2024-04-02T13:17:20.913 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-02T13:17:20.913 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593 --verbose --name pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi --owner scheduled_pdonnell@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} -- /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593/orig.config.yaml 2024-04-02T13:17:20.972 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T13:17:21.080 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593 branch: wip-batrick-testing-20240402.004512 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} email: pdonnell@redhat.com first_in_suite: false job_id: '7635593' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240402.004512 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client dirsize rbytes: false client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 kclient: client.0: mount_subvol_num: 1 rbytes: false selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240402.004512 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2763 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 sleep_before_teardown: 0 subset: 318/512 suite: fs suite_branch: wip-batrick-testing-20240402.004512 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLhTnsnNgTEqvZ0xH6rc6tnsPMCrui9NGPD3iGUtxEMvHGDqTEvq7KKmVY4TO0VTw0K1wbELcKMqMk8zzOdAznQ= smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbXHHwuJ3dMVVWHiHynexISlW2DTUTPBSllC2jgAn4HenuiXzJZonUO7EZFtpyDiOH8hsbyyGD5kvyFEZaO9jI= smithi145.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG4bNyFXHEQGaXTNcf2ir12eNpiralK8fj87y7QdLX0e6bOBhm/YW4OGLshUUeUizs1UMAnlk2Mxoekaq1y0euQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - fs/norstats teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-04-02_11:52:43 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-02T13:17:21.080 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa; will attempt to use it 2024-04-02T13:17:21.081 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks 2024-04-02T13:17:21.081 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-02T13:17:21.083 INFO:teuthology.task.internal:Checking packages... 2024-04-02T13:17:21.105 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '191d963ac9de7322da8aff20b567b3fa458e2ef6' 2024-04-02T13:17:21.105 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T13:17:21.105 INFO:teuthology.packaging:ref: None 2024-04-02T13:17:21.106 INFO:teuthology.packaging:tag: None 2024-04-02T13:17:21.106 INFO:teuthology.packaging:branch: wip-batrick-testing-20240402.004512 2024-04-02T13:17:21.106 INFO:teuthology.packaging:sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:17:21.106 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240402.004512 2024-04-02T13:17:21.244 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2609.g191d963a 2024-04-02T13:17:21.245 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-02T13:17:21.252 INFO:teuthology.task.internal:no buildpackages task found 2024-04-02T13:17:21.252 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-02T13:17:21.274 INFO:teuthology.task.internal:Saving configuration 2024-04-02T13:17:21.295 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-02T13:17:21.301 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T13:17:21.325 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:12:23.631068', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLhTnsnNgTEqvZ0xH6rc6tnsPMCrui9NGPD3iGUtxEMvHGDqTEvq7KKmVY4TO0VTw0K1wbELcKMqMk8zzOdAznQ='} 2024-04-02T13:17:21.347 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:12:23.632177', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbXHHwuJ3dMVVWHiHynexISlW2DTUTPBSllC2jgAn4HenuiXzJZonUO7EZFtpyDiOH8hsbyyGD5kvyFEZaO9jI='} 2024-04-02T13:17:21.367 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:12:23.629403', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG4bNyFXHEQGaXTNcf2ir12eNpiralK8fj87y7QdLX0e6bOBhm/YW4OGLshUUeUizs1UMAnlk2Mxoekaq1y0euQ='} 2024-04-02T13:17:21.367 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-02T13:17:21.374 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-02T13:17:21.375 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-02T13:17:21.375 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-02T13:17:21.375 INFO:teuthology.run_tasks:Running task console_log... 2024-04-02T13:17:21.475 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbf24cb1160>, signals=[15]) 2024-04-02T13:17:21.476 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-02T13:17:21.482 INFO:teuthology.task.internal:Opening connections... 2024-04-02T13:17:21.482 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-04-02T13:17:21.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:17:21.589 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2024-04-02T13:17:21.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:17:21.691 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi145.front.sepia.ceph.com 2024-04-02T13:17:21.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:17:21.780 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-02T13:17:21.787 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-04-02T13:17:21.805 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-04-02T13:17:21.805 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-04-02T13:17:21.862 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-04-02T13:17:21.862 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-04-02T13:17:21.862 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-04-02T13:17:21.863 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-04-02T13:17:21.863 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-04-02T13:17:21.863 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-04-02T13:17:21.863 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T13:17:21.863 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-04-02T13:17:21.863 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-04-02T13:17:21.863 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T13:17:21.863 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-04-02T13:17:21.863 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T13:17:21.863 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T13:17:21.863 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T13:17:21.864 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-04-02T13:17:21.899 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2024-04-02T13:17:21.918 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2024-04-02T13:17:21.918 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2024-04-02T13:17:21.976 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2024-04-02T13:17:21.976 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2024-04-02T13:17:21.976 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2024-04-02T13:17:21.976 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2024-04-02T13:17:21.976 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2024-04-02T13:17:21.976 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2024-04-02T13:17:21.976 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T13:17:21.976 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2024-04-02T13:17:21.976 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2024-04-02T13:17:21.976 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T13:17:21.976 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2024-04-02T13:17:21.977 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T13:17:21.977 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T13:17:21.977 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T13:17:21.977 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2024-04-02T13:17:21.997 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2024-04-02T13:17:22.015 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2024-04-02T13:17:22.015 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2024-04-02T13:17:22.072 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2024-04-02T13:17:22.072 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="9" 2024-04-02T13:17:22.072 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2024-04-02T13:17:22.072 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2024-04-02T13:17:22.072 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="9" 2024-04-02T13:17:22.072 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el9" 2024-04-02T13:17:22.072 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T13:17:22.072 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2024-04-02T13:17:22.072 INFO:teuthology.orchestra.run.smithi145.stdout:LOGO="fedora-logo-icon" 2024-04-02T13:17:22.072 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T13:17:22.072 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2024-04-02T13:17:22.072 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T13:17:22.073 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T13:17:22.073 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T13:17:22.073 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2024-04-02T13:17:22.097 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-02T13:17:22.109 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-02T13:17:22.119 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-02T13:17:22.119 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-04-02T13:17:22.122 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2024-04-02T13:17:22.124 DEBUG:teuthology.orchestra.run.smithi145:> test '!' -e /home/ubuntu/cephtest 2024-04-02T13:17:22.140 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-02T13:17:22.146 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-02T13:17:22.147 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-04-02T13:17:22.179 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2024-04-02T13:17:22.181 DEBUG:teuthology.orchestra.run.smithi145:> test -z $(ls -A /var/lib/ceph) 2024-04-02T13:17:22.278 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-02T13:17:22.356 INFO:teuthology.run_tasks:Running task kernel... 2024-04-02T13:17:22.369 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-02T13:17:22.369 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-02T13:17:22.370 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T13:17:22.370 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T13:17:22.370 DEBUG:teuthology.orchestra.run.smithi145:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T13:17:22.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:17:22.387 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-04-02T13:17:22.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:17:22.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:17:22.389 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-04-02T13:17:22.389 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-04-02T13:17:22.444 INFO:teuthology.orchestra.run.smithi145.stdout:5.14.0-361.el9.x86_64 2024-04-02T13:17:22.444 INFO:teuthology.task.kernel:Running kernel on smithi145: 5.14.0-361.el9.x86_64 2024-04-02T13:17:22.445 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum install -y kernel 2024-04-02T13:17:22.446 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-361.el9.x86_64 2024-04-02T13:17:22.446 INFO:teuthology.task.kernel:Running kernel on smithi050: 5.14.0-361.el9.x86_64 2024-04-02T13:17:22.447 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum install -y kernel 2024-04-02T13:17:22.447 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-361.el9.x86_64 2024-04-02T13:17:22.448 INFO:teuthology.task.kernel:Running kernel on smithi033: 5.14.0-361.el9.x86_64 2024-04-02T13:17:22.448 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum install -y kernel 2024-04-02T13:17:24.722 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-04-02T13:17:24.760 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-04-02T13:17:24.905 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-04-02T13:17:26.247 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - BaseOS 5.6 MB/s | 2.0 MB 00:00 2024-04-02T13:17:26.285 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2024-04-02T13:17:26.663 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS-9 - BaseOS 3.1 MB/s | 2.0 MB 00:00 2024-04-02T13:17:27.662 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-02T13:17:28.554 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.0 MB 00:01 2024-04-02T13:17:29.488 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-02T13:17:31.281 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-02T13:17:34.439 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.1 MB 00:01 2024-04-02T13:17:35.433 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 935 kB/s | 8.0 MB 00:08 2024-04-02T13:17:35.590 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-04-02T13:17:35.965 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-02T13:17:37.324 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-02T13:17:37.368 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-02T13:17:39.358 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 19 MB 00:02 2024-04-02T13:17:39.519 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-02T13:17:42.399 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-02T13:17:44.639 INFO:teuthology.orchestra.run.smithi145.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-02T13:17:44.639 INFO:teuthology.orchestra.run.smithi145.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-02T13:17:44.732 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-04-02T13:17:44.740 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-04-02T13:17:44.740 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-04-02T13:17:44.740 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-04-02T13:17:44.740 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-04-02T13:17:44.740 INFO:teuthology.orchestra.run.smithi145.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-02T13:17:44.740 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-04-02T13:17:44.740 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-02T13:17:44.741 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-02T13:17:44.741 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-02T13:17:44.741 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:17:44.741 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-04-02T13:17:44.741 INFO:teuthology.orchestra.run.smithi145.stdout:================================================================================ 2024-04-02T13:17:44.741 INFO:teuthology.orchestra.run.smithi145.stdout:Install 4 Packages 2024-04-02T13:17:44.741 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:17:44.741 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 76 M 2024-04-02T13:17:44.741 INFO:teuthology.orchestra.run.smithi145.stdout:Installed size: 123 M 2024-04-02T13:17:44.741 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-04-02T13:17:44.892 INFO:teuthology.orchestra.run.smithi145.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-02T13:17:44.992 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-02T13:17:45.375 INFO:teuthology.orchestra.run.smithi145.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 24 MB/s | 15 MB 00:00 2024-04-02T13:17:46.084 INFO:teuthology.orchestra.run.smithi145.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 25 MB/s | 34 MB 00:01 2024-04-02T13:17:46.359 INFO:teuthology.orchestra.run.smithi145.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 19 MB/s | 28 MB 00:01 2024-04-02T13:17:46.362 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:17:46.362 INFO:teuthology.orchestra.run.smithi145.stdout:Total 47 MB/s | 76 MB 00:01 2024-04-02T13:17:46.363 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-04-02T13:17:46.752 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-04-02T13:17:46.752 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-04-02T13:17:46.808 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.1 MB 00:03 2024-04-02T13:17:47.382 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-02T13:17:47.383 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-02T13:17:47.479 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-04-02T13:17:47.486 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-04-02T13:17:47.486 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-04-02T13:17:47.486 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-04-02T13:17:47.486 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-04-02T13:17:47.486 INFO:teuthology.orchestra.run.smithi050.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-02T13:17:47.486 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-04-02T13:17:47.486 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-02T13:17:47.486 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-02T13:17:47.486 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-02T13:17:47.486 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:17:47.487 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-04-02T13:17:47.487 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-04-02T13:17:47.487 INFO:teuthology.orchestra.run.smithi050.stdout:Install 4 Packages 2024-04-02T13:17:47.487 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:17:47.487 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 76 M 2024-04-02T13:17:47.487 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 123 M 2024-04-02T13:17:47.488 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-04-02T13:17:47.593 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-04-02T13:17:47.594 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-04-02T13:17:47.677 INFO:teuthology.orchestra.run.smithi050.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-04-02T13:17:48.210 INFO:teuthology.orchestra.run.smithi050.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 21 MB/s | 15 MB 00:00 2024-04-02T13:17:48.643 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-04-02T13:17:48.910 INFO:teuthology.orchestra.run.smithi050.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 24 MB/s | 34 MB 00:01 2024-04-02T13:17:49.177 INFO:teuthology.orchestra.run.smithi050.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 18 MB/s | 28 MB 00:01 2024-04-02T13:17:49.180 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:17:49.181 INFO:teuthology.orchestra.run.smithi050.stdout:Total 45 MB/s | 76 MB 00:01 2024-04-02T13:17:49.181 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-04-02T13:17:49.618 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-04-02T13:17:49.618 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-04-02T13:17:49.924 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-04-02T13:17:50.227 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-02T13:17:50.508 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-04-02T13:17:50.508 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-04-02T13:17:50.526 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-02T13:17:50.609 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:17:52.932 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-04-02T13:17:53.354 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:17:53.488 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:17:53.550 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-02T13:17:53.637 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:17:55.435 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-02T13:17:56.518 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:17:56.658 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:17:57.826 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-02T13:17:57.826 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-02T13:17:57.917 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-02T13:17:57.931 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-02T13:17:57.931 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-02T13:17:57.931 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-02T13:17:57.931 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-02T13:17:57.931 INFO:teuthology.orchestra.run.smithi033.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-02T13:17:57.931 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-04-02T13:17:57.931 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-02T13:17:57.931 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-02T13:17:57.931 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-02T13:17:57.932 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:17:57.932 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-02T13:17:57.932 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-02T13:17:57.932 INFO:teuthology.orchestra.run.smithi033.stdout:Install 4 Packages 2024-04-02T13:17:57.932 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:17:57.932 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 76 M 2024-04-02T13:17:57.932 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 123 M 2024-04-02T13:17:57.933 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-02T13:17:58.186 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:17:58.187 INFO:teuthology.orchestra.run.smithi033.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 755 kB/s | 192 kB 00:00 2024-04-02T13:17:58.220 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:17:58.862 INFO:teuthology.orchestra.run.smithi033.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 16 MB/s | 15 MB 00:00 2024-04-02T13:17:59.420 INFO:teuthology.orchestra.run.smithi033.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 23 MB/s | 34 MB 00:01 2024-04-02T13:17:59.887 INFO:teuthology.orchestra.run.smithi033.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 16 MB/s | 28 MB 00:01 2024-04-02T13:17:59.890 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:17:59.890 INFO:teuthology.orchestra.run.smithi033.stdout:Total 39 MB/s | 76 MB 00:01 2024-04-02T13:17:59.891 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-02T13:18:00.340 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-02T13:18:00.340 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-02T13:18:01.181 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-02T13:18:01.181 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-02T13:18:01.737 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:18:01.774 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:02.932 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:03.615 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-02T13:18:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-02T13:18:04.312 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:18:06.879 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:07.199 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:18:07.340 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:18:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:18:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:17.561 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:32.403 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:32.404 INFO:teuthology.orchestra.run.smithi145.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T13:18:32.404 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:18:32.447 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:32.857 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:32.857 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-02T13:18:32.857 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:18:32.857 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:18:33.278 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:33.278 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:18:33.278 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-04-02T13:18:33.278 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-02T13:18:33.278 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-02T13:18:33.279 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-02T13:18:33.279 INFO:teuthology.orchestra.run.smithi145.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-02T13:18:33.279 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:18:33.279 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-04-02T13:18:33.604 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T13:18:33.713 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T13:18:33.713 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T13:18:33.713 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-02T13:18:33.714 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T13:18:33.714 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T13:18:33.714 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-04-02T13:18:33.714 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi145.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T13:18:33.714 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T13:18:33.793 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T13:18:33.794 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T13:18:33.794 DEBUG:teuthology.orchestra.run.smithi145:> sudo rpm -qi grub2-tools 2024-04-02T13:18:33.832 INFO:teuthology.orchestra.run.smithi145.stdout:Name : grub2-tools 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:Epoch : 1 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:Version : 2.06 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:Release : 68.el9 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:Architecture: x86_64 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:Group : Unspecified 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:Size : 8264254 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:License : GPLv3+ 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:Packager : builder@centos.org 2024-04-02T13:18:33.833 INFO:teuthology.orchestra.run.smithi145.stdout:Vendor : CentOS 2024-04-02T13:18:33.834 INFO:teuthology.orchestra.run.smithi145.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T13:18:33.834 INFO:teuthology.orchestra.run.smithi145.stdout:Summary : Support tools for GRUB. 2024-04-02T13:18:33.834 INFO:teuthology.orchestra.run.smithi145.stdout:Description : 2024-04-02T13:18:33.834 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:18:33.834 INFO:teuthology.orchestra.run.smithi145.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T13:18:33.834 INFO:teuthology.orchestra.run.smithi145.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T13:18:33.834 INFO:teuthology.orchestra.run.smithi145.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T13:18:33.834 INFO:teuthology.orchestra.run.smithi145.stdout:hardware devices. 2024-04-02T13:18:33.834 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:18:33.834 INFO:teuthology.orchestra.run.smithi145.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T13:18:33.835 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T13:18:33.835 INFO:teuthology.task.kernel:Updating grub on smithi145 to boot 5.14.0-434.el9.x86_64 2024-04-02T13:18:33.835 DEBUG:teuthology.orchestra.run.smithi145:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T13:18:34.504 INFO:teuthology.orchestra.run.smithi145.stderr:Generating grub configuration file ... 2024-04-02T13:18:35.917 INFO:teuthology.orchestra.run.smithi145.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T13:18:35.964 INFO:teuthology.orchestra.run.smithi145.stderr:done 2024-04-02T13:18:35.966 DEBUG:teuthology.orchestra.run.smithi145:> mktemp 2024-04-02T13:18:35.981 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/tmp.EqBFtaRcH5 2024-04-02T13:18:35.981 DEBUG:teuthology.orchestra.run.smithi145:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EqBFtaRcH5 2024-04-02T13:18:36.048 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0666 /tmp/tmp.EqBFtaRcH5 2024-04-02T13:18:36.182 DEBUG:teuthology.orchestra.remote:smithi145:/tmp/tmp.EqBFtaRcH5 is 6KB 2024-04-02T13:18:36.230 DEBUG:teuthology.orchestra.run.smithi145:> rm -fr /tmp/tmp.EqBFtaRcH5 2024-04-02T13:18:36.244 DEBUG:teuthology.orchestra.run.smithi145:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T13:18:36.309 INFO:teuthology.orchestra.run.smithi145.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T13:18:36.309 INFO:teuthology.orchestra.run.smithi145.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T13:18:36.309 INFO:teuthology.orchestra.run.smithi145.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T13:18:36.309 INFO:teuthology.orchestra.run.smithi145.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-02T13:18:36.310 DEBUG:teuthology.orchestra.run.smithi145:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-02T13:18:36.468 DEBUG:teuthology.orchestra.run.smithi145:> sudo shutdown -r now 2024-04-02T13:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T13:18:37.703 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:18:37.748 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-02T13:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:18:38.110 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:18:38.513 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:38.513 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:18:38.513 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-04-02T13:18:38.513 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-02T13:18:38.513 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-02T13:18:38.513 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-02T13:18:38.514 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-02T13:18:38.514 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:18:38.514 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-04-02T13:18:38.855 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T13:18:38.967 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T13:18:38.967 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T13:18:38.967 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-02T13:18:38.967 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T13:18:38.967 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T13:18:38.967 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-02T13:18:38.967 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi050.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T13:18:38.968 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T13:18:39.051 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T13:18:39.051 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T13:18:39.052 DEBUG:teuthology.orchestra.run.smithi050:> sudo rpm -qi grub2-tools 2024-04-02T13:18:39.090 INFO:teuthology.orchestra.run.smithi050.stdout:Name : grub2-tools 2024-04-02T13:18:39.090 INFO:teuthology.orchestra.run.smithi050.stdout:Epoch : 1 2024-04-02T13:18:39.090 INFO:teuthology.orchestra.run.smithi050.stdout:Version : 2.06 2024-04-02T13:18:39.090 INFO:teuthology.orchestra.run.smithi050.stdout:Release : 68.el9 2024-04-02T13:18:39.090 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: x86_64 2024-04-02T13:18:39.090 INFO:teuthology.orchestra.run.smithi050.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T13:18:39.090 INFO:teuthology.orchestra.run.smithi050.stdout:Group : Unspecified 2024-04-02T13:18:39.090 INFO:teuthology.orchestra.run.smithi050.stdout:Size : 8264254 2024-04-02T13:18:39.090 INFO:teuthology.orchestra.run.smithi050.stdout:License : GPLv3+ 2024-04-02T13:18:39.090 INFO:teuthology.orchestra.run.smithi050.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T13:18:39.090 INFO:teuthology.orchestra.run.smithi050.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T13:18:39.090 INFO:teuthology.orchestra.run.smithi050.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T13:18:39.091 INFO:teuthology.orchestra.run.smithi050.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T13:18:39.091 INFO:teuthology.orchestra.run.smithi050.stdout:Packager : builder@centos.org 2024-04-02T13:18:39.091 INFO:teuthology.orchestra.run.smithi050.stdout:Vendor : CentOS 2024-04-02T13:18:39.091 INFO:teuthology.orchestra.run.smithi050.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T13:18:39.091 INFO:teuthology.orchestra.run.smithi050.stdout:Summary : Support tools for GRUB. 2024-04-02T13:18:39.091 INFO:teuthology.orchestra.run.smithi050.stdout:Description : 2024-04-02T13:18:39.091 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:18:39.091 INFO:teuthology.orchestra.run.smithi050.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T13:18:39.091 INFO:teuthology.orchestra.run.smithi050.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T13:18:39.091 INFO:teuthology.orchestra.run.smithi050.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T13:18:39.091 INFO:teuthology.orchestra.run.smithi050.stdout:hardware devices. 2024-04-02T13:18:39.091 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:18:39.091 INFO:teuthology.orchestra.run.smithi050.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T13:18:39.092 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T13:18:39.092 INFO:teuthology.task.kernel:Updating grub on smithi050 to boot 5.14.0-434.el9.x86_64 2024-04-02T13:18:39.093 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T13:18:39.756 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2024-04-02T13:18:41.175 INFO:teuthology.orchestra.run.smithi050.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T13:18:41.232 INFO:teuthology.orchestra.run.smithi050.stderr:done 2024-04-02T13:18:41.234 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2024-04-02T13:18:41.290 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.M5yGwHEcbY 2024-04-02T13:18:41.290 DEBUG:teuthology.orchestra.run.smithi050:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.M5yGwHEcbY 2024-04-02T13:18:41.366 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.M5yGwHEcbY 2024-04-02T13:18:41.508 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.M5yGwHEcbY is 6KB 2024-04-02T13:18:41.556 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.M5yGwHEcbY 2024-04-02T13:18:41.571 DEBUG:teuthology.orchestra.run.smithi050:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T13:18:41.639 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T13:18:41.640 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T13:18:41.640 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T13:18:41.640 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-02T13:18:41.641 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-02T13:18:41.782 DEBUG:teuthology.orchestra.run.smithi050:> sudo shutdown -r now 2024-04-02T13:18:48.490 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T13:18:48.490 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:18:48.535 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:48.912 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:48.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-02T13:18:48.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:18:48.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:18:49.326 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:18:49.326 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:18:49.326 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-02T13:18:49.327 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-02T13:18:49.327 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-02T13:18:49.327 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-02T13:18:49.327 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-02T13:18:49.327 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:18:49.327 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-02T13:18:49.702 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T13:18:49.807 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T13:18:49.807 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T13:18:49.807 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-02T13:18:49.808 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T13:18:49.808 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T13:18:49.808 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-02T13:18:49.808 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T13:18:49.808 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T13:18:49.892 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T13:18:49.892 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T13:18:49.892 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -qi grub2-tools 2024-04-02T13:18:49.940 INFO:teuthology.orchestra.run.smithi033.stdout:Name : grub2-tools 2024-04-02T13:18:49.940 INFO:teuthology.orchestra.run.smithi033.stdout:Epoch : 1 2024-04-02T13:18:49.940 INFO:teuthology.orchestra.run.smithi033.stdout:Version : 2.06 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:Release : 68.el9 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: x86_64 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:Group : Unspecified 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:Size : 8264254 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:License : GPLv3+ 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:Packager : builder@centos.org 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:Vendor : CentOS 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:Summary : Support tools for GRUB. 2024-04-02T13:18:49.941 INFO:teuthology.orchestra.run.smithi033.stdout:Description : 2024-04-02T13:18:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:18:49.942 INFO:teuthology.orchestra.run.smithi033.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T13:18:49.942 INFO:teuthology.orchestra.run.smithi033.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T13:18:49.942 INFO:teuthology.orchestra.run.smithi033.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T13:18:49.942 INFO:teuthology.orchestra.run.smithi033.stdout:hardware devices. 2024-04-02T13:18:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:18:49.942 INFO:teuthology.orchestra.run.smithi033.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T13:18:49.943 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T13:18:49.943 INFO:teuthology.task.kernel:Updating grub on smithi033 to boot 5.14.0-434.el9.x86_64 2024-04-02T13:18:49.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T13:18:50.445 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2024-04-02T13:18:51.851 INFO:teuthology.orchestra.run.smithi033.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T13:18:51.888 INFO:teuthology.orchestra.run.smithi033.stderr:done 2024-04-02T13:18:51.890 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-04-02T13:18:51.905 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.rUiwotdIC3 2024-04-02T13:18:51.906 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rUiwotdIC3 2024-04-02T13:18:51.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.rUiwotdIC3 2024-04-02T13:18:52.123 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.rUiwotdIC3 is 6KB 2024-04-02T13:18:52.171 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.rUiwotdIC3 2024-04-02T13:18:52.189 DEBUG:teuthology.orchestra.run.smithi033:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T13:18:52.254 INFO:teuthology.orchestra.run.smithi033.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T13:18:52.254 INFO:teuthology.orchestra.run.smithi033.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T13:18:52.254 INFO:teuthology.orchestra.run.smithi033.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T13:18:52.255 INFO:teuthology.orchestra.run.smithi033.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-02T13:18:52.256 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-02T13:18:52.411 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2024-04-02T13:19:06.485 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-04-02T13:19:06.485 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-04-02T13:19:06.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:19:11.790 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-02T13:19:11.790 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-04-02T13:19:11.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:19:22.426 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-02T13:19:22.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-02T13:19:22.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:19:24.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-04-02T13:19:30.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-04-02T13:19:33.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-04-02T13:19:33.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:19:39.341 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-04-02T13:19:39.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:19:40.830 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-04-02T13:19:49.838 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-02T13:19:49.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:20:08.223 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-04-02T13:20:20.235 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-02T13:20:20.236 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:20:20.838 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-04-02T13:20:21.276 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-02T13:20:21.277 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-434.el9.x86_64"... 2024-04-02T13:20:21.277 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-04-02T13:20:21.330 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-434.el9.x86_64 2024-04-02T13:20:21.331 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-02T13:20:21.331 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T13:20:21.331 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T13:20:22.331 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-02T13:20:22.332 DEBUG:teuthology.orchestra.run.smithi033:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-02T13:20:22.408 INFO:teuthology.orchestra.run.smithi033.stdout:ttyS1 2024-04-02T13:20:22.448 DEBUG:teuthology.parallel:result is None 2024-04-02T13:20:33.965 DEBUG:teuthology.orchestra.remote:timed out 2024-04-02T13:20:39.344 DEBUG:teuthology.orchestra.remote:timed out 2024-04-02T13:20:45.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-04-02T13:20:45.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:20:46.444 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-04-02T13:20:46.830 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-04-02T13:20:46.833 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-434.el9.x86_64"... 2024-04-02T13:20:46.834 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-04-02T13:20:46.850 INFO:teuthology.orchestra.run.smithi145.stdout:5.14.0-434.el9.x86_64 2024-04-02T13:20:46.850 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-02T13:20:46.851 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T13:20:46.851 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T13:20:47.851 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-02T13:20:47.852 DEBUG:teuthology.orchestra.run.smithi145:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-02T13:20:47.931 INFO:teuthology.orchestra.run.smithi145.stdout:ttyS1 2024-04-02T13:20:47.963 DEBUG:teuthology.parallel:result is None 2024-04-02T13:20:51.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-04-02T13:20:51.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:20:51.650 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-04-02T13:20:52.013 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-04-02T13:20:52.013 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-434.el9.x86_64"... 2024-04-02T13:20:52.013 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-04-02T13:20:52.028 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-434.el9.x86_64 2024-04-02T13:20:52.028 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-02T13:20:52.028 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T13:20:52.028 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T13:20:53.029 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-02T13:20:53.029 DEBUG:teuthology.orchestra.run.smithi050:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-02T13:20:53.100 INFO:teuthology.orchestra.run.smithi050.stdout:ttyS1 2024-04-02T13:20:53.135 DEBUG:teuthology.parallel:result is None 2024-04-02T13:20:53.136 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-02T13:20:53.143 INFO:teuthology.task.internal:Creating test directory... 2024-04-02T13:20:53.143 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T13:20:53.145 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T13:20:53.148 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T13:20:53.166 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-02T13:20:53.172 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-02T13:20:53.178 INFO:teuthology.task.internal:Creating archive directory... 2024-04-02T13:20:53.178 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T13:20:53.202 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T13:20:53.204 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T13:20:53.235 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-02T13:20:53.242 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-02T13:20:53.242 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T13:20:53.260 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T13:20:53.271 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T13:20:53.294 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:20:53.301 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:20:53.308 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:20:53.315 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:20:53.315 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:20:53.328 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:20:53.330 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-02T13:20:53.337 INFO:teuthology.task.internal:Configuring sudo... 2024-04-02T13:20:53.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T13:20:53.351 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T13:20:53.362 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T13:20:53.400 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-02T13:20:53.410 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-02T13:20:53.410 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T13:20:53.419 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T13:20:53.427 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T13:20:53.456 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:20:53.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:20:53.569 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:20:53.652 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:20:53.703 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:20:53.703 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T13:20:53.773 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:20:53.818 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:20:53.870 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:20:53.954 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:20:53.999 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T13:20:53.999 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T13:20:54.068 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:20:54.117 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:20:54.166 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:20:54.245 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:20:54.296 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-02T13:20:54.296 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T13:20:54.366 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-04-02T13:20:54.369 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2024-04-02T13:20:54.371 DEBUG:teuthology.orchestra.run.smithi145:> sudo service rsyslog restart 2024-04-02T13:20:54.414 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T13:20:54.426 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T13:20:54.460 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T13:20:54.796 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-02T13:20:54.803 INFO:teuthology.task.internal:Starting timer... 2024-04-02T13:20:54.803 INFO:teuthology.run_tasks:Running task pcp... 2024-04-02T13:20:54.820 INFO:teuthology.run_tasks:Running task selinux... 2024-04-02T13:20:54.828 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-02T13:20:54.828 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-04-02T13:20:54.901 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-04-02T13:20:54.903 DEBUG:teuthology.orchestra.run.smithi050:> sudo service auditd rotate 2024-04-02T13:20:54.955 INFO:teuthology.orchestra.run.smithi050.stdout:Rotating logs: 2024-04-02T13:20:54.956 DEBUG:teuthology.orchestra.run.smithi145:> sudo service auditd rotate 2024-04-02T13:20:55.023 INFO:teuthology.orchestra.run.smithi145.stdout:Rotating logs: 2024-04-02T13:20:55.025 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-02T13:20:55.025 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-04-02T13:20:55.046 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-04-02T13:20:55.046 DEBUG:teuthology.orchestra.run.smithi050:> /usr/sbin/getenforce 2024-04-02T13:20:55.076 INFO:teuthology.orchestra.run.smithi050.stdout:Permissive 2024-04-02T13:20:55.076 DEBUG:teuthology.orchestra.run.smithi145:> /usr/sbin/getenforce 2024-04-02T13:20:55.108 INFO:teuthology.orchestra.run.smithi145.stdout:Permissive 2024-04-02T13:20:55.108 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi050.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi145.front.sepia.ceph.com': 'permissive'} 2024-04-02T13:20:55.109 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T13:20:55.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:20:55.139 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T13:20:55.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:20:55.168 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T13:20:55.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:20:55.199 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-02T13:20:55.199 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-04-02T13:20:55.231 DEBUG:teuthology.orchestra.run.smithi050:> sudo /usr/sbin/setenforce permissive 2024-04-02T13:20:55.270 DEBUG:teuthology.orchestra.run.smithi145:> sudo /usr/sbin/setenforce permissive 2024-04-02T13:20:55.302 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-02T13:20:55.311 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-02T13:20:55.317 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-02T13:20:55.457 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-02T13:20:55.472 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-02T13:20:55.473 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi050.front.sepia.ceph.com,smithi145.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-02T13:25:15.154 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi050.front.sepia.ceph.com'), Remote(name='ubuntu@smithi145.front.sepia.ceph.com')] 2024-04-02T13:25:15.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-02T13:25:15.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:25:15.249 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-04-02T13:25:15.324 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-02T13:25:15.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-04-02T13:25:15.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:25:15.412 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-04-02T13:25:15.489 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-04-02T13:25:15.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-04-02T13:25:15.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:25:15.583 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-04-02T13:25:15.670 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-04-02T13:25:15.671 INFO:teuthology.run_tasks:Running task clock... 2024-04-02T13:25:15.682 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-02T13:25:15.682 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T13:25:15.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T13:25:15.685 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T13:25:15.685 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T13:25:15.687 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T13:25:15.687 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T13:25:15.715 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T13:25:15.717 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T13:25:15.732 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T13:25:15.735 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T13:25:15.742 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T13:25:15.759 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T13:25:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-04-02T13:25:15.765 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: ntpd: command not found 2024-04-02T13:25:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-04-02T13:25:15.778 INFO:teuthology.orchestra.run.smithi050.stdout:506 Cannot talk to daemon 2024-04-02T13:25:15.789 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: ntpd: command not found 2024-04-02T13:25:15.792 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T13:25:15.795 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T13:25:15.802 INFO:teuthology.orchestra.run.smithi145.stdout:506 Cannot talk to daemon 2024-04-02T13:25:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T13:25:15.811 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T13:25:15.819 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T13:25:15.835 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T13:25:15.861 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-04-02T13:25:15.861 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-04-02T13:25:15.864 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T13:25:15.864 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-04-02T13:25:15.864 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:25:15.864 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:25:15.864 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:25:15.864 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:25:15.865 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T13:25:15.865 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-04-02T13:25:15.865 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:25:15.865 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:25:15.865 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:25:15.865 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:25:15.895 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-04-02T13:25:15.899 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T13:25:15.899 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-04-02T13:25:15.899 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:25:15.899 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:25:15.899 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:25:15.899 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:25:15.900 INFO:teuthology.run_tasks:Running task install... 2024-04-02T13:25:15.908 DEBUG:teuthology.task.install:project ceph 2024-04-02T13:25:15.908 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}} 2024-04-02T13:25:15.908 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'} 2024-04-02T13:25:15.909 INFO:teuthology.task.install:Using flavor: default 2024-04-02T13:25:15.914 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-02T13:25:15.914 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-02T13:25:15.915 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T13:25:15.916 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:25:15.918 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T13:25:15.918 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:25:15.919 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T13:25:15.919 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:25:16.058 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T13:25:16.058 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T13:25:16.066 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T13:25:16.066 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T13:25:16.071 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T13:25:16.071 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T13:25:16.203 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T13:25:16.203 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T13:25:16.203 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T13:25:16.231 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T13:25:16.231 DEBUG:teuthology.orchestra.run.smithi050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T13:25:16.259 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T13:25:16.259 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:25:16.259 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T13:25:16.269 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T13:25:16.269 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-02T13:25:16.269 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T13:25:16.287 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T13:25:16.287 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T13:25:16.298 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T13:25:16.298 DEBUG:teuthology.orchestra.run.smithi145:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T13:25:16.300 DEBUG:teuthology.orchestra.run.smithi050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T13:25:16.355 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T13:25:16.366 DEBUG:teuthology.orchestra.run.smithi145:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T13:25:16.380 DEBUG:teuthology.orchestra.run.smithi050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T13:25:16.435 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T13:25:16.446 INFO:teuthology.orchestra.run.smithi050.stdout:check_obsoletes = 1 2024-04-02T13:25:16.446 DEBUG:teuthology.orchestra.run.smithi145:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T13:25:16.448 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum clean all 2024-04-02T13:25:16.502 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-04-02T13:25:16.503 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-04-02T13:25:16.512 INFO:teuthology.orchestra.run.smithi145.stdout:check_obsoletes = 1 2024-04-02T13:25:16.514 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum clean all 2024-04-02T13:25:16.699 INFO:teuthology.orchestra.run.smithi050.stdout:74 files removed 2024-04-02T13:25:16.726 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T13:25:16.755 INFO:teuthology.orchestra.run.smithi033.stdout:74 files removed 2024-04-02T13:25:16.758 INFO:teuthology.orchestra.run.smithi145.stdout:74 files removed 2024-04-02T13:25:16.782 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T13:25:16.784 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T13:25:17.412 INFO:teuthology.orchestra.run.smithi145.stdout:ceph packages for x86_64 256 kB/s | 83 kB 00:00 2024-04-02T13:25:17.455 INFO:teuthology.orchestra.run.smithi050.stdout:ceph packages for x86_64 198 kB/s | 83 kB 00:00 2024-04-02T13:25:17.566 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 174 kB/s | 83 kB 00:00 2024-04-02T13:25:17.854 INFO:teuthology.orchestra.run.smithi145.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-02T13:25:17.855 INFO:teuthology.orchestra.run.smithi050.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-02T13:25:18.008 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-02T13:25:18.239 INFO:teuthology.orchestra.run.smithi050.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-04-02T13:25:18.255 INFO:teuthology.orchestra.run.smithi145.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-04-02T13:25:18.392 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-04-02T13:25:18.725 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-04-02T13:25:18.911 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-04-02T13:25:19.441 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS-9 - AppStream 5.8 MB/s | 6.7 MB 00:01 2024-04-02T13:25:20.509 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - BaseOS 3.2 MB/s | 2.0 MB 00:00 2024-04-02T13:25:20.678 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2024-04-02T13:25:21.343 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS-9 - BaseOS 2.5 MB/s | 2.0 MB 00:00 2024-04-02T13:25:22.202 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-02T13:25:22.594 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-04-02T13:25:23.440 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.0 MB 00:02 2024-04-02T13:25:23.968 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-02T13:25:25.208 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-04-02T13:25:25.403 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-02T13:25:29.035 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.1 MB 00:01 2024-04-02T13:25:29.310 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-04-02T13:25:30.004 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.1 MB 00:00 2024-04-02T13:25:30.619 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 16 kB 00:00 2024-04-02T13:25:31.869 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-02T13:25:32.106 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 16 kB 00:00 2024-04-02T13:25:32.371 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-02T13:25:33.247 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-02T13:25:37.379 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-02T13:25:38.413 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-04-02T13:25:39.739 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 2.8 MB/s | 21 MB 00:07 2024-04-02T13:25:39.955 INFO:teuthology.orchestra.run.smithi145.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:25:39.956 INFO:teuthology.orchestra.run.smithi145.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:25:39.970 INFO:teuthology.orchestra.run.smithi145.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T13:25:39.971 INFO:teuthology.orchestra.run.smithi145.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T13:25:39.972 INFO:teuthology.orchestra.run.smithi145.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T13:25:39.973 INFO:teuthology.orchestra.run.smithi145.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T13:25:39.975 INFO:teuthology.orchestra.run.smithi145.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T13:25:39.978 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T13:25:39.979 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T13:25:40.060 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-04-02T13:25:40.072 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================================ 2024-04-02T13:25:40.073 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-04-02T13:25:40.073 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================================ 2024-04-02T13:25:40.073 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-04-02T13:25:40.073 INFO:teuthology.orchestra.run.smithi145.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T13:25:40.073 INFO:teuthology.orchestra.run.smithi145.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T13:25:40.073 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T13:25:40.073 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T13:25:40.073 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T13:25:40.073 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T13:25:40.073 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T13:25:40.074 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T13:25:40.074 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T13:25:40.074 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T13:25:40.074 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T13:25:40.074 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T13:25:40.074 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T13:25:40.074 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T13:25:40.074 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T13:25:40.074 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T13:25:40.074 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T13:25:40.074 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T13:25:40.075 INFO:teuthology.orchestra.run.smithi145.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T13:25:40.075 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T13:25:40.075 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T13:25:40.075 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T13:25:40.075 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T13:25:40.075 INFO:teuthology.orchestra.run.smithi145.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T13:25:40.075 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T13:25:40.075 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T13:25:40.075 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T13:25:40.075 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T13:25:40.075 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T13:25:40.076 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T13:25:40.076 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T13:25:40.076 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T13:25:40.076 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T13:25:40.076 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T13:25:40.076 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:25:40.076 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T13:25:40.076 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T13:25:40.076 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrading: 2024-04-02T13:25:40.076 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T13:25:40.076 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T13:25:40.076 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T13:25:40.077 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T13:25:40.077 INFO:teuthology.orchestra.run.smithi145.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T13:25:40.077 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T13:25:40.077 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T13:25:40.077 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T13:25:40.077 INFO:teuthology.orchestra.run.smithi145.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T13:25:40.077 INFO:teuthology.orchestra.run.smithi145.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T13:25:40.077 INFO:teuthology.orchestra.run.smithi145.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T13:25:40.077 INFO:teuthology.orchestra.run.smithi145.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T13:25:40.077 INFO:teuthology.orchestra.run.smithi145.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T13:25:40.077 INFO:teuthology.orchestra.run.smithi145.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T13:25:40.078 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:25:40.078 INFO:teuthology.orchestra.run.smithi145.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T13:25:40.078 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T13:25:40.078 INFO:teuthology.orchestra.run.smithi145.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T13:25:40.078 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T13:25:40.078 INFO:teuthology.orchestra.run.smithi145.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T13:25:40.078 INFO:teuthology.orchestra.run.smithi145.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T13:25:40.078 INFO:teuthology.orchestra.run.smithi145.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T13:25:40.078 INFO:teuthology.orchestra.run.smithi145.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T13:25:40.078 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T13:25:40.078 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-04-02T13:25:40.078 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T13:25:40.079 INFO:teuthology.orchestra.run.smithi145.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T13:25:40.080 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T13:25:40.081 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T13:25:40.081 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T13:25:40.081 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T13:25:40.081 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T13:25:40.081 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T13:25:40.081 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T13:25:40.081 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T13:25:40.082 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T13:25:40.082 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T13:25:40.082 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T13:25:40.082 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T13:25:40.082 INFO:teuthology.orchestra.run.smithi145.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T13:25:40.082 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T13:25:40.082 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T13:25:40.082 INFO:teuthology.orchestra.run.smithi145.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T13:25:40.082 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T13:25:40.082 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T13:25:40.082 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T13:25:40.082 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T13:25:40.082 INFO:teuthology.orchestra.run.smithi145.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T13:25:40.083 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T13:25:40.083 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T13:25:40.083 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T13:25:40.083 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T13:25:40.083 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T13:25:40.083 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T13:25:40.083 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T13:25:40.083 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T13:25:40.083 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T13:25:40.083 INFO:teuthology.orchestra.run.smithi145.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T13:25:40.084 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T13:25:40.084 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T13:25:40.084 INFO:teuthology.orchestra.run.smithi145.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T13:25:40.084 INFO:teuthology.orchestra.run.smithi145.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T13:25:40.084 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T13:25:40.084 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T13:25:40.084 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T13:25:40.084 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T13:25:40.084 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T13:25:40.085 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T13:25:40.086 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T13:25:40.086 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T13:25:40.086 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T13:25:40.086 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T13:25:40.086 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T13:25:40.086 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T13:25:40.086 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T13:25:40.086 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T13:25:40.086 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T13:25:40.086 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T13:25:40.086 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T13:25:40.086 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T13:25:40.086 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T13:25:40.087 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T13:25:40.088 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T13:25:40.089 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T13:25:40.090 INFO:teuthology.orchestra.run.smithi145.stdout:Installing weak dependencies: 2024-04-02T13:25:40.090 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T13:25:40.090 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:25:40.090 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-04-02T13:25:40.090 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================================ 2024-04-02T13:25:40.090 INFO:teuthology.orchestra.run.smithi145.stdout:Install 150 Packages 2024-04-02T13:25:40.090 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrade 24 Packages 2024-04-02T13:25:40.090 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:25:40.091 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 226 M 2024-04-02T13:25:40.091 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-04-02T13:25:41.038 INFO:teuthology.orchestra.run.smithi145.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-04-02T13:25:41.084 INFO:teuthology.orchestra.run.smithi050.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:25:41.085 INFO:teuthology.orchestra.run.smithi050.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:25:41.100 INFO:teuthology.orchestra.run.smithi050.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T13:25:41.101 INFO:teuthology.orchestra.run.smithi050.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T13:25:41.102 INFO:teuthology.orchestra.run.smithi050.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T13:25:41.103 INFO:teuthology.orchestra.run.smithi050.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T13:25:41.105 INFO:teuthology.orchestra.run.smithi050.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T13:25:41.108 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T13:25:41.109 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T13:25:41.195 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-04-02T13:25:41.212 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2024-04-02T13:25:41.212 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-04-02T13:25:41.212 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T13:25:41.213 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T13:25:41.214 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T13:25:41.214 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T13:25:41.214 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T13:25:41.214 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T13:25:41.214 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T13:25:41.214 INFO:teuthology.orchestra.run.smithi050.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T13:25:41.214 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T13:25:41.214 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T13:25:41.214 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T13:25:41.214 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T13:25:41.214 INFO:teuthology.orchestra.run.smithi050.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T13:25:41.214 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T13:25:41.214 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrading: 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T13:25:41.215 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:25:41.216 INFO:teuthology.orchestra.run.smithi050.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T13:25:41.217 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T13:25:41.218 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T13:25:41.219 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T13:25:41.220 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T13:25:41.221 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T13:25:41.222 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T13:25:41.223 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T13:25:41.224 INFO:teuthology.orchestra.run.smithi050.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T13:25:41.225 INFO:teuthology.orchestra.run.smithi050.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T13:25:41.225 INFO:teuthology.orchestra.run.smithi050.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T13:25:41.225 INFO:teuthology.orchestra.run.smithi050.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T13:25:41.225 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T13:25:41.225 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T13:25:41.225 INFO:teuthology.orchestra.run.smithi050.stdout:Installing weak dependencies: 2024-04-02T13:25:41.225 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T13:25:41.225 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:25:41.225 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-04-02T13:25:41.225 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2024-04-02T13:25:41.225 INFO:teuthology.orchestra.run.smithi050.stdout:Install 150 Packages 2024-04-02T13:25:41.225 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrade 24 Packages 2024-04-02T13:25:41.225 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:25:41.226 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 226 M 2024-04-02T13:25:41.226 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-04-02T13:25:41.422 INFO:teuthology.orchestra.run.smithi145.stdout:(2/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 8.7 MB/s | 5.4 MB 00:00 2024-04-02T13:25:41.489 INFO:teuthology.orchestra.run.smithi145.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 2.1 MB/s | 145 kB 00:00 2024-04-02T13:25:41.756 INFO:teuthology.orchestra.run.smithi145.stdout:(4/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 8.9 MB/s | 2.4 MB 00:00 2024-04-02T13:25:41.905 INFO:teuthology.orchestra.run.smithi050.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-02T13:25:41.914 INFO:teuthology.orchestra.run.smithi145.stdout:(5/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 9.4 MB/s | 1.5 MB 00:00 2024-04-02T13:25:41.939 INFO:teuthology.orchestra.run.smithi145.stdout:(6/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 988 kB/s | 890 kB 00:00 2024-04-02T13:25:42.323 INFO:teuthology.orchestra.run.smithi145.stdout:(7/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 12 MB/s | 4.7 MB 00:00 2024-04-02T13:25:42.523 INFO:teuthology.orchestra.run.smithi145.stdout:(8/174): ceph-common-19.0.0-2609.g191d963a.el9. 12 MB/s | 21 MB 00:01 2024-04-02T13:25:42.555 INFO:teuthology.orchestra.run.smithi050.stdout:(2/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 1.3 MB/s | 890 kB 00:00 2024-04-02T13:25:42.574 INFO:teuthology.orchestra.run.smithi145.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 505 kB/s | 25 kB 00:00 2024-04-02T13:25:42.706 INFO:teuthology.orchestra.run.smithi050.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 964 kB/s | 145 kB 00:00 2024-04-02T13:25:43.224 INFO:teuthology.orchestra.run.smithi145.stdout:(10/174): ceph-radosgw-19.0.0-2609.g191d963a.el 11 MB/s | 9.8 MB 00:00 2024-04-02T13:25:43.307 INFO:teuthology.orchestra.run.smithi145.stdout:(11/174): cephfs-mirror-19.0.0-2609.g191d963a.e 2.6 MB/s | 225 kB 00:00 2024-04-02T13:25:43.391 INFO:teuthology.orchestra.run.smithi145.stdout:(12/174): cephfs-shell-19.0.0-2609.g191d963a.el 325 kB/s | 27 kB 00:00 2024-04-02T13:25:43.475 INFO:teuthology.orchestra.run.smithi145.stdout:(13/174): libcephfs-devel-19.0.0-2609.g191d963a 380 kB/s | 32 kB 00:00 2024-04-02T13:25:43.739 INFO:teuthology.orchestra.run.smithi050.stdout:(4/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 2.3 MB/s | 2.4 MB 00:01 2024-04-02T13:25:43.833 INFO:teuthology.orchestra.run.smithi145.stdout:(14/174): libcephfs2-19.0.0-2609.g191d963a.el9. 2.0 MB/s | 747 kB 00:00 2024-04-02T13:25:43.917 INFO:teuthology.orchestra.run.smithi145.stdout:(15/174): libcephsqlite-19.0.0-2609.g191d963a.e 1.9 MB/s | 164 kB 00:00 2024-04-02T13:25:44.001 INFO:teuthology.orchestra.run.smithi145.stdout:(16/174): librados-devel-19.0.0-2609.g191d963a. 1.5 MB/s | 127 kB 00:00 2024-04-02T13:25:44.065 INFO:teuthology.orchestra.run.smithi050.stdout:(5/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 4.6 MB/s | 1.5 MB 00:00 2024-04-02T13:25:44.101 INFO:teuthology.orchestra.run.smithi145.stdout:(17/174): libradosstriper1-19.0.0-2609.g191d963 4.9 MB/s | 500 kB 00:00 2024-04-02T13:25:44.198 INFO:teuthology.orchestra.run.smithi050.stdout:(6/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-02T13:25:44.872 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-04-02T13:25:44.951 INFO:teuthology.orchestra.run.smithi145.stdout:(18/174): librgw2-19.0.0-2609.g191d963a.el9.x86 6.0 MB/s | 5.1 MB 00:00 2024-04-02T13:25:45.026 INFO:teuthology.orchestra.run.smithi145.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 602 kB/s | 45 kB 00:00 2024-04-02T13:25:45.032 INFO:teuthology.orchestra.run.smithi050.stdout:(7/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 4.9 MB/s | 4.7 MB 00:00 2024-04-02T13:25:45.093 INFO:teuthology.orchestra.run.smithi145.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 2.0 MB/s | 134 kB 00:00 2024-04-02T13:25:45.185 INFO:teuthology.orchestra.run.smithi145.stdout:(21/174): python3-cephfs-19.0.0-2609.g191d963a. 1.7 MB/s | 161 kB 00:00 2024-04-02T13:25:45.285 INFO:teuthology.orchestra.run.smithi145.stdout:(22/174): python3-rados-19.0.0-2609.g191d963a.e 3.1 MB/s | 320 kB 00:00 2024-04-02T13:25:45.386 INFO:teuthology.orchestra.run.smithi145.stdout:(23/174): python3-rbd-19.0.0-2609.g191d963a.el9 2.9 MB/s | 299 kB 00:00 2024-04-02T13:25:45.478 INFO:teuthology.orchestra.run.smithi145.stdout:(24/174): python3-rgw-19.0.0-2609.g191d963a.el9 1.1 MB/s | 99 kB 00:00 2024-04-02T13:25:45.578 INFO:teuthology.orchestra.run.smithi145.stdout:(25/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 857 kB/s | 85 kB 00:00 2024-04-02T13:25:46.199 INFO:teuthology.orchestra.run.smithi050.stdout:(8/174): ceph-radosgw-19.0.0-2609.g191d963a.el9 8.4 MB/s | 9.8 MB 00:01 2024-04-02T13:25:46.253 INFO:teuthology.orchestra.run.smithi145.stdout:(26/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 4.6 MB/s | 3.1 MB 00:00 2024-04-02T13:25:46.257 INFO:teuthology.orchestra.run.smithi050.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 434 kB/s | 25 kB 00:00 2024-04-02T13:25:46.303 INFO:teuthology.orchestra.run.smithi145.stdout:(27/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 3.3 MB/s | 170 kB 00:00 2024-04-02T13:25:46.370 INFO:teuthology.orchestra.run.smithi145.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2609.g 508 kB/s | 34 kB 00:00 2024-04-02T13:25:46.454 INFO:teuthology.orchestra.run.smithi145.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 1.7 MB/s | 141 kB 00:00 2024-04-02T13:25:46.930 INFO:teuthology.orchestra.run.smithi145.stdout:(30/174): ceph-test-19.0.0-2609.g191d963a.el9.x 11 MB/s | 48 MB 00:04 2024-04-02T13:25:47.221 INFO:teuthology.orchestra.run.smithi145.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 4.6 MB/s | 3.5 MB 00:00 2024-04-02T13:25:47.313 INFO:teuthology.orchestra.run.smithi145.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 2.7 MB/s | 249 kB 00:00 2024-04-02T13:25:47.397 INFO:teuthology.orchestra.run.smithi145.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 593 kB/s | 49 kB 00:00 2024-04-02T13:25:47.481 INFO:teuthology.orchestra.run.smithi145.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 197 kB/s | 16 kB 00:00 2024-04-02T13:25:47.572 INFO:teuthology.orchestra.run.smithi145.stdout:(35/174): ceph-volume-19.0.0-2609.g191d963a.el9 2.8 MB/s | 264 kB 00:00 2024-04-02T13:25:47.688 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:25:47.689 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:25:47.704 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T13:25:47.706 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T13:25:47.706 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T13:25:47.707 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T13:25:47.710 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T13:25:47.713 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T13:25:47.714 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T13:25:47.756 INFO:teuthology.orchestra.run.smithi145.stdout:(36/174): cephadm-19.0.0-2609.g191d963a.el9.noa 4.1 MB/s | 762 kB 00:00 2024-04-02T13:25:47.804 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-02T13:25:47.825 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================================ 2024-04-02T13:25:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-02T13:25:47.826 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================================ 2024-04-02T13:25:47.826 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-02T13:25:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T13:25:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T13:25:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T13:25:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T13:25:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T13:25:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T13:25:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T13:25:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T13:25:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T13:25:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T13:25:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T13:25:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T13:25:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T13:25:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T13:25:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T13:25:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T13:25:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T13:25:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T13:25:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T13:25:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T13:25:47.827 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T13:25:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T13:25:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T13:25:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T13:25:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T13:25:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T13:25:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T13:25:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T13:25:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T13:25:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T13:25:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T13:25:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T13:25:47.828 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T13:25:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T13:25:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:25:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T13:25:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T13:25:47.829 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-04-02T13:25:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T13:25:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T13:25:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T13:25:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T13:25:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T13:25:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T13:25:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T13:25:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T13:25:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T13:25:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T13:25:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T13:25:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T13:25:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T13:25:47.830 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T13:25:47.831 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T13:25:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T13:25:47.833 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T13:25:47.834 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T13:25:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T13:25:47.836 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T13:25:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T13:25:47.838 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout:Installing weak dependencies: 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-02T13:25:47.839 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================================ 2024-04-02T13:25:47.840 INFO:teuthology.orchestra.run.smithi033.stdout:Install 150 Packages 2024-04-02T13:25:47.840 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 24 Packages 2024-04-02T13:25:47.840 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:25:47.840 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 226 M 2024-04-02T13:25:47.840 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-02T13:25:47.898 INFO:teuthology.orchestra.run.smithi145.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2024-04-02T13:25:47.923 INFO:teuthology.orchestra.run.smithi145.stdout:(38/174): cephfs-top-19.0.0-2609.g191d963a.el9. 109 kB/s | 18 kB 00:00 2024-04-02T13:25:48.298 INFO:teuthology.orchestra.run.smithi145.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2024-04-02T13:25:48.365 INFO:teuthology.orchestra.run.smithi145.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-04-02T13:25:48.391 INFO:teuthology.orchestra.run.smithi145.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 1.9 MB/s | 920 kB 00:00 2024-04-02T13:25:48.441 INFO:teuthology.orchestra.run.smithi145.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-04-02T13:25:48.466 INFO:teuthology.orchestra.run.smithi145.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 422 kB/s | 32 kB 00:00 2024-04-02T13:25:48.603 INFO:teuthology.orchestra.run.smithi145.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 128 kB/s | 17 kB 00:00 2024-04-02T13:25:48.650 INFO:teuthology.orchestra.run.smithi145.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-04-02T13:25:48.675 INFO:teuthology.orchestra.run.smithi145.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 137 kB/s | 10 kB 00:00 2024-04-02T13:25:48.701 INFO:teuthology.orchestra.run.smithi145.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-04-02T13:25:48.714 INFO:teuthology.orchestra.run.smithi033.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-02T13:25:48.726 INFO:teuthology.orchestra.run.smithi145.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-04-02T13:25:48.768 INFO:teuthology.orchestra.run.smithi145.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-04-02T13:25:48.802 INFO:teuthology.orchestra.run.smithi145.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 578 kB 00:00 2024-04-02T13:25:48.835 INFO:teuthology.orchestra.run.smithi145.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 700 kB/s | 46 kB 00:00 2024-04-02T13:25:48.878 INFO:teuthology.orchestra.run.smithi145.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-04-02T13:25:48.969 INFO:teuthology.orchestra.run.smithi145.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2024-04-02T13:25:48.995 INFO:teuthology.orchestra.run.smithi145.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.8 MB/s | 217 kB 00:00 2024-04-02T13:25:49.037 INFO:teuthology.orchestra.run.smithi145.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 823 kB/s | 34 kB 00:00 2024-04-02T13:25:49.154 INFO:teuthology.orchestra.run.smithi145.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-04-02T13:25:49.196 INFO:teuthology.orchestra.run.smithi145.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 9.8 MB/s | 1.5 MB 00:00 2024-04-02T13:25:49.238 INFO:teuthology.orchestra.run.smithi145.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 9.9 MB/s | 817 kB 00:00 2024-04-02T13:25:49.263 INFO:teuthology.orchestra.run.smithi145.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 132 kB 00:00 2024-04-02T13:25:49.463 INFO:teuthology.orchestra.run.smithi145.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.7 MB 00:00 2024-04-02T13:25:49.497 INFO:teuthology.orchestra.run.smithi145.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-04-02T13:25:49.499 INFO:teuthology.orchestra.run.smithi050.stdout:(10/174): ceph-common-19.0.0-2609.g191d963a.el9 2.6 MB/s | 21 MB 00:07 2024-04-02T13:25:49.522 INFO:teuthology.orchestra.run.smithi145.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-04-02T13:25:49.656 INFO:teuthology.orchestra.run.smithi033.stdout:(2/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 4.1 MB/s | 5.4 MB 00:01 2024-04-02T13:25:49.697 INFO:teuthology.orchestra.run.smithi145.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 5.8 MB 00:00 2024-04-02T13:25:49.706 INFO:teuthology.orchestra.run.smithi033.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 2.8 MB/s | 145 kB 00:00 2024-04-02T13:25:49.722 INFO:teuthology.orchestra.run.smithi145.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 122 kB/s | 24 kB 00:00 2024-04-02T13:25:49.748 INFO:teuthology.orchestra.run.smithi145.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2024-04-02T13:25:49.773 INFO:teuthology.orchestra.run.smithi145.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-04-02T13:25:49.807 INFO:teuthology.orchestra.run.smithi145.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 539 kB/s | 32 kB 00:00 2024-04-02T13:25:49.874 INFO:teuthology.orchestra.run.smithi033.stdout:(4/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 14 MB/s | 2.4 MB 00:00 2024-04-02T13:25:49.925 INFO:teuthology.orchestra.run.smithi050.stdout:(11/174): ceph-test-19.0.0-2609.g191d963a.el9.x 13 MB/s | 48 MB 00:03 2024-04-02T13:25:49.950 INFO:teuthology.orchestra.run.smithi050.stdout:(12/174): cephfs-mirror-19.0.0-2609.g191d963a.e 499 kB/s | 225 kB 00:00 2024-04-02T13:25:49.957 INFO:teuthology.orchestra.run.smithi145.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-04-02T13:25:49.966 INFO:teuthology.orchestra.run.smithi050.stdout:(13/174): cephfs-shell-19.0.0-2609.g191d963a.el 652 kB/s | 27 kB 00:00 2024-04-02T13:25:49.982 INFO:teuthology.orchestra.run.smithi145.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-04-02T13:25:49.990 INFO:teuthology.orchestra.run.smithi033.stdout:(5/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 13 MB/s | 1.5 MB 00:00 2024-04-02T13:25:50.007 INFO:teuthology.orchestra.run.smithi145.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-04-02T13:25:50.041 INFO:teuthology.orchestra.run.smithi145.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 69 kB 00:00 2024-04-02T13:25:50.067 INFO:teuthology.orchestra.run.smithi145.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 703 kB/s | 41 kB 00:00 2024-04-02T13:25:50.109 INFO:teuthology.orchestra.run.smithi145.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-04-02T13:25:50.134 INFO:teuthology.orchestra.run.smithi145.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-04-02T13:25:50.150 INFO:teuthology.orchestra.run.smithi050.stdout:(14/174): ceph-osd-19.0.0-2609.g191d963a.el9.x8 2.8 MB/s | 17 MB 00:05 2024-04-02T13:25:50.167 INFO:teuthology.orchestra.run.smithi050.stdout:(15/174): libcephfs-devel-19.0.0-2609.g191d963a 146 kB/s | 32 kB 00:00 2024-04-02T13:25:50.176 INFO:teuthology.orchestra.run.smithi145.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-04-02T13:25:50.207 INFO:teuthology.orchestra.run.smithi033.stdout:(6/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 596 kB/s | 890 kB 00:01 2024-04-02T13:25:50.209 INFO:teuthology.orchestra.run.smithi050.stdout:(16/174): libcephfs2-19.0.0-2609.g191d963a.el9. 3.0 MB/s | 747 kB 00:00 2024-04-02T13:25:50.218 INFO:teuthology.orchestra.run.smithi145.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 566 kB/s | 47 kB 00:00 2024-04-02T13:25:50.234 INFO:teuthology.orchestra.run.smithi145.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 735 kB/s | 43 kB 00:00 2024-04-02T13:25:50.242 INFO:teuthology.orchestra.run.smithi050.stdout:(17/174): libcephsqlite-19.0.0-2609.g191d963a.e 1.7 MB/s | 164 kB 00:00 2024-04-02T13:25:50.268 INFO:teuthology.orchestra.run.smithi050.stdout:(18/174): librados-devel-19.0.0-2609.g191d963a. 1.2 MB/s | 127 kB 00:00 2024-04-02T13:25:50.274 INFO:teuthology.orchestra.run.smithi033.stdout:(7/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 17 MB/s | 4.7 MB 00:00 2024-04-02T13:25:50.301 INFO:teuthology.orchestra.run.smithi050.stdout:(19/174): libradosstriper1-19.0.0-2609.g191d963 5.3 MB/s | 500 kB 00:00 2024-04-02T13:25:50.318 INFO:teuthology.orchestra.run.smithi050.stdout:(20/174): python3-ceph-argparse-19.0.0-2609.g19 898 kB/s | 45 kB 00:00 2024-04-02T13:25:50.369 INFO:teuthology.orchestra.run.smithi050.stdout:(21/174): python3-ceph-common-19.0.0-2609.g191d 2.0 MB/s | 134 kB 00:00 2024-04-02T13:25:50.394 INFO:teuthology.orchestra.run.smithi050.stdout:(22/174): python3-cephfs-19.0.0-2609.g191d963a. 2.1 MB/s | 161 kB 00:00 2024-04-02T13:25:50.401 INFO:teuthology.orchestra.run.smithi145.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 232 kB/s | 39 kB 00:00 2024-04-02T13:25:50.453 INFO:teuthology.orchestra.run.smithi050.stdout:(23/174): python3-rados-19.0.0-2609.g191d963a.e 3.7 MB/s | 320 kB 00:00 2024-04-02T13:25:50.536 INFO:teuthology.orchestra.run.smithi050.stdout:(24/174): python3-rbd-19.0.0-2609.g191d963a.el9 2.1 MB/s | 299 kB 00:00 2024-04-02T13:25:50.562 INFO:teuthology.orchestra.run.smithi050.stdout:(25/174): python3-rgw-19.0.0-2609.g191d963a.el9 916 kB/s | 99 kB 00:00 2024-04-02T13:25:50.612 INFO:teuthology.orchestra.run.smithi050.stdout:(26/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-02T13:25:50.677 INFO:teuthology.orchestra.run.smithi145.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-04-02T13:25:50.694 INFO:teuthology.orchestra.run.smithi145.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 142 kB/s | 41 kB 00:00 2024-04-02T13:25:50.719 INFO:teuthology.orchestra.run.smithi145.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 5.1 MB/s | 220 kB 00:00 2024-04-02T13:25:50.720 INFO:teuthology.orchestra.run.smithi050.stdout:(27/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 1.5 MB/s | 170 kB 00:00 2024-04-02T13:25:50.744 INFO:teuthology.orchestra.run.smithi145.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 697 kB/s | 35 kB 00:00 2024-04-02T13:25:50.770 INFO:teuthology.orchestra.run.smithi145.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 923 kB/s | 46 kB 00:00 2024-04-02T13:25:50.795 INFO:teuthology.orchestra.run.smithi145.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-04-02T13:25:50.812 INFO:teuthology.orchestra.run.smithi050.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2609.g 370 kB/s | 34 kB 00:00 2024-04-02T13:25:50.820 INFO:teuthology.orchestra.run.smithi145.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-04-02T13:25:50.846 INFO:teuthology.orchestra.run.smithi145.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-02T13:25:50.871 INFO:teuthology.orchestra.run.smithi145.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 880 kB/s | 44 kB 00:00 2024-04-02T13:25:50.896 INFO:teuthology.orchestra.run.smithi050.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 1.7 MB/s | 141 kB 00:00 2024-04-02T13:25:50.896 INFO:teuthology.orchestra.run.smithi145.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-02T13:25:50.922 INFO:teuthology.orchestra.run.smithi145.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-02T13:25:50.966 INFO:teuthology.orchestra.run.smithi033.stdout:(8/174): ceph-radosgw-19.0.0-2609.g191d963a.el9 14 MB/s | 9.8 MB 00:00 2024-04-02T13:25:50.972 INFO:teuthology.orchestra.run.smithi145.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-04-02T13:25:50.988 INFO:teuthology.orchestra.run.smithi050.stdout:(30/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 7.2 MB/s | 3.1 MB 00:00 2024-04-02T13:25:50.997 INFO:teuthology.orchestra.run.smithi145.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-04-02T13:25:51.016 INFO:teuthology.orchestra.run.smithi033.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 507 kB/s | 25 kB 00:00 2024-04-02T13:25:51.022 INFO:teuthology.orchestra.run.smithi145.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-04-02T13:25:51.048 INFO:teuthology.orchestra.run.smithi145.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-04-02T13:25:51.571 INFO:teuthology.orchestra.run.smithi050.stdout:(31/174): librgw2-19.0.0-2609.g191d963a.el9.x86 3.9 MB/s | 5.1 MB 00:01 2024-04-02T13:25:51.573 INFO:teuthology.orchestra.run.smithi145.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 340 kB/s | 187 kB 00:00 2024-04-02T13:25:51.682 INFO:teuthology.orchestra.run.smithi145.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-02T13:25:51.738 INFO:teuthology.orchestra.run.smithi050.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 1.5 MB/s | 249 kB 00:00 2024-04-02T13:25:51.748 INFO:teuthology.orchestra.run.smithi145.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-02T13:25:51.790 INFO:teuthology.orchestra.run.smithi145.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-02T13:25:51.830 INFO:teuthology.orchestra.run.smithi050.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 539 kB/s | 49 kB 00:00 2024-04-02T13:25:51.849 INFO:teuthology.orchestra.run.smithi145.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-04-02T13:25:51.874 INFO:teuthology.orchestra.run.smithi145.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 305 kB/s | 25 kB 00:00 2024-04-02T13:25:51.956 INFO:teuthology.orchestra.run.smithi050.stdout:(34/174): ceph-mgr-diskprediction-local-19.0.0- 7.7 MB/s | 7.4 MB 00:00 2024-04-02T13:25:51.966 INFO:teuthology.orchestra.run.smithi145.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.6 MB/s | 245 kB 00:00 2024-04-02T13:25:51.973 INFO:teuthology.orchestra.run.smithi050.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2609.g1 115 kB/s | 16 kB 00:00 2024-04-02T13:25:52.017 INFO:teuthology.orchestra.run.smithi145.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 4.8 MB/s | 248 kB 00:00 2024-04-02T13:25:52.031 INFO:teuthology.orchestra.run.smithi050.stdout:(36/174): ceph-volume-19.0.0-2609.g191d963a.el9 3.4 MB/s | 264 kB 00:00 2024-04-02T13:25:52.084 INFO:teuthology.orchestra.run.smithi145.stdout:(102/174): openssl-devel-3.0.7-27.el9.x86_64.rp 18 MB/s | 4.1 MB 00:00 2024-04-02T13:25:52.101 INFO:teuthology.orchestra.run.smithi145.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-04-02T13:25:52.168 INFO:teuthology.orchestra.run.smithi050.stdout:(37/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 2.8 MB/s | 3.5 MB 00:01 2024-04-02T13:25:52.207 INFO:teuthology.orchestra.run.smithi050.stdout:(38/174): cephfs-top-19.0.0-2609.g191d963a.el9. 104 kB/s | 18 kB 00:00 2024-04-02T13:25:52.241 INFO:teuthology.orchestra.run.smithi050.stdout:(39/174): cephadm-19.0.0-2609.g191d963a.el9.noa 2.8 MB/s | 762 kB 00:00 2024-04-02T13:25:52.301 INFO:teuthology.orchestra.run.smithi145.stdout:(104/174): ceph-osd-19.0.0-2609.g191d963a.el9.x 1.6 MB/s | 17 MB 00:10 2024-04-02T13:25:52.433 INFO:teuthology.orchestra.run.smithi050.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 766 kB/s | 106 kB 00:00 2024-04-02T13:25:52.508 INFO:teuthology.orchestra.run.smithi050.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 662 kB 00:00 2024-04-02T13:25:52.542 INFO:teuthology.orchestra.run.smithi050.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 920 kB 00:00 2024-04-02T13:25:52.567 INFO:teuthology.orchestra.run.smithi050.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 307 kB 00:00 2024-04-02T13:25:52.592 INFO:teuthology.orchestra.run.smithi050.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 380 kB/s | 32 kB 00:00 2024-04-02T13:25:52.809 INFO:teuthology.orchestra.run.smithi050.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 79 kB/s | 17 kB 00:00 2024-04-02T13:25:52.834 INFO:teuthology.orchestra.run.smithi050.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 42 kB/s | 10 kB 00:00 2024-04-02T13:25:52.901 INFO:teuthology.orchestra.run.smithi145.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 27 kB/s | 22 kB 00:00 2024-04-02T13:25:52.926 INFO:teuthology.orchestra.run.smithi050.stdout:(47/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.8 MB/s | 3.0 MB 00:00 2024-04-02T13:25:52.952 INFO:teuthology.orchestra.run.smithi050.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2024-04-02T13:25:52.977 INFO:teuthology.orchestra.run.smithi050.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 313 kB/s | 44 kB 00:00 2024-04-02T13:25:53.019 INFO:teuthology.orchestra.run.smithi050.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 661 kB 00:00 2024-04-02T13:25:53.026 INFO:teuthology.orchestra.run.smithi145.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 756 kB/s | 548 kB 00:00 2024-04-02T13:25:53.061 INFO:teuthology.orchestra.run.smithi050.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-04-02T13:25:53.086 INFO:teuthology.orchestra.run.smithi050.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 429 kB/s | 46 kB 00:00 2024-04-02T13:25:53.111 INFO:teuthology.orchestra.run.smithi050.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-04-02T13:25:53.135 INFO:teuthology.orchestra.run.smithi145.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-04-02T13:25:53.137 INFO:teuthology.orchestra.run.smithi050.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-04-02T13:25:53.162 INFO:teuthology.orchestra.run.smithi050.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-04-02T13:25:53.187 INFO:teuthology.orchestra.run.smithi050.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 456 kB/s | 34 kB 00:00 2024-04-02T13:25:53.502 INFO:teuthology.orchestra.run.smithi145.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 514 kB/s | 309 kB 00:00 2024-04-02T13:25:53.569 INFO:teuthology.orchestra.run.smithi145.stdout:(109/174): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-02T13:25:53.603 INFO:teuthology.orchestra.run.smithi145.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-02T13:25:53.604 INFO:teuthology.orchestra.run.smithi050.stdout:(57/174): postgresql-contrib-13.14-1.el9.x86_64 1.9 MB/s | 817 kB 00:00 2024-04-02T13:25:53.669 INFO:teuthology.orchestra.run.smithi145.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-04-02T13:25:53.671 INFO:teuthology.orchestra.run.smithi050.stdout:(58/174): openblas-openmp-0.3.21-2.el9.x86_64.r 8.9 MB/s | 4.7 MB 00:00 2024-04-02T13:25:53.711 INFO:teuthology.orchestra.run.smithi145.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-04-02T13:25:53.713 INFO:teuthology.orchestra.run.smithi050.stdout:(59/174): postgresql-13.14-1.el9.x86_64.rpm 2.9 MB/s | 1.5 MB 00:00 2024-04-02T13:25:53.738 INFO:teuthology.orchestra.run.smithi050.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 992 kB/s | 132 kB 00:00 2024-04-02T13:25:53.778 INFO:teuthology.orchestra.run.smithi145.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-02T13:25:53.887 INFO:teuthology.orchestra.run.smithi145.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 5.0 MB/s | 548 kB 00:00 2024-04-02T13:25:53.946 INFO:teuthology.orchestra.run.smithi145.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 838 kB 00:00 2024-04-02T13:25:53.979 INFO:teuthology.orchestra.run.smithi145.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2024-04-02T13:25:54.030 INFO:teuthology.orchestra.run.smithi050.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 268 kB/s | 78 kB 00:00 2024-04-02T13:25:54.038 INFO:teuthology.orchestra.run.smithi145.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-04-02T13:25:54.071 INFO:teuthology.orchestra.run.smithi145.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-02T13:25:54.097 INFO:teuthology.orchestra.run.smithi050.stdout:(62/174): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.7 MB 00:00 2024-04-02T13:25:54.134 INFO:teuthology.orchestra.run.smithi033.stdout:(10/174): ceph-test-19.0.0-2609.g191d963a.el9.x 15 MB/s | 48 MB 00:03 2024-04-02T13:25:54.139 INFO:teuthology.orchestra.run.smithi050.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 2.5 MB/s | 1.0 MB 00:00 2024-04-02T13:25:54.147 INFO:teuthology.orchestra.run.smithi145.stdout:(119/174): protobuf-compiler-3.14.0-13.el9.x86_ 422 kB/s | 863 kB 00:02 2024-04-02T13:25:54.172 INFO:teuthology.orchestra.run.smithi145.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-02T13:25:54.192 INFO:teuthology.orchestra.run.smithi033.stdout:(11/174): cephfs-mirror-19.0.0-2609.g191d963a.e 3.8 MB/s | 225 kB 00:00 2024-04-02T13:25:54.242 INFO:teuthology.orchestra.run.smithi033.stdout:(12/174): cephfs-shell-19.0.0-2609.g191d963a.el 544 kB/s | 27 kB 00:00 2024-04-02T13:25:54.273 INFO:teuthology.orchestra.run.smithi145.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-04-02T13:25:54.284 INFO:teuthology.orchestra.run.smithi033.stdout:(13/174): libcephfs-devel-19.0.0-2609.g191d963a 759 kB/s | 32 kB 00:00 2024-04-02T13:25:54.356 INFO:teuthology.orchestra.run.smithi145.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 192 kB/s | 35 kB 00:00 2024-04-02T13:25:54.357 INFO:teuthology.orchestra.run.smithi050.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2024-04-02T13:25:54.359 INFO:teuthology.orchestra.run.smithi033.stdout:(14/174): libcephfs2-19.0.0-2609.g191d963a.el9. 9.7 MB/s | 747 kB 00:00 2024-04-02T13:25:54.381 INFO:teuthology.orchestra.run.smithi050.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 86 kB/s | 24 kB 00:00 2024-04-02T13:25:54.389 INFO:teuthology.orchestra.run.smithi145.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-04-02T13:25:54.407 INFO:teuthology.orchestra.run.smithi050.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 68 kB/s | 18 kB 00:00 2024-04-02T13:25:54.410 INFO:teuthology.orchestra.run.smithi033.stdout:(15/174): libcephsqlite-19.0.0-2609.g191d963a.e 3.2 MB/s | 164 kB 00:00 2024-04-02T13:25:54.432 INFO:teuthology.orchestra.run.smithi050.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-04-02T13:25:54.457 INFO:teuthology.orchestra.run.smithi050.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 631 kB/s | 32 kB 00:00 2024-04-02T13:25:54.460 INFO:teuthology.orchestra.run.smithi033.stdout:(16/174): librados-devel-19.0.0-2609.g191d963a. 2.5 MB/s | 127 kB 00:00 2024-04-02T13:25:54.498 INFO:teuthology.orchestra.run.smithi145.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-02T13:25:54.519 INFO:teuthology.orchestra.run.smithi033.stdout:(17/174): libradosstriper1-19.0.0-2609.g191d963 8.4 MB/s | 500 kB 00:00 2024-04-02T13:25:54.590 INFO:teuthology.orchestra.run.smithi145.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-04-02T13:25:54.616 INFO:teuthology.orchestra.run.smithi145.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-04-02T13:25:54.649 INFO:teuthology.orchestra.run.smithi050.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-04-02T13:25:54.674 INFO:teuthology.orchestra.run.smithi050.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 899 kB/s | 217 kB 00:00 2024-04-02T13:25:54.691 INFO:teuthology.orchestra.run.smithi145.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-02T13:25:54.691 INFO:teuthology.orchestra.run.smithi050.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 750 kB/s | 174 kB 00:00 2024-04-02T13:25:54.716 INFO:teuthology.orchestra.run.smithi145.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-02T13:25:54.725 INFO:teuthology.orchestra.run.smithi050.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 919 kB/s | 69 kB 00:00 2024-04-02T13:25:54.767 INFO:teuthology.orchestra.run.smithi050.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 546 kB/s | 41 kB 00:00 2024-04-02T13:25:54.791 INFO:teuthology.orchestra.run.smithi145.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-02T13:25:54.792 INFO:teuthology.orchestra.run.smithi050.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 238 kB 00:00 2024-04-02T13:25:54.802 INFO:teuthology.orchestra.run.smithi033.stdout:(18/174): librgw2-19.0.0-2609.g191d963a.el9.x86 18 MB/s | 5.1 MB 00:00 2024-04-02T13:25:54.817 INFO:teuthology.orchestra.run.smithi145.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-02T13:25:54.842 INFO:teuthology.orchestra.run.smithi145.stdout:(131/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 437 kB/s | 303 kB 00:00 2024-04-02T13:25:54.853 INFO:teuthology.orchestra.run.smithi033.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 901 kB/s | 45 kB 00:00 2024-04-02T13:25:54.867 INFO:teuthology.orchestra.run.smithi050.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 929 kB/s | 132 kB 00:00 2024-04-02T13:25:54.892 INFO:teuthology.orchestra.run.smithi050.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 211 kB 00:00 2024-04-02T13:25:54.903 INFO:teuthology.orchestra.run.smithi033.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 2.6 MB/s | 134 kB 00:00 2024-04-02T13:25:54.917 INFO:teuthology.orchestra.run.smithi145.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-04-02T13:25:54.918 INFO:teuthology.orchestra.run.smithi050.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 377 kB/s | 47 kB 00:00 2024-04-02T13:25:54.942 INFO:teuthology.orchestra.run.smithi145.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 369 kB/s | 46 kB 00:00 2024-04-02T13:25:54.943 INFO:teuthology.orchestra.run.smithi050.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 856 kB/s | 43 kB 00:00 2024-04-02T13:25:54.953 INFO:teuthology.orchestra.run.smithi033.stdout:(21/174): python3-cephfs-19.0.0-2609.g191d963a. 3.2 MB/s | 161 kB 00:00 2024-04-02T13:25:54.968 INFO:teuthology.orchestra.run.smithi145.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 628 kB/s | 79 kB 00:00 2024-04-02T13:25:55.004 INFO:teuthology.orchestra.run.smithi033.stdout:(22/174): python3-rados-19.0.0-2609.g191d963a.e 6.2 MB/s | 320 kB 00:00 2024-04-02T13:25:55.018 INFO:teuthology.orchestra.run.smithi145.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 574 kB/s | 58 kB 00:00 2024-04-02T13:25:55.043 INFO:teuthology.orchestra.run.smithi145.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-02T13:25:55.054 INFO:teuthology.orchestra.run.smithi033.stdout:(23/174): python3-rbd-19.0.0-2609.g191d963a.el9 5.9 MB/s | 299 kB 00:00 2024-04-02T13:25:55.068 INFO:teuthology.orchestra.run.smithi145.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-02T13:25:55.096 INFO:teuthology.orchestra.run.smithi033.stdout:(24/174): python3-rgw-19.0.0-2609.g191d963a.el9 2.3 MB/s | 99 kB 00:00 2024-04-02T13:25:55.127 INFO:teuthology.orchestra.run.smithi145.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 822 kB/s | 90 kB 00:00 2024-04-02T13:25:55.146 INFO:teuthology.orchestra.run.smithi033.stdout:(25/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-02T13:25:55.161 INFO:teuthology.orchestra.run.smithi145.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 240 kB/s | 28 kB 00:00 2024-04-02T13:25:55.185 INFO:teuthology.orchestra.run.smithi050.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 145 kB/s | 39 kB 00:00 2024-04-02T13:25:55.186 INFO:teuthology.orchestra.run.smithi145.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 262 kB/s | 31 kB 00:00 2024-04-02T13:25:55.210 INFO:teuthology.orchestra.run.smithi050.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 154 kB/s | 41 kB 00:00 2024-04-02T13:25:55.228 INFO:teuthology.orchestra.run.smithi145.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-02T13:25:55.261 INFO:teuthology.orchestra.run.smithi145.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2024-04-02T13:25:55.295 INFO:teuthology.orchestra.run.smithi145.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-04-02T13:25:55.329 INFO:teuthology.orchestra.run.smithi145.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-02T13:25:55.339 INFO:teuthology.orchestra.run.smithi033.stdout:(26/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 16 MB/s | 3.1 MB 00:00 2024-04-02T13:25:55.362 INFO:teuthology.orchestra.run.smithi145.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-02T13:25:55.385 INFO:teuthology.orchestra.run.smithi050.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2024-04-02T13:25:55.389 INFO:teuthology.orchestra.run.smithi033.stdout:(27/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 3.3 MB/s | 170 kB 00:00 2024-04-02T13:25:55.421 INFO:teuthology.orchestra.run.smithi145.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 713 kB/s | 90 kB 00:00 2024-04-02T13:25:55.431 INFO:teuthology.orchestra.run.smithi033.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2609.g 816 kB/s | 34 kB 00:00 2024-04-02T13:25:55.446 INFO:teuthology.orchestra.run.smithi145.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-02T13:25:55.472 INFO:teuthology.orchestra.run.smithi145.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-04-02T13:25:55.481 INFO:teuthology.orchestra.run.smithi033.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 2.8 MB/s | 141 kB 00:00 2024-04-02T13:25:55.572 INFO:teuthology.orchestra.run.smithi145.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-02T13:25:55.577 INFO:teuthology.orchestra.run.smithi050.stdout:(82/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-02T13:25:55.603 INFO:teuthology.orchestra.run.smithi050.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 89 kB/s | 35 kB 00:00 2024-04-02T13:25:55.614 INFO:teuthology.orchestra.run.smithi145.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-02T13:25:55.628 INFO:teuthology.orchestra.run.smithi050.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 193 kB/s | 46 kB 00:00 2024-04-02T13:25:55.653 INFO:teuthology.orchestra.run.smithi050.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-04-02T13:25:55.679 INFO:teuthology.orchestra.run.smithi050.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-04-02T13:25:55.690 INFO:teuthology.orchestra.run.smithi033.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 17 MB/s | 3.5 MB 00:00 2024-04-02T13:25:55.703 INFO:teuthology.orchestra.run.smithi050.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 838 kB/s | 63 kB 00:00 2024-04-02T13:25:55.714 INFO:teuthology.orchestra.run.smithi145.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 23 MB/s | 2.3 MB 00:00 2024-04-02T13:25:55.729 INFO:teuthology.orchestra.run.smithi050.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 585 kB/s | 44 kB 00:00 2024-04-02T13:25:55.754 INFO:teuthology.orchestra.run.smithi050.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-02T13:25:55.779 INFO:teuthology.orchestra.run.smithi050.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-04-02T13:25:55.813 INFO:teuthology.orchestra.run.smithi050.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-04-02T13:25:55.823 INFO:teuthology.orchestra.run.smithi145.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 17 MB/s | 1.8 MB 00:00 2024-04-02T13:25:55.838 INFO:teuthology.orchestra.run.smithi050.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-04-02T13:25:55.848 INFO:teuthology.orchestra.run.smithi145.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.6 MB/s | 41 kB 00:00 2024-04-02T13:25:55.863 INFO:teuthology.orchestra.run.smithi050.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-04-02T13:25:55.889 INFO:teuthology.orchestra.run.smithi050.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-04-02T13:25:55.890 INFO:teuthology.orchestra.run.smithi145.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-04-02T13:25:55.940 INFO:teuthology.orchestra.run.smithi145.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-04-02T13:25:56.031 INFO:teuthology.orchestra.run.smithi050.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-04-02T13:25:56.064 INFO:teuthology.orchestra.run.smithi050.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-02T13:25:56.131 INFO:teuthology.orchestra.run.smithi050.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-02T13:25:56.157 INFO:teuthology.orchestra.run.smithi050.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-04-02T13:25:56.174 INFO:teuthology.orchestra.run.smithi145.stdout:(156/174): librbd1-19.0.0-2609.g191d963a.el9.x8 5.2 MB/s | 3.1 MB 00:00 2024-04-02T13:25:56.181 INFO:teuthology.orchestra.run.smithi033.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-04-02T13:25:56.224 INFO:teuthology.orchestra.run.smithi145.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 252 kB/s | 71 kB 00:00 2024-04-02T13:25:56.232 INFO:teuthology.orchestra.run.smithi033.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 4.9 MB/s | 249 kB 00:00 2024-04-02T13:25:56.282 INFO:teuthology.orchestra.run.smithi033.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 990 kB/s | 49 kB 00:00 2024-04-02T13:25:56.324 INFO:teuthology.orchestra.run.smithi033.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 394 kB/s | 16 kB 00:00 2024-04-02T13:25:56.365 INFO:teuthology.orchestra.run.smithi050.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 71 kB/s | 21 kB 00:00 2024-04-02T13:25:56.374 INFO:teuthology.orchestra.run.smithi145.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 182 kB/s | 36 kB 00:00 2024-04-02T13:25:56.374 INFO:teuthology.orchestra.run.smithi033.stdout:(35/174): ceph-volume-19.0.0-2609.g191d963a.el9 5.2 MB/s | 264 kB 00:00 2024-04-02T13:25:56.432 INFO:teuthology.orchestra.run.smithi050.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 85 kB/s | 25 kB 00:00 2024-04-02T13:25:56.441 INFO:teuthology.orchestra.run.smithi145.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 887 kB/s | 192 kB 00:00 2024-04-02T13:25:56.441 INFO:teuthology.orchestra.run.smithi033.stdout:(36/174): cephadm-19.0.0-2609.g191d963a.el9.noa 11 MB/s | 762 kB 00:00 2024-04-02T13:25:56.484 INFO:teuthology.orchestra.run.smithi033.stdout:(37/174): cephfs-top-19.0.0-2609.g191d963a.el9. 437 kB/s | 18 kB 00:00 2024-04-02T13:25:56.558 INFO:teuthology.orchestra.run.smithi145.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 197 kB/s | 23 kB 00:00 2024-04-02T13:25:56.667 INFO:teuthology.orchestra.run.smithi145.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 875 kB/s | 256 kB 00:00 2024-04-02T13:25:56.692 INFO:teuthology.orchestra.run.smithi145.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 812 kB/s | 108 kB 00:00 2024-04-02T13:25:56.725 INFO:teuthology.orchestra.run.smithi033.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 662 kB 00:00 2024-04-02T13:25:56.749 INFO:teuthology.orchestra.run.smithi050.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 639 kB/s | 245 kB 00:00 2024-04-02T13:25:56.784 INFO:teuthology.orchestra.run.smithi145.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 155 kB 00:00 2024-04-02T13:25:56.792 INFO:teuthology.orchestra.run.smithi033.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 920 kB 00:00 2024-04-02T13:25:56.809 INFO:teuthology.orchestra.run.smithi145.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-04-02T13:25:56.816 INFO:teuthology.orchestra.run.smithi050.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 646 kB/s | 248 kB 00:00 2024-04-02T13:25:56.834 INFO:teuthology.orchestra.run.smithi033.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2024-04-02T13:25:56.858 INFO:teuthology.orchestra.run.smithi050.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 863 kB/s | 94 kB 00:00 2024-04-02T13:25:56.876 INFO:teuthology.orchestra.run.smithi033.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-04-02T13:25:56.901 INFO:teuthology.orchestra.run.smithi145.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 547 kB/s | 64 kB 00:00 2024-04-02T13:25:56.918 INFO:teuthology.orchestra.run.smithi033.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 758 kB/s | 32 kB 00:00 2024-04-02T13:25:56.926 INFO:teuthology.orchestra.run.smithi145.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 245 kB/s | 29 kB 00:00 2024-04-02T13:25:56.958 INFO:teuthology.orchestra.run.smithi050.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 157 kB/s | 22 kB 00:00 2024-04-02T13:25:57.052 INFO:teuthology.orchestra.run.smithi033.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-04-02T13:25:57.094 INFO:teuthology.orchestra.run.smithi033.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 413 kB/s | 17 kB 00:00 2024-04-02T13:25:57.134 INFO:teuthology.orchestra.run.smithi050.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.2 MB/s | 4.1 MB 00:00 2024-04-02T13:25:57.153 INFO:teuthology.orchestra.run.smithi033.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 171 kB/s | 10 kB 00:00 2024-04-02T13:25:57.195 INFO:teuthology.orchestra.run.smithi033.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2024-04-02T13:25:57.227 INFO:teuthology.orchestra.run.smithi145.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 3.7 MB/s | 1.2 MB 00:00 2024-04-02T13:25:57.245 INFO:teuthology.orchestra.run.smithi033.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-04-02T13:25:57.295 INFO:teuthology.orchestra.run.smithi033.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-02T13:25:57.318 INFO:teuthology.orchestra.run.smithi050.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-04-02T13:25:57.360 INFO:teuthology.orchestra.run.smithi145.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.0 MB/s | 2.2 MB 00:00 2024-04-02T13:25:57.362 INFO:teuthology.orchestra.run.smithi033.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 578 kB 00:00 2024-04-02T13:25:57.404 INFO:teuthology.orchestra.run.smithi033.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-04-02T13:25:57.409 INFO:teuthology.orchestra.run.smithi050.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-02T13:25:57.411 INFO:teuthology.orchestra.run.smithi145.stdout:(169/174): librados2-19.0.0-2609.g191d963a.el9. 1.7 MB/s | 3.4 MB 00:01 2024-04-02T13:25:57.434 INFO:teuthology.orchestra.run.smithi050.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-04-02T13:25:57.455 INFO:teuthology.orchestra.run.smithi033.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-04-02T13:25:57.494 INFO:teuthology.orchestra.run.smithi145.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.5 MB/s | 2.3 MB 00:00 2024-04-02T13:25:57.505 INFO:teuthology.orchestra.run.smithi033.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-02T13:25:57.510 INFO:teuthology.orchestra.run.smithi050.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-04-02T13:25:57.519 INFO:teuthology.orchestra.run.smithi145.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 2.9 MB/s | 465 kB 00:00 2024-04-02T13:25:57.535 INFO:teuthology.orchestra.run.smithi050.stdout:(110/174): gperftools-libs-2.9.1-2.el9.x86_64.r 770 kB/s | 309 kB 00:00 2024-04-02T13:25:57.555 INFO:teuthology.orchestra.run.smithi033.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-04-02T13:25:57.580 INFO:teuthology.orchestra.run.smithi033.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 1.3 MB/s | 34 kB 00:00 2024-04-02T13:25:57.603 INFO:teuthology.orchestra.run.smithi145.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 892 kB/s | 74 kB 00:00 2024-04-02T13:25:57.613 INFO:teuthology.orchestra.run.smithi050.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-04-02T13:25:57.637 INFO:teuthology.orchestra.run.smithi145.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2024-04-02T13:25:57.704 INFO:teuthology.orchestra.run.smithi145.stdout:(174/174): util-linux-user-2.37.4-18.el9.x86_64 106 kB/s | 31 kB 00:00 2024-04-02T13:25:57.712 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:25:57.712 INFO:teuthology.orchestra.run.smithi145.stdout:Total 13 MB/s | 226 MB 00:17 2024-04-02T13:25:57.748 INFO:teuthology.orchestra.run.smithi033.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-04-02T13:25:57.823 INFO:teuthology.orchestra.run.smithi033.stdout:(56/174): postgresql-13.14-1.el9.x86_64.rpm 20 MB/s | 1.5 MB 00:00 2024-04-02T13:25:57.882 INFO:teuthology.orchestra.run.smithi033.stdout:(57/174): postgresql-contrib-13.14-1.el9.x86_64 14 MB/s | 817 kB 00:00 2024-04-02T13:25:57.924 INFO:teuthology.orchestra.run.smithi033.stdout:(58/174): postgresql-private-libs-13.14-1.el9.x 3.1 MB/s | 132 kB 00:00 2024-04-02T13:25:58.061 INFO:teuthology.orchestra.run.smithi050.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 128 kB/s | 67 kB 00:00 2024-04-02T13:25:58.086 INFO:teuthology.orchestra.run.smithi050.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 318 kB/s | 151 kB 00:00 2024-04-02T13:25:58.095 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-04-02T13:25:58.116 INFO:teuthology.orchestra.run.smithi033.stdout:(59/174): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.7 MB 00:00 2024-04-02T13:25:58.195 INFO:teuthology.orchestra.run.smithi050.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-04-02T13:25:58.245 INFO:teuthology.orchestra.run.smithi050.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-04-02T13:25:58.270 INFO:teuthology.orchestra.run.smithi050.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 578 kB/s | 43 kB 00:00 2024-04-02T13:25:58.274 INFO:teuthology.orchestra.run.smithi033.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-04-02T13:25:58.296 INFO:teuthology.orchestra.run.smithi050.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-04-02T13:25:58.316 INFO:teuthology.orchestra.run.smithi033.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.8 MB/s | 78 kB 00:00 2024-04-02T13:25:58.321 INFO:teuthology.orchestra.run.smithi050.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 297 kB/s | 15 kB 00:00 2024-04-02T13:25:58.354 INFO:teuthology.orchestra.run.smithi050.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-02T13:25:58.383 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-04-02T13:25:58.384 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-04-02T13:25:58.405 INFO:teuthology.orchestra.run.smithi050.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-02T13:25:58.430 INFO:teuthology.orchestra.run.smithi050.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-02T13:25:58.464 INFO:teuthology.orchestra.run.smithi050.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 603 kB/s | 35 kB 00:00 2024-04-02T13:25:58.489 INFO:teuthology.orchestra.run.smithi050.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-04-02T13:25:58.540 INFO:teuthology.orchestra.run.smithi050.stdout:(124/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 838 kB 00:00 2024-04-02T13:25:58.542 INFO:teuthology.orchestra.run.smithi033.stdout:(62/174): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2024-04-02T13:25:58.565 INFO:teuthology.orchestra.run.smithi050.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-02T13:25:58.584 INFO:teuthology.orchestra.run.smithi033.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 589 kB/s | 24 kB 00:00 2024-04-02T13:25:58.590 INFO:teuthology.orchestra.run.smithi050.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-02T13:25:58.607 INFO:teuthology.orchestra.run.smithi050.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 422 kB/s | 18 kB 00:00 2024-04-02T13:25:58.625 INFO:teuthology.orchestra.run.smithi033.stdout:(64/174): python3-jsonpointer-2.0-4.el9.noarch. 435 kB/s | 18 kB 00:00 2024-04-02T13:25:58.649 INFO:teuthology.orchestra.run.smithi050.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-02T13:25:58.666 INFO:teuthology.orchestra.run.smithi050.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 306 kB/s | 23 kB 00:00 2024-04-02T13:25:58.676 INFO:teuthology.orchestra.run.smithi033.stdout:(65/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-04-02T13:25:58.683 INFO:teuthology.orchestra.run.smithi050.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-04-02T13:25:58.700 INFO:teuthology.orchestra.run.smithi050.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-04-02T13:25:58.718 INFO:teuthology.orchestra.run.smithi033.stdout:(66/174): python3-markupsafe-1.1.1-12.el9.x86_6 755 kB/s | 32 kB 00:00 2024-04-02T13:25:58.725 INFO:teuthology.orchestra.run.smithi050.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-02T13:25:58.784 INFO:teuthology.orchestra.run.smithi050.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 942 kB/s | 79 kB 00:00 2024-04-02T13:25:58.809 INFO:teuthology.orchestra.run.smithi050.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 690 kB/s | 58 kB 00:00 2024-04-02T13:25:58.851 INFO:teuthology.orchestra.run.smithi050.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-02T13:25:58.876 INFO:teuthology.orchestra.run.smithi050.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-04-02T13:25:58.901 INFO:teuthology.orchestra.run.smithi050.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-02T13:25:58.926 INFO:teuthology.orchestra.run.smithi033.stdout:(67/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2024-04-02T13:25:58.927 INFO:teuthology.orchestra.run.smithi050.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-04-02T13:25:58.952 INFO:teuthology.orchestra.run.smithi050.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-04-02T13:25:58.968 INFO:teuthology.orchestra.run.smithi033.stdout:(68/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.1 MB/s | 217 kB 00:00 2024-04-02T13:25:58.986 INFO:teuthology.orchestra.run.smithi050.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-04-02T13:25:59.011 INFO:teuthology.orchestra.run.smithi050.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-02T13:25:59.027 INFO:teuthology.orchestra.run.smithi033.stdout:(69/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 174 kB 00:00 2024-04-02T13:25:59.045 INFO:teuthology.orchestra.run.smithi050.stdout:(142/174): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2024-04-02T13:25:59.070 INFO:teuthology.orchestra.run.smithi050.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-04-02T13:25:59.078 INFO:teuthology.orchestra.run.smithi033.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-04-02T13:25:59.096 INFO:teuthology.orchestra.run.smithi050.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 607 kB/s | 51 kB 00:00 2024-04-02T13:25:59.129 INFO:teuthology.orchestra.run.smithi050.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-02T13:25:59.136 INFO:teuthology.orchestra.run.smithi033.stdout:(71/174): python3-prettytable-0.7.2-27.el9.noar 707 kB/s | 41 kB 00:00 2024-04-02T13:25:59.163 INFO:teuthology.orchestra.run.smithi050.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-02T13:25:59.187 INFO:teuthology.orchestra.run.smithi033.stdout:(72/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-04-02T13:25:59.188 INFO:teuthology.orchestra.run.smithi050.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-04-02T13:25:59.213 INFO:teuthology.orchestra.run.smithi050.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-02T13:25:59.245 INFO:teuthology.orchestra.run.smithi033.stdout:(73/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-04-02T13:25:59.296 INFO:teuthology.orchestra.run.smithi033.stdout:(74/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-04-02T13:25:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:(75/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-04-02T13:25:59.364 INFO:teuthology.orchestra.run.smithi050.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 954 kB/s | 192 kB 00:00 2024-04-02T13:25:59.396 INFO:teuthology.orchestra.run.smithi033.stdout:(76/174): python3-requests-oauthlib-1.3.0-12.el 732 kB/s | 43 kB 00:00 2024-04-02T13:25:59.406 INFO:teuthology.orchestra.run.smithi050.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-02T13:25:59.756 INFO:teuthology.orchestra.run.smithi050.stdout:(151/174): librados2-19.0.0-2609.g191d963a.el9. 6.3 MB/s | 3.4 MB 00:00 2024-04-02T13:25:59.815 INFO:teuthology.orchestra.run.smithi050.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 5.6 MB/s | 2.3 MB 00:00 2024-04-02T13:25:59.890 INFO:teuthology.orchestra.run.smithi050.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 14 MB/s | 1.8 MB 00:00 2024-04-02T13:25:59.915 INFO:teuthology.orchestra.run.smithi050.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 422 kB/s | 41 kB 00:00 2024-04-02T13:25:59.941 INFO:teuthology.orchestra.run.smithi050.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.8 MB/s | 247 kB 00:00 2024-04-02T13:25:59.966 INFO:teuthology.orchestra.run.smithi050.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-04-02T13:26:00.049 INFO:teuthology.orchestra.run.smithi050.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 438 kB/s | 36 kB 00:00 2024-04-02T13:26:00.055 INFO:teuthology.orchestra.run.smithi033.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-02T13:26:00.078 INFO:teuthology.orchestra.run.smithi050.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 536 kB/s | 71 kB 00:00 2024-04-02T13:26:00.097 INFO:teuthology.orchestra.run.smithi033.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 934 kB/s | 39 kB 00:00 2024-04-02T13:26:00.139 INFO:teuthology.orchestra.run.smithi033.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 984 kB/s | 41 kB 00:00 2024-04-02T13:26:00.158 INFO:teuthology.orchestra.run.smithi050.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-04-02T13:26:00.197 INFO:teuthology.orchestra.run.smithi033.stdout:(80/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2024-04-02T13:26:00.233 INFO:teuthology.orchestra.run.smithi050.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 307 kB/s | 23 kB 00:00 2024-04-02T13:26:00.239 INFO:teuthology.orchestra.run.smithi033.stdout:(81/174): qatlib-service-24.02.0-1.el9.x86_64.r 835 kB/s | 35 kB 00:00 2024-04-02T13:26:00.309 INFO:teuthology.orchestra.run.smithi050.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-04-02T13:26:00.359 INFO:teuthology.orchestra.run.smithi050.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-04-02T13:26:00.384 INFO:teuthology.orchestra.run.smithi050.stdout:(163/174): elfutils-libs-0.190-2.el9.x86_64.rpm 827 kB/s | 256 kB 00:00 2024-04-02T13:26:00.406 INFO:teuthology.orchestra.run.smithi033.stdout:(82/174): ceph-osd-19.0.0-2609.g191d963a.el9.x8 1.7 MB/s | 17 MB 00:10 2024-04-02T13:26:00.426 INFO:teuthology.orchestra.run.smithi050.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-04-02T13:26:00.431 INFO:teuthology.orchestra.run.smithi033.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 241 kB/s | 46 kB 00:00 2024-04-02T13:26:00.452 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-04-02T13:26:00.452 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-04-02T13:26:00.457 INFO:teuthology.orchestra.run.smithi033.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-04-02T13:26:00.474 INFO:teuthology.orchestra.run.smithi033.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-04-02T13:26:00.485 INFO:teuthology.orchestra.run.smithi050.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 492 kB/s | 29 kB 00:00 2024-04-02T13:26:00.491 INFO:teuthology.orchestra.run.smithi033.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-04-02T13:26:00.508 INFO:teuthology.orchestra.run.smithi033.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-04-02T13:26:00.510 INFO:teuthology.orchestra.run.smithi050.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 508 kB/s | 64 kB 00:00 2024-04-02T13:26:00.533 INFO:teuthology.orchestra.run.smithi033.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-04-02T13:26:00.558 INFO:teuthology.orchestra.run.smithi033.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-02T13:26:00.610 INFO:teuthology.orchestra.run.smithi050.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 9.7 MB/s | 1.2 MB 00:00 2024-04-02T13:26:00.738 INFO:teuthology.orchestra.run.smithi050.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 18 MB/s | 2.3 MB 00:00 2024-04-02T13:26:00.751 INFO:teuthology.orchestra.run.smithi033.stdout:(90/174): ceph-common-19.0.0-2609.g191d963a.el9 1.7 MB/s | 21 MB 00:12 2024-04-02T13:26:00.784 INFO:teuthology.orchestra.run.smithi033.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 4.6 MB/s | 1.1 MB 00:00 2024-04-02T13:26:00.794 INFO:teuthology.orchestra.run.smithi050.stdout:(169/174): librbd1-19.0.0-2609.g191d963a.el9.x8 2.2 MB/s | 3.1 MB 00:01 2024-04-02T13:26:00.801 INFO:teuthology.orchestra.run.smithi033.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 426 kB/s | 103 kB 00:00 2024-04-02T13:26:00.820 INFO:teuthology.orchestra.run.smithi050.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 5.4 MB/s | 465 kB 00:00 2024-04-02T13:26:00.826 INFO:teuthology.orchestra.run.smithi033.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.7 MB/s | 124 kB 00:00 2024-04-02T13:26:00.852 INFO:teuthology.orchestra.run.smithi033.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-04-02T13:26:00.886 INFO:teuthology.orchestra.run.smithi050.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 339 kB/s | 31 kB 00:00 2024-04-02T13:26:01.137 INFO:teuthology.orchestra.run.smithi050.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.4 MB/s | 1.1 MB 00:00 2024-04-02T13:26:01.154 INFO:teuthology.orchestra.run.smithi050.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 278 kB/s | 74 kB 00:00 2024-04-02T13:26:01.343 INFO:teuthology.orchestra.run.smithi033.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 346 kB/s | 187 kB 00:00 2024-04-02T13:26:01.369 INFO:teuthology.orchestra.run.smithi033.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 378 kB/s | 195 kB 00:00 2024-04-02T13:26:01.453 INFO:teuthology.orchestra.run.smithi033.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 257 kB/s | 21 kB 00:00 2024-04-02T13:26:01.469 INFO:teuthology.orchestra.run.smithi033.stdout:(98/174): python3-requests-2.25.1-8.el9.noarch. 993 kB/s | 125 kB 00:00 2024-04-02T13:26:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-04-02T13:26:01.528 INFO:teuthology.orchestra.run.smithi033.stdout:(100/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-02T13:26:01.570 INFO:teuthology.orchestra.run.smithi050.stdout:(174/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 2.0 MB/s | 2.2 MB 00:01 2024-04-02T13:26:01.575 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:26:01.575 INFO:teuthology.orchestra.run.smithi050.stdout:Total 11 MB/s | 226 MB 00:20 2024-04-02T13:26:01.604 INFO:teuthology.orchestra.run.smithi033.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.4 MB/s | 245 kB 00:00 2024-04-02T13:26:01.646 INFO:teuthology.orchestra.run.smithi033.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-04-02T13:26:01.671 INFO:teuthology.orchestra.run.smithi033.stdout:(103/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.7 MB/s | 248 kB 00:00 2024-04-02T13:26:01.755 INFO:teuthology.orchestra.run.smithi033.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 15 MB/s | 4.1 MB 00:00 2024-04-02T13:26:01.780 INFO:teuthology.orchestra.run.smithi033.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 166 kB/s | 22 kB 00:00 2024-04-02T13:26:01.839 INFO:teuthology.orchestra.run.smithi033.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.0 MB/s | 863 kB 00:00 2024-04-02T13:26:01.959 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-04-02T13:26:01.964 INFO:teuthology.orchestra.run.smithi033.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 156 kB/s | 19 kB 00:00 2024-04-02T13:26:02.056 INFO:teuthology.orchestra.run.smithi033.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-04-02T13:26:02.114 INFO:teuthology.orchestra.run.smithi033.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-04-02T13:26:02.139 INFO:teuthology.orchestra.run.smithi033.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 311 kB/s | 26 kB 00:00 2024-04-02T13:26:02.173 INFO:teuthology.orchestra.run.smithi033.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 831 kB/s | 49 kB 00:00 2024-04-02T13:26:02.198 INFO:teuthology.orchestra.run.smithi033.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-02T13:26:02.232 INFO:teuthology.orchestra.run.smithi033.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-02T13:26:02.256 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-04-02T13:26:02.256 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-04-02T13:26:02.324 INFO:teuthology.orchestra.run.smithi033.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-04-02T13:26:02.357 INFO:teuthology.orchestra.run.smithi033.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 838 kB 00:00 2024-04-02T13:26:02.383 INFO:teuthology.orchestra.run.smithi033.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-04-02T13:26:02.417 INFO:teuthology.orchestra.run.smithi033.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-04-02T13:26:02.442 INFO:teuthology.orchestra.run.smithi033.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-04-02T13:26:02.509 INFO:teuthology.orchestra.run.smithi033.stdout:(119/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-04-02T13:26:02.534 INFO:teuthology.orchestra.run.smithi033.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-02T13:26:02.559 INFO:teuthology.orchestra.run.smithi033.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-02T13:26:02.593 INFO:teuthology.orchestra.run.smithi033.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-02T13:26:02.610 INFO:teuthology.orchestra.run.smithi033.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 468 kB/s | 35 kB 00:00 2024-04-02T13:26:02.635 INFO:teuthology.orchestra.run.smithi033.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-04-02T13:26:02.677 INFO:teuthology.orchestra.run.smithi033.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-02T13:26:02.711 INFO:teuthology.orchestra.run.smithi033.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 141 kB/s | 11 kB 00:00 2024-04-02T13:26:02.736 INFO:teuthology.orchestra.run.smithi033.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 300 kB/s | 18 kB 00:00 2024-04-02T13:26:02.778 INFO:teuthology.orchestra.run.smithi033.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-02T13:26:02.803 INFO:teuthology.orchestra.run.smithi033.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-04-02T13:26:02.829 INFO:teuthology.orchestra.run.smithi033.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-02T13:26:02.854 INFO:teuthology.orchestra.run.smithi033.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-02T13:26:02.887 INFO:teuthology.orchestra.run.smithi033.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 794 kB/s | 46 kB 00:00 2024-04-02T13:26:02.913 INFO:teuthology.orchestra.run.smithi033.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-02T13:26:02.955 INFO:teuthology.orchestra.run.smithi033.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-04-02T13:26:02.980 INFO:teuthology.orchestra.run.smithi033.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 626 kB/s | 58 kB 00:00 2024-04-02T13:26:03.005 INFO:teuthology.orchestra.run.smithi033.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-02T13:26:03.031 INFO:teuthology.orchestra.run.smithi033.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-04-02T13:26:03.056 INFO:teuthology.orchestra.run.smithi033.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-02T13:26:03.081 INFO:teuthology.orchestra.run.smithi033.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-02T13:26:03.106 INFO:teuthology.orchestra.run.smithi033.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-04-02T13:26:03.132 INFO:teuthology.orchestra.run.smithi033.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-02T13:26:03.157 INFO:teuthology.orchestra.run.smithi033.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-04-02T13:26:03.190 INFO:teuthology.orchestra.run.smithi033.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 427 kB/s | 36 kB 00:00 2024-04-02T13:26:03.216 INFO:teuthology.orchestra.run.smithi033.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 609 kB/s | 51 kB 00:00 2024-04-02T13:26:03.241 INFO:teuthology.orchestra.run.smithi033.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-02T13:26:03.266 INFO:teuthology.orchestra.run.smithi033.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-02T13:26:03.292 INFO:teuthology.orchestra.run.smithi033.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-02T13:26:03.476 INFO:teuthology.orchestra.run.smithi033.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 85 kB/s | 20 kB 00:00 2024-04-02T13:26:03.501 INFO:teuthology.orchestra.run.smithi033.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 819 kB/s | 192 kB 00:00 2024-04-02T13:26:03.543 INFO:teuthology.orchestra.run.smithi033.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-04-02T13:26:03.685 INFO:teuthology.orchestra.run.smithi033.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 16 MB/s | 2.3 MB 00:00 2024-04-02T13:26:03.777 INFO:teuthology.orchestra.run.smithi033.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-04-02T13:26:03.810 INFO:teuthology.orchestra.run.smithi033.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-02T13:26:03.861 INFO:teuthology.orchestra.run.smithi033.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.8 MB/s | 247 kB 00:00 2024-04-02T13:26:03.903 INFO:teuthology.orchestra.run.smithi033.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-04-02T13:26:04.178 INFO:teuthology.orchestra.run.smithi033.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 259 kB/s | 71 kB 00:00 2024-04-02T13:26:04.278 INFO:teuthology.orchestra.run.smithi033.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 364 kB/s | 36 kB 00:00 2024-04-02T13:26:04.354 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-04-02T13:26:04.354 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-04-02T13:26:04.462 INFO:teuthology.orchestra.run.smithi033.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.0 MB/s | 192 kB 00:00 2024-04-02T13:26:04.579 INFO:teuthology.orchestra.run.smithi033.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.1 MB/s | 256 kB 00:00 2024-04-02T13:26:04.629 INFO:teuthology.orchestra.run.smithi033.stdout:(160/174): librbd1-19.0.0-2609.g191d963a.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-02T13:26:04.671 INFO:teuthology.orchestra.run.smithi033.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 251 kB/s | 23 kB 00:00 2024-04-02T13:26:04.780 INFO:teuthology.orchestra.run.smithi033.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-04-02T13:26:04.888 INFO:teuthology.orchestra.run.smithi033.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-04-02T13:26:04.981 INFO:teuthology.orchestra.run.smithi033.stdout:(164/174): libblkid-2.37.4-18.el9.x86_64.rpm 309 kB/s | 108 kB 00:00 2024-04-02T13:26:05.006 INFO:teuthology.orchestra.run.smithi033.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 545 kB/s | 64 kB 00:00 2024-04-02T13:26:05.056 INFO:teuthology.orchestra.run.smithi033.stdout:(166/174): librados2-19.0.0-2609.g191d963a.el9. 2.1 MB/s | 3.4 MB 00:01 2024-04-02T13:26:05.081 INFO:teuthology.orchestra.run.smithi033.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 286 kB/s | 29 kB 00:00 2024-04-02T13:26:05.207 INFO:teuthology.orchestra.run.smithi033.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 6.0 MB/s | 1.2 MB 00:00 2024-04-02T13:26:05.315 INFO:teuthology.orchestra.run.smithi033.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 4.2 MB/s | 465 kB 00:00 2024-04-02T13:26:05.432 INFO:teuthology.orchestra.run.smithi033.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 267 kB/s | 31 kB 00:00 2024-04-02T13:26:05.541 INFO:teuthology.orchestra.run.smithi033.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.0 MB/s | 2.3 MB 00:00 2024-04-02T13:26:05.574 INFO:teuthology.orchestra.run.smithi033.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2024-04-02T13:26:05.625 INFO:teuthology.orchestra.run.smithi033.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 890 kB/s | 74 kB 00:00 2024-04-02T13:26:05.866 INFO:teuthology.orchestra.run.smithi033.stdout:(174/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 2.7 MB/s | 2.2 MB 00:00 2024-04-02T13:26:05.870 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:26:05.870 INFO:teuthology.orchestra.run.smithi033.stdout:Total 13 MB/s | 226 MB 00:18 2024-04-02T13:26:06.248 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-02T13:26:06.546 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-02T13:26:06.546 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-02T13:26:07.320 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-04-02T13:26:07.408 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T13:26:07.443 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T13:26:07.522 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:26:07.565 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:26:07.839 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T13:26:07.914 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T13:26:07.952 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T13:26:07.985 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T13:26:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T13:26:08.367 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T13:26:08.428 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:26:08.550 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:26:08.579 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:26:08.612 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-02T13:26:08.612 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-02T13:26:08.680 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:26:08.709 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:26:08.747 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T13:26:08.933 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T13:26:08.990 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T13:26:09.010 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:26:09.098 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:26:09.119 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:26:09.185 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:26:09.224 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T13:26:09.246 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:26:09.262 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:26:09.463 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:26:09.495 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:26:09.534 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T13:26:09.722 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T13:26:09.845 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:26:09.896 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:26:09.992 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:26:10.051 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:26:10.253 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:26:10.355 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T13:26:10.442 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T13:26:10.473 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T13:26:10.500 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T13:26:10.543 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T13:26:10.618 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T13:26:10.672 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T13:26:10.718 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T13:26:10.783 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T13:26:10.817 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T13:26:10.860 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T13:26:10.893 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T13:26:10.921 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T13:26:10.991 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T13:26:11.026 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T13:26:11.069 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T13:26:11.098 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T13:26:11.169 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T13:26:11.212 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T13:26:11.287 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T13:26:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T13:26:11.454 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-04-02T13:26:11.474 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T13:26:11.559 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T13:26:11.832 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T13:26:11.890 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:26:11.902 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:26:11.966 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T13:26:11.991 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T13:26:12.023 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T13:26:12.057 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T13:26:12.078 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T13:26:12.081 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T13:26:12.128 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T13:26:12.170 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T13:26:12.214 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:26:12.224 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:26:12.255 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T13:26:12.295 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T13:26:12.333 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T13:26:12.363 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T13:26:12.478 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T13:26:12.532 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:26:12.633 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T13:26:12.657 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:26:12.683 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:26:12.776 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:26:12.805 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:26:12.814 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T13:26:12.842 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T13:26:12.890 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T13:26:12.935 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T13:26:13.031 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T13:26:13.068 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T13:26:13.086 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T13:26:13.112 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T13:26:13.114 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:26:13.176 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T13:26:13.203 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:26:13.232 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:26:13.249 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T13:26:13.282 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T13:26:13.288 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:26:13.318 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T13:26:13.336 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T13:26:13.358 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:26:13.366 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T13:26:13.374 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:26:13.401 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T13:26:13.444 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T13:26:13.486 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T13:26:13.526 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:26:13.557 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:26:13.600 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T13:26:13.622 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T13:26:13.675 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T13:26:13.705 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T13:26:13.747 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T13:26:13.765 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T13:26:13.841 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T13:26:13.887 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:26:13.888 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T13:26:13.928 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T13:26:13.954 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T13:26:13.955 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:26:13.979 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T13:26:14.056 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:26:14.064 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T13:26:14.098 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:26:14.187 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T13:26:14.288 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T13:26:14.300 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:26:14.337 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T13:26:14.386 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T13:26:14.416 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T13:26:14.442 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T13:26:14.511 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T13:26:14.545 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T13:26:14.572 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T13:26:14.616 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T13:26:14.627 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T13:26:14.683 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T13:26:14.692 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T13:26:14.745 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T13:26:14.783 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T13:26:14.823 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T13:26:14.864 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T13:26:14.873 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T13:26:14.896 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T13:26:14.907 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T13:26:14.927 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:26:14.951 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T13:26:14.954 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T13:26:14.984 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T13:26:14.989 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T13:26:15.011 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T13:26:15.028 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T13:26:15.060 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T13:26:15.082 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T13:26:15.090 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T13:26:15.116 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T13:26:15.160 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T13:26:15.167 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T13:26:15.187 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T13:26:15.208 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T13:26:15.242 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T13:26:15.259 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T13:26:15.283 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T13:26:15.311 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T13:26:15.318 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T13:26:15.387 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T13:26:15.407 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T13:26:15.489 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T13:26:15.524 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T13:26:15.540 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T13:26:15.572 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T13:26:15.580 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T13:26:15.619 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T13:26:15.716 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T13:26:15.733 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-02T13:26:15.774 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T13:26:15.855 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T13:26:16.073 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T13:26:16.132 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T13:26:16.194 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:26:16.206 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:26:16.219 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T13:26:16.274 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T13:26:16.304 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T13:26:16.340 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T13:26:16.361 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:26:16.371 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:26:16.378 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T13:26:16.402 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T13:26:16.410 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T13:26:16.458 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T13:26:16.459 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T13:26:16.497 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T13:26:16.501 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T13:26:16.519 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T13:26:16.545 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T13:26:16.579 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T13:26:16.628 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T13:26:16.717 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T13:26:16.746 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T13:26:16.800 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T13:26:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T13:26:16.853 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:26:16.923 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T13:26:16.950 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T13:26:16.977 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:26:16.977 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T13:26:17.004 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:26:17.046 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T13:26:17.057 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T13:26:17.090 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T13:26:17.096 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:26:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:26:17.176 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T13:26:17.234 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T13:26:17.284 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T13:26:17.341 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T13:26:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T13:26:17.413 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T13:26:17.432 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T13:26:17.451 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T13:26:17.459 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:26:17.505 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T13:26:17.553 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T13:26:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:26:17.588 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T13:26:17.602 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:26:17.631 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T13:26:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:26:17.673 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T13:26:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T13:26:17.707 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T13:26:17.729 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:26:17.744 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:26:17.812 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T13:26:17.834 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T13:26:17.862 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T13:26:17.883 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T13:26:17.926 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T13:26:17.933 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:26:17.965 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:26:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T13:26:18.022 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T13:26:18.058 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T13:26:18.099 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T13:26:18.124 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T13:26:18.157 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T13:26:18.236 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T13:26:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T13:26:18.389 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T13:26:18.496 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T13:26:18.533 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:26:18.549 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T13:26:18.593 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:26:18.607 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T13:26:18.672 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T13:26:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:26:18.778 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:26:18.862 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T13:26:18.912 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T13:26:18.987 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:26:19.052 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T13:26:19.055 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T13:26:19.093 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T13:26:19.095 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T13:26:19.125 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T13:26:19.144 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T13:26:19.155 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:26:19.175 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T13:26:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T13:26:19.192 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T13:26:19.206 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T13:26:19.224 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T13:26:19.226 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T13:26:19.234 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T13:26:19.251 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T13:26:19.266 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T13:26:19.296 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T13:26:19.305 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T13:26:19.335 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T13:26:19.372 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T13:26:19.415 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T13:26:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T13:26:19.454 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T13:26:19.462 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T13:26:19.487 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T13:26:19.529 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T13:26:19.537 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T13:26:19.552 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T13:26:19.563 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T13:26:19.580 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:26:19.586 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T13:26:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T13:26:19.656 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T13:26:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T13:26:19.671 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:26:19.701 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T13:26:19.708 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T13:26:19.738 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T13:26:19.746 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T13:26:19.788 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T13:26:19.803 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T13:26:19.820 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T13:26:19.834 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T13:26:19.863 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T13:26:19.882 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T13:26:19.891 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T13:26:19.965 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T13:26:19.991 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T13:26:20.006 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T13:26:20.045 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T13:26:20.091 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T13:26:20.195 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T13:26:20.225 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:26:20.229 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T13:26:20.294 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T13:26:20.295 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:26:20.326 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T13:26:20.811 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T13:26:20.851 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T13:26:20.857 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T13:26:20.891 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T13:26:20.931 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T13:26:20.950 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T13:26:21.026 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T13:26:21.040 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T13:26:21.073 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:26:21.082 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:26:21.085 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T13:26:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T13:26:21.179 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T13:26:21.217 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T13:26:21.270 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T13:26:21.272 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T13:26:21.299 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T13:26:21.379 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T13:26:21.584 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T13:26:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T13:26:21.857 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T13:26:21.902 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T13:26:22.046 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T13:26:22.057 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T13:26:22.104 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T13:26:22.161 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T13:26:22.176 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T13:26:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T13:26:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T13:26:22.308 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T13:26:22.356 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T13:26:22.391 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T13:26:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T13:26:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T13:26:22.564 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:26:22.598 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:26:22.626 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T13:26:22.682 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T13:26:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T13:26:22.754 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T13:26:22.852 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T13:26:22.886 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T13:26:22.928 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T13:26:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T13:26:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T13:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T13:26:23.133 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:26:23.167 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:26:23.216 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T13:26:23.241 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:26:23.241 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T13:26:23.241 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T13:26:23.241 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:26:23.285 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:26:23.322 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T13:26:23.368 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T13:26:23.427 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T13:26:23.473 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T13:26:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T13:26:23.550 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T13:26:23.581 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T13:26:23.611 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T13:26:23.639 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T13:26:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T13:26:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T13:26:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T13:26:23.936 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T13:26:23.969 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T13:26:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T13:26:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:26:24.027 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T13:26:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T13:26:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T13:26:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T13:26:24.162 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T13:26:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T13:26:24.250 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:26:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T13:26:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T13:26:24.336 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:26:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T13:26:24.365 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T13:26:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T13:26:24.403 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T13:26:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T13:26:24.583 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T13:26:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T13:26:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T13:26:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T13:26:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T13:26:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T13:26:24.879 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T13:26:24.905 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:26:24.960 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:26:24.982 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T13:26:25.646 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T13:26:25.701 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T13:26:25.735 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T13:26:25.784 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T13:26:25.876 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T13:26:25.911 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T13:26:26.088 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T13:26:26.108 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T13:26:26.189 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T13:26:26.883 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T13:26:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T13:26:27.363 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:26:27.397 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:26:27.976 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:26:28.025 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:26:28.116 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:26:28.116 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T13:26:28.116 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T13:26:28.116 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:26:28.151 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:26:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T13:26:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T13:26:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T13:26:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T13:26:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T13:26:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T13:26:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:26:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:26:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T13:26:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T13:26:29.470 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T13:26:29.497 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:26:29.569 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:26:29.599 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T13:26:31.979 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:26:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:26:32.594 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:26:32.632 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:26:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:26:32.725 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T13:26:32.725 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T13:26:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:26:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:26:34.925 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:26:34.925 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /sys 2024-04-02T13:26:34.925 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /proc 2024-04-02T13:26:34.926 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /mnt 2024-04-02T13:26:34.926 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /var/tmp 2024-04-02T13:26:34.926 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /home 2024-04-02T13:26:34.926 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /root 2024-04-02T13:26:34.926 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /tmp 2024-04-02T13:26:34.926 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:26:35.089 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:26:35.105 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:26:35.105 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:35.105 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T13:26:35.105 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:26:36.584 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:26:36.610 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:26:36.610 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:36.610 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T13:26:36.610 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:26:36.610 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:26:36.610 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:26:36.834 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:26:36.857 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:26:36.857 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:36.857 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T13:26:36.857 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:26:36.857 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:26:36.857 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:26:37.302 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:26:37.325 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:26:37.325 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:37.325 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T13:26:37.325 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:26:37.325 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:26:37.325 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:26:37.412 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T13:26:37.781 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:26:37.810 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:26:37.843 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T13:26:37.903 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T13:26:37.933 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:26:38.974 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:26:39.011 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:26:39.978 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:26:40.011 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:26:40.111 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:26:40.274 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T13:26:40.304 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:26:40.327 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:26:40.327 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:40.328 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T13:26:40.328 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:26:40.328 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:26:40.328 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:26:40.371 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:26:40.383 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:26:40.434 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T13:26:40.452 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:26:40.452 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /sys 2024-04-02T13:26:40.452 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /proc 2024-04-02T13:26:40.452 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /mnt 2024-04-02T13:26:40.452 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /var/tmp 2024-04-02T13:26:40.452 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /home 2024-04-02T13:26:40.453 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /root 2024-04-02T13:26:40.453 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /tmp 2024-04-02T13:26:40.453 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:26:40.631 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:26:40.649 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:26:40.649 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:40.649 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T13:26:40.649 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:26:41.309 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T13:26:41.340 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:26:41.363 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:26:41.363 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:41.364 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T13:26:41.364 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:26:41.364 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:26:41.364 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:26:41.407 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:26:41.429 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:26:41.429 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:41.429 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T13:26:41.430 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:26:41.475 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:26:41.497 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:26:41.497 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:41.497 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T13:26:41.498 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:26:41.498 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:26:41.498 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:26:41.817 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:26:41.840 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:26:41.840 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:41.840 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T13:26:41.840 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:26:41.840 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:26:41.840 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:26:42.184 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:26:42.209 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:26:42.210 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:42.210 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T13:26:42.210 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:26:42.210 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:26:42.210 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:26:42.443 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:26:42.465 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:26:42.466 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:42.466 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T13:26:42.466 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:26:42.466 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:26:42.466 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:26:42.927 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:26:42.949 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:26:42.949 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:42.949 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T13:26:42.949 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:26:42.949 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:26:42.950 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:26:43.045 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T13:26:43.380 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:26:43.411 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:26:43.443 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T13:26:43.507 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T13:26:43.542 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:26:44.637 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:26:44.677 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:26:45.109 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:26:45.109 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-04-02T13:26:45.109 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-04-02T13:26:45.109 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-04-02T13:26:45.109 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-04-02T13:26:45.109 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-04-02T13:26:45.109 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /root 2024-04-02T13:26:45.109 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-04-02T13:26:45.109 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:26:45.273 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:26:45.289 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:26:45.289 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:45.289 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T13:26:45.289 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:26:45.691 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:26:45.719 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:26:45.823 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:26:46.006 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T13:26:46.038 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:26:46.061 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:26:46.061 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:46.061 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T13:26:46.061 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:26:46.061 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:26:46.061 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:26:46.105 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:26:46.116 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:26:46.166 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T13:26:46.313 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:26:46.362 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T13:26:46.393 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T13:26:46.420 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T13:26:46.602 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T13:26:46.711 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T13:26:46.759 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T13:26:46.814 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T13:26:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:26:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:26:46.852 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:46.852 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T13:26:46.852 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:26:46.852 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:26:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:26:46.860 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T13:26:46.894 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T13:26:46.927 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T13:26:46.967 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T13:26:46.997 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T13:26:47.054 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T13:26:47.085 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:26:47.086 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T13:26:47.109 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:26:47.109 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:47.109 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T13:26:47.109 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:26:47.110 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:26:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:26:47.124 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:26:47.127 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T13:26:47.146 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:26:47.146 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:47.146 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T13:26:47.146 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:26:47.146 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:26:47.146 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:26:47.162 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T13:26:47.191 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:26:47.213 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:26:47.213 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:47.213 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T13:26:47.213 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:26:47.258 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:26:47.281 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:26:47.281 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:47.281 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T13:26:47.281 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:26:47.281 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:26:47.281 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:26:47.570 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:26:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:26:47.592 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:47.592 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T13:26:47.592 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:26:47.592 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:26:47.592 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:26:47.601 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:26:47.623 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:26:47.623 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:47.623 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T13:26:47.623 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:26:47.623 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:26:47.623 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:26:47.688 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T13:26:47.724 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:26:47.760 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:26:47.780 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:26:48.057 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:26:48.086 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:26:48.119 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T13:26:48.182 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T13:26:48.209 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:26:48.595 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T13:26:48.672 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T13:26:48.713 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T13:26:48.804 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T13:26:48.833 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T13:26:48.859 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T13:26:48.884 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T13:26:48.910 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T13:26:48.910 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:26:48.998 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:26:49.029 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T13:26:49.029 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:26:49.113 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:26:49.139 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T13:26:49.172 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T13:26:49.197 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T13:26:49.222 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T13:26:49.256 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T13:26:49.303 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T13:26:49.309 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:26:49.344 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:26:49.345 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T13:26:49.371 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T13:26:49.396 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T13:26:49.421 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:26:49.446 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T13:26:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:26:49.453 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:26:49.641 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:26:49.641 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:26:49.724 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:26:49.755 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:26:49.782 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T13:26:49.808 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T13:26:49.833 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T13:26:49.915 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:26:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:26:50.395 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:26:50.501 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:26:50.665 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T13:26:50.697 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:26:50.719 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:26:50.719 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:50.719 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T13:26:50.719 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:26:50.720 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:26:50.720 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:26:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:26:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:26:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T13:26:51.748 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T13:26:51.774 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:26:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:26:51.796 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:51.796 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T13:26:51.796 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:26:51.796 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:26:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:26:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:26:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:26:51.863 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:51.863 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T13:26:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:26:51.909 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:26:51.931 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:26:51.931 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:51.931 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T13:26:51.931 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:26:51.931 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:26:51.931 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:26:52.251 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:26:52.273 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:26:52.273 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:26:52.274 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T13:26:52.274 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:26:52.274 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:26:52.274 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:26:52.339 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:26:52.389 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T13:26:52.421 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T13:26:52.448 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T13:26:52.647 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T13:26:52.760 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T13:26:52.804 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T13:26:52.856 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T13:26:52.913 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T13:26:52.948 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T13:26:52.980 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T13:26:53.012 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T13:26:53.042 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T13:26:53.111 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T13:26:53.173 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T13:26:53.207 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T13:26:53.843 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:26:53.880 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:26:53.900 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:26:54.771 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T13:26:54.859 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T13:26:54.901 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T13:26:54.985 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T13:26:55.012 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T13:26:55.037 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T13:26:55.063 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T13:26:55.089 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T13:26:55.090 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:26:55.169 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:26:55.198 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T13:26:55.198 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:26:55.283 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:26:55.309 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T13:26:55.359 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T13:26:55.388 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T13:26:55.402 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:26:55.402 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T13:26:55.402 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T13:26:55.402 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T13:26:55.402 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T13:26:55.402 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T13:26:55.402 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T13:26:55.403 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T13:26:55.404 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T13:26:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T13:26:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T13:26:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T13:26:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T13:26:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T13:26:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T13:26:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T13:26:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T13:26:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T13:26:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T13:26:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T13:26:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T13:26:55.406 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T13:26:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T13:26:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T13:26:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:26:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T13:26:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T13:26:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T13:26:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T13:26:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T13:26:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T13:26:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T13:26:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T13:26:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T13:26:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T13:26:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T13:26:55.409 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T13:26:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T13:26:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T13:26:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T13:26:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T13:26:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T13:26:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T13:26:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T13:26:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T13:26:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T13:26:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T13:26:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T13:26:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T13:26:55.410 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T13:26:55.411 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T13:26:55.412 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T13:26:55.413 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T13:26:55.414 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T13:26:55.415 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T13:26:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T13:26:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T13:26:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T13:26:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T13:26:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:26:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T13:26:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T13:26:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T13:26:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:26:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T13:26:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T13:26:55.416 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T13:26:55.417 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T13:26:55.444 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T13:26:55.482 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T13:26:55.514 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T13:26:55.540 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T13:26:55.564 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T13:26:55.590 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:26:55.615 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T13:26:55.622 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:26:55.622 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:26:55.802 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:26:55.802 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:26:55.884 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:26:55.917 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:26:55.943 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T13:26:55.968 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T13:26:55.994 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T13:26:56.071 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:26:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:26:57.056 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T13:26:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T13:26:57.121 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T13:26:57.326 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T13:26:57.442 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T13:26:57.487 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T13:26:57.537 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T13:26:57.586 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T13:26:57.621 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T13:26:57.653 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T13:26:57.694 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T13:26:57.723 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T13:26:57.797 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T13:26:57.860 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T13:26:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T13:26:58.439 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:26:58.480 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:26:58.499 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:26:58.594 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T13:26:58.594 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:26:58.594 INFO:teuthology.orchestra.run.smithi145.stdout:Upgraded: 2024-04-02T13:26:58.594 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T13:26:58.594 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T13:26:58.594 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T13:26:58.595 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.596 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:26:58.597 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T13:26:58.598 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: jq-1.6-16.el9.x86_64 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T13:26:58.599 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T13:26:58.600 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.601 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.602 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.602 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T13:26:58.602 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T13:26:58.602 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T13:26:58.602 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T13:26:58.602 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T13:26:58.602 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T13:26:58.603 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T13:26:58.603 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T13:26:58.603 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T13:26:58.603 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T13:26:58.603 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T13:26:58.603 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T13:26:58.603 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T13:26:58.603 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T13:26:58.603 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T13:26:58.603 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T13:26:58.604 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T13:26:58.605 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T13:26:58.606 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T13:26:58.607 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T13:26:58.607 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T13:26:58.607 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.607 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.607 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:26:58.607 INFO:teuthology.orchestra.run.smithi145.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T13:26:58.607 INFO:teuthology.orchestra.run.smithi145.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T13:26:58.607 INFO:teuthology.orchestra.run.smithi145.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T13:26:58.607 INFO:teuthology.orchestra.run.smithi145.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T13:26:58.607 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T13:26:58.607 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T13:26:58.608 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T13:26:58.608 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:26:58.608 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-04-02T13:26:59.064 DEBUG:teuthology.parallel:result is None 2024-04-02T13:26:59.398 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T13:26:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T13:26:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T13:26:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T13:26:59.645 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T13:26:59.670 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T13:26:59.695 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T13:26:59.722 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T13:26:59.722 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:26:59.827 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:26:59.856 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T13:26:59.856 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:26:59.941 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:26:59.967 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T13:27:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T13:27:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T13:27:00.050 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T13:27:00.084 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T13:27:00.123 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T13:27:00.155 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T13:27:00.180 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T13:27:00.206 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T13:27:00.231 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:27:00.256 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T13:27:00.263 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:27:00.263 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:27:00.442 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:27:00.442 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:27:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:27:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:27:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T13:27:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T13:27:00.634 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T13:27:00.712 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:27:01.392 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:27:01.393 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T13:27:01.393 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T13:27:01.393 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T13:27:01.393 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T13:27:01.393 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T13:27:01.393 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T13:27:01.393 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T13:27:01.393 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T13:27:01.393 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T13:27:01.393 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:27:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:27:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:27:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T13:27:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T13:27:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T13:27:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:27:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T13:27:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T13:27:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T13:27:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T13:27:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T13:27:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T13:27:01.395 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T13:27:01.395 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T13:27:01.395 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T13:27:01.395 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T13:27:01.395 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T13:27:01.395 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T13:27:01.413 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T13:27:01.413 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T13:27:01.413 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T13:27:01.413 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T13:27:01.413 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T13:27:01.413 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T13:27:01.414 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T13:27:01.414 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T13:27:01.414 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T13:27:01.414 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T13:27:01.414 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T13:27:01.414 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T13:27:01.414 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T13:27:01.414 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T13:27:01.414 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T13:27:01.414 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T13:27:01.415 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T13:27:01.415 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T13:27:01.415 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T13:27:01.415 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T13:27:01.415 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T13:27:01.415 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T13:27:01.415 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T13:27:01.415 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T13:27:01.415 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T13:27:01.415 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T13:27:01.415 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T13:27:01.415 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T13:27:01.416 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T13:27:01.416 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T13:27:01.416 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T13:27:01.416 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T13:27:01.416 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T13:27:01.416 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T13:27:01.416 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T13:27:01.416 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T13:27:01.416 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T13:27:01.416 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T13:27:01.416 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T13:27:01.416 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T13:27:01.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T13:27:01.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T13:27:01.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T13:27:01.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T13:27:01.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T13:27:01.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T13:27:01.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T13:27:01.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T13:27:01.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T13:27:01.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T13:27:01.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T13:27:01.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T13:27:01.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T13:27:01.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T13:27:01.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T13:27:01.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T13:27:01.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T13:27:01.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T13:27:01.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T13:27:01.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T13:27:01.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:27:01.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T13:27:01.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T13:27:01.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T13:27:01.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T13:27:01.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T13:27:01.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T13:27:01.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T13:27:01.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T13:27:01.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T13:27:01.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T13:27:01.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T13:27:01.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T13:27:01.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T13:27:01.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T13:27:01.420 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T13:27:01.420 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T13:27:01.420 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T13:27:01.420 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T13:27:01.420 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T13:27:01.420 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T13:27:01.420 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T13:27:01.420 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T13:27:01.420 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T13:27:01.420 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T13:27:01.420 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T13:27:01.420 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T13:27:01.420 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T13:27:01.421 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T13:27:01.421 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T13:27:01.421 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T13:27:01.421 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T13:27:01.421 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T13:27:01.421 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T13:27:01.421 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T13:27:01.421 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T13:27:01.421 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T13:27:01.421 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T13:27:01.421 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T13:27:01.421 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T13:27:01.422 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T13:27:01.422 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T13:27:01.422 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T13:27:01.422 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T13:27:01.422 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T13:27:01.422 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T13:27:01.422 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T13:27:01.422 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T13:27:01.422 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T13:27:01.422 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T13:27:01.422 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T13:27:01.422 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T13:27:01.423 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T13:27:01.423 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T13:27:01.423 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T13:27:01.423 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T13:27:01.423 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T13:27:01.423 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T13:27:01.423 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T13:27:01.423 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T13:27:01.423 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T13:27:01.423 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T13:27:01.423 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:27:01.423 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T13:27:01.424 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:27:01.424 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T13:27:01.424 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T13:27:01.424 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T13:27:01.424 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T13:27:01.424 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T13:27:01.424 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T13:27:01.424 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T13:27:01.424 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T13:27:01.424 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T13:27:01.424 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T13:27:01.424 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T13:27:01.425 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T13:27:01.425 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T13:27:01.425 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T13:27:01.425 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T13:27:01.425 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T13:27:01.425 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T13:27:01.425 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T13:27:01.425 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T13:27:01.425 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T13:27:01.425 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T13:27:01.425 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T13:27:01.425 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T13:27:01.426 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T13:27:01.426 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T13:27:01.426 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T13:27:01.426 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T13:27:01.426 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T13:27:01.426 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T13:27:01.426 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T13:27:01.426 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T13:27:01.426 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T13:27:01.426 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T13:27:01.426 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T13:27:01.427 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T13:27:01.427 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T13:27:01.427 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:27:01.427 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T13:27:01.427 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T13:27:01.427 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T13:27:01.427 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:27:01.427 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T13:27:01.427 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T13:27:01.427 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T13:27:04.317 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T13:27:04.317 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:27:04.317 INFO:teuthology.orchestra.run.smithi050.stdout:Upgraded: 2024-04-02T13:27:04.317 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T13:27:04.317 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T13:27:04.317 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T13:27:04.317 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T13:27:04.317 INFO:teuthology.orchestra.run.smithi050.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T13:27:04.318 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T13:27:04.318 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T13:27:04.318 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T13:27:04.318 INFO:teuthology.orchestra.run.smithi050.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T13:27:04.318 INFO:teuthology.orchestra.run.smithi050.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T13:27:04.318 INFO:teuthology.orchestra.run.smithi050.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T13:27:04.318 INFO:teuthology.orchestra.run.smithi050.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T13:27:04.318 INFO:teuthology.orchestra.run.smithi050.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T13:27:04.318 INFO:teuthology.orchestra.run.smithi050.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.318 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.318 INFO:teuthology.orchestra.run.smithi050.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T13:27:04.318 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T13:27:04.318 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.319 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.320 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: jq-1.6-16.el9.x86_64 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T13:27:04.321 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T13:27:04.322 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T13:27:04.323 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T13:27:04.323 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T13:27:04.323 INFO:teuthology.orchestra.run.smithi050.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T13:27:04.323 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T13:27:04.323 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T13:27:04.323 INFO:teuthology.orchestra.run.smithi050.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T13:27:04.323 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T13:27:04.323 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T13:27:04.323 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T13:27:04.323 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T13:27:04.323 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T13:27:04.323 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T13:27:04.323 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T13:27:04.324 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T13:27:04.324 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T13:27:04.324 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T13:27:04.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T13:27:04.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T13:27:04.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T13:27:04.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T13:27:04.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T13:27:04.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T13:27:04.324 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T13:27:04.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T13:27:04.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T13:27:04.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T13:27:04.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T13:27:04.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T13:27:04.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T13:27:04.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T13:27:04.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T13:27:04.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T13:27:04.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T13:27:04.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T13:27:04.325 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T13:27:04.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T13:27:04.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T13:27:04.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T13:27:04.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T13:27:04.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T13:27:04.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T13:27:04.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T13:27:04.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T13:27:04.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T13:27:04.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T13:27:04.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T13:27:04.326 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T13:27:04.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T13:27:04.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T13:27:04.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T13:27:04.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T13:27:04.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T13:27:04.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T13:27:04.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T13:27:04.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T13:27:04.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T13:27:04.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T13:27:04.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T13:27:04.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T13:27:04.327 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T13:27:04.328 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T13:27:04.329 INFO:teuthology.orchestra.run.smithi050.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T13:27:04.330 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T13:27:04.330 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T13:27:04.330 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T13:27:04.330 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-04-02T13:27:04.330 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-04-02T13:27:04.776 DEBUG:teuthology.parallel:result is None 2024-04-02T13:27:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:27:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T13:27:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T13:27:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T13:27:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T13:27:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T13:27:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T13:27:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T13:27:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T13:27:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T13:27:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T13:27:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T13:27:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T13:27:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T13:27:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T13:27:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T13:27:05.849 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T13:27:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T13:27:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T13:27:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T13:27:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T13:27:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T13:27:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T13:27:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T13:27:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T13:27:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T13:27:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T13:27:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T13:27:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T13:27:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T13:27:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T13:27:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T13:27:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T13:27:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T13:27:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T13:27:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T13:27:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T13:27:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T13:27:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T13:27:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T13:27:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T13:27:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T13:27:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T13:27:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T13:27:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T13:27:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T13:27:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T13:27:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T13:27:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T13:27:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T13:27:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T13:27:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T13:27:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T13:27:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T13:27:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T13:27:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T13:27:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T13:27:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T13:27:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T13:27:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T13:27:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T13:27:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T13:27:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T13:27:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:27:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T13:27:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T13:27:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T13:27:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T13:27:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T13:27:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T13:27:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T13:27:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T13:27:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T13:27:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T13:27:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T13:27:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T13:27:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T13:27:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T13:27:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T13:27:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T13:27:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T13:27:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T13:27:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T13:27:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T13:27:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T13:27:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T13:27:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T13:27:09.003 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T13:27:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:27:09.004 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-04-02T13:27:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T13:27:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T13:27:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T13:27:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T13:27:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T13:27:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T13:27:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T13:27:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T13:27:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T13:27:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T13:27:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T13:27:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T13:27:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T13:27:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T13:27:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T13:27:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T13:27:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T13:27:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T13:27:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T13:27:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T13:27:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T13:27:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T13:27:09.006 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-02T13:27:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T13:27:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T13:27:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T13:27:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T13:27:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:27:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T13:27:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T13:27:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T13:27:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T13:27:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T13:27:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T13:27:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T13:27:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T13:27:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: jq-1.6-16.el9.x86_64 2024-04-02T13:27:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T13:27:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T13:27:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T13:27:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T13:27:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T13:27:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T13:27:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T13:27:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T13:27:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T13:27:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T13:27:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T13:27:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T13:27:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T13:27:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T13:27:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T13:27:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T13:27:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T13:27:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T13:27:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T13:27:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T13:27:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T13:27:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T13:27:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T13:27:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T13:27:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T13:27:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T13:27:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T13:27:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T13:27:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T13:27:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T13:27:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T13:27:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T13:27:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T13:27:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T13:27:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T13:27:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T13:27:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T13:27:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T13:27:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T13:27:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T13:27:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T13:27:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T13:27:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T13:27:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T13:27:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T13:27:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T13:27:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T13:27:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T13:27:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T13:27:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T13:27:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T13:27:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T13:27:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T13:27:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T13:27:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T13:27:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T13:27:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T13:27:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T13:27:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T13:27:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T13:27:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T13:27:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T13:27:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T13:27:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T13:27:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T13:27:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T13:27:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T13:27:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T13:27:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T13:27:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T13:27:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T13:27:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T13:27:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T13:27:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T13:27:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T13:27:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T13:27:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T13:27:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T13:27:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T13:27:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T13:27:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T13:27:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T13:27:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T13:27:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T13:27:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T13:27:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T13:27:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T13:27:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T13:27:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T13:27:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T13:27:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T13:27:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:27:09.019 INFO:teuthology.orchestra.run.smithi033.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T13:27:09.019 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T13:27:09.019 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T13:27:09.019 INFO:teuthology.orchestra.run.smithi033.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T13:27:09.019 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T13:27:09.019 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T13:27:09.020 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T13:27:09.020 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:27:09.020 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-02T13:27:09.440 DEBUG:teuthology.parallel:result is None 2024-04-02T13:27:09.440 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:27:09.584 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T13:27:09.605 INFO:teuthology.orchestra.run.smithi033.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T13:27:09.605 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T13:27:09.605 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T13:27:09.608 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:27:09.750 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T13:27:09.775 INFO:teuthology.orchestra.run.smithi050.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T13:27:09.775 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T13:27:09.775 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T13:27:09.777 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:27:09.999 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T13:27:10.025 INFO:teuthology.orchestra.run.smithi145.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T13:27:10.026 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T13:27:10.026 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T13:27:10.028 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-02T13:27:10.028 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:27:10.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T13:27:10.058 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T13:27:10.058 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T13:27:10.094 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-02T13:27:10.094 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T13:27:10.129 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-02T13:27:10.129 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:27:10.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T13:27:10.155 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T13:27:10.222 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T13:27:10.222 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T13:27:10.251 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T13:27:10.320 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-02T13:27:10.320 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T13:27:10.351 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T13:27:10.421 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-02T13:27:10.421 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:27:10.421 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T13:27:10.447 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T13:27:10.515 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T13:27:10.515 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T13:27:10.543 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T13:27:10.612 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-02T13:27:10.612 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T13:27:10.640 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T13:27:10.707 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-02T13:27:10.707 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:27:10.707 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T13:27:10.734 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T13:27:10.801 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T13:27:10.802 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T13:27:10.831 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T13:27:10.900 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-02T13:27:10.900 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T13:27:10.928 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T13:27:10.995 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-02T13:27:11.103 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client dirsize rbytes': False, 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-02T13:27:11.104 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:27:11.104 INFO:tasks.cephadm:Cluster fsid is b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:27:11.104 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-02T13:27:11.105 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.33', 'mon.b': '172.21.15.50', 'mon.c': '172.21.15.145'} 2024-04-02T13:27:11.105 INFO:tasks.cephadm:First mon is mon.a on smithi033 2024-04-02T13:27:11.105 INFO:tasks.cephadm:First mgr is x 2024-04-02T13:27:11.105 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-02T13:27:11.105 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname $(hostname -s) 2024-04-02T13:27:11.133 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname $(hostname -s) 2024-04-02T13:27:11.165 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname $(hostname -s) 2024-04-02T13:27:11.193 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-02T13:27:11.193 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:27:11.337 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78154/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2609-g191d963a', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2609.g191d963a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-02 02:04:04.615852', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240402.004512', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-02T13:27:11.582 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-batrick-testing-20240402.004512, sha1 191d963ac9de7322da8aff20b567b3fa458e2ef6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:27:11.584 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm 2024-04-02T13:27:11.584 DEBUG:teuthology.orchestra.run.smithi033:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T13:27:12.054 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 13:27 /home/ubuntu/cephtest/cephadm 2024-04-02T13:27:12.054 DEBUG:teuthology.orchestra.run.smithi050:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T13:27:12.536 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 13:27 /home/ubuntu/cephtest/cephadm 2024-04-02T13:27:12.537 DEBUG:teuthology.orchestra.run.smithi145:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T13:27:13.340 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 13:27 /home/ubuntu/cephtest/cephadm 2024-04-02T13:27:13.340 DEBUG:teuthology.orchestra.run.smithi033:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T13:27:13.359 DEBUG:teuthology.orchestra.run.smithi050:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T13:27:13.377 DEBUG:teuthology.orchestra.run.smithi145:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T13:27:13.411 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 on all hosts... 2024-04-02T13:27:13.412 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T13:27:13.414 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T13:27:13.420 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T13:27:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:27:14.441 INFO:teuthology.orchestra.run.smithi050.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:27:14.574 INFO:teuthology.orchestra.run.smithi145.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:27:46.629 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:27:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T13:27:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T13:27:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: "repo_digests": [ 2024-04-02T13:27:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T13:27:46.630 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-02T13:27:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:28:10.822 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-04-02T13:28:10.822 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T13:28:10.822 INFO:teuthology.orchestra.run.smithi050.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T13:28:10.822 INFO:teuthology.orchestra.run.smithi050.stdout: "repo_digests": [ 2024-04-02T13:28:10.822 INFO:teuthology.orchestra.run.smithi050.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T13:28:10.822 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-04-02T13:28:10.822 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-04-02T13:28:12.749 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-04-02T13:28:12.749 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T13:28:12.749 INFO:teuthology.orchestra.run.smithi145.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T13:28:12.749 INFO:teuthology.orchestra.run.smithi145.stdout: "repo_digests": [ 2024-04-02T13:28:12.749 INFO:teuthology.orchestra.run.smithi145.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T13:28:12.749 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-04-02T13:28:12.749 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-04-02T13:28:12.774 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph 2024-04-02T13:28:12.809 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph 2024-04-02T13:28:12.844 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /etc/ceph 2024-04-02T13:28:12.873 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /etc/ceph 2024-04-02T13:28:12.903 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /etc/ceph 2024-04-02T13:28:12.933 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 777 /etc/ceph 2024-04-02T13:28:12.959 INFO:tasks.cephadm:Writing seed config... 2024-04-02T13:28:12.961 INFO:tasks.cephadm: override: [client] client dirsize rbytes = False 2024-04-02T13:28:12.961 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-02T13:28:12.961 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-02T13:28:12.961 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-02T13:28:12.961 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-02T13:28:12.962 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-02T13:28:12.962 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-02T13:28:12.962 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-02T13:28:12.962 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-02T13:28:12.962 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-02T13:28:12.962 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-02T13:28:12.962 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-02T13:28:12.962 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-02T13:28:12.962 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-02T13:28:12.962 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-02T13:28:12.962 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-02T13:28:12.962 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-02T13:28:12.963 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-02T13:28:12.964 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-02T13:28:12.965 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-02T13:28:12.965 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-02T13:28:12.965 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:28:12.965 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-02T13:28:12.983 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b5fd982e-f0f4-11ee-b647-cb9ed24678a4 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client dirsize rbytes = False client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-02T13:28:12.983 DEBUG:teuthology.orchestra.run.smithi033:mon.a> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.a.service 2024-04-02T13:28:13.026 DEBUG:teuthology.orchestra.run.smithi033:mgr.x> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.x.service 2024-04-02T13:28:13.069 INFO:tasks.cephadm:Bootstrapping... 2024-04-02T13:28:13.070 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 -v bootstrap --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.33 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:28:13.271 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:28:13.272 INFO:teuthology.orchestra.run.smithi033.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6', '-v', 'bootstrap', '--fsid', 'b5fd982e-f0f4-11ee-b647-cb9ed24678a4', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.33', '--skip-admin-label'] 2024-04-02T13:28:13.272 INFO:teuthology.orchestra.run.smithi033.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-02T13:28:13.272 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying podman|docker is present... 2024-04-02T13:28:13.297 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T13:28:13.297 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying lvm2 is present... 2024-04-02T13:28:13.297 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying time synchronization is in place... 2024-04-02T13:28:13.304 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T13:28:13.304 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T13:28:13.310 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T13:28:13.311 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-02T13:28:13.316 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-04-02T13:28:13.322 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-04-02T13:28:13.322 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-04-02T13:28:13.323 INFO:teuthology.orchestra.run.smithi033.stdout:Repeating the final host check... 2024-04-02T13:28:13.346 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T13:28:13.346 INFO:teuthology.orchestra.run.smithi033.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-02T13:28:13.346 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl is present 2024-04-02T13:28:13.346 INFO:teuthology.orchestra.run.smithi033.stdout:lvcreate is present 2024-04-02T13:28:13.352 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T13:28:13.352 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T13:28:13.358 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T13:28:13.358 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-02T13:28:13.363 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-04-02T13:28:13.370 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-04-02T13:28:13.370 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-04-02T13:28:13.370 INFO:teuthology.orchestra.run.smithi033.stdout:Host looks OK 2024-04-02T13:28:13.370 INFO:teuthology.orchestra.run.smithi033.stdout:Cluster fsid: b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:28:13.370 INFO:teuthology.orchestra.run.smithi033.stdout:Acquiring lock 140120870068912 on /run/cephadm/b5fd982e-f0f4-11ee-b647-cb9ed24678a4.lock 2024-04-02T13:28:13.370 INFO:teuthology.orchestra.run.smithi033.stdout:Lock 140120870068912 acquired on /run/cephadm/b5fd982e-f0f4-11ee-b647-cb9ed24678a4.lock 2024-04-02T13:28:13.371 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 3300 ... 2024-04-02T13:28:13.371 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 6789 ... 2024-04-02T13:28:13.371 INFO:teuthology.orchestra.run.smithi033.stdout:Base mon IP(s) is [172.21.15.33:3300, 172.21.15.33:6789], mon addrv is [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-04-02T13:28:13.374 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.33 metric 100 2024-04-02T13:28:13.374 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.33 metric 100 2024-04-02T13:28:13.376 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-02T13:28:13.376 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-02T13:28:13.377 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-02T13:28:13.379 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-02T13:28:13.379 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-02T13:28:13.379 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T13:28:13.379 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-02T13:28:13.379 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-04-02T13:28:13.379 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T13:28:13.380 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-04-02T13:28:13.380 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-04-02T13:28:13.380 INFO:teuthology.orchestra.run.smithi033.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-02T13:28:13.380 INFO:teuthology.orchestra.run.smithi033.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-02T13:28:13.381 INFO:teuthology.orchestra.run.smithi033.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:28:13.840 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f 2024-04-02T13:28:13.840 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:28:13.840 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Getting image source signatures 2024-04-02T13:28:13.840 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:db8c11fadded9ebce76f2e720ef87f32e8913958c61c526c89a5e4e2c41a8200 2024-04-02T13:28:13.840 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:11da15500eca1f826c328eb3265caa30b0a3730510e2a32023084d6fe9cee20c 2024-04-02T13:28:13.840 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying config sha256:f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f 2024-04-02T13:28:13.840 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-02T13:28:14.763 INFO:teuthology.orchestra.run.smithi033.stdout:ceph: stdout ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T13:28:14.763 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph version: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T13:28:14.763 INFO:teuthology.orchestra.run.smithi033.stdout:Extracting ceph user uid/gid from container image... 2024-04-02T13:28:15.636 INFO:teuthology.orchestra.run.smithi033.stdout:stat: stdout 167 167 2024-04-02T13:28:15.636 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial keys... 2024-04-02T13:28:16.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQDvBwxmSqfJOhAA/tAtpF+Yuiv6qHiALiWrtA== 2024-04-02T13:28:17.398 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQDwBwxmfR+bMRAAuvf9aZ4JBTtr3UbvMfSNQg== 2024-04-02T13:28:18.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQDxBwxmKP56KBAAIGen1JmbmUHvvSYVNq3p5A== 2024-04-02T13:28:18.354 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial monmap... 2024-04-02T13:28:19.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T13:28:19.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-02T13:28:19.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:28:19.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T13:28:19.278 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool for a [v2:172.21.15.33:3300,v1:172.21.15.33:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T13:28:19.278 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = quincy 2024-04-02T13:28:19.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: set fsid to b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:28:19.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T13:28:19.279 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:28:19.279 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mon... 2024-04-02T13:28:20.126 INFO:teuthology.orchestra.run.smithi033.stdout:create mon.a on 2024-04-02T13:28:20.340 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-02T13:28:20.477 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-02T13:28:20.656 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4.target. 2024-04-02T13:28:20.656 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4.target. 2024-04-02T13:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.a 2024-04-02T13:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.a.service: Unit ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.a.service not loaded. 2024-04-02T13:28:21.043 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4.target.wants/ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.a.service → /etc/systemd/system/ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@.service. 2024-04-02T13:28:21.536 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:21 smithi033 ceph-mon[32019]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-02T13:28:21.549 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T13:28:21.549 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-02T13:28:21.555 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T13:28:21.555 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-02T13:28:21.556 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-02T13:28:21.556 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T13:28:21.556 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon to start... 2024-04-02T13:28:21.556 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon... 2024-04-02T13:28:21.839 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:21 smithi033 bash[32001]: ca2430c9120d2dd57b12c6f9bdc5e9c27d39684fa4781fc25f95b7f4fc1bacc3 2024-04-02T13:28:21.839 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:21 smithi033 systemd[1]: Started Ceph mon.a for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:28:22.537 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout cluster: 2024-04-02T13:28:22.538 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout id: b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:28:22.538 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-02T13:28:22.538 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-02T13:28:22.538 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout services: 2024-04-02T13:28:22.538 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.511708s) 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout data: 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pgs: 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.935+0000 7f6887901700 1 Processor -- start 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.935+0000 7f6887901700 1 -- start start 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.936+0000 7f6887901700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 0x7f6880108540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.936+0000 7f6887901700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6880108b10 con 0x7f6880108120 2024-04-02T13:28:22.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.936+0000 7f688569d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 0x7f6880108540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.936+0000 7f688569d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 0x7f6880108540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48964/0 (socket says 172.21.15.33:48964) 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.936+0000 7f688569d700 1 -- 172.21.15.33:0/2110067146 learned_addr learned my addr 172.21.15.33:0/2110067146 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.936+0000 7f688569d700 1 -- 172.21.15.33:0/2110067146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6880109320 con 0x7f6880108120 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.937+0000 7f688569d700 1 --2- 172.21.15.33:0/2110067146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 0x7f6880108540 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f687c0098a0 tx=0x7f687c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5d6b21e0d6bff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.937+0000 7f6873fff700 1 -- 172.21.15.33:0/2110067146 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f687c004160 con 0x7f6880108120 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.937+0000 7f6873fff700 1 -- 172.21.15.33:0/2110067146 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f687c004300 con 0x7f6880108120 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.937+0000 7f6873fff700 1 -- 172.21.15.33:0/2110067146 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f687c0036a0 con 0x7f6880108120 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.937+0000 7f6887901700 1 -- 172.21.15.33:0/2110067146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 msgr2=0x7f6880108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.937+0000 7f6887901700 1 --2- 172.21.15.33:0/2110067146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 0x7f6880108540 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f687c0098a0 tx=0x7f687c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.937+0000 7f6887901700 1 -- 172.21.15.33:0/2110067146 shutdown_connections 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.937+0000 7f6887901700 1 --2- 172.21.15.33:0/2110067146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 0x7f6880108540 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.937+0000 7f6887901700 1 -- 172.21.15.33:0/2110067146 >> 172.21.15.33:0/2110067146 conn(0x7f68801036e0 msgr2=0x7f6880105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.937+0000 7f6887901700 1 -- 172.21.15.33:0/2110067146 shutdown_connections 2024-04-02T13:28:22.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.937+0000 7f6887901700 1 -- 172.21.15.33:0/2110067146 wait complete. 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.938+0000 7f6887901700 1 Processor -- start 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.938+0000 7f6887901700 1 -- start start 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.938+0000 7f6887901700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 0x7f688019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.938+0000 7f6887901700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6880109850 con 0x7f6880108120 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.938+0000 7f688569d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 0x7f688019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.938+0000 7f688569d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 0x7f688019b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48976/0 (socket says 172.21.15.33:48976) 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.938+0000 7f688569d700 1 -- 172.21.15.33:0/274898765 learned_addr learned my addr 172.21.15.33:0/274898765 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.939+0000 7f688569d700 1 -- 172.21.15.33:0/274898765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688019bc70 con 0x7f6880108120 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.939+0000 7f688569d700 1 --2- 172.21.15.33:0/274898765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 0x7f688019b730 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f687c000c00 tx=0x7f687c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.939+0000 7f68727fc700 1 -- 172.21.15.33:0/274898765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f687c003db0 con 0x7f6880108120 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.939+0000 7f68727fc700 1 -- 172.21.15.33:0/274898765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f687c011630 con 0x7f6880108120 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.939+0000 7f6887901700 1 -- 172.21.15.33:0/274898765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688019bf00 con 0x7f6880108120 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.939+0000 7f68727fc700 1 -- 172.21.15.33:0/274898765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f687c011950 con 0x7f6880108120 2024-04-02T13:28:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.939+0000 7f6887901700 1 -- 172.21.15.33:0/274898765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688019ebf0 con 0x7f6880108120 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.939+0000 7f68727fc700 1 -- 172.21.15.33:0/274898765 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f687c011af0 con 0x7f6880108120 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.939+0000 7f68727fc700 1 -- 172.21.15.33:0/274898765 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f687c01c450 con 0x7f6880108120 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.940+0000 7f6887901700 1 -- 172.21.15.33:0/274898765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6880063c20 con 0x7f6880108120 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:21.942+0000 7f68727fc700 1 -- 172.21.15.33:0/274898765 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f687c032020 con 0x7f6880108120 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.014+0000 7f6887901700 1 -- 172.21.15.33:0/274898765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f688002deb0 con 0x7f6880108120 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.015+0000 7f68727fc700 1 -- 172.21.15.33:0/274898765 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f687c004260 con 0x7f6880108120 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.015+0000 7f6887901700 1 -- 172.21.15.33:0/274898765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 msgr2=0x7f688019b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.015+0000 7f6887901700 1 --2- 172.21.15.33:0/274898765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 0x7f688019b730 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f687c000c00 tx=0x7f687c004750 comp rx=0 tx=0).stop 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.015+0000 7f6887901700 1 -- 172.21.15.33:0/274898765 shutdown_connections 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.015+0000 7f6887901700 1 --2- 172.21.15.33:0/274898765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6880108120 0x7f688019b730 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.015+0000 7f6887901700 1 -- 172.21.15.33:0/274898765 >> 172.21.15.33:0/274898765 conn(0x7f68801036e0 msgr2=0x7f6880192430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.015+0000 7f6887901700 1 -- 172.21.15.33:0/274898765 shutdown_connections 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.015+0000 7f6887901700 1 -- 172.21.15.33:0/274898765 wait complete. 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:mon is available 2024-04-02T13:28:22.547 INFO:teuthology.orchestra.run.smithi033.stdout:Assimilating anything we can from ceph.conf... 2024-04-02T13:28:22.662 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:22 smithi033 ceph-mon[32019]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-02T13:28:22.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:22 smithi033 ceph-mon[32019]: monmap epoch 1 2024-04-02T13:28:22.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:22 smithi033 ceph-mon[32019]: fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:28:22.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:22 smithi033 ceph-mon[32019]: last_changed 2024-04-02T13:28:18.700852+0000 2024-04-02T13:28:22.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:22 smithi033 ceph-mon[32019]: created 2024-04-02T13:28:18.700852+0000 2024-04-02T13:28:22.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:22 smithi033 ceph-mon[32019]: min_mon_release 19 (squid) 2024-04-02T13:28:22.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:22 smithi033 ceph-mon[32019]: election_strategy: 1 2024-04-02T13:28:22.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:22 smithi033 ceph-mon[32019]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-02T13:28:22.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:22 smithi033 ceph-mon[32019]: fsmap 2024-04-02T13:28:22.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:22 smithi033 ceph-mon[32019]: osdmap e1: 0 total, 0 up, 0 in 2024-04-02T13:28:22.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:22 smithi033 ceph-mon[32019]: mgrmap e1: no daemons active 2024-04-02T13:28:22.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:22 smithi033 ceph-mon[32019]: from='client.? 172.21.15.33:0/274898765' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-02T13:28:23.636 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:23 smithi033 ceph-mon[32019]: from='client.? 172.21.15.33:0/3863648640' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-02T13:28:23.636 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:23 smithi033 ceph-mon[32019]: from='client.? 172.21.15.33:0/3863648640' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-02T13:28:23.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-02T13:28:23.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T13:28:23.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:28:23.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-04-02T13:28:23.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T13:28:23.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T13:28:23.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T13:28:23.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T13:28:23.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.928+0000 7f4c24b2b700 1 Processor -- start 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.928+0000 7f4c24b2b700 1 -- start start 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.928+0000 7f4c24b2b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 0x7f4c20108540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.928+0000 7f4c24b2b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4c20108b10 con 0x7f4c20108120 2024-04-02T13:28:23.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.928+0000 7f4c1e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 0x7f4c20108540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:23.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.928+0000 7f4c1e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 0x7f4c20108540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48978/0 (socket says 172.21.15.33:48978) 2024-04-02T13:28:23.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.928+0000 7f4c1e59c700 1 -- 172.21.15.33:0/233815262 learned_addr learned my addr 172.21.15.33:0/233815262 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:23.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.929+0000 7f4c1e59c700 1 -- 172.21.15.33:0/233815262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c20109320 con 0x7f4c20108120 2024-04-02T13:28:23.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.929+0000 7f4c1e59c700 1 --2- 172.21.15.33:0/233815262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 0x7f4c20108540 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4c18009e80 tx=0x7f4c1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddd25514c08f220 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:23.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.929+0000 7f4c1d59a700 1 -- 172.21.15.33:0/233815262 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c18017070 con 0x7f4c20108120 2024-04-02T13:28:23.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.929+0000 7f4c1d59a700 1 -- 172.21.15.33:0/233815262 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c18003a20 con 0x7f4c20108120 2024-04-02T13:28:23.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.929+0000 7f4c1d59a700 1 -- 172.21.15.33:0/233815262 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c18003d20 con 0x7f4c20108120 2024-04-02T13:28:23.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.930+0000 7f4c24b2b700 1 -- 172.21.15.33:0/233815262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 msgr2=0x7f4c20108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:23.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.930+0000 7f4c24b2b700 1 --2- 172.21.15.33:0/233815262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 0x7f4c20108540 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4c18009e80 tx=0x7f4c1800b330 comp rx=0 tx=0).stop 2024-04-02T13:28:23.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.930+0000 7f4c24b2b700 1 -- 172.21.15.33:0/233815262 shutdown_connections 2024-04-02T13:28:23.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.930+0000 7f4c24b2b700 1 --2- 172.21.15.33:0/233815262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 0x7f4c20108540 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:23.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.930+0000 7f4c24b2b700 1 -- 172.21.15.33:0/233815262 >> 172.21.15.33:0/233815262 conn(0x7f4c201036e0 msgr2=0x7f4c20105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:23.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.930+0000 7f4c24b2b700 1 -- 172.21.15.33:0/233815262 shutdown_connections 2024-04-02T13:28:23.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.930+0000 7f4c24b2b700 1 -- 172.21.15.33:0/233815262 wait complete. 2024-04-02T13:28:23.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.931+0000 7f4c24b2b700 1 Processor -- start 2024-04-02T13:28:23.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.931+0000 7f4c24b2b700 1 -- start start 2024-04-02T13:28:23.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.931+0000 7f4c24b2b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 0x7f4c2019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:23.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.931+0000 7f4c24b2b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4c20109850 con 0x7f4c20108120 2024-04-02T13:28:23.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.931+0000 7f4c1e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 0x7f4c2019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:23.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.931+0000 7f4c1e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 0x7f4c2019b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48980/0 (socket says 172.21.15.33:48980) 2024-04-02T13:28:23.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.931+0000 7f4c1e59c700 1 -- 172.21.15.33:0/3863648640 learned_addr learned my addr 172.21.15.33:0/3863648640 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:23.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.931+0000 7f4c1e59c700 1 -- 172.21.15.33:0/3863648640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2019bc70 con 0x7f4c20108120 2024-04-02T13:28:23.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.931+0000 7f4c1e59c700 1 --2- 172.21.15.33:0/3863648640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 0x7f4c2019b730 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4c18004000 tx=0x7f4c18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.931+0000 7f4c0f7fe700 1 -- 172.21.15.33:0/3863648640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c18022020 con 0x7f4c20108120 2024-04-02T13:28:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.931+0000 7f4c0f7fe700 1 -- 172.21.15.33:0/3863648640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c18018040 con 0x7f4c20108120 2024-04-02T13:28:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.931+0000 7f4c24b2b700 1 -- 172.21.15.33:0/3863648640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2019bf00 con 0x7f4c20108120 2024-04-02T13:28:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.932+0000 7f4c0f7fe700 1 -- 172.21.15.33:0/3863648640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c18017070 con 0x7f4c20108120 2024-04-02T13:28:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.932+0000 7f4c24b2b700 1 -- 172.21.15.33:0/3863648640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c2019ebf0 con 0x7f4c20108120 2024-04-02T13:28:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.932+0000 7f4c0f7fe700 1 -- 172.21.15.33:0/3863648640 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4c18025020 con 0x7f4c20108120 2024-04-02T13:28:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.932+0000 7f4c0f7fe700 1 -- 172.21.15.33:0/3863648640 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4c1800f050 con 0x7f4c20108120 2024-04-02T13:28:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.932+0000 7f4c24b2b700 1 -- 172.21.15.33:0/3863648640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c20063c20 con 0x7f4c20108120 2024-04-02T13:28:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:22.935+0000 7f4c0f7fe700 1 -- 172.21.15.33:0/3863648640 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4c1802c030 con 0x7f4c20108120 2024-04-02T13:28:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:23.005+0000 7f4c24b2b700 1 -- 172.21.15.33:0/3863648640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f4c2002def0 con 0x7f4c20108120 2024-04-02T13:28:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:23.008+0000 7f4c0f7fe700 1 -- 172.21.15.33:0/3863648640 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f4c1801e480 con 0x7f4c20108120 2024-04-02T13:28:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:23.009+0000 7f4c0f7fe700 1 -- 172.21.15.33:0/3863648640 <== mon.0 v2:172.21.15.33:3300/0 8 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f4c18035070 con 0x7f4c20108120 2024-04-02T13:28:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:23.009+0000 7f4c24b2b700 1 -- 172.21.15.33:0/3863648640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 msgr2=0x7f4c2019b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:23.650 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:23.009+0000 7f4c24b2b700 1 --2- 172.21.15.33:0/3863648640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 0x7f4c2019b730 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4c18004000 tx=0x7f4c18004ad0 comp rx=0 tx=0).stop 2024-04-02T13:28:23.650 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:23.009+0000 7f4c24b2b700 1 -- 172.21.15.33:0/3863648640 shutdown_connections 2024-04-02T13:28:23.650 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:23.009+0000 7f4c24b2b700 1 --2- 172.21.15.33:0/3863648640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c20108120 0x7f4c2019b730 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:23.650 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:23.009+0000 7f4c24b2b700 1 -- 172.21.15.33:0/3863648640 >> 172.21.15.33:0/3863648640 conn(0x7f4c201036e0 msgr2=0x7f4c20192430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:23.650 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:23.009+0000 7f4c24b2b700 1 -- 172.21.15.33:0/3863648640 shutdown_connections 2024-04-02T13:28:23.650 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:23.009+0000 7f4c24b2b700 1 -- 172.21.15.33:0/3863648640 wait complete. 2024-04-02T13:28:23.650 INFO:teuthology.orchestra.run.smithi033.stdout:Generating new minimal ceph.conf... 2024-04-02T13:28:24.865 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:24 smithi033 ceph-mon[32019]: from='client.? 172.21.15.33:0/877624888' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:24.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.065+0000 7f2251c50700 1 Processor -- start 2024-04-02T13:28:24.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.065+0000 7f2251c50700 1 -- start start 2024-04-02T13:28:24.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.066+0000 7f2251c50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 0x7f224c10af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:24.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.066+0000 7f2251c50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f224c07beb0 con 0x7f224c108ba0 2024-04-02T13:28:24.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.066+0000 7f224b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 0x7f224c10af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:24.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.066+0000 7f224b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 0x7f224c10af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35036/0 (socket says 172.21.15.33:35036) 2024-04-02T13:28:24.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.066+0000 7f224b7fe700 1 -- 172.21.15.33:0/3015804909 learned_addr learned my addr 172.21.15.33:0/3015804909 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:24.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.066+0000 7f224b7fe700 1 -- 172.21.15.33:0/3015804909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c10b4d0 con 0x7f224c108ba0 2024-04-02T13:28:24.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.066+0000 7f224b7fe700 1 --2- 172.21.15.33:0/3015804909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 0x7f224c10af90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f223c0098a0 tx=0x7f223c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120b794616603465 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:24.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.066+0000 7f224a7fc700 1 -- 172.21.15.33:0/3015804909 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f223c004160 con 0x7f224c108ba0 2024-04-02T13:28:24.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.067+0000 7f224a7fc700 1 -- 172.21.15.33:0/3015804909 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f223c004300 con 0x7f224c108ba0 2024-04-02T13:28:24.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.067+0000 7f224a7fc700 1 -- 172.21.15.33:0/3015804909 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f223c011b40 con 0x7f224c108ba0 2024-04-02T13:28:24.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.067+0000 7f2251c50700 1 -- 172.21.15.33:0/3015804909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 msgr2=0x7f224c10af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:24.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.067+0000 7f2251c50700 1 --2- 172.21.15.33:0/3015804909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 0x7f224c10af90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f223c0098a0 tx=0x7f223c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:28:24.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.067+0000 7f2251c50700 1 -- 172.21.15.33:0/3015804909 shutdown_connections 2024-04-02T13:28:24.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.067+0000 7f2251c50700 1 --2- 172.21.15.33:0/3015804909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 0x7f224c10af90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:24.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.067+0000 7f2251c50700 1 -- 172.21.15.33:0/3015804909 >> 172.21.15.33:0/3015804909 conn(0x7f224c1025f0 msgr2=0x7f224c104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:24.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.067+0000 7f2251c50700 1 -- 172.21.15.33:0/3015804909 shutdown_connections 2024-04-02T13:28:24.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.067+0000 7f2251c50700 1 -- 172.21.15.33:0/3015804909 wait complete. 2024-04-02T13:28:24.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.068+0000 7f2251c50700 1 Processor -- start 2024-04-02T13:28:24.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.068+0000 7f2251c50700 1 -- start start 2024-04-02T13:28:24.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.068+0000 7f2251c50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 0x7f224c19c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:24.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.068+0000 7f2251c50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f224c10c500 con 0x7f224c108ba0 2024-04-02T13:28:24.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.068+0000 7f224b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 0x7f224c19c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:24.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.068+0000 7f224b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 0x7f224c19c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35046/0 (socket says 172.21.15.33:35046) 2024-04-02T13:28:24.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.068+0000 7f224b7fe700 1 -- 172.21.15.33:0/877624888 learned_addr learned my addr 172.21.15.33:0/877624888 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:24.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.068+0000 7f224b7fe700 1 -- 172.21.15.33:0/877624888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c19c580 con 0x7f224c108ba0 2024-04-02T13:28:24.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.069+0000 7f224b7fe700 1 --2- 172.21.15.33:0/877624888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 0x7f224c19c040 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f223c004180 tx=0x7f223c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:24.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.069+0000 7f2248ff9700 1 -- 172.21.15.33:0/877624888 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f223c0039e0 con 0x7f224c108ba0 2024-04-02T13:28:24.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.069+0000 7f2251c50700 1 -- 172.21.15.33:0/877624888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c19c810 con 0x7f224c108ba0 2024-04-02T13:28:24.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.069+0000 7f2248ff9700 1 -- 172.21.15.33:0/877624888 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f223c020070 con 0x7f224c108ba0 2024-04-02T13:28:24.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.069+0000 7f2248ff9700 1 -- 172.21.15.33:0/877624888 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f223c0193c0 con 0x7f224c108ba0 2024-04-02T13:28:24.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.069+0000 7f2251c50700 1 -- 172.21.15.33:0/877624888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c19f500 con 0x7f224c108ba0 2024-04-02T13:28:24.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.069+0000 7f2248ff9700 1 -- 172.21.15.33:0/877624888 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f223c019a10 con 0x7f224c108ba0 2024-04-02T13:28:24.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.069+0000 7f2248ff9700 1 -- 172.21.15.33:0/877624888 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f223c00b720 con 0x7f224c108ba0 2024-04-02T13:28:24.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.070+0000 7f2251c50700 1 -- 172.21.15.33:0/877624888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224c063c20 con 0x7f224c108ba0 2024-04-02T13:28:24.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.072+0000 7f2248ff9700 1 -- 172.21.15.33:0/877624888 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f223c003b80 con 0x7f224c108ba0 2024-04-02T13:28:24.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.142+0000 7f2251c50700 1 -- 172.21.15.33:0/877624888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f224c19f6d0 con 0x7f224c108ba0 2024-04-02T13:28:24.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.142+0000 7f2248ff9700 1 -- 172.21.15.33:0/877624888 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f223c025460 con 0x7f224c108ba0 2024-04-02T13:28:24.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.143+0000 7f2251c50700 1 -- 172.21.15.33:0/877624888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 msgr2=0x7f224c19c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:24.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.143+0000 7f2251c50700 1 --2- 172.21.15.33:0/877624888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 0x7f224c19c040 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f223c004180 tx=0x7f223c0046a0 comp rx=0 tx=0).stop 2024-04-02T13:28:24.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.143+0000 7f2251c50700 1 -- 172.21.15.33:0/877624888 shutdown_connections 2024-04-02T13:28:24.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.143+0000 7f2251c50700 1 --2- 172.21.15.33:0/877624888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f224c108ba0 0x7f224c19c040 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:24.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.143+0000 7f2251c50700 1 -- 172.21.15.33:0/877624888 >> 172.21.15.33:0/877624888 conn(0x7f224c1025f0 msgr2=0x7f224c1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:24.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.143+0000 7f2251c50700 1 -- 172.21.15.33:0/877624888 shutdown_connections 2024-04-02T13:28:24.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:24.143+0000 7f2251c50700 1 -- 172.21.15.33:0/877624888 wait complete. 2024-04-02T13:28:24.877 INFO:teuthology.orchestra.run.smithi033.stdout:Restarting the monitor... 2024-04-02T13:28:25.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:24 smithi033 systemd[1]: Stopping Ceph mon.a for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:28:25.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:25 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a[32015]: 2024-04-02T13:28:25.140+0000 7f82a0daa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:28:25.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:25 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a[32015]: 2024-04-02T13:28:25.140+0000 7f82a0daa700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-02T13:28:25.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:25 smithi033 podman[32264]: 2024-04-02 13:28:25.192622901 +0000 UTC m=+0.160023942 container died ca2430c9120d2dd57b12c6f9bdc5e9c27d39684fa4781fc25f95b7f4fc1bacc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-02T13:28:25.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:25 smithi033 podman[32264]: 2024-04-02 13:28:25.56606968 +0000 UTC m=+0.533470707 container cleanup ca2430c9120d2dd57b12c6f9bdc5e9c27d39684fa4781fc25f95b7f4fc1bacc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:28:25.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:25 smithi033 bash[32264]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a 2024-04-02T13:28:26.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:25 smithi033 podman[32278]: 2024-04-02 13:28:25.817426422 +0000 UTC m=+0.620773988 container remove ca2430c9120d2dd57b12c6f9bdc5e9c27d39684fa4781fc25f95b7f4fc1bacc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326) 2024-04-02T13:28:26.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:25 smithi033 systemd[1]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.a.service: Deactivated successfully. 2024-04-02T13:28:26.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:25 smithi033 systemd[1]: Stopped Ceph mon.a for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:28:26.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:25 smithi033 systemd[1]: Starting Ceph mon.a for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:28:26.423 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 podman[32372]: 2024-04-02 13:28:26.134932127 +0000 UTC m=+0.112633426 container create 35cc49612e7c9978e7549cf9201b6fab093fd9913d514223163a73692b6ade50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 podman[32372]: 2024-04-02 13:28:26.039952205 +0000 UTC m=+0.017653469 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 podman[32372]: 2024-04-02 13:28:26.345470457 +0000 UTC m=+0.323171707 container init 35cc49612e7c9978e7549cf9201b6fab093fd9913d514223163a73692b6ade50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 podman[32372]: 2024-04-02 13:28:26.34822777 +0000 UTC m=+0.325929015 container start 35cc49612e7c9978e7549cf9201b6fab093fd9913d514223163a73692b6ade50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True) 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev), process ceph-mon, pid 2 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: pidfile_write: ignore empty --pid-file 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: load: jerasure load: lrc 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: RocksDB version: 7.9.2 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Git sha 0 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Compile date 2024-04-02 01:02:34 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: DB SUMMARY 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: DB Session ID: QAIUWVCAQTMQWVLZOGRB 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: CURRENT file: CURRENT 2024-04-02T13:28:26.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T13:28:26.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-02T13:28:26.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-02T13:28:26.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106194 ; 2024-04-02T13:28:26.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.error_if_exists: 0 2024-04-02T13:28:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.create_if_missing: 0 2024-04-02T13:28:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.paranoid_checks: 1 2024-04-02T13:28:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T13:28:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T13:28:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T13:28:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.env: 0x55b2ee6cac60 2024-04-02T13:28:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T13:28:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.info_log: 0x55b2efbcfae0 2024-04-02T13:28:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T13:28:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.statistics: (nil) 2024-04-02T13:28:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.use_fsync: 0 2024-04-02T13:28:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_log_file_size: 0 2024-04-02T13:28:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.allow_fallocate: 1 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.use_direct_reads: 0 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.db_log_dir: 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.wal_dir: 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T13:28:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.write_buffer_manager: 0x55b2efba01e0 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T13:28:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.unordered_write: 0 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.row_cache: None 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.wal_filter: None 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.two_write_queues: 0 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.wal_compression: 0 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.atomic_flush: 0 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T13:28:26.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.log_readahead_size: 0 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_background_jobs: 2 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_background_compactions: -1 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_subcompactions: 1 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T13:28:26.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_open_files: -1 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_background_flushes: -1 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Compression algorithms supported: 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: kZSTD supported: 0 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: kXpressCompression supported: 0 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T13:28:26.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: kZlibCompression supported: 1 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: kSnappyCompression supported: 1 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: kLZ4Compression supported: 1 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: kBZip2Compression supported: 0 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.merge_operator: 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compaction_filter: None 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T13:28:26.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b2efbcfcc0) 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks: 1 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_top_level_index_and_filter: 1 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_type: 0 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_index_type: 0 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_shortening: 1 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: checksum: 4 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: no_block_cache: 0 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache: 0x55b2efbd3090 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_name: BinnedLRUCache 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_options: 2024-04-02T13:28:26.433 INFO:journalctl@ceph.mon.a.smithi033.stdout: capacity : 536870912 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_shard_bits : 4 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: strict_capacity_limit : 0 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: high_pri_pool_ratio: 0.000 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_compressed: (nil) 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: persistent_cache: (nil) 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size: 4096 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size_deviation: 10 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_restart_interval: 16 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_block_restart_interval: 1 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: metadata_block_size: 4096 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: partition_filters: 0 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: use_delta_encoding: 1 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: filter_policy: bloomfilter 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: whole_key_filtering: 1 2024-04-02T13:28:26.434 INFO:journalctl@ceph.mon.a.smithi033.stdout: verify_compression: 0 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout: read_amp_bytes_per_bit: 0 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout: format_version: 5 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_index_compression: 1 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_align: 0 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout: max_auto_readahead_size: 262144 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout: prepopulate_block_cache: 0 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout: initial_auto_readahead_size: 8192 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compression: NoCompression 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T13:28:26.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T13:28:26.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.num_levels: 7 2024-04-02T13:28:26.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T13:28:26.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T13:28:26.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T13:28:26.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T13:28:26.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T13:28:26.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T13:28:26.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T13:28:26.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:28:26.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T13:28:26.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T13:28:26.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:28:26.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:28:26.437 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T13:28:26.437 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T13:28:26.437 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T13:28:26.437 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T13:28:26.437 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:28:26.437 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:28:26.437 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T13:28:26.437 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T13:28:26.437 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:28:26.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T13:28:26.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T13:28:26.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T13:28:26.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T13:28:26.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T13:28:26.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T13:28:26.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T13:28:26.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T13:28:26.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T13:28:26.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.inplace_update_support: 0 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T13:28:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.bloom_locality: 0 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.max_successive_merges: 0 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.ttl: 2592000 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.enable_blob_files: false 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.min_blob_size: 0 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T13:28:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T13:28:26.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T13:28:26.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T13:28:26.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T13:28:26.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T13:28:26.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-02T13:28:26.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-02T13:28:26.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f4604e30-e157-4e6c-a4d5-57e2db1b1a54 2024-04-02T13:28:26.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064506376120, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-02T13:28:26.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-02T13:28:26.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064506377085, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101279, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 99166, "index_size": 305, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14431, "raw_average_key_size": 49, "raw_value_size": 91288, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712064506, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f4604e30-e157-4e6c-a4d5-57e2db1b1a54", "db_session_id": "QAIUWVCAQTMQWVLZOGRB", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-02T13:28:26.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064506377169, "job": 1, "event": "recovery_finished"} 2024-04-02T13:28:26.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-02T13:28:26.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b2efcc2000 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: DB pointer 0x55b2efcac000 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** DB Stats ** 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-04-02T13:28:26.443 INFO:journalctl@ceph.mon.a.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: L0 2/0 100.77 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: Sum 2/0 100.77 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 155.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-02T13:28:26.444 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative compaction: 0.00 GB write, 33.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval compaction: 0.00 GB write, 33.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache BinnedLRUCache@0x55b2efbd3090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: starting mon.a rank 0 at public addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] at bind addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: mon.a@-1(???) e1 preinit fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: mon.a@-1(???).mds e1 new map 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: mon.a@-1(???).mds e1 print_map 2024-04-02T13:28:26.445 INFO:journalctl@ceph.mon.a.smithi033.stdout: e1 2024-04-02T13:28:26.446 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T13:28:26.446 INFO:journalctl@ceph.mon.a.smithi033.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T13:28:26.446 INFO:journalctl@ceph.mon.a.smithi033.stdout: legacy client fscid: -1 2024-04-02T13:28:26.446 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-02T13:28:26.446 INFO:journalctl@ceph.mon.a.smithi033.stdout: No filesystems configured 2024-04-02T13:28:26.446 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-02T13:28:26.446 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:28:26.446 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:28:26.446 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:28:26.446 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-02T13:28:26.446 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 bash[32372]: 35cc49612e7c9978e7549cf9201b6fab093fd9913d514223163a73692b6ade50 2024-04-02T13:28:26.446 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 systemd[1]: Started Ceph mon.a for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:28:26.446 INFO:teuthology.orchestra.run.smithi033.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-02T13:28:26.751 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-02T13:28:26.751 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: monmap epoch 1 2024-04-02T13:28:26.752 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:28:26.752 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: last_changed 2024-04-02T13:28:18.700852+0000 2024-04-02T13:28:26.752 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: created 2024-04-02T13:28:18.700852+0000 2024-04-02T13:28:26.752 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: min_mon_release 19 (squid) 2024-04-02T13:28:26.752 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: election_strategy: 1 2024-04-02T13:28:26.752 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-02T13:28:26.752 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: fsmap 2024-04-02T13:28:26.752 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: osdmap e1: 0 total, 0 up, 0 in 2024-04-02T13:28:26.752 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:26 smithi033 ceph-mon[32390]: mgrmap e1: no daemons active 2024-04-02T13:28:27.389 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.851+0000 7fa8cd397700 1 Processor -- start 2024-04-02T13:28:27.389 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.851+0000 7fa8cd397700 1 -- start start 2024-04-02T13:28:27.389 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.851+0000 7fa8cd397700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 0x7fa8c8108550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:27.389 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.851+0000 7fa8cd397700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c8108b20 con 0x7fa8c8108130 2024-04-02T13:28:27.389 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.852+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 0x7fa8c8108550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:27.389 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.852+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 0x7fa8c8108550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35052/0 (socket says 172.21.15.33:35052) 2024-04-02T13:28:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.852+0000 7fa8c6ffd700 1 -- 172.21.15.33:0/2987795465 learned_addr learned my addr 172.21.15.33:0/2987795465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.852+0000 7fa8c6ffd700 1 -- 172.21.15.33:0/2987795465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c8109330 con 0x7fa8c8108130 2024-04-02T13:28:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.852+0000 7fa8c6ffd700 1 --2- 172.21.15.33:0/2987795465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 0x7fa8c8108550 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009e80 tx=0x7fa8b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dd6e5ae17d3f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.852+0000 7fa8c5ffb700 1 -- 172.21.15.33:0/2987795465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b8017070 con 0x7fa8c8108130 2024-04-02T13:28:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.852+0000 7fa8c5ffb700 1 -- 172.21.15.33:0/2987795465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fa8b800b8c0 con 0x7fa8c8108130 2024-04-02T13:28:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.852+0000 7fa8c5ffb700 1 -- 172.21.15.33:0/2987795465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b80045a0 con 0x7fa8c8108130 2024-04-02T13:28:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.853+0000 7fa8cd397700 1 -- 172.21.15.33:0/2987795465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 msgr2=0x7fa8c8108550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.853+0000 7fa8cd397700 1 --2- 172.21.15.33:0/2987795465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 0x7fa8c8108550 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009e80 tx=0x7fa8b800b330 comp rx=0 tx=0).stop 2024-04-02T13:28:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.853+0000 7fa8cd397700 1 -- 172.21.15.33:0/2987795465 shutdown_connections 2024-04-02T13:28:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.853+0000 7fa8cd397700 1 --2- 172.21.15.33:0/2987795465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 0x7fa8c8108550 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.853+0000 7fa8cd397700 1 -- 172.21.15.33:0/2987795465 >> 172.21.15.33:0/2987795465 conn(0x7fa8c81036f0 msgr2=0x7fa8c8105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.853+0000 7fa8cd397700 1 -- 172.21.15.33:0/2987795465 shutdown_connections 2024-04-02T13:28:27.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.853+0000 7fa8cd397700 1 -- 172.21.15.33:0/2987795465 wait complete. 2024-04-02T13:28:27.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.853+0000 7fa8cd397700 1 Processor -- start 2024-04-02T13:28:27.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.854+0000 7fa8cd397700 1 -- start start 2024-04-02T13:28:27.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.854+0000 7fa8cd397700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 0x7fa8c807e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:27.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.854+0000 7fa8cd397700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c810c6e0 con 0x7fa8c8108130 2024-04-02T13:28:27.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.854+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 0x7fa8c807e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:27.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.854+0000 7fa8c6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 0x7fa8c807e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35066/0 (socket says 172.21.15.33:35066) 2024-04-02T13:28:27.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.854+0000 7fa8c6ffd700 1 -- 172.21.15.33:0/919404121 learned_addr learned my addr 172.21.15.33:0/919404121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:27.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.854+0000 7fa8c6ffd700 1 -- 172.21.15.33:0/919404121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c807e940 con 0x7fa8c8108130 2024-04-02T13:28:27.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.854+0000 7fa8c6ffd700 1 --2- 172.21.15.33:0/919404121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 0x7fa8c807e400 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80072a0 tx=0x7fa8b8003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:27.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.855+0000 7fa8bffff700 1 -- 172.21.15.33:0/919404121 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b8021020 con 0x7fa8c8108130 2024-04-02T13:28:27.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.855+0000 7fa8bffff700 1 -- 172.21.15.33:0/919404121 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fa8b80048b0 con 0x7fa8c8108130 2024-04-02T13:28:27.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.855+0000 7fa8cd397700 1 -- 172.21.15.33:0/919404121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c807cb30 con 0x7fa8c8108130 2024-04-02T13:28:27.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.855+0000 7fa8cd397700 1 -- 172.21.15.33:0/919404121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c807d070 con 0x7fa8c8108130 2024-04-02T13:28:27.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.855+0000 7fa8bffff700 1 -- 172.21.15.33:0/919404121 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b8017040 con 0x7fa8c8108130 2024-04-02T13:28:27.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.855+0000 7fa8bffff700 1 -- 172.21.15.33:0/919404121 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa8b8024020 con 0x7fa8c8108130 2024-04-02T13:28:27.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.855+0000 7fa8bffff700 1 -- 172.21.15.33:0/919404121 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa8b800f050 con 0x7fa8c8108130 2024-04-02T13:28:27.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.856+0000 7fa8cd397700 1 -- 172.21.15.33:0/919404121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c8063c20 con 0x7fa8c8108130 2024-04-02T13:28:27.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.858+0000 7fa8bffff700 1 -- 172.21.15.33:0/919404121 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa8b802b020 con 0x7fa8c8108130 2024-04-02T13:28:27.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.927+0000 7fa8cd397700 1 -- 172.21.15.33:0/919404121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fa8c807d3b0 con 0x7fa8c8108130 2024-04-02T13:28:27.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.931+0000 7fa8bffff700 1 -- 172.21.15.33:0/919404121 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fa8b80094d0 con 0x7fa8c8108130 2024-04-02T13:28:27.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.932+0000 7fa8cd397700 1 -- 172.21.15.33:0/919404121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 msgr2=0x7fa8c807e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:27.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.932+0000 7fa8cd397700 1 --2- 172.21.15.33:0/919404121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 0x7fa8c807e400 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80072a0 tx=0x7fa8b8003d00 comp rx=0 tx=0).stop 2024-04-02T13:28:27.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.932+0000 7fa8cd397700 1 -- 172.21.15.33:0/919404121 shutdown_connections 2024-04-02T13:28:27.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.932+0000 7fa8cd397700 1 --2- 172.21.15.33:0/919404121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8108130 0x7fa8c807e400 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:27.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.932+0000 7fa8cd397700 1 -- 172.21.15.33:0/919404121 >> 172.21.15.33:0/919404121 conn(0x7fa8c81036f0 msgr2=0x7fa8c8104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:27.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.932+0000 7fa8cd397700 1 -- 172.21.15.33:0/919404121 shutdown_connections 2024-04-02T13:28:27.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:26.932+0000 7fa8cd397700 1 -- 172.21.15.33:0/919404121 wait complete. 2024-04-02T13:28:27.394 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-02T13:28:27.413 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:28:27.413 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mgr... 2024-04-02T13:28:27.413 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-02T13:28:27.413 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-02T13:28:27.655 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.x 2024-04-02T13:28:27.655 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.x.service: Unit ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.x.service not loaded. 2024-04-02T13:28:27.802 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4.target.wants/ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.x.service → /etc/systemd/system/ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@.service. 2024-04-02T13:28:27.936 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:27 smithi033 systemd[1]: Starting Ceph mgr.x for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:28:28.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:27 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/919404121' entity='client.admin' 2024-04-02T13:28:28.234 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:28 smithi033 podman[32606]: 2024-04-02 13:28:28.021441265 +0000 UTC m=+0.102728177 container create 0bdee59f1aa5110f95979969ef77694eb11e4cf0e633ec8d57f935a409ed7b10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-02T13:28:28.234 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:28 smithi033 podman[32606]: 2024-04-02 13:28:27.936407038 +0000 UTC m=+0.017693928 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:28:28.235 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:28 smithi033 podman[32606]: 2024-04-02 13:28:28.232137555 +0000 UTC m=+0.313424427 container init 0bdee59f1aa5110f95979969ef77694eb11e4cf0e633ec8d57f935a409ed7b10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-02T13:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T13:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-02T13:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T13:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-02T13:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-02T13:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T13:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T13:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-02T13:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T13:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-02T13:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-02T13:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-02T13:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr to start... 2024-04-02T13:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr... 2024-04-02T13:28:28.494 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:28 smithi033 podman[32606]: 2024-04-02 13:28:28.234567958 +0000 UTC m=+0.315854829 container start 0bdee59f1aa5110f95979969ef77694eb11e4cf0e633ec8d57f935a409ed7b10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-02T13:28:28.494 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:28 smithi033 bash[32606]: 0bdee59f1aa5110f95979969ef77694eb11e4cf0e633ec8d57f935a409ed7b10 2024-04-02T13:28:28.494 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:28 smithi033 systemd[1]: Started Ceph mgr.x for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:28:29.182 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:28 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2999077436' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T13:28:29.183 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:28 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:28.804+0000 7ff552226200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:28:29.183 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:28 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:28.903+0000 7ff552226200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:28:29.515 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-02T13:28:29.515 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:28:29.515 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "b5fd982e-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:28:29.515 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T13:28:29.515 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T13:28:29.515 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T13:28:29.515 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T13:28:29.515 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:29.515 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T13:28:29.515 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T13:28:29.516 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-04-02T13:28:29.516 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:28:29.516 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T13:28:29.516 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T13:28:29.516 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:28:29.516 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-02T13:28:29.516 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T13:28:29.516 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:29.516 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T13:28:29.516 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T13:28:29.516 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:29.516 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T13:28:29.516 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:29.517 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T13:28:29.517 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T13:28:29.517 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T13:28:29.517 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T13:28:29.524 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T13:28:29.524 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T13:28:29.524 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:29.524 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:29.525 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T13:28:21.505369+0000", 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:29.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.801+0000 7f53fe023700 1 Processor -- start 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.801+0000 7f53fe023700 1 -- start start 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.801+0000 7f53fe023700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 0x7f53f8106de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.801+0000 7f53fe023700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f53f807c700 con 0x7f53f81069c0 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.802+0000 7f53f77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 0x7f53f8106de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.802+0000 7f53f77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 0x7f53f8106de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35106/0 (socket says 172.21.15.33:35106) 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.802+0000 7f53f77fe700 1 -- 172.21.15.33:0/1368069584 learned_addr learned my addr 172.21.15.33:0/1368069584 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.802+0000 7f53f77fe700 1 -- 172.21.15.33:0/1368069584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f8107320 con 0x7f53f81069c0 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.802+0000 7f53f77fe700 1 --2- 172.21.15.33:0/1368069584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 0x7f53f8106de0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f53e80098a0 tx=0x7f53e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512062d505f0ea62 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.802+0000 7f53f67fc700 1 -- 172.21.15.33:0/1368069584 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53e8004160 con 0x7f53f81069c0 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.802+0000 7f53f67fc700 1 -- 172.21.15.33:0/1368069584 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f53e8004300 con 0x7f53f81069c0 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.802+0000 7f53f67fc700 1 -- 172.21.15.33:0/1368069584 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53e8011b40 con 0x7f53f81069c0 2024-04-02T13:28:29.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.803+0000 7f53fe023700 1 -- 172.21.15.33:0/1368069584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 msgr2=0x7f53f8106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.803+0000 7f53fe023700 1 --2- 172.21.15.33:0/1368069584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 0x7f53f8106de0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f53e80098a0 tx=0x7f53e8009bb0 comp rx=0 tx=0).stop 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.803+0000 7f53fe023700 1 -- 172.21.15.33:0/1368069584 shutdown_connections 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.803+0000 7f53fe023700 1 --2- 172.21.15.33:0/1368069584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 0x7f53f8106de0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.803+0000 7f53fe023700 1 -- 172.21.15.33:0/1368069584 >> 172.21.15.33:0/1368069584 conn(0x7f53f81025e0 msgr2=0x7f53f8104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.803+0000 7f53fe023700 1 -- 172.21.15.33:0/1368069584 shutdown_connections 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.803+0000 7f53fe023700 1 -- 172.21.15.33:0/1368069584 wait complete. 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.803+0000 7f53fe023700 1 Processor -- start 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.804+0000 7f53fe023700 1 -- start start 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.804+0000 7f53fe023700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 0x7f53f819e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.804+0000 7f53fe023700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f53f81083b0 con 0x7f53f81069c0 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.804+0000 7f53f77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 0x7f53f819e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.804+0000 7f53f77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 0x7f53f819e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35118/0 (socket says 172.21.15.33:35118) 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.804+0000 7f53f77fe700 1 -- 172.21.15.33:0/2999077436 learned_addr learned my addr 172.21.15.33:0/2999077436 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.804+0000 7f53f77fe700 1 -- 172.21.15.33:0/2999077436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f819e700 con 0x7f53f81069c0 2024-04-02T13:28:29.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.804+0000 7f53f77fe700 1 --2- 172.21.15.33:0/2999077436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 0x7f53f819e1c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f53e8011840 tx=0x7f53e80119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.805+0000 7f53f4ff9700 1 -- 172.21.15.33:0/2999077436 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53e80039e0 con 0x7f53f81069c0 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.805+0000 7f53f4ff9700 1 -- 172.21.15.33:0/2999077436 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f53e8020070 con 0x7f53f81069c0 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.805+0000 7f53fe023700 1 -- 172.21.15.33:0/2999077436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f819e990 con 0x7f53f81069c0 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.805+0000 7f53f4ff9700 1 -- 172.21.15.33:0/2999077436 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53e80193c0 con 0x7f53f81069c0 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.805+0000 7f53fe023700 1 -- 172.21.15.33:0/2999077436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f81a1680 con 0x7f53f81069c0 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.805+0000 7f53f4ff9700 1 -- 172.21.15.33:0/2999077436 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f53e8019a10 con 0x7f53f81069c0 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.805+0000 7f53f4ff9700 1 -- 172.21.15.33:0/2999077436 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f53e800b720 con 0x7f53f81069c0 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.806+0000 7f53fe023700 1 -- 172.21.15.33:0/2999077436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f8063c20 con 0x7f53f81069c0 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.808+0000 7f53f4ff9700 1 -- 172.21.15.33:0/2999077436 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f53e8003b80 con 0x7f53f81069c0 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.881+0000 7f53fe023700 1 -- 172.21.15.33:0/2999077436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f53f802e070 con 0x7f53f81069c0 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.882+0000 7f53f4ff9700 1 -- 172.21.15.33:0/2999077436 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f53e8025460 con 0x7f53f81069c0 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.883+0000 7f53fe023700 1 -- 172.21.15.33:0/2999077436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 msgr2=0x7f53f819e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.883+0000 7f53fe023700 1 --2- 172.21.15.33:0/2999077436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 0x7f53f819e1c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f53e8011840 tx=0x7f53e80119d0 comp rx=0 tx=0).stop 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.883+0000 7f53fe023700 1 -- 172.21.15.33:0/2999077436 shutdown_connections 2024-04-02T13:28:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.883+0000 7f53fe023700 1 --2- 172.21.15.33:0/2999077436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f81069c0 0x7f53f819e1c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:29.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.883+0000 7f53fe023700 1 -- 172.21.15.33:0/2999077436 >> 172.21.15.33:0/2999077436 conn(0x7f53f81025e0 msgr2=0x7f53f8194e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:29.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.883+0000 7f53fe023700 1 -- 172.21.15.33:0/2999077436 shutdown_connections 2024-04-02T13:28:29.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:28.883+0000 7f53fe023700 1 -- 172.21.15.33:0/2999077436 wait complete. 2024-04-02T13:28:29.530 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (1/15)... 2024-04-02T13:28:29.821 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:29 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:29.579+0000 7ff552226200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:28:29.821 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:29 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:29.674+0000 7ff552226200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:28:30.254 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:29 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:29.817+0000 7ff552226200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:28:30.254 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:29 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:29.915+0000 7ff552226200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:28:30.254 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:30 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:30.000+0000 7ff552226200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:28:30.578 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:30 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:30.270+0000 7ff552226200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:28:30.579 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:30 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:30.387+0000 7ff552226200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:28:30.579 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:30 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:30.477+0000 7ff552226200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:28:30.579 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:30 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:30.574+0000 7ff552226200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:28:31.004 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:30 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:30.752+0000 7ff552226200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:28:31.909 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:31 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:31.768+0000 7ff552226200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:28:31.909 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:31 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:31.905+0000 7ff552226200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:28:32.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:32 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2226915478' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T13:28:32.557 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:32 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:32.255+0000 7ff552226200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:28:32.557 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:32 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:32.348+0000 7ff552226200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:28:32.557 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:32 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:32.553+0000 7ff552226200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:28:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-02T13:28:32.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:28:32.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "b5fd982e-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:28:32.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T13:28:32.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T13:28:32.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T13:28:32.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T13:28:32.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:32.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:32.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T13:28:32.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:28:32.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T13:28:21.505369+0000", 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.957+0000 7f10c6be0700 1 Processor -- start 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.957+0000 7f10c6be0700 1 -- start start 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.957+0000 7f10c6be0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 0x7f10c010afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.957+0000 7f10c6be0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f10c007c6e0 con 0x7f10c0108bf0 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.957+0000 7f10c497c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 0x7f10c010afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.958+0000 7f10c497c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 0x7f10c010afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35132/0 (socket says 172.21.15.33:35132) 2024-04-02T13:28:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.958+0000 7f10c497c700 1 -- 172.21.15.33:0/688281545 learned_addr learned my addr 172.21.15.33:0/688281545 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.958+0000 7f10c497c700 1 -- 172.21.15.33:0/688281545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c010b520 con 0x7f10c0108bf0 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.958+0000 7f10c497c700 1 --2- 172.21.15.33:0/688281545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 0x7f10c010afe0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f10b40098a0 tx=0x7f10b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1aef13d420b436 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.958+0000 7f10bf7fe700 1 -- 172.21.15.33:0/688281545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10b4004160 con 0x7f10c0108bf0 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.958+0000 7f10bf7fe700 1 -- 172.21.15.33:0/688281545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f10b4004300 con 0x7f10c0108bf0 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.958+0000 7f10bf7fe700 1 -- 172.21.15.33:0/688281545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10b4011b40 con 0x7f10c0108bf0 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.959+0000 7f10c6be0700 1 -- 172.21.15.33:0/688281545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 msgr2=0x7f10c010afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.959+0000 7f10c6be0700 1 --2- 172.21.15.33:0/688281545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 0x7f10c010afe0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f10b40098a0 tx=0x7f10b4009bb0 comp rx=0 tx=0).stop 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.959+0000 7f10c6be0700 1 -- 172.21.15.33:0/688281545 shutdown_connections 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.959+0000 7f10c6be0700 1 --2- 172.21.15.33:0/688281545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 0x7f10c010afe0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.959+0000 7f10c6be0700 1 -- 172.21.15.33:0/688281545 >> 172.21.15.33:0/688281545 conn(0x7f10c01025e0 msgr2=0x7f10c0104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.959+0000 7f10c6be0700 1 -- 172.21.15.33:0/688281545 shutdown_connections 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.959+0000 7f10c6be0700 1 -- 172.21.15.33:0/688281545 wait complete. 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.959+0000 7f10c6be0700 1 Processor -- start 2024-04-02T13:28:32.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.959+0000 7f10c6be0700 1 -- start start 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.960+0000 7f10c6be0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 0x7f10c0108470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.960+0000 7f10c6be0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f10c010c4f0 con 0x7f10c0108bf0 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.960+0000 7f10c497c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 0x7f10c0108470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.960+0000 7f10c497c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 0x7f10c0108470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35142/0 (socket says 172.21.15.33:35142) 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.960+0000 7f10c497c700 1 -- 172.21.15.33:0/2226915478 learned_addr learned my addr 172.21.15.33:0/2226915478 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.960+0000 7f10c497c700 1 -- 172.21.15.33:0/2226915478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c01089b0 con 0x7f10c0108bf0 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.960+0000 7f10c497c700 1 --2- 172.21.15.33:0/2226915478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 0x7f10c0108470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f10b4011840 tx=0x7f10b40119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.960+0000 7f10bdffb700 1 -- 172.21.15.33:0/2226915478 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10b40039e0 con 0x7f10c0108bf0 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.960+0000 7f10bdffb700 1 -- 172.21.15.33:0/2226915478 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f10b4020070 con 0x7f10c0108bf0 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.960+0000 7f10bdffb700 1 -- 172.21.15.33:0/2226915478 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10b40193c0 con 0x7f10c0108bf0 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.960+0000 7f10c6be0700 1 -- 172.21.15.33:0/2226915478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c0106ba0 con 0x7f10c0108bf0 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.961+0000 7f10c6be0700 1 -- 172.21.15.33:0/2226915478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c01070e0 con 0x7f10c0108bf0 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.961+0000 7f10bdffb700 1 -- 172.21.15.33:0/2226915478 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f10b40199c0 con 0x7f10c0108bf0 2024-04-02T13:28:32.634 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.961+0000 7f10bdffb700 1 -- 172.21.15.33:0/2226915478 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f10b400b720 con 0x7f10c0108bf0 2024-04-02T13:28:32.635 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.961+0000 7f10c6be0700 1 -- 172.21.15.33:0/2226915478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c0063c20 con 0x7f10c0108bf0 2024-04-02T13:28:32.635 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:31.964+0000 7f10bdffb700 1 -- 172.21.15.33:0/2226915478 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f10b4003b80 con 0x7f10c0108bf0 2024-04-02T13:28:32.635 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:32.037+0000 7f10c6be0700 1 -- 172.21.15.33:0/2226915478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f10c0107420 con 0x7f10c0108bf0 2024-04-02T13:28:32.635 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:32.038+0000 7f10bdffb700 1 -- 172.21.15.33:0/2226915478 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f10b4025460 con 0x7f10c0108bf0 2024-04-02T13:28:32.635 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:32.039+0000 7f10c6be0700 1 -- 172.21.15.33:0/2226915478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 msgr2=0x7f10c0108470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:32.635 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:32.039+0000 7f10c6be0700 1 --2- 172.21.15.33:0/2226915478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 0x7f10c0108470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f10b4011840 tx=0x7f10b40119d0 comp rx=0 tx=0).stop 2024-04-02T13:28:32.635 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:32.039+0000 7f10c6be0700 1 -- 172.21.15.33:0/2226915478 shutdown_connections 2024-04-02T13:28:32.635 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:32.039+0000 7f10c6be0700 1 --2- 172.21.15.33:0/2226915478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c0108bf0 0x7f10c0108470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:32.635 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:32.039+0000 7f10c6be0700 1 -- 172.21.15.33:0/2226915478 >> 172.21.15.33:0/2226915478 conn(0x7f10c01025e0 msgr2=0x7f10c0102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:32.635 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:32.039+0000 7f10c6be0700 1 -- 172.21.15.33:0/2226915478 shutdown_connections 2024-04-02T13:28:32.635 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:32.039+0000 7f10c6be0700 1 -- 172.21.15.33:0/2226915478 wait complete. 2024-04-02T13:28:32.635 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (2/15)... 2024-04-02T13:28:33.004 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:32 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:32.744+0000 7ff552226200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:28:33.307 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:33 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:33.041+0000 7ff552226200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:28:33.754 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:33 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:33.303+0000 7ff552226200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:28:33.754 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:33 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:33.550+0000 7ff552226200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:28:34.746 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:34 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:34.395+0000 7ff552226200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:28:35.004 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:34 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:34.989+0000 7ff552226200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:28:35.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:35 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1742453495' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T13:28:35.324 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:35 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:35.093+0000 7ff552226200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:28:35.696 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:35 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:35.320+0000 7ff552226200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:28:35.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-02T13:28:35.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:28:35.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "b5fd982e-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:28:35.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T13:28:35.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T13:28:35.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T13:28:35.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T13:28:35.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:35.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T13:28:35.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T13:28:35.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T13:28:35.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T13:28:35.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:28:35.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T13:28:21.505369+0000", 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.072+0000 7f449e1be700 1 Processor -- start 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.072+0000 7f449e1be700 1 -- start start 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.072+0000 7f449e1be700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 0x7f449810afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.072+0000 7f449e1be700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f449807c720 con 0x7f4498108bb0 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.072+0000 7f44977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 0x7f449810afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.072+0000 7f44977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 0x7f449810afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58666/0 (socket says 172.21.15.33:58666) 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.072+0000 7f44977fe700 1 -- 172.21.15.33:0/211761982 learned_addr learned my addr 172.21.15.33:0/211761982 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:35.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.073+0000 7f44977fe700 1 -- 172.21.15.33:0/211761982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449810b4e0 con 0x7f4498108bb0 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.073+0000 7f44977fe700 1 --2- 172.21.15.33:0/211761982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 0x7f449810afa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f44800098a0 tx=0x7f4480009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88866e90c1723eda server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.073+0000 7f44967fc700 1 -- 172.21.15.33:0/211761982 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4480004160 con 0x7f4498108bb0 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.073+0000 7f44967fc700 1 -- 172.21.15.33:0/211761982 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f4480004300 con 0x7f4498108bb0 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.073+0000 7f44967fc700 1 -- 172.21.15.33:0/211761982 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4480011b40 con 0x7f4498108bb0 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.074+0000 7f449e1be700 1 -- 172.21.15.33:0/211761982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 msgr2=0x7f449810afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.074+0000 7f449e1be700 1 --2- 172.21.15.33:0/211761982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 0x7f449810afa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f44800098a0 tx=0x7f4480009bb0 comp rx=0 tx=0).stop 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.074+0000 7f449e1be700 1 -- 172.21.15.33:0/211761982 shutdown_connections 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.074+0000 7f449e1be700 1 --2- 172.21.15.33:0/211761982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 0x7f449810afa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.074+0000 7f449e1be700 1 -- 172.21.15.33:0/211761982 >> 172.21.15.33:0/211761982 conn(0x7f44981025e0 msgr2=0x7f4498104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.074+0000 7f449e1be700 1 -- 172.21.15.33:0/211761982 shutdown_connections 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.074+0000 7f449e1be700 1 -- 172.21.15.33:0/211761982 wait complete. 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.074+0000 7f449e1be700 1 Processor -- start 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.074+0000 7f449e1be700 1 -- start start 2024-04-02T13:28:35.714 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.075+0000 7f449e1be700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 0x7f4498108430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.075+0000 7f449e1be700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f449810c490 con 0x7f4498108bb0 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.075+0000 7f44977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 0x7f4498108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.075+0000 7f44977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 0x7f4498108430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58676/0 (socket says 172.21.15.33:58676) 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.075+0000 7f44977fe700 1 -- 172.21.15.33:0/1742453495 learned_addr learned my addr 172.21.15.33:0/1742453495 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.075+0000 7f44977fe700 1 -- 172.21.15.33:0/1742453495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4498108970 con 0x7f4498108bb0 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.075+0000 7f44977fe700 1 --2- 172.21.15.33:0/1742453495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 0x7f4498108430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4480011840 tx=0x7f44800119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.075+0000 7f4494ff9700 1 -- 172.21.15.33:0/1742453495 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44800039e0 con 0x7f4498108bb0 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.075+0000 7f4494ff9700 1 -- 172.21.15.33:0/1742453495 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f4480020070 con 0x7f4498108bb0 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.075+0000 7f449e1be700 1 -- 172.21.15.33:0/1742453495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4498106b60 con 0x7f4498108bb0 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.075+0000 7f4494ff9700 1 -- 172.21.15.33:0/1742453495 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44800193c0 con 0x7f4498108bb0 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.075+0000 7f449e1be700 1 -- 172.21.15.33:0/1742453495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44981070a0 con 0x7f4498108bb0 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.076+0000 7f4494ff9700 1 -- 172.21.15.33:0/1742453495 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4480019a90 con 0x7f4498108bb0 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.076+0000 7f4494ff9700 1 -- 172.21.15.33:0/1742453495 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4480025d70 con 0x7f4498108bb0 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.077+0000 7f449e1be700 1 -- 172.21.15.33:0/1742453495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4498063c20 con 0x7f4498108bb0 2024-04-02T13:28:35.715 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.079+0000 7f4494ff9700 1 -- 172.21.15.33:0/1742453495 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4480019d80 con 0x7f4498108bb0 2024-04-02T13:28:35.716 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.152+0000 7f449e1be700 1 -- 172.21.15.33:0/1742453495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4498107350 con 0x7f4498108bb0 2024-04-02T13:28:35.716 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.152+0000 7f4494ff9700 1 -- 172.21.15.33:0/1742453495 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4480003b80 con 0x7f4498108bb0 2024-04-02T13:28:35.716 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.153+0000 7f449e1be700 1 -- 172.21.15.33:0/1742453495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 msgr2=0x7f4498108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:35.716 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.153+0000 7f449e1be700 1 --2- 172.21.15.33:0/1742453495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 0x7f4498108430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4480011840 tx=0x7f44800119d0 comp rx=0 tx=0).stop 2024-04-02T13:28:35.716 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.153+0000 7f449e1be700 1 -- 172.21.15.33:0/1742453495 shutdown_connections 2024-04-02T13:28:35.716 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.153+0000 7f449e1be700 1 --2- 172.21.15.33:0/1742453495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4498108bb0 0x7f4498108430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:35.716 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.153+0000 7f449e1be700 1 -- 172.21.15.33:0/1742453495 >> 172.21.15.33:0/1742453495 conn(0x7f44981025e0 msgr2=0x7f4498102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:35.716 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.153+0000 7f449e1be700 1 -- 172.21.15.33:0/1742453495 shutdown_connections 2024-04-02T13:28:35.716 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:35.153+0000 7f449e1be700 1 -- 172.21.15.33:0/1742453495 wait complete. 2024-04-02T13:28:35.716 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (3/15)... 2024-04-02T13:28:36.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:36 smithi033 ceph-mon[32390]: Activating manager daemon x 2024-04-02T13:28:36.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:36 smithi033 ceph-mon[32390]: mgrmap e2: x(active, starting, since 0.00335366s) 2024-04-02T13:28:36.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:36 smithi033 ceph-mon[32390]: from='mgr.14100 172.21.15.33:0/255061912' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:28:36.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:36 smithi033 ceph-mon[32390]: from='mgr.14100 172.21.15.33:0/255061912' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:28:36.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:36 smithi033 ceph-mon[32390]: from='mgr.14100 172.21.15.33:0/255061912' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:28:36.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:36 smithi033 ceph-mon[32390]: from='mgr.14100 172.21.15.33:0/255061912' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:28:36.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:36 smithi033 ceph-mon[32390]: from='mgr.14100 172.21.15.33:0/255061912' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:28:36.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:36 smithi033 ceph-mon[32390]: Manager daemon x is now available 2024-04-02T13:28:36.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:36 smithi033 ceph-mon[32390]: from='mgr.14100 172.21.15.33:0/255061912' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:28:36.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:36 smithi033 ceph-mon[32390]: from='mgr.14100 172.21.15.33:0/255061912' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:28:36.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:36 smithi033 ceph-mon[32390]: from='mgr.14100 172.21.15.33:0/255061912' entity='mgr.x' 2024-04-02T13:28:36.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:36 smithi033 ceph-mon[32390]: from='mgr.14100 172.21.15.33:0/255061912' entity='mgr.x' 2024-04-02T13:28:36.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:36 smithi033 ceph-mon[32390]: from='mgr.14100 172.21.15.33:0/255061912' entity='mgr.x' 2024-04-02T13:28:37.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:37 smithi033 ceph-mon[32390]: mgrmap e3: x(active, since 1.00758s) 2024-04-02T13:28:38.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:38 smithi033 ceph-mon[32390]: mgrmap e4: x(active, since 2s) 2024-04-02T13:28:39.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-02T13:28:39.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:28:39.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "b5fd982e-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:28:39.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T13:28:39.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T13:28:39.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T13:28:39.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T13:28:39.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:39.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:39.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T13:28:39.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T13:28:39.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T13:28:39.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T13:28:39.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T13:28:39.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T13:28:39.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:39.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T13:28:39.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T13:28:39.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T13:28:39.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T13:28:39.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T13:28:39.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T13:28:21.505369+0000", 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.130+0000 7ffa666e0700 1 Processor -- start 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.130+0000 7ffa666e0700 1 -- start start 2024-04-02T13:28:39.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.130+0000 7ffa666e0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 0x7ffa60107440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.130+0000 7ffa666e0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60107a10 con 0x7ffa60107020 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.131+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 0x7ffa60107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.131+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 0x7ffa60107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58740/0 (socket says 172.21.15.33:58740) 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.131+0000 7ffa5ffff700 1 -- 172.21.15.33:0/1784582683 learned_addr learned my addr 172.21.15.33:0/1784582683 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.131+0000 7ffa5ffff700 1 -- 172.21.15.33:0/1784582683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa60108220 con 0x7ffa60107020 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.131+0000 7ffa5ffff700 1 --2- 172.21.15.33:0/1784582683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 0x7ffa60107440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffa500098a0 tx=0x7ffa50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97151a2cf880fe86 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.131+0000 7ffa5effd700 1 -- 172.21.15.33:0/1784582683 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa50004160 con 0x7ffa60107020 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.131+0000 7ffa5effd700 1 -- 172.21.15.33:0/1784582683 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7ffa50004300 con 0x7ffa60107020 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.131+0000 7ffa5effd700 1 -- 172.21.15.33:0/1784582683 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa50011b40 con 0x7ffa60107020 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.132+0000 7ffa666e0700 1 -- 172.21.15.33:0/1784582683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 msgr2=0x7ffa60107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.132+0000 7ffa666e0700 1 --2- 172.21.15.33:0/1784582683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 0x7ffa60107440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffa500098a0 tx=0x7ffa50009bb0 comp rx=0 tx=0).stop 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.132+0000 7ffa666e0700 1 -- 172.21.15.33:0/1784582683 shutdown_connections 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.132+0000 7ffa666e0700 1 --2- 172.21.15.33:0/1784582683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 0x7ffa60107440 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:39.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.132+0000 7ffa666e0700 1 -- 172.21.15.33:0/1784582683 >> 172.21.15.33:0/1784582683 conn(0x7ffa601025e0 msgr2=0x7ffa60104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.132+0000 7ffa666e0700 1 -- 172.21.15.33:0/1784582683 shutdown_connections 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.132+0000 7ffa666e0700 1 -- 172.21.15.33:0/1784582683 wait complete. 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.133+0000 7ffa666e0700 1 Processor -- start 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.133+0000 7ffa666e0700 1 -- start start 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.133+0000 7ffa666e0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 0x7ffa6019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.133+0000 7ffa666e0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6010b5d0 con 0x7ffa60107020 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.133+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 0x7ffa6019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.133+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 0x7ffa6019c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58744/0 (socket says 172.21.15.33:58744) 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.133+0000 7ffa5ffff700 1 -- 172.21.15.33:0/797669810 learned_addr learned my addr 172.21.15.33:0/797669810 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.133+0000 7ffa5ffff700 1 -- 172.21.15.33:0/797669810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6019c550 con 0x7ffa60107020 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.134+0000 7ffa5ffff700 1 --2- 172.21.15.33:0/797669810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 0x7ffa6019c010 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffa50006af0 tx=0x7ffa50011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.134+0000 7ffa5d7fa700 1 -- 172.21.15.33:0/797669810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa50003970 con 0x7ffa60107020 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.134+0000 7ffa5d7fa700 1 -- 172.21.15.33:0/797669810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7ffa50003b10 con 0x7ffa60107020 2024-04-02T13:28:39.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.134+0000 7ffa666e0700 1 -- 172.21.15.33:0/797669810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6019c7e0 con 0x7ffa60107020 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.134+0000 7ffa5d7fa700 1 -- 172.21.15.33:0/797669810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa500196b0 con 0x7ffa60107020 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.134+0000 7ffa666e0700 1 -- 172.21.15.33:0/797669810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6019f4d0 con 0x7ffa60107020 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.135+0000 7ffa5d7fa700 1 -- 172.21.15.33:0/797669810 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ffa50019960 con 0x7ffa60107020 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.135+0000 7ffa666e0700 1 -- 172.21.15.33:0/797669810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa60063c20 con 0x7ffa60107020 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.135+0000 7ffa5d7fa700 1 --2- 172.21.15.33:0/797669810 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffa48039620 0x7ffa4803bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.135+0000 7ffa5d7fa700 1 -- 172.21.15.33:0/797669810 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffa5004da60 con 0x7ffa60107020 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.135+0000 7ffa5f7fe700 1 --2- 172.21.15.33:0/797669810 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffa48039620 0x7ffa4803bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.135+0000 7ffa5f7fe700 1 --2- 172.21.15.33:0/797669810 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffa48039620 0x7ffa4803bae0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffa54006fd0 tx=0x7ffa54006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.140+0000 7ffa5d7fa700 1 -- 172.21.15.33:0/797669810 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ffa50003cc0 con 0x7ffa60107020 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.432+0000 7ffa666e0700 1 -- 172.21.15.33:0/797669810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ffa600008d0 con 0x7ffa60107020 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.433+0000 7ffa5d7fa700 1 -- 172.21.15.33:0/797669810 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ffa5000bc70 con 0x7ffa60107020 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.436+0000 7ffa666e0700 1 -- 172.21.15.33:0/797669810 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffa48039620 msgr2=0x7ffa4803bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.436+0000 7ffa666e0700 1 --2- 172.21.15.33:0/797669810 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffa48039620 0x7ffa4803bae0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffa54006fd0 tx=0x7ffa54006e40 comp rx=0 tx=0).stop 2024-04-02T13:28:39.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.436+0000 7ffa666e0700 1 -- 172.21.15.33:0/797669810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 msgr2=0x7ffa6019c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:39.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.436+0000 7ffa666e0700 1 --2- 172.21.15.33:0/797669810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 0x7ffa6019c010 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffa50006af0 tx=0x7ffa50011c00 comp rx=0 tx=0).stop 2024-04-02T13:28:39.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.436+0000 7ffa666e0700 1 -- 172.21.15.33:0/797669810 shutdown_connections 2024-04-02T13:28:39.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.436+0000 7ffa666e0700 1 --2- 172.21.15.33:0/797669810 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffa48039620 0x7ffa4803bae0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:39.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.436+0000 7ffa666e0700 1 --2- 172.21.15.33:0/797669810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa60107020 0x7ffa6019c010 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:39.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.436+0000 7ffa666e0700 1 -- 172.21.15.33:0/797669810 >> 172.21.15.33:0/797669810 conn(0x7ffa601025e0 msgr2=0x7ffa60192bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:39.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.436+0000 7ffa666e0700 1 -- 172.21.15.33:0/797669810 shutdown_connections 2024-04-02T13:28:39.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:38.436+0000 7ffa666e0700 1 -- 172.21.15.33:0/797669810 wait complete. 2024-04-02T13:28:39.024 INFO:teuthology.orchestra.run.smithi033.stdout:mgr is available 2024-04-02T13:28:39.659 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:39 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/797669810' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T13:28:40.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-02T13:28:40.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T13:28:40.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:28:40.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T13:28:40.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T13:28:40.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T13:28:40.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T13:28:40.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-02T13:28:40.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T13:28:40.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T13:28:40.195 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-02T13:28:40.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T13:28:40.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T13:28:40.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T13:28:40.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T13:28:40.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T13:28:40.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T13:28:40.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.439+0000 7f261180a700 1 Processor -- start 2024-04-02T13:28:40.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.439+0000 7f261180a700 1 -- start start 2024-04-02T13:28:40.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.439+0000 7f261180a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 0x7f260c07cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:40.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.439+0000 7f261180a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f260c07d160 con 0x7f260c07e730 2024-04-02T13:28:40.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.439+0000 7f260affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 0x7f260c07cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:40.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.439+0000 7f260affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 0x7f260c07cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58754/0 (socket says 172.21.15.33:58754) 2024-04-02T13:28:40.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.439+0000 7f260affd700 1 -- 172.21.15.33:0/2282477888 learned_addr learned my addr 172.21.15.33:0/2282477888 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:40.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.440+0000 7f260affd700 1 -- 172.21.15.33:0/2282477888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c07d2e0 con 0x7f260c07e730 2024-04-02T13:28:40.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.440+0000 7f260affd700 1 --2- 172.21.15.33:0/2282477888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 0x7f260c07cb90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009e80 tx=0x7f25fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f3d4788036d27d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:40.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.440+0000 7f2609ffb700 1 -- 172.21.15.33:0/2282477888 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25fc017070 con 0x7f260c07e730 2024-04-02T13:28:40.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.440+0000 7f2609ffb700 1 -- 172.21.15.33:0/2282477888 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f25fc00b8c0 con 0x7f260c07e730 2024-04-02T13:28:40.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.440+0000 7f2609ffb700 1 -- 172.21.15.33:0/2282477888 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25fc0045a0 con 0x7f260c07e730 2024-04-02T13:28:40.198 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.441+0000 7f261180a700 1 -- 172.21.15.33:0/2282477888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 msgr2=0x7f260c07cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:40.198 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.441+0000 7f261180a700 1 --2- 172.21.15.33:0/2282477888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 0x7f260c07cb90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009e80 tx=0x7f25fc00b330 comp rx=0 tx=0).stop 2024-04-02T13:28:40.198 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.441+0000 7f261180a700 1 -- 172.21.15.33:0/2282477888 shutdown_connections 2024-04-02T13:28:40.198 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.441+0000 7f261180a700 1 --2- 172.21.15.33:0/2282477888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 0x7f260c07cb90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:40.198 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.441+0000 7f261180a700 1 -- 172.21.15.33:0/2282477888 >> 172.21.15.33:0/2282477888 conn(0x7f260c1036b0 msgr2=0x7f260c105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:40.198 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.441+0000 7f261180a700 1 -- 172.21.15.33:0/2282477888 shutdown_connections 2024-04-02T13:28:40.198 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.441+0000 7f261180a700 1 -- 172.21.15.33:0/2282477888 wait complete. 2024-04-02T13:28:40.198 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.441+0000 7f261180a700 1 Processor -- start 2024-04-02T13:28:40.198 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.442+0000 7f261180a700 1 -- start start 2024-04-02T13:28:40.198 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.442+0000 7f261180a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 0x7f260c19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:40.198 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.442+0000 7f261180a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f260c10e720 con 0x7f260c07e730 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.442+0000 7f260affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 0x7f260c19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.442+0000 7f260affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 0x7f260c19e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58756/0 (socket says 172.21.15.33:58756) 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.442+0000 7f260affd700 1 -- 172.21.15.33:0/2675379965 learned_addr learned my addr 172.21.15.33:0/2675379965 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.442+0000 7f260affd700 1 -- 172.21.15.33:0/2675379965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c19e6e0 con 0x7f260c07e730 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.442+0000 7f260affd700 1 --2- 172.21.15.33:0/2675379965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 0x7f260c19e1a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0072a0 tx=0x7f25fc003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.442+0000 7f2610808700 1 -- 172.21.15.33:0/2675379965 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25fc021020 con 0x7f260c07e730 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.443+0000 7f2610808700 1 -- 172.21.15.33:0/2675379965 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f25fc0048b0 con 0x7f260c07e730 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.443+0000 7f261180a700 1 -- 172.21.15.33:0/2675379965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260c19e970 con 0x7f260c07e730 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.443+0000 7f2610808700 1 -- 172.21.15.33:0/2675379965 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25fc017040 con 0x7f260c07e730 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.443+0000 7f261180a700 1 -- 172.21.15.33:0/2675379965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f260c1a1660 con 0x7f260c07e730 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.443+0000 7f2610808700 1 -- 172.21.15.33:0/2675379965 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f25fc024020 con 0x7f260c07e730 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.444+0000 7f2610808700 1 --2- 172.21.15.33:0/2675379965 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f25f80395d0 0x7f25f803ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.444+0000 7f261180a700 1 -- 172.21.15.33:0/2675379965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f260c063c20 con 0x7f260c07e730 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.444+0000 7f2610808700 1 -- 172.21.15.33:0/2675379965 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f25fc00f050 con 0x7f260c07e730 2024-04-02T13:28:40.199 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.444+0000 7f260a7fc700 1 --2- 172.21.15.33:0/2675379965 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f25f80395d0 0x7f25f803ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:40.200 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.444+0000 7f260a7fc700 1 --2- 172.21.15.33:0/2675379965 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f25f80395d0 0x7f25f803ba90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f25f4006fd0 tx=0x7f25f4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:40.200 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.449+0000 7f2610808700 1 -- 172.21.15.33:0/2675379965 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f25fc02b020 con 0x7f260c07e730 2024-04-02T13:28:40.200 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.656+0000 7f261180a700 1 -- 172.21.15.33:0/2675379965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f260c1a1830 con 0x7f260c07e730 2024-04-02T13:28:40.200 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.657+0000 7f2610808700 1 -- 172.21.15.33:0/2675379965 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f260c1a1830 con 0x7f260c07e730 2024-04-02T13:28:40.200 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.660+0000 7f261180a700 1 -- 172.21.15.33:0/2675379965 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f25f80395d0 msgr2=0x7f25f803ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:40.200 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.660+0000 7f261180a700 1 --2- 172.21.15.33:0/2675379965 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f25f80395d0 0x7f25f803ba90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f25f4006fd0 tx=0x7f25f4006e40 comp rx=0 tx=0).stop 2024-04-02T13:28:40.200 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.660+0000 7f261180a700 1 -- 172.21.15.33:0/2675379965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 msgr2=0x7f260c19e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:40.200 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.660+0000 7f261180a700 1 --2- 172.21.15.33:0/2675379965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 0x7f260c19e1a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0072a0 tx=0x7f25fc003d00 comp rx=0 tx=0).stop 2024-04-02T13:28:40.200 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.660+0000 7f261180a700 1 -- 172.21.15.33:0/2675379965 shutdown_connections 2024-04-02T13:28:40.200 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.660+0000 7f261180a700 1 --2- 172.21.15.33:0/2675379965 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f25f80395d0 0x7f25f803ba90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:40.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.660+0000 7f261180a700 1 --2- 172.21.15.33:0/2675379965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f260c07e730 0x7f260c19e1a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:40.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.660+0000 7f261180a700 1 -- 172.21.15.33:0/2675379965 >> 172.21.15.33:0/2675379965 conn(0x7f260c1036b0 msgr2=0x7f260c104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:40.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.660+0000 7f261180a700 1 -- 172.21.15.33:0/2675379965 shutdown_connections 2024-04-02T13:28:40.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:39.660+0000 7f261180a700 1 -- 172.21.15.33:0/2675379965 wait complete. 2024-04-02T13:28:40.201 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling cephadm module... 2024-04-02T13:28:40.396 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:40 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2675379965' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-02T13:28:41.641 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:41 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2655487604' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-02T13:28:41.641 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:41 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: ignoring --setuser ceph since I am not root 2024-04-02T13:28:41.641 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:41 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: ignoring --setgroup ceph since I am not root 2024-04-02T13:28:41.932 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.592+0000 7f0c74574700 1 Processor -- start 2024-04-02T13:28:41.932 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.592+0000 7f0c74574700 1 -- start start 2024-04-02T13:28:41.932 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.592+0000 7f0c74574700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 0x7f0c6c107440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:41.932 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.592+0000 7f0c74574700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c107a10 con 0x7f0c6c107020 2024-04-02T13:28:41.932 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.593+0000 7f0c72310700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 0x7f0c6c107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:41.932 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.593+0000 7f0c72310700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 0x7f0c6c107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58770/0 (socket says 172.21.15.33:58770) 2024-04-02T13:28:41.932 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.593+0000 7f0c72310700 1 -- 172.21.15.33:0/311451684 learned_addr learned my addr 172.21.15.33:0/311451684 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:41.932 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.593+0000 7f0c72310700 1 -- 172.21.15.33:0/311451684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c108220 con 0x7f0c6c107020 2024-04-02T13:28:41.932 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.593+0000 7f0c72310700 1 --2- 172.21.15.33:0/311451684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 0x7f0c6c107440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0c640098a0 tx=0x7f0c64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc21da0db475842 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.593+0000 7f0c7130e700 1 -- 172.21.15.33:0/311451684 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c64004160 con 0x7f0c6c107020 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.593+0000 7f0c7130e700 1 -- 172.21.15.33:0/311451684 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f0c64004300 con 0x7f0c6c107020 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.593+0000 7f0c7130e700 1 -- 172.21.15.33:0/311451684 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c64011b40 con 0x7f0c6c107020 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.594+0000 7f0c74574700 1 -- 172.21.15.33:0/311451684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 msgr2=0x7f0c6c107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.594+0000 7f0c74574700 1 --2- 172.21.15.33:0/311451684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 0x7f0c6c107440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0c640098a0 tx=0x7f0c64009bb0 comp rx=0 tx=0).stop 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.594+0000 7f0c74574700 1 -- 172.21.15.33:0/311451684 shutdown_connections 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.594+0000 7f0c74574700 1 --2- 172.21.15.33:0/311451684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 0x7f0c6c107440 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.594+0000 7f0c74574700 1 -- 172.21.15.33:0/311451684 >> 172.21.15.33:0/311451684 conn(0x7f0c6c1025e0 msgr2=0x7f0c6c104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.594+0000 7f0c74574700 1 -- 172.21.15.33:0/311451684 shutdown_connections 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.594+0000 7f0c74574700 1 -- 172.21.15.33:0/311451684 wait complete. 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.595+0000 7f0c74574700 1 Processor -- start 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.595+0000 7f0c74574700 1 -- start start 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.595+0000 7f0c74574700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 0x7f0c6c19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.595+0000 7f0c74574700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c10b5d0 con 0x7f0c6c107020 2024-04-02T13:28:41.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.595+0000 7f0c72310700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 0x7f0c6c19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:41.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.595+0000 7f0c72310700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 0x7f0c6c19c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58782/0 (socket says 172.21.15.33:58782) 2024-04-02T13:28:41.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.595+0000 7f0c72310700 1 -- 172.21.15.33:0/2655487604 learned_addr learned my addr 172.21.15.33:0/2655487604 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:41.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.595+0000 7f0c72310700 1 -- 172.21.15.33:0/2655487604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c19c5a0 con 0x7f0c6c107020 2024-04-02T13:28:41.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.595+0000 7f0c72310700 1 --2- 172.21.15.33:0/2655487604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 0x7f0c6c19c060 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0c64011670 tx=0x7f0c64011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:41.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.595+0000 7f0c6b7fe700 1 -- 172.21.15.33:0/2655487604 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c640038b0 con 0x7f0c6c107020 2024-04-02T13:28:41.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.595+0000 7f0c6b7fe700 1 -- 172.21.15.33:0/2655487604 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f0c6401a430 con 0x7f0c6c107020 2024-04-02T13:28:41.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.595+0000 7f0c74574700 1 -- 172.21.15.33:0/2655487604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6c19c830 con 0x7f0c6c107020 2024-04-02T13:28:41.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.596+0000 7f0c6b7fe700 1 -- 172.21.15.33:0/2655487604 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c64019550 con 0x7f0c6c107020 2024-04-02T13:28:41.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.596+0000 7f0c74574700 1 -- 172.21.15.33:0/2655487604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c6c19f520 con 0x7f0c6c107020 2024-04-02T13:28:41.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.596+0000 7f0c6b7fe700 1 -- 172.21.15.33:0/2655487604 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f0c64019860 con 0x7f0c6c107020 2024-04-02T13:28:41.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.596+0000 7f0c6b7fe700 1 --2- 172.21.15.33:0/2655487604 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f0c54039620 0x7f0c5403bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:41.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.596+0000 7f0c74574700 1 -- 172.21.15.33:0/2655487604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6c063c20 con 0x7f0c6c107020 2024-04-02T13:28:41.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.597+0000 7f0c71b0f700 1 --2- 172.21.15.33:0/2655487604 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f0c54039620 0x7f0c5403bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.597+0000 7f0c6b7fe700 1 -- 172.21.15.33:0/2655487604 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0c6404da10 con 0x7f0c6c107020 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.597+0000 7f0c71b0f700 1 --2- 172.21.15.33:0/2655487604 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f0c54039620 0x7f0c5403bae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0c60006fd0 tx=0x7f0c60006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.602+0000 7f0c6b7fe700 1 -- 172.21.15.33:0/2655487604 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0c64022590 con 0x7f0c6c107020 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:40.868+0000 7f0c74574700 1 -- 172.21.15.33:0/2655487604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f0c6c19fba0 con 0x7f0c6c107020 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:41.335+0000 7f0c6b7fe700 1 -- 172.21.15.33:0/2655487604 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f0c64022340 con 0x7f0c6c107020 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:41.337+0000 7f0c6b7fe700 1 -- 172.21.15.33:0/2655487604 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f0c6402d960 con 0x7f0c6c107020 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:41.339+0000 7f0c74574700 1 -- 172.21.15.33:0/2655487604 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f0c54039620 msgr2=0x7f0c5403bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:41.339+0000 7f0c74574700 1 --2- 172.21.15.33:0/2655487604 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f0c54039620 0x7f0c5403bae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0c60006fd0 tx=0x7f0c60006e40 comp rx=0 tx=0).stop 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:41.339+0000 7f0c74574700 1 -- 172.21.15.33:0/2655487604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 msgr2=0x7f0c6c19c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:41.339+0000 7f0c74574700 1 --2- 172.21.15.33:0/2655487604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 0x7f0c6c19c060 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0c64011670 tx=0x7f0c64011b50 comp rx=0 tx=0).stop 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:41.339+0000 7f0c74574700 1 -- 172.21.15.33:0/2655487604 shutdown_connections 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:41.339+0000 7f0c74574700 1 --2- 172.21.15.33:0/2655487604 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f0c54039620 0x7f0c5403bae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:41.339+0000 7f0c74574700 1 --2- 172.21.15.33:0/2655487604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c107020 0x7f0c6c19c060 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:41.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:41.339+0000 7f0c74574700 1 -- 172.21.15.33:0/2655487604 >> 172.21.15.33:0/2655487604 conn(0x7f0c6c1025e0 msgr2=0x7f0c6c103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:41.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:41.339+0000 7f0c74574700 1 -- 172.21.15.33:0/2655487604 shutdown_connections 2024-04-02T13:28:41.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:41.339+0000 7f0c74574700 1 -- 172.21.15.33:0/2655487604 wait complete. 2024-04-02T13:28:42.174 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:41 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:41.917+0000 7f34e10fe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:28:42.174 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:42 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:42.016+0000 7f34e10fe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:28:42.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:42 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2655487604' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-02T13:28:42.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:42 smithi033 ceph-mon[32390]: mgrmap e5: x(active, since 6s) 2024-04-02T13:28:42.941 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:42 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:42.693+0000 7f34e10fe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:28:42.941 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:42 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:42.789+0000 7f34e10fe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:28:42.941 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:42 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:42.936+0000 7f34e10fe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:28:43.242 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:43 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:43.036+0000 7f34e10fe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:28:43.243 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:43 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:43.122+0000 7f34e10fe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:28:43.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:28:43.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-02T13:28:43.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T13:28:43.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-02T13:28:43.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T13:28:43.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:28:43.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.368+0000 7f1facda1700 1 Processor -- start 2024-04-02T13:28:43.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.368+0000 7f1facda1700 1 -- start start 2024-04-02T13:28:43.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.368+0000 7f1facda1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 0x7f1fa810ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:43.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.368+0000 7f1facda1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa807c680 con 0x7f1fa8108a50 2024-04-02T13:28:43.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.368+0000 7f1fa77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 0x7f1fa810ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:43.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.368+0000 7f1fa77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 0x7f1fa810ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58800/0 (socket says 172.21.15.33:58800) 2024-04-02T13:28:43.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.368+0000 7f1fa77fe700 1 -- 172.21.15.33:0/2958934308 learned_addr learned my addr 172.21.15.33:0/2958934308 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:43.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.368+0000 7f1fa77fe700 1 -- 172.21.15.33:0/2958934308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa810b380 con 0x7f1fa8108a50 2024-04-02T13:28:43.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.369+0000 7f1fa77fe700 1 --2- 172.21.15.33:0/2958934308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 0x7f1fa810ae40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1f980098a0 tx=0x7f1f98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182de9f1fbfcd068 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:43.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.369+0000 7f1fa67fc700 1 -- 172.21.15.33:0/2958934308 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f98004160 con 0x7f1fa8108a50 2024-04-02T13:28:43.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.369+0000 7f1fa67fc700 1 -- 172.21.15.33:0/2958934308 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f1f98004300 con 0x7f1fa8108a50 2024-04-02T13:28:43.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.369+0000 7f1fa67fc700 1 -- 172.21.15.33:0/2958934308 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f98011b40 con 0x7f1fa8108a50 2024-04-02T13:28:43.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.370+0000 7f1facda1700 1 -- 172.21.15.33:0/2958934308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 msgr2=0x7f1fa810ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:43.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.370+0000 7f1facda1700 1 --2- 172.21.15.33:0/2958934308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 0x7f1fa810ae40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1f980098a0 tx=0x7f1f98009bb0 comp rx=0 tx=0).stop 2024-04-02T13:28:43.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.370+0000 7f1facda1700 1 -- 172.21.15.33:0/2958934308 shutdown_connections 2024-04-02T13:28:43.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.370+0000 7f1facda1700 1 --2- 172.21.15.33:0/2958934308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 0x7f1fa810ae40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.370+0000 7f1facda1700 1 -- 172.21.15.33:0/2958934308 >> 172.21.15.33:0/2958934308 conn(0x7f1fa8102400 msgr2=0x7f1fa8104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.370+0000 7f1facda1700 1 -- 172.21.15.33:0/2958934308 shutdown_connections 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.370+0000 7f1facda1700 1 -- 172.21.15.33:0/2958934308 wait complete. 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.370+0000 7f1facda1700 1 Processor -- start 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.370+0000 7f1facda1700 1 -- start start 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.371+0000 7f1facda1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 0x7f1fa819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.371+0000 7f1facda1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa810c2f0 con 0x7f1fa8108a50 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.371+0000 7f1fa77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 0x7f1fa819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.371+0000 7f1fa77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 0x7f1fa819bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58804/0 (socket says 172.21.15.33:58804) 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.371+0000 7f1fa77fe700 1 -- 172.21.15.33:0/1276087324 learned_addr learned my addr 172.21.15.33:0/1276087324 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.371+0000 7f1fa77fe700 1 -- 172.21.15.33:0/1276087324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa819c2e0 con 0x7f1fa8108a50 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.371+0000 7f1fa77fe700 1 --2- 172.21.15.33:0/1276087324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 0x7f1fa819bda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f98011840 tx=0x7f1f980119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.371+0000 7f1fa4ff9700 1 -- 172.21.15.33:0/1276087324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f980039e0 con 0x7f1fa8108a50 2024-04-02T13:28:43.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.371+0000 7f1fa4ff9700 1 -- 172.21.15.33:0/1276087324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f1f98020070 con 0x7f1fa8108a50 2024-04-02T13:28:43.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.371+0000 7f1facda1700 1 -- 172.21.15.33:0/1276087324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa819c570 con 0x7f1fa8108a50 2024-04-02T13:28:43.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.371+0000 7f1fa4ff9700 1 -- 172.21.15.33:0/1276087324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f980193c0 con 0x7f1fa8108a50 2024-04-02T13:28:43.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.371+0000 7f1facda1700 1 -- 172.21.15.33:0/1276087324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa819f260 con 0x7f1fa8108a50 2024-04-02T13:28:43.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.372+0000 7f1fa4ff9700 1 -- 172.21.15.33:0/1276087324 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f1f980196c0 con 0x7f1fa8108a50 2024-04-02T13:28:43.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.373+0000 7f1facda1700 1 -- 172.21.15.33:0/1276087324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa8063c20 con 0x7f1fa8108a50 2024-04-02T13:28:43.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.373+0000 7f1fa4ff9700 1 --2- 172.21.15.33:0/1276087324 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f1f90039670 0x7f1f9003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:43.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.373+0000 7f1fa6ffd700 1 -- 172.21.15.33:0/1276087324 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f1f90039670 msgr2=0x7f1f9003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2869265288 2024-04-02T13:28:43.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.373+0000 7f1fa4ff9700 1 -- 172.21.15.33:0/1276087324 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1f9804d7b0 con 0x7f1fa8108a50 2024-04-02T13:28:43.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.373+0000 7f1fa6ffd700 1 --2- 172.21.15.33:0/1276087324 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f1f90039670 0x7f1f9003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:28:43.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.378+0000 7f1fa4ff9700 1 -- 172.21.15.33:0/1276087324 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1f9804d1d0 con 0x7f1fa8108a50 2024-04-02T13:28:43.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.574+0000 7f1fa6ffd700 1 -- 172.21.15.33:0/1276087324 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f1f90039670 msgr2=0x7f1f9003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2869265288 2024-04-02T13:28:43.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.574+0000 7f1fa6ffd700 1 --2- 172.21.15.33:0/1276087324 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f1f90039670 0x7f1f9003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:28:43.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.645+0000 7f1facda1700 1 -- 172.21.15.33:0/1276087324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1fa819f430 con 0x7f1fa8108a50 2024-04-02T13:28:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.645+0000 7f1fa4ff9700 1 -- 172.21.15.33:0/1276087324 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f1fa819f430 con 0x7f1fa8108a50 2024-04-02T13:28:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.648+0000 7f1facda1700 1 -- 172.21.15.33:0/1276087324 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f1f90039670 msgr2=0x7f1f9003bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:28:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.648+0000 7f1facda1700 1 --2- 172.21.15.33:0/1276087324 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f1f90039670 0x7f1f9003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.648+0000 7f1facda1700 1 -- 172.21.15.33:0/1276087324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 msgr2=0x7f1fa819bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.648+0000 7f1facda1700 1 --2- 172.21.15.33:0/1276087324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 0x7f1fa819bda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f98011840 tx=0x7f1f980119d0 comp rx=0 tx=0).stop 2024-04-02T13:28:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.648+0000 7f1facda1700 1 -- 172.21.15.33:0/1276087324 shutdown_connections 2024-04-02T13:28:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.648+0000 7f1facda1700 1 --2- 172.21.15.33:0/1276087324 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7f1f90039670 0x7f1f9003bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.648+0000 7f1facda1700 1 --2- 172.21.15.33:0/1276087324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fa8108a50 0x7f1fa819bda0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.648+0000 7f1facda1700 1 -- 172.21.15.33:0/1276087324 >> 172.21.15.33:0/1276087324 conn(0x7f1fa8102400 msgr2=0x7f1fa8102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.648+0000 7f1facda1700 1 -- 172.21.15.33:0/1276087324 shutdown_connections 2024-04-02T13:28:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:42.648+0000 7f1facda1700 1 -- 172.21.15.33:0/1276087324 wait complete. 2024-04-02T13:28:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-04-02T13:28:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 5... 2024-04-02T13:28:43.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:43 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1276087324' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T13:28:43.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:43 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:43.404+0000 7f34e10fe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:28:43.875 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:43 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:43.519+0000 7f34e10fe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:28:43.875 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:43 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:43.610+0000 7f34e10fe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:28:43.875 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:43 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:43.710+0000 7f34e10fe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:28:44.254 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:43 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:43.891+0000 7f34e10fe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:28:45.254 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:44 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:44.881+0000 7f34e10fe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:28:45.254 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:45 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:45.016+0000 7f34e10fe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:28:45.659 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:45 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:45.360+0000 7f34e10fe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:28:45.659 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:45 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:45.453+0000 7f34e10fe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:28:45.659 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:45 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:45.656+0000 7f34e10fe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:28:46.004 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:45 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:45.847+0000 7f34e10fe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:28:46.408 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:46 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:46.141+0000 7f34e10fe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:28:46.408 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:46 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:46.404+0000 7f34e10fe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:28:47.004 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:46 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:46.657+0000 7f34e10fe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:28:47.754 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:47.509+0000 7f34e10fe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:28:48.426 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:48.093+0000 7f34e10fe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:28:48.426 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:48.196+0000 7f34e10fe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:28:48.426 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:28:48.423+0000 7f34e10fe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:28:48.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: Active manager daemon x restarted 2024-04-02T13:28:48.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: Activating manager daemon x 2024-04-02T13:28:48.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: osdmap e2: 0 total, 0 up, 0 in 2024-04-02T13:28:48.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: mgrmap e6: x(active, starting, since 0.00338429s) 2024-04-02T13:28:48.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:28:48.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:28:48.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:28:48.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:28:48.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:28:48.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: Manager daemon x is now available 2024-04-02T13:28:48.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' 2024-04-02T13:28:48.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' 2024-04-02T13:28:48.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:48.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:48.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:48 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:28:49.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:49 smithi033 ceph-mon[32390]: Found migration_current of "None". Setting to last migration. 2024-04-02T13:28:49.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:49 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:28:49.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:49 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' 2024-04-02T13:28:49.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:49 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' 2024-04-02T13:28:49.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:49 smithi033 ceph-mon[32390]: mgrmap e7: x(active, since 1.00533s) 2024-04-02T13:28:50.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:28:50.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-02T13:28:50.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T13:28:50.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:28:50.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.666+0000 7ffb2db41700 1 Processor -- start 2024-04-02T13:28:50.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.666+0000 7ffb2db41700 1 -- start start 2024-04-02T13:28:50.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.666+0000 7ffb2db41700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 0x7ffb28107440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:50.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.666+0000 7ffb2db41700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffb28107a10 con 0x7ffb28107020 2024-04-02T13:28:50.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.666+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 0x7ffb28107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:50.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.666+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 0x7ffb28107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42858/0 (socket says 172.21.15.33:42858) 2024-04-02T13:28:50.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.666+0000 7ffb277fe700 1 -- 172.21.15.33:0/2462910278 learned_addr learned my addr 172.21.15.33:0/2462910278 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:50.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.667+0000 7ffb277fe700 1 -- 172.21.15.33:0/2462910278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb28108220 con 0x7ffb28107020 2024-04-02T13:28:50.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.667+0000 7ffb277fe700 1 --2- 172.21.15.33:0/2462910278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 0x7ffb28107440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0098a0 tx=0x7ffb1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc62fc1b9bd9a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:50.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.667+0000 7ffb267fc700 1 -- 172.21.15.33:0/2462910278 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb1c004160 con 0x7ffb28107020 2024-04-02T13:28:50.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.667+0000 7ffb267fc700 1 -- 172.21.15.33:0/2462910278 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7ffb1c004300 con 0x7ffb28107020 2024-04-02T13:28:50.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.667+0000 7ffb267fc700 1 -- 172.21.15.33:0/2462910278 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb1c011b40 con 0x7ffb28107020 2024-04-02T13:28:50.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.668+0000 7ffb2db41700 1 -- 172.21.15.33:0/2462910278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 msgr2=0x7ffb28107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:50.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.668+0000 7ffb2db41700 1 --2- 172.21.15.33:0/2462910278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 0x7ffb28107440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0098a0 tx=0x7ffb1c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:28:50.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.668+0000 7ffb2db41700 1 -- 172.21.15.33:0/2462910278 shutdown_connections 2024-04-02T13:28:50.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.668+0000 7ffb2db41700 1 --2- 172.21.15.33:0/2462910278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 0x7ffb28107440 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:50.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.668+0000 7ffb2db41700 1 -- 172.21.15.33:0/2462910278 >> 172.21.15.33:0/2462910278 conn(0x7ffb281025e0 msgr2=0x7ffb28104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:50.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.668+0000 7ffb2db41700 1 -- 172.21.15.33:0/2462910278 shutdown_connections 2024-04-02T13:28:50.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.668+0000 7ffb2db41700 1 -- 172.21.15.33:0/2462910278 wait complete. 2024-04-02T13:28:50.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.669+0000 7ffb2db41700 1 Processor -- start 2024-04-02T13:28:50.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.669+0000 7ffb2db41700 1 -- start start 2024-04-02T13:28:50.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.669+0000 7ffb2db41700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 0x7ffb2819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:50.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.669+0000 7ffb2db41700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2810b5d0 con 0x7ffb28107020 2024-04-02T13:28:50.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.669+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 0x7ffb2819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:50.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.669+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 0x7ffb2819bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42872/0 (socket says 172.21.15.33:42872) 2024-04-02T13:28:50.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.669+0000 7ffb277fe700 1 -- 172.21.15.33:0/3890314858 learned_addr learned my addr 172.21.15.33:0/3890314858 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:50.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.670+0000 7ffb277fe700 1 -- 172.21.15.33:0/3890314858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2819c510 con 0x7ffb28107020 2024-04-02T13:28:50.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.670+0000 7ffb277fe700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 0x7ffb2819bfd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c006af0 tx=0x7ffb1c011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:50.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.670+0000 7ffb24ff9700 1 -- 172.21.15.33:0/3890314858 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb1c003970 con 0x7ffb28107020 2024-04-02T13:28:50.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.670+0000 7ffb2db41700 1 -- 172.21.15.33:0/3890314858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2819c7a0 con 0x7ffb28107020 2024-04-02T13:28:50.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.670+0000 7ffb2db41700 1 -- 172.21.15.33:0/3890314858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2819f490 con 0x7ffb28107020 2024-04-02T13:28:50.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.670+0000 7ffb24ff9700 1 -- 172.21.15.33:0/3890314858 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7ffb1c003b10 con 0x7ffb28107020 2024-04-02T13:28:50.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.670+0000 7ffb24ff9700 1 -- 172.21.15.33:0/3890314858 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb1c0196b0 con 0x7ffb28107020 2024-04-02T13:28:50.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.671+0000 7ffb24ff9700 1 -- 172.21.15.33:0/3890314858 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ffb1c01a430 con 0x7ffb28107020 2024-04-02T13:28:50.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.671+0000 7ffb24ff9700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 0x7ffb1003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:50.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.671+0000 7ffb24ff9700 1 -- 172.21.15.33:0/3890314858 --> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffb1003c1f0 con 0x7ffb10039620 2024-04-02T13:28:50.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.671+0000 7ffb24ff9700 1 -- 172.21.15.33:0/3890314858 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffb1c04db50 con 0x7ffb28107020 2024-04-02T13:28:50.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.671+0000 7ffb26ffd700 1 -- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 msgr2=0x7ffb1003bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2869265288 2024-04-02T13:28:50.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.671+0000 7ffb26ffd700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 0x7ffb1003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:28:50.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.872+0000 7ffb26ffd700 1 -- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 msgr2=0x7ffb1003bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2869265288 2024-04-02T13:28:50.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:43.872+0000 7ffb26ffd700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 0x7ffb1003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:28:50.099 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:44.273+0000 7ffb26ffd700 1 -- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 msgr2=0x7ffb1003bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2869265288 2024-04-02T13:28:50.099 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:44.273+0000 7ffb26ffd700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 0x7ffb1003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T13:28:50.099 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:45.074+0000 7ffb26ffd700 1 -- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 msgr2=0x7ffb1003bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2869265288 2024-04-02T13:28:50.099 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:45.074+0000 7ffb26ffd700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 0x7ffb1003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T13:28:50.099 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:46.675+0000 7ffb26ffd700 1 -- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 msgr2=0x7ffb1003bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2869265288 2024-04-02T13:28:50.099 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:46.675+0000 7ffb26ffd700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 0x7ffb1003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T13:28:50.099 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:48.426+0000 7ffb24ff9700 1 -- 172.21.15.33:0/3890314858 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7ffb1c019960 con 0x7ffb28107020 2024-04-02T13:28:50.099 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:48.426+0000 7ffb24ff9700 1 -- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 msgr2=0x7ffb1003bae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:28:50.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:48.426+0000 7ffb24ff9700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 0x7ffb1003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:50.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.427+0000 7ffb24ff9700 1 -- 172.21.15.33:0/3890314858 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7ffb1c04f960 con 0x7ffb28107020 2024-04-02T13:28:50.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.427+0000 7ffb24ff9700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7ffb1003cf10 0x7ffb1003f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:50.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.427+0000 7ffb24ff9700 1 -- 172.21.15.33:0/3890314858 --> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffb1003c1f0 con 0x7ffb1003cf10 2024-04-02T13:28:50.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.427+0000 7ffb26ffd700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7ffb1003cf10 0x7ffb1003f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:50.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.428+0000 7ffb26ffd700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7ffb1003cf10 0x7ffb1003f300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffb18003a10 tx=0x7ffb18009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:50.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.429+0000 7ffb24ff9700 1 -- 172.21.15.33:0/3890314858 <== mgr.14120 v2:172.21.15.33:6800/2666001000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ffb1003c1f0 con 0x7ffb1003cf10 2024-04-02T13:28:50.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.435+0000 7ffb2db41700 1 -- 172.21.15.33:0/3890314858 --> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ffb28063c20 con 0x7ffb1003cf10 2024-04-02T13:28:50.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.436+0000 7ffb24ff9700 1 -- 172.21.15.33:0/3890314858 <== mgr.14120 v2:172.21.15.33:6800/2666001000 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ffb28063c20 con 0x7ffb1003cf10 2024-04-02T13:28:50.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.436+0000 7ffb2db41700 1 -- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7ffb1003cf10 msgr2=0x7ffb1003f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:50.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.436+0000 7ffb2db41700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7ffb1003cf10 0x7ffb1003f300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffb18003a10 tx=0x7ffb18009290 comp rx=0 tx=0).stop 2024-04-02T13:28:50.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.436+0000 7ffb2db41700 1 -- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 msgr2=0x7ffb2819bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:50.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.436+0000 7ffb2db41700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 0x7ffb2819bfd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c006af0 tx=0x7ffb1c011c00 comp rx=0 tx=0).stop 2024-04-02T13:28:50.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.436+0000 7ffb2db41700 1 -- 172.21.15.33:0/3890314858 shutdown_connections 2024-04-02T13:28:50.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.436+0000 7ffb2db41700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7ffb1003cf10 0x7ffb1003f300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:50.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.436+0000 7ffb2db41700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb28107020 0x7ffb2819bfd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:50.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.436+0000 7ffb2db41700 1 --2- 172.21.15.33:0/3890314858 >> [v2:172.21.15.33:6800/2869265288,v1:172.21.15.33:6801/2869265288] conn(0x7ffb10039620 0x7ffb1003bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:50.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.436+0000 7ffb2db41700 1 -- 172.21.15.33:0/3890314858 >> 172.21.15.33:0/3890314858 conn(0x7ffb281025e0 msgr2=0x7ffb28103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:50.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.436+0000 7ffb2db41700 1 -- 172.21.15.33:0/3890314858 shutdown_connections 2024-04-02T13:28:50.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:49.436+0000 7ffb2db41700 1 -- 172.21.15.33:0/3890314858 wait complete. 2024-04-02T13:28:50.101 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 5 is available 2024-04-02T13:28:50.101 INFO:teuthology.orchestra.run.smithi033.stdout:Setting orchestrator backend to cephadm... 2024-04-02T13:28:50.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:50 smithi033 ceph-mon[32390]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-02T13:28:50.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:50 smithi033 ceph-mon[32390]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-02T13:28:51.363 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.517+0000 7f0dbd5da700 1 Processor -- start 2024-04-02T13:28:51.363 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.517+0000 7f0dbd5da700 1 -- start start 2024-04-02T13:28:51.363 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.517+0000 7f0dbd5da700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 0x7f0db810b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:51.363 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.517+0000 7f0dbd5da700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0db807c6c0 con 0x7f0db8108c10 2024-04-02T13:28:51.363 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.518+0000 7f0db6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 0x7f0db810b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:51.363 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.518+0000 7f0db6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 0x7f0db810b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42968/0 (socket says 172.21.15.33:42968) 2024-04-02T13:28:51.363 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.518+0000 7f0db6ffd700 1 -- 172.21.15.33:0/2579362909 learned_addr learned my addr 172.21.15.33:0/2579362909 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:51.363 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.518+0000 7f0db6ffd700 1 -- 172.21.15.33:0/2579362909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db810b540 con 0x7f0db8108c10 2024-04-02T13:28:51.363 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.518+0000 7f0db6ffd700 1 --2- 172.21.15.33:0/2579362909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 0x7f0db810b000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0da80098a0 tx=0x7f0da8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4302b59ac1b7d97e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.518+0000 7f0db5ffb700 1 -- 172.21.15.33:0/2579362909 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0da8004160 con 0x7f0db8108c10 2024-04-02T13:28:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.518+0000 7f0db5ffb700 1 -- 172.21.15.33:0/2579362909 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f0da8004300 con 0x7f0db8108c10 2024-04-02T13:28:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.518+0000 7f0db5ffb700 1 -- 172.21.15.33:0/2579362909 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0da8011b40 con 0x7f0db8108c10 2024-04-02T13:28:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.519+0000 7f0dbd5da700 1 -- 172.21.15.33:0/2579362909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 msgr2=0x7f0db810b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.519+0000 7f0dbd5da700 1 --2- 172.21.15.33:0/2579362909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 0x7f0db810b000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0da80098a0 tx=0x7f0da8009bb0 comp rx=0 tx=0).stop 2024-04-02T13:28:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.519+0000 7f0dbd5da700 1 -- 172.21.15.33:0/2579362909 shutdown_connections 2024-04-02T13:28:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.519+0000 7f0dbd5da700 1 --2- 172.21.15.33:0/2579362909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 0x7f0db810b000 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.519+0000 7f0dbd5da700 1 -- 172.21.15.33:0/2579362909 >> 172.21.15.33:0/2579362909 conn(0x7f0db81025e0 msgr2=0x7f0db8104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.519+0000 7f0dbd5da700 1 -- 172.21.15.33:0/2579362909 shutdown_connections 2024-04-02T13:28:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.519+0000 7f0dbd5da700 1 -- 172.21.15.33:0/2579362909 wait complete. 2024-04-02T13:28:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.520+0000 7f0dbd5da700 1 Processor -- start 2024-04-02T13:28:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.520+0000 7f0dbd5da700 1 -- start start 2024-04-02T13:28:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.520+0000 7f0dbd5da700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 0x7f0db819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.520+0000 7f0dbd5da700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0db810c500 con 0x7f0db8108c10 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.520+0000 7f0db6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 0x7f0db819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.520+0000 7f0db6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 0x7f0db819c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42970/0 (socket says 172.21.15.33:42970) 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.520+0000 7f0db6ffd700 1 -- 172.21.15.33:0/1291077105 learned_addr learned my addr 172.21.15.33:0/1291077105 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.520+0000 7f0db6ffd700 1 -- 172.21.15.33:0/1291077105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db819c550 con 0x7f0db8108c10 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.521+0000 7f0db6ffd700 1 --2- 172.21.15.33:0/1291077105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 0x7f0db819c010 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0da8011670 tx=0x7f0da8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.521+0000 7f0d9ffff700 1 -- 172.21.15.33:0/1291077105 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0da80038b0 con 0x7f0db8108c10 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.521+0000 7f0d9ffff700 1 -- 172.21.15.33:0/1291077105 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f0da801a430 con 0x7f0db8108c10 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.521+0000 7f0dbd5da700 1 -- 172.21.15.33:0/1291077105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db819c7e0 con 0x7f0db8108c10 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.521+0000 7f0d9ffff700 1 -- 172.21.15.33:0/1291077105 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0da8019550 con 0x7f0db8108c10 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.521+0000 7f0dbd5da700 1 -- 172.21.15.33:0/1291077105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db819f4d0 con 0x7f0db8108c10 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.522+0000 7f0d9ffff700 1 -- 172.21.15.33:0/1291077105 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0da8019850 con 0x7f0db8108c10 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.522+0000 7f0d9ffff700 1 --2- 172.21.15.33:0/1291077105 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f0da0039620 0x7f0da003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.522+0000 7f0dbd5da700 1 -- 172.21.15.33:0/1291077105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db8063c20 con 0x7f0db8108c10 2024-04-02T13:28:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.522+0000 7f0d9ffff700 1 -- 172.21.15.33:0/1291077105 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0da804da20 con 0x7f0db8108c10 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.522+0000 7f0db67fc700 1 --2- 172.21.15.33:0/1291077105 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f0da0039620 0x7f0da003bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.522+0000 7f0db67fc700 1 --2- 172.21.15.33:0/1291077105 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f0da0039620 0x7f0da003bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0dac006fd0 tx=0x7f0dac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.532+0000 7f0d9ffff700 1 -- 172.21.15.33:0/1291077105 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0da800bc70 con 0x7f0db8108c10 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.756+0000 7f0dbd5da700 1 -- 172.21.15.33:0/1291077105 --> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f0db81036e0 con 0x7f0da0039620 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.761+0000 7f0d9ffff700 1 -- 172.21.15.33:0/1291077105 <== mgr.14120 v2:172.21.15.33:6800/2666001000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0db81036e0 con 0x7f0da0039620 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.763+0000 7f0dbd5da700 1 -- 172.21.15.33:0/1291077105 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f0da0039620 msgr2=0x7f0da003bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.763+0000 7f0dbd5da700 1 --2- 172.21.15.33:0/1291077105 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f0da0039620 0x7f0da003bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0dac006fd0 tx=0x7f0dac006e40 comp rx=0 tx=0).stop 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.763+0000 7f0dbd5da700 1 -- 172.21.15.33:0/1291077105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 msgr2=0x7f0db819c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.763+0000 7f0dbd5da700 1 --2- 172.21.15.33:0/1291077105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 0x7f0db819c010 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0da8011670 tx=0x7f0da8011b50 comp rx=0 tx=0).stop 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.763+0000 7f0dbd5da700 1 -- 172.21.15.33:0/1291077105 shutdown_connections 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.763+0000 7f0dbd5da700 1 --2- 172.21.15.33:0/1291077105 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f0da0039620 0x7f0da003bae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.763+0000 7f0dbd5da700 1 --2- 172.21.15.33:0/1291077105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8108c10 0x7f0db819c010 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.763+0000 7f0dbd5da700 1 -- 172.21.15.33:0/1291077105 >> 172.21.15.33:0/1291077105 conn(0x7f0db81025e0 msgr2=0x7f0db8102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.763+0000 7f0dbd5da700 1 -- 172.21.15.33:0/1291077105 shutdown_connections 2024-04-02T13:28:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:50.764+0000 7f0dbd5da700 1 -- 172.21.15.33:0/1291077105 wait complete. 2024-04-02T13:28:51.682 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:51 smithi033 ceph-mon[32390]: mgrmap e8: x(active, since 2s) 2024-04-02T13:28:51.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:51 smithi033 ceph-mon[32390]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:28:51.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:51 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' 2024-04-02T13:28:51.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:51 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:51.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:51 smithi033 ceph-mon[32390]: [02/Apr/2024:13:28:51] ENGINE Bus STARTING 2024-04-02T13:28:51.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:51 smithi033 ceph-mon[32390]: [02/Apr/2024:13:28:51] ENGINE Serving on https://172.21.15.33:7150 2024-04-02T13:28:51.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:51 smithi033 ceph-mon[32390]: [02/Apr/2024:13:28:51] ENGINE Serving on http://172.21.15.33:8765 2024-04-02T13:28:51.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:51 smithi033 ceph-mon[32390]: [02/Apr/2024:13:28:51] ENGINE Bus STARTED 2024-04-02T13:28:51.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:51 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:52.586 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:52 smithi033 ceph-mon[32390]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:28:52.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-02T13:28:52.594 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.780+0000 7f04dfcf3700 1 Processor -- start 2024-04-02T13:28:52.594 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.780+0000 7f04dfcf3700 1 -- start start 2024-04-02T13:28:52.594 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.780+0000 7f04dfcf3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 0x7f04d8108530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.780+0000 7f04dfcf3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8108b00 con 0x7f04d8108110 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.780+0000 7f04dda8f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 0x7f04d8108530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.780+0000 7f04dda8f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 0x7f04d8108530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42982/0 (socket says 172.21.15.33:42982) 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.780+0000 7f04dda8f700 1 -- 172.21.15.33:0/1134968752 learned_addr learned my addr 172.21.15.33:0/1134968752 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.781+0000 7f04dda8f700 1 -- 172.21.15.33:0/1134968752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d8109310 con 0x7f04d8108110 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.781+0000 7f04dda8f700 1 --2- 172.21.15.33:0/1134968752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 0x7f04d8108530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009e80 tx=0x7f04d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f3f7fbdc6d6207 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.781+0000 7f04dca8d700 1 -- 172.21.15.33:0/1134968752 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d0017070 con 0x7f04d8108110 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.781+0000 7f04dca8d700 1 -- 172.21.15.33:0/1134968752 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f04d000b8c0 con 0x7f04d8108110 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.781+0000 7f04dca8d700 1 -- 172.21.15.33:0/1134968752 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d00045a0 con 0x7f04d8108110 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.782+0000 7f04dfcf3700 1 -- 172.21.15.33:0/1134968752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 msgr2=0x7f04d8108530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.782+0000 7f04dfcf3700 1 --2- 172.21.15.33:0/1134968752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 0x7f04d8108530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009e80 tx=0x7f04d000b330 comp rx=0 tx=0).stop 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.782+0000 7f04dfcf3700 1 -- 172.21.15.33:0/1134968752 shutdown_connections 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.782+0000 7f04dfcf3700 1 --2- 172.21.15.33:0/1134968752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 0x7f04d8108530 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:52.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.782+0000 7f04dfcf3700 1 -- 172.21.15.33:0/1134968752 >> 172.21.15.33:0/1134968752 conn(0x7f04d81036f0 msgr2=0x7f04d8105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.782+0000 7f04dfcf3700 1 -- 172.21.15.33:0/1134968752 shutdown_connections 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.782+0000 7f04dfcf3700 1 -- 172.21.15.33:0/1134968752 wait complete. 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.782+0000 7f04dfcf3700 1 Processor -- start 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.782+0000 7f04dfcf3700 1 -- start start 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.783+0000 7f04dfcf3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 0x7f04d819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.783+0000 7f04dfcf3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04d810c6c0 con 0x7f04d8108110 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.783+0000 7f04dda8f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 0x7f04d819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.783+0000 7f04dda8f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 0x7f04d819bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42996/0 (socket says 172.21.15.33:42996) 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.783+0000 7f04dda8f700 1 -- 172.21.15.33:0/763589326 learned_addr learned my addr 172.21.15.33:0/763589326 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.783+0000 7f04dda8f700 1 -- 172.21.15.33:0/763589326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d819c4d0 con 0x7f04d8108110 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.783+0000 7f04dda8f700 1 --2- 172.21.15.33:0/763589326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 0x7f04d819bf90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f04d00072a0 tx=0x7f04d0003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.783+0000 7f04d6ffd700 1 -- 172.21.15.33:0/763589326 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d0021020 con 0x7f04d8108110 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.783+0000 7f04d6ffd700 1 -- 172.21.15.33:0/763589326 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f04d00048b0 con 0x7f04d8108110 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.783+0000 7f04dfcf3700 1 -- 172.21.15.33:0/763589326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d819c760 con 0x7f04d8108110 2024-04-02T13:28:52.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.783+0000 7f04d6ffd700 1 -- 172.21.15.33:0/763589326 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d0017040 con 0x7f04d8108110 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.783+0000 7f04dfcf3700 1 -- 172.21.15.33:0/763589326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d819f450 con 0x7f04d8108110 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.784+0000 7f04d6ffd700 1 -- 172.21.15.33:0/763589326 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f04d0024020 con 0x7f04d8108110 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.784+0000 7f04dfcf3700 1 -- 172.21.15.33:0/763589326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d8063c20 con 0x7f04d8108110 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.784+0000 7f04d6ffd700 1 --2- 172.21.15.33:0/763589326 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f04c0039620 0x7f04c003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.784+0000 7f04d6ffd700 1 -- 172.21.15.33:0/763589326 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f04d000f050 con 0x7f04d8108110 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.785+0000 7f04dd28e700 1 --2- 172.21.15.33:0/763589326 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f04c0039620 0x7f04c003bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.785+0000 7f04dd28e700 1 --2- 172.21.15.33:0/763589326 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f04c0039620 0x7f04c003bae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f04cc006fd0 tx=0x7f04cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.790+0000 7f04d6ffd700 1 -- 172.21.15.33:0/763589326 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f04d002b020 con 0x7f04d8108110 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.997+0000 7f04dfcf3700 1 -- 172.21.15.33:0/763589326 --> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f04d8104790 con 0x7f04c0039620 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:51.998+0000 7f04d6ffd700 1 -- 172.21.15.33:0/763589326 <== mgr.14120 v2:172.21.15.33:6800/2666001000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f04d8104790 con 0x7f04c0039620 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:52.000+0000 7f04dfcf3700 1 -- 172.21.15.33:0/763589326 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f04c0039620 msgr2=0x7f04c003bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:52.000+0000 7f04dfcf3700 1 --2- 172.21.15.33:0/763589326 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f04c0039620 0x7f04c003bae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f04cc006fd0 tx=0x7f04cc006e40 comp rx=0 tx=0).stop 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:52.000+0000 7f04dfcf3700 1 -- 172.21.15.33:0/763589326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 msgr2=0x7f04d819bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:52.000+0000 7f04dfcf3700 1 --2- 172.21.15.33:0/763589326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 0x7f04d819bf90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f04d00072a0 tx=0x7f04d0003d00 comp rx=0 tx=0).stop 2024-04-02T13:28:52.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:52.000+0000 7f04dfcf3700 1 -- 172.21.15.33:0/763589326 shutdown_connections 2024-04-02T13:28:52.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:52.000+0000 7f04dfcf3700 1 --2- 172.21.15.33:0/763589326 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f04c0039620 0x7f04c003bae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:52.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:52.000+0000 7f04dfcf3700 1 --2- 172.21.15.33:0/763589326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04d8108110 0x7f04d819bf90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:52.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:52.000+0000 7f04dfcf3700 1 -- 172.21.15.33:0/763589326 >> 172.21.15.33:0/763589326 conn(0x7f04d81036f0 msgr2=0x7f04d8104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:52.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:52.000+0000 7f04dfcf3700 1 -- 172.21.15.33:0/763589326 shutdown_connections 2024-04-02T13:28:52.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:52.000+0000 7f04dfcf3700 1 -- 172.21.15.33:0/763589326 wait complete. 2024-04-02T13:28:52.598 INFO:teuthology.orchestra.run.smithi033.stdout:Generating ssh key... 2024-04-02T13:28:53.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: Generating public/private rsa key pair. 2024-04-02T13:28:53.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: Your identification has been saved in /tmp/tmpj9qqhilc/key. 2024-04-02T13:28:53.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: Your public key has been saved in /tmp/tmpj9qqhilc/key.pub. 2024-04-02T13:28:53.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: The key fingerprint is: 2024-04-02T13:28:53.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: SHA256:ezGVcW0+JifpAqtiwhAkpwnbGiZnEKCzWTK+8nIQjwA ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:28:53.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: The key's randomart image is: 2024-04-02T13:28:53.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: +---[RSA 3072]----+ 2024-04-02T13:28:53.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: |+. . .. | 2024-04-02T13:28:53.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: |E.. + o| 2024-04-02T13:28:53.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: |XB. o .o | 2024-04-02T13:28:53.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: |XX+ . . + +.| 2024-04-02T13:28:53.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: |*X. S = . = .| 2024-04-02T13:28:53.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: |+.o o + . | 2024-04-02T13:28:53.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: |.oo o . . | 2024-04-02T13:28:53.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: |o..o o . . | 2024-04-02T13:28:53.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: | o. o . | 2024-04-02T13:28:53.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:28:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: +----[SHA256]-----+ 2024-04-02T13:28:53.869 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.007+0000 7f5281216700 1 Processor -- start 2024-04-02T13:28:53.869 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.007+0000 7f5281216700 1 -- start start 2024-04-02T13:28:53.869 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.007+0000 7f5281216700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 0x7f527c10afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:53.869 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.007+0000 7f5281216700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f527c07c6e0 con 0x7f527c108bf0 2024-04-02T13:28:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.007+0000 7f527ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 0x7f527c10afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.007+0000 7f527ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 0x7f527c10afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43006/0 (socket says 172.21.15.33:43006) 2024-04-02T13:28:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.007+0000 7f527ad9d700 1 -- 172.21.15.33:0/4010261521 learned_addr learned my addr 172.21.15.33:0/4010261521 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.007+0000 7f527ad9d700 1 -- 172.21.15.33:0/4010261521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527c10b520 con 0x7f527c108bf0 2024-04-02T13:28:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.008+0000 7f527ad9d700 1 --2- 172.21.15.33:0/4010261521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 0x7f527c10afe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f526c0098a0 tx=0x7f526c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99276dcbf297899c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.008+0000 7f5279d9b700 1 -- 172.21.15.33:0/4010261521 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f526c004160 con 0x7f527c108bf0 2024-04-02T13:28:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.008+0000 7f5279d9b700 1 -- 172.21.15.33:0/4010261521 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f526c004300 con 0x7f527c108bf0 2024-04-02T13:28:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.008+0000 7f5279d9b700 1 -- 172.21.15.33:0/4010261521 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f526c011b40 con 0x7f527c108bf0 2024-04-02T13:28:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.008+0000 7f5281216700 1 -- 172.21.15.33:0/4010261521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 msgr2=0x7f527c10afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.008+0000 7f5281216700 1 --2- 172.21.15.33:0/4010261521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 0x7f527c10afe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f526c0098a0 tx=0x7f526c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:28:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.008+0000 7f5281216700 1 -- 172.21.15.33:0/4010261521 shutdown_connections 2024-04-02T13:28:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.008+0000 7f5281216700 1 --2- 172.21.15.33:0/4010261521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 0x7f527c10afe0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.008+0000 7f5281216700 1 -- 172.21.15.33:0/4010261521 >> 172.21.15.33:0/4010261521 conn(0x7f527c1025e0 msgr2=0x7f527c104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.008+0000 7f5281216700 1 -- 172.21.15.33:0/4010261521 shutdown_connections 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.008+0000 7f5281216700 1 -- 172.21.15.33:0/4010261521 wait complete. 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.009+0000 7f5281216700 1 Processor -- start 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.009+0000 7f5281216700 1 -- start start 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.010+0000 7f5281216700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 0x7f527c19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.010+0000 7f5281216700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f527c10c4f0 con 0x7f527c108bf0 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.010+0000 7f527ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 0x7f527c19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.010+0000 7f527ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 0x7f527c19bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43016/0 (socket says 172.21.15.33:43016) 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.010+0000 7f527ad9d700 1 -- 172.21.15.33:0/2995166401 learned_addr learned my addr 172.21.15.33:0/2995166401 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.010+0000 7f527ad9d700 1 -- 172.21.15.33:0/2995166401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527c19c4f0 con 0x7f527c108bf0 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.010+0000 7f527ad9d700 1 --2- 172.21.15.33:0/2995166401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 0x7f527c19bfb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f526c011840 tx=0x7f526c0119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.010+0000 7f5263fff700 1 -- 172.21.15.33:0/2995166401 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f526c0039e0 con 0x7f527c108bf0 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.010+0000 7f5263fff700 1 -- 172.21.15.33:0/2995166401 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f526c020070 con 0x7f527c108bf0 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.010+0000 7f5281216700 1 -- 172.21.15.33:0/2995166401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527c19c780 con 0x7f527c108bf0 2024-04-02T13:28:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.010+0000 7f5263fff700 1 -- 172.21.15.33:0/2995166401 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f526c0193c0 con 0x7f527c108bf0 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.010+0000 7f5281216700 1 -- 172.21.15.33:0/2995166401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f527c19f470 con 0x7f527c108bf0 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.011+0000 7f5263fff700 1 -- 172.21.15.33:0/2995166401 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f526c0196a0 con 0x7f527c108bf0 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.011+0000 7f5281216700 1 -- 172.21.15.33:0/2995166401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f527c063c20 con 0x7f527c108bf0 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.011+0000 7f5263fff700 1 --2- 172.21.15.33:0/2995166401 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f5264039670 0x7f526403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.011+0000 7f5263fff700 1 -- 172.21.15.33:0/2995166401 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f526c04d810 con 0x7f527c108bf0 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.012+0000 7f527a59c700 1 --2- 172.21.15.33:0/2995166401 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f5264039670 0x7f526403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.012+0000 7f527a59c700 1 --2- 172.21.15.33:0/2995166401 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f5264039670 0x7f526403bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5270006fd0 tx=0x7f5270006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.016+0000 7f5263fff700 1 -- 172.21.15.33:0/2995166401 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f526c023770 con 0x7f527c108bf0 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.225+0000 7f5281216700 1 -- 172.21.15.33:0/2995166401 --> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f527c1036a0 con 0x7f5264039670 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.373+0000 7f5263fff700 1 -- 172.21.15.33:0/2995166401 <== mgr.14120 v2:172.21.15.33:6800/2666001000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f527c1036a0 con 0x7f5264039670 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.375+0000 7f5281216700 1 -- 172.21.15.33:0/2995166401 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f5264039670 msgr2=0x7f526403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.375+0000 7f5281216700 1 --2- 172.21.15.33:0/2995166401 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f5264039670 0x7f526403bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5270006fd0 tx=0x7f5270006e40 comp rx=0 tx=0).stop 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.375+0000 7f5281216700 1 -- 172.21.15.33:0/2995166401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 msgr2=0x7f527c19bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.375+0000 7f5281216700 1 --2- 172.21.15.33:0/2995166401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 0x7f527c19bfb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f526c011840 tx=0x7f526c0119d0 comp rx=0 tx=0).stop 2024-04-02T13:28:53.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.375+0000 7f5281216700 1 -- 172.21.15.33:0/2995166401 shutdown_connections 2024-04-02T13:28:53.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.375+0000 7f5281216700 1 --2- 172.21.15.33:0/2995166401 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f5264039670 0x7f526403bb30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:53.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.375+0000 7f5281216700 1 --2- 172.21.15.33:0/2995166401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f527c108bf0 0x7f527c19bfb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:53.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.375+0000 7f5281216700 1 -- 172.21.15.33:0/2995166401 >> 172.21.15.33:0/2995166401 conn(0x7f527c1025e0 msgr2=0x7f527c102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:53.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.375+0000 7f5281216700 1 -- 172.21.15.33:0/2995166401 shutdown_connections 2024-04-02T13:28:53.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:53.375+0000 7f5281216700 1 -- 172.21.15.33:0/2995166401 wait complete. 2024-04-02T13:28:54.633 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:54 smithi033 ceph-mon[32390]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:28:54.633 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:54 smithi033 ceph-mon[32390]: Generating ssh key... 2024-04-02T13:28:54.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:54 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' 2024-04-02T13:28:54.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:54 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' 2024-04-02T13:28:55.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHk6AOx3zvViF2UrhsT/hujKp5nhsMrjKHYtRhSCJvyfYfaOJP/EpUPV3pBTnUD4C93LOIIGB4hbUZ2uvACuz6GBmQrcJbOaV093q70vNGNJ4u6ERk7/ALNf2WycCmQFomi5jOKsNY4B3xtLuBuXTEAoUOsDBnTN5U3XxVrqnAA9nmzPoMBnXiLVORe5DFFUSoClCQMS5SAQIlNTh0vhFtkV6Rn0UFuCFVITpIkRhQKKl8ejOy0EtsI+6rvSXVZWgamj2wcUlDmCGeZsBTZbRnc8ummuG2xFLXj9AxRYquoZhEMevLCXlvek+dgZq7S9BU8CevoytqErQhOEr4cJwJ3JpY0pMd3bTuWTzNPBi3jQUoeTKE5CzdOO6X0BKYtwzfd9Pk9GfshzMuEd8kWRic2/KpkwbHLZ/WL4+Pb4KueaDeco0Be1C+hQsjccUZGploBvLrI+tD6AFNm/hK67oKyMQ4EV350omHFNy0I3SZWe8aPrt9gFikqrw+4M2XW5E= ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:28:55.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.408+0000 7f9efe7f8700 1 Processor -- start 2024-04-02T13:28:55.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.408+0000 7f9efe7f8700 1 -- start start 2024-04-02T13:28:55.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.408+0000 7f9efe7f8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 0x7f9ef8106e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:55.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.408+0000 7f9efe7f8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef807c6e0 con 0x7f9ef81069e0 2024-04-02T13:28:55.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.408+0000 7f9ef7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 0x7f9ef8106e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:55.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.409+0000 7f9ef7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 0x7f9ef8106e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58438/0 (socket says 172.21.15.33:58438) 2024-04-02T13:28:55.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.409+0000 7f9ef7fff700 1 -- 172.21.15.33:0/3624971138 learned_addr learned my addr 172.21.15.33:0/3624971138 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:55.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.409+0000 7f9ef7fff700 1 -- 172.21.15.33:0/3624971138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef8107340 con 0x7f9ef81069e0 2024-04-02T13:28:55.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.409+0000 7f9ef7fff700 1 --2- 172.21.15.33:0/3624971138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 0x7f9ef8106e00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80098a0 tx=0x7f9ee8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559759353067ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:55.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.409+0000 7f9ef6ffd700 1 -- 172.21.15.33:0/3624971138 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee8004160 con 0x7f9ef81069e0 2024-04-02T13:28:55.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.409+0000 7f9ef6ffd700 1 -- 172.21.15.33:0/3624971138 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f9ee8004300 con 0x7f9ef81069e0 2024-04-02T13:28:55.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.409+0000 7f9ef6ffd700 1 -- 172.21.15.33:0/3624971138 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee8011b40 con 0x7f9ef81069e0 2024-04-02T13:28:55.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.410+0000 7f9efe7f8700 1 -- 172.21.15.33:0/3624971138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 msgr2=0x7f9ef8106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:55.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.410+0000 7f9efe7f8700 1 --2- 172.21.15.33:0/3624971138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 0x7f9ef8106e00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80098a0 tx=0x7f9ee8009bb0 comp rx=0 tx=0).stop 2024-04-02T13:28:55.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.410+0000 7f9efe7f8700 1 -- 172.21.15.33:0/3624971138 shutdown_connections 2024-04-02T13:28:55.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.410+0000 7f9efe7f8700 1 --2- 172.21.15.33:0/3624971138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 0x7f9ef8106e00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:55.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.410+0000 7f9efe7f8700 1 -- 172.21.15.33:0/3624971138 >> 172.21.15.33:0/3624971138 conn(0x7f9ef81025e0 msgr2=0x7f9ef8104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:55.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.410+0000 7f9efe7f8700 1 -- 172.21.15.33:0/3624971138 shutdown_connections 2024-04-02T13:28:55.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.410+0000 7f9efe7f8700 1 -- 172.21.15.33:0/3624971138 wait complete. 2024-04-02T13:28:55.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.410+0000 7f9efe7f8700 1 Processor -- start 2024-04-02T13:28:55.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.410+0000 7f9efe7f8700 1 -- start start 2024-04-02T13:28:55.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.411+0000 7f9efe7f8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 0x7f9ef819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:55.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.411+0000 7f9efe7f8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef81083d0 con 0x7f9ef81069e0 2024-04-02T13:28:55.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.411+0000 7f9ef7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 0x7f9ef819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:55.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.411+0000 7f9ef7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 0x7f9ef819e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58444/0 (socket says 172.21.15.33:58444) 2024-04-02T13:28:55.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.411+0000 7f9ef7fff700 1 -- 172.21.15.33:0/1772904004 learned_addr learned my addr 172.21.15.33:0/1772904004 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:28:55.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.411+0000 7f9ef7fff700 1 -- 172.21.15.33:0/1772904004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef819e6f0 con 0x7f9ef81069e0 2024-04-02T13:28:55.153 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.411+0000 7f9ef7fff700 1 --2- 172.21.15.33:0/1772904004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 0x7f9ef819e1b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8011840 tx=0x7f9ee80119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:55.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.411+0000 7f9ef57fa700 1 -- 172.21.15.33:0/1772904004 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee80039e0 con 0x7f9ef81069e0 2024-04-02T13:28:55.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.412+0000 7f9ef57fa700 1 -- 172.21.15.33:0/1772904004 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f9ee8020070 con 0x7f9ef81069e0 2024-04-02T13:28:55.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.412+0000 7f9efe7f8700 1 -- 172.21.15.33:0/1772904004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef819e980 con 0x7f9ef81069e0 2024-04-02T13:28:55.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.412+0000 7f9ef57fa700 1 -- 172.21.15.33:0/1772904004 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee80193c0 con 0x7f9ef81069e0 2024-04-02T13:28:55.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.412+0000 7f9efe7f8700 1 -- 172.21.15.33:0/1772904004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef81a1670 con 0x7f9ef81069e0 2024-04-02T13:28:55.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.412+0000 7f9ef57fa700 1 -- 172.21.15.33:0/1772904004 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9ee8019630 con 0x7f9ef81069e0 2024-04-02T13:28:55.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.413+0000 7f9ef57fa700 1 --2- 172.21.15.33:0/1772904004 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f9ee00396c0 0x7f9ee003bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:55.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.413+0000 7f9ef57fa700 1 -- 172.21.15.33:0/1772904004 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9ee804d9d0 con 0x7f9ef81069e0 2024-04-02T13:28:55.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.413+0000 7f9efe7f8700 1 -- 172.21.15.33:0/1772904004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef8063c20 con 0x7f9ef81069e0 2024-04-02T13:28:55.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.413+0000 7f9ef77fe700 1 --2- 172.21.15.33:0/1772904004 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f9ee00396c0 0x7f9ee003bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:55.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.413+0000 7f9ef77fe700 1 --2- 172.21.15.33:0/1772904004 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f9ee00396c0 0x7f9ee003bb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9eec006fd0 tx=0x7f9eec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:55.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.418+0000 7f9ef57fa700 1 -- 172.21.15.33:0/1772904004 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9ee8003b80 con 0x7f9ef81069e0 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.630+0000 7f9efe7f8700 1 -- 172.21.15.33:0/1772904004 --> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f9ef802dfb0 con 0x7f9ee00396c0 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.631+0000 7f9ef57fa700 1 -- 172.21.15.33:0/1772904004 <== mgr.14120 v2:172.21.15.33:6800/2666001000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f9ef802dfb0 con 0x7f9ee00396c0 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.633+0000 7f9efe7f8700 1 -- 172.21.15.33:0/1772904004 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f9ee00396c0 msgr2=0x7f9ee003bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.633+0000 7f9efe7f8700 1 --2- 172.21.15.33:0/1772904004 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f9ee00396c0 0x7f9ee003bb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9eec006fd0 tx=0x7f9eec006e40 comp rx=0 tx=0).stop 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.633+0000 7f9efe7f8700 1 -- 172.21.15.33:0/1772904004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 msgr2=0x7f9ef819e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.633+0000 7f9efe7f8700 1 --2- 172.21.15.33:0/1772904004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 0x7f9ef819e1b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8011840 tx=0x7f9ee80119d0 comp rx=0 tx=0).stop 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.633+0000 7f9efe7f8700 1 -- 172.21.15.33:0/1772904004 shutdown_connections 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.633+0000 7f9efe7f8700 1 --2- 172.21.15.33:0/1772904004 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f9ee00396c0 0x7f9ee003bb80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.633+0000 7f9efe7f8700 1 --2- 172.21.15.33:0/1772904004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ef81069e0 0x7f9ef819e1b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.633+0000 7f9efe7f8700 1 -- 172.21.15.33:0/1772904004 >> 172.21.15.33:0/1772904004 conn(0x7f9ef81025e0 msgr2=0x7f9ef8194e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.633+0000 7f9efe7f8700 1 -- 172.21.15.33:0/1772904004 shutdown_connections 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:54.633+0000 7f9efe7f8700 1 -- 172.21.15.33:0/1772904004 wait complete. 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:Adding key to root@localhost authorized_keys... 2024-04-02T13:28:55.155 INFO:teuthology.orchestra.run.smithi033.stdout:Adding host smithi033... 2024-04-02T13:28:55.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:55 smithi033 ceph-mon[32390]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:28:56.603 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:56 smithi033 ceph-mon[32390]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:28:57.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:28:57 smithi033 ceph-mon[32390]: Deploying cephadm binary to smithi033 2024-04-02T13:29:00.785 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Added host 'smithi033' with addr '172.21.15.33' 2024-04-02T13:29:00.785 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.542+0000 7faef324d700 1 Processor -- start 2024-04-02T13:29:00.785 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.542+0000 7faef324d700 1 -- start start 2024-04-02T13:29:00.785 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.543+0000 7faef324d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 0x7faeec07cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:00.785 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.543+0000 7faef324d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faeec07d160 con 0x7faeec07e730 2024-04-02T13:29:00.785 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.543+0000 7faef0fe9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 0x7faeec07cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:00.785 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.543+0000 7faef0fe9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 0x7faeec07cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58458/0 (socket says 172.21.15.33:58458) 2024-04-02T13:29:00.786 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.543+0000 7faef0fe9700 1 -- 172.21.15.33:0/3297284548 learned_addr learned my addr 172.21.15.33:0/3297284548 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:00.786 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.543+0000 7faef0fe9700 1 -- 172.21.15.33:0/3297284548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeec07d2e0 con 0x7faeec07e730 2024-04-02T13:29:00.786 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.543+0000 7faef0fe9700 1 --2- 172.21.15.33:0/3297284548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 0x7faeec07cb90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faee00098a0 tx=0x7faee0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65e6c5488be5cc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:00.786 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.543+0000 7faeeb7fe700 1 -- 172.21.15.33:0/3297284548 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee0004160 con 0x7faeec07e730 2024-04-02T13:29:00.786 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.543+0000 7faeeb7fe700 1 -- 172.21.15.33:0/3297284548 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7faee0004300 con 0x7faeec07e730 2024-04-02T13:29:00.786 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.543+0000 7faeeb7fe700 1 -- 172.21.15.33:0/3297284548 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee0011b40 con 0x7faeec07e730 2024-04-02T13:29:00.786 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.544+0000 7faef324d700 1 -- 172.21.15.33:0/3297284548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 msgr2=0x7faeec07cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:00.786 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.544+0000 7faef324d700 1 --2- 172.21.15.33:0/3297284548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 0x7faeec07cb90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faee00098a0 tx=0x7faee0009bb0 comp rx=0 tx=0).stop 2024-04-02T13:29:00.786 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.544+0000 7faef324d700 1 -- 172.21.15.33:0/3297284548 shutdown_connections 2024-04-02T13:29:00.786 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.544+0000 7faef324d700 1 --2- 172.21.15.33:0/3297284548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 0x7faeec07cb90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:00.786 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.544+0000 7faef324d700 1 -- 172.21.15.33:0/3297284548 >> 172.21.15.33:0/3297284548 conn(0x7faeec1036e0 msgr2=0x7faeec105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:00.786 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.544+0000 7faef324d700 1 -- 172.21.15.33:0/3297284548 shutdown_connections 2024-04-02T13:29:00.786 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.544+0000 7faef324d700 1 -- 172.21.15.33:0/3297284548 wait complete. 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.545+0000 7faef324d700 1 Processor -- start 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.545+0000 7faef324d700 1 -- start start 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.545+0000 7faef324d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 0x7faeec19e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.545+0000 7faef324d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faeec10e730 con 0x7faeec07e730 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.545+0000 7faef0fe9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 0x7faeec19e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.545+0000 7faef0fe9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 0x7faeec19e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58468/0 (socket says 172.21.15.33:58468) 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.545+0000 7faef0fe9700 1 -- 172.21.15.33:0/2872592632 learned_addr learned my addr 172.21.15.33:0/2872592632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.545+0000 7faef0fe9700 1 -- 172.21.15.33:0/2872592632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeec19e810 con 0x7faeec07e730 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.546+0000 7faef0fe9700 1 --2- 172.21.15.33:0/2872592632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 0x7faeec19e2d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faee0011670 tx=0x7faee0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.546+0000 7faee9ffb700 1 -- 172.21.15.33:0/2872592632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee00038b0 con 0x7faeec07e730 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.546+0000 7faee9ffb700 1 -- 172.21.15.33:0/2872592632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7faee001a430 con 0x7faeec07e730 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.546+0000 7faee9ffb700 1 -- 172.21.15.33:0/2872592632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faee0019550 con 0x7faeec07e730 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.546+0000 7faef324d700 1 -- 172.21.15.33:0/2872592632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeec19eaa0 con 0x7faeec07e730 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.546+0000 7faef324d700 1 -- 172.21.15.33:0/2872592632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeec1a1740 con 0x7faeec07e730 2024-04-02T13:29:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.547+0000 7faee9ffb700 1 -- 172.21.15.33:0/2872592632 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7faee001aa50 con 0x7faeec07e730 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.547+0000 7faee9ffb700 1 --2- 172.21.15.33:0/2872592632 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7faedc039670 0x7faedc03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.547+0000 7faee9ffb700 1 -- 172.21.15.33:0/2872592632 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faee004d9b0 con 0x7faeec07e730 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.547+0000 7faef324d700 1 -- 172.21.15.33:0/2872592632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeec063c20 con 0x7faeec07e730 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.547+0000 7faeebfff700 1 --2- 172.21.15.33:0/2872592632 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7faedc039670 0x7faedc03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.547+0000 7faeebfff700 1 --2- 172.21.15.33:0/2872592632 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7faedc039670 0x7faedc03bb30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faed8006fd0 tx=0x7faed8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.552+0000 7faee9ffb700 1 -- 172.21.15.33:0/2872592632 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7faee0023e50 con 0x7faeec07e730 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:28:55.763+0000 7faef324d700 1 -- 172.21.15.33:0/2872592632 --> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}) -- 0x7faeec104930 con 0x7faedc039670 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:00.161+0000 7faee9ffb700 1 -- 172.21.15.33:0/2872592632 <== mgr.14120 v2:172.21.15.33:6800/2666001000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7faeec104930 con 0x7faedc039670 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:00.164+0000 7faef324d700 1 -- 172.21.15.33:0/2872592632 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7faedc039670 msgr2=0x7faedc03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:00.164+0000 7faef324d700 1 --2- 172.21.15.33:0/2872592632 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7faedc039670 0x7faedc03bb30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faed8006fd0 tx=0x7faed8006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:00.164+0000 7faef324d700 1 -- 172.21.15.33:0/2872592632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 msgr2=0x7faeec19e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:00.164+0000 7faef324d700 1 --2- 172.21.15.33:0/2872592632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 0x7faeec19e2d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faee0011670 tx=0x7faee0011b50 comp rx=0 tx=0).stop 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:00.164+0000 7faef324d700 1 -- 172.21.15.33:0/2872592632 shutdown_connections 2024-04-02T13:29:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:00.164+0000 7faef324d700 1 --2- 172.21.15.33:0/2872592632 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7faedc039670 0x7faedc03bb30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:00.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:00.164+0000 7faef324d700 1 --2- 172.21.15.33:0/2872592632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faeec07e730 0x7faeec19e2d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:00.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:00.164+0000 7faef324d700 1 -- 172.21.15.33:0/2872592632 >> 172.21.15.33:0/2872592632 conn(0x7faeec1036e0 msgr2=0x7faeec104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:00.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:00.164+0000 7faef324d700 1 -- 172.21.15.33:0/2872592632 shutdown_connections 2024-04-02T13:29:00.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:00.164+0000 7faef324d700 1 -- 172.21.15.33:0/2872592632 wait complete. 2024-04-02T13:29:00.789 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mon service... 2024-04-02T13:29:01.449 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:01 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' 2024-04-02T13:29:01.450 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:01 smithi033 ceph-mon[32390]: Added host smithi033 2024-04-02T13:29:01.450 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:01 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:02.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-02T13:29:02.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.191+0000 7f3439a8a700 1 Processor -- start 2024-04-02T13:29:02.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.191+0000 7f3439a8a700 1 -- start start 2024-04-02T13:29:02.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.191+0000 7f3439a8a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 0x7f3434107450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:02.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.191+0000 7f3439a8a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3434107a20 con 0x7f3434107030 2024-04-02T13:29:02.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.191+0000 7f34337fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 0x7f3434107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:02.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.191+0000 7f34337fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 0x7f3434107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58480/0 (socket says 172.21.15.33:58480) 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.191+0000 7f34337fe700 1 -- 172.21.15.33:0/2476815715 learned_addr learned my addr 172.21.15.33:0/2476815715 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.191+0000 7f34337fe700 1 -- 172.21.15.33:0/2476815715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3434108230 con 0x7f3434107030 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.191+0000 7f34337fe700 1 --2- 172.21.15.33:0/2476815715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 0x7f3434107450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f342400bb40 tx=0x7f342400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06bf6533e446f25 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.192+0000 7f3432ffd700 1 -- 172.21.15.33:0/2476815715 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f342400dbe0 con 0x7f3434107030 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.192+0000 7f3432ffd700 1 -- 172.21.15.33:0/2476815715 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f34240044f0 con 0x7f3434107030 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.192+0000 7f3432ffd700 1 -- 172.21.15.33:0/2476815715 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3424010470 con 0x7f3434107030 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.192+0000 7f3439a8a700 1 -- 172.21.15.33:0/2476815715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 msgr2=0x7f3434107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.192+0000 7f3439a8a700 1 --2- 172.21.15.33:0/2476815715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 0x7f3434107450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f342400bb40 tx=0x7f342400be50 comp rx=0 tx=0).stop 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.192+0000 7f3439a8a700 1 -- 172.21.15.33:0/2476815715 shutdown_connections 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.192+0000 7f3439a8a700 1 --2- 172.21.15.33:0/2476815715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 0x7f3434107450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.192+0000 7f3439a8a700 1 -- 172.21.15.33:0/2476815715 >> 172.21.15.33:0/2476815715 conn(0x7f34341025f0 msgr2=0x7f3434104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.192+0000 7f3439a8a700 1 -- 172.21.15.33:0/2476815715 shutdown_connections 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.192+0000 7f3439a8a700 1 -- 172.21.15.33:0/2476815715 wait complete. 2024-04-02T13:29:02.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.193+0000 7f3439a8a700 1 Processor -- start 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.193+0000 7f3439a8a700 1 -- start start 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.193+0000 7f3439a8a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 0x7f3434199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.193+0000 7f3439a8a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f343407afd0 con 0x7f3434107030 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.193+0000 7f34337fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 0x7f3434199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.193+0000 7f34337fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 0x7f3434199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58488/0 (socket says 172.21.15.33:58488) 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.193+0000 7f34337fe700 1 -- 172.21.15.33:0/3853108307 learned_addr learned my addr 172.21.15.33:0/3853108307 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.194+0000 7f34337fe700 1 -- 172.21.15.33:0/3853108307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343419a350 con 0x7f3434107030 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.194+0000 7f34337fe700 1 --2- 172.21.15.33:0/3853108307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 0x7f3434199e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3424003ff0 tx=0x7f34240040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.194+0000 7f34317fa700 1 -- 172.21.15.33:0/3853108307 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3424027070 con 0x7f3434107030 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.194+0000 7f34317fa700 1 -- 172.21.15.33:0/3853108307 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7f3424010e40 con 0x7f3434107030 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.194+0000 7f3439a8a700 1 -- 172.21.15.33:0/3853108307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343419a5e0 con 0x7f3434107030 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.194+0000 7f34317fa700 1 -- 172.21.15.33:0/3853108307 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3424022c90 con 0x7f3434107030 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.194+0000 7f3439a8a700 1 -- 172.21.15.33:0/3853108307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343419d2d0 con 0x7f3434107030 2024-04-02T13:29:02.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.195+0000 7f34317fa700 1 -- 172.21.15.33:0/3853108307 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f342402a070 con 0x7f3434107030 2024-04-02T13:29:02.206 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.195+0000 7f3439a8a700 1 -- 172.21.15.33:0/3853108307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3434063c20 con 0x7f3434107030 2024-04-02T13:29:02.206 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.195+0000 7f34317fa700 1 --2- 172.21.15.33:0/3853108307 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f3420039670 0x7f342003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:02.206 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.195+0000 7f34317fa700 1 -- 172.21.15.33:0/3853108307 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f342401f070 con 0x7f3434107030 2024-04-02T13:29:02.206 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.195+0000 7f342affd700 1 --2- 172.21.15.33:0/3853108307 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f3420039670 0x7f342003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:02.206 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.196+0000 7f342affd700 1 --2- 172.21.15.33:0/3853108307 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f3420039670 0x7f342003bb30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f341c006fd0 tx=0x7f341c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:02.206 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.200+0000 7f34317fa700 1 -- 172.21.15.33:0/3853108307 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3424015070 con 0x7f3434107030 2024-04-02T13:29:02.206 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.446+0000 7f3439a8a700 1 -- 172.21.15.33:0/3853108307 --> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3434062a30 con 0x7f3420039670 2024-04-02T13:29:02.206 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.453+0000 7f34317fa700 1 -- 172.21.15.33:0/3853108307 <== mgr.14120 v2:172.21.15.33:6800/2666001000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3434062a30 con 0x7f3420039670 2024-04-02T13:29:02.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.455+0000 7f3439a8a700 1 -- 172.21.15.33:0/3853108307 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f3420039670 msgr2=0x7f342003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:02.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.455+0000 7f3439a8a700 1 --2- 172.21.15.33:0/3853108307 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f3420039670 0x7f342003bb30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f341c006fd0 tx=0x7f341c006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:02.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.455+0000 7f3439a8a700 1 -- 172.21.15.33:0/3853108307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 msgr2=0x7f3434199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:02.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.455+0000 7f3439a8a700 1 --2- 172.21.15.33:0/3853108307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 0x7f3434199e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3424003ff0 tx=0x7f34240040d0 comp rx=0 tx=0).stop 2024-04-02T13:29:02.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.455+0000 7f3439a8a700 1 -- 172.21.15.33:0/3853108307 shutdown_connections 2024-04-02T13:29:02.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.455+0000 7f3439a8a700 1 --2- 172.21.15.33:0/3853108307 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f3420039670 0x7f342003bb30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:02.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.455+0000 7f3439a8a700 1 --2- 172.21.15.33:0/3853108307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3434107030 0x7f3434199e10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:02.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.455+0000 7f3439a8a700 1 -- 172.21.15.33:0/3853108307 >> 172.21.15.33:0/3853108307 conn(0x7f34341025f0 msgr2=0x7f3434190450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:02.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.455+0000 7f3439a8a700 1 -- 172.21.15.33:0/3853108307 shutdown_connections 2024-04-02T13:29:02.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:01.456+0000 7f3439a8a700 1 -- 172.21.15.33:0/3853108307 wait complete. 2024-04-02T13:29:02.207 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mgr service... 2024-04-02T13:29:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:02 smithi033 ceph-mon[32390]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:02 smithi033 ceph-mon[32390]: Saving service mon spec with placement count:5 2024-04-02T13:29:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:02 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' 2024-04-02T13:29:04.158 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:03 smithi033 ceph-mon[32390]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:04.158 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:03 smithi033 ceph-mon[32390]: Saving service mgr spec with placement count:2 2024-04-02T13:29:04.158 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:03 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' 2024-04-02T13:29:04.158 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:03 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' 2024-04-02T13:29:04.166 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-02T13:29:04.166 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.645+0000 7fcd906a9700 1 Processor -- start 2024-04-02T13:29:04.166 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.645+0000 7fcd906a9700 1 -- start start 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.646+0000 7fcd906a9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 0x7fcd88108540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.646+0000 7fcd906a9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcd88108b10 con 0x7fcd88108120 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.646+0000 7fcd8e445700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 0x7fcd88108540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.646+0000 7fcd8e445700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 0x7fcd88108540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58494/0 (socket says 172.21.15.33:58494) 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.646+0000 7fcd8e445700 1 -- 172.21.15.33:0/396644327 learned_addr learned my addr 172.21.15.33:0/396644327 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.646+0000 7fcd8e445700 1 -- 172.21.15.33:0/396644327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd88109320 con 0x7fcd88108120 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.646+0000 7fcd8e445700 1 --2- 172.21.15.33:0/396644327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 0x7fcd88108540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcd780098a0 tx=0x7fcd78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2d6eeb8ee6eeee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.647+0000 7fcd8d443700 1 -- 172.21.15.33:0/396644327 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd78004160 con 0x7fcd88108120 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.647+0000 7fcd8d443700 1 -- 172.21.15.33:0/396644327 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fcd78004300 con 0x7fcd88108120 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.647+0000 7fcd8d443700 1 -- 172.21.15.33:0/396644327 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd78011b40 con 0x7fcd88108120 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.647+0000 7fcd906a9700 1 -- 172.21.15.33:0/396644327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 msgr2=0x7fcd88108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.647+0000 7fcd906a9700 1 --2- 172.21.15.33:0/396644327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 0x7fcd88108540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcd780098a0 tx=0x7fcd78009bb0 comp rx=0 tx=0).stop 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.647+0000 7fcd906a9700 1 -- 172.21.15.33:0/396644327 shutdown_connections 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.647+0000 7fcd906a9700 1 --2- 172.21.15.33:0/396644327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 0x7fcd88108540 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:04.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.647+0000 7fcd906a9700 1 -- 172.21.15.33:0/396644327 >> 172.21.15.33:0/396644327 conn(0x7fcd881036e0 msgr2=0x7fcd88105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.647+0000 7fcd906a9700 1 -- 172.21.15.33:0/396644327 shutdown_connections 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.647+0000 7fcd906a9700 1 -- 172.21.15.33:0/396644327 wait complete. 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.648+0000 7fcd906a9700 1 Processor -- start 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.648+0000 7fcd906a9700 1 -- start start 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.648+0000 7fcd906a9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 0x7fcd8807e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.648+0000 7fcd906a9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8810c6d0 con 0x7fcd88108120 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.648+0000 7fcd8e445700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 0x7fcd8807e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.648+0000 7fcd8e445700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 0x7fcd8807e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58502/0 (socket says 172.21.15.33:58502) 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.648+0000 7fcd8e445700 1 -- 172.21.15.33:0/4270177766 learned_addr learned my addr 172.21.15.33:0/4270177766 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.649+0000 7fcd8e445700 1 -- 172.21.15.33:0/4270177766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8807e940 con 0x7fcd88108120 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.649+0000 7fcd8e445700 1 --2- 172.21.15.33:0/4270177766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 0x7fcd8807e400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcd78011670 tx=0x7fcd78011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.649+0000 7fcd7f7fe700 1 -- 172.21.15.33:0/4270177766 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd780038b0 con 0x7fcd88108120 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.649+0000 7fcd906a9700 1 -- 172.21.15.33:0/4270177766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8807cb30 con 0x7fcd88108120 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.649+0000 7fcd7f7fe700 1 -- 172.21.15.33:0/4270177766 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1105+0+0 (secure 0 0 0) 0x7fcd7801a430 con 0x7fcd88108120 2024-04-02T13:29:04.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.649+0000 7fcd7f7fe700 1 -- 172.21.15.33:0/4270177766 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd78019550 con 0x7fcd88108120 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.649+0000 7fcd906a9700 1 -- 172.21.15.33:0/4270177766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd8807d070 con 0x7fcd88108120 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.650+0000 7fcd7f7fe700 1 -- 172.21.15.33:0/4270177766 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fcd78003a50 con 0x7fcd88108120 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.650+0000 7fcd906a9700 1 -- 172.21.15.33:0/4270177766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd88063c20 con 0x7fcd88108120 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.650+0000 7fcd7f7fe700 1 --2- 172.21.15.33:0/4270177766 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7fcd74039620 0x7fcd7403bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.650+0000 7fcd7f7fe700 1 -- 172.21.15.33:0/4270177766 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcd7804d950 con 0x7fcd88108120 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.650+0000 7fcd8dc44700 1 --2- 172.21.15.33:0/4270177766 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7fcd74039620 0x7fcd7403bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.650+0000 7fcd8dc44700 1 --2- 172.21.15.33:0/4270177766 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7fcd74039620 0x7fcd7403bae0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcd84006fd0 tx=0x7fcd84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.655+0000 7fcd7f7fe700 1 -- 172.21.15.33:0/4270177766 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcd78003d40 con 0x7fcd88108120 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.880+0000 7fcd906a9700 1 -- 172.21.15.33:0/4270177766 --> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcd881048f0 con 0x7fcd74039620 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.887+0000 7fcd7f7fe700 1 -- 172.21.15.33:0/4270177766 <== mgr.14120 v2:172.21.15.33:6800/2666001000 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcd881048f0 con 0x7fcd74039620 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.889+0000 7fcd906a9700 1 -- 172.21.15.33:0/4270177766 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7fcd74039620 msgr2=0x7fcd7403bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.889+0000 7fcd906a9700 1 --2- 172.21.15.33:0/4270177766 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7fcd74039620 0x7fcd7403bae0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcd84006fd0 tx=0x7fcd84006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.889+0000 7fcd906a9700 1 -- 172.21.15.33:0/4270177766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 msgr2=0x7fcd8807e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:04.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.889+0000 7fcd906a9700 1 --2- 172.21.15.33:0/4270177766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 0x7fcd8807e400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcd78011670 tx=0x7fcd78011b50 comp rx=0 tx=0).stop 2024-04-02T13:29:04.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.889+0000 7fcd906a9700 1 -- 172.21.15.33:0/4270177766 shutdown_connections 2024-04-02T13:29:04.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.889+0000 7fcd906a9700 1 --2- 172.21.15.33:0/4270177766 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7fcd74039620 0x7fcd7403bae0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:04.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.889+0000 7fcd906a9700 1 --2- 172.21.15.33:0/4270177766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd88108120 0x7fcd8807e400 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:04.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.889+0000 7fcd906a9700 1 -- 172.21.15.33:0/4270177766 >> 172.21.15.33:0/4270177766 conn(0x7fcd881036e0 msgr2=0x7fcd881041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:04.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.889+0000 7fcd906a9700 1 -- 172.21.15.33:0/4270177766 shutdown_connections 2024-04-02T13:29:04.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:02.889+0000 7fcd906a9700 1 -- 172.21.15.33:0/4270177766 wait complete. 2024-04-02T13:29:05.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.648+0000 7f483ed9f700 1 Processor -- start 2024-04-02T13:29:05.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.648+0000 7f483ed9f700 1 -- start start 2024-04-02T13:29:05.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.648+0000 7f483ed9f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 0x7f483810af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.648+0000 7f483ed9f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f483807beb0 con 0x7f4838108ba0 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.648+0000 7f483cb3b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 0x7f483810af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.648+0000 7f483cb3b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 0x7f483810af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53962/0 (socket says 172.21.15.33:53962) 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.648+0000 7f483cb3b700 1 -- 172.21.15.33:0/2731739903 learned_addr learned my addr 172.21.15.33:0/2731739903 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.648+0000 7f483cb3b700 1 -- 172.21.15.33:0/2731739903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483810b4d0 con 0x7f4838108ba0 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.648+0000 7f483cb3b700 1 --2- 172.21.15.33:0/2731739903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 0x7f483810af90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f482c0098a0 tx=0x7f482c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38e2c0e87a7e33e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.649+0000 7f48377fe700 1 -- 172.21.15.33:0/2731739903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f482c004160 con 0x7f4838108ba0 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.649+0000 7f48377fe700 1 -- 172.21.15.33:0/2731739903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f482c004300 con 0x7f4838108ba0 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.649+0000 7f48377fe700 1 -- 172.21.15.33:0/2731739903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f482c011b10 con 0x7f4838108ba0 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.649+0000 7f483ed9f700 1 -- 172.21.15.33:0/2731739903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 msgr2=0x7f483810af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.649+0000 7f483ed9f700 1 --2- 172.21.15.33:0/2731739903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 0x7f483810af90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f482c0098a0 tx=0x7f482c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.649+0000 7f483ed9f700 1 -- 172.21.15.33:0/2731739903 shutdown_connections 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.649+0000 7f483ed9f700 1 --2- 172.21.15.33:0/2731739903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 0x7f483810af90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:05.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.649+0000 7f483ed9f700 1 -- 172.21.15.33:0/2731739903 >> 172.21.15.33:0/2731739903 conn(0x7f48381025f0 msgr2=0x7f4838104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.649+0000 7f483ed9f700 1 -- 172.21.15.33:0/2731739903 shutdown_connections 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.649+0000 7f483ed9f700 1 -- 172.21.15.33:0/2731739903 wait complete. 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.650+0000 7f483ed9f700 1 Processor -- start 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.650+0000 7f483ed9f700 1 -- start start 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.650+0000 7f483ed9f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 0x7f4838108420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.650+0000 7f483ed9f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f483810c540 con 0x7f4838108ba0 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.650+0000 7f483cb3b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 0x7f4838108420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.650+0000 7f483cb3b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 0x7f4838108420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53966/0 (socket says 172.21.15.33:53966) 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.650+0000 7f483cb3b700 1 -- 172.21.15.33:0/2679383723 learned_addr learned my addr 172.21.15.33:0/2679383723 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.651+0000 7f483cb3b700 1 -- 172.21.15.33:0/2679383723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4838108960 con 0x7f4838108ba0 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.651+0000 7f483cb3b700 1 --2- 172.21.15.33:0/2679383723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 0x7f4838108420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f482c003fa0 tx=0x7f482c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.651+0000 7f4835ffb700 1 -- 172.21.15.33:0/2679383723 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f482c0038b0 con 0x7f4838108ba0 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.651+0000 7f4835ffb700 1 -- 172.21.15.33:0/2679383723 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f482c01a430 con 0x7f4838108ba0 2024-04-02T13:29:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.651+0000 7f483ed9f700 1 -- 172.21.15.33:0/2679383723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4838106b50 con 0x7f4838108ba0 2024-04-02T13:29:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.651+0000 7f4835ffb700 1 -- 172.21.15.33:0/2679383723 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f482c019550 con 0x7f4838108ba0 2024-04-02T13:29:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.651+0000 7f483ed9f700 1 -- 172.21.15.33:0/2679383723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4838107090 con 0x7f4838108ba0 2024-04-02T13:29:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.652+0000 7f4835ffb700 1 -- 172.21.15.33:0/2679383723 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f482c0196f0 con 0x7f4838108ba0 2024-04-02T13:29:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.652+0000 7f483ed9f700 1 -- 172.21.15.33:0/2679383723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4838063c20 con 0x7f4838108ba0 2024-04-02T13:29:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.652+0000 7f4835ffb700 1 --2- 172.21.15.33:0/2679383723 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f4828039670 0x7f482803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.652+0000 7f4835ffb700 1 -- 172.21.15.33:0/2679383723 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f482c04d9d0 con 0x7f4838108ba0 2024-04-02T13:29:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.652+0000 7f4837fff700 1 --2- 172.21.15.33:0/2679383723 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f4828039670 0x7f482803bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.653+0000 7f4837fff700 1 --2- 172.21.15.33:0/2679383723 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f4828039670 0x7f482803bb30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4824006fd0 tx=0x7f4824006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.657+0000 7f4835ffb700 1 -- 172.21.15.33:0/2679383723 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f482c0199e0 con 0x7f4838108ba0 2024-04-02T13:29:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.866+0000 7f483ed9f700 1 -- 172.21.15.33:0/2679383723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f4838107930 con 0x7f4838108ba0 2024-04-02T13:29:05.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.869+0000 7f4835ffb700 1 -- 172.21.15.33:0/2679383723 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f482c019db0 con 0x7f4838108ba0 2024-04-02T13:29:05.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.871+0000 7f483ed9f700 1 -- 172.21.15.33:0/2679383723 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f4828039670 msgr2=0x7f482803bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:05.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.871+0000 7f483ed9f700 1 --2- 172.21.15.33:0/2679383723 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f4828039670 0x7f482803bb30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4824006fd0 tx=0x7f4824006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:05.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.871+0000 7f483ed9f700 1 -- 172.21.15.33:0/2679383723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 msgr2=0x7f4838108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:05.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.871+0000 7f483ed9f700 1 --2- 172.21.15.33:0/2679383723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 0x7f4838108420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f482c003fa0 tx=0x7f482c0046a0 comp rx=0 tx=0).stop 2024-04-02T13:29:05.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.872+0000 7f483ed9f700 1 -- 172.21.15.33:0/2679383723 shutdown_connections 2024-04-02T13:29:05.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.872+0000 7f483ed9f700 1 --2- 172.21.15.33:0/2679383723 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f4828039670 0x7f482803bb30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:05.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.872+0000 7f483ed9f700 1 --2- 172.21.15.33:0/2679383723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4838108ba0 0x7f4838108420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:05.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.872+0000 7f483ed9f700 1 -- 172.21.15.33:0/2679383723 >> 172.21.15.33:0/2679383723 conn(0x7f48381025f0 msgr2=0x7f48381030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:05.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.872+0000 7f483ed9f700 1 -- 172.21.15.33:0/2679383723 shutdown_connections 2024-04-02T13:29:05.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:04.872+0000 7f483ed9f700 1 -- 172.21.15.33:0/2679383723 wait complete. 2024-04-02T13:29:05.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:05 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' 2024-04-02T13:29:05.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:05 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2679383723' entity='client.admin' 2024-04-02T13:29:07.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.916+0000 7f680d8b2700 1 Processor -- start 2024-04-02T13:29:07.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.916+0000 7f680d8b2700 1 -- start start 2024-04-02T13:29:07.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.916+0000 7f680d8b2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 0x7f6808108550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:07.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.916+0000 7f680d8b2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6808108b20 con 0x7f6808108130 2024-04-02T13:29:07.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.916+0000 7f6806ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 0x7f6808108550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:07.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.916+0000 7f6806ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 0x7f6808108550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53972/0 (socket says 172.21.15.33:53972) 2024-04-02T13:29:07.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.916+0000 7f6806ffd700 1 -- 172.21.15.33:0/3565477175 learned_addr learned my addr 172.21.15.33:0/3565477175 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:07.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.917+0000 7f6806ffd700 1 -- 172.21.15.33:0/3565477175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6808109330 con 0x7f6808108130 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.917+0000 7f6806ffd700 1 --2- 172.21.15.33:0/3565477175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 0x7f6808108550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67f80098a0 tx=0x7f67f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794ff8effd828a0e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.917+0000 7f6805ffb700 1 -- 172.21.15.33:0/3565477175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f8004160 con 0x7f6808108130 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.917+0000 7f6805ffb700 1 -- 172.21.15.33:0/3565477175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f67f8004300 con 0x7f6808108130 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.917+0000 7f6805ffb700 1 -- 172.21.15.33:0/3565477175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f8011b10 con 0x7f6808108130 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.918+0000 7f680d8b2700 1 -- 172.21.15.33:0/3565477175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 msgr2=0x7f6808108550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.918+0000 7f680d8b2700 1 --2- 172.21.15.33:0/3565477175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 0x7f6808108550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67f80098a0 tx=0x7f67f8009bb0 comp rx=0 tx=0).stop 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.918+0000 7f680d8b2700 1 -- 172.21.15.33:0/3565477175 shutdown_connections 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.918+0000 7f680d8b2700 1 --2- 172.21.15.33:0/3565477175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 0x7f6808108550 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.918+0000 7f680d8b2700 1 -- 172.21.15.33:0/3565477175 >> 172.21.15.33:0/3565477175 conn(0x7f68081036f0 msgr2=0x7f6808105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.918+0000 7f680d8b2700 1 -- 172.21.15.33:0/3565477175 shutdown_connections 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.918+0000 7f680d8b2700 1 -- 172.21.15.33:0/3565477175 wait complete. 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.918+0000 7f680d8b2700 1 Processor -- start 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.919+0000 7f680d8b2700 1 -- start start 2024-04-02T13:29:07.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.919+0000 7f680d8b2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 0x7f680819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.919+0000 7f680d8b2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f680810c6e0 con 0x7f6808108130 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.919+0000 7f6806ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 0x7f680819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.919+0000 7f6806ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 0x7f680819e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53986/0 (socket says 172.21.15.33:53986) 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.919+0000 7f6806ffd700 1 -- 172.21.15.33:0/2707616191 learned_addr learned my addr 172.21.15.33:0/2707616191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.919+0000 7f6806ffd700 1 -- 172.21.15.33:0/2707616191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680819e7e0 con 0x7f6808108130 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.919+0000 7f6806ffd700 1 --2- 172.21.15.33:0/2707616191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 0x7f680819e2a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67f8006af0 tx=0x7f67f8011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.919+0000 7f680c8b0700 1 -- 172.21.15.33:0/2707616191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f8003af0 con 0x7f6808108130 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.920+0000 7f680c8b0700 1 -- 172.21.15.33:0/2707616191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f67f8003c90 con 0x7f6808108130 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.920+0000 7f680c8b0700 1 -- 172.21.15.33:0/2707616191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f8024830 con 0x7f6808108130 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.920+0000 7f680d8b2700 1 -- 172.21.15.33:0/2707616191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680819ea70 con 0x7f6808108130 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.920+0000 7f680d8b2700 1 -- 172.21.15.33:0/2707616191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68081a1760 con 0x7f6808108130 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.920+0000 7f680c8b0700 1 -- 172.21.15.33:0/2707616191 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f67f8022020 con 0x7f6808108130 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.921+0000 7f680c8b0700 1 --2- 172.21.15.33:0/2707616191 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f67f0039670 0x7f67f003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.921+0000 7f680c8b0700 1 -- 172.21.15.33:0/2707616191 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f67f804eab0 con 0x7f6808108130 2024-04-02T13:29:07.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.921+0000 7f68067fc700 1 --2- 172.21.15.33:0/2707616191 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f67f0039670 0x7f67f003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:07.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.921+0000 7f680d8b2700 1 -- 172.21.15.33:0/2707616191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6808063c20 con 0x7f6808108130 2024-04-02T13:29:07.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.921+0000 7f68067fc700 1 --2- 172.21.15.33:0/2707616191 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f67f0039670 0x7f67f003bb30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:07.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:05.926+0000 7f680c8b0700 1 -- 172.21.15.33:0/2707616191 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f67f802ad60 con 0x7f6808108130 2024-04-02T13:29:07.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:06.144+0000 7f680d8b2700 1 -- 172.21.15.33:0/2707616191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f68081a1930 con 0x7f6808108130 2024-04-02T13:29:07.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:06.146+0000 7f680c8b0700 1 -- 172.21.15.33:0/2707616191 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f67f8006ea0 con 0x7f6808108130 2024-04-02T13:29:07.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:06.148+0000 7f680d8b2700 1 -- 172.21.15.33:0/2707616191 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f67f0039670 msgr2=0x7f67f003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:07.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:06.148+0000 7f680d8b2700 1 --2- 172.21.15.33:0/2707616191 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f67f0039670 0x7f67f003bb30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:07.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:06.149+0000 7f680d8b2700 1 -- 172.21.15.33:0/2707616191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 msgr2=0x7f680819e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:07.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:06.149+0000 7f680d8b2700 1 --2- 172.21.15.33:0/2707616191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 0x7f680819e2a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67f8006af0 tx=0x7f67f8011bd0 comp rx=0 tx=0).stop 2024-04-02T13:29:07.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:06.149+0000 7f680d8b2700 1 -- 172.21.15.33:0/2707616191 shutdown_connections 2024-04-02T13:29:07.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:06.149+0000 7f680d8b2700 1 --2- 172.21.15.33:0/2707616191 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f67f0039670 0x7f67f003bb30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:07.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:06.149+0000 7f680d8b2700 1 --2- 172.21.15.33:0/2707616191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808108130 0x7f680819e2a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:07.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:06.149+0000 7f680d8b2700 1 -- 172.21.15.33:0/2707616191 >> 172.21.15.33:0/2707616191 conn(0x7f68081036f0 msgr2=0x7f6808104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:07.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:06.149+0000 7f680d8b2700 1 -- 172.21.15.33:0/2707616191 shutdown_connections 2024-04-02T13:29:07.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:06.149+0000 7f680d8b2700 1 -- 172.21.15.33:0/2707616191 wait complete. 2024-04-02T13:29:07.163 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling the dashboard module... 2024-04-02T13:29:07.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:07 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2707616191' entity='client.admin' 2024-04-02T13:29:08.990 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:08 smithi033 ceph-mon[32390]: from='mgr.14120 172.21.15.33:0/3218318227' entity='mgr.x' 2024-04-02T13:29:08.991 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:08 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1568021887' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-02T13:29:08.991 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:08 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: ignoring --setuser ceph since I am not root 2024-04-02T13:29:08.991 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:08 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: ignoring --setgroup ceph since I am not root 2024-04-02T13:29:09.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.574+0000 7fe3afc0f700 1 Processor -- start 2024-04-02T13:29:09.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.575+0000 7fe3afc0f700 1 -- start start 2024-04-02T13:29:09.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.575+0000 7fe3afc0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 0x7fe3a8107440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:09.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.575+0000 7fe3afc0f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8107a10 con 0x7fe3a8107020 2024-04-02T13:29:09.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.575+0000 7fe3ad9ab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 0x7fe3a8107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:09.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.575+0000 7fe3ad9ab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 0x7fe3a8107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53996/0 (socket says 172.21.15.33:53996) 2024-04-02T13:29:09.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.575+0000 7fe3ad9ab700 1 -- 172.21.15.33:0/1304411462 learned_addr learned my addr 172.21.15.33:0/1304411462 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:09.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.575+0000 7fe3ad9ab700 1 -- 172.21.15.33:0/1304411462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a8108220 con 0x7fe3a8107020 2024-04-02T13:29:09.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.576+0000 7fe3ad9ab700 1 --2- 172.21.15.33:0/1304411462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 0x7fe3a8107440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3980098a0 tx=0x7fe398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31616ab31fc326b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:09.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.576+0000 7fe3ac9a9700 1 -- 172.21.15.33:0/1304411462 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe398004160 con 0x7fe3a8107020 2024-04-02T13:29:09.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.576+0000 7fe3ac9a9700 1 -- 172.21.15.33:0/1304411462 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe398004300 con 0x7fe3a8107020 2024-04-02T13:29:09.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.576+0000 7fe3ac9a9700 1 -- 172.21.15.33:0/1304411462 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe398011b10 con 0x7fe3a8107020 2024-04-02T13:29:09.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.576+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1304411462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 msgr2=0x7fe3a8107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:09.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.576+0000 7fe3afc0f700 1 --2- 172.21.15.33:0/1304411462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 0x7fe3a8107440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3980098a0 tx=0x7fe398009bb0 comp rx=0 tx=0).stop 2024-04-02T13:29:09.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.576+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1304411462 shutdown_connections 2024-04-02T13:29:09.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.576+0000 7fe3afc0f700 1 --2- 172.21.15.33:0/1304411462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 0x7fe3a8107440 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.576+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1304411462 >> 172.21.15.33:0/1304411462 conn(0x7fe3a81025e0 msgr2=0x7fe3a8104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.576+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1304411462 shutdown_connections 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.576+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1304411462 wait complete. 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.577+0000 7fe3afc0f700 1 Processor -- start 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.577+0000 7fe3afc0f700 1 -- start start 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.577+0000 7fe3afc0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 0x7fe3a819c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.577+0000 7fe3afc0f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a810b5d0 con 0x7fe3a8107020 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.577+0000 7fe3ad9ab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 0x7fe3a819c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.577+0000 7fe3ad9ab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 0x7fe3a819c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53998/0 (socket says 172.21.15.33:53998) 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.577+0000 7fe3ad9ab700 1 -- 172.21.15.33:0/1568021887 learned_addr learned my addr 172.21.15.33:0/1568021887 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.578+0000 7fe3ad9ab700 1 -- 172.21.15.33:0/1568021887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a819c5e0 con 0x7fe3a8107020 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.578+0000 7fe3ad9ab700 1 --2- 172.21.15.33:0/1568021887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 0x7fe3a819c0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe398006af0 tx=0x7fe398011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.578+0000 7fe39effd700 1 -- 172.21.15.33:0/1568021887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3980038b0 con 0x7fe3a8107020 2024-04-02T13:29:09.293 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.578+0000 7fe39effd700 1 -- 172.21.15.33:0/1568021887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe39801a430 con 0x7fe3a8107020 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.578+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1568021887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a819c870 con 0x7fe3a8107020 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.578+0000 7fe39effd700 1 -- 172.21.15.33:0/1568021887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe398019530 con 0x7fe3a8107020 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.578+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1568021887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a819f560 con 0x7fe3a8107020 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.579+0000 7fe39effd700 1 -- 172.21.15.33:0/1568021887 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe398003a50 con 0x7fe3a8107020 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.579+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1568021887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a8063c20 con 0x7fe3a8107020 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.579+0000 7fe39effd700 1 --2- 172.21.15.33:0/1568021887 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7fe394039620 0x7fe39403bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.579+0000 7fe39effd700 1 -- 172.21.15.33:0/1568021887 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe39804d9d0 con 0x7fe3a8107020 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.579+0000 7fe3ad1aa700 1 --2- 172.21.15.33:0/1568021887 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7fe394039620 0x7fe39403bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.579+0000 7fe3ad1aa700 1 --2- 172.21.15.33:0/1568021887 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7fe394039620 0x7fe39403bae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4006fd0 tx=0x7fe3a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.584+0000 7fe39effd700 1 -- 172.21.15.33:0/1568021887 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe398003d40 con 0x7fe3a8107020 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:07.858+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1568021887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fe3a802dfc0 con 0x7fe3a8107020 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:08.666+0000 7fe39effd700 1 -- 172.21.15.33:0/1568021887 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fe398023e00 con 0x7fe3a8107020 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:08.669+0000 7fe39effd700 1 -- 172.21.15.33:0/1568021887 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fe3980197f0 con 0x7fe3a8107020 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:08.669+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1568021887 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7fe394039620 msgr2=0x7fe39403bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:09.294 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:08.669+0000 7fe3afc0f700 1 --2- 172.21.15.33:0/1568021887 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7fe394039620 0x7fe39403bae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4006fd0 tx=0x7fe3a4006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:09.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:08.669+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1568021887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 msgr2=0x7fe3a819c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:09.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:08.669+0000 7fe3afc0f700 1 --2- 172.21.15.33:0/1568021887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 0x7fe3a819c0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe398006af0 tx=0x7fe398011bd0 comp rx=0 tx=0).stop 2024-04-02T13:29:09.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:08.670+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1568021887 shutdown_connections 2024-04-02T13:29:09.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:08.670+0000 7fe3afc0f700 1 --2- 172.21.15.33:0/1568021887 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7fe394039620 0x7fe39403bae0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:09.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:08.670+0000 7fe3afc0f700 1 --2- 172.21.15.33:0/1568021887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3a8107020 0x7fe3a819c0a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:09.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:08.670+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1568021887 >> 172.21.15.33:0/1568021887 conn(0x7fe3a81025e0 msgr2=0x7fe3a8192c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:09.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:08.670+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1568021887 shutdown_connections 2024-04-02T13:29:09.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:08.670+0000 7fe3afc0f700 1 -- 172.21.15.33:0/1568021887 wait complete. 2024-04-02T13:29:09.539 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:09 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:09.266+0000 7f854c2b2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:29:09.539 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:09 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:09.375+0000 7f854c2b2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:29:09.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:09 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1568021887' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-02T13:29:09.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:09 smithi033 ceph-mon[32390]: mgrmap e9: x(active, since 20s) 2024-04-02T13:29:10.155 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:10 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:10.054+0000 7f854c2b2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:29:10.155 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:10 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:10.152+0000 7f854c2b2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:29:10.486 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:10 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:10.297+0000 7f854c2b2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:29:10.487 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:10 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:10.396+0000 7f854c2b2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:29:10.487 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:10 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:10.483+0000 7f854c2b2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:29:10.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:29:10.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-02T13:29:10.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T13:29:10.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-02T13:29:10.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T13:29:10.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:29:10.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.744+0000 7f0612dc9700 1 Processor -- start 2024-04-02T13:29:10.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.744+0000 7f0612dc9700 1 -- start start 2024-04-02T13:29:10.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.744+0000 7f0612dc9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 0x7f060c108540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:10.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.744+0000 7f0612dc9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f060c108b10 con 0x7f060c108120 2024-04-02T13:29:10.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.744+0000 7f0610b65700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 0x7f060c108540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:10.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.744+0000 7f0610b65700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 0x7f060c108540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54030/0 (socket says 172.21.15.33:54030) 2024-04-02T13:29:10.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.744+0000 7f0610b65700 1 -- 172.21.15.33:0/2691697904 learned_addr learned my addr 172.21.15.33:0/2691697904 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.744+0000 7f0610b65700 1 -- 172.21.15.33:0/2691697904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c109320 con 0x7f060c108120 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.745+0000 7f0610b65700 1 --2- 172.21.15.33:0/2691697904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 0x7f060c108540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06000098a0 tx=0x7f0600009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4631b022b77fe012 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.745+0000 7f060b7fe700 1 -- 172.21.15.33:0/2691697904 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0600004160 con 0x7f060c108120 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.745+0000 7f060b7fe700 1 -- 172.21.15.33:0/2691697904 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0600004300 con 0x7f060c108120 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.745+0000 7f060b7fe700 1 -- 172.21.15.33:0/2691697904 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0600011b10 con 0x7f060c108120 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.745+0000 7f0612dc9700 1 -- 172.21.15.33:0/2691697904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 msgr2=0x7f060c108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.745+0000 7f0612dc9700 1 --2- 172.21.15.33:0/2691697904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 0x7f060c108540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06000098a0 tx=0x7f0600009bb0 comp rx=0 tx=0).stop 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.745+0000 7f0612dc9700 1 -- 172.21.15.33:0/2691697904 shutdown_connections 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.745+0000 7f0612dc9700 1 --2- 172.21.15.33:0/2691697904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 0x7f060c108540 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.745+0000 7f0612dc9700 1 -- 172.21.15.33:0/2691697904 >> 172.21.15.33:0/2691697904 conn(0x7f060c1036e0 msgr2=0x7f060c105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.745+0000 7f0612dc9700 1 -- 172.21.15.33:0/2691697904 shutdown_connections 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.745+0000 7f0612dc9700 1 -- 172.21.15.33:0/2691697904 wait complete. 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.746+0000 7f0612dc9700 1 Processor -- start 2024-04-02T13:29:10.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.746+0000 7f0612dc9700 1 -- start start 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.746+0000 7f0612dc9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 0x7f060c19c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.746+0000 7f0612dc9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f060c10c6d0 con 0x7f060c108120 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.746+0000 7f0610b65700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 0x7f060c19c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.746+0000 7f0610b65700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 0x7f060c19c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54038/0 (socket says 172.21.15.33:54038) 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.746+0000 7f0610b65700 1 -- 172.21.15.33:0/1437817153 learned_addr learned my addr 172.21.15.33:0/1437817153 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.747+0000 7f0610b65700 1 -- 172.21.15.33:0/1437817153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c19c5f0 con 0x7f060c108120 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.747+0000 7f0610b65700 1 --2- 172.21.15.33:0/1437817153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 0x7f060c19c0b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0600006af0 tx=0x7f0600011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.747+0000 7f0609ffb700 1 -- 172.21.15.33:0/1437817153 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06000038b0 con 0x7f060c108120 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.747+0000 7f0609ffb700 1 -- 172.21.15.33:0/1437817153 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f060001a430 con 0x7f060c108120 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.747+0000 7f0609ffb700 1 -- 172.21.15.33:0/1437817153 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0600019530 con 0x7f060c108120 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.747+0000 7f0612dc9700 1 -- 172.21.15.33:0/1437817153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060c19c880 con 0x7f060c108120 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.747+0000 7f0612dc9700 1 -- 172.21.15.33:0/1437817153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060c19f570 con 0x7f060c108120 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.748+0000 7f0612dc9700 1 -- 172.21.15.33:0/1437817153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f060c063c20 con 0x7f060c108120 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.748+0000 7f0609ffb700 1 -- 172.21.15.33:0/1437817153 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f06000197f0 con 0x7f060c108120 2024-04-02T13:29:10.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.748+0000 7f0609ffb700 1 --2- 172.21.15.33:0/1437817153 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f05fc039670 0x7f05fc03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.748+0000 7f0609ffb700 1 -- 172.21.15.33:0/1437817153 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f060004da50 con 0x7f060c108120 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.749+0000 7f060bfff700 1 -- 172.21.15.33:0/1437817153 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f05fc039670 msgr2=0x7f05fc03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2666001000 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.749+0000 7f060bfff700 1 --2- 172.21.15.33:0/1437817153 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f05fc039670 0x7f05fc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.753+0000 7f0609ffb700 1 -- 172.21.15.33:0/1437817153 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0600023e00 con 0x7f060c108120 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.950+0000 7f060bfff700 1 -- 172.21.15.33:0/1437817153 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f05fc039670 msgr2=0x7f05fc03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2666001000 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:09.950+0000 7f060bfff700 1 --2- 172.21.15.33:0/1437817153 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f05fc039670 0x7f05fc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:10.038+0000 7f0612dc9700 1 -- 172.21.15.33:0/1437817153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f060c19f740 con 0x7f060c108120 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:10.038+0000 7f0609ffb700 1 -- 172.21.15.33:0/1437817153 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f060000fda0 con 0x7f060c108120 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:10.040+0000 7f0612dc9700 1 -- 172.21.15.33:0/1437817153 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f05fc039670 msgr2=0x7f05fc03bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:10.040+0000 7f0612dc9700 1 --2- 172.21.15.33:0/1437817153 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f05fc039670 0x7f05fc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:10.040+0000 7f0612dc9700 1 -- 172.21.15.33:0/1437817153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 msgr2=0x7f060c19c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:10.040+0000 7f0612dc9700 1 --2- 172.21.15.33:0/1437817153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 0x7f060c19c0b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0600006af0 tx=0x7f0600011bd0 comp rx=0 tx=0).stop 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:10.040+0000 7f0612dc9700 1 -- 172.21.15.33:0/1437817153 shutdown_connections 2024-04-02T13:29:10.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:10.040+0000 7f0612dc9700 1 --2- 172.21.15.33:0/1437817153 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f05fc039670 0x7f05fc03bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:10.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:10.040+0000 7f0612dc9700 1 --2- 172.21.15.33:0/1437817153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f060c108120 0x7f060c19c0b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:10.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:10.040+0000 7f0612dc9700 1 -- 172.21.15.33:0/1437817153 >> 172.21.15.33:0/1437817153 conn(0x7f060c1036e0 msgr2=0x7f060c1041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:10.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:10.040+0000 7f0612dc9700 1 -- 172.21.15.33:0/1437817153 shutdown_connections 2024-04-02T13:29:10.713 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:10.040+0000 7f0612dc9700 1 -- 172.21.15.33:0/1437817153 wait complete. 2024-04-02T13:29:10.713 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-04-02T13:29:10.713 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 9... 2024-04-02T13:29:10.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:10 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1437817153' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T13:29:11.027 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:10 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:10.764+0000 7f854c2b2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:29:11.027 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:10 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:10.878+0000 7f854c2b2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:29:11.027 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:10 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:10.968+0000 7f854c2b2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:29:11.334 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:11 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:11.067+0000 7f854c2b2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:29:11.334 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:11 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:11.249+0000 7f854c2b2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:29:12.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:12 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:12.242+0000 7f854c2b2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:29:12.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:12 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:12.378+0000 7f854c2b2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:29:12.815 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:12 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:12.719+0000 7f854c2b2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:29:12.815 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:12 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:12.811+0000 7f854c2b2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:29:13.210 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:13 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:13.015+0000 7f854c2b2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:29:13.210 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:13 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:13.207+0000 7f854c2b2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:29:13.769 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:13 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:13.504+0000 7f854c2b2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:29:13.769 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:13 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:13.766+0000 7f854c2b2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:29:14.138 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:14 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:14.012+0000 7f854c2b2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:29:15.254 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:14 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:14.848+0000 7f854c2b2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:29:15.527 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:15.437+0000 7f854c2b2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:29:15.780 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:15.544+0000 7f854c2b2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:29:15.780 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:29:15.776+0000 7f854c2b2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:29:16.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-mon[32390]: Active manager daemon x restarted 2024-04-02T13:29:16.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-mon[32390]: Activating manager daemon x 2024-04-02T13:29:16.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-mon[32390]: osdmap e3: 0 total, 0 up, 0 in 2024-04-02T13:29:16.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-mon[32390]: mgrmap e10: x(active, starting, since 0.0038973s) 2024-04-02T13:29:16.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:29:16.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:29:16.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:29:16.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:29:16.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:29:16.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-mon[32390]: Manager daemon x is now available 2024-04-02T13:29:16.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:16.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:29:17.051 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:29:17.051 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:16 smithi033 ceph-mon[32390]: [02/Apr/2024:13:29:16] ENGINE Bus STARTING 2024-04-02T13:29:17.051 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:16 smithi033 ceph-mon[32390]: [02/Apr/2024:13:29:16] ENGINE Serving on http://172.21.15.33:8765 2024-04-02T13:29:17.051 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:16 smithi033 ceph-mon[32390]: mgrmap e11: x(active, since 1.00604s) 2024-04-02T13:29:17.963 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:29:17.963 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-02T13:29:17.963 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.125+0000 7f13fc712700 1 Processor -- start 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.125+0000 7f13fc712700 1 -- start start 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.125+0000 7f13fc712700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 0x7f13f407cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.125+0000 7f13fc712700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13f407d160 con 0x7f13f407e730 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.126+0000 7f13fa4ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 0x7f13f407cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.126+0000 7f13fa4ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 0x7f13f407cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54050/0 (socket says 172.21.15.33:54050) 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.126+0000 7f13fa4ae700 1 -- 172.21.15.33:0/1939652919 learned_addr learned my addr 172.21.15.33:0/1939652919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.126+0000 7f13fa4ae700 1 -- 172.21.15.33:0/1939652919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f407d2e0 con 0x7f13f407e730 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.126+0000 7f13fa4ae700 1 --2- 172.21.15.33:0/1939652919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 0x7f13f407cb90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13f00098a0 tx=0x7f13f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2ba7e7d98ba948 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.126+0000 7f13f94ac700 1 -- 172.21.15.33:0/1939652919 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13f0004160 con 0x7f13f407e730 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.127+0000 7f13f94ac700 1 -- 172.21.15.33:0/1939652919 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f13f0004300 con 0x7f13f407e730 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.127+0000 7f13f94ac700 1 -- 172.21.15.33:0/1939652919 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13f0011b10 con 0x7f13f407e730 2024-04-02T13:29:17.964 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.127+0000 7f13fc712700 1 -- 172.21.15.33:0/1939652919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 msgr2=0x7f13f407cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:17.965 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.127+0000 7f13fc712700 1 --2- 172.21.15.33:0/1939652919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 0x7f13f407cb90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13f00098a0 tx=0x7f13f0009bb0 comp rx=0 tx=0).stop 2024-04-02T13:29:17.965 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.127+0000 7f13fc712700 1 -- 172.21.15.33:0/1939652919 shutdown_connections 2024-04-02T13:29:17.965 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.127+0000 7f13fc712700 1 --2- 172.21.15.33:0/1939652919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 0x7f13f407cb90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:17.965 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.127+0000 7f13fc712700 1 -- 172.21.15.33:0/1939652919 >> 172.21.15.33:0/1939652919 conn(0x7f13f41036e0 msgr2=0x7f13f4105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:17.965 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.127+0000 7f13fc712700 1 -- 172.21.15.33:0/1939652919 shutdown_connections 2024-04-02T13:29:17.965 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.127+0000 7f13fc712700 1 -- 172.21.15.33:0/1939652919 wait complete. 2024-04-02T13:29:17.965 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.128+0000 7f13fc712700 1 Processor -- start 2024-04-02T13:29:17.965 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.128+0000 7f13fc712700 1 -- start start 2024-04-02T13:29:17.965 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.128+0000 7f13fc712700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 0x7f13f419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:17.965 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.128+0000 7f13fc712700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13f410e780 con 0x7f13f407e730 2024-04-02T13:29:17.965 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.128+0000 7f13fa4ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 0x7f13f419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:17.966 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.128+0000 7f13fa4ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 0x7f13f419e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54054/0 (socket says 172.21.15.33:54054) 2024-04-02T13:29:17.966 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.128+0000 7f13fa4ae700 1 -- 172.21.15.33:0/588195255 learned_addr learned my addr 172.21.15.33:0/588195255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:17.966 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.129+0000 7f13fa4ae700 1 -- 172.21.15.33:0/588195255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f419e830 con 0x7f13f407e730 2024-04-02T13:29:17.966 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.129+0000 7f13fa4ae700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 0x7f13f419e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f13f0006af0 tx=0x7f13f0011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:17.966 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.129+0000 7f13e77fe700 1 -- 172.21.15.33:0/588195255 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13f00038b0 con 0x7f13f407e730 2024-04-02T13:29:17.966 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.129+0000 7f13e77fe700 1 -- 172.21.15.33:0/588195255 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f13f001a430 con 0x7f13f407e730 2024-04-02T13:29:17.966 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.129+0000 7f13fc712700 1 -- 172.21.15.33:0/588195255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f419eac0 con 0x7f13f407e730 2024-04-02T13:29:17.966 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.129+0000 7f13e77fe700 1 -- 172.21.15.33:0/588195255 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13f0019530 con 0x7f13f407e730 2024-04-02T13:29:17.966 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.129+0000 7f13fc712700 1 -- 172.21.15.33:0/588195255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f41a17b0 con 0x7f13f407e730 2024-04-02T13:29:17.966 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.130+0000 7f13e77fe700 1 -- 172.21.15.33:0/588195255 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f13f00197f0 con 0x7f13f407e730 2024-04-02T13:29:17.966 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.130+0000 7f13e77fe700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 0x7f13e003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:17.966 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.130+0000 7f13fc712700 1 -- 172.21.15.33:0/588195255 --> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f13f4063c20 con 0x7f13e0039670 2024-04-02T13:29:17.966 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.130+0000 7f13e77fe700 1 -- 172.21.15.33:0/588195255 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f13f004db30 con 0x7f13f407e730 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.130+0000 7f13f9cad700 1 -- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 msgr2=0x7f13e003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2666001000 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.130+0000 7f13f9cad700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 0x7f13e003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.331+0000 7f13f9cad700 1 -- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 msgr2=0x7f13e003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2666001000 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.331+0000 7f13f9cad700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 0x7f13e003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.732+0000 7f13f9cad700 1 -- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 msgr2=0x7f13e003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2666001000 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:11.732+0000 7f13f9cad700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 0x7f13e003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:12.533+0000 7f13f9cad700 1 -- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 msgr2=0x7f13e003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2666001000 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:12.533+0000 7f13f9cad700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 0x7f13e003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:14.134+0000 7f13f9cad700 1 -- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 msgr2=0x7f13e003bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2666001000 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:14.134+0000 7f13f9cad700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 0x7f13e003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:15.780+0000 7f13e77fe700 1 -- 172.21.15.33:0/588195255 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f13f002d960 con 0x7f13f407e730 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:15.780+0000 7f13e77fe700 1 -- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 msgr2=0x7f13e003bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:15.780+0000 7f13e77fe700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 0x7f13e003bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:17.967 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.781+0000 7f13e77fe700 1 -- 172.21.15.33:0/588195255 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f13f00205d0 con 0x7f13f407e730 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.782+0000 7f13e77fe700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f13e003cf00 0x7f13e003f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.782+0000 7f13e77fe700 1 -- 172.21.15.33:0/588195255 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f13f002be50 con 0x7f13e003cf00 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.782+0000 7f13f9cad700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f13e003cf00 0x7f13e003f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.783+0000 7f13f9cad700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f13e003cf00 0x7f13e003f2f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f13e8003a10 tx=0x7f13e80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.784+0000 7f13e77fe700 1 -- 172.21.15.33:0/588195255 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f13f002be50 con 0x7f13e003cf00 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.790+0000 7f13fc712700 1 -- 172.21.15.33:0/588195255 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f13f41048e0 con 0x7f13e003cf00 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.791+0000 7f13e77fe700 1 -- 172.21.15.33:0/588195255 <== mgr.14152 v2:172.21.15.33:6800/2562306694 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f13f41048e0 con 0x7f13e003cf00 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.791+0000 7f13fc712700 1 -- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f13e003cf00 msgr2=0x7f13e003f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.791+0000 7f13fc712700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f13e003cf00 0x7f13e003f2f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f13e8003a10 tx=0x7f13e80092b0 comp rx=0 tx=0).stop 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.791+0000 7f13fc712700 1 -- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 msgr2=0x7f13f419e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.791+0000 7f13fc712700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 0x7f13f419e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f13f0006af0 tx=0x7f13f0011bd0 comp rx=0 tx=0).stop 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.791+0000 7f13fc712700 1 -- 172.21.15.33:0/588195255 shutdown_connections 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.791+0000 7f13fc712700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f13e003cf00 0x7f13e003f2f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.791+0000 7f13fc712700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407e730 0x7f13f419e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.791+0000 7f13fc712700 1 --2- 172.21.15.33:0/588195255 >> [v2:172.21.15.33:6800/2666001000,v1:172.21.15.33:6801/2666001000] conn(0x7f13e0039670 0x7f13e003bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:17.969 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.791+0000 7f13fc712700 1 -- 172.21.15.33:0/588195255 >> 172.21.15.33:0/588195255 conn(0x7f13f41036e0 msgr2=0x7f13f41041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:17.969 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.791+0000 7f13fc712700 1 -- 172.21.15.33:0/588195255 shutdown_connections 2024-04-02T13:29:17.969 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:16.791+0000 7f13fc712700 1 -- 172.21.15.33:0/588195255 wait complete. 2024-04-02T13:29:17.969 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 9 is available 2024-04-02T13:29:17.969 INFO:teuthology.orchestra.run.smithi033.stdout:Generating a dashboard self-signed certificate... 2024-04-02T13:29:18.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:18 smithi033 ceph-mon[32390]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-02T13:29:18.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:18 smithi033 ceph-mon[32390]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-02T13:29:18.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:18 smithi033 ceph-mon[32390]: [02/Apr/2024:13:29:16] ENGINE Serving on https://172.21.15.33:7150 2024-04-02T13:29:18.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:18 smithi033 ceph-mon[32390]: [02/Apr/2024:13:29:16] ENGINE Bus STARTED 2024-04-02T13:29:18.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:19.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-02T13:29:19.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.393+0000 7efd79e63700 1 Processor -- start 2024-04-02T13:29:19.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.393+0000 7efd79e63700 1 -- start start 2024-04-02T13:29:19.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.393+0000 7efd79e63700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 0x7efd7410b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:19.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.393+0000 7efd79e63700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd7407c660 con 0x7efd74108c70 2024-04-02T13:29:19.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.393+0000 7efd737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 0x7efd7410b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:19.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.393+0000 7efd737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 0x7efd7410b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36550/0 (socket says 172.21.15.33:36550) 2024-04-02T13:29:19.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.393+0000 7efd737fe700 1 -- 172.21.15.33:0/2735332247 learned_addr learned my addr 172.21.15.33:0/2735332247 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:19.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.393+0000 7efd737fe700 1 -- 172.21.15.33:0/2735332247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7410b5a0 con 0x7efd74108c70 2024-04-02T13:29:19.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.394+0000 7efd737fe700 1 --2- 172.21.15.33:0/2735332247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 0x7efd7410b060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd640098a0 tx=0x7efd64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56af7d24dbc57b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:19.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.394+0000 7efd727fc700 1 -- 172.21.15.33:0/2735332247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd64004160 con 0x7efd74108c70 2024-04-02T13:29:19.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.394+0000 7efd727fc700 1 -- 172.21.15.33:0/2735332247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efd64004300 con 0x7efd74108c70 2024-04-02T13:29:19.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.394+0000 7efd727fc700 1 -- 172.21.15.33:0/2735332247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd64011b10 con 0x7efd74108c70 2024-04-02T13:29:19.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.394+0000 7efd79e63700 1 -- 172.21.15.33:0/2735332247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 msgr2=0x7efd7410b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:19.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.394+0000 7efd79e63700 1 --2- 172.21.15.33:0/2735332247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 0x7efd7410b060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd640098a0 tx=0x7efd64009bb0 comp rx=0 tx=0).stop 2024-04-02T13:29:19.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.395+0000 7efd79e63700 1 -- 172.21.15.33:0/2735332247 shutdown_connections 2024-04-02T13:29:19.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.395+0000 7efd79e63700 1 --2- 172.21.15.33:0/2735332247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 0x7efd7410b060 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:19.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.395+0000 7efd79e63700 1 -- 172.21.15.33:0/2735332247 >> 172.21.15.33:0/2735332247 conn(0x7efd741025e0 msgr2=0x7efd74104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:19.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.395+0000 7efd79e63700 1 -- 172.21.15.33:0/2735332247 shutdown_connections 2024-04-02T13:29:19.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.395+0000 7efd79e63700 1 -- 172.21.15.33:0/2735332247 wait complete. 2024-04-02T13:29:19.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.395+0000 7efd79e63700 1 Processor -- start 2024-04-02T13:29:19.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.395+0000 7efd79e63700 1 -- start start 2024-04-02T13:29:19.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.395+0000 7efd79e63700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 0x7efd7419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:19.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.395+0000 7efd79e63700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd7410c540 con 0x7efd74108c70 2024-04-02T13:29:19.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.396+0000 7efd737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 0x7efd7419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:19.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.396+0000 7efd737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 0x7efd7419bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36558/0 (socket says 172.21.15.33:36558) 2024-04-02T13:29:19.579 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.396+0000 7efd737fe700 1 -- 172.21.15.33:0/630691393 learned_addr learned my addr 172.21.15.33:0/630691393 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:19.579 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.396+0000 7efd737fe700 1 -- 172.21.15.33:0/630691393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7419c4f0 con 0x7efd74108c70 2024-04-02T13:29:19.579 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.396+0000 7efd737fe700 1 --2- 172.21.15.33:0/630691393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 0x7efd7419bfb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efd64011670 tx=0x7efd64011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:19.579 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.396+0000 7efd70ff9700 1 -- 172.21.15.33:0/630691393 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd64003930 con 0x7efd74108c70 2024-04-02T13:29:19.579 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.396+0000 7efd70ff9700 1 -- 172.21.15.33:0/630691393 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efd64003ad0 con 0x7efd74108c70 2024-04-02T13:29:19.579 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.396+0000 7efd79e63700 1 -- 172.21.15.33:0/630691393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7419c780 con 0x7efd74108c70 2024-04-02T13:29:19.579 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.396+0000 7efd70ff9700 1 -- 172.21.15.33:0/630691393 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd64019550 con 0x7efd74108c70 2024-04-02T13:29:19.579 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.396+0000 7efd79e63700 1 -- 172.21.15.33:0/630691393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7419f470 con 0x7efd74108c70 2024-04-02T13:29:19.579 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.397+0000 7efd70ff9700 1 -- 172.21.15.33:0/630691393 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7efd64019800 con 0x7efd74108c70 2024-04-02T13:29:19.579 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.397+0000 7efd79e63700 1 -- 172.21.15.33:0/630691393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd74063c20 con 0x7efd74108c70 2024-04-02T13:29:19.580 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.397+0000 7efd70ff9700 1 --2- 172.21.15.33:0/630691393 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7efd5c0395a0 0x7efd5c03ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:19.580 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.397+0000 7efd70ff9700 1 -- 172.21.15.33:0/630691393 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efd6404da60 con 0x7efd74108c70 2024-04-02T13:29:19.580 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.397+0000 7efd72ffd700 1 --2- 172.21.15.33:0/630691393 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7efd5c0395a0 0x7efd5c03ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:19.580 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.398+0000 7efd72ffd700 1 --2- 172.21.15.33:0/630691393 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7efd5c0395a0 0x7efd5c03ba60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efd68006fd0 tx=0x7efd68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:19.580 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.402+0000 7efd70ff9700 1 -- 172.21.15.33:0/630691393 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efd64003c80 con 0x7efd74108c70 2024-04-02T13:29:19.580 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.595+0000 7efd70ff9700 1 -- 172.21.15.33:0/630691393 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7efd6402d960 con 0x7efd74108c70 2024-04-02T13:29:19.580 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.621+0000 7efd79e63700 1 -- 172.21.15.33:0/630691393 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7efd741036d0 con 0x7efd5c0395a0 2024-04-02T13:29:19.580 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.827+0000 7efd70ff9700 1 -- 172.21.15.33:0/630691393 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7efd741036d0 con 0x7efd5c0395a0 2024-04-02T13:29:19.580 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.829+0000 7efd79e63700 1 -- 172.21.15.33:0/630691393 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7efd5c0395a0 msgr2=0x7efd5c03ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:19.580 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.829+0000 7efd79e63700 1 --2- 172.21.15.33:0/630691393 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7efd5c0395a0 0x7efd5c03ba60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efd68006fd0 tx=0x7efd68006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:19.580 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.829+0000 7efd79e63700 1 -- 172.21.15.33:0/630691393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 msgr2=0x7efd7419bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:19.581 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.829+0000 7efd79e63700 1 --2- 172.21.15.33:0/630691393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 0x7efd7419bfb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efd64011670 tx=0x7efd64011bd0 comp rx=0 tx=0).stop 2024-04-02T13:29:19.581 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.829+0000 7efd79e63700 1 -- 172.21.15.33:0/630691393 shutdown_connections 2024-04-02T13:29:19.581 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.829+0000 7efd79e63700 1 --2- 172.21.15.33:0/630691393 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7efd5c0395a0 0x7efd5c03ba60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:19.581 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.829+0000 7efd79e63700 1 --2- 172.21.15.33:0/630691393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd74108c70 0x7efd7419bfb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:19.581 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.829+0000 7efd79e63700 1 -- 172.21.15.33:0/630691393 >> 172.21.15.33:0/630691393 conn(0x7efd741025e0 msgr2=0x7efd74102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:19.581 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.829+0000 7efd79e63700 1 -- 172.21.15.33:0/630691393 shutdown_connections 2024-04-02T13:29:19.581 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:18.829+0000 7efd79e63700 1 -- 172.21.15.33:0/630691393 wait complete. 2024-04-02T13:29:19.581 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial admin user... 2024-04-02T13:29:19.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:19 smithi033 ceph-mon[32390]: mgrmap e12: x(active, since 2s) 2024-04-02T13:29:19.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:19 smithi033 ceph-mon[32390]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:19.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:19 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:19.879 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:19 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:19.879 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:19 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:21.252 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$YHo7oLlmLkhvH9YvwNnr2uUS8J5P0xxssp16hMhXvrWjIsegPwCja", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1712064560, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-02T13:29:21.253 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.979+0000 7f0b4a16f700 1 Processor -- start 2024-04-02T13:29:21.253 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.979+0000 7f0b4a16f700 1 -- start start 2024-04-02T13:29:21.253 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.979+0000 7f0b4a16f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 0x7f0b44108540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:21.253 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.979+0000 7f0b4a16f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44108b10 con 0x7f0b44108120 2024-04-02T13:29:21.253 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.979+0000 7f0b437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 0x7f0b44108540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:21.253 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.980+0000 7f0b437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 0x7f0b44108540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36570/0 (socket says 172.21.15.33:36570) 2024-04-02T13:29:21.253 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.980+0000 7f0b437fe700 1 -- 172.21.15.33:0/1523531730 learned_addr learned my addr 172.21.15.33:0/1523531730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:21.253 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.980+0000 7f0b437fe700 1 -- 172.21.15.33:0/1523531730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b44109320 con 0x7f0b44108120 2024-04-02T13:29:21.253 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.980+0000 7f0b437fe700 1 --2- 172.21.15.33:0/1523531730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 0x7f0b44108540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009e80 tx=0x7f0b3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef8a029add638c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:21.253 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.980+0000 7f0b427fc700 1 -- 172.21.15.33:0/1523531730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34017070 con 0x7f0b44108120 2024-04-02T13:29:21.253 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.980+0000 7f0b427fc700 1 -- 172.21.15.33:0/1523531730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0b3400b8c0 con 0x7f0b44108120 2024-04-02T13:29:21.253 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.980+0000 7f0b427fc700 1 -- 172.21.15.33:0/1523531730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34004520 con 0x7f0b44108120 2024-04-02T13:29:21.253 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.981+0000 7f0b4a16f700 1 -- 172.21.15.33:0/1523531730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 msgr2=0x7f0b44108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:21.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.981+0000 7f0b4a16f700 1 --2- 172.21.15.33:0/1523531730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 0x7f0b44108540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009e80 tx=0x7f0b3400b330 comp rx=0 tx=0).stop 2024-04-02T13:29:21.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.981+0000 7f0b4a16f700 1 -- 172.21.15.33:0/1523531730 shutdown_connections 2024-04-02T13:29:21.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.981+0000 7f0b4a16f700 1 --2- 172.21.15.33:0/1523531730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 0x7f0b44108540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:21.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.981+0000 7f0b4a16f700 1 -- 172.21.15.33:0/1523531730 >> 172.21.15.33:0/1523531730 conn(0x7f0b441036e0 msgr2=0x7f0b44105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:21.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.981+0000 7f0b4a16f700 1 -- 172.21.15.33:0/1523531730 shutdown_connections 2024-04-02T13:29:21.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.981+0000 7f0b4a16f700 1 -- 172.21.15.33:0/1523531730 wait complete. 2024-04-02T13:29:21.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.981+0000 7f0b4a16f700 1 Processor -- start 2024-04-02T13:29:21.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.981+0000 7f0b4a16f700 1 -- start start 2024-04-02T13:29:21.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.982+0000 7f0b4a16f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 0x7f0b4407e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:21.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.982+0000 7f0b4a16f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4410c6d0 con 0x7f0b44108120 2024-04-02T13:29:21.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.982+0000 7f0b437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 0x7f0b4407e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:21.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.982+0000 7f0b437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 0x7f0b4407e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36580/0 (socket says 172.21.15.33:36580) 2024-04-02T13:29:21.254 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.982+0000 7f0b437fe700 1 -- 172.21.15.33:0/3662059786 learned_addr learned my addr 172.21.15.33:0/3662059786 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.982+0000 7f0b437fe700 1 -- 172.21.15.33:0/3662059786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4407e940 con 0x7f0b44108120 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.982+0000 7f0b437fe700 1 --2- 172.21.15.33:0/3662059786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 0x7f0b4407e400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009420 tx=0x7f0b3400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.982+0000 7f0b40ff9700 1 -- 172.21.15.33:0/3662059786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34017070 con 0x7f0b44108120 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.982+0000 7f0b4a16f700 1 -- 172.21.15.33:0/3662059786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4407cb30 con 0x7f0b44108120 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.982+0000 7f0b40ff9700 1 -- 172.21.15.33:0/3662059786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0b34022070 con 0x7f0b44108120 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.982+0000 7f0b40ff9700 1 -- 172.21.15.33:0/3662059786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b3401d690 con 0x7f0b44108120 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.982+0000 7f0b4a16f700 1 -- 172.21.15.33:0/3662059786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b4407d070 con 0x7f0b44108120 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.984+0000 7f0b40ff9700 1 -- 172.21.15.33:0/3662059786 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f0b3401d910 con 0x7f0b44108120 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.984+0000 7f0b4a16f700 1 -- 172.21.15.33:0/3662059786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b44063c20 con 0x7f0b44108120 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.984+0000 7f0b40ff9700 1 --2- 172.21.15.33:0/3662059786 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0b30039720 0x7f0b3003bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.984+0000 7f0b40ff9700 1 -- 172.21.15.33:0/3662059786 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0b3400f050 con 0x7f0b44108120 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.984+0000 7f0b42ffd700 1 --2- 172.21.15.33:0/3662059786 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0b30039720 0x7f0b3003bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.984+0000 7f0b42ffd700 1 --2- 172.21.15.33:0/3662059786 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0b30039720 0x7f0b3003bbe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c006fd0 tx=0x7f0b2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:21.255 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:19.989+0000 7f0b40ff9700 1 -- 172.21.15.33:0/3662059786 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0b34025360 con 0x7f0b44108120 2024-04-02T13:29:21.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:20.241+0000 7f0b4a16f700 1 -- 172.21.15.33:0/3662059786 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f0b4407d2f0 con 0x7f0b30039720 2024-04-02T13:29:21.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:20.488+0000 7f0b40ff9700 1 -- 172.21.15.33:0/3662059786 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f0b4407d2f0 con 0x7f0b30039720 2024-04-02T13:29:21.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:20.490+0000 7f0b4a16f700 1 -- 172.21.15.33:0/3662059786 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0b30039720 msgr2=0x7f0b3003bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:21.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:20.490+0000 7f0b4a16f700 1 --2- 172.21.15.33:0/3662059786 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0b30039720 0x7f0b3003bbe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c006fd0 tx=0x7f0b2c006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:21.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:20.490+0000 7f0b4a16f700 1 -- 172.21.15.33:0/3662059786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 msgr2=0x7f0b4407e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:21.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:20.490+0000 7f0b4a16f700 1 --2- 172.21.15.33:0/3662059786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 0x7f0b4407e400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009420 tx=0x7f0b3400bd50 comp rx=0 tx=0).stop 2024-04-02T13:29:21.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:20.490+0000 7f0b4a16f700 1 -- 172.21.15.33:0/3662059786 shutdown_connections 2024-04-02T13:29:21.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:20.490+0000 7f0b4a16f700 1 --2- 172.21.15.33:0/3662059786 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0b30039720 0x7f0b3003bbe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:21.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:20.490+0000 7f0b4a16f700 1 --2- 172.21.15.33:0/3662059786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b44108120 0x7f0b4407e400 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:21.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:20.490+0000 7f0b4a16f700 1 -- 172.21.15.33:0/3662059786 >> 172.21.15.33:0/3662059786 conn(0x7f0b441036e0 msgr2=0x7f0b44104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:21.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:20.490+0000 7f0b4a16f700 1 -- 172.21.15.33:0/3662059786 shutdown_connections 2024-04-02T13:29:21.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:20.491+0000 7f0b4a16f700 1 -- 172.21.15.33:0/3662059786 wait complete. 2024-04-02T13:29:21.257 INFO:teuthology.orchestra.run.smithi033.stdout:Fetching dashboard port number... 2024-04-02T13:29:21.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:21 smithi033 ceph-mon[32390]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:21.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:21 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:22.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:22 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2843422855' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-02T13:29:22.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 8443 2024-04-02T13:29:22.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.677+0000 7fdea290d700 1 Processor -- start 2024-04-02T13:29:22.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.677+0000 7fdea290d700 1 -- start start 2024-04-02T13:29:22.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.677+0000 7fdea290d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 0x7fde9c108560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:22.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.677+0000 7fdea290d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c108b30 con 0x7fde9c108140 2024-04-02T13:29:22.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.677+0000 7fde9bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 0x7fde9c108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:22.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.677+0000 7fde9bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 0x7fde9c108560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36582/0 (socket says 172.21.15.33:36582) 2024-04-02T13:29:22.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.677+0000 7fde9bfff700 1 -- 172.21.15.33:0/1593777033 learned_addr learned my addr 172.21.15.33:0/1593777033 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:22.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.677+0000 7fde9bfff700 1 -- 172.21.15.33:0/1593777033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c109340 con 0x7fde9c108140 2024-04-02T13:29:22.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.678+0000 7fde9bfff700 1 --2- 172.21.15.33:0/1593777033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 0x7fde9c108560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0098a0 tx=0x7fde8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c0f22a1bb3d1db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:22.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.678+0000 7fde9affd700 1 -- 172.21.15.33:0/1593777033 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde8c004160 con 0x7fde9c108140 2024-04-02T13:29:22.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.678+0000 7fde9affd700 1 -- 172.21.15.33:0/1593777033 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde8c004300 con 0x7fde9c108140 2024-04-02T13:29:22.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.678+0000 7fde9affd700 1 -- 172.21.15.33:0/1593777033 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde8c011b10 con 0x7fde9c108140 2024-04-02T13:29:22.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.678+0000 7fdea290d700 1 -- 172.21.15.33:0/1593777033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 msgr2=0x7fde9c108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:22.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.678+0000 7fdea290d700 1 --2- 172.21.15.33:0/1593777033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 0x7fde9c108560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0098a0 tx=0x7fde8c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:29:22.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.678+0000 7fdea290d700 1 -- 172.21.15.33:0/1593777033 shutdown_connections 2024-04-02T13:29:22.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.678+0000 7fdea290d700 1 --2- 172.21.15.33:0/1593777033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 0x7fde9c108560 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:22.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.678+0000 7fdea290d700 1 -- 172.21.15.33:0/1593777033 >> 172.21.15.33:0/1593777033 conn(0x7fde9c103700 msgr2=0x7fde9c105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:22.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.678+0000 7fdea290d700 1 -- 172.21.15.33:0/1593777033 shutdown_connections 2024-04-02T13:29:22.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.678+0000 7fdea290d700 1 -- 172.21.15.33:0/1593777033 wait complete. 2024-04-02T13:29:22.844 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.679+0000 7fdea290d700 1 Processor -- start 2024-04-02T13:29:22.844 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.679+0000 7fdea290d700 1 -- start start 2024-04-02T13:29:22.844 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.679+0000 7fdea290d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 0x7fde9c07e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:22.844 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.679+0000 7fdea290d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c10c6f0 con 0x7fde9c108140 2024-04-02T13:29:22.844 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.679+0000 7fde9bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 0x7fde9c07e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:22.844 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.679+0000 7fde9bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 0x7fde9c07e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36588/0 (socket says 172.21.15.33:36588) 2024-04-02T13:29:22.845 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.679+0000 7fde9bfff700 1 -- 172.21.15.33:0/2843422855 learned_addr learned my addr 172.21.15.33:0/2843422855 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:22.845 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.680+0000 7fde9bfff700 1 -- 172.21.15.33:0/2843422855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c07e940 con 0x7fde9c108140 2024-04-02T13:29:22.845 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.680+0000 7fde9bfff700 1 --2- 172.21.15.33:0/2843422855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 0x7fde9c07e400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fde8c006af0 tx=0x7fde8c011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:22.845 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.680+0000 7fde997fa700 1 -- 172.21.15.33:0/2843422855 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde8c003af0 con 0x7fde9c108140 2024-04-02T13:29:22.845 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.680+0000 7fdea290d700 1 -- 172.21.15.33:0/2843422855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c07cb30 con 0x7fde9c108140 2024-04-02T13:29:22.846 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.680+0000 7fde997fa700 1 -- 172.21.15.33:0/2843422855 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde8c003c90 con 0x7fde9c108140 2024-04-02T13:29:22.846 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.680+0000 7fde997fa700 1 -- 172.21.15.33:0/2843422855 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde8c024830 con 0x7fde9c108140 2024-04-02T13:29:22.846 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.680+0000 7fdea290d700 1 -- 172.21.15.33:0/2843422855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c07d070 con 0x7fde9c108140 2024-04-02T13:29:22.846 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.681+0000 7fde997fa700 1 -- 172.21.15.33:0/2843422855 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fde8c022020 con 0x7fde9c108140 2024-04-02T13:29:22.846 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.681+0000 7fde997fa700 1 --2- 172.21.15.33:0/2843422855 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fde840396c0 0x7fde8403bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:22.846 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.681+0000 7fde997fa700 1 -- 172.21.15.33:0/2843422855 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fde8c04d790 con 0x7fde9c108140 2024-04-02T13:29:22.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.681+0000 7fdea290d700 1 -- 172.21.15.33:0/2843422855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9c063c20 con 0x7fde9c108140 2024-04-02T13:29:22.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.681+0000 7fde9b7fe700 1 --2- 172.21.15.33:0/2843422855 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fde840396c0 0x7fde8403bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:22.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.682+0000 7fde9b7fe700 1 --2- 172.21.15.33:0/2843422855 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fde840396c0 0x7fde8403bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fde90006fd0 tx=0x7fde90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:22.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.686+0000 7fde997fa700 1 -- 172.21.15.33:0/2843422855 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fde8c02ad60 con 0x7fde9c108140 2024-04-02T13:29:22.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.894+0000 7fdea290d700 1 -- 172.21.15.33:0/2843422855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fde9c07d2f0 con 0x7fde9c108140 2024-04-02T13:29:22.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.895+0000 7fde997fa700 1 -- 172.21.15.33:0/2843422855 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fde8c02dc90 con 0x7fde9c108140 2024-04-02T13:29:22.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.897+0000 7fdea290d700 1 -- 172.21.15.33:0/2843422855 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fde840396c0 msgr2=0x7fde8403bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:22.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.897+0000 7fdea290d700 1 --2- 172.21.15.33:0/2843422855 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fde840396c0 0x7fde8403bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fde90006fd0 tx=0x7fde90006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:22.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.897+0000 7fdea290d700 1 -- 172.21.15.33:0/2843422855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 msgr2=0x7fde9c07e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:22.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.897+0000 7fdea290d700 1 --2- 172.21.15.33:0/2843422855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 0x7fde9c07e400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fde8c006af0 tx=0x7fde8c011bd0 comp rx=0 tx=0).stop 2024-04-02T13:29:22.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.897+0000 7fdea290d700 1 -- 172.21.15.33:0/2843422855 shutdown_connections 2024-04-02T13:29:22.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.897+0000 7fdea290d700 1 --2- 172.21.15.33:0/2843422855 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fde840396c0 0x7fde8403bb80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:22.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.897+0000 7fdea290d700 1 --2- 172.21.15.33:0/2843422855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c108140 0x7fde9c07e400 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:22.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.897+0000 7fdea290d700 1 -- 172.21.15.33:0/2843422855 >> 172.21.15.33:0/2843422855 conn(0x7fde9c103700 msgr2=0x7fde9c104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:22.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.897+0000 7fdea290d700 1 -- 172.21.15.33:0/2843422855 shutdown_connections 2024-04-02T13:29:22.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:21.897+0000 7fdea290d700 1 -- 172.21.15.33:0/2843422855 wait complete. 2024-04-02T13:29:22.850 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T13:29:22.850 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-02T13:29:22.856 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T13:29:22.856 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-02T13:29:22.857 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-02T13:29:22.857 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-02T13:29:22.857 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph Dashboard is now available at: 2024-04-02T13:29:22.858 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:29:22.858 INFO:teuthology.orchestra.run.smithi033.stdout: URL: https://smithi033.front.sepia.ceph.com:8443/ 2024-04-02T13:29:22.858 INFO:teuthology.orchestra.run.smithi033.stdout: User: admin 2024-04-02T13:29:22.858 INFO:teuthology.orchestra.run.smithi033.stdout: Password: n3ngtt1v95 2024-04-02T13:29:22.858 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:29:22.859 INFO:teuthology.orchestra.run.smithi033.stdout:Saving cluster configuration to /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config directory 2024-04-02T13:29:22.859 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling autotune for osd_memory_target 2024-04-02T13:29:23.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:23 smithi033 ceph-mon[32390]: mgrmap e13: x(active, since 6s) 2024-04-02T13:29:24.086 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.317+0000 7f5907795700 1 Processor -- start 2024-04-02T13:29:24.086 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.317+0000 7f5907795700 1 -- start start 2024-04-02T13:29:24.086 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.317+0000 7f5907795700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 0x7f5900107440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.317+0000 7f5907795700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5900107a10 con 0x7f5900107020 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.317+0000 7f5905531700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 0x7f5900107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.317+0000 7f5905531700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 0x7f5900107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36604/0 (socket says 172.21.15.33:36604) 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.317+0000 7f5905531700 1 -- 172.21.15.33:0/1207713235 learned_addr learned my addr 172.21.15.33:0/1207713235 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.318+0000 7f5905531700 1 -- 172.21.15.33:0/1207713235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5900108220 con 0x7f5900107020 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.318+0000 7f5905531700 1 --2- 172.21.15.33:0/1207713235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 0x7f5900107440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0098a0 tx=0x7f58fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d1a28c6cc9433b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.318+0000 7f58f3fff700 1 -- 172.21.15.33:0/1207713235 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58fc004160 con 0x7f5900107020 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.318+0000 7f58f3fff700 1 -- 172.21.15.33:0/1207713235 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f58fc004300 con 0x7f5900107020 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.318+0000 7f58f3fff700 1 -- 172.21.15.33:0/1207713235 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58fc011b10 con 0x7f5900107020 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.318+0000 7f5907795700 1 -- 172.21.15.33:0/1207713235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 msgr2=0x7f5900107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.318+0000 7f5907795700 1 --2- 172.21.15.33:0/1207713235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 0x7f5900107440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0098a0 tx=0x7f58fc009bb0 comp rx=0 tx=0).stop 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.318+0000 7f5907795700 1 -- 172.21.15.33:0/1207713235 shutdown_connections 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.318+0000 7f5907795700 1 --2- 172.21.15.33:0/1207713235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 0x7f5900107440 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:24.087 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.318+0000 7f5907795700 1 -- 172.21.15.33:0/1207713235 >> 172.21.15.33:0/1207713235 conn(0x7f59001025e0 msgr2=0x7f5900104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.319+0000 7f5907795700 1 -- 172.21.15.33:0/1207713235 shutdown_connections 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.319+0000 7f5907795700 1 -- 172.21.15.33:0/1207713235 wait complete. 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.319+0000 7f5907795700 1 Processor -- start 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.319+0000 7f5907795700 1 -- start start 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.319+0000 7f5907795700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 0x7f590007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.319+0000 7f5907795700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f590010b5d0 con 0x7f5900107020 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.319+0000 7f5905531700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 0x7f590007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.319+0000 7f5905531700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 0x7f590007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36618/0 (socket says 172.21.15.33:36618) 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.319+0000 7f5905531700 1 -- 172.21.15.33:0/3784174624 learned_addr learned my addr 172.21.15.33:0/3784174624 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.320+0000 7f5905531700 1 -- 172.21.15.33:0/3784174624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590019edb0 con 0x7f5900107020 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.320+0000 7f5905531700 1 --2- 172.21.15.33:0/3784174624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 0x7f590007c2e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f58fc006af0 tx=0x7f58fc011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.320+0000 7f58f27fc700 1 -- 172.21.15.33:0/3784174624 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58fc003af0 con 0x7f5900107020 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.320+0000 7f5907795700 1 -- 172.21.15.33:0/3784174624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590019ef80 con 0x7f5900107020 2024-04-02T13:29:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.320+0000 7f58f27fc700 1 -- 172.21.15.33:0/3784174624 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f58fc003c90 con 0x7f5900107020 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.320+0000 7f58f27fc700 1 -- 172.21.15.33:0/3784174624 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58fc024830 con 0x7f5900107020 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.320+0000 7f5907795700 1 -- 172.21.15.33:0/3784174624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f590019f320 con 0x7f5900107020 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.321+0000 7f58f27fc700 1 -- 172.21.15.33:0/3784174624 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f58fc022020 con 0x7f5900107020 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.321+0000 7f5907795700 1 -- 172.21.15.33:0/3784174624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5900063c20 con 0x7f5900107020 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.321+0000 7f58f27fc700 1 --2- 172.21.15.33:0/3784174624 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f58ec039760 0x7f58ec03bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.321+0000 7f58f27fc700 1 -- 172.21.15.33:0/3784174624 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f58fc04d860 con 0x7f5900107020 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.321+0000 7f5904d30700 1 --2- 172.21.15.33:0/3784174624 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f58ec039760 0x7f58ec03bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.322+0000 7f5904d30700 1 --2- 172.21.15.33:0/3784174624 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f58ec039760 0x7f58ec03bc20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f58f4006fd0 tx=0x7f58f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.326+0000 7f58f27fc700 1 -- 172.21.15.33:0/3784174624 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f58fc02a2a0 con 0x7f5900107020 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.534+0000 7f5907795700 1 -- 172.21.15.33:0/3784174624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f590019f5a0 con 0x7f5900107020 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.535+0000 7f58f27fc700 1 -- 172.21.15.33:0/3784174624 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f58fc02d3f0 con 0x7f5900107020 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.537+0000 7f5907795700 1 -- 172.21.15.33:0/3784174624 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f58ec039760 msgr2=0x7f58ec03bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.537+0000 7f5907795700 1 --2- 172.21.15.33:0/3784174624 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f58ec039760 0x7f58ec03bc20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f58f4006fd0 tx=0x7f58f4006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:24.089 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.537+0000 7f5907795700 1 -- 172.21.15.33:0/3784174624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 msgr2=0x7f590007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:24.090 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.537+0000 7f5907795700 1 --2- 172.21.15.33:0/3784174624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 0x7f590007c2e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f58fc006af0 tx=0x7f58fc011bd0 comp rx=0 tx=0).stop 2024-04-02T13:29:24.090 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.537+0000 7f5907795700 1 -- 172.21.15.33:0/3784174624 shutdown_connections 2024-04-02T13:29:24.090 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.537+0000 7f5907795700 1 --2- 172.21.15.33:0/3784174624 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f58ec039760 0x7f58ec03bc20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:24.090 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.537+0000 7f5907795700 1 --2- 172.21.15.33:0/3784174624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5900107020 0x7f590007c2e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:24.090 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.537+0000 7f5907795700 1 -- 172.21.15.33:0/3784174624 >> 172.21.15.33:0/3784174624 conn(0x7f59001025e0 msgr2=0x7f5900103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:24.090 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.537+0000 7f5907795700 1 -- 172.21.15.33:0/3784174624 shutdown_connections 2024-04-02T13:29:24.090 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:23.537+0000 7f5907795700 1 -- 172.21.15.33:0/3784174624 wait complete. 2024-04-02T13:29:25.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.450+0000 7f1ca61bb700 1 Processor -- start 2024-04-02T13:29:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.450+0000 7f1ca61bb700 1 -- start start 2024-04-02T13:29:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.450+0000 7f1ca61bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 0x7f1ca0106e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.450+0000 7f1ca61bb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca007c6a0 con 0x7f1ca0106a20 2024-04-02T13:29:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.450+0000 7f1c9f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 0x7f1ca0106e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.450+0000 7f1c9f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 0x7f1ca0106e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47924/0 (socket says 172.21.15.33:47924) 2024-04-02T13:29:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.450+0000 7f1c9f7fe700 1 -- 172.21.15.33:0/2430666012 learned_addr learned my addr 172.21.15.33:0/2430666012 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.450+0000 7f1c9f7fe700 1 -- 172.21.15.33:0/2430666012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca0107380 con 0x7f1ca0106a20 2024-04-02T13:29:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.451+0000 7f1c9f7fe700 1 --2- 172.21.15.33:0/2430666012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 0x7f1ca0106e40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1c900098a0 tx=0x7f1c90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151a732b2294a2ba server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.451+0000 7f1c9e7fc700 1 -- 172.21.15.33:0/2430666012 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c90004160 con 0x7f1ca0106a20 2024-04-02T13:29:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.451+0000 7f1c9e7fc700 1 -- 172.21.15.33:0/2430666012 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1c90004300 con 0x7f1ca0106a20 2024-04-02T13:29:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.451+0000 7f1c9e7fc700 1 -- 172.21.15.33:0/2430666012 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c90011b10 con 0x7f1ca0106a20 2024-04-02T13:29:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.451+0000 7f1ca61bb700 1 -- 172.21.15.33:0/2430666012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 msgr2=0x7f1ca0106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.451+0000 7f1ca61bb700 1 --2- 172.21.15.33:0/2430666012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 0x7f1ca0106e40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1c900098a0 tx=0x7f1c90009bb0 comp rx=0 tx=0).stop 2024-04-02T13:29:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.451+0000 7f1ca61bb700 1 -- 172.21.15.33:0/2430666012 shutdown_connections 2024-04-02T13:29:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.451+0000 7f1ca61bb700 1 --2- 172.21.15.33:0/2430666012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 0x7f1ca0106e40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.451+0000 7f1ca61bb700 1 -- 172.21.15.33:0/2430666012 >> 172.21.15.33:0/2430666012 conn(0x7f1ca01025e0 msgr2=0x7f1ca0104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.451+0000 7f1ca61bb700 1 -- 172.21.15.33:0/2430666012 shutdown_connections 2024-04-02T13:29:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.451+0000 7f1ca61bb700 1 -- 172.21.15.33:0/2430666012 wait complete. 2024-04-02T13:29:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.452+0000 7f1ca61bb700 1 Processor -- start 2024-04-02T13:29:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.452+0000 7f1ca61bb700 1 -- start start 2024-04-02T13:29:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.452+0000 7f1ca61bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 0x7f1ca019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.452+0000 7f1ca61bb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca0108450 con 0x7f1ca0106a20 2024-04-02T13:29:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.452+0000 7f1c9f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 0x7f1ca019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.452+0000 7f1c9f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 0x7f1ca019e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47928/0 (socket says 172.21.15.33:47928) 2024-04-02T13:29:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.452+0000 7f1c9f7fe700 1 -- 172.21.15.33:0/3938738325 learned_addr learned my addr 172.21.15.33:0/3938738325 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:25.314 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.453+0000 7f1c9f7fe700 1 -- 172.21.15.33:0/3938738325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca019e770 con 0x7f1ca0106a20 2024-04-02T13:29:25.314 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.453+0000 7f1c9f7fe700 1 --2- 172.21.15.33:0/3938738325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 0x7f1ca019e230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c90003fa0 tx=0x7f1c900046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:25.314 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.453+0000 7f1c9cff9700 1 -- 172.21.15.33:0/3938738325 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c900038b0 con 0x7f1ca0106a20 2024-04-02T13:29:25.314 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.453+0000 7f1c9cff9700 1 -- 172.21.15.33:0/3938738325 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1c9001a430 con 0x7f1ca0106a20 2024-04-02T13:29:25.314 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.453+0000 7f1ca61bb700 1 -- 172.21.15.33:0/3938738325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca019ea00 con 0x7f1ca0106a20 2024-04-02T13:29:25.314 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.453+0000 7f1c9cff9700 1 -- 172.21.15.33:0/3938738325 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c90019550 con 0x7f1ca0106a20 2024-04-02T13:29:25.314 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.453+0000 7f1ca61bb700 1 -- 172.21.15.33:0/3938738325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca01a16f0 con 0x7f1ca0106a20 2024-04-02T13:29:25.314 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.454+0000 7f1c9cff9700 1 -- 172.21.15.33:0/3938738325 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1c900197e0 con 0x7f1ca0106a20 2024-04-02T13:29:25.314 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.454+0000 7f1ca61bb700 1 -- 172.21.15.33:0/3938738325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca0063c20 con 0x7f1ca0106a20 2024-04-02T13:29:25.314 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.454+0000 7f1c9cff9700 1 --2- 172.21.15.33:0/3938738325 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1c88039710 0x7f1c8803bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:25.314 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.454+0000 7f1c9cff9700 1 -- 172.21.15.33:0/3938738325 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1c9004dd00 con 0x7f1ca0106a20 2024-04-02T13:29:25.315 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.454+0000 7f1c9effd700 1 --2- 172.21.15.33:0/3938738325 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1c88039710 0x7f1c8803bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:25.315 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.454+0000 7f1c9effd700 1 --2- 172.21.15.33:0/3938738325 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1c88039710 0x7f1c8803bbd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1c94006fd0 tx=0x7f1c94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:25.315 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.459+0000 7f1c9cff9700 1 -- 172.21.15.33:0/3938738325 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1c9002d9c0 con 0x7f1ca0106a20 2024-04-02T13:29:25.315 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.781+0000 7f1ca61bb700 1 -- 172.21.15.33:0/3938738325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f1ca002dfc0 con 0x7f1ca0106a20 2024-04-02T13:29:25.315 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.784+0000 7f1c9cff9700 1 -- 172.21.15.33:0/3938738325 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f1c9002d440 con 0x7f1ca0106a20 2024-04-02T13:29:25.315 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.786+0000 7f1ca61bb700 1 -- 172.21.15.33:0/3938738325 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1c88039710 msgr2=0x7f1c8803bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:25.315 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.786+0000 7f1ca61bb700 1 --2- 172.21.15.33:0/3938738325 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1c88039710 0x7f1c8803bbd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1c94006fd0 tx=0x7f1c94006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:25.315 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.786+0000 7f1ca61bb700 1 -- 172.21.15.33:0/3938738325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 msgr2=0x7f1ca019e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:25.315 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.786+0000 7f1ca61bb700 1 --2- 172.21.15.33:0/3938738325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 0x7f1ca019e230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c90003fa0 tx=0x7f1c900046a0 comp rx=0 tx=0).stop 2024-04-02T13:29:25.315 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.786+0000 7f1ca61bb700 1 -- 172.21.15.33:0/3938738325 shutdown_connections 2024-04-02T13:29:25.315 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.786+0000 7f1ca61bb700 1 --2- 172.21.15.33:0/3938738325 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1c88039710 0x7f1c8803bbd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:25.315 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.786+0000 7f1ca61bb700 1 --2- 172.21.15.33:0/3938738325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ca0106a20 0x7f1ca019e230 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:25.315 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.786+0000 7f1ca61bb700 1 -- 172.21.15.33:0/3938738325 >> 172.21.15.33:0/3938738325 conn(0x7f1ca01025e0 msgr2=0x7f1ca0194e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.786+0000 7f1ca61bb700 1 -- 172.21.15.33:0/3938738325 shutdown_connections 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-02T13:29:24.786+0000 7f1ca61bb700 1 -- 172.21.15.33:0/3938738325 wait complete. 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout:Or, if you are only running a single cluster on this host: 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:29:25.316 INFO:teuthology.orchestra.run.smithi033.stdout: ceph telemetry on 2024-04-02T13:29:25.317 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:29:25.317 INFO:teuthology.orchestra.run.smithi033.stdout:For more information see: 2024-04-02T13:29:25.317 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:29:25.317 INFO:teuthology.orchestra.run.smithi033.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-02T13:29:25.317 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:29:25.317 INFO:teuthology.orchestra.run.smithi033.stdout:Bootstrap complete. 2024-04-02T13:29:25.355 INFO:tasks.cephadm:Fetching config... 2024-04-02T13:29:25.355 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:29:25.355 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-02T13:29:25.375 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-02T13:29:25.375 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:29:25.375 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-02T13:29:25.431 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-02T13:29:25.432 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:29:25.432 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/keyring of=/dev/stdout 2024-04-02T13:29:25.500 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-02T13:29:25.500 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:29:25.501 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-02T13:29:25.559 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-02T13:29:25.559 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHk6AOx3zvViF2UrhsT/hujKp5nhsMrjKHYtRhSCJvyfYfaOJP/EpUPV3pBTnUD4C93LOIIGB4hbUZ2uvACuz6GBmQrcJbOaV093q70vNGNJ4u6ERk7/ALNf2WycCmQFomi5jOKsNY4B3xtLuBuXTEAoUOsDBnTN5U3XxVrqnAA9nmzPoMBnXiLVORe5DFFUSoClCQMS5SAQIlNTh0vhFtkV6Rn0UFuCFVITpIkRhQKKl8ejOy0EtsI+6rvSXVZWgamj2wcUlDmCGeZsBTZbRnc8ummuG2xFLXj9AxRYquoZhEMevLCXlvek+dgZq7S9BU8CevoytqErQhOEr4cJwJ3JpY0pMd3bTuWTzNPBi3jQUoeTKE5CzdOO6X0BKYtwzfd9Pk9GfshzMuEd8kWRic2/KpkwbHLZ/WL4+Pb4KueaDeco0Be1C+hQsjccUZGploBvLrI+tD6AFNm/hK67oKyMQ4EV350omHFNy0I3SZWe8aPrt9gFikqrw+4M2XW5E= ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T13:29:25.641 INFO:teuthology.orchestra.run.smithi033.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHk6AOx3zvViF2UrhsT/hujKp5nhsMrjKHYtRhSCJvyfYfaOJP/EpUPV3pBTnUD4C93LOIIGB4hbUZ2uvACuz6GBmQrcJbOaV093q70vNGNJ4u6ERk7/ALNf2WycCmQFomi5jOKsNY4B3xtLuBuXTEAoUOsDBnTN5U3XxVrqnAA9nmzPoMBnXiLVORe5DFFUSoClCQMS5SAQIlNTh0vhFtkV6Rn0UFuCFVITpIkRhQKKl8ejOy0EtsI+6rvSXVZWgamj2wcUlDmCGeZsBTZbRnc8ummuG2xFLXj9AxRYquoZhEMevLCXlvek+dgZq7S9BU8CevoytqErQhOEr4cJwJ3JpY0pMd3bTuWTzNPBi3jQUoeTKE5CzdOO6X0BKYtwzfd9Pk9GfshzMuEd8kWRic2/KpkwbHLZ/WL4+Pb4KueaDeco0Be1C+hQsjccUZGploBvLrI+tD6AFNm/hK67oKyMQ4EV350omHFNy0I3SZWe8aPrt9gFikqrw+4M2XW5E= ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:29:25.655 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHk6AOx3zvViF2UrhsT/hujKp5nhsMrjKHYtRhSCJvyfYfaOJP/EpUPV3pBTnUD4C93LOIIGB4hbUZ2uvACuz6GBmQrcJbOaV093q70vNGNJ4u6ERk7/ALNf2WycCmQFomi5jOKsNY4B3xtLuBuXTEAoUOsDBnTN5U3XxVrqnAA9nmzPoMBnXiLVORe5DFFUSoClCQMS5SAQIlNTh0vhFtkV6Rn0UFuCFVITpIkRhQKKl8ejOy0EtsI+6rvSXVZWgamj2wcUlDmCGeZsBTZbRnc8ummuG2xFLXj9AxRYquoZhEMevLCXlvek+dgZq7S9BU8CevoytqErQhOEr4cJwJ3JpY0pMd3bTuWTzNPBi3jQUoeTKE5CzdOO6X0BKYtwzfd9Pk9GfshzMuEd8kWRic2/KpkwbHLZ/WL4+Pb4KueaDeco0Be1C+hQsjccUZGploBvLrI+tD6AFNm/hK67oKyMQ4EV350omHFNy0I3SZWe8aPrt9gFikqrw+4M2XW5E= ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T13:29:25.704 INFO:teuthology.orchestra.run.smithi050.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHk6AOx3zvViF2UrhsT/hujKp5nhsMrjKHYtRhSCJvyfYfaOJP/EpUPV3pBTnUD4C93LOIIGB4hbUZ2uvACuz6GBmQrcJbOaV093q70vNGNJ4u6ERk7/ALNf2WycCmQFomi5jOKsNY4B3xtLuBuXTEAoUOsDBnTN5U3XxVrqnAA9nmzPoMBnXiLVORe5DFFUSoClCQMS5SAQIlNTh0vhFtkV6Rn0UFuCFVITpIkRhQKKl8ejOy0EtsI+6rvSXVZWgamj2wcUlDmCGeZsBTZbRnc8ummuG2xFLXj9AxRYquoZhEMevLCXlvek+dgZq7S9BU8CevoytqErQhOEr4cJwJ3JpY0pMd3bTuWTzNPBi3jQUoeTKE5CzdOO6X0BKYtwzfd9Pk9GfshzMuEd8kWRic2/KpkwbHLZ/WL4+Pb4KueaDeco0Be1C+hQsjccUZGploBvLrI+tD6AFNm/hK67oKyMQ4EV350omHFNy0I3SZWe8aPrt9gFikqrw+4M2XW5E= ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:29:25.719 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHk6AOx3zvViF2UrhsT/hujKp5nhsMrjKHYtRhSCJvyfYfaOJP/EpUPV3pBTnUD4C93LOIIGB4hbUZ2uvACuz6GBmQrcJbOaV093q70vNGNJ4u6ERk7/ALNf2WycCmQFomi5jOKsNY4B3xtLuBuXTEAoUOsDBnTN5U3XxVrqnAA9nmzPoMBnXiLVORe5DFFUSoClCQMS5SAQIlNTh0vhFtkV6Rn0UFuCFVITpIkRhQKKl8ejOy0EtsI+6rvSXVZWgamj2wcUlDmCGeZsBTZbRnc8ummuG2xFLXj9AxRYquoZhEMevLCXlvek+dgZq7S9BU8CevoytqErQhOEr4cJwJ3JpY0pMd3bTuWTzNPBi3jQUoeTKE5CzdOO6X0BKYtwzfd9Pk9GfshzMuEd8kWRic2/KpkwbHLZ/WL4+Pb4KueaDeco0Be1C+hQsjccUZGploBvLrI+tD6AFNm/hK67oKyMQ4EV350omHFNy0I3SZWe8aPrt9gFikqrw+4M2XW5E= ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T13:29:25.766 INFO:teuthology.orchestra.run.smithi145.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHk6AOx3zvViF2UrhsT/hujKp5nhsMrjKHYtRhSCJvyfYfaOJP/EpUPV3pBTnUD4C93LOIIGB4hbUZ2uvACuz6GBmQrcJbOaV093q70vNGNJ4u6ERk7/ALNf2WycCmQFomi5jOKsNY4B3xtLuBuXTEAoUOsDBnTN5U3XxVrqnAA9nmzPoMBnXiLVORe5DFFUSoClCQMS5SAQIlNTh0vhFtkV6Rn0UFuCFVITpIkRhQKKl8ejOy0EtsI+6rvSXVZWgamj2wcUlDmCGeZsBTZbRnc8ummuG2xFLXj9AxRYquoZhEMevLCXlvek+dgZq7S9BU8CevoytqErQhOEr4cJwJ3JpY0pMd3bTuWTzNPBi3jQUoeTKE5CzdOO6X0BKYtwzfd9Pk9GfshzMuEd8kWRic2/KpkwbHLZ/WL4+Pb4KueaDeco0Be1C+hQsjccUZGploBvLrI+tD6AFNm/hK67oKyMQ4EV350omHFNy0I3SZWe8aPrt9gFikqrw+4M2XW5E= ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:29:25.781 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-02T13:29:25.889 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:25 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3938738325' entity='client.admin' 2024-04-02T13:29:26.045 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:29:28.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.121+0000 7f9011f14700 1 -- 172.21.15.33:0/4088304622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c102040 msgr2=0x7f900c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:28.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.121+0000 7f9011f14700 1 --2- 172.21.15.33:0/4088304622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c102040 0x7f900c102460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009910 tx=0x7f8ffc009c20 comp rx=0 tx=0).stop 2024-04-02T13:29:28.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.121+0000 7f9011f14700 1 -- 172.21.15.33:0/4088304622 shutdown_connections 2024-04-02T13:29:28.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.121+0000 7f9011f14700 1 --2- 172.21.15.33:0/4088304622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c102040 0x7f900c102460 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:28.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.121+0000 7f9011f14700 1 -- 172.21.15.33:0/4088304622 >> 172.21.15.33:0/4088304622 conn(0x7f900c0fd5e0 msgr2=0x7f900c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:28.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.121+0000 7f9011f14700 1 -- 172.21.15.33:0/4088304622 shutdown_connections 2024-04-02T13:29:28.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.121+0000 7f9011f14700 1 -- 172.21.15.33:0/4088304622 wait complete. 2024-04-02T13:29:28.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.121+0000 7f9011f14700 1 Processor -- start 2024-04-02T13:29:28.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.122+0000 7f9011f14700 1 -- start start 2024-04-02T13:29:28.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.122+0000 7f9011f14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c102040 0x7f900c0762b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:28.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.122+0000 7f9011f14700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f900c1080c0 con 0x7f900c102040 2024-04-02T13:29:28.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.122+0000 7f900b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c102040 0x7f900c0762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:28.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.122+0000 7f900b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c102040 0x7f900c0762b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47944/0 (socket says 172.21.15.33:47944) 2024-04-02T13:29:28.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.122+0000 7f900b7fe700 1 -- 172.21.15.33:0/1549884716 learned_addr learned my addr 172.21.15.33:0/1549884716 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.122+0000 7f900b7fe700 1 -- 172.21.15.33:0/1549884716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c0767f0 con 0x7f900c102040 2024-04-02T13:29:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.122+0000 7f900b7fe700 1 --2- 172.21.15.33:0/1549884716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c102040 0x7f900c0762b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc004f80 tx=0x7f8ffc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.122+0000 7f9008ff9700 1 -- 172.21.15.33:0/1549884716 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ffc027070 con 0x7f900c102040 2024-04-02T13:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.123+0000 7f9011f14700 1 -- 172.21.15.33:0/1549884716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c0749e0 con 0x7f900c102040 2024-04-02T13:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.123+0000 7f9008ff9700 1 -- 172.21.15.33:0/1549884716 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8ffc00be30 con 0x7f900c102040 2024-04-02T13:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.123+0000 7f9008ff9700 1 -- 172.21.15.33:0/1549884716 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ffc00f460 con 0x7f900c102040 2024-04-02T13:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.123+0000 7f9011f14700 1 -- 172.21.15.33:0/1549884716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c074f20 con 0x7f900c102040 2024-04-02T13:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.124+0000 7f9008ff9700 1 -- 172.21.15.33:0/1549884716 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8ffc02a070 con 0x7f900c102040 2024-04-02T13:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.124+0000 7f9011f14700 1 -- 172.21.15.33:0/1549884716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c108840 con 0x7f900c102040 2024-04-02T13:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.124+0000 7f9008ff9700 1 --2- 172.21.15.33:0/1549884716 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8ff8039650 0x7f8ff803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.124+0000 7f9008ff9700 1 -- 172.21.15.33:0/1549884716 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8ffc01e030 con 0x7f900c102040 2024-04-02T13:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.124+0000 7f900affd700 1 --2- 172.21.15.33:0/1549884716 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8ff8039650 0x7f8ff803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.124+0000 7f900affd700 1 --2- 172.21.15.33:0/1549884716 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8ff8039650 0x7f8ff803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4006fd0 tx=0x7f8ff4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:28.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.129+0000 7f9008ff9700 1 -- 172.21.15.33:0/1549884716 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8ffc013070 con 0x7f900c102040 2024-04-02T13:29:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.337+0000 7f9011f14700 1 -- 172.21.15.33:0/1549884716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f900c190b40 con 0x7f900c102040 2024-04-02T13:29:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.339+0000 7f9008ff9700 1 -- 172.21.15.33:0/1549884716 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f8ffc02b3b0 con 0x7f900c102040 2024-04-02T13:29:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.341+0000 7f9011f14700 1 -- 172.21.15.33:0/1549884716 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8ff8039650 msgr2=0x7f8ff803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.341+0000 7f9011f14700 1 --2- 172.21.15.33:0/1549884716 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8ff8039650 0x7f8ff803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4006fd0 tx=0x7f8ff4006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.341+0000 7f9011f14700 1 -- 172.21.15.33:0/1549884716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c102040 msgr2=0x7f900c0762b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.341+0000 7f9011f14700 1 --2- 172.21.15.33:0/1549884716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c102040 0x7f900c0762b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc004f80 tx=0x7f8ffc009f90 comp rx=0 tx=0).stop 2024-04-02T13:29:28.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.341+0000 7f9011f14700 1 -- 172.21.15.33:0/1549884716 shutdown_connections 2024-04-02T13:29:28.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.341+0000 7f9011f14700 1 --2- 172.21.15.33:0/1549884716 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8ff8039650 0x7f8ff803bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:28.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.341+0000 7f9011f14700 1 --2- 172.21.15.33:0/1549884716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c102040 0x7f900c0762b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:28.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.341+0000 7f9011f14700 1 -- 172.21.15.33:0/1549884716 >> 172.21.15.33:0/1549884716 conn(0x7f900c0fd5e0 msgr2=0x7f900c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:28.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.341+0000 7f9011f14700 1 -- 172.21.15.33:0/1549884716 shutdown_connections 2024-04-02T13:29:28.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:28.341+0000 7f9011f14700 1 -- 172.21.15.33:0/1549884716 wait complete. 2024-04-02T13:29:28.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:28.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:28.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi033", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:28.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:28.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:28.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:28.959 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-02T13:29:28.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-02T13:29:29.198 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:29:29.650 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:29 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1549884716' entity='client.admin' 2024-04-02T13:29:29.650 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:29.651 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:30.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.712+0000 7f49b567c700 1 -- 172.21.15.33:0/1470026344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49b01006c0 msgr2=0x7f49b0100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:30.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.712+0000 7f49b567c700 1 --2- 172.21.15.33:0/1470026344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49b01006c0 0x7f49b0100ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009910 tx=0x7f49a0009c20 comp rx=0 tx=0).stop 2024-04-02T13:29:30.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.712+0000 7f49b567c700 1 -- 172.21.15.33:0/1470026344 shutdown_connections 2024-04-02T13:29:30.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.712+0000 7f49b567c700 1 --2- 172.21.15.33:0/1470026344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49b01006c0 0x7f49b0100ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:30.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.712+0000 7f49b567c700 1 -- 172.21.15.33:0/1470026344 >> 172.21.15.33:0/1470026344 conn(0x7f49b00fbc80 msgr2=0x7f49b00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:30.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.712+0000 7f49b567c700 1 -- 172.21.15.33:0/1470026344 shutdown_connections 2024-04-02T13:29:30.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.712+0000 7f49b567c700 1 -- 172.21.15.33:0/1470026344 wait complete. 2024-04-02T13:29:30.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.713+0000 7f49b567c700 1 Processor -- start 2024-04-02T13:29:30.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.713+0000 7f49b567c700 1 -- start start 2024-04-02T13:29:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.713+0000 7f49b567c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49b01006c0 0x7f49b0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.713+0000 7f49b567c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49b0106740 con 0x7f49b01006c0 2024-04-02T13:29:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.713+0000 7f49aeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49b01006c0 0x7f49b0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.713+0000 7f49aeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49b01006c0 0x7f49b0199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47960/0 (socket says 172.21.15.33:47960) 2024-04-02T13:29:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.713+0000 7f49aeffd700 1 -- 172.21.15.33:0/3682574193 learned_addr learned my addr 172.21.15.33:0/3682574193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.714+0000 7f49aeffd700 1 -- 172.21.15.33:0/3682574193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b0199ec0 con 0x7f49b01006c0 2024-04-02T13:29:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.714+0000 7f49aeffd700 1 --2- 172.21.15.33:0/3682574193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49b01006c0 0x7f49b0199980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49a0004f80 tx=0x7f49a0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.714+0000 7f4997fff700 1 -- 172.21.15.33:0/3682574193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49a0027070 con 0x7f49b01006c0 2024-04-02T13:29:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.714+0000 7f49b567c700 1 -- 172.21.15.33:0/3682574193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b019a150 con 0x7f49b01006c0 2024-04-02T13:29:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.714+0000 7f4997fff700 1 -- 172.21.15.33:0/3682574193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f49a000be30 con 0x7f49b01006c0 2024-04-02T13:29:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.714+0000 7f4997fff700 1 -- 172.21.15.33:0/3682574193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49a000f460 con 0x7f49b01006c0 2024-04-02T13:29:30.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.714+0000 7f49b567c700 1 -- 172.21.15.33:0/3682574193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b019cdf0 con 0x7f49b01006c0 2024-04-02T13:29:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.715+0000 7f4997fff700 1 -- 172.21.15.33:0/3682574193 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f49a002a070 con 0x7f49b01006c0 2024-04-02T13:29:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.715+0000 7f49b567c700 1 -- 172.21.15.33:0/3682574193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b0106ec0 con 0x7f49b01006c0 2024-04-02T13:29:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.715+0000 7f4997fff700 1 --2- 172.21.15.33:0/3682574193 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f4998039650 0x7f499803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.715+0000 7f4997fff700 1 -- 172.21.15.33:0/3682574193 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f49a001e030 con 0x7f49b01006c0 2024-04-02T13:29:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.715+0000 7f49ae7fc700 1 --2- 172.21.15.33:0/3682574193 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f4998039650 0x7f499803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.715+0000 7f49ae7fc700 1 --2- 172.21.15.33:0/3682574193 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f4998039650 0x7f499803bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f49a4006fd0 tx=0x7f49a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.720+0000 7f4997fff700 1 -- 172.21.15.33:0/3682574193 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f49a0013070 con 0x7f49b01006c0 2024-04-02T13:29:30.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.932+0000 7f49b567c700 1 -- 172.21.15.33:0/3682574193 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f49b0107510 con 0x7f4998039650 2024-04-02T13:29:30.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.936+0000 7f4997fff700 1 -- 172.21.15.33:0/3682574193 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f49b0107510 con 0x7f4998039650 2024-04-02T13:29:30.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.938+0000 7f49b567c700 1 -- 172.21.15.33:0/3682574193 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f4998039650 msgr2=0x7f499803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:30.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.938+0000 7f49b567c700 1 --2- 172.21.15.33:0/3682574193 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f4998039650 0x7f499803bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f49a4006fd0 tx=0x7f49a4006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:30.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.938+0000 7f49b567c700 1 -- 172.21.15.33:0/3682574193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49b01006c0 msgr2=0x7f49b0199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:30.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.938+0000 7f49b567c700 1 --2- 172.21.15.33:0/3682574193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49b01006c0 0x7f49b0199980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49a0004f80 tx=0x7f49a0009f90 comp rx=0 tx=0).stop 2024-04-02T13:29:30.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.938+0000 7f49b567c700 1 -- 172.21.15.33:0/3682574193 shutdown_connections 2024-04-02T13:29:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.938+0000 7f49b567c700 1 --2- 172.21.15.33:0/3682574193 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f4998039650 0x7f499803bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.938+0000 7f49b567c700 1 --2- 172.21.15.33:0/3682574193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49b01006c0 0x7f49b0199980 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.938+0000 7f49b567c700 1 -- 172.21.15.33:0/3682574193 >> 172.21.15.33:0/3682574193 conn(0x7f49b00fbc80 msgr2=0x7f49b00fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.938+0000 7f49b567c700 1 -- 172.21.15.33:0/3682574193 shutdown_connections 2024-04-02T13:29:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:30.938+0000 7f49b567c700 1 -- 172.21.15.33:0/3682574193 wait complete. 2024-04-02T13:29:31.666 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi050 2024-04-02T13:29:31.667 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T13:29:31.667 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/etc/ceph/ceph.conf 2024-04-02T13:29:31.687 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T13:29:31.687 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:29:31.744 INFO:tasks.cephadm:Adding host smithi050 to orchestrator... 2024-04-02T13:29:31.745 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi050 2024-04-02T13:29:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:29:32.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:31 smithi033 ceph-mon[32390]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:32.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:31 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:32.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:31 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:32.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:31 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:32.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:31 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:32.210 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:31 smithi033 ceph-mon[32390]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-02T13:29:32.210 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:31 smithi033 ceph-mon[32390]: Updating smithi033:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:29:32.210 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:31 smithi033 ceph-mon[32390]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:29:32.210 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:31 smithi033 ceph-mon[32390]: Updating smithi033:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:29:32.210 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:31 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:32.210 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:31 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:32.210 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:31 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:33.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.335+0000 7fa9cba82700 1 -- 172.21.15.33:0/569913553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c4101120 msgr2=0x7fa9c4101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.335+0000 7fa9cba82700 1 --2- 172.21.15.33:0/569913553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c4101120 0x7fa9c4101500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009910 tx=0x7fa9c0009c20 comp rx=0 tx=0).stop 2024-04-02T13:29:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.335+0000 7fa9cba82700 1 -- 172.21.15.33:0/569913553 shutdown_connections 2024-04-02T13:29:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.335+0000 7fa9cba82700 1 --2- 172.21.15.33:0/569913553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c4101120 0x7fa9c4101500 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.335+0000 7fa9cba82700 1 -- 172.21.15.33:0/569913553 >> 172.21.15.33:0/569913553 conn(0x7fa9c40fc9b0 msgr2=0x7fa9c40fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.336+0000 7fa9cba82700 1 -- 172.21.15.33:0/569913553 shutdown_connections 2024-04-02T13:29:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.336+0000 7fa9cba82700 1 -- 172.21.15.33:0/569913553 wait complete. 2024-04-02T13:29:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.336+0000 7fa9cba82700 1 Processor -- start 2024-04-02T13:29:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.336+0000 7fa9cba82700 1 -- start start 2024-04-02T13:29:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.336+0000 7fa9cba82700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c4101120 0x7fa9c419d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.336+0000 7fa9cba82700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c410b4e0 con 0x7fa9c4101120 2024-04-02T13:29:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.337+0000 7fa9c981e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c4101120 0x7fa9c419d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.337+0000 7fa9c981e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c4101120 0x7fa9c419d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47978/0 (socket says 172.21.15.33:47978) 2024-04-02T13:29:33.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.337+0000 7fa9c981e700 1 -- 172.21.15.33:0/1282610762 learned_addr learned my addr 172.21.15.33:0/1282610762 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:33.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.337+0000 7fa9c981e700 1 -- 172.21.15.33:0/1282610762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c419dcb0 con 0x7fa9c4101120 2024-04-02T13:29:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.337+0000 7fa9c981e700 1 --2- 172.21.15.33:0/1282610762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c4101120 0x7fa9c419d770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0006010 tx=0x7fa9c0005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.337+0000 7fa9baffd700 1 -- 172.21.15.33:0/1282610762 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9c0027070 con 0x7fa9c4101120 2024-04-02T13:29:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.337+0000 7fa9baffd700 1 -- 172.21.15.33:0/1282610762 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa9c000be30 con 0x7fa9c4101120 2024-04-02T13:29:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.337+0000 7fa9cba82700 1 -- 172.21.15.33:0/1282610762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c419dfa0 con 0x7fa9c4101120 2024-04-02T13:29:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.337+0000 7fa9baffd700 1 -- 172.21.15.33:0/1282610762 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9c0022d70 con 0x7fa9c4101120 2024-04-02T13:29:33.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.337+0000 7fa9cba82700 1 -- 172.21.15.33:0/1282610762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c4197930 con 0x7fa9c4101120 2024-04-02T13:29:33.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.338+0000 7fa9baffd700 1 -- 172.21.15.33:0/1282610762 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa9c002a070 con 0x7fa9c4101120 2024-04-02T13:29:33.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.338+0000 7fa9cba82700 1 -- 172.21.15.33:0/1282610762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c4102600 con 0x7fa9c4101120 2024-04-02T13:29:33.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.338+0000 7fa9baffd700 1 --2- 172.21.15.33:0/1282610762 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa9b00396b0 0x7fa9b003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:33.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.338+0000 7fa9baffd700 1 -- 172.21.15.33:0/1282610762 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa9c001e030 con 0x7fa9c4101120 2024-04-02T13:29:33.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.338+0000 7fa9c901d700 1 --2- 172.21.15.33:0/1282610762 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa9b00396b0 0x7fa9b003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:33.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.339+0000 7fa9c901d700 1 --2- 172.21.15.33:0/1282610762 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa9b00396b0 0x7fa9b003bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4006fd0 tx=0x7fa9b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:33.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.343+0000 7fa9baffd700 1 -- 172.21.15.33:0/1282610762 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa9c0013070 con 0x7fa9c4101120 2024-04-02T13:29:33.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:33.559+0000 7fa9cba82700 1 -- 172.21.15.33:0/1282610762 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi050", "target": ["mon-mgr", ""]}) -- 0x7fa9c4198ed0 con 0x7fa9b00396b0 2024-04-02T13:29:34.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:33 smithi033 ceph-mon[32390]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi050", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:35.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:34 smithi033 ceph-mon[32390]: Deploying cephadm binary to smithi050 2024-04-02T13:29:37.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:36 smithi033 ceph-mon[32390]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:29:37.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:37.977+0000 7fa9baffd700 1 -- 172.21.15.33:0/1282610762 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fa9c4198ed0 con 0x7fa9b00396b0 2024-04-02T13:29:37.980 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi050' with addr '172.21.15.50' 2024-04-02T13:29:37.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:37.979+0000 7fa9cba82700 1 -- 172.21.15.33:0/1282610762 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa9b00396b0 msgr2=0x7fa9b003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:37.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:37.979+0000 7fa9cba82700 1 --2- 172.21.15.33:0/1282610762 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa9b00396b0 0x7fa9b003bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4006fd0 tx=0x7fa9b4006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:37.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:37.979+0000 7fa9cba82700 1 -- 172.21.15.33:0/1282610762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c4101120 msgr2=0x7fa9c419d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:37.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:37.979+0000 7fa9cba82700 1 --2- 172.21.15.33:0/1282610762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c4101120 0x7fa9c419d770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0006010 tx=0x7fa9c0005810 comp rx=0 tx=0).stop 2024-04-02T13:29:37.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:37.979+0000 7fa9cba82700 1 -- 172.21.15.33:0/1282610762 shutdown_connections 2024-04-02T13:29:37.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:37.979+0000 7fa9cba82700 1 --2- 172.21.15.33:0/1282610762 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa9b00396b0 0x7fa9b003bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:37.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:37.979+0000 7fa9cba82700 1 --2- 172.21.15.33:0/1282610762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c4101120 0x7fa9c419d770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:37.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:37.979+0000 7fa9cba82700 1 -- 172.21.15.33:0/1282610762 >> 172.21.15.33:0/1282610762 conn(0x7fa9c40fc9b0 msgr2=0x7fa9c406a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:37.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:37.979+0000 7fa9cba82700 1 -- 172.21.15.33:0/1282610762 shutdown_connections 2024-04-02T13:29:37.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:37.979+0000 7fa9cba82700 1 -- 172.21.15.33:0/1282610762 wait complete. 2024-04-02T13:29:38.573 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-02T13:29:38.816 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:29:39.252 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:38 smithi033 ceph-mon[32390]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:29:39.252 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:38 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:39.252 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:38 smithi033 ceph-mon[32390]: Added host smithi050 2024-04-02T13:29:39.252 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:38 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:39.252 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:38 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:40.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:40 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.345+0000 7f534be9f700 1 -- 172.21.15.33:0/2646156136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5344102230 msgr2=0x7f5344102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.345+0000 7f534be9f700 1 --2- 172.21.15.33:0/2646156136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5344102230 0x7f5344102610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5340009910 tx=0x7f5340009c20 comp rx=0 tx=0).stop 2024-04-02T13:29:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.345+0000 7f534be9f700 1 -- 172.21.15.33:0/2646156136 shutdown_connections 2024-04-02T13:29:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.345+0000 7f534be9f700 1 --2- 172.21.15.33:0/2646156136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5344102230 0x7f5344102610 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.345+0000 7f534be9f700 1 -- 172.21.15.33:0/2646156136 >> 172.21.15.33:0/2646156136 conn(0x7f53440fdb00 msgr2=0x7f53440fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.345+0000 7f534be9f700 1 -- 172.21.15.33:0/2646156136 shutdown_connections 2024-04-02T13:29:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.345+0000 7f534be9f700 1 -- 172.21.15.33:0/2646156136 wait complete. 2024-04-02T13:29:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.345+0000 7f534be9f700 1 Processor -- start 2024-04-02T13:29:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.345+0000 7f534be9f700 1 -- start start 2024-04-02T13:29:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.346+0000 7f534be9f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5344102230 0x7f53440748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.346+0000 7f534be9f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f534410c5f0 con 0x7f5344102230 2024-04-02T13:29:40.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.346+0000 7f5349c3b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5344102230 0x7f53440748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:40.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.346+0000 7f5349c3b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5344102230 0x7f53440748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58002/0 (socket says 172.21.15.33:58002) 2024-04-02T13:29:40.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.346+0000 7f5349c3b700 1 -- 172.21.15.33:0/926439293 learned_addr learned my addr 172.21.15.33:0/926439293 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:40.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.346+0000 7f5349c3b700 1 -- 172.21.15.33:0/926439293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53440767e0 con 0x7f5344102230 2024-04-02T13:29:40.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.346+0000 7f5349c3b700 1 --2- 172.21.15.33:0/926439293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5344102230 0x7f53440748f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5340006010 tx=0x7f5340005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:40.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.346+0000 7f533affd700 1 -- 172.21.15.33:0/926439293 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5340027070 con 0x7f5344102230 2024-04-02T13:29:40.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.346+0000 7f533affd700 1 -- 172.21.15.33:0/926439293 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f534000be30 con 0x7f5344102230 2024-04-02T13:29:40.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.346+0000 7f534be9f700 1 -- 172.21.15.33:0/926439293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5344074e90 con 0x7f5344102230 2024-04-02T13:29:40.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.346+0000 7f533affd700 1 -- 172.21.15.33:0/926439293 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5340022d70 con 0x7f5344102230 2024-04-02T13:29:40.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.346+0000 7f534be9f700 1 -- 172.21.15.33:0/926439293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5344075350 con 0x7f5344102230 2024-04-02T13:29:40.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.347+0000 7f533affd700 1 -- 172.21.15.33:0/926439293 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f534002a070 con 0x7f5344102230 2024-04-02T13:29:40.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.347+0000 7f534be9f700 1 -- 172.21.15.33:0/926439293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5344103710 con 0x7f5344102230 2024-04-02T13:29:40.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.347+0000 7f533affd700 1 --2- 172.21.15.33:0/926439293 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f53300396b0 0x7f533003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:40.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.347+0000 7f533affd700 1 -- 172.21.15.33:0/926439293 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f534001e030 con 0x7f5344102230 2024-04-02T13:29:40.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.348+0000 7f534943a700 1 --2- 172.21.15.33:0/926439293 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f53300396b0 0x7f533003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:40.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.348+0000 7f534943a700 1 --2- 172.21.15.33:0/926439293 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f53300396b0 0x7f533003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5334006fd0 tx=0x7f5334006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:40.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.352+0000 7f533affd700 1 -- 172.21.15.33:0/926439293 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5340013070 con 0x7f5344102230 2024-04-02T13:29:40.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.564+0000 7f534be9f700 1 -- 172.21.15.33:0/926439293 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5344075e90 con 0x7f53300396b0 2024-04-02T13:29:40.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.565+0000 7f533affd700 1 -- 172.21.15.33:0/926439293 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f5344075e90 con 0x7f53300396b0 2024-04-02T13:29:40.568 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:29:40.569 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}] 2024-04-02T13:29:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.567+0000 7f534be9f700 1 -- 172.21.15.33:0/926439293 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f53300396b0 msgr2=0x7f533003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.567+0000 7f534be9f700 1 --2- 172.21.15.33:0/926439293 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f53300396b0 0x7f533003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5334006fd0 tx=0x7f5334006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.567+0000 7f534be9f700 1 -- 172.21.15.33:0/926439293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5344102230 msgr2=0x7f53440748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.567+0000 7f534be9f700 1 --2- 172.21.15.33:0/926439293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5344102230 0x7f53440748f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5340006010 tx=0x7f5340005810 comp rx=0 tx=0).stop 2024-04-02T13:29:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.567+0000 7f534be9f700 1 -- 172.21.15.33:0/926439293 shutdown_connections 2024-04-02T13:29:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.567+0000 7f534be9f700 1 --2- 172.21.15.33:0/926439293 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f53300396b0 0x7f533003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.567+0000 7f534be9f700 1 --2- 172.21.15.33:0/926439293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5344102230 0x7f53440748f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.567+0000 7f534be9f700 1 -- 172.21.15.33:0/926439293 >> 172.21.15.33:0/926439293 conn(0x7f53440fdb00 msgr2=0x7f5344105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:40.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.567+0000 7f534be9f700 1 -- 172.21.15.33:0/926439293 shutdown_connections 2024-04-02T13:29:40.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:40.567+0000 7f534be9f700 1 -- 172.21.15.33:0/926439293 wait complete. 2024-04-02T13:29:41.184 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi145 2024-04-02T13:29:41.184 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-02T13:29:41.184 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/etc/ceph/ceph.conf 2024-04-02T13:29:41.209 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-02T13:29:41.210 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:29:41.266 INFO:tasks.cephadm:Adding host smithi145 to orchestrator... 2024-04-02T13:29:41.267 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi145 2024-04-02T13:29:41.510 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:29:41.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:41 smithi033 ceph-mon[32390]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:29:41.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:41 smithi033 ceph-mon[32390]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:29:41.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.014+0000 7f084a422700 1 -- 172.21.15.33:0/1907009381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844101120 msgr2=0x7f0844101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.014+0000 7f084a422700 1 --2- 172.21.15.33:0/1907009381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844101120 0x7f0844101500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0834009910 tx=0x7f0834009c20 comp rx=0 tx=0).stop 2024-04-02T13:29:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.014+0000 7f084a422700 1 -- 172.21.15.33:0/1907009381 shutdown_connections 2024-04-02T13:29:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.014+0000 7f084a422700 1 --2- 172.21.15.33:0/1907009381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844101120 0x7f0844101500 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.014+0000 7f084a422700 1 -- 172.21.15.33:0/1907009381 >> 172.21.15.33:0/1907009381 conn(0x7f0844077b30 msgr2=0x7f08440fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.014+0000 7f084a422700 1 -- 172.21.15.33:0/1907009381 shutdown_connections 2024-04-02T13:29:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.014+0000 7f084a422700 1 -- 172.21.15.33:0/1907009381 wait complete. 2024-04-02T13:29:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.015+0000 7f084a422700 1 Processor -- start 2024-04-02T13:29:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.015+0000 7f084a422700 1 -- start start 2024-04-02T13:29:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.015+0000 7f084a422700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844101120 0x7f0844102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.015+0000 7f084a422700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f084410b4e0 con 0x7f0844101120 2024-04-02T13:29:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.015+0000 7f0843fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844101120 0x7f0844102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.015+0000 7f0843fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844101120 0x7f0844102bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58008/0 (socket says 172.21.15.33:58008) 2024-04-02T13:29:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.015+0000 7f0843fff700 1 -- 172.21.15.33:0/3109121105 learned_addr learned my addr 172.21.15.33:0/3109121105 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.015+0000 7f0843fff700 1 -- 172.21.15.33:0/3109121105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0844104ab0 con 0x7f0844101120 2024-04-02T13:29:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.016+0000 7f0843fff700 1 --2- 172.21.15.33:0/3109121105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844101120 0x7f0844102bc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0834006010 tx=0x7f0834005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.016+0000 7f08417fa700 1 -- 172.21.15.33:0/3109121105 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0834027070 con 0x7f0844101120 2024-04-02T13:29:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.016+0000 7f08417fa700 1 -- 172.21.15.33:0/3109121105 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f083400be30 con 0x7f0844101120 2024-04-02T13:29:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.016+0000 7f08417fa700 1 -- 172.21.15.33:0/3109121105 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0834022d70 con 0x7f0844101120 2024-04-02T13:29:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.016+0000 7f084a422700 1 -- 172.21.15.33:0/3109121105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0844103160 con 0x7f0844101120 2024-04-02T13:29:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.016+0000 7f084a422700 1 -- 172.21.15.33:0/3109121105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0844103620 con 0x7f0844101120 2024-04-02T13:29:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.017+0000 7f08417fa700 1 -- 172.21.15.33:0/3109121105 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f083402a070 con 0x7f0844101120 2024-04-02T13:29:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.017+0000 7f084a422700 1 -- 172.21.15.33:0/3109121105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0844102600 con 0x7f0844101120 2024-04-02T13:29:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.017+0000 7f08417fa700 1 --2- 172.21.15.33:0/3109121105 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0830039650 0x7f083003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.017+0000 7f08417fa700 1 -- 172.21.15.33:0/3109121105 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f083401e030 con 0x7f0844101120 2024-04-02T13:29:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.017+0000 7f08437fe700 1 --2- 172.21.15.33:0/3109121105 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0830039650 0x7f083003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.018+0000 7f08437fe700 1 --2- 172.21.15.33:0/3109121105 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0830039650 0x7f083003bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f082c006fd0 tx=0x7f082c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:43.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.022+0000 7f08417fa700 1 -- 172.21.15.33:0/3109121105 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0834013070 con 0x7f0844101120 2024-04-02T13:29:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:43.237+0000 7f084a422700 1 -- 172.21.15.33:0/3109121105 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi145", "target": ["mon-mgr", ""]}) -- 0x7f08441040f0 con 0x7f0830039650 2024-04-02T13:29:44.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:43 smithi033 ceph-mon[32390]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:29:45.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:44 smithi033 ceph-mon[32390]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi145", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:45.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:45.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:45.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:45.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:45.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:45.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:45.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:46.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:45 smithi033 ceph-mon[32390]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:29:46.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:45 smithi033 ceph-mon[32390]: Updating smithi050:/etc/ceph/ceph.conf 2024-04-02T13:29:46.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:45 smithi033 ceph-mon[32390]: Deploying cephadm binary to smithi145 2024-04-02T13:29:46.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:45 smithi033 ceph-mon[32390]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:29:46.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:45 smithi033 ceph-mon[32390]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:29:46.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:45 smithi033 ceph-mon[32390]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:29:47.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:47.599+0000 7f08417fa700 1 -- 172.21.15.33:0/3109121105 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f08441040f0 con 0x7f0830039650 2024-04-02T13:29:47.603 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi145' with addr '172.21.15.145' 2024-04-02T13:29:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:47.601+0000 7f084a422700 1 -- 172.21.15.33:0/3109121105 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0830039650 msgr2=0x7f083003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:47.601+0000 7f084a422700 1 --2- 172.21.15.33:0/3109121105 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0830039650 0x7f083003bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f082c006fd0 tx=0x7f082c006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:47.601+0000 7f084a422700 1 -- 172.21.15.33:0/3109121105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844101120 msgr2=0x7f0844102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:47.601+0000 7f084a422700 1 --2- 172.21.15.33:0/3109121105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844101120 0x7f0844102bc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0834006010 tx=0x7f0834005810 comp rx=0 tx=0).stop 2024-04-02T13:29:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:47.602+0000 7f084a422700 1 -- 172.21.15.33:0/3109121105 shutdown_connections 2024-04-02T13:29:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:47.602+0000 7f084a422700 1 --2- 172.21.15.33:0/3109121105 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0830039650 0x7f083003bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:47.602+0000 7f084a422700 1 --2- 172.21.15.33:0/3109121105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844101120 0x7f0844102bc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:47.602+0000 7f084a422700 1 -- 172.21.15.33:0/3109121105 >> 172.21.15.33:0/3109121105 conn(0x7f0844077b30 msgr2=0x7f08440699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:47.602+0000 7f084a422700 1 -- 172.21.15.33:0/3109121105 shutdown_connections 2024-04-02T13:29:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:47.602+0000 7f084a422700 1 -- 172.21.15.33:0/3109121105 wait complete. 2024-04-02T13:29:47.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:47 smithi033 ceph-mon[32390]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:29:47.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:47.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:47.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:47.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:47.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:48.342 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-02T13:29:48.590 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:29:48.977 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:48 smithi033 ceph-mon[32390]: Added host smithi145 2024-04-02T13:29:48.977 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:48.977 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:50.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:49 smithi033 ceph-mon[32390]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:29:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.035+0000 7fbe96eb1700 1 -- 172.21.15.33:0/3025936450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102230 msgr2=0x7fbe90102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.035+0000 7fbe96eb1700 1 --2- 172.21.15.33:0/3025936450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102230 0x7fbe90102610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009910 tx=0x7fbe84009c20 comp rx=0 tx=0).stop 2024-04-02T13:29:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.035+0000 7fbe96eb1700 1 -- 172.21.15.33:0/3025936450 shutdown_connections 2024-04-02T13:29:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.035+0000 7fbe96eb1700 1 --2- 172.21.15.33:0/3025936450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102230 0x7fbe90102610 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.035+0000 7fbe96eb1700 1 -- 172.21.15.33:0/3025936450 >> 172.21.15.33:0/3025936450 conn(0x7fbe900fdb00 msgr2=0x7fbe900fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.035+0000 7fbe96eb1700 1 -- 172.21.15.33:0/3025936450 shutdown_connections 2024-04-02T13:29:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.035+0000 7fbe96eb1700 1 -- 172.21.15.33:0/3025936450 wait complete. 2024-04-02T13:29:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.035+0000 7fbe96eb1700 1 Processor -- start 2024-04-02T13:29:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.035+0000 7fbe96eb1700 1 -- start start 2024-04-02T13:29:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.036+0000 7fbe96eb1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102230 0x7fbe9019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.036+0000 7fbe96eb1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9010c5f0 con 0x7fbe90102230 2024-04-02T13:29:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.036+0000 7fbe94c4d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102230 0x7fbe9019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.036+0000 7fbe94c4d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102230 0x7fbe9019d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54318/0 (socket says 172.21.15.33:54318) 2024-04-02T13:29:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.036+0000 7fbe94c4d700 1 -- 172.21.15.33:0/2623202314 learned_addr learned my addr 172.21.15.33:0/2623202314 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.036+0000 7fbe94c4d700 1 -- 172.21.15.33:0/2623202314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9019dc90 con 0x7fbe90102230 2024-04-02T13:29:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.036+0000 7fbe94c4d700 1 --2- 172.21.15.33:0/2623202314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102230 0x7fbe9019d750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe84006010 tx=0x7fbe84005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.036+0000 7fbe8dffb700 1 -- 172.21.15.33:0/2623202314 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe84027070 con 0x7fbe90102230 2024-04-02T13:29:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.036+0000 7fbe8dffb700 1 -- 172.21.15.33:0/2623202314 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbe8400be30 con 0x7fbe90102230 2024-04-02T13:29:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.036+0000 7fbe96eb1700 1 -- 172.21.15.33:0/2623202314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9019df80 con 0x7fbe90102230 2024-04-02T13:29:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.036+0000 7fbe8dffb700 1 -- 172.21.15.33:0/2623202314 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe84022d70 con 0x7fbe90102230 2024-04-02T13:29:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.036+0000 7fbe96eb1700 1 -- 172.21.15.33:0/2623202314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe90197910 con 0x7fbe90102230 2024-04-02T13:29:50.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.037+0000 7fbe8dffb700 1 -- 172.21.15.33:0/2623202314 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbe8402a070 con 0x7fbe90102230 2024-04-02T13:29:50.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.037+0000 7fbe96eb1700 1 -- 172.21.15.33:0/2623202314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe90103710 con 0x7fbe90102230 2024-04-02T13:29:50.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.038+0000 7fbe8dffb700 1 --2- 172.21.15.33:0/2623202314 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbe80039650 0x7fbe8003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:50.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.038+0000 7fbe8dffb700 1 -- 172.21.15.33:0/2623202314 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbe8401e030 con 0x7fbe90102230 2024-04-02T13:29:50.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.038+0000 7fbe8ffff700 1 --2- 172.21.15.33:0/2623202314 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbe80039650 0x7fbe8003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:50.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.038+0000 7fbe8ffff700 1 --2- 172.21.15.33:0/2623202314 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbe80039650 0x7fbe8003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c006fd0 tx=0x7fbe7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.043+0000 7fbe8dffb700 1 -- 172.21.15.33:0/2623202314 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbe84013070 con 0x7fbe90102230 2024-04-02T13:29:50.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.260+0000 7fbe96eb1700 1 -- 172.21.15.33:0/2623202314 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe90198e40 con 0x7fbe80039650 2024-04-02T13:29:50.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.261+0000 7fbe8dffb700 1 -- 172.21.15.33:0/2623202314 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fbe90198e40 con 0x7fbe80039650 2024-04-02T13:29:50.264 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:29:50.264 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}, {"addr": "172.21.15.145", "hostname": "smithi145", "labels": [], "status": ""}] 2024-04-02T13:29:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.263+0000 7fbe96eb1700 1 -- 172.21.15.33:0/2623202314 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbe80039650 msgr2=0x7fbe8003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.263+0000 7fbe96eb1700 1 --2- 172.21.15.33:0/2623202314 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbe80039650 0x7fbe8003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c006fd0 tx=0x7fbe7c006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.263+0000 7fbe96eb1700 1 -- 172.21.15.33:0/2623202314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102230 msgr2=0x7fbe9019d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.263+0000 7fbe96eb1700 1 --2- 172.21.15.33:0/2623202314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102230 0x7fbe9019d750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe84006010 tx=0x7fbe84005810 comp rx=0 tx=0).stop 2024-04-02T13:29:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.263+0000 7fbe96eb1700 1 -- 172.21.15.33:0/2623202314 shutdown_connections 2024-04-02T13:29:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.263+0000 7fbe96eb1700 1 --2- 172.21.15.33:0/2623202314 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbe80039650 0x7fbe8003bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.263+0000 7fbe96eb1700 1 --2- 172.21.15.33:0/2623202314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe90102230 0x7fbe9019d750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:50.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.263+0000 7fbe96eb1700 1 -- 172.21.15.33:0/2623202314 >> 172.21.15.33:0/2623202314 conn(0x7fbe900fdb00 msgr2=0x7fbe90105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:50.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.263+0000 7fbe96eb1700 1 -- 172.21.15.33:0/2623202314 shutdown_connections 2024-04-02T13:29:50.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:50.263+0000 7fbe96eb1700 1 -- 172.21.15.33:0/2623202314 wait complete. 2024-04-02T13:29:50.902 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-02T13:29:50.902 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd crush tunables default 2024-04-02T13:29:51.144 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:29:51.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:51 smithi033 ceph-mon[32390]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:29:51.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:51 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:51.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:51 smithi033 ceph-mon[32390]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:29:52.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.644+0000 7f1afc553700 1 -- 172.21.15.33:0/1340827189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af4101130 msgr2=0x7f1af4101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:52.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.644+0000 7f1afc553700 1 --2- 172.21.15.33:0/1340827189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af4101130 0x7f1af4101510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009910 tx=0x7f1af0009c20 comp rx=0 tx=0).stop 2024-04-02T13:29:52.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.644+0000 7f1afc553700 1 -- 172.21.15.33:0/1340827189 shutdown_connections 2024-04-02T13:29:52.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.644+0000 7f1afc553700 1 --2- 172.21.15.33:0/1340827189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af4101130 0x7f1af4101510 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:52.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.644+0000 7f1afc553700 1 -- 172.21.15.33:0/1340827189 >> 172.21.15.33:0/1340827189 conn(0x7f1af40fc9e0 msgr2=0x7f1af40fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:52.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.644+0000 7f1afc553700 1 -- 172.21.15.33:0/1340827189 shutdown_connections 2024-04-02T13:29:52.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.644+0000 7f1afc553700 1 -- 172.21.15.33:0/1340827189 wait complete. 2024-04-02T13:29:52.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.644+0000 7f1afc553700 1 Processor -- start 2024-04-02T13:29:52.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.644+0000 7f1afc553700 1 -- start start 2024-04-02T13:29:52.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.645+0000 7f1afc553700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af4101130 0x7f1af419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:52.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.645+0000 7f1afc553700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1af410b4f0 con 0x7f1af4101130 2024-04-02T13:29:52.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.645+0000 7f1afa2ef700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af4101130 0x7f1af419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:52.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.645+0000 7f1afa2ef700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af4101130 0x7f1af419d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54338/0 (socket says 172.21.15.33:54338) 2024-04-02T13:29:52.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.645+0000 7f1afa2ef700 1 -- 172.21.15.33:0/1572715621 learned_addr learned my addr 172.21.15.33:0/1572715621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:29:52.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.645+0000 7f1afa2ef700 1 -- 172.21.15.33:0/1572715621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af419dcc0 con 0x7f1af4101130 2024-04-02T13:29:52.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.645+0000 7f1afa2ef700 1 --2- 172.21.15.33:0/1572715621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af4101130 0x7f1af419d780 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1af0006010 tx=0x7f1af0005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:52.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.645+0000 7f1aeb7fe700 1 -- 172.21.15.33:0/1572715621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1af0026070 con 0x7f1af4101130 2024-04-02T13:29:52.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.645+0000 7f1aeb7fe700 1 -- 172.21.15.33:0/1572715621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1af000bbc0 con 0x7f1af4101130 2024-04-02T13:29:52.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.645+0000 7f1aeb7fe700 1 -- 172.21.15.33:0/1572715621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1af00038a0 con 0x7f1af4101130 2024-04-02T13:29:52.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.645+0000 7f1afc553700 1 -- 172.21.15.33:0/1572715621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af419dfb0 con 0x7f1af4101130 2024-04-02T13:29:52.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.645+0000 7f1afc553700 1 -- 172.21.15.33:0/1572715621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af4197940 con 0x7f1af4101130 2024-04-02T13:29:52.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.646+0000 7f1aeb7fe700 1 -- 172.21.15.33:0/1572715621 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1af0029070 con 0x7f1af4101130 2024-04-02T13:29:52.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.646+0000 7f1afc553700 1 -- 172.21.15.33:0/1572715621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af4102610 con 0x7f1af4101130 2024-04-02T13:29:52.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.646+0000 7f1aeb7fe700 1 --2- 172.21.15.33:0/1572715621 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1ae00396b0 0x7f1ae003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:52.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.646+0000 7f1aeb7fe700 1 -- 172.21.15.33:0/1572715621 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1af001e070 con 0x7f1af4101130 2024-04-02T13:29:52.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.646+0000 7f1af9aee700 1 --2- 172.21.15.33:0/1572715621 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1ae00396b0 0x7f1ae003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:52.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.647+0000 7f1af9aee700 1 --2- 172.21.15.33:0/1572715621 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1ae00396b0 0x7f1ae003bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4006fd0 tx=0x7f1ae4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:52.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.652+0000 7f1aeb7fe700 1 -- 172.21.15.33:0/1572715621 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1af0013070 con 0x7f1af4101130 2024-04-02T13:29:52.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:52.867+0000 7f1afc553700 1 -- 172.21.15.33:0/1572715621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1af4198e70 con 0x7f1af4101130 2024-04-02T13:29:53.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:53.199+0000 7f1aeb7fe700 1 -- 172.21.15.33:0/1572715621 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f1af00142b0 con 0x7f1af4101130 2024-04-02T13:29:53.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:53.201+0000 7f1afc553700 1 -- 172.21.15.33:0/1572715621 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1ae00396b0 msgr2=0x7f1ae003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:53.201+0000 7f1afc553700 1 --2- 172.21.15.33:0/1572715621 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1ae00396b0 0x7f1ae003bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4006fd0 tx=0x7f1ae4006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:53.201+0000 7f1afc553700 1 -- 172.21.15.33:0/1572715621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af4101130 msgr2=0x7f1af419d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:53.201+0000 7f1afc553700 1 --2- 172.21.15.33:0/1572715621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af4101130 0x7f1af419d780 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1af0006010 tx=0x7f1af0005810 comp rx=0 tx=0).stop 2024-04-02T13:29:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:53.201+0000 7f1afc553700 1 -- 172.21.15.33:0/1572715621 shutdown_connections 2024-04-02T13:29:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:53.201+0000 7f1afc553700 1 --2- 172.21.15.33:0/1572715621 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1ae00396b0 0x7f1ae003bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:53.201+0000 7f1afc553700 1 --2- 172.21.15.33:0/1572715621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1af4101130 0x7f1af419d780 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:53.201+0000 7f1afc553700 1 -- 172.21.15.33:0/1572715621 >> 172.21.15.33:0/1572715621 conn(0x7f1af40fc9e0 msgr2=0x7f1af406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:53.201+0000 7f1afc553700 1 -- 172.21.15.33:0/1572715621 shutdown_connections 2024-04-02T13:29:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:29:53.201+0000 7f1afc553700 1 -- 172.21.15.33:0/1572715621 wait complete. 2024-04-02T13:29:53.205 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2024-04-02T13:29:53.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:53 smithi033 ceph-mon[32390]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:29:53.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:53 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1572715621' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-02T13:29:53.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:53.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:53.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:53.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:53.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:53.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:53.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:53.817 INFO:tasks.cephadm:Adding mon.a on smithi033 2024-04-02T13:29:53.817 INFO:tasks.cephadm:Adding mon.b on smithi050 2024-04-02T13:29:53.817 INFO:tasks.cephadm:Adding mon.c on smithi145 2024-04-02T13:29:53.818 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch apply mon '3;smithi033:172.21.15.33=a;smithi050:172.21.15.50=b;smithi145:172.21.15.145=c' 2024-04-02T13:29:54.054 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:29:54.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:54 smithi033 ceph-mon[32390]: Updating smithi145:/etc/ceph/ceph.conf 2024-04-02T13:29:54.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:54 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1572715621' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-02T13:29:54.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:54 smithi033 ceph-mon[32390]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:29:54.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:54 smithi033 ceph-mon[32390]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:29:54.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:54 smithi033 ceph-mon[32390]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:29:54.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:54 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:54.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:54 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:54.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:54 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:55.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.398+0000 7fe9dfd21700 1 -- 172.21.15.145:0/4048654281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d8100da0 msgr2=0x7fe9d8101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:55.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.398+0000 7fe9dfd21700 1 --2- 172.21.15.145:0/4048654281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d8100da0 0x7fe9d8101180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009910 tx=0x7fe9d4009c20 comp rx=0 tx=0).stop 2024-04-02T13:29:55.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.398+0000 7fe9dfd21700 1 -- 172.21.15.145:0/4048654281 shutdown_connections 2024-04-02T13:29:55.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.398+0000 7fe9dfd21700 1 --2- 172.21.15.145:0/4048654281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d8100da0 0x7fe9d8101180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:55.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.398+0000 7fe9dfd21700 1 -- 172.21.15.145:0/4048654281 >> 172.21.15.145:0/4048654281 conn(0x7fe9d80fc9e0 msgr2=0x7fe9d80fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:55.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.398+0000 7fe9dfd21700 1 -- 172.21.15.145:0/4048654281 shutdown_connections 2024-04-02T13:29:55.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.398+0000 7fe9dfd21700 1 -- 172.21.15.145:0/4048654281 wait complete. 2024-04-02T13:29:55.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.399+0000 7fe9dfd21700 1 Processor -- start 2024-04-02T13:29:55.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.399+0000 7fe9dfd21700 1 -- start start 2024-04-02T13:29:55.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.399+0000 7fe9dfd21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d8100da0 0x7fe9d8199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:55.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.399+0000 7fe9dfd21700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d810e860 con 0x7fe9d8100da0 2024-04-02T13:29:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.399+0000 7fe9ddabd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d8100da0 0x7fe9d8199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.399+0000 7fe9ddabd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d8100da0 0x7fe9d8199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.145:57454/0 (socket says 172.21.15.145:57454) 2024-04-02T13:29:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.399+0000 7fe9ddabd700 1 -- 172.21.15.145:0/2495522762 learned_addr learned my addr 172.21.15.145:0/2495522762 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.400+0000 7fe9ddabd700 1 -- 172.21.15.145:0/2495522762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d819a2d0 con 0x7fe9d8100da0 2024-04-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.400+0000 7fe9ddabd700 1 --2- 172.21.15.145:0/2495522762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d8100da0 0x7fe9d8199d90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4006010 tx=0x7fe9d4005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.400+0000 7fe9ceffd700 1 -- 172.21.15.145:0/2495522762 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9d4027070 con 0x7fe9d8100da0 2024-04-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.400+0000 7fe9dfd21700 1 -- 172.21.15.145:0/2495522762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d819a560 con 0x7fe9d8100da0 2024-04-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.400+0000 7fe9ceffd700 1 -- 172.21.15.145:0/2495522762 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe9d400be30 con 0x7fe9d8100da0 2024-04-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.400+0000 7fe9ceffd700 1 -- 172.21.15.145:0/2495522762 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9d4022d70 con 0x7fe9d8100da0 2024-04-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.400+0000 7fe9dfd21700 1 -- 172.21.15.145:0/2495522762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d819a9c0 con 0x7fe9d8100da0 2024-04-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.401+0000 7fe9dfd21700 1 -- 172.21.15.145:0/2495522762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d806acb0 con 0x7fe9d8100da0 2024-04-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.401+0000 7fe9ceffd700 1 -- 172.21.15.145:0/2495522762 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe9d402a070 con 0x7fe9d8100da0 2024-04-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.401+0000 7fe9ceffd700 1 --2- 172.21.15.145:0/2495522762 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe9c4039650 0x7fe9c403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.401+0000 7fe9ceffd700 1 -- 172.21.15.145:0/2495522762 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe9d401e030 con 0x7fe9d8100da0 2024-04-02T13:29:55.406 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.402+0000 7fe9dd2bc700 1 --2- 172.21.15.145:0/2495522762 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe9c4039650 0x7fe9c403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:55.406 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.402+0000 7fe9dd2bc700 1 --2- 172.21.15.145:0/2495522762 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe9c4039650 0x7fe9c403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006fd0 tx=0x7fe9c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:55.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.406+0000 7fe9ceffd700 1 -- 172.21.15.145:0/2495522762 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe9d4013070 con 0x7fe9d8100da0 2024-04-02T13:29:55.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:55 smithi033 ceph-mon[32390]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:29:55.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:55 smithi033 ceph-mon[32390]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:29:55.620 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.616+0000 7fe9dfd21700 1 -- 172.21.15.145:0/2495522762 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi050:172.21.15.50=b;smithi145:172.21.15.145=c", "target": ["mon-mgr", ""]}) -- 0x7fe9d80fe180 con 0x7fe9c4039650 2024-04-02T13:29:55.627 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.624+0000 7fe9ceffd700 1 -- 172.21.15.145:0/2495522762 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe9d80fe180 con 0x7fe9c4039650 2024-04-02T13:29:55.627 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mon update... 2024-04-02T13:29:55.629 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.625+0000 7fe9dfd21700 1 -- 172.21.15.145:0/2495522762 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe9c4039650 msgr2=0x7fe9c403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:55.629 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.625+0000 7fe9dfd21700 1 --2- 172.21.15.145:0/2495522762 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe9c4039650 0x7fe9c403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006fd0 tx=0x7fe9c8006e40 comp rx=0 tx=0).stop 2024-04-02T13:29:55.629 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.625+0000 7fe9dfd21700 1 -- 172.21.15.145:0/2495522762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d8100da0 msgr2=0x7fe9d8199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:55.629 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.625+0000 7fe9dfd21700 1 --2- 172.21.15.145:0/2495522762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d8100da0 0x7fe9d8199d90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4006010 tx=0x7fe9d4005810 comp rx=0 tx=0).stop 2024-04-02T13:29:55.629 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.625+0000 7fe9dfd21700 1 -- 172.21.15.145:0/2495522762 shutdown_connections 2024-04-02T13:29:55.629 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.625+0000 7fe9dfd21700 1 --2- 172.21.15.145:0/2495522762 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe9c4039650 0x7fe9c403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:55.629 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.625+0000 7fe9dfd21700 1 --2- 172.21.15.145:0/2495522762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9d8100da0 0x7fe9d8199d90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:55.629 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.625+0000 7fe9dfd21700 1 -- 172.21.15.145:0/2495522762 >> 172.21.15.145:0/2495522762 conn(0x7fe9d80fc9e0 msgr2=0x7fe9d80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:55.629 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.626+0000 7fe9dfd21700 1 -- 172.21.15.145:0/2495522762 shutdown_connections 2024-04-02T13:29:55.630 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:29:55.626+0000 7fe9dfd21700 1 -- 172.21.15.145:0/2495522762 wait complete. 2024-04-02T13:29:56.205 DEBUG:teuthology.orchestra.run.smithi050:mon.b> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.b.service 2024-04-02T13:29:56.208 DEBUG:teuthology.orchestra.run.smithi145:mon.c> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.c.service 2024-04-02T13:29:56.210 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T13:29:56.210 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T13:29:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:56 smithi033 ceph-mon[32390]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi050:172.21.15.50=b;smithi145:172.21.15.145=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:56 smithi033 ceph-mon[32390]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi050:172.21.15.50=b;smithi145:172.21.15.145=c;count:3 2024-04-02T13:29:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:57.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:57.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:57.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:57.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:29:57.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:29:57.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:57.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:56 smithi033 ceph-mon[32390]: Deploying daemon mon.c on smithi145 2024-04-02T13:29:57.252 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:29:58.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:57 smithi033 ceph-mon[32390]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:29:59 smithi033 ceph-mon[32390]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:00.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.068+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1135723382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e81031d0 msgr2=0x7f91e81035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:00.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.068+0000 7f91ef2e5700 1 --2- 172.21.15.145:0/1135723382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e81031d0 0x7f91e81035b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009910 tx=0x7f91e4009c20 comp rx=0 tx=0).stop 2024-04-02T13:30:00.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.068+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1135723382 shutdown_connections 2024-04-02T13:30:00.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.068+0000 7f91ef2e5700 1 --2- 172.21.15.145:0/1135723382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e81031d0 0x7f91e81035b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:00.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.068+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1135723382 >> 172.21.15.145:0/1135723382 conn(0x7f91e80fc9a0 msgr2=0x7f91e80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.068+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1135723382 shutdown_connections 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.068+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1135723382 wait complete. 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.069+0000 7f91ef2e5700 1 Processor -- start 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.069+0000 7f91ef2e5700 1 -- start start 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.069+0000 7f91ef2e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e81031d0 0x7f91e819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.069+0000 7f91ef2e5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f91e810c640 con 0x7f91e81031d0 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.069+0000 7f91ed081700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e81031d0 0x7f91e819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.069+0000 7f91ed081700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e81031d0 0x7f91e819d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.145:33804/0 (socket says 172.21.15.145:33804) 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.069+0000 7f91ed081700 1 -- 172.21.15.145:0/1436418016 learned_addr learned my addr 172.21.15.145:0/1436418016 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.070+0000 7f91ed081700 1 -- 172.21.15.145:0/1436418016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e819dd50 con 0x7f91e81031d0 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.070+0000 7f91ed081700 1 --2- 172.21.15.145:0/1436418016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e81031d0 0x7f91e819d810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91e4006010 tx=0x7f91e4005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.070+0000 7f91de7fc700 1 -- 172.21.15.145:0/1436418016 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91e4027070 con 0x7f91e81031d0 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.070+0000 7f91de7fc700 1 -- 172.21.15.145:0/1436418016 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f91e400be30 con 0x7f91e81031d0 2024-04-02T13:30:00.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.070+0000 7f91de7fc700 1 -- 172.21.15.145:0/1436418016 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91e4022e10 con 0x7f91e81031d0 2024-04-02T13:30:00.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.070+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1436418016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e819dfe0 con 0x7f91e81031d0 2024-04-02T13:30:00.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.070+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1436418016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e81979d0 con 0x7f91e81031d0 2024-04-02T13:30:00.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.071+0000 7f91de7fc700 1 -- 172.21.15.145:0/1436418016 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f91e402a070 con 0x7f91e81031d0 2024-04-02T13:30:00.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.071+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1436418016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e806aaf0 con 0x7f91e81031d0 2024-04-02T13:30:00.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.071+0000 7f91de7fc700 1 --2- 172.21.15.145:0/1436418016 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f91d4039600 0x7f91d403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:00.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.071+0000 7f91de7fc700 1 -- 172.21.15.145:0/1436418016 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f91e401e030 con 0x7f91e81031d0 2024-04-02T13:30:00.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.072+0000 7f91ec880700 1 --2- 172.21.15.145:0/1436418016 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f91d4039600 0x7f91d403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:00.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.072+0000 7f91ec880700 1 --2- 172.21.15.145:0/1436418016 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f91d4039600 0x7f91d403bac0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f91d8006fd0 tx=0x7f91d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:00.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.077+0000 7f91de7fc700 1 -- 172.21.15.145:0/1436418016 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f91e4013070 con 0x7f91e81031d0 2024-04-02T13:30:00.396 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.392+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1436418016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f91e8198f00 con 0x7f91e81031d0 2024-04-02T13:30:00.396 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.393+0000 7f91de7fc700 1 -- 172.21.15.145:0/1436418016 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f91e40292c0 con 0x7f91e81031d0 2024-04-02T13:30:00.397 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:30:00.397 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":1,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","modified":"2024-04-02T13:28:18.700852Z","created":"2024-04-02T13:28:18.700852Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T13:30:00.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.395+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1436418016 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f91d4039600 msgr2=0x7f91d403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:00.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.395+0000 7f91ef2e5700 1 --2- 172.21.15.145:0/1436418016 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f91d4039600 0x7f91d403bac0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f91d8006fd0 tx=0x7f91d8006e40 comp rx=0 tx=0).stop 2024-04-02T13:30:00.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.395+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1436418016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e81031d0 msgr2=0x7f91e819d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:00.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.395+0000 7f91ef2e5700 1 --2- 172.21.15.145:0/1436418016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e81031d0 0x7f91e819d810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91e4006010 tx=0x7f91e4005810 comp rx=0 tx=0).stop 2024-04-02T13:30:00.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.395+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1436418016 shutdown_connections 2024-04-02T13:30:00.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.395+0000 7f91ef2e5700 1 --2- 172.21.15.145:0/1436418016 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f91d4039600 0x7f91d403bac0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:00.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.395+0000 7f91ef2e5700 1 --2- 172.21.15.145:0/1436418016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e81031d0 0x7f91e819d810 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:00.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.395+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1436418016 >> 172.21.15.145:0/1436418016 conn(0x7f91e80fc9a0 msgr2=0x7f91e8103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:00.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.395+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1436418016 shutdown_connections 2024-04-02T13:30:00.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:00.395+0000 7f91ef2e5700 1 -- 172.21.15.145:0/1436418016 wait complete. 2024-04-02T13:30:00.400 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 1 2024-04-02T13:30:00.441 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 podman[35600]: 2024-04-02 13:30:00.414546461 +0000 UTC m=+0.349705723 container start 6fc32637c96fb23f295b8eeb947efd83e50fd843f68652aebb0f5061c1f93f10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-c, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD) 2024-04-02T13:30:00.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T13:30:00.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev), process ceph-mon, pid 2 2024-04-02T13:30:00.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: pidfile_write: ignore empty --pid-file 2024-04-02T13:30:00.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: load: jerasure load: lrc 2024-04-02T13:30:00.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: RocksDB version: 7.9.2 2024-04-02T13:30:00.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Git sha 0 2024-04-02T13:30:00.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Compile date 2024-04-02 01:02:34 2024-04-02T13:30:00.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: DB SUMMARY 2024-04-02T13:30:00.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: DB Session ID: YQWLFP11YASYQ05RG65Z 2024-04-02T13:30:00.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: CURRENT file: CURRENT 2024-04-02T13:30:00.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T13:30:00.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-02T13:30:00.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-02T13:30:00.733 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-02T13:30:00.733 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.error_if_exists: 0 2024-04-02T13:30:00.733 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.create_if_missing: 0 2024-04-02T13:30:00.733 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.paranoid_checks: 1 2024-04-02T13:30:00.733 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T13:30:00.733 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T13:30:00.733 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T13:30:00.733 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.env: 0x5582b55c5c60 2024-04-02T13:30:00.733 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T13:30:00.733 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.info_log: 0x5582b79e7ae0 2024-04-02T13:30:00.734 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T13:30:00.734 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.statistics: (nil) 2024-04-02T13:30:00.734 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.use_fsync: 0 2024-04-02T13:30:00.734 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_log_file_size: 0 2024-04-02T13:30:00.734 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T13:30:00.734 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T13:30:00.734 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T13:30:00.734 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T13:30:00.734 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.allow_fallocate: 1 2024-04-02T13:30:00.734 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T13:30:00.734 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T13:30:00.734 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.use_direct_reads: 0 2024-04-02T13:30:00.735 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T13:30:00.735 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T13:30:00.735 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.db_log_dir: 2024-04-02T13:30:00.735 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.wal_dir: 2024-04-02T13:30:00.735 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T13:30:00.735 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T13:30:00.735 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T13:30:00.735 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T13:30:00.735 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T13:30:00.735 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T13:30:00.735 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T13:30:00.735 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T13:30:00.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.write_buffer_manager: 0x5582b79b81e0 2024-04-02T13:30:00.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T13:30:00.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T13:30:00.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T13:30:00.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T13:30:00.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T13:30:00.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T13:30:00.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T13:30:00.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T13:30:00.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.unordered_write: 0 2024-04-02T13:30:00.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T13:30:00.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T13:30:00.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T13:30:00.737 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T13:30:00.737 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.row_cache: None 2024-04-02T13:30:00.737 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.wal_filter: None 2024-04-02T13:30:00.737 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T13:30:00.737 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T13:30:00.737 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.two_write_queues: 0 2024-04-02T13:30:00.737 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T13:30:00.737 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.wal_compression: 0 2024-04-02T13:30:00.737 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.atomic_flush: 0 2024-04-02T13:30:00.737 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T13:30:00.738 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T13:30:00.738 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T13:30:00.738 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.log_readahead_size: 0 2024-04-02T13:30:00.738 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T13:30:00.738 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T13:30:00.738 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T13:30:00.738 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T13:30:00.738 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T13:30:00.738 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T13:30:00.738 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T13:30:00.738 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_background_jobs: 2 2024-04-02T13:30:00.738 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_background_compactions: -1 2024-04-02T13:30:00.738 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_subcompactions: 1 2024-04-02T13:30:00.739 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T13:30:00.739 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T13:30:00.739 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T13:30:00.739 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T13:30:00.739 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T13:30:00.742 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T13:30:00.742 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T13:30:00.742 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T13:30:00.742 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_open_files: -1 2024-04-02T13:30:00.742 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T13:30:00.742 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T13:30:00.743 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T13:30:00.743 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T13:30:00.743 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_background_flushes: -1 2024-04-02T13:30:00.743 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Compression algorithms supported: 2024-04-02T13:30:00.743 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T13:30:00.743 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: kZSTD supported: 0 2024-04-02T13:30:00.743 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: kXpressCompression supported: 0 2024-04-02T13:30:00.743 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T13:30:00.744 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: kZlibCompression supported: 1 2024-04-02T13:30:00.744 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: kSnappyCompression supported: 1 2024-04-02T13:30:00.744 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: kLZ4Compression supported: 1 2024-04-02T13:30:00.744 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: kBZip2Compression supported: 0 2024-04-02T13:30:00.744 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T13:30:00.744 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T13:30:00.744 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-02T13:30:00.744 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T13:30:00.744 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T13:30:00.744 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.merge_operator: 2024-04-02T13:30:00.744 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compaction_filter: None 2024-04-02T13:30:00.744 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T13:30:00.744 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T13:30:00.745 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T13:30:00.745 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T13:30:00.745 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5582b79e7cc0) 2024-04-02T13:30:00.745 INFO:journalctl@ceph.mon.c.smithi145.stdout: cache_index_and_filter_blocks: 1 2024-04-02T13:30:00.745 INFO:journalctl@ceph.mon.c.smithi145.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T13:30:00.745 INFO:journalctl@ceph.mon.c.smithi145.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T13:30:00.745 INFO:journalctl@ceph.mon.c.smithi145.stdout: pin_top_level_index_and_filter: 1 2024-04-02T13:30:00.745 INFO:journalctl@ceph.mon.c.smithi145.stdout: index_type: 0 2024-04-02T13:30:00.745 INFO:journalctl@ceph.mon.c.smithi145.stdout: data_block_index_type: 0 2024-04-02T13:30:00.745 INFO:journalctl@ceph.mon.c.smithi145.stdout: index_shortening: 1 2024-04-02T13:30:00.745 INFO:journalctl@ceph.mon.c.smithi145.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T13:30:00.745 INFO:journalctl@ceph.mon.c.smithi145.stdout: checksum: 4 2024-04-02T13:30:00.745 INFO:journalctl@ceph.mon.c.smithi145.stdout: no_block_cache: 0 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_cache: 0x5582b79eb090 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_cache_name: BinnedLRUCache 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_cache_options: 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: capacity : 536870912 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: num_shard_bits : 4 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: strict_capacity_limit : 0 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: high_pri_pool_ratio: 0.000 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_cache_compressed: (nil) 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: persistent_cache: (nil) 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_size: 4096 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_size_deviation: 10 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_restart_interval: 16 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: index_block_restart_interval: 1 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: metadata_block_size: 4096 2024-04-02T13:30:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout: partition_filters: 0 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout: use_delta_encoding: 1 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout: filter_policy: bloomfilter 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout: whole_key_filtering: 1 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout: verify_compression: 0 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout: read_amp_bytes_per_bit: 0 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout: format_version: 5 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout: enable_index_compression: 1 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout: block_align: 0 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout: max_auto_readahead_size: 262144 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout: prepopulate_block_cache: 0 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout: initial_auto_readahead_size: 8192 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T13:30:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compression: NoCompression 2024-04-02T13:30:00.748 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T13:30:00.748 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T13:30:00.748 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T13:30:00.748 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.num_levels: 7 2024-04-02T13:30:00.748 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T13:30:00.748 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T13:30:00.748 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T13:30:00.748 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T13:30:00.748 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T13:30:00.748 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T13:30:00.748 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T13:30:00.748 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:30:00.748 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T13:30:00.749 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T13:30:00.750 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T13:30:00.750 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T13:30:00.750 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T13:30:00.750 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T13:30:00.750 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T13:30:00.750 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T13:30:00.750 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T13:30:00.750 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T13:30:00.750 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T13:30:00.750 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T13:30:00.750 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T13:30:00.750 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T13:30:00.750 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T13:30:00.751 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T13:30:00.751 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T13:30:00.751 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T13:30:00.751 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T13:30:00.751 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T13:30:00.751 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T13:30:00.751 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T13:30:00.751 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T13:30:00.751 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T13:30:00.751 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T13:30:00.751 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T13:30:00.751 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T13:30:00.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T13:30:00.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T13:30:00.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T13:30:00.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T13:30:00.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-02T13:30:00.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.inplace_update_support: 0 2024-04-02T13:30:00.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T13:30:00.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T13:30:00.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T13:30:00.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T13:30:00.753 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.bloom_locality: 0 2024-04-02T13:30:00.753 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.max_successive_merges: 0 2024-04-02T13:30:00.753 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T13:30:00.753 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T13:30:00.753 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T13:30:00.753 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T13:30:00.753 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.ttl: 2592000 2024-04-02T13:30:00.753 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T13:30:00.753 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T13:30:00.753 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T13:30:00.753 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.enable_blob_files: false 2024-04-02T13:30:00.754 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.min_blob_size: 0 2024-04-02T13:30:00.754 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T13:30:00.754 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T13:30:00.754 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T13:30:00.754 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T13:30:00.754 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T13:30:00.754 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T13:30:00.754 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T13:30:00.754 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T13:30:00.754 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-02T13:30:00.755 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-02T13:30:00.755 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f9d27625-c04f-4ce8-a4e8-527d0a3b8c12 2024-04-02T13:30:00.755 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064600445253, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-02T13:30:00.756 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-02T13:30:00.756 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064600445776, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712064600, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f9d27625-c04f-4ce8-a4e8-527d0a3b8c12", "db_session_id": "YQWLFP11YASYQ05RG65Z", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-02T13:30:00.756 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064600445871, "job": 1, "event": "recovery_finished"} 2024-04-02T13:30:00.756 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-02T13:30:00.756 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T13:30:00.756 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5582b7ada000 2024-04-02T13:30:00.756 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: DB pointer 0x5582b7ac4000 2024-04-02T13:30:00.756 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T13:30:00.756 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T13:30:00.757 INFO:journalctl@ceph.mon.c.smithi145.stdout: ** DB Stats ** 2024-04-02T13:30:00.757 INFO:journalctl@ceph.mon.c.smithi145.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:30:00.757 INFO:journalctl@ceph.mon.c.smithi145.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T13:30:00.757 INFO:journalctl@ceph.mon.c.smithi145.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:30:00.757 INFO:journalctl@ceph.mon.c.smithi145.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:30:00.757 INFO:journalctl@ceph.mon.c.smithi145.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T13:30:00.757 INFO:journalctl@ceph.mon.c.smithi145.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:30:00.757 INFO:journalctl@ceph.mon.c.smithi145.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:30:00.758 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-04-02T13:30:00.758 INFO:journalctl@ceph.mon.c.smithi145.stdout: ** Compaction Stats [default] ** 2024-04-02T13:30:00.758 INFO:journalctl@ceph.mon.c.smithi145.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:30:00.758 INFO:journalctl@ceph.mon.c.smithi145.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T13:30:00.758 INFO:journalctl@ceph.mon.c.smithi145.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:30:00.758 INFO:journalctl@ceph.mon.c.smithi145.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:30:00.758 INFO:journalctl@ceph.mon.c.smithi145.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:30:00.758 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-04-02T13:30:00.758 INFO:journalctl@ceph.mon.c.smithi145.stdout: ** Compaction Stats [default] ** 2024-04-02T13:30:00.758 INFO:journalctl@ceph.mon.c.smithi145.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:30:00.758 INFO:journalctl@ceph.mon.c.smithi145.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T13:30:00.758 INFO:journalctl@ceph.mon.c.smithi145.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:30:00.758 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: Cumulative compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: Interval compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: Block cache BinnedLRUCache@0x5582b79eb090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T13:30:00.759 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-02T13:30:00.760 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: using public_addr v2:172.21.15.145:0/0 -> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] 2024-04-02T13:30:00.760 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: starting mon.c rank -1 at public addrs [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] at bind addrs [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:00.760 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(???) e0 preinit fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:00.761 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(synchronizing).mds e1 new map 2024-04-02T13:30:00.761 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-02T13:30:00.761 INFO:journalctl@ceph.mon.c.smithi145.stdout: e1 2024-04-02T13:30:00.761 INFO:journalctl@ceph.mon.c.smithi145.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T13:30:00.761 INFO:journalctl@ceph.mon.c.smithi145.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T13:30:00.761 INFO:journalctl@ceph.mon.c.smithi145.stdout: legacy client fscid: -1 2024-04-02T13:30:00.761 INFO:journalctl@ceph.mon.c.smithi145.stdout: 2024-04-02T13:30:00.762 INFO:journalctl@ceph.mon.c.smithi145.stdout: No filesystems configured 2024-04-02T13:30:00.762 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-02T13:30:00.762 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-02T13:30:00.762 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-02T13:30:00.762 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-02T13:30:00.762 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-02T13:30:00.762 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-02T13:30:00.762 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-02T13:30:00.762 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:30:00.762 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:30:00.762 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:30:00.762 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:00.763 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:30:00.763 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.763 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:00.763 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi145", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:00.763 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.763 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.763 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.763 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.763 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:00.763 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:00.763 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:00.763 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:00.763 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: Updating smithi050:/etc/ceph/ceph.conf 2024-04-02T13:30:00.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: Deploying cephadm binary to smithi145 2024-04-02T13:30:00.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:30:00.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:30:00.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:30:00.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:00.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:00.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: Added host smithi145 2024-04-02T13:30:00.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1572715621' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:00.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: Updating smithi145:/etc/ceph/ceph.conf 2024-04-02T13:30:00.766 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1572715621' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-02T13:30:00.766 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:30:00.766 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:30:00.766 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:30:00.766 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.766 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.766 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.766 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:30:00.766 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:00.766 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi050:172.21.15.50=b;smithi145:172.21.15.145=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:00.766 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi050:172.21.15.50=b;smithi145:172.21.15.145=c;count:3 2024-04-02T13:30:00.767 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.767 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:00.767 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:00.767 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:00.767 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:00.767 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:00.767 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:00.767 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: Deploying daemon mon.c on smithi145 2024-04-02T13:30:00.767 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:00.767 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:00.767 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 ceph-mon[35646]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-02T13:30:01.090 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 bash[35600]: 6fc32637c96fb23f295b8eeb947efd83e50fd843f68652aebb0f5061c1f93f10 2024-04-02T13:30:01.091 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:00 smithi145 systemd[1]: Started Ceph mon.c for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:30:02.211 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T13:30:02.212 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T13:30:02.459 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:30:05.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: mon.a calling monitor election 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: mon.c calling monitor election 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: monmap epoch 2 2024-04-02T13:30:05.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: last_changed 2024-04-02T13:30:00.470114+0000 2024-04-02T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: created 2024-04-02T13:28:18.700852+0000 2024-04-02T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: min_mon_release 19 (squid) 2024-04-02T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: election_strategy: 1 2024-04-02T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-02T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-04-02T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: fsmap 2024-04-02T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: mgrmap e13: x(active, since 49s) 2024-04-02T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: overall HEALTH_OK 2024-04-02T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:05 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:06.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:06.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:30:06.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: mon.a calling monitor election 2024-04-02T13:30:06.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:06.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:06.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:06.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:06.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: mon.c calling monitor election 2024-04-02T13:30:06.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:06.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:06.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:06.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:06.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:30:06.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: monmap epoch 2 2024-04-02T13:30:06.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:06.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: last_changed 2024-04-02T13:30:00.470114+0000 2024-04-02T13:30:06.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: created 2024-04-02T13:28:18.700852+0000 2024-04-02T13:30:06.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: min_mon_release 19 (squid) 2024-04-02T13:30:06.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: election_strategy: 1 2024-04-02T13:30:06.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-02T13:30:06.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-04-02T13:30:06.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: fsmap 2024-04-02T13:30:06.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:30:06.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: mgrmap e13: x(active, since 49s) 2024-04-02T13:30:06.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: overall HEALTH_OK 2024-04-02T13:30:06.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:06.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:05 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:06.979 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.975+0000 7eff8de73700 1 -- 172.21.15.145:0/884967985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff88101b50 msgr2=0x7eff70005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:06.979 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.975+0000 7eff8de73700 1 --2- 172.21.15.145:0/884967985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff88101b50 0x7eff70005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff78005fa0 tx=0x7eff7800ffc0 comp rx=0 tx=0).stop 2024-04-02T13:30:06.980 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.975+0000 7eff8de73700 1 -- 172.21.15.145:0/884967985 shutdown_connections 2024-04-02T13:30:06.980 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.975+0000 7eff8de73700 1 --2- 172.21.15.145:0/884967985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff88101b50 0x7eff70005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:06.980 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.975+0000 7eff8de73700 1 -- 172.21.15.145:0/884967985 >> 172.21.15.145:0/884967985 conn(0x7eff880775b0 msgr2=0x7eff880779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:06.980 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.975+0000 7eff8de73700 1 -- 172.21.15.145:0/884967985 shutdown_connections 2024-04-02T13:30:06.980 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.976+0000 7eff8de73700 1 -- 172.21.15.145:0/884967985 wait complete. 2024-04-02T13:30:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:06 smithi145 ceph-mon[35646]: Deploying daemon mon.b on smithi050 2024-04-02T13:30:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:06.985 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.976+0000 7eff8de73700 1 Processor -- start 2024-04-02T13:30:06.985 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.977+0000 7eff8de73700 1 -- start start 2024-04-02T13:30:06.985 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.977+0000 7eff8de73700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff88101b50 0x7eff88199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:06.985 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.977+0000 7eff8de73700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8819a360 0x7eff8819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:06.985 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.977+0000 7eff8de73700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff88109070 con 0x7eff88101b50 2024-04-02T13:30:06.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.977+0000 7eff8de73700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7eff88108ef0 con 0x7eff8819a360 2024-04-02T13:30:06.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.977+0000 7eff86ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8819a360 0x7eff8819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:06.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.977+0000 7eff877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff88101b50 0x7eff88199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:06.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.977+0000 7eff877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff88101b50 0x7eff88199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.145:54804/0 (socket says 172.21.15.145:54804) 2024-04-02T13:30:06.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.977+0000 7eff877fe700 1 -- 172.21.15.145:0/1990725262 learned_addr learned my addr 172.21.15.145:0/1990725262 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-02T13:30:06.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.977+0000 7eff86ffd700 1 -- 172.21.15.145:0/1990725262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8819a360 msgr2=0x7eff8819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-02T13:30:06.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.977+0000 7eff86ffd700 1 -- 172.21.15.145:0/1990725262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8819a360 msgr2=0x7eff8819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T13:30:06.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.977+0000 7eff86ffd700 1 --2- 172.21.15.145:0/1990725262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8819a360 0x7eff8819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T13:30:06.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.977+0000 7eff86ffd700 1 --2- 172.21.15.145:0/1990725262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8819a360 0x7eff8819e6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:30:06.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.978+0000 7eff877fe700 1 -- 172.21.15.145:0/1990725262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8819a360 msgr2=0x7eff8819e6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:30:06.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.978+0000 7eff877fe700 1 --2- 172.21.15.145:0/1990725262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8819a360 0x7eff8819e6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:06.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.978+0000 7eff877fe700 1 -- 172.21.15.145:0/1990725262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8819a9a0 con 0x7eff88101b50 2024-04-02T13:30:06.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.978+0000 7eff877fe700 1 --2- 172.21.15.145:0/1990725262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff88101b50 0x7eff88199e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff780059d0 tx=0x7eff78004d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:06.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.978+0000 7eff84ff9700 1 -- 172.21.15.145:0/1990725262 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7eff780052b0 con 0x7eff88101b50 2024-04-02T13:30:06.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.978+0000 7eff84ff9700 1 -- 172.21.15.145:0/1990725262 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7eff78005450 con 0x7eff88101b50 2024-04-02T13:30:06.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.978+0000 7eff84ff9700 1 -- 172.21.15.145:0/1990725262 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7eff78011670 con 0x7eff88101b50 2024-04-02T13:30:06.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.978+0000 7eff8de73700 1 -- 172.21.15.145:0/1990725262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8819ec30 con 0x7eff88101b50 2024-04-02T13:30:06.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.978+0000 7eff8de73700 1 -- 172.21.15.145:0/1990725262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8819f190 con 0x7eff88101b50 2024-04-02T13:30:06.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.979+0000 7eff84ff9700 1 -- 172.21.15.145:0/1990725262 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7eff78011920 con 0x7eff88101b50 2024-04-02T13:30:06.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.979+0000 7eff84ff9700 1 --2- 172.21.15.145:0/1990725262 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7eff74039460 0x7eff7403b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:06.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.979+0000 7eff84ff9700 1 -- 172.21.15.145:0/1990725262 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7eff7804fdb0 con 0x7eff88101b50 2024-04-02T13:30:06.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.980+0000 7eff8de73700 1 -- 172.21.15.145:0/1990725262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff88102900 con 0x7eff88101b50 2024-04-02T13:30:06.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.980+0000 7eff86ffd700 1 --2- 172.21.15.145:0/1990725262 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7eff74039460 0x7eff7403b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:06.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.980+0000 7eff86ffd700 1 --2- 172.21.15.145:0/1990725262 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7eff74039460 0x7eff7403b920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009850 tx=0x7eff7c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:06.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:06.986+0000 7eff84ff9700 1 -- 172.21.15.145:0/1990725262 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7eff78022070 con 0x7eff88101b50 2024-04-02T13:30:07.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:07.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:07.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:07.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:06 smithi033 ceph-mon[32390]: Deploying daemon mon.b on smithi050 2024-04-02T13:30:07.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:07.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:07.265+0000 7eff8de73700 1 -- 172.21.15.145:0/1990725262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7eff88063c20 con 0x7eff88101b50 2024-04-02T13:30:07.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:07.266+0000 7eff84ff9700 1 -- 172.21.15.145:0/1990725262 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7eff7802d360 con 0x7eff88101b50 2024-04-02T13:30:07.269 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:30:07.269 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":2,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","modified":"2024-04-02T13:30:00.470114Z","created":"2024-04-02T13:28:18.700852Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-02T13:30:07.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:07.268+0000 7eff8de73700 1 -- 172.21.15.145:0/1990725262 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7eff74039460 msgr2=0x7eff7403b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:07.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:07.268+0000 7eff8de73700 1 --2- 172.21.15.145:0/1990725262 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7eff74039460 0x7eff7403b920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009850 tx=0x7eff7c008040 comp rx=0 tx=0).stop 2024-04-02T13:30:07.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:07.268+0000 7eff8de73700 1 -- 172.21.15.145:0/1990725262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff88101b50 msgr2=0x7eff88199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:07.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:07.268+0000 7eff8de73700 1 --2- 172.21.15.145:0/1990725262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff88101b50 0x7eff88199e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff780059d0 tx=0x7eff78004d80 comp rx=0 tx=0).stop 2024-04-02T13:30:07.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:07.268+0000 7eff8de73700 1 -- 172.21.15.145:0/1990725262 shutdown_connections 2024-04-02T13:30:07.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:07.268+0000 7eff8de73700 1 --2- 172.21.15.145:0/1990725262 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7eff74039460 0x7eff7403b920 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:07.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:07.268+0000 7eff8de73700 1 --2- 172.21.15.145:0/1990725262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff88101b50 0x7eff88199e20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:07.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:07.268+0000 7eff8de73700 1 --2- 172.21.15.145:0/1990725262 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff8819a360 0x7eff8819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:07.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:07.268+0000 7eff8de73700 1 -- 172.21.15.145:0/1990725262 >> 172.21.15.145:0/1990725262 conn(0x7eff880775b0 msgr2=0x7eff88104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:07.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:07.268+0000 7eff8de73700 1 -- 172.21.15.145:0/1990725262 shutdown_connections 2024-04-02T13:30:07.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:07.268+0000 7eff8de73700 1 -- 172.21.15.145:0/1990725262 wait complete. 2024-04-02T13:30:07.272 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 2 2024-04-02T13:30:07.830 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:07 smithi145 ceph-mon[35646]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:07.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:07 smithi145 ceph-mon[35646]: from='client.? 172.21.15.145:0/1990725262' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:30:08.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:07 smithi033 ceph-mon[32390]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:08.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:07 smithi033 ceph-mon[32390]: from='client.? 172.21.15.145:0/1990725262' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:30:08.858 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T13:30:08.858 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T13:30:09.103 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:30:09.354 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 podman[35502]: 2024-04-02 13:30:09.202529574 +0000 UTC m=+0.019189839 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:30:09.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 podman[35502]: 2024-04-02 13:30:09.508939043 +0000 UTC m=+0.325599291 container init d9039c32ac9c28b12a710aa174ecba1dfec048b4c6ac817d2df3896a71414d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:30:09.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 podman[35502]: 2024-04-02 13:30:09.512242018 +0000 UTC m=+0.328902267 container start d9039c32ac9c28b12a710aa174ecba1dfec048b4c6ac817d2df3896a71414d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:30:09.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T13:30:09.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev), process ceph-mon, pid 2 2024-04-02T13:30:09.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pidfile_write: ignore empty --pid-file 2024-04-02T13:30:09.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: load: jerasure load: lrc 2024-04-02T13:30:09.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: RocksDB version: 7.9.2 2024-04-02T13:30:09.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Git sha 0 2024-04-02T13:30:09.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Compile date 2024-04-02 01:02:34 2024-04-02T13:30:09.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: DB SUMMARY 2024-04-02T13:30:09.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: DB Session ID: GCPVM5QI9SXJME5ORFVK 2024-04-02T13:30:09.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: CURRENT file: CURRENT 2024-04-02T13:30:09.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T13:30:09.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-02T13:30:09.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-02T13:30:09.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-02T13:30:09.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.error_if_exists: 0 2024-04-02T13:30:09.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.create_if_missing: 0 2024-04-02T13:30:09.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.paranoid_checks: 1 2024-04-02T13:30:09.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T13:30:09.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T13:30:09.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T13:30:09.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.env: 0x55fe85798c60 2024-04-02T13:30:09.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T13:30:09.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.info_log: 0x55fe86b89ae0 2024-04-02T13:30:09.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T13:30:09.748 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.statistics: (nil) 2024-04-02T13:30:09.749 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.use_fsync: 0 2024-04-02T13:30:09.749 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_log_file_size: 0 2024-04-02T13:30:09.749 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T13:30:09.749 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T13:30:09.749 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T13:30:09.749 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T13:30:09.749 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.allow_fallocate: 1 2024-04-02T13:30:09.749 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T13:30:09.749 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T13:30:09.749 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.use_direct_reads: 0 2024-04-02T13:30:09.749 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T13:30:09.749 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T13:30:09.749 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.db_log_dir: 2024-04-02T13:30:09.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.wal_dir: 2024-04-02T13:30:09.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T13:30:09.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T13:30:09.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T13:30:09.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T13:30:09.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T13:30:09.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T13:30:09.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T13:30:09.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T13:30:09.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.write_buffer_manager: 0x55fe86b5a1e0 2024-04-02T13:30:09.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T13:30:09.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T13:30:09.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.unordered_write: 0 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.row_cache: None 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.wal_filter: None 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T13:30:09.751 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.two_write_queues: 0 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.wal_compression: 0 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.atomic_flush: 0 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.log_readahead_size: 0 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T13:30:09.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_background_jobs: 2 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_background_compactions: -1 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_subcompactions: 1 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_open_files: -1 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T13:30:09.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_background_flushes: -1 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Compression algorithms supported: 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: kZSTD supported: 0 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: kXpressCompression supported: 0 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: kZlibCompression supported: 1 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: kSnappyCompression supported: 1 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: kLZ4Compression supported: 1 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: kBZip2Compression supported: 0 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T13:30:09.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.merge_operator: 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compaction_filter: None 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fe86b89cc0) 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout: cache_index_and_filter_blocks: 1 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout: pin_top_level_index_and_filter: 1 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout: index_type: 0 2024-04-02T13:30:09.755 INFO:journalctl@ceph.mon.b.smithi050.stdout: data_block_index_type: 0 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: index_shortening: 1 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: checksum: 4 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: no_block_cache: 0 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_cache: 0x55fe86b8d090 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_cache_name: BinnedLRUCache 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_cache_options: 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: capacity : 536870912 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: num_shard_bits : 4 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: strict_capacity_limit : 0 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: high_pri_pool_ratio: 0.000 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_cache_compressed: (nil) 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: persistent_cache: (nil) 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_size: 4096 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_size_deviation: 10 2024-04-02T13:30:09.756 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_restart_interval: 16 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: index_block_restart_interval: 1 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: metadata_block_size: 4096 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: partition_filters: 0 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: use_delta_encoding: 1 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: filter_policy: bloomfilter 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: whole_key_filtering: 1 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: verify_compression: 0 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: read_amp_bytes_per_bit: 0 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: format_version: 5 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: enable_index_compression: 1 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_align: 0 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: max_auto_readahead_size: 262144 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: prepopulate_block_cache: 0 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: initial_auto_readahead_size: 8192 2024-04-02T13:30:09.757 INFO:journalctl@ceph.mon.b.smithi050.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T13:30:09.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T13:30:09.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T13:30:09.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compression: NoCompression 2024-04-02T13:30:09.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T13:30:09.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T13:30:09.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T13:30:09.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.num_levels: 7 2024-04-02T13:30:09.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T13:30:09.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T13:30:09.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T13:30:09.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T13:30:09.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T13:30:09.758 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T13:30:09.759 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T13:30:09.760 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T13:30:09.761 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.inplace_update_support: 0 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.bloom_locality: 0 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.max_successive_merges: 0 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T13:30:09.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.ttl: 2592000 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.enable_blob_files: false 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.min_blob_size: 0 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T13:30:09.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8e766e64-32b0-4c48-b28c-35a8f7503589 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064609542491, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064609542954, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712064609, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8e766e64-32b0-4c48-b28c-35a8f7503589", "db_session_id": "GCPVM5QI9SXJME5ORFVK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064609543058, "job": 1, "event": "recovery_finished"} 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fe86c7c000 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: DB pointer 0x55fe86c66000 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T13:30:09.764 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** DB Stats ** 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** Compaction Stats [default] ** 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:30:09.765 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-04-02T13:30:09.766 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** Compaction Stats [default] ** 2024-04-02T13:30:09.766 INFO:journalctl@ceph.mon.b.smithi050.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:30:09.766 INFO:journalctl@ceph.mon.b.smithi050.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T13:30:09.766 INFO:journalctl@ceph.mon.b.smithi050.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:30:09.766 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-04-02T13:30:09.766 INFO:journalctl@ceph.mon.b.smithi050.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T13:30:09.766 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-04-02T13:30:09.766 INFO:journalctl@ceph.mon.b.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:30:09.766 INFO:journalctl@ceph.mon.b.smithi050.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-02T13:30:09.766 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T13:30:09.766 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T13:30:09.766 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T13:30:09.766 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T13:30:09.767 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:30:09.767 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:30:09.767 INFO:journalctl@ceph.mon.b.smithi050.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T13:30:09.767 INFO:journalctl@ceph.mon.b.smithi050.stdout: Block cache BinnedLRUCache@0x55fe86b8d090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: using public_addr v2:172.21.15.50:0/0 -> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: starting mon.b rank -1 at public addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] at bind addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(???) e0 preinit fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(synchronizing).mds e1 new map 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout: e1 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout: legacy client fscid: -1 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-04-02T13:30:09.768 INFO:journalctl@ceph.mon.b.smithi050.stdout: No filesystems configured 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi145", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: Updating smithi050:/etc/ceph/ceph.conf 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: Deploying cephadm binary to smithi145 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.770 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: Added host smithi145 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1572715621' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.771 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: Updating smithi145:/etc/ceph/ceph.conf 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1572715621' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:30:09.773 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.773 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi050:172.21.15.50=b;smithi145:172.21.15.145=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:09.773 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi050:172.21.15.50=b;smithi145:172.21.15.145=c;count:3 2024-04-02T13:30:09.773 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.773 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:09.773 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:09.773 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:09.773 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.773 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:09.773 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:09.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: Deploying daemon mon.c on smithi145 2024-04-02T13:30:09.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:30:09.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.a calling monitor election 2024-04-02T13:30:09.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:09.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:09.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:09.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.c calling monitor election 2024-04-02T13:30:09.774 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: monmap epoch 2 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: last_changed 2024-04-02T13:30:00.470114+0000 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: created 2024-04-02T13:28:18.700852+0000 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: min_mon_release 19 (squid) 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: election_strategy: 1 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: fsmap 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:30:09.775 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mgrmap e13: x(active, since 49s) 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: overall HEALTH_OK 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: Deploying daemon mon.b on smithi050 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: from='client.? 172.21.15.145:0/1990725262' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 ceph-mon[35524]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 bash[35502]: d9039c32ac9c28b12a710aa174ecba1dfec048b4c6ac817d2df3896a71414d3c 2024-04-02T13:30:09.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:09 smithi050 systemd[1]: Started Ceph mon.b for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:30:09.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:09 smithi145 ceph-mon[35646]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:10.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:09 smithi033 ceph-mon[32390]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:14.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.840+0000 7f2999b74700 1 -- 172.21.15.145:0/2531883840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c0035c0 msgr2=0x7f297c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:14.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.840+0000 7f2999b74700 1 --2- 172.21.15.145:0/2531883840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c0035c0 0x7f297c005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f299406a810 tx=0x7f298c009e80 comp rx=0 tx=0).stop 2024-04-02T13:30:14.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.840+0000 7f2999b74700 1 -- 172.21.15.145:0/2531883840 shutdown_connections 2024-04-02T13:30:14.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.840+0000 7f2999b74700 1 --2- 172.21.15.145:0/2531883840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c0035c0 0x7f297c005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:14.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.840+0000 7f2999b74700 1 -- 172.21.15.145:0/2531883840 >> 172.21.15.145:0/2531883840 conn(0x7f29940fcab0 msgr2=0x7f29940feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:14.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.840+0000 7f2999b74700 1 -- 172.21.15.145:0/2531883840 shutdown_connections 2024-04-02T13:30:14.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.840+0000 7f2999b74700 1 -- 172.21.15.145:0/2531883840 wait complete. 2024-04-02T13:30:14.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.841+0000 7f2999b74700 1 Processor -- start 2024-04-02T13:30:14.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.841+0000 7f2999b74700 1 -- start start 2024-04-02T13:30:14.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.841+0000 7f2999b74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c0035c0 0x7f2994197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:14.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.841+0000 7f2999b74700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2994198290 0x7f299419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:14.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.841+0000 7f2999b74700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f299419cb60 0x7f299419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:14.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.841+0000 7f2999b74700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f299410a0f0 con 0x7f297c0035c0 2024-04-02T13:30:14.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.841+0000 7f2999b74700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2994109f70 con 0x7f299419cb60 2024-04-02T13:30:14.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.841+0000 7f2999b74700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f299410a270 con 0x7f2994198290 2024-04-02T13:30:14.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.841+0000 7f2992ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2994198290 0x7f299419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:14.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.841+0000 7f29937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c0035c0 0x7f2994197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:14.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f29937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c0035c0 0x7f2994197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.145:54874/0 (socket says 172.21.15.145:54874) 2024-04-02T13:30:14.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2992ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2994198290 0x7f299419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57086/0 (socket says 172.21.15.145:57086) 2024-04-02T13:30:14.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f29937fe700 1 -- 172.21.15.145:0/3519200387 learned_addr learned my addr 172.21.15.145:0/3519200387 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-02T13:30:14.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2993fff700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f299419cb60 0x7f299419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:14.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2992ffd700 1 -- 172.21.15.145:0/3519200387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2994198290 msgr2=0x7f299419c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-02T13:30:14.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2992ffd700 1 -- 172.21.15.145:0/3519200387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2994198290 msgr2=0x7f299419c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T13:30:14.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2992ffd700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2994198290 0x7f299419c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2993fff700 1 -- 172.21.15.145:0/3519200387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f299419cb60 msgr2=0x7f299419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2993fff700 1 -- 172.21.15.145:0/3519200387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f299419cb60 msgr2=0x7f299419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2992ffd700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2994198290 0x7f299419c620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2993fff700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f299419cb60 0x7f299419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2993fff700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f299419cb60 0x7f299419d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f29937fe700 1 -- 172.21.15.145:0/3519200387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2994198290 msgr2=0x7f299419c620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f29937fe700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2994198290 0x7f299419c620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f29937fe700 1 -- 172.21.15.145:0/3519200387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f299419cb60 msgr2=0x7f299419d010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f29937fe700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f299419cb60 0x7f299419d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f29937fe700 1 -- 172.21.15.145:0/3519200387 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29941a48d0 con 0x7f297c0035c0 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f29937fe700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c0035c0 0x7f2994197d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f298c002230 tx=0x7f298c0049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2990ff9700 1 -- 172.21.15.145:0/3519200387 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c004c90 con 0x7f297c0035c0 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2990ff9700 1 -- 172.21.15.145:0/3519200387 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f298c026030 con 0x7f297c0035c0 2024-04-02T13:30:14.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2990ff9700 1 -- 172.21.15.145:0/3519200387 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c0059b0 con 0x7f297c0035c0 2024-04-02T13:30:14.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.842+0000 7f2999b74700 1 -- 172.21.15.145:0/3519200387 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29941a4aa0 con 0x7f297c0035c0 2024-04-02T13:30:14.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.843+0000 7f2999b74700 1 -- 172.21.15.145:0/3519200387 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29941a4f90 con 0x7f297c0035c0 2024-04-02T13:30:14.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.844+0000 7f2990ff9700 1 -- 172.21.15.145:0/3519200387 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f298c015040 con 0x7f297c0035c0 2024-04-02T13:30:14.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.844+0000 7f2999b74700 1 -- 172.21.15.145:0/3519200387 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2994063c20 con 0x7f297c0035c0 2024-04-02T13:30:14.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.844+0000 7f2990ff9700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2980039c80 0x7f298003c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:14.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.844+0000 7f2990ff9700 1 -- 172.21.15.145:0/3519200387 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f298c04e870 con 0x7f297c0035c0 2024-04-02T13:30:14.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.844+0000 7f2992ffd700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2980039c80 0x7f298003c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:14.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.845+0000 7f2992ffd700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2980039c80 0x7f298003c140 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2988009850 tx=0x7f2988008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:14.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:14.849+0000 7f2990ff9700 1 -- 172.21.15.145:0/3519200387 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f298c016340 con 0x7f297c0035c0 2024-04-02T13:30:14.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: mon.c calling monitor election 2024-04-02T13:30:14.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:30:14.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: mon.a calling monitor election 2024-04-02T13:30:14.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:14.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:14.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:14.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:14.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:14.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:14.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:14.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:14.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:14.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:30:14.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: monmap epoch 3 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: last_changed 2024-04-02T13:30:09.566182+0000 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: created 2024-04-02T13:28:18.700852+0000 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: min_mon_release 19 (squid) 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: election_strategy: 1 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: fsmap 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: mgrmap e13: x(active, since 58s) 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: overall HEALTH_OK 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:14.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:15.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: mon.c calling monitor election 2024-04-02T13:30:15.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:30:15.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: mon.a calling monitor election 2024-04-02T13:30:15.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:15.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:15.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:15.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:15.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:15.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:15.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: monmap epoch 3 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: last_changed 2024-04-02T13:30:09.566182+0000 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: created 2024-04-02T13:28:18.700852+0000 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: min_mon_release 19 (squid) 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: election_strategy: 1 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: fsmap 2024-04-02T13:30:15.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:30:15.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: mgrmap e13: x(active, since 58s) 2024-04-02T13:30:15.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: overall HEALTH_OK 2024-04-02T13:30:15.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:15.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:15.153 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.150+0000 7f2999b74700 1 -- 172.21.15.145:0/3519200387 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f299402dd30 con 0x7f297c0035c0 2024-04-02T13:30:15.154 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.150+0000 7f2990ff9700 1 -- 172.21.15.145:0/3519200387 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f298c0050e0 con 0x7f297c0035c0 2024-04-02T13:30:15.154 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-04-02T13:30:15.154 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":3,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","modified":"2024-04-02T13:30:09.566182Z","created":"2024-04-02T13:28:18.700852Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-02T13:30:15.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.153+0000 7f2999b74700 1 -- 172.21.15.145:0/3519200387 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2980039c80 msgr2=0x7f298003c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:15.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.153+0000 7f2999b74700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2980039c80 0x7f298003c140 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2988009850 tx=0x7f2988008040 comp rx=0 tx=0).stop 2024-04-02T13:30:15.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.153+0000 7f2999b74700 1 -- 172.21.15.145:0/3519200387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c0035c0 msgr2=0x7f2994197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:15.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.153+0000 7f2999b74700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c0035c0 0x7f2994197d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f298c002230 tx=0x7f298c0049e0 comp rx=0 tx=0).stop 2024-04-02T13:30:15.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.153+0000 7f2999b74700 1 -- 172.21.15.145:0/3519200387 shutdown_connections 2024-04-02T13:30:15.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.153+0000 7f2999b74700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2980039c80 0x7f298003c140 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:15.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.153+0000 7f2999b74700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f299419cb60 0x7f299419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:15.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.153+0000 7f2999b74700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c0035c0 0x7f2994197d50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:15.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.153+0000 7f2999b74700 1 --2- 172.21.15.145:0/3519200387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2994198290 0x7f299419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:15.158 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.153+0000 7f2999b74700 1 -- 172.21.15.145:0/3519200387 >> 172.21.15.145:0/3519200387 conn(0x7f29940fcab0 msgr2=0x7f2994107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:15.158 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.153+0000 7f2999b74700 1 -- 172.21.15.145:0/3519200387 shutdown_connections 2024-04-02T13:30:15.158 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:15.153+0000 7f2999b74700 1 -- 172.21.15.145:0/3519200387 wait complete. 2024-04-02T13:30:15.158 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 3 2024-04-02T13:30:15.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:15 smithi145 ceph-mon[35646]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:15.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:15 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:15.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:15 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:15.726 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:15 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:15.727 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:15 smithi145 ceph-mon[35646]: from='client.? 172.21.15.145:0/3519200387' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:30:15.727 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:15 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:15.752 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-02T13:30:15.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph config generate-minimal-conf 2024-04-02T13:30:15.997 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:30:16.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:15 smithi033 ceph-mon[32390]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:16.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:16.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:16.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:16.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:15 smithi033 ceph-mon[32390]: from='client.? 172.21.15.145:0/3519200387' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:30:16.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: mon.c calling monitor election 2024-04-02T13:30:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:30:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: mon.a calling monitor election 2024-04-02T13:30:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:30:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: monmap epoch 3 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: last_changed 2024-04-02T13:30:09.566182+0000 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: created 2024-04-02T13:28:18.700852+0000 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: min_mon_release 19 (squid) 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: election_strategy: 1 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-04-02T13:30:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: fsmap 2024-04-02T13:30:16.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:30:16.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: mgrmap e13: x(active, since 58s) 2024-04-02T13:30:16.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: overall HEALTH_OK 2024-04-02T13:30:16.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:16.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:16.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:16.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:16.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:16.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:16.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='client.? 172.21.15.145:0/3519200387' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:30:16.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:17.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.559+0000 7f62bf96c700 1 -- 172.21.15.33:0/4245239595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b8101120 msgr2=0x7f62b8101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.559+0000 7f62bf96c700 1 --2- 172.21.15.33:0/4245239595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b8101120 0x7f62b8101500 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009910 tx=0x7f62b4009c20 comp rx=0 tx=0).stop 2024-04-02T13:30:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.559+0000 7f62bf96c700 1 -- 172.21.15.33:0/4245239595 shutdown_connections 2024-04-02T13:30:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.559+0000 7f62bf96c700 1 --2- 172.21.15.33:0/4245239595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b8101120 0x7f62b8101500 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.559+0000 7f62bf96c700 1 -- 172.21.15.33:0/4245239595 >> 172.21.15.33:0/4245239595 conn(0x7f62b8077b30 msgr2=0x7f62b80fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.559+0000 7f62bf96c700 1 -- 172.21.15.33:0/4245239595 shutdown_connections 2024-04-02T13:30:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.559+0000 7f62bf96c700 1 -- 172.21.15.33:0/4245239595 wait complete. 2024-04-02T13:30:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.560+0000 7f62bf96c700 1 Processor -- start 2024-04-02T13:30:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.560+0000 7f62bf96c700 1 -- start start 2024-04-02T13:30:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.560+0000 7f62bf96c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b8101120 0x7f62b8197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.560+0000 7f62bf96c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62b81982d0 0x7f62b819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.560+0000 7f62bf96c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62b819cba0 0x7f62b819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.560+0000 7f62bf96c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f62b810b4a0 con 0x7f62b8101120 2024-04-02T13:30:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.560+0000 7f62bf96c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f62b810b320 con 0x7f62b81982d0 2024-04-02T13:30:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.560+0000 7f62bf96c700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f62b810b620 con 0x7f62b819cba0 2024-04-02T13:30:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.560+0000 7f62bd708700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b8101120 0x7f62b8197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.560+0000 7f62bd708700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b8101120 0x7f62b8197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52738/0 (socket says 172.21.15.33:52738) 2024-04-02T13:30:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.560+0000 7f62bd708700 1 -- 172.21.15.33:0/4193318481 learned_addr learned my addr 172.21.15.33:0/4193318481 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:30:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.560+0000 7f62bdf09700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62b819cba0 0x7f62b819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.560+0000 7f62bcf07700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62b81982d0 0x7f62b819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.561+0000 7f62bd708700 1 -- 172.21.15.33:0/4193318481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62b819cba0 msgr2=0x7f62b819d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.561+0000 7f62bd708700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62b819cba0 0x7f62b819d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.561+0000 7f62bd708700 1 -- 172.21.15.33:0/4193318481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62b81982d0 msgr2=0x7f62b819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.561+0000 7f62bd708700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62b81982d0 0x7f62b819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.561+0000 7f62bd708700 1 -- 172.21.15.33:0/4193318481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b8102af0 con 0x7f62b8101120 2024-04-02T13:30:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.561+0000 7f62bdf09700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62b819cba0 0x7f62b819d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:30:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.561+0000 7f62bd708700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b8101120 0x7f62b8197d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f62b4006010 tx=0x7f62b4004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.561+0000 7f62ae7fc700 1 -- 172.21.15.33:0/4193318481 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4027070 con 0x7f62b8101120 2024-04-02T13:30:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.561+0000 7f62ae7fc700 1 -- 172.21.15.33:0/4193318481 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f62b4005da0 con 0x7f62b8101120 2024-04-02T13:30:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.561+0000 7f62bf96c700 1 -- 172.21.15.33:0/4193318481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b8102de0 con 0x7f62b8101120 2024-04-02T13:30:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.561+0000 7f62ae7fc700 1 -- 172.21.15.33:0/4193318481 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4022e30 con 0x7f62b8101120 2024-04-02T13:30:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.561+0000 7f62bf96c700 1 -- 172.21.15.33:0/4193318481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b8103320 con 0x7f62b8101120 2024-04-02T13:30:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.562+0000 7f62ae7fc700 1 -- 172.21.15.33:0/4193318481 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f62b402a070 con 0x7f62b8101120 2024-04-02T13:30:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.562+0000 7f62bf96c700 1 -- 172.21.15.33:0/4193318481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b8102600 con 0x7f62b8101120 2024-04-02T13:30:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.562+0000 7f62ae7fc700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f62a4039ae0 0x7f62a403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.562+0000 7f62ae7fc700 1 -- 172.21.15.33:0/4193318481 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f62b404db50 con 0x7f62b8101120 2024-04-02T13:30:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.562+0000 7f62bcf07700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f62a4039ae0 0x7f62a403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.563+0000 7f62bcf07700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f62a4039ae0 0x7f62a403bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009850 tx=0x7f62a8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.567+0000 7f62ae7fc700 1 -- 172.21.15.33:0/4193318481 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f62b4013070 con 0x7f62b8101120 2024-04-02T13:30:17.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: mon.b calling monitor election 2024-04-02T13:30:17.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:17.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: Updating smithi050:/etc/ceph/ceph.conf 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: Updating smithi145:/etc/ceph/ceph.conf 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: mon.b calling monitor election 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: mon.c calling monitor election 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: mon.a calling monitor election 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: monmap epoch 3 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: last_changed 2024-04-02T13:30:09.566182+0000 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: created 2024-04-02T13:28:18.700852+0000 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: min_mon_release 19 (squid) 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: election_strategy: 1 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-04-02T13:30:17.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: fsmap 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: mgrmap e13: x(active, since 60s) 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: overall HEALTH_OK 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: Updating smithi033:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:17.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:30:17.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:17.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.798+0000 7f62bf96c700 1 -- 172.21.15.33:0/4193318481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f62b8063c20 con 0x7f62b8101120 2024-04-02T13:30:17.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.798+0000 7f62ae7fc700 1 -- 172.21.15.33:0/4193318481 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f62b400e3e0 con 0x7f62b8101120 2024-04-02T13:30:17.801 INFO:teuthology.orchestra.run.smithi033.stdout:# minimal ceph.conf for b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:17.802 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-04-02T13:30:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: mon_host = [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] 2024-04-02T13:30:17.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.801+0000 7f62bf96c700 1 -- 172.21.15.33:0/4193318481 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f62a4039ae0 msgr2=0x7f62a403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:17.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.801+0000 7f62bf96c700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f62a4039ae0 0x7f62a403bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009850 tx=0x7f62a8008040 comp rx=0 tx=0).stop 2024-04-02T13:30:17.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.801+0000 7f62bf96c700 1 -- 172.21.15.33:0/4193318481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b8101120 msgr2=0x7f62b8197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:17.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.801+0000 7f62bf96c700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b8101120 0x7f62b8197d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f62b4006010 tx=0x7f62b4004cc0 comp rx=0 tx=0).stop 2024-04-02T13:30:17.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.801+0000 7f62bf96c700 1 -- 172.21.15.33:0/4193318481 shutdown_connections 2024-04-02T13:30:17.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.801+0000 7f62bf96c700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f62a4039ae0 0x7f62a403bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:17.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.801+0000 7f62bf96c700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62b819cba0 0x7f62b819d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:17.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.801+0000 7f62bf96c700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b8101120 0x7f62b8197d90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:17.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.801+0000 7f62bf96c700 1 --2- 172.21.15.33:0/4193318481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62b81982d0 0x7f62b819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:17.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.801+0000 7f62bf96c700 1 -- 172.21.15.33:0/4193318481 >> 172.21.15.33:0/4193318481 conn(0x7f62b8077b30 msgr2=0x7f62b80699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:17.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.801+0000 7f62bf96c700 1 -- 172.21.15.33:0/4193318481 shutdown_connections 2024-04-02T13:30:17.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:17.801+0000 7f62bf96c700 1 -- 172.21.15.33:0/4193318481 wait complete. 2024-04-02T13:30:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: mon.b calling monitor election 2024-04-02T13:30:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-02T13:30:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: Updating smithi050:/etc/ceph/ceph.conf 2024-04-02T13:30:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: Updating smithi145:/etc/ceph/ceph.conf 2024-04-02T13:30:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: mon.b calling monitor election 2024-04-02T13:30:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: mon.c calling monitor election 2024-04-02T13:30:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: mon.a calling monitor election 2024-04-02T13:30:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T13:30:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: monmap epoch 3 2024-04-02T13:30:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: last_changed 2024-04-02T13:30:09.566182+0000 2024-04-02T13:30:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: created 2024-04-02T13:28:18.700852+0000 2024-04-02T13:30:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: min_mon_release 19 (squid) 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: election_strategy: 1 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: fsmap 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: mgrmap e13: x(active, since 60s) 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: overall HEALTH_OK 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: Updating smithi033:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:17.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:30:17.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: mon.b calling monitor election 2024-04-02T13:30:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-02T13:30:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: Updating smithi050:/etc/ceph/ceph.conf 2024-04-02T13:30:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: Updating smithi145:/etc/ceph/ceph.conf 2024-04-02T13:30:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: mon.b calling monitor election 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: mon.c calling monitor election 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: mon.a calling monitor election 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: monmap epoch 3 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: last_changed 2024-04-02T13:30:09.566182+0000 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: created 2024-04-02T13:28:18.700852+0000 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: min_mon_release 19 (squid) 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: election_strategy: 1 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: 1: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.c 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: fsmap 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:30:17.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: mgrmap e13: x(active, since 60s) 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: overall HEALTH_OK 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: Updating smithi033:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:30:17.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:18.703 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-02T13:30:18.704 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:30:18.704 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T13:30:18.735 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:30:18.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:30:18.799 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:18 smithi033 ceph-mon[32390]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:30:18.799 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:18 smithi033 ceph-mon[32390]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:30:18.799 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:18 smithi033 ceph-mon[32390]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T13:30:18.799 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:18 smithi033 ceph-mon[32390]: Reconfiguring daemon mon.a on smithi033 2024-04-02T13:30:18.799 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:18.799 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:18 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4193318481' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:18.803 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T13:30:18.803 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T13:30:18.833 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:18 smithi050 ceph-mon[35524]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:30:18.833 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:18 smithi050 ceph-mon[35524]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:30:18.833 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:18 smithi050 ceph-mon[35524]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T13:30:18.833 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:18 smithi050 ceph-mon[35524]: Reconfiguring daemon mon.a on smithi033 2024-04-02T13:30:18.833 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:18.833 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:18 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4193318481' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:18.838 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T13:30:18.838 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:30:18.909 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-02T13:30:18.909 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T13:30:18.939 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:18 smithi145 ceph-mon[35646]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:30:18.939 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:18 smithi145 ceph-mon[35646]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:30:18.939 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:18 smithi145 ceph-mon[35646]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T13:30:18.939 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:18 smithi145 ceph-mon[35646]: Reconfiguring daemon mon.a on smithi033 2024-04-02T13:30:18.939 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:30:18.939 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:18 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4193318481' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:18.944 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-02T13:30:18.944 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:30:19.014 INFO:tasks.cephadm:Adding mgr.x on smithi033 2024-04-02T13:30:19.014 INFO:tasks.cephadm:Adding mgr.y on smithi050 2024-04-02T13:30:19.014 INFO:tasks.cephadm:Adding mgr.z on smithi145 2024-04-02T13:30:19.015 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch apply mgr '3;smithi033=x;smithi050=y;smithi145=z' 2024-04-02T13:30:19.288 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:30:19.707 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:19 smithi145 ceph-mon[35646]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:19.708 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:19 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:19.708 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:19 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:19.708 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:19 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:19.708 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:19 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:30:19.708 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:19 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:19.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:19 smithi033 ceph-mon[32390]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:19.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:19 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:19.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:19 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:19.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:19 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:19.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:19 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:30:19.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:19 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:19.817 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:19 smithi050 ceph-mon[35524]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:19.817 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:19 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:19.817 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:19 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:19.817 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:19 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:19.817 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:19 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:30:19.817 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:19 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:20.708 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.703+0000 7f60e84b1700 1 -- 172.21.15.145:0/1001315178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60e0101050 msgr2=0x7f60e0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.703+0000 7f60e84b1700 1 --2- 172.21.15.145:0/1001315178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60e0101050 0x7f60e0101470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009910 tx=0x7f60dc009c20 comp rx=0 tx=0).stop 2024-04-02T13:30:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.703+0000 7f60e84b1700 1 -- 172.21.15.145:0/1001315178 shutdown_connections 2024-04-02T13:30:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.703+0000 7f60e84b1700 1 --2- 172.21.15.145:0/1001315178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60e0101050 0x7f60e0101470 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.703+0000 7f60e84b1700 1 -- 172.21.15.145:0/1001315178 >> 172.21.15.145:0/1001315178 conn(0x7f60e00fc5f0 msgr2=0x7f60e00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.703+0000 7f60e84b1700 1 -- 172.21.15.145:0/1001315178 shutdown_connections 2024-04-02T13:30:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.703+0000 7f60e84b1700 1 -- 172.21.15.145:0/1001315178 wait complete. 2024-04-02T13:30:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.704+0000 7f60e84b1700 1 Processor -- start 2024-04-02T13:30:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.704+0000 7f60e84b1700 1 -- start start 2024-04-02T13:30:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.704+0000 7f60e84b1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e0101050 0x7f60e006b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.704+0000 7f60e84b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60e019a6a0 0x7f60e019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.704+0000 7f60e84b1700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60e019d020 0x7f60e019f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.704+0000 7f60e84b1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01070d0 con 0x7f60e019a6a0 2024-04-02T13:30:20.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.704+0000 7f60e84b1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f60e0106f50 con 0x7f60e0101050 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.704+0000 7f60e84b1700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f60e0107250 con 0x7f60e019d020 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.704+0000 7f60e6a4e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60e019d020 0x7f60e019f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.704+0000 7f60e6a4e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60e019d020 0x7f60e019f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52808/0 (socket says 172.21.15.145:52808) 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.704+0000 7f60e6a4e700 1 -- 172.21.15.145:0/290798983 learned_addr learned my addr 172.21.15.145:0/290798983 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.704+0000 7f60e5a4c700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60e019a6a0 0x7f60e019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.704+0000 7f60e624d700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e0101050 0x7f60e006b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.705+0000 7f60e5a4c700 1 -- 172.21.15.145:0/290798983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60e019d020 msgr2=0x7f60e019f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.705+0000 7f60e5a4c700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60e019d020 0x7f60e019f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.705+0000 7f60e5a4c700 1 -- 172.21.15.145:0/290798983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e0101050 msgr2=0x7f60e006b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.705+0000 7f60e5a4c700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e0101050 0x7f60e006b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.705+0000 7f60e5a4c700 1 -- 172.21.15.145:0/290798983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e019fd90 con 0x7f60e019a6a0 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.705+0000 7f60e6a4e700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60e019d020 0x7f60e019f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.705+0000 7f60e624d700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e0101050 0x7f60e006b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:30:20.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.705+0000 7f60e5a4c700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60e019a6a0 0x7f60e019ca90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f60d000b550 tx=0x7f60d000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:20.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.705+0000 7f60d77fe700 1 -- 172.21.15.145:0/290798983 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0004230 con 0x7f60e019a6a0 2024-04-02T13:30:20.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.705+0000 7f60e84b1700 1 -- 172.21.15.145:0/290798983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e01a0080 con 0x7f60e019a6a0 2024-04-02T13:30:20.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.705+0000 7f60d77fe700 1 -- 172.21.15.145:0/290798983 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f60d00043d0 con 0x7f60e019a6a0 2024-04-02T13:30:20.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.705+0000 7f60d77fe700 1 -- 172.21.15.145:0/290798983 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0011bd0 con 0x7f60e019a6a0 2024-04-02T13:30:20.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.705+0000 7f60e84b1700 1 -- 172.21.15.145:0/290798983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e01a0640 con 0x7f60e019a6a0 2024-04-02T13:30:20.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.706+0000 7f60e84b1700 1 -- 172.21.15.145:0/290798983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e0063c20 con 0x7f60e019a6a0 2024-04-02T13:30:20.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.706+0000 7f60d77fe700 1 -- 172.21.15.145:0/290798983 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f60d00036a0 con 0x7f60e019a6a0 2024-04-02T13:30:20.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.706+0000 7f60d77fe700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f60cc039ae0 0x7f60cc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:20.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.706+0000 7f60d77fe700 1 -- 172.21.15.145:0/290798983 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f60d004c2e0 con 0x7f60e019a6a0 2024-04-02T13:30:20.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.706+0000 7f60e624d700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f60cc039ae0 0x7f60cc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:20.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.707+0000 7f60e624d700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f60cc039ae0 0x7f60cc03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f60dc005950 tx=0x7f60dc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:20.715 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.712+0000 7f60d77fe700 1 -- 172.21.15.145:0/290798983 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f60d00206f0 con 0x7f60e019a6a0 2024-04-02T13:30:20.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:20 smithi050 ceph-mon[35524]: Reconfiguring mon.b (monmap changed)... 2024-04-02T13:30:20.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:20 smithi050 ceph-mon[35524]: Reconfiguring daemon mon.b on smithi050 2024-04-02T13:30:20.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.919+0000 7f60e84b1700 1 -- 172.21.15.145:0/290798983 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi050=y;smithi145=z", "target": ["mon-mgr", ""]}) -- 0x7f60e0062b80 con 0x7f60cc039ae0 2024-04-02T13:30:20.923 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:20 smithi145 ceph-mon[35646]: Reconfiguring mon.b (monmap changed)... 2024-04-02T13:30:20.923 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:20 smithi145 ceph-mon[35646]: Reconfiguring daemon mon.b on smithi050 2024-04-02T13:30:20.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.925+0000 7f60d77fe700 1 -- 172.21.15.145:0/290798983 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f60e0062b80 con 0x7f60cc039ae0 2024-04-02T13:30:20.929 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mgr update... 2024-04-02T13:30:20.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.927+0000 7f60e84b1700 1 -- 172.21.15.145:0/290798983 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f60cc039ae0 msgr2=0x7f60cc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:20.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.927+0000 7f60e84b1700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f60cc039ae0 0x7f60cc03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f60dc005950 tx=0x7f60dc0058e0 comp rx=0 tx=0).stop 2024-04-02T13:30:20.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.927+0000 7f60e84b1700 1 -- 172.21.15.145:0/290798983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60e019a6a0 msgr2=0x7f60e019ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:20.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.927+0000 7f60e84b1700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60e019a6a0 0x7f60e019ca90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f60d000b550 tx=0x7f60d000b860 comp rx=0 tx=0).stop 2024-04-02T13:30:20.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.928+0000 7f60e84b1700 1 -- 172.21.15.145:0/290798983 shutdown_connections 2024-04-02T13:30:20.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.928+0000 7f60e84b1700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f60cc039ae0 0x7f60cc03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:20.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.928+0000 7f60e84b1700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60e019d020 0x7f60e019f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:20.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.928+0000 7f60e84b1700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e0101050 0x7f60e006b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:20.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.928+0000 7f60e84b1700 1 --2- 172.21.15.145:0/290798983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60e019a6a0 0x7f60e019ca90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:20.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.928+0000 7f60e84b1700 1 -- 172.21.15.145:0/290798983 >> 172.21.15.145:0/290798983 conn(0x7f60e00fc5f0 msgr2=0x7f60e00fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:20.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.928+0000 7f60e84b1700 1 -- 172.21.15.145:0/290798983 shutdown_connections 2024-04-02T13:30:20.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:30:20.928+0000 7f60e84b1700 1 -- 172.21.15.145:0/290798983 wait complete. 2024-04-02T13:30:21.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:20 smithi033 ceph-mon[32390]: Reconfiguring mon.b (monmap changed)... 2024-04-02T13:30:21.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:20 smithi033 ceph-mon[32390]: Reconfiguring daemon mon.b on smithi050 2024-04-02T13:30:21.534 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:21 smithi145 ceph-mon[35646]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:21.534 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:21 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:21.534 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:21 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:21.535 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:21 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:21.535 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:21 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:21.535 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:21 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:30:21.535 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:21 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:21.564 DEBUG:teuthology.orchestra.run.smithi050:mgr.y> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.y.service 2024-04-02T13:30:21.567 DEBUG:teuthology.orchestra.run.smithi145:mgr.z> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.z.service 2024-04-02T13:30:21.569 INFO:tasks.cephadm:Deploying OSDs... 2024-04-02T13:30:21.569 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:30:21.570 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T13:30:21.588 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T13:30:21.588 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-04-02T13:30:21.646 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T13:30:21.646 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:30:21.646 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-02T13:30:21.646 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:30:21.646 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:30:21.646 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-02 13:29:26.350982193 +0000 2024-04-02T13:30:21.646 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-02 13:24:48.641890713 +0000 2024-04-02T13:30:21.646 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-02 13:24:48.641890713 +0000 2024-04-02T13:30:21.646 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-02 13:24:48.641890713 +0000 2024-04-02T13:30:21.646 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T13:30:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-02T13:30:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-02T13:30:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000218721 s, 2.3 MB/s 2024-04-02T13:30:21.716 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T13:30:21.773 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-04-02T13:30:21.829 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T13:30:21.829 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:30:21.829 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-02T13:30:21.829 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:30:21.829 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:30:21.829 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-02 13:29:26.377981752 +0000 2024-04-02T13:30:21.829 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-02 13:24:49.001884801 +0000 2024-04-02T13:30:21.829 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-02 13:24:49.001884801 +0000 2024-04-02T13:30:21.829 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-02 13:24:49.001884801 +0000 2024-04-02T13:30:21.830 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T13:30:21.894 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:21 smithi033 ceph-mon[32390]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:21.894 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:21 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:21.894 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:21 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:21.894 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:21 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:21.894 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:21 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:21.894 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:21 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:30:21.894 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:21 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:21.896 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-02T13:30:21.896 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-02T13:30:21.896 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.00018855 s, 2.7 MB/s 2024-04-02T13:30:21.897 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T13:30:21.954 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-04-02T13:30:21.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:21 smithi050 ceph-mon[35524]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:21.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:21 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:21.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:21 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:21.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:21 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:21.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:21 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:30:21.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:21 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:30:21.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:21 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T13:30:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:30:22.011 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-04-02T13:30:22.011 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:30:22.011 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:30:22.011 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-02 13:29:26.404981311 +0000 2024-04-02T13:30:22.011 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-02 13:24:49.352879037 +0000 2024-04-02T13:30:22.011 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-02 13:24:49.352879037 +0000 2024-04-02T13:30:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-02 13:24:49.352879037 +0000 2024-04-02T13:30:22.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T13:30:22.077 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-02T13:30:22.077 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-02T13:30:22.077 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000188559 s, 2.7 MB/s 2024-04-02T13:30:22.079 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T13:30:22.135 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-04-02T13:30:22.190 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T13:30:22.190 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:30:22.190 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-02T13:30:22.190 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:30:22.190 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:30:22.190 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-02 13:29:26.429980903 +0000 2024-04-02T13:30:22.190 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-02 13:24:49.675873731 +0000 2024-04-02T13:30:22.190 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-02 13:24:49.675873731 +0000 2024-04-02T13:30:22.190 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-02 13:24:49.675873731 +0000 2024-04-02T13:30:22.191 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T13:30:22.256 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-02T13:30:22.256 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-02T13:30:22.256 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000179416 s, 2.9 MB/s 2024-04-02T13:30:22.257 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T13:30:22.314 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-04-02T13:30:22.314 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T13:30:22.332 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T13:30:22.332 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2024-04-02T13:30:22.390 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T13:30:22.390 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:30:22.391 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-02T13:30:22.391 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:30:22.391 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:30:22.391 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-04-02 13:30:08.998419401 +0000 2024-04-02T13:30:22.391 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-04-02 13:24:50.591319953 +0000 2024-04-02T13:30:22.391 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-04-02 13:24:50.591319953 +0000 2024-04-02T13:30:22.391 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-04-02 13:24:50.591319953 +0000 2024-04-02T13:30:22.391 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T13:30:22.461 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-04-02T13:30:22.461 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-04-02T13:30:22.461 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000128335 s, 4.0 MB/s 2024-04-02T13:30:22.462 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T13:30:22.522 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2024-04-02T13:30:22.580 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T13:30:22.580 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:30:22.580 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-04-02T13:30:22.580 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:30:22.580 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:30:22.580 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-04-02 13:30:08.998419401 +0000 2024-04-02T13:30:22.580 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-04-02 13:24:50.933311348 +0000 2024-04-02T13:30:22.580 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-04-02 13:24:50.933311348 +0000 2024-04-02T13:30:22.580 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-04-02 13:24:50.933311348 +0000 2024-04-02T13:30:22.581 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T13:30:22.650 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-04-02T13:30:22.650 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-04-02T13:30:22.650 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000115559 s, 4.4 MB/s 2024-04-02T13:30:22.651 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T13:30:22.711 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2024-04-02T13:30:22.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:22 smithi050 ceph-mon[35524]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi050=y;smithi145=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:22.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:22 smithi050 ceph-mon[35524]: Saving service mgr spec with placement smithi033=x;smithi050=y;smithi145=z;count:3 2024-04-02T13:30:22.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:22 smithi050 ceph-mon[35524]: Reconfiguring mon.c (monmap changed)... 2024-04-02T13:30:22.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:22 smithi050 ceph-mon[35524]: Reconfiguring daemon mon.c on smithi145 2024-04-02T13:30:22.763 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T13:30:22.763 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:30:22.763 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-02T13:30:22.763 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:30:22.763 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:30:22.764 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-04-02 13:30:08.999419375 +0000 2024-04-02T13:30:22.764 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-04-02 13:24:51.258303170 +0000 2024-04-02T13:30:22.764 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-04-02 13:24:51.258303170 +0000 2024-04-02T13:30:22.764 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-04-02 13:24:51.258303170 +0000 2024-04-02T13:30:22.764 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T13:30:22.834 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-04-02T13:30:22.834 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-04-02T13:30:22.834 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000109378 s, 4.7 MB/s 2024-04-02T13:30:22.835 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T13:30:22.895 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2024-04-02T13:30:22.953 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T13:30:22.953 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:30:22.953 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-04-02T13:30:22.953 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:30:22.954 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:30:22.954 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-04-02 13:30:08.999419375 +0000 2024-04-02T13:30:22.954 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-04-02 13:24:51.585294942 +0000 2024-04-02T13:30:22.954 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-04-02 13:24:51.585294942 +0000 2024-04-02T13:30:22.954 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-04-02 13:24:51.585294942 +0000 2024-04-02T13:30:22.954 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T13:30:22.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:22 smithi145 ceph-mon[35646]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi050=y;smithi145=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:22.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:22 smithi145 ceph-mon[35646]: Saving service mgr spec with placement smithi033=x;smithi050=y;smithi145=z;count:3 2024-04-02T13:30:22.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:22 smithi145 ceph-mon[35646]: Reconfiguring mon.c (monmap changed)... 2024-04-02T13:30:22.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:22 smithi145 ceph-mon[35646]: Reconfiguring daemon mon.c on smithi145 2024-04-02T13:30:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:22 smithi033 ceph-mon[32390]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi050=y;smithi145=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:22 smithi033 ceph-mon[32390]: Saving service mgr spec with placement smithi033=x;smithi050=y;smithi145=z;count:3 2024-04-02T13:30:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:22 smithi033 ceph-mon[32390]: Reconfiguring mon.c (monmap changed)... 2024-04-02T13:30:23.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:22 smithi033 ceph-mon[32390]: Reconfiguring daemon mon.c on smithi145 2024-04-02T13:30:23.023 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-04-02T13:30:23.023 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-04-02T13:30:23.023 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000114737 s, 4.5 MB/s 2024-04-02T13:30:23.025 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T13:30:23.086 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-04-02T13:30:23.086 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T13:30:23.103 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T13:30:23.104 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_1 2024-04-02T13:30:23.168 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T13:30:23.168 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:30:23.168 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-04-02T13:30:23.168 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:30:23.168 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:30:23.168 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-04-02 13:29:59.230247939 +0000 2024-04-02T13:30:23.168 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-04-02 13:24:47.939495434 +0000 2024-04-02T13:30:23.168 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-04-02 13:24:47.939495434 +0000 2024-04-02T13:30:23.168 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-04-02 13:24:47.939495434 +0000 2024-04-02T13:30:23.169 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T13:30:23.247 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-04-02T13:30:23.247 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-04-02T13:30:23.248 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000215814 s, 2.4 MB/s 2024-04-02T13:30:23.249 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T13:30:23.313 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_2 2024-04-02T13:30:23.378 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T13:30:23.378 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:30:23.378 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-02T13:30:23.378 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:30:23.378 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:30:23.378 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-04-02 13:29:59.230247939 +0000 2024-04-02T13:30:23.378 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-04-02 13:24:48.288487962 +0000 2024-04-02T13:30:23.378 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-04-02 13:24:48.288487962 +0000 2024-04-02T13:30:23.378 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-04-02 13:24:48.288487962 +0000 2024-04-02T13:30:23.379 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T13:30:23.451 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-04-02T13:30:23.451 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-04-02T13:30:23.451 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000201647 s, 2.5 MB/s 2024-04-02T13:30:23.452 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T13:30:23.513 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_3 2024-04-02T13:30:23.575 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T13:30:23.575 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:30:23.575 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-02T13:30:23.575 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:30:23.575 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:30:23.575 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-04-02 13:29:59.230247939 +0000 2024-04-02T13:30:23.575 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-04-02 13:24:48.639480449 +0000 2024-04-02T13:30:23.576 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-04-02 13:24:48.639480449 +0000 2024-04-02T13:30:23.576 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-04-02 13:24:48.639480449 +0000 2024-04-02T13:30:23.576 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T13:30:23.648 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-04-02T13:30:23.648 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-04-02T13:30:23.648 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000234776 s, 2.2 MB/s 2024-04-02T13:30:23.649 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T13:30:23.715 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_4 2024-04-02T13:30:23.776 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T13:30:23.776 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:30:23.777 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-02T13:30:23.777 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:30:23.777 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:30:23.777 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-04-02 13:29:59.231247918 +0000 2024-04-02T13:30:23.777 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-04-02 13:24:48.993472870 +0000 2024-04-02T13:30:23.777 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-04-02 13:24:48.993472870 +0000 2024-04-02T13:30:23.777 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-04-02 13:24:48.993472870 +0000 2024-04-02T13:30:23.777 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T13:30:23.844 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:23 smithi145 ceph-mon[35646]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:23.844 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:23.844 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:23.844 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:23.845 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:23.845 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:23.845 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:23.845 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:30:23.845 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:30:23.845 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:30:23.845 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:23.847 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-04-02T13:30:23.847 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-04-02T13:30:23.847 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000165828 s, 3.1 MB/s 2024-04-02T13:30:23.848 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T13:30:23.910 INFO:tasks.cephadm:Deploying osd.0 on smithi033 with /dev/vg_nvme/lv_4... 2024-04-02T13:30:23.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T13:30:23.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:23 smithi033 ceph-mon[32390]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:30:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:30:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:30:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:23 smithi050 ceph-mon[35524]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:30:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:30:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:30:23.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:24.158 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:30:24.881 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:24 smithi145 ceph-mon[35646]: Deploying daemon mgr.z on smithi145 2024-04-02T13:30:24.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:24 smithi050 ceph-mon[35524]: Deploying daemon mgr.z on smithi145 2024-04-02T13:30:25.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:24 smithi033 ceph-mon[32390]: Deploying daemon mgr.z on smithi145 2024-04-02T13:30:25.229 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:25 smithi145 systemd[1]: Starting Ceph mgr.z for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:30:25.536 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:25 smithi145 podman[37493]: 2024-04-02 13:30:25.199987026 +0000 UTC m=+0.017619982 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:30:25.536 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:25 smithi145 podman[37493]: 2024-04-02 13:30:25.31368023 +0000 UTC m=+0.131313244 container create fb0412f4a58914714965d03d20b25ab96ba4cf3c242c607654c880a19afff1c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-02T13:30:25.536 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:25 smithi145 podman[37493]: 2024-04-02 13:30:25.532832321 +0000 UTC m=+0.350465275 container init fb0412f4a58914714965d03d20b25ab96ba4cf3c242c607654c880a19afff1c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:30:25.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:25 smithi033 ceph-mon[32390]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:25.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:25 smithi145 ceph-mon[35646]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:25.979 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:25 smithi145 podman[37493]: 2024-04-02 13:30:25.535780319 +0000 UTC m=+0.353413269 container start fb0412f4a58914714965d03d20b25ab96ba4cf3c242c607654c880a19afff1c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:30:25.979 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:25 smithi145 ceph-mgr[37512]: -- 172.21.15.145:0/3078225674 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55757e1ec5a0 con 0x55757e1c0c00 2024-04-02T13:30:25.980 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:25 smithi145 bash[37493]: fb0412f4a58914714965d03d20b25ab96ba4cf3c242c607654c880a19afff1c9 2024-04-02T13:30:25.980 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:25 smithi145 systemd[1]: Started Ceph mgr.z for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:30:25.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:25 smithi050 ceph-mon[35524]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T13:30:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T13:30:26.031 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-02T13:30:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-02T13:30:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0119864 s, 875 MB/s 2024-04-02T13:30:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-02T13:30:26.479 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:26 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:26.083+0000 7f89c0b5e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:30:26.479 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:26 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:26.176+0000 7f89c0b5e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:30:26.628 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi033:vg_nvme/lv_4 2024-04-02T13:30:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:30:26.979 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:26 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:26.814+0000 7f89c0b5e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:30:26.979 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:26 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:26.906+0000 7f89c0b5e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:30:26.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:26 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:26.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:26 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:26.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:26 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:26.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:26 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:30:26.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:26 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:30:26.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:26 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:30:26.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:26 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:26.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:26 smithi145 ceph-mon[35646]: Deploying daemon mgr.y on smithi050 2024-04-02T13:30:26.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:26 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:26.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:26 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:26.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:26 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:26.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:26 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:30:26.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:26 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:30:26.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:26 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:30:26.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:26 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:26.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:26 smithi050 ceph-mon[35524]: Deploying daemon mgr.y on smithi050 2024-04-02T13:30:27.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:26 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:27.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:26 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:27.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:26 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:27.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:26 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:30:27.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:26 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:30:27.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:26 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:30:27.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:26 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:27.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:26 smithi033 ceph-mon[32390]: Deploying daemon mgr.y on smithi050 2024-04-02T13:30:27.479 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:27.045+0000 7f89c0b5e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:30:27.479 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:27.138+0000 7f89c0b5e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:30:27.479 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:27.219+0000 7f89c0b5e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:30:27.781 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:27 smithi145 ceph-mon[35646]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:27.781 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:27.487+0000 7f89c0b5e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:30:27.781 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:27.596+0000 7f89c0b5e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:30:27.781 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:27.679+0000 7f89c0b5e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:30:27.781 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:27.777+0000 7f89c0b5e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:30:27.863 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:27 smithi050 ceph-mon[35524]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:28.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:27 smithi033 ceph-mon[32390]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:28.124 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:28 smithi050 ceph-mgr[36907]: -- 172.21.15.50:0/3345078839 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d25dcc45a0 con 0x55d25dc99000 2024-04-02T13:30:28.150 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:27.960+0000 7f89c0b5e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:30:28.406 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:28 smithi050 bash[36888]: aea4ca15923545d6298de6eb1a8ea92347b151d3de47bd0418bde4133818ad7a 2024-04-02T13:30:28.406 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:28 smithi050 systemd[1]: Started Ceph mgr.y for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:30:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.550+0000 7f1b343b2700 1 -- 172.21.15.33:0/3866201638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2c103100 msgr2=0x7f1b2c1054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.550+0000 7f1b343b2700 1 --2- 172.21.15.33:0/3866201638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2c103100 0x7f1b2c1054f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00b0a0 tx=0x7f1b1c00b3b0 comp rx=0 tx=0).stop 2024-04-02T13:30:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.550+0000 7f1b343b2700 1 -- 172.21.15.33:0/3866201638 shutdown_connections 2024-04-02T13:30:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.550+0000 7f1b343b2700 1 --2- 172.21.15.33:0/3866201638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b2c108360 0x7f1b2c10a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.550+0000 7f1b343b2700 1 --2- 172.21.15.33:0/3866201638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2c103100 0x7f1b2c1054f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.550+0000 7f1b343b2700 1 --2- 172.21.15.33:0/3866201638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c105a30 0x7f1b2c107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.550+0000 7f1b343b2700 1 -- 172.21.15.33:0/3866201638 >> 172.21.15.33:0/3866201638 conn(0x7f1b2c0fcab0 msgr2=0x7f1b2c0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.550+0000 7f1b343b2700 1 -- 172.21.15.33:0/3866201638 shutdown_connections 2024-04-02T13:30:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.550+0000 7f1b343b2700 1 -- 172.21.15.33:0/3866201638 wait complete. 2024-04-02T13:30:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.550+0000 7f1b343b2700 1 Processor -- start 2024-04-02T13:30:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.550+0000 7f1b343b2700 1 -- start start 2024-04-02T13:30:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b343b2700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b2c103100 0x7f1b2c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b343b2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c105a30 0x7f1b2c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b343b2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2c108360 0x7f1b2c19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b343b2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c10e1f0 con 0x7f1b2c108360 2024-04-02T13:30:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b343b2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c10e070 con 0x7f1b2c105a30 2024-04-02T13:30:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b343b2700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c10e370 con 0x7f1b2c103100 2024-04-02T13:30:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b3294f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2c108360 0x7f1b2c19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b3294f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2c108360 0x7f1b2c19fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40234/0 (socket says 172.21.15.33:40234) 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b3194d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c105a30 0x7f1b2c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b3294f700 1 -- 172.21.15.33:0/182835160 learned_addr learned my addr 172.21.15.33:0/182835160 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b3214e700 1 --2- 172.21.15.33:0/182835160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b2c103100 0x7f1b2c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b3294f700 1 -- 172.21.15.33:0/182835160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b2c103100 msgr2=0x7f1b2c198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b3294f700 1 --2- 172.21.15.33:0/182835160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b2c103100 0x7f1b2c198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b3294f700 1 -- 172.21.15.33:0/182835160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c105a30 msgr2=0x7f1b2c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b3294f700 1 --2- 172.21.15.33:0/182835160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c105a30 0x7f1b2c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b3294f700 1 -- 172.21.15.33:0/182835160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c1a0490 con 0x7f1b2c108360 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b3294f700 1 --2- 172.21.15.33:0/182835160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2c108360 0x7f1b2c19fdf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400d980 tx=0x7f1b2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.551+0000 7f1b2b7fe700 1 -- 172.21.15.33:0/182835160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24017070 con 0x7f1b2c108360 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.552+0000 7f1b2b7fe700 1 -- 172.21.15.33:0/182835160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1b24005020 con 0x7f1b2c108360 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.552+0000 7f1b2b7fe700 1 -- 172.21.15.33:0/182835160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2400fb50 con 0x7f1b2c108360 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.552+0000 7f1b343b2700 1 -- 172.21.15.33:0/182835160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2c1a0780 con 0x7f1b2c108360 2024-04-02T13:30:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.552+0000 7f1b343b2700 1 -- 172.21.15.33:0/182835160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2c1a0c70 con 0x7f1b2c108360 2024-04-02T13:30:28.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.552+0000 7f1b2b7fe700 1 -- 172.21.15.33:0/182835160 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1b2400fcf0 con 0x7f1b2c108360 2024-04-02T13:30:28.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.553+0000 7f1b2b7fe700 1 --2- 172.21.15.33:0/182835160 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b14039890 0x7f1b1403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:28.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.553+0000 7f1b343b2700 1 -- 172.21.15.33:0/182835160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2c0692b0 con 0x7f1b2c108360 2024-04-02T13:30:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.553+0000 7f1b2b7fe700 1 -- 172.21.15.33:0/182835160 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1b2404c3a0 con 0x7f1b2c108360 2024-04-02T13:30:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.553+0000 7f1b3214e700 1 --2- 172.21.15.33:0/182835160 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b14039890 0x7f1b1403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.553+0000 7f1b3214e700 1 --2- 172.21.15.33:0/182835160 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b14039890 0x7f1b1403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009fd0 tx=0x7f1b1c0060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:28.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.558+0000 7f1b2b7fe700 1 -- 172.21.15.33:0/182835160 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1b240243a0 con 0x7f1b2c108360 2024-04-02T13:30:28.745 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:28 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:28.643+0000 7fbeff420200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:30:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:28.777+0000 7f1b343b2700 1 -- 172.21.15.33:0/182835160 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1b2c062a30 con 0x7f1b14039890 2024-04-02T13:30:29.208 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:28 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:28.743+0000 7fbeff420200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:30:29.392 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:29.009+0000 7f89c0b5e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:30:29.392 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:29.164+0000 7f89c0b5e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:30:29.392 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-mon[35646]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:29.392 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:29.393 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:29.393 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:29.393 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:29.393 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:29.393 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:30:29.393 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:30:29.393 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:29.469 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-mon[35524]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:29.470 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:29.470 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:29.470 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:29.470 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:29.470 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:29.470 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:30:29.470 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:30:29.470 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:29.497 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:29 smithi033 ceph-mon[32390]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:29.497 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:29.497 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:29.497 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:29.498 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:29.498 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:29.498 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:30:29.498 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:30:29.498 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:29.729 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:29.510+0000 7f89c0b5e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:30:29.729 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:29.601+0000 7f89c0b5e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:30:29.745 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:29.465+0000 7fbeff420200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:30:29.745 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:29.560+0000 7fbeff420200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:30:29.745 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:29.713+0000 7fbeff420200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:30:30.172 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:29.810+0000 7fbeff420200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:30:30.172 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:29 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:29.895+0000 7fbeff420200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:30:30.172 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:30.167+0000 7fbeff420200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:30:30.229 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:29.797+0000 7f89c0b5e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:30:30.229 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:29.992+0000 7f89c0b5e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:30:30.468 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:30.279+0000 7fbeff420200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:30:30.468 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:30.367+0000 7fbeff420200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:30:30.468 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:30.463+0000 7fbeff420200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:30:30.558 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:30.279+0000 7f89c0b5e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:30:30.558 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:30.554+0000 7f89c0b5e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:30:30.745 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:30.641+0000 7fbeff420200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:30:30.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-mon[35524]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:30.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:30.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:30.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:30:30.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:30:30.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:30.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:30 smithi033 ceph-mon[32390]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:30.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:30.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:30.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:30:30.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:30:30.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:30.979 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:30.793+0000 7f89c0b5e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:30:30.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-mon[35646]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:30.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:30.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:30:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:30:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:31.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:31 smithi033 ceph-mon[32390]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:31.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:31 smithi033 ceph-mon[32390]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T13:30:31.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:31 smithi033 ceph-mon[32390]: Reconfiguring daemon mgr.x on smithi033 2024-04-02T13:30:31.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:31 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:31.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:31 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2567137097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78695977-281e-41e3-9c9e-9a3c1fca4744"}]: dispatch 2024-04-02T13:30:31.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:31 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2567137097' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78695977-281e-41e3-9c9e-9a3c1fca4744"}]': finished 2024-04-02T13:30:31.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:31 smithi033 ceph-mon[32390]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T13:30:31.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:31 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:31.979 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:31 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:31.594+0000 7f89c0b5e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:30:31.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:31 smithi145 ceph-mon[35646]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:31.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:31 smithi145 ceph-mon[35646]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T13:30:31.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:31 smithi145 ceph-mon[35646]: Reconfiguring daemon mgr.x on smithi033 2024-04-02T13:30:31.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:31 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:31.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:31 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2567137097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78695977-281e-41e3-9c9e-9a3c1fca4744"}]: dispatch 2024-04-02T13:30:31.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:31 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2567137097' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78695977-281e-41e3-9c9e-9a3c1fca4744"}]': finished 2024-04-02T13:30:31.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:31 smithi145 ceph-mon[35646]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T13:30:31.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:31 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:31.995 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:31 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:31.639+0000 7fbeff420200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:30:31.995 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:31 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:31.773+0000 7fbeff420200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:30:31.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:31 smithi050 ceph-mon[35524]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:31.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:31 smithi050 ceph-mon[35524]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T13:30:31.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:31 smithi050 ceph-mon[35524]: Reconfiguring daemon mgr.x on smithi033 2024-04-02T13:30:31.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:31 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:31.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:31 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2567137097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78695977-281e-41e3-9c9e-9a3c1fca4744"}]: dispatch 2024-04-02T13:30:31.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:31 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2567137097' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78695977-281e-41e3-9c9e-9a3c1fca4744"}]': finished 2024-04-02T13:30:31.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:31 smithi050 ceph-mon[35524]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T13:30:31.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:31 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:32.414 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:32.119+0000 7fbeff420200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:30:32.414 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:32.210+0000 7fbeff420200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:30:32.414 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:32.410+0000 7fbeff420200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:30:32.479 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:32 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:32.163+0000 7f89c0b5e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:30:32.479 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:32 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:32.260+0000 7f89c0b5e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:30:32.678 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:32.598+0000 7fbeff420200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:30:32.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:32.677+0000 7f1b2b7fe700 1 -- 172.21.15.33:0/182835160 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f1b24007d00 con 0x7f1b2c108360 2024-04-02T13:30:32.979 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:30:32 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:30:32.476+0000 7f89c0b5e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:30:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:32 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3877008446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:30:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:32 smithi145 ceph-mon[35646]: Standby manager daemon z started 2024-04-02T13:30:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:32 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.145:0/315574316' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:30:32.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:32 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.145:0/315574316' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:30:32.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:32 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.145:0/315574316' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:30:32.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:32 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.145:0/315574316' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:30:32.995 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:32.889+0000 7fbeff420200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:30:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:32 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3877008446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:30:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:32 smithi050 ceph-mon[35524]: Standby manager daemon z started 2024-04-02T13:30:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:32 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.145:0/315574316' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:30:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:32 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.145:0/315574316' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:30:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:32 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.145:0/315574316' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:30:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:32 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.145:0/315574316' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:30:33.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:32 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3877008446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:30:33.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:32 smithi033 ceph-mon[32390]: Standby manager daemon z started 2024-04-02T13:30:33.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:32 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.145:0/315574316' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:30:33.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:32 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.145:0/315574316' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:30:33.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:32 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.145:0/315574316' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:30:33.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:32 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.145:0/315574316' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:30:33.495 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:33 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:33.152+0000 7fbeff420200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:30:33.495 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:33 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:33.394+0000 7fbeff420200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:30:33.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:33 smithi145 ceph-mon[35646]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:33.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:33 smithi145 ceph-mon[35646]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-02T13:30:33.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:33 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:30:33.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:33 smithi050 ceph-mon[35524]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:33.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:33 smithi050 ceph-mon[35524]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-02T13:30:33.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:33 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:30:34.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:33 smithi033 ceph-mon[32390]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:34.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:33 smithi033 ceph-mon[32390]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-02T13:30:34.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:33 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:30:34.495 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:34 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:34.219+0000 7fbeff420200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:30:35.125 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:34 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:34.802+0000 7fbeff420200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:30:35.125 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:34 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:34.901+0000 7fbeff420200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:30:35.125 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:30:35 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:30:35.121+0000 7fbeff420200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:30:35.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:35.688+0000 7f1b2b7fe700 1 -- 172.21.15.33:0/182835160 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f1b240188d0 con 0x7f1b2c108360 2024-04-02T13:30:35.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:35 smithi033 ceph-mon[32390]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:35.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:35 smithi033 ceph-mon[32390]: Standby manager daemon y started 2024-04-02T13:30:35.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:35 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.50:0/1237600951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:30:35.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:35 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.50:0/1237600951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:30:35.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:35 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.50:0/1237600951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:30:35.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:35 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.50:0/1237600951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:30:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:35 smithi145 ceph-mon[35646]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:35 smithi145 ceph-mon[35646]: Standby manager daemon y started 2024-04-02T13:30:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:35 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.50:0/1237600951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:30:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:35 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.50:0/1237600951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:30:35.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:35 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.50:0/1237600951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:30:35.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:35 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.50:0/1237600951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:30:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:35 smithi050 ceph-mon[35524]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:35 smithi050 ceph-mon[35524]: Standby manager daemon y started 2024-04-02T13:30:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:35 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.50:0/1237600951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:30:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:35 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.50:0/1237600951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:30:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:35 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.50:0/1237600951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:30:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:35 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.50:0/1237600951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:30:36.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:36 smithi145 ceph-mon[35646]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-02T13:30:36.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:36 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:30:36.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:36 smithi050 ceph-mon[35524]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-02T13:30:36.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:36 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:30:37.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:36 smithi033 ceph-mon[32390]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-02T13:30:37.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:36 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:30:37.900 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:37 smithi033 ceph-mon[32390]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:37.900 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:37 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:37.900 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:37 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:37.900 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:37 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:37.900 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:37 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:37.900 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:37 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:37.900 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:37 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:37 smithi145 ceph-mon[35646]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:37 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:37 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:37 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:37 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:37 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:37.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:37 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:37.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:37 smithi050 ceph-mon[35524]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:37.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:37 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:37.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:37 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:37.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:37 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:37.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:37 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:37.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:37 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:37.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:37 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:39.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:39 smithi050 ceph-mon[35524]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:39.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:39 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T13:30:39.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:39 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:39.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:39 smithi145 ceph-mon[35646]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:39.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:39 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T13:30:39.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:39 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:40.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:39 smithi033 ceph-mon[32390]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:40.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:39 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T13:30:40.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:39 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:40.972 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:40 smithi033 ceph-mon[32390]: Deploying daemon osd.0 on smithi033 2024-04-02T13:30:40.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:40 smithi145 ceph-mon[35646]: Deploying daemon osd.0 on smithi033 2024-04-02T13:30:40.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:40 smithi050 ceph-mon[35524]: Deploying daemon osd.0 on smithi033 2024-04-02T13:30:41.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:41 smithi145 ceph-mon[35646]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:41.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:41 smithi050 ceph-mon[35524]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:42.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:41 smithi033 ceph-mon[32390]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:43.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:43 smithi033 ceph-mon[32390]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:43.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:43 smithi145 ceph-mon[35646]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:43.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:43 smithi050 ceph-mon[35524]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:45.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:45 smithi145 ceph-mon[35646]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:45.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:45 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:45.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:45 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:45.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:45 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:45.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:45 smithi050 ceph-mon[35524]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:45.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:45 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:45.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:45 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:45.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:45 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:46.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:45 smithi033 ceph-mon[32390]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:46.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:45 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:46.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:45 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:46.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:45 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:47.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:47 smithi145 ceph-mon[35646]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:47.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:47 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:47.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:47 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:47.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:47 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:47.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:47 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:47.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:47 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:47.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:47 smithi050 ceph-mon[35524]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:47.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:47 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:47.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:47 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:47.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:47 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:47.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:47 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:47.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:47 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:48.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:47 smithi033 ceph-mon[32390]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:48.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:48.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:48.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:48.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:48.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:48.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:48.286+0000 7f1b2b7fe700 1 -- 172.21.15.33:0/182835160 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1b2c062a30 con 0x7f1b14039890 2024-04-02T13:30:48.290 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 0 on host 'smithi033' 2024-04-02T13:30:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:48.288+0000 7f1b343b2700 1 -- 172.21.15.33:0/182835160 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b14039890 msgr2=0x7f1b1403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:48.288+0000 7f1b343b2700 1 --2- 172.21.15.33:0/182835160 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b14039890 0x7f1b1403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009fd0 tx=0x7f1b1c0060c0 comp rx=0 tx=0).stop 2024-04-02T13:30:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:48.288+0000 7f1b343b2700 1 -- 172.21.15.33:0/182835160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2c108360 msgr2=0x7f1b2c19fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:48.288+0000 7f1b343b2700 1 --2- 172.21.15.33:0/182835160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2c108360 0x7f1b2c19fdf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400d980 tx=0x7f1b2400dc90 comp rx=0 tx=0).stop 2024-04-02T13:30:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:48.288+0000 7f1b343b2700 1 -- 172.21.15.33:0/182835160 shutdown_connections 2024-04-02T13:30:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:48.288+0000 7f1b343b2700 1 --2- 172.21.15.33:0/182835160 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b14039890 0x7f1b1403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:48.288+0000 7f1b343b2700 1 --2- 172.21.15.33:0/182835160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2c108360 0x7f1b2c19fdf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:48.288+0000 7f1b343b2700 1 --2- 172.21.15.33:0/182835160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b2c103100 0x7f1b2c198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:48.288+0000 7f1b343b2700 1 --2- 172.21.15.33:0/182835160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c105a30 0x7f1b2c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:48.288+0000 7f1b343b2700 1 -- 172.21.15.33:0/182835160 >> 172.21.15.33:0/182835160 conn(0x7f1b2c0fcab0 msgr2=0x7f1b2c103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:48.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:48.289+0000 7f1b343b2700 1 -- 172.21.15.33:0/182835160 shutdown_connections 2024-04-02T13:30:48.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:30:48.289+0000 7f1b343b2700 1 -- 172.21.15.33:0/182835160 wait complete. 2024-04-02T13:30:48.977 DEBUG:teuthology.orchestra.run.smithi033:osd.0> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.0.service 2024-04-02T13:30:48.979 INFO:tasks.cephadm:Deploying osd.1 on smithi050 with /dev/vg_nvme/lv_4... 2024-04-02T13:30:48.979 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T13:30:49.229 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:30:49.538 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:49 smithi050 ceph-mon[35524]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:49.538 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:49 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:49.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:49 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:49.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:49 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:49.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:49 smithi050 ceph-mon[35524]: from='osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T13:30:49.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:49 smithi145 ceph-mon[35646]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:49.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:49 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:49.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:49 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:49.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:49 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:49.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:49 smithi145 ceph-mon[35646]: from='osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T13:30:49.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:49 smithi033 ceph-mon[32390]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:49.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:49 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:49.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:49 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:49.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:49 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:49.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:49 smithi033 ceph-mon[32390]: from='osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T13:30:50.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:50 smithi033 ceph-mon[32390]: from='osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T13:30:50.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:50 smithi033 ceph-mon[32390]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T13:30:50.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:50 smithi033 ceph-mon[32390]: from='osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-02T13:30:50.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:50 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:50.431 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:30:50 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0[44516]: 2024-04-02T13:30:50.288+0000 7f83fb5f0700 -1 osd.0 0 waiting for initial osdmap 2024-04-02T13:30:50.431 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:30:50 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0[44516]: 2024-04-02T13:30:50.291+0000 7f83f73e3700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:30:50.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:50 smithi050 ceph-mon[35524]: from='osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T13:30:50.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:50 smithi050 ceph-mon[35524]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T13:30:50.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:50 smithi050 ceph-mon[35524]: from='osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-02T13:30:50.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:50 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:50.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:50 smithi145 ceph-mon[35646]: from='osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T13:30:50.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:50 smithi145 ceph-mon[35646]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T13:30:50.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:50 smithi145 ceph-mon[35646]: from='osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-02T13:30:50.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:50 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:51.186 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T13:30:51.187 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T13:30:51.187 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-04-02T13:30:51.187 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-04-02T13:30:51.187 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00839347 s, 1.2 GB/s 2024-04-02T13:30:51.187 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-04-02T13:30:51.499 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:51 smithi050 ceph-mon[35524]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:51.500 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:51 smithi050 ceph-mon[35524]: from='osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-02T13:30:51.500 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:51 smithi050 ceph-mon[35524]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T13:30:51.500 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:51 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:51.500 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:51 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:51.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:51 smithi033 ceph-mon[32390]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:51.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:51 smithi033 ceph-mon[32390]: from='osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-02T13:30:51.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:51 smithi033 ceph-mon[32390]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T13:30:51.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:51 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:51.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:51 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:51.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:51 smithi145 ceph-mon[35646]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:51.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:51 smithi145 ceph-mon[35646]: from='osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-02T13:30:51.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:51 smithi145 ceph-mon[35646]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T13:30:51.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:51 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:51.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:51 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:51.871 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi050:vg_nvme/lv_4 2024-04-02T13:30:52.122 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:30:52.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:52 smithi050 ceph-mon[35524]: purged_snaps scrub starts 2024-04-02T13:30:52.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:52 smithi050 ceph-mon[35524]: purged_snaps scrub ok 2024-04-02T13:30:52.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:52 smithi050 ceph-mon[35524]: osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839] boot 2024-04-02T13:30:52.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:52 smithi050 ceph-mon[35524]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T13:30:52.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:52 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:52.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:52 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:52.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:52 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:52.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:52 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:52.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:52 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:52.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:52 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:52.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:52 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:52.497 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:52 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:52.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:52 smithi145 ceph-mon[35646]: purged_snaps scrub starts 2024-04-02T13:30:52.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:52 smithi145 ceph-mon[35646]: purged_snaps scrub ok 2024-04-02T13:30:52.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:52 smithi145 ceph-mon[35646]: osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839] boot 2024-04-02T13:30:52.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:52 smithi145 ceph-mon[35646]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T13:30:52.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:52 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:52.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:52 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:52.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:52 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:52.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:52 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:52.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:52 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:52.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:52 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:52.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:52 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:52.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:52 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:52.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:52 smithi033 ceph-mon[32390]: purged_snaps scrub starts 2024-04-02T13:30:52.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:52 smithi033 ceph-mon[32390]: purged_snaps scrub ok 2024-04-02T13:30:52.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:52 smithi033 ceph-mon[32390]: osd.0 [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839] boot 2024-04-02T13:30:52.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:52 smithi033 ceph-mon[32390]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T13:30:52.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:52 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:30:52.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:52 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:52.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:52 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:52.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:52 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:52.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:52 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:52.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:52 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:52.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:52 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:52.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:52 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:30:53.526 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:53 smithi050 ceph-mon[35524]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:53.526 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:53 smithi050 ceph-mon[35524]: Detected new or changed devices on smithi033 2024-04-02T13:30:53.527 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:53 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi033 to 17093M 2024-04-02T13:30:53.527 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:53 smithi050 ceph-mon[35524]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T13:30:53.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:53 smithi145 ceph-mon[35646]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:53.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:53 smithi145 ceph-mon[35646]: Detected new or changed devices on smithi033 2024-04-02T13:30:53.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:53 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi033 to 17093M 2024-04-02T13:30:53.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:53 smithi145 ceph-mon[35646]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T13:30:53.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:53 smithi033 ceph-mon[32390]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:30:53.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:53 smithi033 ceph-mon[32390]: Detected new or changed devices on smithi033 2024-04-02T13:30:53.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:53 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi033 to 17093M 2024-04-02T13:30:53.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:53 smithi033 ceph-mon[32390]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T13:30:53.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.756+0000 7f2f19625700 1 -- 172.21.15.50:0/3043368536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f141008e0 msgr2=0x7f2f14100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:53.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.756+0000 7f2f19625700 1 --2- 172.21.15.50:0/3043368536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f141008e0 0x7f2f14100d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400b0a0 tx=0x7f2f0400b3b0 comp rx=0 tx=0).stop 2024-04-02T13:30:53.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.756+0000 7f2f19625700 1 -- 172.21.15.50:0/3043368536 shutdown_connections 2024-04-02T13:30:53.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.756+0000 7f2f19625700 1 --2- 172.21.15.50:0/3043368536 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f14104260 0x7f2f1410ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:53.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.756+0000 7f2f19625700 1 --2- 172.21.15.50:0/3043368536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f141008e0 0x7f2f14100d00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:53.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.756+0000 7f2f19625700 1 --2- 172.21.15.50:0/3043368536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f141038c0 0x7f2f14103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:53.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.756+0000 7f2f19625700 1 -- 172.21.15.50:0/3043368536 >> 172.21.15.50:0/3043368536 conn(0x7f2f140fc260 msgr2=0x7f2f140fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:53.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.756+0000 7f2f19625700 1 -- 172.21.15.50:0/3043368536 shutdown_connections 2024-04-02T13:30:53.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.756+0000 7f2f19625700 1 -- 172.21.15.50:0/3043368536 wait complete. 2024-04-02T13:30:53.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.756+0000 7f2f19625700 1 Processor -- start 2024-04-02T13:30:53.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f19625700 1 -- start start 2024-04-02T13:30:53.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f19625700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f141008e0 0x7f2f14198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:53.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f19625700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f141038c0 0x7f2f14198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:53.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f19625700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f14104260 0x7f2f1419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:53.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f19625700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1410e1e0 con 0x7f2f141038c0 2024-04-02T13:30:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f19625700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1410e060 con 0x7f2f141008e0 2024-04-02T13:30:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f19625700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1410e360 con 0x7f2f14104260 2024-04-02T13:30:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f12ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f141008e0 0x7f2f14198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f12ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f141008e0 0x7f2f14198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37418/0 (socket says 172.21.15.50:37418) 2024-04-02T13:30:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f12ffd700 1 -- 172.21.15.50:0/2389765826 learned_addr learned my addr 172.21.15.50:0/2389765826 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T13:30:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f137fe700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f14104260 0x7f2f1419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f127fc700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f141038c0 0x7f2f14198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f12ffd700 1 -- 172.21.15.50:0/2389765826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f14104260 msgr2=0x7f2f1419fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f12ffd700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f14104260 0x7f2f1419fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f12ffd700 1 -- 172.21.15.50:0/2389765826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f141038c0 msgr2=0x7f2f14198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f12ffd700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f141038c0 0x7f2f14198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f12ffd700 1 -- 172.21.15.50:0/2389765826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f141a0440 con 0x7f2f141008e0 2024-04-02T13:30:53.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.757+0000 7f2f12ffd700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f141008e0 0x7f2f14198880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2f04006130 tx=0x7f2f04008fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:53.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.758+0000 7f2f0bfff700 1 -- 172.21.15.50:0/2389765826 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04021070 con 0x7f2f141008e0 2024-04-02T13:30:53.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.758+0000 7f2f19625700 1 -- 172.21.15.50:0/2389765826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f141a06d0 con 0x7f2f141008e0 2024-04-02T13:30:53.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.758+0000 7f2f0bfff700 1 -- 172.21.15.50:0/2389765826 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2f04007c80 con 0x7f2f141008e0 2024-04-02T13:30:53.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.758+0000 7f2f0bfff700 1 -- 172.21.15.50:0/2389765826 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0401b8f0 con 0x7f2f141008e0 2024-04-02T13:30:53.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.758+0000 7f2f19625700 1 -- 172.21.15.50:0/2389765826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f141a0b80 con 0x7f2f141008e0 2024-04-02T13:30:53.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.759+0000 7f2f19625700 1 -- 172.21.15.50:0/2389765826 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f140692b0 con 0x7f2f141008e0 2024-04-02T13:30:53.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.760+0000 7f2f0bfff700 1 -- 172.21.15.50:0/2389765826 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f2f04024070 con 0x7f2f141008e0 2024-04-02T13:30:53.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.760+0000 7f2f0bfff700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2f000a3bb0 0x7f2f000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:53.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.760+0000 7f2f0bfff700 1 -- 172.21.15.50:0/2389765826 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f2f0400f020 con 0x7f2f141008e0 2024-04-02T13:30:53.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.761+0000 7f2f127fc700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2f000a3bb0 0x7f2f000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:53.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.761+0000 7f2f127fc700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2f000a3bb0 0x7f2f000a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2efc005fd0 tx=0x7f2efc00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:53.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.765+0000 7f2f0bfff700 1 -- 172.21.15.50:0/2389765826 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2f0402b020 con 0x7f2f141008e0 2024-04-02T13:30:53.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:30:53.981+0000 7f2f19625700 1 -- 172.21.15.50:0/2389765826 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2f14062a30 con 0x7f2f000a3bb0 2024-04-02T13:30:54.608 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:54 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:30:54.608 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:54 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:30:54.608 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:54 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:54.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:54 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:30:54.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:54 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:30:54.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:54 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:54.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:54 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:30:54.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:54 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:30:54.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:54 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:55.629 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:55 smithi050 ceph-mon[35524]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:30:55.629 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:55 smithi050 ceph-mon[35524]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:55.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:55 smithi145 ceph-mon[35646]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:30:55.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:55 smithi145 ceph-mon[35646]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:55.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:55 smithi033 ceph-mon[32390]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:30:55.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:55 smithi033 ceph-mon[32390]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:57.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:57 smithi050 ceph-mon[35524]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:30:57.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:57 smithi050 ceph-mon[35524]: from='client.? 172.21.15.50:0/1277761888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e22709-5642-4aae-a9b3-f7b27c4bed96"}]: dispatch 2024-04-02T13:30:57.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:57 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e22709-5642-4aae-a9b3-f7b27c4bed96"}]: dispatch 2024-04-02T13:30:57.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:57 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6e22709-5642-4aae-a9b3-f7b27c4bed96"}]': finished 2024-04-02T13:30:57.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:57 smithi050 ceph-mon[35524]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T13:30:57.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:57 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:30:57.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:57 smithi145 ceph-mon[35646]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:30:57.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:57 smithi145 ceph-mon[35646]: from='client.? 172.21.15.50:0/1277761888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e22709-5642-4aae-a9b3-f7b27c4bed96"}]: dispatch 2024-04-02T13:30:57.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:57 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e22709-5642-4aae-a9b3-f7b27c4bed96"}]: dispatch 2024-04-02T13:30:57.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:57 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6e22709-5642-4aae-a9b3-f7b27c4bed96"}]': finished 2024-04-02T13:30:57.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:57 smithi145 ceph-mon[35646]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T13:30:57.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:57 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:30:57.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:57 smithi033 ceph-mon[32390]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:30:57.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:57 smithi033 ceph-mon[32390]: from='client.? 172.21.15.50:0/1277761888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e22709-5642-4aae-a9b3-f7b27c4bed96"}]: dispatch 2024-04-02T13:30:57.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:57 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e22709-5642-4aae-a9b3-f7b27c4bed96"}]: dispatch 2024-04-02T13:30:57.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:57 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6e22709-5642-4aae-a9b3-f7b27c4bed96"}]': finished 2024-04-02T13:30:57.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:57 smithi033 ceph-mon[32390]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T13:30:57.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:57 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:30:58.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:58 smithi145 ceph-mon[35646]: from='client.? 172.21.15.50:0/236029154' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:30:58.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:58 smithi050 ceph-mon[35524]: from='client.? 172.21.15.50:0/236029154' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:30:58.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:58 smithi033 ceph-mon[32390]: from='client.? 172.21.15.50:0/236029154' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:30:59.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:30:59 smithi145 ceph-mon[35646]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:30:59.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:30:59 smithi050 ceph-mon[35524]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:30:59.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:30:59 smithi033 ceph-mon[32390]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:01.718 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:01 smithi050 ceph-mon[35524]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:01.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:01 smithi145 ceph-mon[35646]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:01.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:01 smithi033 ceph-mon[32390]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:03 smithi145 ceph-mon[35646]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:03.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:03 smithi050 ceph-mon[35524]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:03.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:03 smithi033 ceph-mon[32390]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:04.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T13:31:04.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:04.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T13:31:04.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:04.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T13:31:04.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:05.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:05 smithi050 ceph-mon[35524]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:05.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:05 smithi050 ceph-mon[35524]: Deploying daemon osd.1 on smithi050 2024-04-02T13:31:05.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:05 smithi145 ceph-mon[35646]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:05.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:05 smithi145 ceph-mon[35646]: Deploying daemon osd.1 on smithi050 2024-04-02T13:31:05.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:05 smithi033 ceph-mon[32390]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:05.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:05 smithi033 ceph-mon[32390]: Deploying daemon osd.1 on smithi050 2024-04-02T13:31:07.692 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:07 smithi050 ceph-mon[35524]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:07.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:07 smithi145 ceph-mon[35646]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:07.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:07 smithi033 ceph-mon[32390]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:09.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:09 smithi050 ceph-mon[35524]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:09.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:09 smithi145 ceph-mon[35646]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:09.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:09 smithi033 ceph-mon[32390]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:10.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:10 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:10.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:10 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:10.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:10 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:10.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:10 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:10.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:10 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:10.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:10 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:11.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:10 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:11.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:10 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:11.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:10 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:11.597 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:11 smithi050 ceph-mon[35524]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:11.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:11 smithi145 ceph-mon[35646]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:12.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:11 smithi033 ceph-mon[32390]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:13.105 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 1 on host 'smithi050' 2024-04-02T13:31:13.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:31:13.100+0000 7f2f0bfff700 1 -- 172.21.15.50:0/2389765826 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2f14062a30 con 0x7f2f000a3bb0 2024-04-02T13:31:13.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:31:13.103+0000 7f2f19625700 1 -- 172.21.15.50:0/2389765826 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2f000a3bb0 msgr2=0x7f2f000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:13.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:31:13.103+0000 7f2f19625700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2f000a3bb0 0x7f2f000a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2efc005fd0 tx=0x7f2efc00a560 comp rx=0 tx=0).stop 2024-04-02T13:31:13.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:31:13.103+0000 7f2f19625700 1 -- 172.21.15.50:0/2389765826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f141008e0 msgr2=0x7f2f14198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:31:13.103+0000 7f2f19625700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f141008e0 0x7f2f14198880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2f04006130 tx=0x7f2f04008fe0 comp rx=0 tx=0).stop 2024-04-02T13:31:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:31:13.103+0000 7f2f19625700 1 -- 172.21.15.50:0/2389765826 shutdown_connections 2024-04-02T13:31:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:31:13.103+0000 7f2f19625700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2f000a3bb0 0x7f2f000a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:31:13.103+0000 7f2f19625700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2f14104260 0x7f2f1419fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:31:13.103+0000 7f2f19625700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f141008e0 0x7f2f14198880 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:31:13.103+0000 7f2f19625700 1 --2- 172.21.15.50:0/2389765826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f141038c0 0x7f2f14198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:31:13.103+0000 7f2f19625700 1 -- 172.21.15.50:0/2389765826 >> 172.21.15.50:0/2389765826 conn(0x7f2f140fc260 msgr2=0x7f2f14106bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:31:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:31:13.103+0000 7f2f19625700 1 -- 172.21.15.50:0/2389765826 shutdown_connections 2024-04-02T13:31:13.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:31:13.103+0000 7f2f19625700 1 -- 172.21.15.50:0/2389765826 wait complete. 2024-04-02T13:31:13.726 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:13 smithi050 ceph-mon[35524]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:13.727 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:13 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:13.727 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:13 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:13.727 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:13 smithi050 ceph-mon[35524]: from='osd.1 [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:31:13.727 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:13 smithi050 ceph-mon[35524]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:31:13.758 DEBUG:teuthology.orchestra.run.smithi050:osd.1> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.1.service 2024-04-02T13:31:13.800 INFO:tasks.cephadm:Deploying osd.2 on smithi145 with /dev/vg_nvme/lv_4... 2024-04-02T13:31:13.801 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T13:31:13.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:13 smithi145 ceph-mon[35646]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:13.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:13 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:13.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:13 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:13.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:13 smithi145 ceph-mon[35646]: from='osd.1 [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:31:13.831 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:13 smithi145 ceph-mon[35646]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:31:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:13 smithi033 ceph-mon[32390]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:13 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:13 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:13 smithi033 ceph-mon[32390]: from='osd.1 [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:31:14.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:13 smithi033 ceph-mon[32390]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:31:14.047 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:31:15.090 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:14 smithi145 ceph-mon[35646]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:15.091 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:15.091 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:15.091 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:14 smithi145 ceph-mon[35646]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T13:31:15.091 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:14 smithi145 ceph-mon[35646]: from='osd.1 [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:31:15.091 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:14 smithi145 ceph-mon[35646]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T13:31:15.091 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:15.091 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:14 smithi145 ceph-mon[35646]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:31:15.159 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:14 smithi050 ceph-mon[35524]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:15.160 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:14 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:15.160 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:14 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:15.160 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:14 smithi050 ceph-mon[35524]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T13:31:15.160 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:14 smithi050 ceph-mon[35524]: from='osd.1 [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:31:15.160 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:14 smithi050 ceph-mon[35524]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T13:31:15.160 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:14 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:15.160 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:14 smithi050 ceph-mon[35524]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:31:15.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:14 smithi033 ceph-mon[32390]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:15.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:15.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:15.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:14 smithi033 ceph-mon[32390]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T13:31:15.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:14 smithi033 ceph-mon[32390]: from='osd.1 [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:31:15.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:14 smithi033 ceph-mon[32390]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T13:31:15.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:15.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:14 smithi033 ceph-mon[32390]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:31:15.745 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:31:15 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1[40110]: 2024-04-02T13:31:15.540+0000 7f4f62845700 -1 osd.1 0 waiting for initial osdmap 2024-04-02T13:31:15.745 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:31:15 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1[40110]: 2024-04-02T13:31:15.543+0000 7f4f5a630700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:31:15.917 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T13:31:15.917 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T13:31:15.917 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2024-04-02T13:31:15.917 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2024-04-02T13:31:15.917 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122339 s, 857 MB/s 2024-04-02T13:31:15.917 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2024-04-02T13:31:16.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:16 smithi033 ceph-mon[32390]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-04-02T13:31:16.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:16 smithi033 ceph-mon[32390]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T13:31:16.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:16.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:16.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:16 smithi033 ceph-mon[32390]: from='osd.1 ' entity='osd.1' 2024-04-02T13:31:16.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:16 smithi145 ceph-mon[35646]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-04-02T13:31:16.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:16 smithi145 ceph-mon[35646]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T13:31:16.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:16.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:16.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:16 smithi145 ceph-mon[35646]: from='osd.1 ' entity='osd.1' 2024-04-02T13:31:16.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:16 smithi050 ceph-mon[35524]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-04-02T13:31:16.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:16 smithi050 ceph-mon[35524]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T13:31:16.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:16.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:16.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:16 smithi050 ceph-mon[35524]: from='osd.1 ' entity='osd.1' 2024-04-02T13:31:16.526 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi145:vg_nvme/lv_4 2024-04-02T13:31:16.811 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:31:17.243 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:17 smithi050 ceph-mon[35524]: purged_snaps scrub starts 2024-04-02T13:31:17.243 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:17 smithi050 ceph-mon[35524]: purged_snaps scrub ok 2024-04-02T13:31:17.243 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:17 smithi050 ceph-mon[35524]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:17.243 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:17.243 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:17 smithi050 ceph-mon[35524]: osd.1 [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736] boot 2024-04-02T13:31:17.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:17 smithi050 ceph-mon[35524]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T13:31:17.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:17.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:17 smithi033 ceph-mon[32390]: purged_snaps scrub starts 2024-04-02T13:31:17.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:17 smithi033 ceph-mon[32390]: purged_snaps scrub ok 2024-04-02T13:31:17.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:17 smithi033 ceph-mon[32390]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:17.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:17.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:17 smithi033 ceph-mon[32390]: osd.1 [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736] boot 2024-04-02T13:31:17.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:17 smithi033 ceph-mon[32390]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T13:31:17.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:17.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:17 smithi145 ceph-mon[35646]: purged_snaps scrub starts 2024-04-02T13:31:17.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:17 smithi145 ceph-mon[35646]: purged_snaps scrub ok 2024-04-02T13:31:17.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:17 smithi145 ceph-mon[35646]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:31:17.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:17.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:17 smithi145 ceph-mon[35646]: osd.1 [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736] boot 2024-04-02T13:31:17.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:17 smithi145 ceph-mon[35646]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T13:31:17.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:31:18.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.382+0000 7f5d443b8700 1 -- 172.21.15.145:0/122356647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d3c103110 msgr2=0x7f5d3c1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:18.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.382+0000 7f5d443b8700 1 --2- 172.21.15.145:0/122356647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d3c103110 0x7f5d3c1099c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400b0a0 tx=0x7f5d3400b3b0 comp rx=0 tx=0).stop 2024-04-02T13:31:18.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 -- 172.21.15.145:0/122356647 shutdown_connections 2024-04-02T13:31:18.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 --2- 172.21.15.145:0/122356647 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d3c103110 0x7f5d3c1099c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:18.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 --2- 172.21.15.145:0/122356647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d3c101570 0x7f5d3c101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:18.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 --2- 172.21.15.145:0/122356647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3c102770 0x7f5d3c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:18.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 -- 172.21.15.145:0/122356647 >> 172.21.15.145:0/122356647 conn(0x7f5d3c0fcaf0 msgr2=0x7f5d3c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:31:18.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 -- 172.21.15.145:0/122356647 shutdown_connections 2024-04-02T13:31:18.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 -- 172.21.15.145:0/122356647 wait complete. 2024-04-02T13:31:18.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 Processor -- start 2024-04-02T13:31:18.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 -- start start 2024-04-02T13:31:18.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3c101570 0x7f5d3c105500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:18.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d3c102770 0x7f5d3c103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:18.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d3c103110 0x7f5d3c104090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:18.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10d230 con 0x7f5d3c101570 2024-04-02T13:31:18.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10d0b0 con 0x7f5d3c103110 2024-04-02T13:31:18.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.383+0000 7f5d443b8700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10d3b0 con 0x7f5d3c102770 2024-04-02T13:31:18.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d41953700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d3c102770 0x7f5d3c103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:18.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d41953700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d3c102770 0x7f5d3c103b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45818/0 (socket says 172.21.15.145:45818) 2024-04-02T13:31:18.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d41953700 1 -- 172.21.15.145:0/312473931 learned_addr learned my addr 172.21.15.145:0/312473931 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-02T13:31:18.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d42154700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3c101570 0x7f5d3c105500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:18.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d42955700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d3c103110 0x7f5d3c104090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:18.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d41953700 1 -- 172.21.15.145:0/312473931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d3c103110 msgr2=0x7f5d3c104090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:18.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d41953700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d3c103110 0x7f5d3c104090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:18.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d41953700 1 -- 172.21.15.145:0/312473931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3c101570 msgr2=0x7f5d3c105500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:18.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d41953700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3c101570 0x7f5d3c105500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:18.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d41953700 1 -- 172.21.15.145:0/312473931 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c104940 con 0x7f5d3c102770 2024-04-02T13:31:18.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d42955700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d3c103110 0x7f5d3c104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:31:18.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d41953700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d3c102770 0x7f5d3c103b50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00d980 tx=0x7f5d2c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:31:18.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d337fe700 1 -- 172.21.15.145:0/312473931 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c017070 con 0x7f5d3c102770 2024-04-02T13:31:18.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d443b8700 1 -- 172.21.15.145:0/312473931 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c1a2ac0 con 0x7f5d3c102770 2024-04-02T13:31:18.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d337fe700 1 -- 172.21.15.145:0/312473931 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5d2c005020 con 0x7f5d3c102770 2024-04-02T13:31:18.391 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.384+0000 7f5d337fe700 1 -- 172.21.15.145:0/312473931 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c009bd0 con 0x7f5d3c102770 2024-04-02T13:31:18.391 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.385+0000 7f5d443b8700 1 -- 172.21.15.145:0/312473931 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1a2fd0 con 0x7f5d3c102770 2024-04-02T13:31:18.391 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.386+0000 7f5d443b8700 1 -- 172.21.15.145:0/312473931 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c0692b0 con 0x7f5d3c102770 2024-04-02T13:31:18.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.387+0000 7f5d337fe700 1 -- 172.21.15.145:0/312473931 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f5d2c0051d0 con 0x7f5d3c102770 2024-04-02T13:31:18.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.387+0000 7f5d337fe700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5d280a3bb0 0x7f5d280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:18.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.387+0000 7f5d337fe700 1 -- 172.21.15.145:0/312473931 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f5d2c0ce720 con 0x7f5d3c102770 2024-04-02T13:31:18.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.387+0000 7f5d42154700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5d280a3bb0 0x7f5d280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:18.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.388+0000 7f5d42154700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5d280a3bb0 0x7f5d280a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c1025d0 tx=0x7f5d38009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:31:18.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.392+0000 7f5d337fe700 1 -- 172.21.15.145:0/312473931 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5d2c020030 con 0x7f5d3c102770 2024-04-02T13:31:18.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:18 smithi145 ceph-mon[35646]: Detected new or changed devices on smithi050 2024-04-02T13:31:18.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:18.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:18 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi050 to 17093M 2024-04-02T13:31:18.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:18.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:18.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:18.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:18.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:18.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:18.601+0000 7f5d443b8700 1 -- 172.21.15.145:0/312473931 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5d3c0febe0 con 0x7f5d280a3bb0 2024-04-02T13:31:18.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:18 smithi050 ceph-mon[35524]: Detected new or changed devices on smithi050 2024-04-02T13:31:18.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:18.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:18 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi050 to 17093M 2024-04-02T13:31:18.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:18.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:18.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:18.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:18.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:18.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:18 smithi033 ceph-mon[32390]: Detected new or changed devices on smithi050 2024-04-02T13:31:18.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:18.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:18 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi050 to 17093M 2024-04-02T13:31:18.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:18.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:18.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:18.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:18.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:18.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:18.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:19.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:19 smithi145 ceph-mon[35646]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:19.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:19 smithi145 ceph-mon[35646]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T13:31:19.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:19 smithi145 ceph-mon[35646]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:31:19.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:19 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:31:19.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:19 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:31:19.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:19 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:19.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:19 smithi050 ceph-mon[35524]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:19.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:19 smithi050 ceph-mon[35524]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T13:31:19.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:19 smithi050 ceph-mon[35524]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:31:19.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:19 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:31:19.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:19 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:31:19.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:19 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:19.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:19 smithi033 ceph-mon[32390]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:19.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:19 smithi033 ceph-mon[32390]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T13:31:19.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:19 smithi033 ceph-mon[32390]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:31:19.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:19 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:31:19.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:19 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:31:19.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:19 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:21.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:21 smithi145 ceph-mon[35646]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:21.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:21 smithi145 ceph-mon[35646]: from='client.? 172.21.15.145:0/2101427665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bdfdc32-0526-4f03-bd3f-50b251e80e53"}]: dispatch 2024-04-02T13:31:21.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:21 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bdfdc32-0526-4f03-bd3f-50b251e80e53"}]: dispatch 2024-04-02T13:31:21.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:21 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bdfdc32-0526-4f03-bd3f-50b251e80e53"}]': finished 2024-04-02T13:31:21.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:21 smithi145 ceph-mon[35646]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T13:31:21.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:21 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:21.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:21 smithi050 ceph-mon[35524]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:21.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:21 smithi050 ceph-mon[35524]: from='client.? 172.21.15.145:0/2101427665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bdfdc32-0526-4f03-bd3f-50b251e80e53"}]: dispatch 2024-04-02T13:31:21.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:21 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bdfdc32-0526-4f03-bd3f-50b251e80e53"}]: dispatch 2024-04-02T13:31:21.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:21 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bdfdc32-0526-4f03-bd3f-50b251e80e53"}]': finished 2024-04-02T13:31:21.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:21 smithi050 ceph-mon[35524]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T13:31:21.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:21 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:21.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:21 smithi033 ceph-mon[32390]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:21.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:21 smithi033 ceph-mon[32390]: from='client.? 172.21.15.145:0/2101427665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bdfdc32-0526-4f03-bd3f-50b251e80e53"}]: dispatch 2024-04-02T13:31:21.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:21 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bdfdc32-0526-4f03-bd3f-50b251e80e53"}]: dispatch 2024-04-02T13:31:21.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:21 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bdfdc32-0526-4f03-bd3f-50b251e80e53"}]': finished 2024-04-02T13:31:21.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:21 smithi033 ceph-mon[32390]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T13:31:21.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:21 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:22.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:22 smithi145 ceph-mon[35646]: from='client.? 172.21.15.145:0/429656992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:31:22.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:22 smithi050 ceph-mon[35524]: from='client.? 172.21.15.145:0/429656992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:31:22.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:22 smithi033 ceph-mon[32390]: from='client.? 172.21.15.145:0/429656992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:31:23.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:23 smithi145 ceph-mon[35646]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:23.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:23 smithi050 ceph-mon[35524]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:23.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:23 smithi033 ceph-mon[32390]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:25.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:25 smithi145 ceph-mon[35646]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:25.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:25 smithi050 ceph-mon[35524]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:25.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:25 smithi033 ceph-mon[32390]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:27.352 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:27 smithi145 ceph-mon[35646]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:27.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:27 smithi050 ceph-mon[35524]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:27.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:27 smithi033 ceph-mon[32390]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:28.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:28 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T13:31:28.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:28 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:28.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:28 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T13:31:28.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:28 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:28.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T13:31:28.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:29.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:29 smithi145 ceph-mon[35646]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:29.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:29 smithi145 ceph-mon[35646]: Deploying daemon osd.2 on smithi145 2024-04-02T13:31:29.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:29 smithi050 ceph-mon[35524]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:29.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:29 smithi050 ceph-mon[35524]: Deploying daemon osd.2 on smithi145 2024-04-02T13:31:29.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:29 smithi033 ceph-mon[32390]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:29.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:29 smithi033 ceph-mon[32390]: Deploying daemon osd.2 on smithi145 2024-04-02T13:31:31.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:31 smithi145 ceph-mon[35646]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:31.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:31 smithi050 ceph-mon[35524]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:31.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:31 smithi033 ceph-mon[32390]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:33.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:33 smithi145 ceph-mon[35646]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:33.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:33 smithi050 ceph-mon[35524]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:33.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:33 smithi033 ceph-mon[32390]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:34.842 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:34 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:34.842 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:34 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:34.842 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:34 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:34.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:34 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:34.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:34 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:34.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:34 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:35.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:34 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:35.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:34 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:35.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:34 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:35.680 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:35 smithi145 ceph-mon[35646]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:35 smithi050 ceph-mon[35524]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:35 smithi033 ceph-mon[32390]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:36.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:36.767+0000 7f5d337fe700 1 -- 172.21.15.145:0/312473931 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5d3c0febe0 con 0x7f5d280a3bb0 2024-04-02T13:31:36.771 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 2 on host 'smithi145' 2024-04-02T13:31:36.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:36.771+0000 7f5d443b8700 1 -- 172.21.15.145:0/312473931 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5d280a3bb0 msgr2=0x7f5d280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:36.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:36.771+0000 7f5d443b8700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5d280a3bb0 0x7f5d280a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c1025d0 tx=0x7f5d38009190 comp rx=0 tx=0).stop 2024-04-02T13:31:36.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:36.771+0000 7f5d443b8700 1 -- 172.21.15.145:0/312473931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d3c102770 msgr2=0x7f5d3c103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:36.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:36.771+0000 7f5d443b8700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d3c102770 0x7f5d3c103b50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00d980 tx=0x7f5d2c00dc90 comp rx=0 tx=0).stop 2024-04-02T13:31:36.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:36.771+0000 7f5d443b8700 1 -- 172.21.15.145:0/312473931 shutdown_connections 2024-04-02T13:31:36.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:36.771+0000 7f5d443b8700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5d280a3bb0 0x7f5d280a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:36.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:36.771+0000 7f5d443b8700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d3c103110 0x7f5d3c104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:36.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:36.771+0000 7f5d443b8700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3c101570 0x7f5d3c105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:36.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:36.771+0000 7f5d443b8700 1 --2- 172.21.15.145:0/312473931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d3c102770 0x7f5d3c103b50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:36.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:36.771+0000 7f5d443b8700 1 -- 172.21.15.145:0/312473931 >> 172.21.15.145:0/312473931 conn(0x7f5d3c0fcaf0 msgr2=0x7f5d3c0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:31:36.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:36.771+0000 7f5d443b8700 1 -- 172.21.15.145:0/312473931 shutdown_connections 2024-04-02T13:31:36.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:31:36.771+0000 7f5d443b8700 1 -- 172.21.15.145:0/312473931 wait complete. 2024-04-02T13:31:37.402 DEBUG:teuthology.orchestra.run.smithi145:osd.2> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.2.service 2024-04-02T13:31:37.404 INFO:tasks.cephadm:Deploying osd.3 on smithi033 with /dev/vg_nvme/lv_3... 2024-04-02T13:31:37.404 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T13:31:37.659 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:31:37.675 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:31:37 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2[40798]: 2024-04-02T13:31:37.432+0000 7fc99d1d27c0 -1 osd.2 0 log_to_monitors true 2024-04-02T13:31:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:37 smithi145 ceph-mon[35646]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:37 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:37 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:37 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:37 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:37 smithi145 ceph-mon[35646]: from='osd.2 [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:31:37.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:37 smithi145 ceph-mon[35646]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:31:38.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:37 smithi033 ceph-mon[32390]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:38.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:37 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:38.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:37 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:38.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:37 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:38.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:37 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:38.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:37 smithi033 ceph-mon[32390]: from='osd.2 [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:31:38.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:37 smithi033 ceph-mon[32390]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:31:38.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:37 smithi050 ceph-mon[35524]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:38.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:37 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:38.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:37 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:38.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:37 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:38.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:37 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:38.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:37 smithi050 ceph-mon[35524]: from='osd.2 [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:31:38.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:37 smithi050 ceph-mon[35524]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:31:39.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:39 smithi050 ceph-mon[35524]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:39.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:39 smithi050 ceph-mon[35524]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T13:31:39.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:39 smithi050 ceph-mon[35524]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T13:31:39.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:39 smithi050 ceph-mon[35524]: from='osd.2 [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:31:39.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:39 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:39.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:39 smithi050 ceph-mon[35524]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:31:39.479 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:31:39 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2[40798]: 2024-04-02T13:31:39.075+0000 7fc992837700 -1 osd.2 0 waiting for initial osdmap 2024-04-02T13:31:39.480 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:31:39 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2[40798]: 2024-04-02T13:31:39.081+0000 7fc98e62a700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:31:39.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:39 smithi145 ceph-mon[35646]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:39.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:39 smithi145 ceph-mon[35646]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T13:31:39.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:39 smithi145 ceph-mon[35646]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T13:31:39.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:39 smithi145 ceph-mon[35646]: from='osd.2 [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:31:39.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:39 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:39.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:39 smithi145 ceph-mon[35646]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:31:39.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:39 smithi033 ceph-mon[32390]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:39.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:39 smithi033 ceph-mon[32390]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T13:31:39.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:39 smithi033 ceph-mon[32390]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T13:31:39.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:39 smithi033 ceph-mon[32390]: from='osd.2 [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:31:39.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:39 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:39.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:39 smithi033 ceph-mon[32390]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:31:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T13:31:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T13:31:39.670 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-02T13:31:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-02T13:31:39.670 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124826 s, 840 MB/s 2024-04-02T13:31:39.671 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-02T13:31:40.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:40 smithi033 ceph-mon[32390]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-04-02T13:31:40.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:40 smithi033 ceph-mon[32390]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T13:31:40.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:40 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:40.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:40 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:40.288 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi033:vg_nvme/lv_3 2024-04-02T13:31:40.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:40 smithi145 ceph-mon[35646]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-04-02T13:31:40.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:40 smithi145 ceph-mon[35646]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T13:31:40.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:40 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:40.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:40 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:40.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:40 smithi050 ceph-mon[35524]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-04-02T13:31:40.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:40 smithi050 ceph-mon[35524]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T13:31:40.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:40 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:40.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:40 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:40.540 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:31:41.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: purged_snaps scrub starts 2024-04-02T13:31:41.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: purged_snaps scrub ok 2024-04-02T13:31:41.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:41.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:41.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: osd.2 [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265] boot 2024-04-02T13:31:41.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T13:31:41.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:41.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: Detected new or changed devices on smithi145 2024-04-02T13:31:41.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:41.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi145 to 17088M 2024-04-02T13:31:41.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:41.256 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:41.256 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.256 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:41.256 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:41.256 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:41.256 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: purged_snaps scrub starts 2024-04-02T13:31:41.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: purged_snaps scrub ok 2024-04-02T13:31:41.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: osd.2 [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265] boot 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: Detected new or changed devices on smithi145 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi145 to 17088M 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:41.481 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:41.481 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:41.481 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: purged_snaps scrub starts 2024-04-02T13:31:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: purged_snaps scrub ok 2024-04-02T13:31:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:31:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: osd.2 [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265] boot 2024-04-02T13:31:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T13:31:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:31:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: Detected new or changed devices on smithi145 2024-04-02T13:31:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:41.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi145 to 17088M 2024-04-02T13:31:41.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:41.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:41.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:41.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:41.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:41.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:41.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.191+0000 7fd3bb241700 1 -- 172.21.15.33:0/2638414313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b41010a0 msgr2=0x7fd3b410ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.191+0000 7fd3bb241700 1 --2- 172.21.15.33:0/2638414313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b41010a0 0x7fd3b410ac70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00b0a0 tx=0x7fd3ac00b3b0 comp rx=0 tx=0).stop 2024-04-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.191+0000 7fd3bb241700 1 -- 172.21.15.33:0/2638414313 shutdown_connections 2024-04-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.191+0000 7fd3bb241700 1 --2- 172.21.15.33:0/2638414313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b41010a0 0x7fd3b410ac70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.191+0000 7fd3bb241700 1 --2- 172.21.15.33:0/2638414313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3b4103c80 0x7fd3b41040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.191+0000 7fd3bb241700 1 --2- 172.21.15.33:0/2638414313 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3b4100700 0x7fd3b4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.191+0000 7fd3bb241700 1 -- 172.21.15.33:0/2638414313 >> 172.21.15.33:0/2638414313 conn(0x7fd3b40fc280 msgr2=0x7fd3b40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.191+0000 7fd3bb241700 1 -- 172.21.15.33:0/2638414313 shutdown_connections 2024-04-02T13:31:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.191+0000 7fd3bb241700 1 -- 172.21.15.33:0/2638414313 wait complete. 2024-04-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.191+0000 7fd3bb241700 1 Processor -- start 2024-04-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.191+0000 7fd3bb241700 1 -- start start 2024-04-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3bb241700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3b4100700 0x7fd3b4103380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3bb241700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b41010a0 0x7fd3b41019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3bb241700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3b4103c80 0x7fd3b4101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3bb241700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b410e230 con 0x7fd3b41010a0 2024-04-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3bb241700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b410e0b0 con 0x7fd3b4103c80 2024-04-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3bb241700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b410e3b0 con 0x7fd3b4100700 2024-04-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3b3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b41010a0 0x7fd3b41019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:42.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3b3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b41010a0 0x7fd3b41019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33374/0 (socket says 172.21.15.33:33374) 2024-04-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3b3fff700 1 -- 172.21.15.33:0/1651661863 learned_addr learned my addr 172.21.15.33:0/1651661863 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3b8fdd700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3b4100700 0x7fd3b4103380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3b97de700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3b4103c80 0x7fd3b4101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3b3fff700 1 -- 172.21.15.33:0/1651661863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3b4100700 msgr2=0x7fd3b4103380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3b3fff700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3b4100700 0x7fd3b4103380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3b3fff700 1 -- 172.21.15.33:0/1651661863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3b4103c80 msgr2=0x7fd3b4101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3b3fff700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3b4103c80 0x7fd3b4101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3b3fff700 1 -- 172.21.15.33:0/1651661863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b41027c0 con 0x7fd3b41010a0 2024-04-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3b3fff700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b41010a0 0x7fd3b41019d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000d980 tx=0x7fd3a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3b1ffb700 1 -- 172.21.15.33:0/1651661863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0016070 con 0x7fd3b41010a0 2024-04-02T13:31:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3bb241700 1 -- 172.21.15.33:0/1651661863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b41a2910 con 0x7fd3b41010a0 2024-04-02T13:31:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.192+0000 7fd3b1ffb700 1 -- 172.21.15.33:0/1651661863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd3a0004d00 con 0x7fd3b41010a0 2024-04-02T13:31:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.193+0000 7fd3b1ffb700 1 -- 172.21.15.33:0/1651661863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a00055d0 con 0x7fd3b41010a0 2024-04-02T13:31:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.193+0000 7fd3bb241700 1 -- 172.21.15.33:0/1651661863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b41a2e50 con 0x7fd3b41010a0 2024-04-02T13:31:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.194+0000 7fd3bb241700 1 -- 172.21.15.33:0/1651661863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3b40692b0 con 0x7fd3b41010a0 2024-04-02T13:31:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.195+0000 7fd3b1ffb700 1 -- 172.21.15.33:0/1651661863 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fd3a0004750 con 0x7fd3b41010a0 2024-04-02T13:31:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.195+0000 7fd3b1ffb700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd3a40a3bb0 0x7fd3a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.196+0000 7fd3b8fdd700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd3a40a3bb0 0x7fd3a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.196+0000 7fd3b1ffb700 1 -- 172.21.15.33:0/1651661863 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7fd3a00cf100 con 0x7fd3b41010a0 2024-04-02T13:31:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.196+0000 7fd3b8fdd700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd3a40a3bb0 0x7fd3a40a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8005950 tx=0x7fd3a800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:31:42.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.199+0000 7fd3b1ffb700 1 -- 172.21.15.33:0/1651661863 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd3a001f020 con 0x7fd3b41010a0 2024-04-02T13:31:42.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:42.419+0000 7fd3bb241700 1 -- 172.21.15.33:0/1651661863 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd3b4062af0 con 0x7fd3a40a3bb0 2024-04-02T13:31:42.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:42 smithi145 ceph-mon[35646]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T13:31:42.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:42 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:31:42.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:42 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:31:42.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:42 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:31:42.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:42 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:42.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:42 smithi050 ceph-mon[35524]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T13:31:42.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:42 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:31:42.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:42 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:31:42.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:42 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:31:42.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:42 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:42.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:42 smithi033 ceph-mon[32390]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T13:31:42.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:42 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:31:42.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:42 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:31:42.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:42 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:31:42.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:42 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:43.721 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44405]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:31:43.721 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44405]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:31:43.721 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44405]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:31:43.721 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44405]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:31:43.721 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:43 smithi145 ceph-mon[35646]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:43.721 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:43 smithi145 ceph-mon[35646]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:31:43.721 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:43 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T13:31:43.722 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:43 smithi145 ceph-mon[35646]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T13:31:43.722 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:43 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:31:43.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:43 smithi050 ceph-mon[35524]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:43.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:43 smithi050 ceph-mon[35524]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:31:43.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:43 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T13:31:43.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:43 smithi050 ceph-mon[35524]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T13:31:43.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:43 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:31:43.745 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44110]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:31:43.745 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44110]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:31:43.745 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44110]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:31:43.745 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44110]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:31:43.746 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44113]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:31:43.746 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44113]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:31:43.746 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44113]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:31:43.746 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44113]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:31:43.754 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49352]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:31:43.754 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49352]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:31:43.754 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49352]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:31:43.754 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49352]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:31:43.754 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49355]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:31:43.754 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49355]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:31:43.755 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49355]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:31:43.755 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49355]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:31:43.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:43 smithi033 ceph-mon[32390]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:43.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:43 smithi033 ceph-mon[32390]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:31:43.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:43 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T13:31:43.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:43 smithi033 ceph-mon[32390]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T13:31:43.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:43 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:31:43.979 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44408]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:31:43.979 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44408]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:31:43.979 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44408]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:31:43.979 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44408]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:31:43.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44411]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:31:43.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44411]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:31:43.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44411]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:31:43.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44411]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:31:43.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44414]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:31:43.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44414]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:31:43.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44414]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:31:43.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:43 smithi145 sudo[44414]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:31:44.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49361]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:31:44.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49361]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:31:44.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49361]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:31:44.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49361]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:31:44.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49364]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:31:44.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49364]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:31:44.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49364]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:31:44.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:43 smithi033 sudo[49364]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:31:44.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44116]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:31:44.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44116]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:31:44.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44116]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:31:44.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44116]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:31:44.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44119]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:31:44.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44119]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:31:44.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44119]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:31:44.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:43 smithi050 sudo[44119]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:31:44.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:31:44.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T13:31:44.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:44.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:44.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:44.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:31:44.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:31:44.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:31:44.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:31:44.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:31:44.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:31:44.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:31:44.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:31:44.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:31:44.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:31:44.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:31:44.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:31:44.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:31:44.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:31:44.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:44 smithi145 ceph-mon[35646]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:31:44.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:31:44.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T13:31:44.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:44.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:44.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:44.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:31:44.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:31:44.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:31:44.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:31:44.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:31:44.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:31:44.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:31:44.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:31:44.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:31:44.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:31:44.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:31:44.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:31:44.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:31:44.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:31:44.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:44 smithi050 ceph-mon[35524]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:31:44.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:31:44.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T13:31:44.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:44.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:44.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:44.756 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:31:44.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:44 smithi033 ceph-mon[32390]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:31:44.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:31:44.753+0000 7fd3b1ffb700 1 -- 172.21.15.33:0/1651661863 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd3a0098fe0 con 0x7fd3b41010a0 2024-04-02T13:31:45.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:45 smithi145 ceph-mon[35646]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:45.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:45 smithi145 ceph-mon[35646]: osdmap e22: 3 total, 3 up, 3 in 2024-04-02T13:31:45.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:45 smithi145 ceph-mon[35646]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T13:31:45.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:45 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3329111175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78b02725-b00b-4bdc-927f-b5d72ad92753"}]: dispatch 2024-04-02T13:31:45.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:45 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3329111175' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78b02725-b00b-4bdc-927f-b5d72ad92753"}]': finished 2024-04-02T13:31:45.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:45 smithi145 ceph-mon[35646]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T13:31:45.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:45 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:31:45.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:45 smithi050 ceph-mon[35524]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:45.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:45 smithi050 ceph-mon[35524]: osdmap e22: 3 total, 3 up, 3 in 2024-04-02T13:31:45.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:45 smithi050 ceph-mon[35524]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T13:31:45.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:45 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3329111175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78b02725-b00b-4bdc-927f-b5d72ad92753"}]: dispatch 2024-04-02T13:31:45.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:45 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3329111175' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78b02725-b00b-4bdc-927f-b5d72ad92753"}]': finished 2024-04-02T13:31:45.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:45 smithi050 ceph-mon[35524]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T13:31:45.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:45 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:31:45.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:45 smithi033 ceph-mon[32390]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:45.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:45 smithi033 ceph-mon[32390]: osdmap e22: 3 total, 3 up, 3 in 2024-04-02T13:31:45.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:45 smithi033 ceph-mon[32390]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T13:31:45.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:45 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3329111175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78b02725-b00b-4bdc-927f-b5d72ad92753"}]: dispatch 2024-04-02T13:31:45.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:45 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3329111175' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78b02725-b00b-4bdc-927f-b5d72ad92753"}]': finished 2024-04-02T13:31:45.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:45 smithi033 ceph-mon[32390]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T13:31:45.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:45 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:31:46.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:46 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3178467648' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:31:46.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:46 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3178467648' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:31:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:46 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3178467648' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:31:47.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:47 smithi050 ceph-mon[35524]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:47.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:47 smithi033 ceph-mon[32390]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:47.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:47 smithi145 ceph-mon[35646]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:49.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:49 smithi050 ceph-mon[35524]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:49.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:49 smithi033 ceph-mon[32390]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:49.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:49 smithi145 ceph-mon[35646]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:51.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:51 smithi050 ceph-mon[35524]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:51.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:51 smithi033 ceph-mon[32390]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:51.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:51 smithi145 ceph-mon[35646]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:53.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:53 smithi050 ceph-mon[35524]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:53.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:53 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T13:31:53.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:53 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:53.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:53 smithi050 ceph-mon[35524]: Deploying daemon osd.3 on smithi033 2024-04-02T13:31:53.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:53 smithi033 ceph-mon[32390]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:53.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T13:31:53.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:53.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:53 smithi033 ceph-mon[32390]: Deploying daemon osd.3 on smithi033 2024-04-02T13:31:53.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:53 smithi145 ceph-mon[35646]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:53.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:53 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T13:31:53.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:53 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:53.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:53 smithi145 ceph-mon[35646]: Deploying daemon osd.3 on smithi033 2024-04-02T13:31:55.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:55 smithi033 ceph-mon[32390]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:55.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:55 smithi050 ceph-mon[35524]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:55.781 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:55 smithi145 ceph-mon[35646]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:57.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:57 smithi050 ceph-mon[35524]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:57.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:57 smithi033 ceph-mon[32390]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:57.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:57 smithi145 ceph-mon[35646]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:59.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:59 smithi145 ceph-mon[35646]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:59.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:59 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:59.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:59 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:59.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:31:59 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:59.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:59 smithi050 ceph-mon[35524]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:59.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:59 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:59.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:59 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:59.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:31:59 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:59.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:59 smithi033 ceph-mon[32390]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:31:59.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:59 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:59.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:59 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:31:59.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:31:59 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:01.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:01 smithi050 ceph-mon[35524]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:32:01.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:01 smithi033 ceph-mon[32390]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:32:01.629 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 3 on host 'smithi033' 2024-04-02T13:32:01.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:01.626+0000 7fd3b1ffb700 1 -- 172.21.15.33:0/1651661863 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd3b4062af0 con 0x7fd3a40a3bb0 2024-04-02T13:32:01.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:01.628+0000 7fd3bb241700 1 -- 172.21.15.33:0/1651661863 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd3a40a3bb0 msgr2=0x7fd3a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:01.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:01.628+0000 7fd3bb241700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd3a40a3bb0 0x7fd3a40a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8005950 tx=0x7fd3a800a300 comp rx=0 tx=0).stop 2024-04-02T13:32:01.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:01.628+0000 7fd3bb241700 1 -- 172.21.15.33:0/1651661863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b41010a0 msgr2=0x7fd3b41019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:01.628+0000 7fd3bb241700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b41010a0 0x7fd3b41019d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000d980 tx=0x7fd3a000dc90 comp rx=0 tx=0).stop 2024-04-02T13:32:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:01.628+0000 7fd3bb241700 1 -- 172.21.15.33:0/1651661863 shutdown_connections 2024-04-02T13:32:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:01.628+0000 7fd3bb241700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd3a40a3bb0 0x7fd3a40a6070 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:01.628+0000 7fd3bb241700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3b4103c80 0x7fd3b4101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:01.628+0000 7fd3bb241700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3b4100700 0x7fd3b4103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:01.628+0000 7fd3bb241700 1 --2- 172.21.15.33:0/1651661863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b41010a0 0x7fd3b41019d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:01.628+0000 7fd3bb241700 1 -- 172.21.15.33:0/1651661863 >> 172.21.15.33:0/1651661863 conn(0x7fd3b40fc280 msgr2=0x7fd3b4106c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:01.628+0000 7fd3bb241700 1 -- 172.21.15.33:0/1651661863 shutdown_connections 2024-04-02T13:32:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:01.628+0000 7fd3bb241700 1 -- 172.21.15.33:0/1651661863 wait complete. 2024-04-02T13:32:01.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:01 smithi145 ceph-mon[35646]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:32:02.275 DEBUG:teuthology.orchestra.run.smithi033:osd.3> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.3.service 2024-04-02T13:32:02.278 INFO:tasks.cephadm:Deploying osd.4 on smithi050 with /dev/vg_nvme/lv_3... 2024-04-02T13:32:02.279 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T13:32:02.531 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:32:02.878 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:02 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:02.878 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:02 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:02.878 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:02 smithi050 ceph-mon[35524]: from='osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T13:32:02.878 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:02 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:02.878 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:02 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:02.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:02 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:02.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:02 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:02.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:02 smithi145 ceph-mon[35646]: from='osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T13:32:02.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:02 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:02.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:02 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:02 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:02 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:02 smithi033 ceph-mon[32390]: from='osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T13:32:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:02 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:02 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:03.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:03 smithi033 ceph-mon[32390]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:32:03.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:03 smithi033 ceph-mon[32390]: from='osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T13:32:03.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:03 smithi033 ceph-mon[32390]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T13:32:03.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:03 smithi033 ceph-mon[32390]: from='osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-02T13:32:03.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:03 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:32:03.755 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:32:03 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3[51469]: 2024-04-02T13:32:03.638+0000 7fd30c242700 -1 osd.3 0 waiting for initial osdmap 2024-04-02T13:32:03.755 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:32:03 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3[51469]: 2024-04-02T13:32:03.644+0000 7fd308035700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:32:03.925 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:03 smithi050 ceph-mon[35524]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:32:03.925 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:03 smithi050 ceph-mon[35524]: from='osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T13:32:03.925 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:03 smithi050 ceph-mon[35524]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T13:32:03.925 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:03 smithi050 ceph-mon[35524]: from='osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-02T13:32:03.925 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:03 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:32:03.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:03 smithi145 ceph-mon[35646]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:32:03.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:03 smithi145 ceph-mon[35646]: from='osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T13:32:03.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:03 smithi145 ceph-mon[35646]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T13:32:03.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:03 smithi145 ceph-mon[35646]: from='osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-02T13:32:03.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:03 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:32:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T13:32:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T13:32:04.491 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-04-02T13:32:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-04-02T13:32:04.491 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124164 s, 845 MB/s 2024-04-02T13:32:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-04-02T13:32:04.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:04 smithi050 ceph-mon[35524]: from='osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-02T13:32:04.881 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:04 smithi050 ceph-mon[35524]: osdmap e25: 4 total, 3 up, 4 in 2024-04-02T13:32:04.881 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:32:04.881 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:32:04.881 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:04 smithi050 ceph-mon[35524]: osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141] boot 2024-04-02T13:32:04.881 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:04 smithi050 ceph-mon[35524]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T13:32:04.881 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:32:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:04 smithi145 ceph-mon[35646]: from='osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-02T13:32:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:04 smithi145 ceph-mon[35646]: osdmap e25: 4 total, 3 up, 4 in 2024-04-02T13:32:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:32:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:32:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:04 smithi145 ceph-mon[35646]: osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141] boot 2024-04-02T13:32:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:04 smithi145 ceph-mon[35646]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T13:32:04.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:32:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:04 smithi033 ceph-mon[32390]: from='osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-02T13:32:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:04 smithi033 ceph-mon[32390]: osdmap e25: 4 total, 3 up, 4 in 2024-04-02T13:32:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:32:05.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:32:05.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:04 smithi033 ceph-mon[32390]: osd.3 [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141] boot 2024-04-02T13:32:05.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:04 smithi033 ceph-mon[32390]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T13:32:05.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:32:05.207 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi050:vg_nvme/lv_3 2024-04-02T13:32:05.452 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:32:05.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:05 smithi050 ceph-mon[35524]: purged_snaps scrub starts 2024-04-02T13:32:05.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:05 smithi050 ceph-mon[35524]: purged_snaps scrub ok 2024-04-02T13:32:05.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:05 smithi050 ceph-mon[35524]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:32:05.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:05 smithi050 ceph-mon[35524]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T13:32:05.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:05 smithi033 ceph-mon[32390]: purged_snaps scrub starts 2024-04-02T13:32:05.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:05 smithi033 ceph-mon[32390]: purged_snaps scrub ok 2024-04-02T13:32:05.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:05 smithi033 ceph-mon[32390]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:32:05.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:05 smithi033 ceph-mon[32390]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T13:32:05.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:05 smithi145 ceph-mon[35646]: purged_snaps scrub starts 2024-04-02T13:32:05.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:05 smithi145 ceph-mon[35646]: purged_snaps scrub ok 2024-04-02T13:32:05.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:05 smithi145 ceph-mon[35646]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:32:05.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:05 smithi145 ceph-mon[35646]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T13:32:07.241 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:07.242 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: Detected new or changed devices on smithi033 2024-04-02T13:32:07.242 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.242 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.242 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:07.242 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:07.242 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi033 to 8546M 2024-04-02T13:32:07.242 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.242 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:07.242 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:07.242 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.242 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:07.242 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:07.243 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:07.243 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.243 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:06 smithi050 ceph-mon[35524]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T13:32:07.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.247+0000 7f3ea7a90700 1 -- 172.21.15.50:0/4112265778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea0101080 msgr2=0x7f3ea010ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:07.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.247+0000 7f3ea7a90700 1 --2- 172.21.15.50:0/4112265778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea0101080 0x7f3ea010ac50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009990 tx=0x7f3e98009ca0 comp rx=0 tx=0).stop 2024-04-02T13:32:07.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.247+0000 7f3ea7a90700 1 -- 172.21.15.50:0/4112265778 shutdown_connections 2024-04-02T13:32:07.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.247+0000 7f3ea7a90700 1 --2- 172.21.15.50:0/4112265778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea0101080 0x7f3ea010ac50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:07.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.247+0000 7f3ea7a90700 1 --2- 172.21.15.50:0/4112265778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ea0103c60 0x7f3ea0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:07.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.247+0000 7f3ea7a90700 1 --2- 172.21.15.50:0/4112265778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ea01006e0 0x7f3ea0100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:07.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.247+0000 7f3ea7a90700 1 -- 172.21.15.50:0/4112265778 >> 172.21.15.50:0/4112265778 conn(0x7f3ea00fc240 msgr2=0x7f3ea00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:07.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.247+0000 7f3ea7a90700 1 -- 172.21.15.50:0/4112265778 shutdown_connections 2024-04-02T13:32:07.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.247+0000 7f3ea7a90700 1 -- 172.21.15.50:0/4112265778 wait complete. 2024-04-02T13:32:07.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea7a90700 1 Processor -- start 2024-04-02T13:32:07.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea7a90700 1 -- start start 2024-04-02T13:32:07.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea7a90700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea01006e0 0x7f3ea0103360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea7a90700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ea0101080 0x7f3ea01019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea7a90700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ea0103c60 0x7f3ea0101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea7a90700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea010e260 con 0x7f3ea0103c60 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea7a90700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea010e0e0 con 0x7f3ea01006e0 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea7a90700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea010e3e0 con 0x7f3ea0101080 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea582c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea01006e0 0x7f3ea0103360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea582c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea01006e0 0x7f3ea0103360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42090/0 (socket says 172.21.15.50:42090) 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea582c700 1 -- 172.21.15.50:0/1972669501 learned_addr learned my addr 172.21.15.50:0/1972669501 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea602d700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ea0103c60 0x7f3ea0101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea502b700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ea0101080 0x7f3ea01019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea582c700 1 -- 172.21.15.50:0/1972669501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ea0101080 msgr2=0x7f3ea01019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea582c700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ea0101080 0x7f3ea01019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea582c700 1 -- 172.21.15.50:0/1972669501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ea0103c60 msgr2=0x7f3ea0101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:07.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea582c700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ea0103c60 0x7f3ea0101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:07.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea582c700 1 -- 172.21.15.50:0/1972669501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea01027a0 con 0x7f3ea01006e0 2024-04-02T13:32:07.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea602d700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ea0103c60 0x7f3ea0101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:07.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.248+0000 7f3ea582c700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea01006e0 0x7f3ea0103360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000ada0 tx=0x7f3e9000c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:07.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.249+0000 7f3e9effd700 1 -- 172.21.15.50:0/1972669501 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90017020 con 0x7f3ea01006e0 2024-04-02T13:32:07.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.249+0000 7f3e9effd700 1 -- 172.21.15.50:0/1972669501 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3e9000c980 con 0x7f3ea01006e0 2024-04-02T13:32:07.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.249+0000 7f3ea7a90700 1 -- 172.21.15.50:0/1972669501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea01a2a30 con 0x7f3ea01006e0 2024-04-02T13:32:07.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.249+0000 7f3e9effd700 1 -- 172.21.15.50:0/1972669501 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90004420 con 0x7f3ea01006e0 2024-04-02T13:32:07.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.249+0000 7f3ea7a90700 1 -- 172.21.15.50:0/1972669501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea01a2f40 con 0x7f3ea01006e0 2024-04-02T13:32:07.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:07.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: Detected new or changed devices on smithi033 2024-04-02T13:32:07.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:07.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:07.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi033 to 8546M 2024-04-02T13:32:07.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:07.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:07.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.256 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:07.256 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:07.256 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:07.256 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.256 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:06 smithi033 ceph-mon[32390]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T13:32:07.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.250+0000 7f3ea7a90700 1 -- 172.21.15.50:0/1972669501 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea002dd30 con 0x7f3ea01006e0 2024-04-02T13:32:07.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.251+0000 7f3e9effd700 1 -- 172.21.15.50:0/1972669501 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3e9000cb30 con 0x7f3ea01006e0 2024-04-02T13:32:07.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.251+0000 7f3e9effd700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3e880a3c40 0x7f3e880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:07.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.251+0000 7f3e9effd700 1 -- 172.21.15.50:0/1972669501 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7f3e900ce9b0 con 0x7f3ea01006e0 2024-04-02T13:32:07.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.251+0000 7f3ea502b700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3e880a3c40 0x7f3e880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:07.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.252+0000 7f3ea502b700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3e880a3c40 0x7f3e880a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3e94006fd0 tx=0x7f3e94008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:07.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.256+0000 7f3e9effd700 1 -- 172.21.15.50:0/1972669501 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3e90021020 con 0x7f3ea01006e0 2024-04-02T13:32:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:07.473+0000 7f3ea7a90700 1 -- 172.21.15.50:0/1972669501 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3ea0062a30 con 0x7f3e880a3c40 2024-04-02T13:32:07.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:07.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: Detected new or changed devices on smithi033 2024-04-02T13:32:07.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:07.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:07.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi033 to 8546M 2024-04-02T13:32:07.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:07.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:07.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:07.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:07.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:07.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:07.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:06 smithi145 ceph-mon[35646]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T13:32:08.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:07 smithi050 ceph-mon[35524]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:32:08.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:32:08.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:32:08.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:08.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:07 smithi050 ceph-mon[35524]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T13:32:08.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:07 smithi033 ceph-mon[32390]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:32:08.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:32:08.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:32:08.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:08.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:07 smithi033 ceph-mon[32390]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T13:32:08.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:07 smithi145 ceph-mon[35646]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:32:08.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:32:08.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:32:08.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:08.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:07 smithi145 ceph-mon[35646]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T13:32:09.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:09 smithi145 ceph-mon[35646]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:32:09.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:09 smithi145 ceph-mon[35646]: osdmap e30: 4 total, 4 up, 4 in 2024-04-02T13:32:09.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:09 smithi050 ceph-mon[35524]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:32:09.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:09 smithi050 ceph-mon[35524]: osdmap e30: 4 total, 4 up, 4 in 2024-04-02T13:32:10.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:09 smithi033 ceph-mon[32390]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:32:10.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:09 smithi033 ceph-mon[32390]: osdmap e30: 4 total, 4 up, 4 in 2024-04-02T13:32:10.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:10 smithi050 ceph-mon[35524]: from='client.? 172.21.15.50:0/1540909088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b02546aa-52cc-43d0-93c8-ba2ad32ceb0d"}]: dispatch 2024-04-02T13:32:10.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:10 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b02546aa-52cc-43d0-93c8-ba2ad32ceb0d"}]: dispatch 2024-04-02T13:32:10.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:10 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b02546aa-52cc-43d0-93c8-ba2ad32ceb0d"}]': finished 2024-04-02T13:32:10.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:10 smithi050 ceph-mon[35524]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T13:32:10.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:10 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:10.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:10 smithi145 ceph-mon[35646]: from='client.? 172.21.15.50:0/1540909088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b02546aa-52cc-43d0-93c8-ba2ad32ceb0d"}]: dispatch 2024-04-02T13:32:10.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:10 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b02546aa-52cc-43d0-93c8-ba2ad32ceb0d"}]: dispatch 2024-04-02T13:32:10.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:10 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b02546aa-52cc-43d0-93c8-ba2ad32ceb0d"}]': finished 2024-04-02T13:32:10.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:10 smithi145 ceph-mon[35646]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T13:32:10.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:10 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:11.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:10 smithi033 ceph-mon[32390]: from='client.? 172.21.15.50:0/1540909088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b02546aa-52cc-43d0-93c8-ba2ad32ceb0d"}]: dispatch 2024-04-02T13:32:11.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:10 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b02546aa-52cc-43d0-93c8-ba2ad32ceb0d"}]: dispatch 2024-04-02T13:32:11.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:10 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b02546aa-52cc-43d0-93c8-ba2ad32ceb0d"}]': finished 2024-04-02T13:32:11.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:10 smithi033 ceph-mon[32390]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T13:32:11.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:10 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:11.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:11 smithi145 ceph-mon[35646]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-02T13:32:11.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:11 smithi145 ceph-mon[35646]: from='client.? 172.21.15.50:0/1701206630' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:32:11.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:11 smithi050 ceph-mon[35524]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-02T13:32:11.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:11 smithi050 ceph-mon[35524]: from='client.? 172.21.15.50:0/1701206630' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:32:12.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:11 smithi033 ceph-mon[32390]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-02T13:32:12.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:11 smithi033 ceph-mon[32390]: from='client.? 172.21.15.50:0/1701206630' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:32:13.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:13 smithi145 ceph-mon[35646]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-02T13:32:13.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:13 smithi050 ceph-mon[35524]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-02T13:32:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:13 smithi033 ceph-mon[32390]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-02T13:32:15.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:15 smithi145 ceph-mon[35646]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:15.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:15 smithi050 ceph-mon[35524]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:16.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:15 smithi033 ceph-mon[32390]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:17 smithi145 ceph-mon[35646]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T13:32:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:17 smithi050 ceph-mon[35524]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T13:32:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:18.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:17 smithi033 ceph-mon[32390]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:18.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T13:32:18.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:18.964 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:18 smithi050 ceph-mon[35524]: Deploying daemon osd.4 on smithi050 2024-04-02T13:32:18.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:18 smithi145 ceph-mon[35646]: Deploying daemon osd.4 on smithi050 2024-04-02T13:32:19.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:18 smithi033 ceph-mon[32390]: Deploying daemon osd.4 on smithi050 2024-04-02T13:32:19.911 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:19 smithi050 ceph-mon[35524]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:19.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:19 smithi145 ceph-mon[35646]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:20.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:19 smithi033 ceph-mon[32390]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:21.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:21 smithi145 ceph-mon[35646]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:21.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:21 smithi050 ceph-mon[35524]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:22.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:21 smithi033 ceph-mon[32390]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:23.820 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:23 smithi050 ceph-mon[35524]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:23.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:23.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:23.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:23.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:23 smithi145 ceph-mon[35646]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:23.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:23.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:23.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:24.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:23 smithi033 ceph-mon[32390]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:24.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:24.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:24.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:25.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:25 smithi145 ceph-mon[35646]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:25.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:25 smithi050 ceph-mon[35524]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:26.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:25 smithi033 ceph-mon[32390]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:26.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:26.439+0000 7f3e9effd700 1 -- 172.21.15.50:0/1972669501 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3ea0062a30 con 0x7f3e880a3c40 2024-04-02T13:32:26.443 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 4 on host 'smithi050' 2024-04-02T13:32:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:26.442+0000 7f3ea7a90700 1 -- 172.21.15.50:0/1972669501 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3e880a3c40 msgr2=0x7f3e880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:26.442+0000 7f3ea7a90700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3e880a3c40 0x7f3e880a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3e94006fd0 tx=0x7f3e94008040 comp rx=0 tx=0).stop 2024-04-02T13:32:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:26.442+0000 7f3ea7a90700 1 -- 172.21.15.50:0/1972669501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea01006e0 msgr2=0x7f3ea0103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:26.442+0000 7f3ea7a90700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea01006e0 0x7f3ea0103360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000ada0 tx=0x7f3e9000c330 comp rx=0 tx=0).stop 2024-04-02T13:32:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:26.442+0000 7f3ea7a90700 1 -- 172.21.15.50:0/1972669501 shutdown_connections 2024-04-02T13:32:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:26.442+0000 7f3ea7a90700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3e880a3c40 0x7f3e880a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:26.442+0000 7f3ea7a90700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ea0103c60 0x7f3ea0101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:26.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:26.442+0000 7f3ea7a90700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ea01006e0 0x7f3ea0103360 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:26.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:26.442+0000 7f3ea7a90700 1 --2- 172.21.15.50:0/1972669501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ea0101080 0x7f3ea01019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:26.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:26.442+0000 7f3ea7a90700 1 -- 172.21.15.50:0/1972669501 >> 172.21.15.50:0/1972669501 conn(0x7f3ea00fc240 msgr2=0x7f3ea0106c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:26.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:26.442+0000 7f3ea7a90700 1 -- 172.21.15.50:0/1972669501 shutdown_connections 2024-04-02T13:32:26.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:32:26.442+0000 7f3ea7a90700 1 -- 172.21.15.50:0/1972669501 wait complete. 2024-04-02T13:32:27.059 DEBUG:teuthology.orchestra.run.smithi050:osd.4> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.4.service 2024-04-02T13:32:27.061 INFO:tasks.cephadm:Deploying osd.5 on smithi145 with /dev/vg_nvme/lv_3... 2024-04-02T13:32:27.061 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T13:32:27.319 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:32:27.712 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:27 smithi145 ceph-mon[35646]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:27.713 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:27 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:27.713 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:27 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:27.713 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:27 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:27.713 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:27 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:27.713 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:27 smithi145 ceph-mon[35646]: from='osd.4 [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:32:27.713 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:27 smithi145 ceph-mon[35646]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:32:27.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:27 smithi050 ceph-mon[35524]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:27.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:27 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:27.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:27 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:27.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:27 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:27.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:27 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:27.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:27 smithi050 ceph-mon[35524]: from='osd.4 [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:32:27.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:27 smithi050 ceph-mon[35524]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:32:27.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:27 smithi033 ceph-mon[32390]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:27.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:27 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:27.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:27 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:27.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:27 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:27.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:27 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:27.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:27 smithi033 ceph-mon[32390]: from='osd.4 [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:32:27.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:27 smithi033 ceph-mon[32390]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:32:28.801 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:28 smithi050 ceph-mon[35524]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T13:32:28.801 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:28 smithi050 ceph-mon[35524]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T13:32:28.801 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:28 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:28.801 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:28 smithi050 ceph-mon[35524]: from='osd.4 [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:32:28.801 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:28 smithi050 ceph-mon[35524]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:32:28.801 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:32:28 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4[46903]: 2024-04-02T13:32:28.702+0000 7f111f28e700 -1 osd.4 0 waiting for initial osdmap 2024-04-02T13:32:28.802 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:32:28 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4[46903]: 2024-04-02T13:32:28.709+0000 7f111b081700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:32:28.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:28 smithi145 ceph-mon[35646]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T13:32:28.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:28 smithi145 ceph-mon[35646]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T13:32:28.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:28 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:28.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:28 smithi145 ceph-mon[35646]: from='osd.4 [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:32:28.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:28 smithi145 ceph-mon[35646]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:32:29.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:28 smithi033 ceph-mon[32390]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T13:32:29.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:28 smithi033 ceph-mon[32390]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T13:32:29.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:29.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:28 smithi033 ceph-mon[32390]: from='osd.4 [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:32:29.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:28 smithi033 ceph-mon[32390]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:32:29.246 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T13:32:29.246 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T13:32:29.246 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2024-04-02T13:32:29.246 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2024-04-02T13:32:29.246 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121868 s, 860 MB/s 2024-04-02T13:32:29.246 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2024-04-02T13:32:29.907 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:29 smithi145 ceph-mon[35646]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:29.908 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:29 smithi145 ceph-mon[35646]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-04-02T13:32:29.908 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:29 smithi145 ceph-mon[35646]: osdmap e33: 5 total, 4 up, 5 in 2024-04-02T13:32:29.908 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:29 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:29.908 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:29 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:29.908 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:29 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:29.935 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi145:vg_nvme/lv_3 2024-04-02T13:32:29.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:29 smithi050 ceph-mon[35524]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:29.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:29 smithi050 ceph-mon[35524]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-04-02T13:32:29.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:29 smithi050 ceph-mon[35524]: osdmap e33: 5 total, 4 up, 5 in 2024-04-02T13:32:29.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:29 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:29.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:29 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:29.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:29 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:30.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:29 smithi033 ceph-mon[32390]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:30.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:29 smithi033 ceph-mon[32390]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-04-02T13:32:30.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:29 smithi033 ceph-mon[32390]: osdmap e33: 5 total, 4 up, 5 in 2024-04-02T13:32:30.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:30.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:30.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:30.169 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:32:30.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: purged_snaps scrub starts 2024-04-02T13:32:30.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: purged_snaps scrub ok 2024-04-02T13:32:30.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: osd.4 [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754] boot 2024-04-02T13:32:30.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T13:32:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:30.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:30.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: purged_snaps scrub starts 2024-04-02T13:32:30.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: purged_snaps scrub ok 2024-04-02T13:32:30.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: osd.4 [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754] boot 2024-04-02T13:32:30.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T13:32:30.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:30.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:30.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:30.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:30.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:30.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:30.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:30.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:30.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:30.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:30.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:30.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:30.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:31.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: purged_snaps scrub starts 2024-04-02T13:32:31.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: purged_snaps scrub ok 2024-04-02T13:32:31.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: osd.4 [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754] boot 2024-04-02T13:32:31.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T13:32:31.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:32:31.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:31.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:31.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:31.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:31.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:31.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:31.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:31.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:31.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:31.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:31.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:31.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:31.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.864+0000 7fb9f6a76700 1 -- 172.21.15.145:0/1417118737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f01041c0 msgr2=0x7fb9f010aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:31.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.864+0000 7fb9f6a76700 1 --2- 172.21.15.145:0/1417118737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f01041c0 0x7fb9f010aa70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800b0a0 tx=0x7fb9e800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:32:31.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 -- 172.21.15.145:0/1417118737 shutdown_connections 2024-04-02T13:32:31.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 --2- 172.21.15.145:0/1417118737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f01041c0 0x7fb9f010aa70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:31.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 --2- 172.21.15.145:0/1417118737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f0102620 0x7fb9f0102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:31.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 --2- 172.21.15.145:0/1417118737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9f0103820 0x7fb9f0103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:31.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 -- 172.21.15.145:0/1417118737 >> 172.21.15.145:0/1417118737 conn(0x7fb9f00fdc00 msgr2=0x7fb9f0100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 -- 172.21.15.145:0/1417118737 shutdown_connections 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 -- 172.21.15.145:0/1417118737 wait complete. 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 Processor -- start 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 -- start start 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f0102620 0x7fb9f0076390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0103820 0x7fb9f00749e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9f01041c0 0x7fb9f0074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f010e200 con 0x7fb9f0102620 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f010e080 con 0x7fb9f01041c0 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9f6a76700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f010e380 con 0x7fb9f0103820 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9effff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0103820 0x7fb9f00749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9effff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0103820 0x7fb9f00749e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40400/0 (socket says 172.21.15.145:40400) 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.865+0000 7fb9effff700 1 -- 172.21.15.145:0/3420270562 learned_addr learned my addr 172.21.15.145:0/3420270562 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-02T13:32:31.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9f4812700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f0102620 0x7fb9f0076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:31.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9f5013700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9f01041c0 0x7fb9f0074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:31.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9effff700 1 -- 172.21.15.145:0/3420270562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9f01041c0 msgr2=0x7fb9f0074f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:31.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9effff700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9f01041c0 0x7fb9f0074f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:31.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9effff700 1 -- 172.21.15.145:0/3420270562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f0102620 msgr2=0x7fb9f0076390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:31.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9effff700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f0102620 0x7fb9f0076390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:31.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9effff700 1 -- 172.21.15.145:0/3420270562 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f00757d0 con 0x7fb9f0103820 2024-04-02T13:32:31.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9f5013700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9f01041c0 0x7fb9f0074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:31.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9effff700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0103820 0x7fb9f00749e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00e9c0 tx=0x7fb9dc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:31.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9edffb700 1 -- 172.21.15.145:0/3420270562 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc00cbd0 con 0x7fb9f0103820 2024-04-02T13:32:31.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9edffb700 1 -- 172.21.15.145:0/3420270562 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb9dc00cd70 con 0x7fb9f0103820 2024-04-02T13:32:31.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9f6a76700 1 -- 172.21.15.145:0/3420270562 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f01a2950 con 0x7fb9f0103820 2024-04-02T13:32:31.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9edffb700 1 -- 172.21.15.145:0/3420270562 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc010430 con 0x7fb9f0103820 2024-04-02T13:32:31.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.866+0000 7fb9f6a76700 1 -- 172.21.15.145:0/3420270562 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f01a2e10 con 0x7fb9f0103820 2024-04-02T13:32:31.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.867+0000 7fb9f6a76700 1 -- 172.21.15.145:0/3420270562 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f00692b0 con 0x7fb9f0103820 2024-04-02T13:32:31.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.868+0000 7fb9edffb700 1 -- 172.21.15.145:0/3420270562 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb9dc004750 con 0x7fb9f0103820 2024-04-02T13:32:31.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.869+0000 7fb9edffb700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9e00a3c40 0x7fb9e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:31.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.869+0000 7fb9edffb700 1 -- 172.21.15.145:0/3420270562 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7fb9dc0cf680 con 0x7fb9f0103820 2024-04-02T13:32:31.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.869+0000 7fb9f4812700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9e00a3c40 0x7fb9e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:31.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.870+0000 7fb9f4812700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9e00a3c40 0x7fb9e00a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0103680 tx=0x7fb9e400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:31.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:31.874+0000 7fb9edffb700 1 -- 172.21.15.145:0/3420270562 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb9dc09d0d0 con 0x7fb9f0103820 2024-04-02T13:32:31.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:31 smithi145 ceph-mon[35646]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:31.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:31 smithi145 ceph-mon[35646]: Detected new or changed devices on smithi050 2024-04-02T13:32:31.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:31 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi050 to 8546M 2024-04-02T13:32:31.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:31 smithi145 ceph-mon[35646]: osdmap e35: 5 total, 5 up, 5 in 2024-04-02T13:32:31.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:31 smithi050 ceph-mon[35524]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:31.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:31 smithi050 ceph-mon[35524]: Detected new or changed devices on smithi050 2024-04-02T13:32:31.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:31 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi050 to 8546M 2024-04-02T13:32:31.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:31 smithi050 ceph-mon[35524]: osdmap e35: 5 total, 5 up, 5 in 2024-04-02T13:32:32.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:31 smithi033 ceph-mon[32390]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:32:32.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:31 smithi033 ceph-mon[32390]: Detected new or changed devices on smithi050 2024-04-02T13:32:32.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:31 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi050 to 8546M 2024-04-02T13:32:32.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:31 smithi033 ceph-mon[32390]: osdmap e35: 5 total, 5 up, 5 in 2024-04-02T13:32:32.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:32.081+0000 7fb9f6a76700 1 -- 172.21.15.145:0/3420270562 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb9f0062a30 con 0x7fb9e00a3c40 2024-04-02T13:32:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:32:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:32:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:32:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:32:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:33.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:32:33.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:32:33.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:33.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:33 smithi145 ceph-mon[35646]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:33.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:33 smithi145 ceph-mon[35646]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:32:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:33 smithi050 ceph-mon[35524]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:33 smithi050 ceph-mon[35524]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:32:34.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:33 smithi033 ceph-mon[32390]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:34.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:33 smithi033 ceph-mon[32390]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:32:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:35 smithi050 ceph-mon[35524]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:35 smithi050 ceph-mon[35524]: from='client.? 172.21.15.145:0/2259943544' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39e973fe-56fa-49d5-b155-17569a72a422"}]: dispatch 2024-04-02T13:32:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:35 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39e973fe-56fa-49d5-b155-17569a72a422"}]: dispatch 2024-04-02T13:32:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:35 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39e973fe-56fa-49d5-b155-17569a72a422"}]': finished 2024-04-02T13:32:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:35 smithi050 ceph-mon[35524]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T13:32:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:35 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:35 smithi050 ceph-mon[35524]: from='client.? 172.21.15.145:0/3138345487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:32:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:35 smithi033 ceph-mon[32390]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:35 smithi033 ceph-mon[32390]: from='client.? 172.21.15.145:0/2259943544' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39e973fe-56fa-49d5-b155-17569a72a422"}]: dispatch 2024-04-02T13:32:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:35 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39e973fe-56fa-49d5-b155-17569a72a422"}]: dispatch 2024-04-02T13:32:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:35 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39e973fe-56fa-49d5-b155-17569a72a422"}]': finished 2024-04-02T13:32:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:35 smithi033 ceph-mon[32390]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T13:32:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:35 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:35 smithi033 ceph-mon[32390]: from='client.? 172.21.15.145:0/3138345487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:32:36.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:35 smithi145 ceph-mon[35646]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:36.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:35 smithi145 ceph-mon[35646]: from='client.? 172.21.15.145:0/2259943544' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39e973fe-56fa-49d5-b155-17569a72a422"}]: dispatch 2024-04-02T13:32:36.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:35 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39e973fe-56fa-49d5-b155-17569a72a422"}]: dispatch 2024-04-02T13:32:36.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:35 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39e973fe-56fa-49d5-b155-17569a72a422"}]': finished 2024-04-02T13:32:36.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:35 smithi145 ceph-mon[35646]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T13:32:36.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:35 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:36.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:35 smithi145 ceph-mon[35646]: from='client.? 172.21.15.145:0/3138345487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:32:38.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:37 smithi033 ceph-mon[32390]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:38.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:37 smithi145 ceph-mon[35646]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:38.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:37 smithi050 ceph-mon[35524]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:40.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:39 smithi033 ceph-mon[32390]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:40.005 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:39 smithi145 ceph-mon[35646]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:40.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:39 smithi050 ceph-mon[35524]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:41.966 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:41 smithi145 ceph-mon[35646]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:42.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:41 smithi033 ceph-mon[32390]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:42.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:41 smithi050 ceph-mon[35524]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:43.022 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:42 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T13:32:43.022 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:42 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:43.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:42 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T13:32:43.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:42 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:43.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:42 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T13:32:43.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:42 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:44.097 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:43 smithi145 ceph-mon[35646]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:44.097 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:43 smithi145 ceph-mon[35646]: Deploying daemon osd.5 on smithi145 2024-04-02T13:32:44.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:43 smithi050 ceph-mon[35524]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:44.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:43 smithi050 ceph-mon[35524]: Deploying daemon osd.5 on smithi145 2024-04-02T13:32:44.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:43 smithi033 ceph-mon[32390]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:44.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:43 smithi033 ceph-mon[32390]: Deploying daemon osd.5 on smithi145 2024-04-02T13:32:45.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:45 smithi145 ceph-mon[35646]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:46.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:45 smithi050 ceph-mon[35524]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:46.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:45 smithi033 ceph-mon[32390]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:47.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:47 smithi145 ceph-mon[35646]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:48.057 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:47 smithi033 ceph-mon[32390]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:48.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:47 smithi050 ceph-mon[35524]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:49.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:48 smithi145 ceph-mon[35646]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:49.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:49.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:49.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:49.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:48 smithi050 ceph-mon[35524]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:49.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:49.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:49.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:49.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:48 smithi033 ceph-mon[32390]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:49.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:49.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:49.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:51.110 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:50 smithi145 ceph-mon[35646]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:51.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:50 smithi050 ceph-mon[35524]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:51.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:50 smithi033 ceph-mon[32390]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:51.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:51.805+0000 7fb9edffb700 1 -- 172.21.15.145:0/3420270562 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb9f0062a30 con 0x7fb9e00a3c40 2024-04-02T13:32:51.809 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 5 on host 'smithi145' 2024-04-02T13:32:51.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:51.807+0000 7fb9f6a76700 1 -- 172.21.15.145:0/3420270562 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9e00a3c40 msgr2=0x7fb9e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:51.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:51.807+0000 7fb9f6a76700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9e00a3c40 0x7fb9e00a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0103680 tx=0x7fb9e400b410 comp rx=0 tx=0).stop 2024-04-02T13:32:51.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:51.807+0000 7fb9f6a76700 1 -- 172.21.15.145:0/3420270562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0103820 msgr2=0x7fb9f00749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:51.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:51.807+0000 7fb9f6a76700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0103820 0x7fb9f00749e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00e9c0 tx=0x7fb9dc00ecd0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:51.807+0000 7fb9f6a76700 1 -- 172.21.15.145:0/3420270562 shutdown_connections 2024-04-02T13:32:51.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:51.807+0000 7fb9f6a76700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9e00a3c40 0x7fb9e00a6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:51.807+0000 7fb9f6a76700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9f01041c0 0x7fb9f0074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:51.807+0000 7fb9f6a76700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f0102620 0x7fb9f0076390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:51.807+0000 7fb9f6a76700 1 --2- 172.21.15.145:0/3420270562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9f0103820 0x7fb9f00749e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:51.807+0000 7fb9f6a76700 1 -- 172.21.15.145:0/3420270562 >> 172.21.15.145:0/3420270562 conn(0x7fb9f00fdc00 msgr2=0x7fb9f0106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:51.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:51.807+0000 7fb9f6a76700 1 -- 172.21.15.145:0/3420270562 shutdown_connections 2024-04-02T13:32:51.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:32:51.807+0000 7fb9f6a76700 1 -- 172.21.15.145:0/3420270562 wait complete. 2024-04-02T13:32:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:51 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:51 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:51 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:51 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:51 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:51 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:51 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:51 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:51 smithi145 ceph-mon[35646]: from='osd.5 [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:32:52.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:51 smithi145 ceph-mon[35646]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:32:52.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:51 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:51 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:51 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:51 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:51 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:51 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:51 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:51 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:51 smithi050 ceph-mon[35524]: from='osd.5 [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:32:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:51 smithi050 ceph-mon[35524]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:32:52.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:51 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:51 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:51 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:52.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:51 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:52.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:51 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:51 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:51 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:52.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:51 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:52.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:51 smithi033 ceph-mon[32390]: from='osd.5 [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:32:52.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:51 smithi033 ceph-mon[32390]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:32:52.455 DEBUG:teuthology.orchestra.run.smithi145:osd.5> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.5.service 2024-04-02T13:32:52.457 INFO:tasks.cephadm:Deploying osd.6 on smithi033 with /dev/vg_nvme/lv_2... 2024-04-02T13:32:52.457 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T13:32:52.711 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:32:53.917 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:53 smithi145 ceph-mon[35646]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:53.918 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:53 smithi145 ceph-mon[35646]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T13:32:53.918 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:53 smithi145 ceph-mon[35646]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T13:32:53.918 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:53 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:53.918 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:53 smithi145 ceph-mon[35646]: from='osd.5 [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:32:53.918 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:53 smithi145 ceph-mon[35646]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:32:53.918 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:32:53 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:32:53.778+0000 7f363baec700 -1 osd.5 0 waiting for initial osdmap 2024-04-02T13:32:53.918 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:32:53 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:32:53.784+0000 7f36378df700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:32:54.168 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:53 smithi033 ceph-mon[32390]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:54.168 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:53 smithi033 ceph-mon[32390]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T13:32:54.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:53 smithi033 ceph-mon[32390]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T13:32:54.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:54.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:53 smithi033 ceph-mon[32390]: from='osd.5 [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:32:54.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:53 smithi033 ceph-mon[32390]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:32:54.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:53 smithi050 ceph-mon[35524]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:54.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:53 smithi050 ceph-mon[35524]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T13:32:54.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:53 smithi050 ceph-mon[35524]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T13:32:54.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:53 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:54.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:53 smithi050 ceph-mon[35524]: from='osd.5 [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:32:54.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:53 smithi050 ceph-mon[35524]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:32:54.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:54 smithi145 ceph-mon[35646]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-04-02T13:32:54.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:54 smithi145 ceph-mon[35646]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T13:32:54.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:54 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:54.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:54 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:54.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:54 smithi145 ceph-mon[35646]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:54.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:54 smithi145 ceph-mon[35646]: osd.5 [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192] boot 2024-04-02T13:32:54.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:54 smithi145 ceph-mon[35646]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T13:32:54.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:54 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T13:32:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T13:32:55.065 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-02T13:32:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-02T13:32:55.065 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110064 s, 953 MB/s 2024-04-02T13:32:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-02T13:32:55.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:54 smithi033 ceph-mon[32390]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-04-02T13:32:55.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:54 smithi033 ceph-mon[32390]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T13:32:55.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:54 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:55.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:54 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:55.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:54 smithi033 ceph-mon[32390]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:55.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:54 smithi033 ceph-mon[32390]: osd.5 [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192] boot 2024-04-02T13:32:55.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:54 smithi033 ceph-mon[32390]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T13:32:55.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:54 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:55.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:54 smithi050 ceph-mon[35524]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-04-02T13:32:55.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:54 smithi050 ceph-mon[35524]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T13:32:55.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:54 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:55.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:54 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:55.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:54 smithi050 ceph-mon[35524]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:32:55.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:54 smithi050 ceph-mon[35524]: osd.5 [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192] boot 2024-04-02T13:32:55.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:54 smithi050 ceph-mon[35524]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T13:32:55.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:54 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:32:55.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi033:vg_nvme/lv_2 2024-04-02T13:32:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:32:56.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:55 smithi033 ceph-mon[32390]: purged_snaps scrub starts 2024-04-02T13:32:56.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:55 smithi033 ceph-mon[32390]: purged_snaps scrub ok 2024-04-02T13:32:56.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:55 smithi033 ceph-mon[32390]: Detected new or changed devices on smithi145 2024-04-02T13:32:56.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:55 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:56.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:55 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:56.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:55 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:56.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:55 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:56.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:55 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi145 to 8544M 2024-04-02T13:32:56.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:55 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:56.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:55 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:56.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:55 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:56.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:55 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:56.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:55 smithi033 ceph-mon[32390]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T13:32:56.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:55 smithi145 ceph-mon[35646]: purged_snaps scrub starts 2024-04-02T13:32:56.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:55 smithi145 ceph-mon[35646]: purged_snaps scrub ok 2024-04-02T13:32:56.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:55 smithi145 ceph-mon[35646]: Detected new or changed devices on smithi145 2024-04-02T13:32:56.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:55 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:56.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:55 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:56.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:55 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:56.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:55 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:56.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:55 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi145 to 8544M 2024-04-02T13:32:56.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:55 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:56.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:55 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:56.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:55 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:56.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:55 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:56.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:55 smithi145 ceph-mon[35646]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T13:32:56.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:55 smithi050 ceph-mon[35524]: purged_snaps scrub starts 2024-04-02T13:32:56.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:55 smithi050 ceph-mon[35524]: purged_snaps scrub ok 2024-04-02T13:32:56.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:55 smithi050 ceph-mon[35524]: Detected new or changed devices on smithi145 2024-04-02T13:32:56.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:55 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:56.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:55 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:56.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:55 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:56.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:55 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:56.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:55 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi145 to 8544M 2024-04-02T13:32:56.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:55 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:56.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:55 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:56.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:55 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:56.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:55 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:32:56.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:55 smithi050 ceph-mon[35524]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T13:32:57.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.625+0000 7ff183da3700 1 -- 172.21.15.33:0/3690431079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c103840 msgr2=0x7ff17c103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.625+0000 7ff183da3700 1 --2- 172.21.15.33:0/3690431079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c103840 0x7ff17c103ca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff1700099c0 tx=0x7ff170009cd0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.625+0000 7ff183da3700 1 -- 172.21.15.33:0/3690431079 shutdown_connections 2024-04-02T13:32:57.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.625+0000 7ff183da3700 1 --2- 172.21.15.33:0/3690431079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff17c1041e0 0x7ff17c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.625+0000 7ff183da3700 1 --2- 172.21.15.33:0/3690431079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff17c102640 0x7ff17c102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.625+0000 7ff183da3700 1 --2- 172.21.15.33:0/3690431079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c103840 0x7ff17c103ca0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.625+0000 7ff183da3700 1 -- 172.21.15.33:0/3690431079 >> 172.21.15.33:0/3690431079 conn(0x7ff17c0fdc00 msgr2=0x7ff17c100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.625+0000 7ff183da3700 1 -- 172.21.15.33:0/3690431079 shutdown_connections 2024-04-02T13:32:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.625+0000 7ff183da3700 1 -- 172.21.15.33:0/3690431079 wait complete. 2024-04-02T13:32:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff183da3700 1 Processor -- start 2024-04-02T13:32:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff183da3700 1 -- start start 2024-04-02T13:32:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff183da3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c102640 0x7ff17c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff183da3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff17c103840 0x7ff17c19afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff183da3700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff17c1041e0 0x7ff17c1a2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff183da3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c10e220 con 0x7ff17c102640 2024-04-02T13:32:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff183da3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c10e0a0 con 0x7ff17c103840 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff183da3700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c10e3a0 con 0x7ff17c1041e0 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff181b3f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c102640 0x7ff17c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff181b3f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c102640 0x7ff17c19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44404/0 (socket says 172.21.15.33:44404) 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff181b3f700 1 -- 172.21.15.33:0/2045966062 learned_addr learned my addr 172.21.15.33:0/2045966062 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff18133e700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff17c103840 0x7ff17c19afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff182340700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff17c1041e0 0x7ff17c1a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff181b3f700 1 -- 172.21.15.33:0/2045966062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff17c1041e0 msgr2=0x7ff17c1a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff181b3f700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff17c1041e0 0x7ff17c1a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff181b3f700 1 -- 172.21.15.33:0/2045966062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff17c103840 msgr2=0x7ff17c19afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff181b3f700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff17c103840 0x7ff17c19afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff181b3f700 1 -- 172.21.15.33:0/2045966062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c1a2700 con 0x7ff17c102640 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.626+0000 7ff182340700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff17c1041e0 0x7ff17c1a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.627+0000 7ff181b3f700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c102640 0x7ff17c19aaa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff174007780 tx=0x7ff174007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.627+0000 7ff17affd700 1 -- 172.21.15.33:0/2045966062 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174023070 con 0x7ff17c102640 2024-04-02T13:32:57.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.627+0000 7ff17affd700 1 -- 172.21.15.33:0/2045966062 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff174004030 con 0x7ff17c102640 2024-04-02T13:32:57.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.627+0000 7ff183da3700 1 -- 172.21.15.33:0/2045966062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c104ac0 con 0x7ff17c102640 2024-04-02T13:32:57.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.627+0000 7ff17affd700 1 -- 172.21.15.33:0/2045966062 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17401e6a0 con 0x7ff17c102640 2024-04-02T13:32:57.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.627+0000 7ff183da3700 1 -- 172.21.15.33:0/2045966062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c104fb0 con 0x7ff17c102640 2024-04-02T13:32:57.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.628+0000 7ff183da3700 1 -- 172.21.15.33:0/2045966062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff17c0692b0 con 0x7ff17c102640 2024-04-02T13:32:57.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.629+0000 7ff17affd700 1 -- 172.21.15.33:0/2045966062 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff174026070 con 0x7ff17c102640 2024-04-02T13:32:57.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.629+0000 7ff17affd700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff1640a3c40 0x7ff1640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.630+0000 7ff18133e700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff1640a3c40 0x7ff1640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.630+0000 7ff17affd700 1 -- 172.21.15.33:0/2045966062 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7ff1740d7100 con 0x7ff17c102640 2024-04-02T13:32:57.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.630+0000 7ff18133e700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff1640a3c40 0x7ff1640a6100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff170009fd0 tx=0x7ff170005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:57.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.634+0000 7ff17affd700 1 -- 172.21.15.33:0/2045966062 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff1740a4980 con 0x7ff17c102640 2024-04-02T13:32:57.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:32:57.855+0000 7ff183da3700 1 -- 172.21.15.33:0/2045966062 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff17c075f40 con 0x7ff1640a3c40 2024-04-02T13:32:58.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:57 smithi033 ceph-mon[32390]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:32:58.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:57 smithi033 ceph-mon[32390]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T13:32:58.047 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:57 smithi050 ceph-mon[35524]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:32:58.047 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:57 smithi050 ceph-mon[35524]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T13:32:58.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:57 smithi145 ceph-mon[35646]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:32:58.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:57 smithi145 ceph-mon[35646]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T13:32:59.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:58 smithi033 ceph-mon[32390]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T13:32:59.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:58 smithi033 ceph-mon[32390]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:32:59.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:58 smithi033 ceph-mon[32390]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:32:59.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:58 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:32:59.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:58 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:32:59.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:58 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:59.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:58 smithi145 ceph-mon[35646]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T13:32:59.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:58 smithi145 ceph-mon[35646]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:32:59.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:58 smithi145 ceph-mon[35646]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:32:59.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:58 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:32:59.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:58 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:32:59.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:58 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:59.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:58 smithi050 ceph-mon[35524]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T13:32:59.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:58 smithi050 ceph-mon[35524]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:32:59.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:58 smithi050 ceph-mon[35524]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:32:59.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:58 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:32:59.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:58 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:32:59.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:58 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:00.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:32:59 smithi145 ceph-mon[35646]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T13:33:00.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:32:59 smithi050 ceph-mon[35524]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T13:33:00.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:32:59 smithi033 ceph-mon[32390]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T13:33:01.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:00 smithi145 ceph-mon[35646]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-04-02T13:33:01.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:00 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1958110773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "523d9581-9067-4a63-8177-403ca4352df7"}]: dispatch 2024-04-02T13:33:01.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:00 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1958110773' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "523d9581-9067-4a63-8177-403ca4352df7"}]': finished 2024-04-02T13:33:01.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:00 smithi145 ceph-mon[35646]: osdmap e44: 7 total, 6 up, 7 in 2024-04-02T13:33:01.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:01.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:00 smithi050 ceph-mon[35524]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-04-02T13:33:01.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:00 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1958110773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "523d9581-9067-4a63-8177-403ca4352df7"}]: dispatch 2024-04-02T13:33:01.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:00 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1958110773' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "523d9581-9067-4a63-8177-403ca4352df7"}]': finished 2024-04-02T13:33:01.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:00 smithi050 ceph-mon[35524]: osdmap e44: 7 total, 6 up, 7 in 2024-04-02T13:33:01.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:00 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:01.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:00 smithi033 ceph-mon[32390]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-04-02T13:33:01.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:00 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1958110773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "523d9581-9067-4a63-8177-403ca4352df7"}]: dispatch 2024-04-02T13:33:01.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:00 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1958110773' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "523d9581-9067-4a63-8177-403ca4352df7"}]': finished 2024-04-02T13:33:01.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:00 smithi033 ceph-mon[32390]: osdmap e44: 7 total, 6 up, 7 in 2024-04-02T13:33:01.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:00 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:02.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:01 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2101915686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:33:02.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:01 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2101915686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:33:02.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:01 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2101915686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:33:03.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:02 smithi145 ceph-mon[35646]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-02T13:33:03.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:02 smithi050 ceph-mon[35524]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-02T13:33:03.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:02 smithi033 ceph-mon[32390]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-02T13:33:04.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:04 smithi033 ceph-mon[32390]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:05.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:04 smithi145 ceph-mon[35646]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:05.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:04 smithi050 ceph-mon[35524]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:06.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:06 smithi033 ceph-mon[32390]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:07.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:06 smithi145 ceph-mon[35646]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:07.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:06 smithi050 ceph-mon[35524]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:09.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:08 smithi033 ceph-mon[32390]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:09.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:08 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T13:33:09.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:08 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:09.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:08 smithi033 ceph-mon[32390]: Deploying daemon osd.6 on smithi033 2024-04-02T13:33:09.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:08 smithi145 ceph-mon[35646]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:09.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:08 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T13:33:09.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:08 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:09.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:08 smithi145 ceph-mon[35646]: Deploying daemon osd.6 on smithi033 2024-04-02T13:33:09.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:08 smithi050 ceph-mon[35524]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:09.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:08 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T13:33:09.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:08 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:09.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:08 smithi050 ceph-mon[35524]: Deploying daemon osd.6 on smithi033 2024-04-02T13:33:11.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:10 smithi033 ceph-mon[32390]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:11.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:10 smithi145 ceph-mon[35646]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:11.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:10 smithi050 ceph-mon[35524]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:13.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:12 smithi145 ceph-mon[35646]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:13.242 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:12 smithi033 ceph-mon[32390]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:13.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:12 smithi050 ceph-mon[35524]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:15.227 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:14 smithi033 ceph-mon[32390]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:15.228 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:15.228 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:15.228 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:15.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:14 smithi145 ceph-mon[35646]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:15.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:15.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:15.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:15.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:14 smithi050 ceph-mon[35524]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:15.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:14 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:15.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:14 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:15.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:14 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:17.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:16 smithi033 ceph-mon[32390]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:17.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:16 smithi145 ceph-mon[35646]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:17.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:16 smithi050 ceph-mon[35524]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:33:17.688+0000 7ff17affd700 1 -- 172.21.15.33:0/2045966062 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff17c075f40 con 0x7ff1640a3c40 2024-04-02T13:33:17.691 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 6 on host 'smithi033' 2024-04-02T13:33:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:33:17.690+0000 7ff183da3700 1 -- 172.21.15.33:0/2045966062 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff1640a3c40 msgr2=0x7ff1640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:33:17.690+0000 7ff183da3700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff1640a3c40 0x7ff1640a6100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff170009fd0 tx=0x7ff170005fb0 comp rx=0 tx=0).stop 2024-04-02T13:33:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:33:17.690+0000 7ff183da3700 1 -- 172.21.15.33:0/2045966062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c102640 msgr2=0x7ff17c19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:33:17.690+0000 7ff183da3700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c102640 0x7ff17c19aaa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff174007780 tx=0x7ff174007a90 comp rx=0 tx=0).stop 2024-04-02T13:33:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:33:17.690+0000 7ff183da3700 1 -- 172.21.15.33:0/2045966062 shutdown_connections 2024-04-02T13:33:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:33:17.690+0000 7ff183da3700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff1640a3c40 0x7ff1640a6100 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:33:17.690+0000 7ff183da3700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff17c1041e0 0x7ff17c1a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:33:17.690+0000 7ff183da3700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c102640 0x7ff17c19aaa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:33:17.690+0000 7ff183da3700 1 --2- 172.21.15.33:0/2045966062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff17c103840 0x7ff17c19afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:33:17.690+0000 7ff183da3700 1 -- 172.21.15.33:0/2045966062 >> 172.21.15.33:0/2045966062 conn(0x7ff17c0fdc00 msgr2=0x7ff17c074820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:33:17.690+0000 7ff183da3700 1 -- 172.21.15.33:0/2045966062 shutdown_connections 2024-04-02T13:33:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:33:17.690+0000 7ff183da3700 1 -- 172.21.15.33:0/2045966062 wait complete. 2024-04-02T13:33:18.447 DEBUG:teuthology.orchestra.run.smithi033:osd.6> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.6.service 2024-04-02T13:33:18.449 INFO:tasks.cephadm:Deploying osd.7 on smithi050 with /dev/vg_nvme/lv_2... 2024-04-02T13:33:18.449 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T13:33:18.703 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:33:18.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:18.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:18.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:18 smithi050 ceph-mon[35524]: from='osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T13:33:18.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:18.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:18.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:18.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:18.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:18 smithi033 ceph-mon[32390]: from='osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T13:33:18.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:18.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:18.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:18.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:18.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:18 smithi145 ceph-mon[35646]: from='osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T13:33:18.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:18.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:19.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:19 smithi033 ceph-mon[32390]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:19.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:19 smithi033 ceph-mon[32390]: from='osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T13:33:19.806 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:19 smithi033 ceph-mon[32390]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T13:33:19.806 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:19 smithi033 ceph-mon[32390]: from='osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-02T13:33:19.806 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:19 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:19.806 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:33:19 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:33:19.673+0000 7efed7691700 -1 osd.6 0 waiting for initial osdmap 2024-04-02T13:33:19.806 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:33:19 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:33:19.685+0000 7efed3484700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:33:19.920 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:19 smithi050 ceph-mon[35524]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:19.920 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:19 smithi050 ceph-mon[35524]: from='osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T13:33:19.920 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:19 smithi050 ceph-mon[35524]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T13:33:19.920 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:19 smithi050 ceph-mon[35524]: from='osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-02T13:33:19.920 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:19 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:19.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:19 smithi145 ceph-mon[35646]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:19.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:19 smithi145 ceph-mon[35646]: from='osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T13:33:19.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:19 smithi145 ceph-mon[35646]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T13:33:19.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:19 smithi145 ceph-mon[35646]: from='osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-02T13:33:19.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:19 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:20.783 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:20 smithi050 ceph-mon[35524]: from='osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-02T13:33:20.783 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:20 smithi050 ceph-mon[35524]: osdmap e46: 7 total, 6 up, 7 in 2024-04-02T13:33:20.783 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:20 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:20.783 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:20 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:20.783 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:20 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:20.783 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T13:33:20.783 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T13:33:20.784 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-04-02T13:33:20.784 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-04-02T13:33:20.784 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00836322 s, 1.3 GB/s 2024-04-02T13:33:20.784 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-04-02T13:33:20.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:20 smithi145 ceph-mon[35646]: from='osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-02T13:33:20.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:20 smithi145 ceph-mon[35646]: osdmap e46: 7 total, 6 up, 7 in 2024-04-02T13:33:20.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:20 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:20.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:20 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:20.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:20 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:21.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:20 smithi033 ceph-mon[32390]: from='osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-02T13:33:21.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:20 smithi033 ceph-mon[32390]: osdmap e46: 7 total, 6 up, 7 in 2024-04-02T13:33:21.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:20 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:21.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:20 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:21.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:20 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:21.456 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi050:vg_nvme/lv_2 2024-04-02T13:33:21.703 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:33:21.945 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:21 smithi050 ceph-mon[35524]: purged_snaps scrub starts 2024-04-02T13:33:21.946 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:21 smithi050 ceph-mon[35524]: purged_snaps scrub ok 2024-04-02T13:33:21.946 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:21 smithi050 ceph-mon[35524]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:21.946 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:21 smithi050 ceph-mon[35524]: osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637] boot 2024-04-02T13:33:21.946 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:21 smithi050 ceph-mon[35524]: osdmap e47: 7 total, 7 up, 7 in 2024-04-02T13:33:21.946 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:21 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:21.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:21 smithi145 ceph-mon[35646]: purged_snaps scrub starts 2024-04-02T13:33:21.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:21 smithi145 ceph-mon[35646]: purged_snaps scrub ok 2024-04-02T13:33:21.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:21 smithi145 ceph-mon[35646]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:21.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:21 smithi145 ceph-mon[35646]: osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637] boot 2024-04-02T13:33:21.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:21 smithi145 ceph-mon[35646]: osdmap e47: 7 total, 7 up, 7 in 2024-04-02T13:33:21.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:21 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:22.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:21 smithi033 ceph-mon[32390]: purged_snaps scrub starts 2024-04-02T13:33:22.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:21 smithi033 ceph-mon[32390]: purged_snaps scrub ok 2024-04-02T13:33:22.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:21 smithi033 ceph-mon[32390]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:33:22.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:21 smithi033 ceph-mon[32390]: osd.6 [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637] boot 2024-04-02T13:33:22.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:21 smithi033 ceph-mon[32390]: osdmap e47: 7 total, 7 up, 7 in 2024-04-02T13:33:22.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:21 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:33:22.860 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: osdmap e48: 7 total, 7 up, 7 in 2024-04-02T13:33:22.860 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:22.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:22.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:22.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:22.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:22.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:22.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:22.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:33:22.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:22.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:22.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:22.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:33:22.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:22 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:22.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: osdmap e48: 7 total, 7 up, 7 in 2024-04-02T13:33:22.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:22.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:22.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:22.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:22.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:22.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:22.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:22.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:33:22.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:22.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:22.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:22.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:33:22.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:22 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: osdmap e48: 7 total, 7 up, 7 in 2024-04-02T13:33:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:33:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:23.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:23.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:33:23.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:22 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:23.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.391+0000 7f1c2204d700 1 -- 172.21.15.50:0/2235692155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c108370 msgr2=0x7f1c1c10a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:23.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.391+0000 7f1c2204d700 1 --2- 172.21.15.50:0/2235692155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c108370 0x7f1c1c10a780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400b0a0 tx=0x7f1c1400b3b0 comp rx=0 tx=0).stop 2024-04-02T13:33:23.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.391+0000 7f1c2204d700 1 -- 172.21.15.50:0/2235692155 shutdown_connections 2024-04-02T13:33:23.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.391+0000 7f1c2204d700 1 --2- 172.21.15.50:0/2235692155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c108370 0x7f1c1c10a780 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:23.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.391+0000 7f1c2204d700 1 --2- 172.21.15.50:0/2235692155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c1c103110 0x7f1c1c105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:23.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.391+0000 7f1c2204d700 1 --2- 172.21.15.50:0/2235692155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c105a40 0x7f1c1c107e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.391+0000 7f1c2204d700 1 -- 172.21.15.50:0/2235692155 >> 172.21.15.50:0/2235692155 conn(0x7f1c1c0fcac0 msgr2=0x7f1c1c0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.391+0000 7f1c2204d700 1 -- 172.21.15.50:0/2235692155 shutdown_connections 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.391+0000 7f1c2204d700 1 -- 172.21.15.50:0/2235692155 wait complete. 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.392+0000 7f1c2204d700 1 Processor -- start 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.392+0000 7f1c2204d700 1 -- start start 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.392+0000 7f1c2204d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c1c103110 0x7f1c1c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.392+0000 7f1c2204d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c105a40 0x7f1c1c198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.392+0000 7f1c2204d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c108370 0x7f1c1c19fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.392+0000 7f1c2204d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10e290 con 0x7f1c1c105a40 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.392+0000 7f1c2204d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10e110 con 0x7f1c1c108370 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.392+0000 7f1c2204d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10e410 con 0x7f1c1c103110 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.392+0000 7f1c1bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c108370 0x7f1c1c19fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.392+0000 7f1c1bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c108370 0x7f1c1c19fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50702/0 (socket says 172.21.15.50:50702) 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.392+0000 7f1c1bfff700 1 -- 172.21.15.50:0/1280914996 learned_addr learned my addr 172.21.15.50:0/1280914996 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T13:33:23.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.392+0000 7f1c1b7fe700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c1c103110 0x7f1c1c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.392+0000 7f1c1affd700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c105a40 0x7f1c1c198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.393+0000 7f1c1bfff700 1 -- 172.21.15.50:0/1280914996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c1c103110 msgr2=0x7f1c1c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.393+0000 7f1c1bfff700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c1c103110 0x7f1c1c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.393+0000 7f1c1bfff700 1 -- 172.21.15.50:0/1280914996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c105a40 msgr2=0x7f1c1c198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.393+0000 7f1c1bfff700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c105a40 0x7f1c1c198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.393+0000 7f1c1bfff700 1 -- 172.21.15.50:0/1280914996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c1a0520 con 0x7f1c1c108370 2024-04-02T13:33:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.393+0000 7f1c1affd700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c105a40 0x7f1c1c198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.393+0000 7f1c1bfff700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c108370 0x7f1c1c19fe80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009cd0 tx=0x7f1c14007d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.393+0000 7f1c18ff9700 1 -- 172.21.15.50:0/1280914996 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14003e10 con 0x7f1c1c108370 2024-04-02T13:33:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.393+0000 7f1c18ff9700 1 -- 172.21.15.50:0/1280914996 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1c1401d940 con 0x7f1c1c108370 2024-04-02T13:33:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.393+0000 7f1c18ff9700 1 -- 172.21.15.50:0/1280914996 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14005720 con 0x7f1c1c108370 2024-04-02T13:33:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.393+0000 7f1c2204d700 1 -- 172.21.15.50:0/1280914996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c1a07b0 con 0x7f1c1c108370 2024-04-02T13:33:23.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.393+0000 7f1c2204d700 1 -- 172.21.15.50:0/1280914996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c1a0c00 con 0x7f1c1c108370 2024-04-02T13:33:23.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.394+0000 7f1c2204d700 1 -- 172.21.15.50:0/1280914996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1c0692b0 con 0x7f1c1c108370 2024-04-02T13:33:23.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.395+0000 7f1c18ff9700 1 -- 172.21.15.50:0/1280914996 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1c140058c0 con 0x7f1c1c108370 2024-04-02T13:33:23.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.396+0000 7f1c18ff9700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1c040a3cc0 0x7f1c040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:23.399 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.396+0000 7f1c18ff9700 1 -- 172.21.15.50:0/1280914996 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f1c140d0b00 con 0x7f1c1c108370 2024-04-02T13:33:23.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.396+0000 7f1c1b7fe700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1c040a3cc0 0x7f1c040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:23.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.396+0000 7f1c1b7fe700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1c040a3cc0 0x7f1c040a6180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c006fd0 tx=0x7f1c0c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:23.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.400+0000 7f1c18ff9700 1 -- 172.21.15.50:0/1280914996 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1c1409e280 con 0x7f1c1c108370 2024-04-02T13:33:23.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:23.617+0000 7f1c2204d700 1 -- 172.21.15.50:0/1280914996 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1c1c062a30 con 0x7f1c040a3cc0 2024-04-02T13:33:23.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:23 smithi145 ceph-mon[35646]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:23.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:23 smithi145 ceph-mon[35646]: Detected new or changed devices on smithi033 2024-04-02T13:33:23.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:23 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi033 to 5697M 2024-04-02T13:33:23.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:33:23.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:33:23.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:23 smithi050 ceph-mon[35524]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:23 smithi050 ceph-mon[35524]: Detected new or changed devices on smithi033 2024-04-02T13:33:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:23 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi033 to 5697M 2024-04-02T13:33:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:33:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:33:23.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:24.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:23 smithi033 ceph-mon[32390]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:24.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:23 smithi033 ceph-mon[32390]: Detected new or changed devices on smithi033 2024-04-02T13:33:24.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:23 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi033 to 5697M 2024-04-02T13:33:24.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:33:24.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:33:24.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:24.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:24 smithi145 ceph-mon[35646]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:24.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:24 smithi050 ceph-mon[35524]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:25.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:24 smithi033 ceph-mon[32390]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:25.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:25 smithi050 ceph-mon[35524]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:25.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:25 smithi145 ceph-mon[35646]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:26.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:25 smithi033 ceph-mon[32390]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:26.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:26 smithi145 ceph-mon[35646]: from='client.? 172.21.15.50:0/3112224683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53f96776-fc11-4b58-b6c7-dce5e873948e"}]: dispatch 2024-04-02T13:33:26.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:26 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53f96776-fc11-4b58-b6c7-dce5e873948e"}]: dispatch 2024-04-02T13:33:26.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:26 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53f96776-fc11-4b58-b6c7-dce5e873948e"}]': finished 2024-04-02T13:33:26.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:26 smithi145 ceph-mon[35646]: osdmap e49: 8 total, 7 up, 8 in 2024-04-02T13:33:26.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:26 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:26.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:26 smithi050 ceph-mon[35524]: from='client.? 172.21.15.50:0/3112224683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53f96776-fc11-4b58-b6c7-dce5e873948e"}]: dispatch 2024-04-02T13:33:26.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:26 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53f96776-fc11-4b58-b6c7-dce5e873948e"}]: dispatch 2024-04-02T13:33:26.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:26 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53f96776-fc11-4b58-b6c7-dce5e873948e"}]': finished 2024-04-02T13:33:26.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:26 smithi050 ceph-mon[35524]: osdmap e49: 8 total, 7 up, 8 in 2024-04-02T13:33:26.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:26 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:27.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:26 smithi033 ceph-mon[32390]: from='client.? 172.21.15.50:0/3112224683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53f96776-fc11-4b58-b6c7-dce5e873948e"}]: dispatch 2024-04-02T13:33:27.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:26 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53f96776-fc11-4b58-b6c7-dce5e873948e"}]: dispatch 2024-04-02T13:33:27.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:26 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53f96776-fc11-4b58-b6c7-dce5e873948e"}]': finished 2024-04-02T13:33:27.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:26 smithi033 ceph-mon[32390]: osdmap e49: 8 total, 7 up, 8 in 2024-04-02T13:33:27.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:26 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:27.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:27 smithi145 ceph-mon[35646]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:27.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:27 smithi145 ceph-mon[35646]: from='client.? 172.21.15.50:0/2010039037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:33:27.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:27 smithi050 ceph-mon[35524]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:27.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:27 smithi050 ceph-mon[35524]: from='client.? 172.21.15.50:0/2010039037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:33:28.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:27 smithi033 ceph-mon[32390]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:28.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:27 smithi033 ceph-mon[32390]: from='client.? 172.21.15.50:0/2010039037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:33:29.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:29 smithi145 ceph-mon[35646]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:29.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:29 smithi050 ceph-mon[35524]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:30.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:29 smithi033 ceph-mon[32390]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:31.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:31 smithi145 ceph-mon[35646]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:31.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:31 smithi050 ceph-mon[35524]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:32.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:31 smithi033 ceph-mon[32390]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:33.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:33 smithi145 ceph-mon[35646]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:33 smithi050 ceph-mon[35524]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:34.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:33 smithi033 ceph-mon[32390]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:34.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:34 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T13:33:34.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:34 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:34 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T13:33:34.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:34 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:35.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:34 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T13:33:35.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:34 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:35 smithi145 ceph-mon[35646]: Deploying daemon osd.7 on smithi050 2024-04-02T13:33:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:35 smithi145 ceph-mon[35646]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:35 smithi050 ceph-mon[35524]: Deploying daemon osd.7 on smithi050 2024-04-02T13:33:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:35 smithi050 ceph-mon[35524]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:35 smithi033 ceph-mon[32390]: Deploying daemon osd.7 on smithi050 2024-04-02T13:33:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:35 smithi033 ceph-mon[32390]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:37 smithi145 ceph-mon[35646]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:37.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:37 smithi050 ceph-mon[35524]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:38.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:37 smithi033 ceph-mon[32390]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:39.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:39 smithi050 ceph-mon[35524]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:39.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:39 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:39.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:39 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:39.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:39 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:40.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:39 smithi033 ceph-mon[32390]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:40.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:39 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:40.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:39 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:40.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:39 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:40.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:39 smithi145 ceph-mon[35646]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:40.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:39 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:40.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:39 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:40.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:39 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:41.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:41 smithi050 ceph-mon[35524]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:42.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:41 smithi033 ceph-mon[32390]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:42.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:41 smithi145 ceph-mon[35646]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:42.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:42.935+0000 7f1c18ff9700 1 -- 172.21.15.50:0/1280914996 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1c1c062a30 con 0x7f1c040a3cc0 2024-04-02T13:33:42.939 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 7 on host 'smithi050' 2024-04-02T13:33:42.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:42.937+0000 7f1c2204d700 1 -- 172.21.15.50:0/1280914996 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1c040a3cc0 msgr2=0x7f1c040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:42.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:42.937+0000 7f1c2204d700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1c040a3cc0 0x7f1c040a6180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c006fd0 tx=0x7f1c0c008040 comp rx=0 tx=0).stop 2024-04-02T13:33:42.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:42.937+0000 7f1c2204d700 1 -- 172.21.15.50:0/1280914996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c108370 msgr2=0x7f1c1c19fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:42.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:42.937+0000 7f1c2204d700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c108370 0x7f1c1c19fe80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009cd0 tx=0x7f1c14007d10 comp rx=0 tx=0).stop 2024-04-02T13:33:42.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:42.937+0000 7f1c2204d700 1 -- 172.21.15.50:0/1280914996 shutdown_connections 2024-04-02T13:33:42.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:42.937+0000 7f1c2204d700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1c040a3cc0 0x7f1c040a6180 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:42.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:42.937+0000 7f1c2204d700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c1c108370 0x7f1c1c19fe80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:42.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:42.937+0000 7f1c2204d700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1c1c103110 0x7f1c1c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:42.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:42.937+0000 7f1c2204d700 1 --2- 172.21.15.50:0/1280914996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c105a40 0x7f1c1c198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:42.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:42.937+0000 7f1c2204d700 1 -- 172.21.15.50:0/1280914996 >> 172.21.15.50:0/1280914996 conn(0x7f1c1c0fcac0 msgr2=0x7f1c1c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:42.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:42.938+0000 7f1c2204d700 1 -- 172.21.15.50:0/1280914996 shutdown_connections 2024-04-02T13:33:42.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:33:42.938+0000 7f1c2204d700 1 -- 172.21.15.50:0/1280914996 wait complete. 2024-04-02T13:33:43.583 DEBUG:teuthology.orchestra.run.smithi050:osd.7> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.7.service 2024-04-02T13:33:43.586 INFO:tasks.cephadm:Deploying osd.8 on smithi145 with /dev/vg_nvme/lv_2... 2024-04-02T13:33:43.587 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T13:33:43.836 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:33:43.958 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:43 smithi145 ceph-mon[35646]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:43.958 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:43 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:43.958 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:43 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:43.958 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:43 smithi145 ceph-mon[35646]: from='osd.7 [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:33:43.958 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:43 smithi145 ceph-mon[35646]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:33:43.958 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:43 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:43.958 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:43 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:43.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:43 smithi050 ceph-mon[35524]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:43.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:43 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:43.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:43 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:43.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:43 smithi050 ceph-mon[35524]: from='osd.7 [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:33:43.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:43 smithi050 ceph-mon[35524]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:33:43.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:43 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:43.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:43 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:44.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:43 smithi033 ceph-mon[32390]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:44.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:43 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:44.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:43 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:44.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:43 smithi033 ceph-mon[32390]: from='osd.7 [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:33:44.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:43 smithi033 ceph-mon[32390]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:33:44.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:43 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:44.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:43 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:44 smithi145 ceph-mon[35646]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:44 smithi145 ceph-mon[35646]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T13:33:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:44 smithi145 ceph-mon[35646]: from='osd.7 [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:33:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:44 smithi145 ceph-mon[35646]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T13:33:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:44 smithi145 ceph-mon[35646]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:33:45.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:44 smithi050 ceph-mon[35524]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:45.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:44 smithi050 ceph-mon[35524]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T13:33:45.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:44 smithi050 ceph-mon[35524]: from='osd.7 [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:33:45.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:44 smithi050 ceph-mon[35524]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T13:33:45.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:45.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:44 smithi050 ceph-mon[35524]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:33:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:44 smithi033 ceph-mon[32390]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:44 smithi033 ceph-mon[32390]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T13:33:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:44 smithi033 ceph-mon[32390]: from='osd.7 [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:33:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:44 smithi033 ceph-mon[32390]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T13:33:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:44 smithi033 ceph-mon[32390]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:33:45.619 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:33:45 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:33:45.412+0000 7f15f004a700 -1 osd.7 0 waiting for initial osdmap 2024-04-02T13:33:45.619 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:33:45 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:33:45.423+0000 7f15e7e35700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:33:46.044 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T13:33:46.045 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T13:33:46.045 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2024-04-02T13:33:46.045 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2024-04-02T13:33:46.045 INFO:teuthology.orchestra.run.smithi145.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128486 s, 816 MB/s 2024-04-02T13:33:46.045 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2024-04-02T13:33:46.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:45 smithi145 ceph-mon[35646]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-04-02T13:33:46.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:45 smithi145 ceph-mon[35646]: osdmap e51: 8 total, 7 up, 8 in 2024-04-02T13:33:46.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:45 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:46.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:45 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:46.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:45 smithi145 ceph-mon[35646]: from='osd.7 ' entity='osd.7' 2024-04-02T13:33:46.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:45 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:46.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:45 smithi050 ceph-mon[35524]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-04-02T13:33:46.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:45 smithi050 ceph-mon[35524]: osdmap e51: 8 total, 7 up, 8 in 2024-04-02T13:33:46.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:45 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:46.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:45 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:46.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:45 smithi050 ceph-mon[35524]: from='osd.7 ' entity='osd.7' 2024-04-02T13:33:46.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:45 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:46.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:45 smithi033 ceph-mon[32390]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-04-02T13:33:46.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:45 smithi033 ceph-mon[32390]: osdmap e51: 8 total, 7 up, 8 in 2024-04-02T13:33:46.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:45 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:46.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:45 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:46.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:45 smithi033 ceph-mon[32390]: from='osd.7 ' entity='osd.7' 2024-04-02T13:33:46.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:45 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:46.762 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi145:vg_nvme/lv_2 2024-04-02T13:33:47.007 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:33:47.136 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:46 smithi145 ceph-mon[35646]: purged_snaps scrub starts 2024-04-02T13:33:47.136 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:46 smithi145 ceph-mon[35646]: purged_snaps scrub ok 2024-04-02T13:33:47.136 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:46 smithi145 ceph-mon[35646]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:47.136 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:46 smithi145 ceph-mon[35646]: osd.7 [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676] boot 2024-04-02T13:33:47.136 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:46 smithi145 ceph-mon[35646]: osdmap e52: 8 total, 8 up, 8 in 2024-04-02T13:33:47.136 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:46 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:47.136 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:46 smithi145 ceph-mon[35646]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T13:33:47.159 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:46 smithi050 ceph-mon[35524]: purged_snaps scrub starts 2024-04-02T13:33:47.159 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:46 smithi050 ceph-mon[35524]: purged_snaps scrub ok 2024-04-02T13:33:47.159 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:46 smithi050 ceph-mon[35524]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:47.159 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:46 smithi050 ceph-mon[35524]: osd.7 [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676] boot 2024-04-02T13:33:47.159 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:46 smithi050 ceph-mon[35524]: osdmap e52: 8 total, 8 up, 8 in 2024-04-02T13:33:47.159 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:46 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:47.159 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:46 smithi050 ceph-mon[35524]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T13:33:47.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:46 smithi033 ceph-mon[32390]: purged_snaps scrub starts 2024-04-02T13:33:47.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:46 smithi033 ceph-mon[32390]: purged_snaps scrub ok 2024-04-02T13:33:47.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:46 smithi033 ceph-mon[32390]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:33:47.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:46 smithi033 ceph-mon[32390]: osd.7 [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676] boot 2024-04-02T13:33:47.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:46 smithi033 ceph-mon[32390]: osdmap e52: 8 total, 8 up, 8 in 2024-04-02T13:33:47.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:46 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:33:47.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:46 smithi033 ceph-mon[32390]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T13:33:48.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: Detected new or changed devices on smithi050 2024-04-02T13:33:48.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:48.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:48.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:48.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi050 to 5697M 2024-04-02T13:33:48.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:48.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:33:48.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:48.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:48.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:33:48.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:48 smithi145 ceph-mon[35646]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T13:33:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: Detected new or changed devices on smithi050 2024-04-02T13:33:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi050 to 5697M 2024-04-02T13:33:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:33:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:48.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:48.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:33:48.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:48 smithi050 ceph-mon[35524]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T13:33:48.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: Detected new or changed devices on smithi050 2024-04-02T13:33:48.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:48.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:48.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:33:48.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi050 to 5697M 2024-04-02T13:33:48.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:48.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:33:48.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:48.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:48.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:33:48.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:33:48.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:48 smithi033 ceph-mon[32390]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T13:33:48.717 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.712+0000 7fdde76c8700 1 -- 172.21.15.145:0/4120027079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdde0102640 msgr2=0x7fdde0102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:48.717 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.712+0000 7fdde76c8700 1 --2- 172.21.15.145:0/4120027079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdde0102640 0x7fdde0102a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdddc009990 tx=0x7fdddc009ca0 comp rx=0 tx=0).stop 2024-04-02T13:33:48.717 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.712+0000 7fdde76c8700 1 -- 172.21.15.145:0/4120027079 shutdown_connections 2024-04-02T13:33:48.717 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.712+0000 7fdde76c8700 1 --2- 172.21.15.145:0/4120027079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdde01041e0 0x7fdde010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.712+0000 7fdde76c8700 1 --2- 172.21.15.145:0/4120027079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdde0102640 0x7fdde0102a60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.712+0000 7fdde76c8700 1 --2- 172.21.15.145:0/4120027079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde0103840 0x7fdde0103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.712+0000 7fdde76c8700 1 -- 172.21.15.145:0/4120027079 >> 172.21.15.145:0/4120027079 conn(0x7fdde00fdc00 msgr2=0x7fdde0100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.712+0000 7fdde76c8700 1 -- 172.21.15.145:0/4120027079 shutdown_connections 2024-04-02T13:33:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.712+0000 7fdde76c8700 1 -- 172.21.15.145:0/4120027079 wait complete. 2024-04-02T13:33:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.712+0000 7fdde76c8700 1 Processor -- start 2024-04-02T13:33:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.712+0000 7fdde76c8700 1 -- start start 2024-04-02T13:33:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde76c8700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdde0102640 0x7fdde0076390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde76c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde0103840 0x7fdde00749e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde76c8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdde01041e0 0x7fdde0074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde76c8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdde010e210 con 0x7fdde0103840 2024-04-02T13:33:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde76c8700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdde010e090 con 0x7fdde01041e0 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde76c8700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdde010e390 con 0x7fdde0102640 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde5464700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdde0102640 0x7fdde0076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde5464700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdde0102640 0x7fdde0076390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45866/0 (socket says 172.21.15.145:45866) 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde5464700 1 -- 172.21.15.145:0/2557265117 learned_addr learned my addr 172.21.15.145:0/2557265117 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde4c63700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde0103840 0x7fdde00749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde5c65700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdde01041e0 0x7fdde0074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde5464700 1 -- 172.21.15.145:0/2557265117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdde01041e0 msgr2=0x7fdde0074f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde5464700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdde01041e0 0x7fdde0074f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde5464700 1 -- 172.21.15.145:0/2557265117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde0103840 msgr2=0x7fdde00749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde5464700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde0103840 0x7fdde00749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde5464700 1 -- 172.21.15.145:0/2557265117 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde00757d0 con 0x7fdde0102640 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fdde5464700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdde0102640 0x7fdde0076390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0098e0 tx=0x7fdddc0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.713+0000 7fddd67fc700 1 -- 172.21.15.145:0/2557265117 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc027070 con 0x7fdde0102640 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.714+0000 7fddd67fc700 1 -- 172.21.15.145:0/2557265117 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdddc0041b0 con 0x7fdde0102640 2024-04-02T13:33:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.714+0000 7fdde76c8700 1 -- 172.21.15.145:0/2557265117 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde01a2980 con 0x7fdde0102640 2024-04-02T13:33:48.720 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.714+0000 7fddd67fc700 1 -- 172.21.15.145:0/2557265117 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc022e60 con 0x7fdde0102640 2024-04-02T13:33:48.720 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.714+0000 7fdde76c8700 1 -- 172.21.15.145:0/2557265117 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde01a2dd0 con 0x7fdde0102640 2024-04-02T13:33:48.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.715+0000 7fdde76c8700 1 -- 172.21.15.145:0/2557265117 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde00692b0 con 0x7fdde0102640 2024-04-02T13:33:48.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.716+0000 7fddd67fc700 1 -- 172.21.15.145:0/2557265117 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdddc02a070 con 0x7fdde0102640 2024-04-02T13:33:48.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.717+0000 7fddd67fc700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fddcc0a3c40 0x7fddcc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:48.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.717+0000 7fddd67fc700 1 -- 172.21.15.145:0/2557265117 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7fdddc0d0af0 con 0x7fdde0102640 2024-04-02T13:33:48.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.717+0000 7fdde4c63700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fddcc0a3c40 0x7fddcc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:48.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.717+0000 7fdde4c63700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fddcc0a3c40 0x7fddcc0a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fddd0007c60 tx=0x7fddd00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:48.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.721+0000 7fddd67fc700 1 -- 172.21.15.145:0/2557265117 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdddc013070 con 0x7fdde0102640 2024-04-02T13:33:48.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:33:48.926+0000 7fdde76c8700 1 -- 172.21.15.145:0/2557265117 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdde0062af0 con 0x7fddcc0a3c40 2024-04-02T13:33:49.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:49 smithi145 ceph-mon[35646]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:49.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:49 smithi145 ceph-mon[35646]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T13:33:49.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:49 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:33:49.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:49 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:33:49.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:49 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:49.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:49 smithi033 ceph-mon[32390]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:49.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:49 smithi033 ceph-mon[32390]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T13:33:49.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:49 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:33:49.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:49 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:33:49.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:49 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:49.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:49 smithi050 ceph-mon[35524]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:49.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:49 smithi050 ceph-mon[35524]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T13:33:49.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:49 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:33:49.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:49 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:33:49.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:49 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:33:50.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:50 smithi145 ceph-mon[35646]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:50.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:50 smithi145 ceph-mon[35646]: osdmap e56: 8 total, 8 up, 8 in 2024-04-02T13:33:50.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:50 smithi050 ceph-mon[35524]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:50.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:50 smithi050 ceph-mon[35524]: osdmap e56: 8 total, 8 up, 8 in 2024-04-02T13:33:50.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:50 smithi033 ceph-mon[32390]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:50.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:50 smithi033 ceph-mon[32390]: osdmap e56: 8 total, 8 up, 8 in 2024-04-02T13:33:51.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:51 smithi145 ceph-mon[35646]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:51.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:51 smithi050 ceph-mon[35524]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:51.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:51 smithi033 ceph-mon[32390]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:52.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:52 smithi145 ceph-mon[35646]: from='client.? 172.21.15.145:0/195889032' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1957ffe4-4131-4157-b7b1-de97b339f24c"}]: dispatch 2024-04-02T13:33:52.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:52 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1957ffe4-4131-4157-b7b1-de97b339f24c"}]: dispatch 2024-04-02T13:33:52.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:52 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1957ffe4-4131-4157-b7b1-de97b339f24c"}]': finished 2024-04-02T13:33:52.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:52 smithi145 ceph-mon[35646]: osdmap e57: 9 total, 8 up, 9 in 2024-04-02T13:33:52.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:52 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:33:52.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:52 smithi145 ceph-mon[35646]: from='client.? 172.21.15.145:0/1339471227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:33:52.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:52 smithi050 ceph-mon[35524]: from='client.? 172.21.15.145:0/195889032' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1957ffe4-4131-4157-b7b1-de97b339f24c"}]: dispatch 2024-04-02T13:33:52.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:52 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1957ffe4-4131-4157-b7b1-de97b339f24c"}]: dispatch 2024-04-02T13:33:52.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:52 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1957ffe4-4131-4157-b7b1-de97b339f24c"}]': finished 2024-04-02T13:33:52.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:52 smithi050 ceph-mon[35524]: osdmap e57: 9 total, 8 up, 9 in 2024-04-02T13:33:52.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:52 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:33:52.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:52 smithi050 ceph-mon[35524]: from='client.? 172.21.15.145:0/1339471227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:33:52.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:52 smithi033 ceph-mon[32390]: from='client.? 172.21.15.145:0/195889032' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1957ffe4-4131-4157-b7b1-de97b339f24c"}]: dispatch 2024-04-02T13:33:52.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:52 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1957ffe4-4131-4157-b7b1-de97b339f24c"}]: dispatch 2024-04-02T13:33:52.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:52 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1957ffe4-4131-4157-b7b1-de97b339f24c"}]': finished 2024-04-02T13:33:52.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:52 smithi033 ceph-mon[32390]: osdmap e57: 9 total, 8 up, 9 in 2024-04-02T13:33:52.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:52 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:33:52.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:52 smithi033 ceph-mon[32390]: from='client.? 172.21.15.145:0/1339471227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:33:53.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:53 smithi050 ceph-mon[35524]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:54.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:53 smithi033 ceph-mon[32390]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:54.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:53 smithi145 ceph-mon[35646]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:55.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:55 smithi050 ceph-mon[35524]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:56.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:55 smithi033 ceph-mon[32390]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:56.128 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:55 smithi145 ceph-mon[35646]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:57.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:57 smithi050 ceph-mon[35524]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:58.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:57 smithi033 ceph-mon[32390]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:58.041 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:57 smithi145 ceph-mon[35646]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:59 smithi145 ceph-mon[35646]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:33:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:59 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T13:33:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:33:59 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:59 smithi033 ceph-mon[32390]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:59 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T13:34:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:33:59 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:00.086 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:59 smithi050 ceph-mon[35524]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:00.086 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:59 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T13:34:00.086 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:33:59 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:00 smithi033 ceph-mon[32390]: Deploying daemon osd.8 on smithi145 2024-04-02T13:34:01.009 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:00 smithi145 ceph-mon[35646]: Deploying daemon osd.8 on smithi145 2024-04-02T13:34:01.058 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:00 smithi050 ceph-mon[35524]: Deploying daemon osd.8 on smithi145 2024-04-02T13:34:02.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:01 smithi033 ceph-mon[32390]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:02.012 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:01 smithi145 ceph-mon[35646]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:02.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:01 smithi050 ceph-mon[35524]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:03.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:03 smithi145 ceph-mon[35646]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:04.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:03 smithi050 ceph-mon[35524]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:04.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:03 smithi033 ceph-mon[32390]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:05.933 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:05 smithi145 ceph-mon[35646]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:05.933 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:05 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:05.933 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:05 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:05.933 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:05 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:06.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:05 smithi050 ceph-mon[35524]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:06.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:05 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:06.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:05 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:06.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:05 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:06.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:05 smithi033 ceph-mon[32390]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:06.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:05 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:06.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:05 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:06.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:05 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:08.109 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:07 smithi145 ceph-mon[35646]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:08.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:07 smithi050 ceph-mon[35524]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:08.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:07 smithi033 ceph-mon[32390]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:08.946 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:34:08.944+0000 7fddd67fc700 1 -- 172.21.15.145:0/2557265117 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdde0062af0 con 0x7fddcc0a3c40 2024-04-02T13:34:08.947 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 8 on host 'smithi145' 2024-04-02T13:34:08.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:34:08.945+0000 7fdde76c8700 1 -- 172.21.15.145:0/2557265117 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fddcc0a3c40 msgr2=0x7fddcc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:08.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:34:08.945+0000 7fdde76c8700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fddcc0a3c40 0x7fddcc0a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fddd0007c60 tx=0x7fddd00073d0 comp rx=0 tx=0).stop 2024-04-02T13:34:08.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:34:08.945+0000 7fdde76c8700 1 -- 172.21.15.145:0/2557265117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdde0102640 msgr2=0x7fdde0076390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:08.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:34:08.945+0000 7fdde76c8700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdde0102640 0x7fdde0076390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0098e0 tx=0x7fdddc0057c0 comp rx=0 tx=0).stop 2024-04-02T13:34:08.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:34:08.945+0000 7fdde76c8700 1 -- 172.21.15.145:0/2557265117 shutdown_connections 2024-04-02T13:34:08.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:34:08.945+0000 7fdde76c8700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fddcc0a3c40 0x7fddcc0a6100 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:08.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:34:08.945+0000 7fdde76c8700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdde01041e0 0x7fdde0074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:08.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:34:08.945+0000 7fdde76c8700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdde0102640 0x7fdde0076390 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:08.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:34:08.945+0000 7fdde76c8700 1 --2- 172.21.15.145:0/2557265117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde0103840 0x7fdde00749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:08.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:34:08.945+0000 7fdde76c8700 1 -- 172.21.15.145:0/2557265117 >> 172.21.15.145:0/2557265117 conn(0x7fdde00fdc00 msgr2=0x7fdde0106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:08.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:34:08.946+0000 7fdde76c8700 1 -- 172.21.15.145:0/2557265117 shutdown_connections 2024-04-02T13:34:08.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:34:08.946+0000 7fdde76c8700 1 -- 172.21.15.145:0/2557265117 wait complete. 2024-04-02T13:34:09.219 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:09 smithi145 ceph-mon[35646]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:09.219 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:09 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.219 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:09 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.219 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:09 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:09.219 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:09 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:09.219 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:09 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.219 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:09 smithi145 ceph-mon[35646]: from='osd.8 [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:34:09.219 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:09 smithi145 ceph-mon[35646]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:34:09.220 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:09 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:09.220 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:09 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.220 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:09 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:09 smithi050 ceph-mon[35524]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:09.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:09.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:09.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:09 smithi050 ceph-mon[35524]: from='osd.8 [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:34:09.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:09 smithi050 ceph-mon[35524]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:34:09.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:09.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:09 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:09 smithi033 ceph-mon[32390]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:09 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:09 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:09 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:09 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:09 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:09 smithi033 ceph-mon[32390]: from='osd.8 [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:34:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:09 smithi033 ceph-mon[32390]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:34:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:09 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:09 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:09 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:09.554 DEBUG:teuthology.orchestra.run.smithi145:osd.8> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.8.service 2024-04-02T13:34:09.556 INFO:tasks.cephadm:Deploying osd.9 on smithi033 with /dev/vg_nvme/lv_1... 2024-04-02T13:34:09.557 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T13:34:09.811 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:10.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:10 smithi050 ceph-mon[35524]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T13:34:10.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:10 smithi050 ceph-mon[35524]: from='osd.8 [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:34:10.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:10 smithi050 ceph-mon[35524]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T13:34:10.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:10 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:34:10.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:10 smithi050 ceph-mon[35524]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:34:11.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:10 smithi145 ceph-mon[35646]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T13:34:11.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:10 smithi145 ceph-mon[35646]: from='osd.8 [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:34:11.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:10 smithi145 ceph-mon[35646]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T13:34:11.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:10 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:34:11.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:10 smithi145 ceph-mon[35646]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:34:11.058 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:10 smithi033 ceph-mon[32390]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T13:34:11.058 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:10 smithi033 ceph-mon[32390]: from='osd.8 [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:34:11.058 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:10 smithi033 ceph-mon[32390]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T13:34:11.058 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:10 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:34:11.058 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:10 smithi033 ceph-mon[32390]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:34:11.291 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:34:11 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:34:11.236+0000 7f74a55c6700 -1 osd.8 0 waiting for initial osdmap 2024-04-02T13:34:11.291 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:34:11 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:34:11.247+0000 7f749d3b1700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:34:11.959 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T13:34:11.959 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T13:34:11.960 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-02T13:34:11.960 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-02T13:34:11.960 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122588 s, 855 MB/s 2024-04-02T13:34:11.960 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-02T13:34:12.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:11 smithi145 ceph-mon[35646]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:12.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:11 smithi145 ceph-mon[35646]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-04-02T13:34:12.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:11 smithi145 ceph-mon[35646]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T13:34:12.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:11 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:34:12.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:11 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:34:12.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:11 smithi145 ceph-mon[35646]: from='osd.8 ' entity='osd.8' 2024-04-02T13:34:12.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:11 smithi145 ceph-mon[35646]: osd.8 [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848] boot 2024-04-02T13:34:12.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:11 smithi145 ceph-mon[35646]: osdmap e60: 9 total, 9 up, 9 in 2024-04-02T13:34:12.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:11 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:34:12.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:11 smithi050 ceph-mon[35524]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:12.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:11 smithi050 ceph-mon[35524]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-04-02T13:34:12.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:11 smithi050 ceph-mon[35524]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T13:34:12.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:11 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:34:12.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:11 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:34:12.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:11 smithi050 ceph-mon[35524]: from='osd.8 ' entity='osd.8' 2024-04-02T13:34:12.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:11 smithi050 ceph-mon[35524]: osd.8 [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848] boot 2024-04-02T13:34:12.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:11 smithi050 ceph-mon[35524]: osdmap e60: 9 total, 9 up, 9 in 2024-04-02T13:34:12.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:11 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:34:12.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:11 smithi033 ceph-mon[32390]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:34:12.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:11 smithi033 ceph-mon[32390]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-04-02T13:34:12.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:11 smithi033 ceph-mon[32390]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T13:34:12.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:11 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:34:12.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:11 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:34:12.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:11 smithi033 ceph-mon[32390]: from='osd.8 ' entity='osd.8' 2024-04-02T13:34:12.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:11 smithi033 ceph-mon[32390]: osd.8 [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848] boot 2024-04-02T13:34:12.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:11 smithi033 ceph-mon[32390]: osdmap e60: 9 total, 9 up, 9 in 2024-04-02T13:34:12.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:11 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:34:12.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi033:vg_nvme/lv_1 2024-04-02T13:34:12.903 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:12.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:12 smithi033 ceph-mon[32390]: purged_snaps scrub starts 2024-04-02T13:34:12.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:12 smithi033 ceph-mon[32390]: purged_snaps scrub ok 2024-04-02T13:34:12.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:12 smithi033 ceph-mon[32390]: osdmap e61: 9 total, 9 up, 9 in 2024-04-02T13:34:13.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:12 smithi145 ceph-mon[35646]: purged_snaps scrub starts 2024-04-02T13:34:13.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:12 smithi145 ceph-mon[35646]: purged_snaps scrub ok 2024-04-02T13:34:13.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:12 smithi145 ceph-mon[35646]: osdmap e61: 9 total, 9 up, 9 in 2024-04-02T13:34:13.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:12 smithi050 ceph-mon[35524]: purged_snaps scrub starts 2024-04-02T13:34:13.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:12 smithi050 ceph-mon[35524]: purged_snaps scrub ok 2024-04-02T13:34:13.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:12 smithi050 ceph-mon[35524]: osdmap e61: 9 total, 9 up, 9 in 2024-04-02T13:34:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:13 smithi033 ceph-mon[32390]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:13 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:13 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:13 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:13 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:13 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:13 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:13 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:13 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:14.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:13 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:14.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:13 smithi033 ceph-mon[32390]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T13:34:14.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:13 smithi145 ceph-mon[35646]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:14.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:13 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:14.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:13 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:14.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:13 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:14.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:13 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:14.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:13 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:14.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:13 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:14.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:13 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:14.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:13 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:14.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:13 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:14.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:13 smithi145 ceph-mon[35646]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T13:34:14.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:13 smithi050 ceph-mon[35524]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:13 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:13 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:13 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:13 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:13 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:13 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:13 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:13 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:13 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:13 smithi050 ceph-mon[35524]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T13:34:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.566+0000 7ff9390d5700 1 -- 172.21.15.33:0/330187329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934101590 msgr2=0x7ff9341019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.566+0000 7ff9390d5700 1 --2- 172.21.15.33:0/330187329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934101590 0x7ff9341019b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff92400b0d0 tx=0x7ff92400b3e0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.566+0000 7ff9390d5700 1 -- 172.21.15.33:0/330187329 shutdown_connections 2024-04-02T13:34:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.566+0000 7ff9390d5700 1 --2- 172.21.15.33:0/330187329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff934103130 0x7ff9341099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.566+0000 7ff9390d5700 1 --2- 172.21.15.33:0/330187329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934101590 0x7ff9341019b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.566+0000 7ff9390d5700 1 --2- 172.21.15.33:0/330187329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff934102790 0x7ff934102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.566+0000 7ff9390d5700 1 -- 172.21.15.33:0/330187329 >> 172.21.15.33:0/330187329 conn(0x7ff9340fcb50 msgr2=0x7ff9340fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.566+0000 7ff9390d5700 1 -- 172.21.15.33:0/330187329 shutdown_connections 2024-04-02T13:34:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.566+0000 7ff9390d5700 1 -- 172.21.15.33:0/330187329 wait complete. 2024-04-02T13:34:14.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.567+0000 7ff9390d5700 1 Processor -- start 2024-04-02T13:34:14.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.567+0000 7ff9390d5700 1 -- start start 2024-04-02T13:34:14.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff9390d5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff934101590 0x7ff93419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:14.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff9390d5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934102790 0x7ff93419afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:14.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff9390d5700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff934103130 0x7ff9341a2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff9390d5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff93410d150 con 0x7ff934102790 2024-04-02T13:34:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff9390d5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff93410cfd0 con 0x7ff934101590 2024-04-02T13:34:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff9390d5700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff93410d2d0 con 0x7ff934103130 2024-04-02T13:34:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff93259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934102790 0x7ff93419afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff93259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934102790 0x7ff93419afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36744/0 (socket says 172.21.15.33:36744) 2024-04-02T13:34:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff93259c700 1 -- 172.21.15.33:0/2820321498 learned_addr learned my addr 172.21.15.33:0/2820321498 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:34:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff932d9d700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff934101590 0x7ff93419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff93359e700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff934103130 0x7ff9341a2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff93259c700 1 -- 172.21.15.33:0/2820321498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff934103130 msgr2=0x7ff9341a2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff93259c700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff934103130 0x7ff9341a2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff93259c700 1 -- 172.21.15.33:0/2820321498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff934101590 msgr2=0x7ff93419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff93259c700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff934101590 0x7ff93419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff93259c700 1 -- 172.21.15.33:0/2820321498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9341a26e0 con 0x7ff934102790 2024-04-02T13:34:14.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff93259c700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934102790 0x7ff93419afc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff91c00b560 tx=0x7ff91c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:14.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff92bfff700 1 -- 172.21.15.33:0/2820321498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c015020 con 0x7ff934102790 2024-04-02T13:34:14.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff9390d5700 1 -- 172.21.15.33:0/2820321498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9341a29d0 con 0x7ff934102790 2024-04-02T13:34:14.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff92bfff700 1 -- 172.21.15.33:0/2820321498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff91c011d00 con 0x7ff934102790 2024-04-02T13:34:14.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.568+0000 7ff92bfff700 1 -- 172.21.15.33:0/2820321498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c010dd0 con 0x7ff934102790 2024-04-02T13:34:14.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.569+0000 7ff9390d5700 1 -- 172.21.15.33:0/2820321498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9341a2f10 con 0x7ff934102790 2024-04-02T13:34:14.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.570+0000 7ff9390d5700 1 -- 172.21.15.33:0/2820321498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9340692b0 con 0x7ff934102790 2024-04-02T13:34:14.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.571+0000 7ff92bfff700 1 -- 172.21.15.33:0/2820321498 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff91c018020 con 0x7ff934102790 2024-04-02T13:34:14.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.571+0000 7ff92bfff700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff9200a3c40 0x7ff9200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:14.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.571+0000 7ff932d9d700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff9200a3c40 0x7ff9200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:14.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.571+0000 7ff92bfff700 1 -- 172.21.15.33:0/2820321498 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7ff91c0cf150 con 0x7ff934102790 2024-04-02T13:34:14.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.572+0000 7ff932d9d700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff9200a3c40 0x7ff9200a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff92400d010 tx=0x7ff92400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:14.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.576+0000 7ff92bfff700 1 -- 172.21.15.33:0/2820321498 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff91c01f020 con 0x7ff934102790 2024-04-02T13:34:14.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:14.796+0000 7ff9390d5700 1 -- 172.21.15.33:0/2820321498 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff9340fed00 con 0x7ff9200a3c40 2024-04-02T13:34:15.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:14 smithi145 ceph-mon[35646]: Detected new or changed devices on smithi145 2024-04-02T13:34:15.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:14 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi145 to 5696M 2024-04-02T13:34:15.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:14 smithi145 ceph-mon[35646]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T13:34:15.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:14 smithi050 ceph-mon[35524]: Detected new or changed devices on smithi145 2024-04-02T13:34:15.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:14 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi145 to 5696M 2024-04-02T13:34:15.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:14 smithi050 ceph-mon[35524]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T13:34:15.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:14 smithi033 ceph-mon[32390]: Detected new or changed devices on smithi145 2024-04-02T13:34:15.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:14 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi145 to 5696M 2024-04-02T13:34:15.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:14 smithi033 ceph-mon[32390]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T13:34:16.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:15 smithi145 ceph-mon[35646]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:16.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:15 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:34:16.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:15 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:34:16.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:15 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:16.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:15 smithi145 ceph-mon[35646]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T13:34:16.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:15 smithi050 ceph-mon[35524]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:16.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:15 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:34:16.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:15 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:34:16.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:15 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:16.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:15 smithi050 ceph-mon[35524]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T13:34:16.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:15 smithi033 ceph-mon[32390]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:16.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:34:16.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:34:16.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:15 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:16.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:15 smithi033 ceph-mon[32390]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T13:34:16.975 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:16 smithi033 ceph-mon[32390]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:17.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:16 smithi145 ceph-mon[35646]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:17.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:16 smithi050 ceph-mon[35524]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:18.155 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:17 smithi033 ceph-mon[32390]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:18.155 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:17 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3802930748' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32d3ed72-1341-4dd0-928e-e0e331751178"}]: dispatch 2024-04-02T13:34:18.155 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:17 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3802930748' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32d3ed72-1341-4dd0-928e-e0e331751178"}]': finished 2024-04-02T13:34:18.156 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:17 smithi033 ceph-mon[32390]: osdmap e65: 10 total, 9 up, 10 in 2024-04-02T13:34:18.156 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:18.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:17 smithi145 ceph-mon[35646]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:18.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:17 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3802930748' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32d3ed72-1341-4dd0-928e-e0e331751178"}]: dispatch 2024-04-02T13:34:18.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:17 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3802930748' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32d3ed72-1341-4dd0-928e-e0e331751178"}]': finished 2024-04-02T13:34:18.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:17 smithi145 ceph-mon[35646]: osdmap e65: 10 total, 9 up, 10 in 2024-04-02T13:34:18.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:18.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:17 smithi050 ceph-mon[35524]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:18.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:17 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3802930748' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32d3ed72-1341-4dd0-928e-e0e331751178"}]: dispatch 2024-04-02T13:34:18.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:17 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3802930748' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32d3ed72-1341-4dd0-928e-e0e331751178"}]': finished 2024-04-02T13:34:18.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:17 smithi050 ceph-mon[35524]: osdmap e65: 10 total, 9 up, 10 in 2024-04-02T13:34:18.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:19.166 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:18 smithi033 ceph-mon[32390]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-02T13:34:19.166 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:18 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3293578938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:34:19.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:18 smithi145 ceph-mon[35646]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-02T13:34:19.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:18 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3293578938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:34:19.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:18 smithi050 ceph-mon[35524]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-02T13:34:19.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:18 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3293578938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:34:21.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:20 smithi145 ceph-mon[35646]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T13:34:21.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:20 smithi050 ceph-mon[35524]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T13:34:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:20 smithi033 ceph-mon[32390]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T13:34:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:22 smithi033 ceph-mon[32390]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-02T13:34:23.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:22 smithi145 ceph-mon[35646]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-02T13:34:23.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:22 smithi050 ceph-mon[35524]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-02T13:34:25.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:24 smithi033 ceph-mon[32390]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-02T13:34:25.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:24 smithi145 ceph-mon[35646]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-02T13:34:25.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:24 smithi050 ceph-mon[35524]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-02T13:34:26.166 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:25 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T13:34:26.167 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:25 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:26.167 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:25 smithi033 ceph-mon[32390]: Deploying daemon osd.9 on smithi033 2024-04-02T13:34:26.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:25 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T13:34:26.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:25 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:26.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:25 smithi145 ceph-mon[35646]: Deploying daemon osd.9 on smithi033 2024-04-02T13:34:26.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:25 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T13:34:26.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:25 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:26.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:25 smithi050 ceph-mon[35524]: Deploying daemon osd.9 on smithi033 2024-04-02T13:34:27.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:26 smithi145 ceph-mon[35646]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T13:34:27.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:26 smithi050 ceph-mon[35524]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T13:34:27.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:26 smithi033 ceph-mon[32390]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T13:34:29.181 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:28 smithi033 ceph-mon[32390]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:29.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:28 smithi145 ceph-mon[35646]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:29.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:28 smithi050 ceph-mon[35524]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:31.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:30 smithi033 ceph-mon[32390]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:31.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:30 smithi145 ceph-mon[35646]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:31.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:30 smithi050 ceph-mon[35524]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:33.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:33.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:33.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:33.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:33 smithi033 ceph-mon[32390]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:33.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:33 smithi145 ceph-mon[35646]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:33 smithi050 ceph-mon[35524]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:35.825 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 9 on host 'smithi033' 2024-04-02T13:34:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:35.820+0000 7ff92bfff700 1 -- 172.21.15.33:0/2820321498 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff9340fed00 con 0x7ff9200a3c40 2024-04-02T13:34:35.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:35.822+0000 7ff9390d5700 1 -- 172.21.15.33:0/2820321498 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff9200a3c40 msgr2=0x7ff9200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:35.822+0000 7ff9390d5700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff9200a3c40 0x7ff9200a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff92400d010 tx=0x7ff92400b040 comp rx=0 tx=0).stop 2024-04-02T13:34:35.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:35.822+0000 7ff9390d5700 1 -- 172.21.15.33:0/2820321498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934102790 msgr2=0x7ff93419afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:35.822+0000 7ff9390d5700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934102790 0x7ff93419afc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff91c00b560 tx=0x7ff91c00b870 comp rx=0 tx=0).stop 2024-04-02T13:34:35.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:35.822+0000 7ff9390d5700 1 -- 172.21.15.33:0/2820321498 shutdown_connections 2024-04-02T13:34:35.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:35.822+0000 7ff9390d5700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff9200a3c40 0x7ff9200a6100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:35.822+0000 7ff9390d5700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff934103130 0x7ff9341a2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:35.822+0000 7ff9390d5700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff934101590 0x7ff93419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:35.822+0000 7ff9390d5700 1 --2- 172.21.15.33:0/2820321498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934102790 0x7ff93419afc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:35.822+0000 7ff9390d5700 1 -- 172.21.15.33:0/2820321498 >> 172.21.15.33:0/2820321498 conn(0x7ff9340fcb50 msgr2=0x7ff9340fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:35.823+0000 7ff9390d5700 1 -- 172.21.15.33:0/2820321498 shutdown_connections 2024-04-02T13:34:35.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:34:35.823+0000 7ff9390d5700 1 -- 172.21.15.33:0/2820321498 wait complete. 2024-04-02T13:34:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:35 smithi145 ceph-mon[35646]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:35 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:35 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:35 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:35 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:35.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:35 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:35.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:35 smithi145 ceph-mon[35646]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T13:34:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:35 smithi050 ceph-mon[35524]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:35 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:35 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:35 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:35 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:35 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:35.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:35 smithi050 ceph-mon[35524]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T13:34:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:35 smithi033 ceph-mon[32390]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:35 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:35 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:35 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:35 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:35 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:36.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:35 smithi033 ceph-mon[32390]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T13:34:36.510 DEBUG:teuthology.orchestra.run.smithi033:osd.9> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.9.service 2024-04-02T13:34:36.512 INFO:tasks.cephadm:Deploying osd.10 on smithi050 with /dev/vg_nvme/lv_1... 2024-04-02T13:34:36.513 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T13:34:36.764 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:34:36.885 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:36 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:36.886 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:36 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:36.886 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:36 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:36.886 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:36 smithi050 ceph-mon[35524]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T13:34:36.886 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:36 smithi050 ceph-mon[35524]: osdmap e66: 10 total, 9 up, 10 in 2024-04-02T13:34:36.886 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:36 smithi050 ceph-mon[35524]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-02T13:34:36.886 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:36 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:37.022 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:36 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:37.023 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:36 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:37.023 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:36 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:37.023 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:36 smithi033 ceph-mon[32390]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T13:34:37.023 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:36 smithi033 ceph-mon[32390]: osdmap e66: 10 total, 9 up, 10 in 2024-04-02T13:34:37.023 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:36 smithi033 ceph-mon[32390]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-02T13:34:37.023 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:36 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:37.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:36 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:37.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:36 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:37.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:36 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:37.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:36 smithi145 ceph-mon[35646]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T13:34:37.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:36 smithi145 ceph-mon[35646]: osdmap e66: 10 total, 9 up, 10 in 2024-04-02T13:34:37.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:36 smithi145 ceph-mon[35646]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-02T13:34:37.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:36 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:37.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:34:37 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:34:37.422+0000 7febed8ef700 -1 osd.9 0 waiting for initial osdmap 2024-04-02T13:34:37.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:34:37 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:34:37.434+0000 7febe56da700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:34:38.126 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:37 smithi050 ceph-mon[35524]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:38.126 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:37 smithi050 ceph-mon[35524]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-02T13:34:38.126 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:37 smithi050 ceph-mon[35524]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T13:34:38.126 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:37 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:38.126 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:37 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:38.126 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:37 smithi050 ceph-mon[35524]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' 2024-04-02T13:34:38.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:37 smithi145 ceph-mon[35646]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:38.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:37 smithi145 ceph-mon[35646]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-02T13:34:38.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:37 smithi145 ceph-mon[35646]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T13:34:38.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:37 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:38.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:37 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:38.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:37 smithi145 ceph-mon[35646]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' 2024-04-02T13:34:38.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:37 smithi033 ceph-mon[32390]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:38.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:37 smithi033 ceph-mon[32390]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-02T13:34:38.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:37 smithi033 ceph-mon[32390]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T13:34:38.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:37 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:38.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:37 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:38.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:37 smithi033 ceph-mon[32390]: from='osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485]' entity='osd.9' 2024-04-02T13:34:38.905 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T13:34:38.905 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T13:34:38.905 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-04-02T13:34:38.905 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-04-02T13:34:38.905 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00845659 s, 1.2 GB/s 2024-04-02T13:34:38.905 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-04-02T13:34:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:38 smithi033 ceph-mon[32390]: purged_snaps scrub starts 2024-04-02T13:34:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:38 smithi033 ceph-mon[32390]: purged_snaps scrub ok 2024-04-02T13:34:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:38 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:38 smithi033 ceph-mon[32390]: osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485] boot 2024-04-02T13:34:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:38 smithi033 ceph-mon[32390]: osdmap e68: 10 total, 10 up, 10 in 2024-04-02T13:34:39.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:38 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:39.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:38 smithi145 ceph-mon[35646]: purged_snaps scrub starts 2024-04-02T13:34:39.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:38 smithi145 ceph-mon[35646]: purged_snaps scrub ok 2024-04-02T13:34:39.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:38 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:39.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:38 smithi145 ceph-mon[35646]: osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485] boot 2024-04-02T13:34:39.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:38 smithi145 ceph-mon[35646]: osdmap e68: 10 total, 10 up, 10 in 2024-04-02T13:34:39.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:38 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:39.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:38 smithi050 ceph-mon[35524]: purged_snaps scrub starts 2024-04-02T13:34:39.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:38 smithi050 ceph-mon[35524]: purged_snaps scrub ok 2024-04-02T13:34:39.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:38 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:39.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:38 smithi050 ceph-mon[35524]: osd.9 [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485] boot 2024-04-02T13:34:39.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:38 smithi050 ceph-mon[35524]: osdmap e68: 10 total, 10 up, 10 in 2024-04-02T13:34:39.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:38 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:34:39.625 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi050:vg_nvme/lv_1 2024-04-02T13:34:39.878 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:34:39.920 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:39 smithi050 ceph-mon[35524]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:39.920 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:39 smithi050 ceph-mon[35524]: osdmap e69: 10 total, 10 up, 10 in 2024-04-02T13:34:40.224 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:39 smithi033 ceph-mon[32390]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:40.224 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:39 smithi033 ceph-mon[32390]: osdmap e69: 10 total, 10 up, 10 in 2024-04-02T13:34:40.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:39 smithi145 ceph-mon[35646]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:34:40.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:39 smithi145 ceph-mon[35646]: osdmap e69: 10 total, 10 up, 10 in 2024-04-02T13:34:41.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.480+0000 7fc943f0c700 1 -- 172.21.15.50:0/464424385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c102720 msgr2=0x7fc93c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:41.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.480+0000 7fc943f0c700 1 --2- 172.21.15.50:0/464424385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c102720 0x7fc93c102b80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc9300099c0 tx=0x7fc930009cd0 comp rx=0 tx=0).stop 2024-04-02T13:34:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.480+0000 7fc943f0c700 1 -- 172.21.15.50:0/464424385 shutdown_connections 2024-04-02T13:34:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.480+0000 7fc943f0c700 1 --2- 172.21.15.50:0/464424385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc93c1030c0 0x7fc93c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.480+0000 7fc943f0c700 1 --2- 172.21.15.50:0/464424385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc93c101520 0x7fc93c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.480+0000 7fc943f0c700 1 --2- 172.21.15.50:0/464424385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c102720 0x7fc93c102b80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.480+0000 7fc943f0c700 1 -- 172.21.15.50:0/464424385 >> 172.21.15.50:0/464424385 conn(0x7fc93c0fcac0 msgr2=0x7fc93c0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.480+0000 7fc943f0c700 1 -- 172.21.15.50:0/464424385 shutdown_connections 2024-04-02T13:34:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.480+0000 7fc943f0c700 1 -- 172.21.15.50:0/464424385 wait complete. 2024-04-02T13:34:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.480+0000 7fc943f0c700 1 Processor -- start 2024-04-02T13:34:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc943f0c700 1 -- start start 2024-04-02T13:34:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc943f0c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc93c101520 0x7fc93c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc943f0c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc93c102720 0x7fc93c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc943f0c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c1030c0 0x7fc93c19ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:41.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc943f0c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c10d1a0 con 0x7fc93c102720 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc943f0c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c10d020 con 0x7fc93c1030c0 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc943f0c700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c10d320 con 0x7fc93c101520 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc9424a9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c1030c0 0x7fc93c19ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc9424a9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c1030c0 0x7fc93c19ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38660/0 (socket says 172.21.15.50:38660) 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc9424a9700 1 -- 172.21.15.50:0/3236476767 learned_addr learned my addr 172.21.15.50:0/3236476767 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc941ca8700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc93c101520 0x7fc93c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc9414a7700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc93c102720 0x7fc93c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc9424a9700 1 -- 172.21.15.50:0/3236476767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc93c101520 msgr2=0x7fc93c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc9424a9700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc93c101520 0x7fc93c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc9424a9700 1 -- 172.21.15.50:0/3236476767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc93c102720 msgr2=0x7fc93c198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc9424a9700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc93c102720 0x7fc93c198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc9424a9700 1 -- 172.21.15.50:0/3236476767 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c1a0650 con 0x7fc93c1030c0 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc9424a9700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c1030c0 0x7fc93c19ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00b970 tx=0x7fc92c00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:41.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.481+0000 7fc93affd700 1 -- 172.21.15.50:0/3236476767 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c019070 con 0x7fc93c1030c0 2024-04-02T13:34:41.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.482+0000 7fc943f0c700 1 -- 172.21.15.50:0/3236476767 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93c1a0940 con 0x7fc93c1030c0 2024-04-02T13:34:41.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.482+0000 7fc93affd700 1 -- 172.21.15.50:0/3236476767 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc92c00ce00 con 0x7fc93c1030c0 2024-04-02T13:34:41.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.482+0000 7fc93affd700 1 -- 172.21.15.50:0/3236476767 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c007370 con 0x7fc93c1030c0 2024-04-02T13:34:41.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.482+0000 7fc943f0c700 1 -- 172.21.15.50:0/3236476767 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93c1a0eb0 con 0x7fc93c1030c0 2024-04-02T13:34:41.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.484+0000 7fc943f0c700 1 -- 172.21.15.50:0/3236476767 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc93c0692b0 con 0x7fc93c1030c0 2024-04-02T13:34:41.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.484+0000 7fc93affd700 1 -- 172.21.15.50:0/3236476767 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc92c01c070 con 0x7fc93c1030c0 2024-04-02T13:34:41.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.484+0000 7fc93affd700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc9240a3c40 0x7fc9240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:41.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.484+0000 7fc93affd700 1 -- 172.21.15.50:0/3236476767 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fc92c0cef20 con 0x7fc93c1030c0 2024-04-02T13:34:41.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.484+0000 7fc941ca8700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc9240a3c40 0x7fc9240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:41.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.485+0000 7fc941ca8700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc9240a3c40 0x7fc9240a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc93c102580 tx=0x7fc934009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:41.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.489+0000 7fc93affd700 1 -- 172.21.15.50:0/3236476767 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc92c09c100 con 0x7fc93c1030c0 2024-04-02T13:34:41.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:41 smithi050 ceph-mon[35524]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:41.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:41 smithi050 ceph-mon[35524]: Detected new or changed devices on smithi033 2024-04-02T13:34:41.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:41.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:41.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:41 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi033 to 4273M 2024-04-02T13:34:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:41.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:41.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:34:41.709+0000 7fc943f0c700 1 -- 172.21.15.50:0/3236476767 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc93c06aa20 con 0x7fc9240a3c40 2024-04-02T13:34:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:41 smithi145 ceph-mon[35646]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:41 smithi145 ceph-mon[35646]: Detected new or changed devices on smithi033 2024-04-02T13:34:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:41 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi033 to 4273M 2024-04-02T13:34:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:41 smithi033 ceph-mon[32390]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:41 smithi033 ceph-mon[32390]: Detected new or changed devices on smithi033 2024-04-02T13:34:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:34:41.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:41 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi033 to 4273M 2024-04-02T13:34:41.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:41.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:41.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:41.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:42.606 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:42 smithi050 ceph-mon[35524]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:42.606 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:42 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:34:42.607 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:42 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:34:42.607 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:42 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:42.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:42 smithi145 ceph-mon[35646]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:42.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:42 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:34:42.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:42 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:34:42.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:42 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:42.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:42 smithi033 ceph-mon[32390]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:42.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:42 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:34:42.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:42 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:34:42.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:42 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:43.655 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:43 smithi050 ceph-mon[35524]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:43.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:43 smithi145 ceph-mon[35646]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:43.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:43 smithi033 ceph-mon[32390]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:45.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:45 smithi145 ceph-mon[35646]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:45.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:45 smithi145 ceph-mon[35646]: from='client.? 172.21.15.50:0/1135007257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0252727c-a17e-4542-b7c3-c409c8a6d0bd"}]: dispatch 2024-04-02T13:34:45.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:45 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0252727c-a17e-4542-b7c3-c409c8a6d0bd"}]: dispatch 2024-04-02T13:34:45.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:45 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0252727c-a17e-4542-b7c3-c409c8a6d0bd"}]': finished 2024-04-02T13:34:45.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:45 smithi145 ceph-mon[35646]: osdmap e70: 11 total, 10 up, 11 in 2024-04-02T13:34:45.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:45 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:34:45.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:45 smithi050 ceph-mon[35524]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:45.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:45 smithi050 ceph-mon[35524]: from='client.? 172.21.15.50:0/1135007257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0252727c-a17e-4542-b7c3-c409c8a6d0bd"}]: dispatch 2024-04-02T13:34:45.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:45 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0252727c-a17e-4542-b7c3-c409c8a6d0bd"}]: dispatch 2024-04-02T13:34:45.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:45 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0252727c-a17e-4542-b7c3-c409c8a6d0bd"}]': finished 2024-04-02T13:34:45.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:45 smithi050 ceph-mon[35524]: osdmap e70: 11 total, 10 up, 11 in 2024-04-02T13:34:45.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:45 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:34:45.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:45 smithi033 ceph-mon[32390]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:45.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:45 smithi033 ceph-mon[32390]: from='client.? 172.21.15.50:0/1135007257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0252727c-a17e-4542-b7c3-c409c8a6d0bd"}]: dispatch 2024-04-02T13:34:45.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:45 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0252727c-a17e-4542-b7c3-c409c8a6d0bd"}]: dispatch 2024-04-02T13:34:45.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:45 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0252727c-a17e-4542-b7c3-c409c8a6d0bd"}]': finished 2024-04-02T13:34:45.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:45 smithi033 ceph-mon[32390]: osdmap e70: 11 total, 10 up, 11 in 2024-04-02T13:34:45.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:45 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:34:46.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:46 smithi145 ceph-mon[35646]: from='client.? 172.21.15.50:0/774597623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:34:46.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:46 smithi050 ceph-mon[35524]: from='client.? 172.21.15.50:0/774597623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:34:46.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:46 smithi033 ceph-mon[32390]: from='client.? 172.21.15.50:0/774597623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:34:47.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:47 smithi145 ceph-mon[35646]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:47.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:47 smithi050 ceph-mon[35524]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:47.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:47 smithi033 ceph-mon[32390]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:49.556 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:49 smithi033 ceph-mon[32390]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:49.675 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:49 smithi050 ceph-mon[35524]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:49.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:49 smithi145 ceph-mon[35646]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:51.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:51 smithi050 ceph-mon[35524]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:51.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:51 smithi145 ceph-mon[35646]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:51.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:51 smithi033 ceph-mon[32390]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:52.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:52 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T13:34:52.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:52 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:52.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:52 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T13:34:52.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:52 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:52.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:52 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T13:34:52.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:52 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:53.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:53 smithi050 ceph-mon[35524]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:53.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:53 smithi050 ceph-mon[35524]: Deploying daemon osd.10 on smithi050 2024-04-02T13:34:53.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:53 smithi145 ceph-mon[35646]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:53.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:53 smithi145 ceph-mon[35646]: Deploying daemon osd.10 on smithi050 2024-04-02T13:34:53.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:53 smithi033 ceph-mon[32390]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:53.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:53 smithi033 ceph-mon[32390]: Deploying daemon osd.10 on smithi050 2024-04-02T13:34:55.432 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:55 smithi050 ceph-mon[35524]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:55.560 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:55 smithi145 ceph-mon[35646]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:55.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:55 smithi033 ceph-mon[32390]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:57.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:57 smithi050 ceph-mon[35524]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:57.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:57 smithi145 ceph-mon[35646]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:57.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:57 smithi033 ceph-mon[32390]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:59.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:58 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:59.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:58 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:59.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:58 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:59.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:34:58 smithi145 ceph-mon[35646]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:59.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:58 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:59.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:58 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:59.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:58 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:59.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:34:58 smithi050 ceph-mon[35524]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:34:59.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:58 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:59.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:58 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:34:59.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:58 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:34:59.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:34:58 smithi033 ceph-mon[32390]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:35:01.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:00 smithi145 ceph-mon[35646]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:35:01.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:01.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:01.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:01.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:01.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:00 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:01.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:00 smithi050 ceph-mon[35524]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:35:01.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:00 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:01.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:00 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:01.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:00 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:01.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:00 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:01.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:00 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:01.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:00 smithi033 ceph-mon[32390]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:35:01.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:00 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:01.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:00 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:01.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:00 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:01.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:00 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:01.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:00 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:01.684 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 10 on host 'smithi050' 2024-04-02T13:35:01.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:35:01.681+0000 7fc93affd700 1 -- 172.21.15.50:0/3236476767 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc93c06aa20 con 0x7fc9240a3c40 2024-04-02T13:35:01.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:35:01.683+0000 7fc943f0c700 1 -- 172.21.15.50:0/3236476767 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc9240a3c40 msgr2=0x7fc9240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:35:01.683+0000 7fc943f0c700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc9240a3c40 0x7fc9240a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc93c102580 tx=0x7fc934009190 comp rx=0 tx=0).stop 2024-04-02T13:35:01.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:35:01.683+0000 7fc943f0c700 1 -- 172.21.15.50:0/3236476767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c1030c0 msgr2=0x7fc93c19ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:35:01.683+0000 7fc943f0c700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c1030c0 0x7fc93c19ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00b970 tx=0x7fc92c00bc80 comp rx=0 tx=0).stop 2024-04-02T13:35:01.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:35:01.683+0000 7fc943f0c700 1 -- 172.21.15.50:0/3236476767 shutdown_connections 2024-04-02T13:35:01.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:35:01.683+0000 7fc943f0c700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc9240a3c40 0x7fc9240a6100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:35:01.683+0000 7fc943f0c700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc93c1030c0 0x7fc93c19ffb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:35:01.683+0000 7fc943f0c700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc93c101520 0x7fc93c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:35:01.683+0000 7fc943f0c700 1 --2- 172.21.15.50:0/3236476767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc93c102720 0x7fc93c198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:35:01.683+0000 7fc943f0c700 1 -- 172.21.15.50:0/3236476767 >> 172.21.15.50:0/3236476767 conn(0x7fc93c0fcac0 msgr2=0x7fc93c06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:35:01.683+0000 7fc943f0c700 1 -- 172.21.15.50:0/3236476767 shutdown_connections 2024-04-02T13:35:01.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-04-02T13:35:01.683+0000 7fc943f0c700 1 -- 172.21.15.50:0/3236476767 wait complete. 2024-04-02T13:35:02.400 DEBUG:teuthology.orchestra.run.smithi050:osd.10> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.10.service 2024-04-02T13:35:02.402 INFO:tasks.cephadm:Deploying osd.11 on smithi145 with /dev/vg_nvme/lv_1... 2024-04-02T13:35:02.403 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T13:35:02.649 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:35:02.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:02 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:02.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:02 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:02.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:02 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:02.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:02 smithi145 ceph-mon[35646]: from='osd.10 [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:35:02.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:02 smithi145 ceph-mon[35646]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:35:02.969 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:02 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:02.969 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:02 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:02.969 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:02 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:02.969 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:02 smithi050 ceph-mon[35524]: from='osd.10 [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:35:02.969 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:02 smithi050 ceph-mon[35524]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:35:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:02 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:02 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:02 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:02 smithi033 ceph-mon[32390]: from='osd.10 [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:35:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:02 smithi033 ceph-mon[32390]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:35:03.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:03 smithi050 ceph-mon[35524]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:35:03.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:03 smithi050 ceph-mon[35524]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T13:35:03.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:03 smithi050 ceph-mon[35524]: from='osd.10 [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:35:03.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:03 smithi050 ceph-mon[35524]: osdmap e71: 11 total, 10 up, 11 in 2024-04-02T13:35:03.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:03 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:03.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:03 smithi050 ceph-mon[35524]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:35:03.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:03 smithi145 ceph-mon[35646]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:35:03.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:03 smithi145 ceph-mon[35646]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T13:35:03.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:03 smithi145 ceph-mon[35646]: from='osd.10 [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:35:03.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:03 smithi145 ceph-mon[35646]: osdmap e71: 11 total, 10 up, 11 in 2024-04-02T13:35:03.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:03 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:03.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:03 smithi145 ceph-mon[35646]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:35:04.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:03 smithi033 ceph-mon[32390]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:35:04.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:03 smithi033 ceph-mon[32390]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T13:35:04.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:03 smithi033 ceph-mon[32390]: from='osd.10 [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:35:04.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:03 smithi033 ceph-mon[32390]: osdmap e71: 11 total, 10 up, 11 in 2024-04-02T13:35:04.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:03 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:04.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:03 smithi033 ceph-mon[32390]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-04-02T13:35:04.246 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:35:04 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:35:04.130+0000 7f5e83530700 -1 osd.10 0 waiting for initial osdmap 2024-04-02T13:35:04.246 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:35:04 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:35:04.147+0000 7f5e7b31b700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:35:04.777 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T13:35:04.777 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T13:35:04.777 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2024-04-02T13:35:04.777 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2024-04-02T13:35:04.777 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0139939 s, 749 MB/s 2024-04-02T13:35:04.777 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2024-04-02T13:35:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:04 smithi145 ceph-mon[35646]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-04-02T13:35:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:04 smithi145 ceph-mon[35646]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T13:35:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:04 smithi145 ceph-mon[35646]: from='osd.10 ' entity='osd.10' 2024-04-02T13:35:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:04 smithi145 ceph-mon[35646]: osd.10 [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410] boot 2024-04-02T13:35:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:04 smithi145 ceph-mon[35646]: osdmap e73: 11 total, 11 up, 11 in 2024-04-02T13:35:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:04.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:04 smithi050 ceph-mon[35524]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-04-02T13:35:04.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:04 smithi050 ceph-mon[35524]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T13:35:04.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:04.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:04.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:04 smithi050 ceph-mon[35524]: from='osd.10 ' entity='osd.10' 2024-04-02T13:35:04.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:04 smithi050 ceph-mon[35524]: osd.10 [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410] boot 2024-04-02T13:35:04.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:04 smithi050 ceph-mon[35524]: osdmap e73: 11 total, 11 up, 11 in 2024-04-02T13:35:04.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:05.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:04 smithi033 ceph-mon[32390]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-04-02T13:35:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:04 smithi033 ceph-mon[32390]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T13:35:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:04 smithi033 ceph-mon[32390]: from='osd.10 ' entity='osd.10' 2024-04-02T13:35:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:04 smithi033 ceph-mon[32390]: osd.10 [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410] boot 2024-04-02T13:35:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:04 smithi033 ceph-mon[32390]: osdmap e73: 11 total, 11 up, 11 in 2024-04-02T13:35:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:05.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi145:vg_nvme/lv_1 2024-04-02T13:35:05.704 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:35:05.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:05 smithi145 ceph-mon[35646]: purged_snaps scrub starts 2024-04-02T13:35:05.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:05 smithi145 ceph-mon[35646]: purged_snaps scrub ok 2024-04-02T13:35:05.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:05 smithi145 ceph-mon[35646]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:35:05.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:05 smithi145 ceph-mon[35646]: osdmap e74: 11 total, 11 up, 11 in 2024-04-02T13:35:05.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:05 smithi050 ceph-mon[35524]: purged_snaps scrub starts 2024-04-02T13:35:05.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:05 smithi050 ceph-mon[35524]: purged_snaps scrub ok 2024-04-02T13:35:05.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:05 smithi050 ceph-mon[35524]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:35:05.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:05 smithi050 ceph-mon[35524]: osdmap e74: 11 total, 11 up, 11 in 2024-04-02T13:35:06.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:05 smithi033 ceph-mon[32390]: purged_snaps scrub starts 2024-04-02T13:35:06.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:05 smithi033 ceph-mon[32390]: purged_snaps scrub ok 2024-04-02T13:35:06.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:05 smithi033 ceph-mon[32390]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:35:06.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:05 smithi033 ceph-mon[32390]: osdmap e74: 11 total, 11 up, 11 in 2024-04-02T13:35:07.159 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:07.159 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: Detected new or changed devices on smithi050 2024-04-02T13:35:07.159 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:07.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:07.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:07.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:07.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:07.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:07.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi050 to 4273M 2024-04-02T13:35:07.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:07.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:07.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:07.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:07.160 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:07 smithi145 ceph-mon[35646]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T13:35:07.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.395+0000 7f5f85aec700 1 -- 172.21.15.145:0/2888808963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f801041b0 msgr2=0x7f5f8010aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:07.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.395+0000 7f5f85aec700 1 --2- 172.21.15.145:0/2888808963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f801041b0 0x7f5f8010aa60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800b0a0 tx=0x7f5f7800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:35:07.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.395+0000 7f5f85aec700 1 -- 172.21.15.145:0/2888808963 shutdown_connections 2024-04-02T13:35:07.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.395+0000 7f5f85aec700 1 --2- 172.21.15.145:0/2888808963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f801041b0 0x7f5f8010aa60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:07.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.395+0000 7f5f85aec700 1 --2- 172.21.15.145:0/2888808963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f80102610 0x7f5f80102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:07.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.395+0000 7f5f85aec700 1 --2- 172.21.15.145:0/2888808963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f80103810 0x7f5f80103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:07.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.395+0000 7f5f85aec700 1 -- 172.21.15.145:0/2888808963 >> 172.21.15.145:0/2888808963 conn(0x7f5f800fdbd0 msgr2=0x7f5f800ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:07.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.395+0000 7f5f85aec700 1 -- 172.21.15.145:0/2888808963 shutdown_connections 2024-04-02T13:35:07.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.395+0000 7f5f85aec700 1 -- 172.21.15.145:0/2888808963 wait complete. 2024-04-02T13:35:07.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.396+0000 7f5f85aec700 1 Processor -- start 2024-04-02T13:35:07.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.396+0000 7f5f85aec700 1 -- start start 2024-04-02T13:35:07.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.396+0000 7f5f85aec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f80102610 0x7f5f8019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:07.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.396+0000 7f5f85aec700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f80103810 0x7f5f8019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:07.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.396+0000 7f5f85aec700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f801041b0 0x7f5f801a2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:07.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.396+0000 7f5f85aec700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8010e2e0 con 0x7f5f80102610 2024-04-02T13:35:07.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.396+0000 7f5f85aec700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8010e160 con 0x7f5f80103810 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.396+0000 7f5f85aec700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8010e460 con 0x7f5f801041b0 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.396+0000 7f5f7ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f801041b0 0x7f5f801a2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f7ffff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f801041b0 0x7f5f801a2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:57614/0 (socket says 172.21.15.145:57614) 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f7ffff700 1 -- 172.21.15.145:0/4157219712 learned_addr learned my addr 172.21.15.145:0/4157219712 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f7f7fe700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f80102610 0x7f5f8019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f7effd700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f80103810 0x7f5f8019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f7ffff700 1 -- 172.21.15.145:0/4157219712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f80103810 msgr2=0x7f5f8019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f7ffff700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f80103810 0x7f5f8019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f7ffff700 1 -- 172.21.15.145:0/4157219712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f80102610 msgr2=0x7f5f8019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f7ffff700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f80102610 0x7f5f8019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f7ffff700 1 -- 172.21.15.145:0/4157219712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f801a26e0 con 0x7f5f801041b0 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f7ffff700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f801041b0 0x7f5f801a2040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009850 tx=0x7f5f7800bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f7cff9700 1 -- 172.21.15.145:0/4157219712 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7801f030 con 0x7f5f801041b0 2024-04-02T13:35:07.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f7cff9700 1 -- 172.21.15.145:0/4157219712 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f78007b40 con 0x7f5f801041b0 2024-04-02T13:35:07.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f7cff9700 1 -- 172.21.15.145:0/4157219712 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f78004850 con 0x7f5f801041b0 2024-04-02T13:35:07.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f85aec700 1 -- 172.21.15.145:0/4157219712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f801a2970 con 0x7f5f801041b0 2024-04-02T13:35:07.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.397+0000 7f5f85aec700 1 -- 172.21.15.145:0/4157219712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f801a2df0 con 0x7f5f801041b0 2024-04-02T13:35:07.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.398+0000 7f5f85aec700 1 -- 172.21.15.145:0/4157219712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f801a2ff0 con 0x7f5f801041b0 2024-04-02T13:35:07.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.399+0000 7f5f7cff9700 1 -- 172.21.15.145:0/4157219712 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5f78004020 con 0x7f5f801041b0 2024-04-02T13:35:07.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.400+0000 7f5f7cff9700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5f680a3c40 0x7f5f680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:07.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.400+0000 7f5f7cff9700 1 -- 172.21.15.145:0/4157219712 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7f5f780d0cc0 con 0x7f5f801041b0 2024-04-02T13:35:07.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.400+0000 7f5f7f7fe700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5f680a3c40 0x7f5f680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:07.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.401+0000 7f5f7f7fe700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5f680a3c40 0x7f5f680a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f80103670 tx=0x7f5f70009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:07.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.404+0000 7f5f7cff9700 1 -- 172.21.15.145:0/4157219712 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5f78007cf0 con 0x7f5f801041b0 2024-04-02T13:35:07.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:07.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: Detected new or changed devices on smithi050 2024-04-02T13:35:07.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:07.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:07.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:07.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:07.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:07.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:07.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi050 to 4273M 2024-04-02T13:35:07.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:07.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:07.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:07.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:07.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:07 smithi050 ceph-mon[35524]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T13:35:07.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:07.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: Detected new or changed devices on smithi050 2024-04-02T13:35:07.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:07.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:07.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:07.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:07.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:07.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:07.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi050 to 4273M 2024-04-02T13:35:07.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:07.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:07.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:07.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:07.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:07 smithi033 ceph-mon[32390]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T13:35:07.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:07.611+0000 7f5f85aec700 1 -- 172.21.15.145:0/4157219712 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5f80075f40 con 0x7f5f680a3c40 2024-04-02T13:35:08.261 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:08 smithi145 ceph-mon[35646]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:08.261 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:08 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:35:08.262 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:08 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:35:08.262 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:08 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:08.262 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:08 smithi145 ceph-mon[35646]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T13:35:08.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:08 smithi050 ceph-mon[35524]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:08.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:08 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:35:08.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:08 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:35:08.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:08 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:08.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:08 smithi050 ceph-mon[35524]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T13:35:08.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:08 smithi033 ceph-mon[32390]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:08.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:08 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:35:08.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:08 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:35:08.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:08 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:08.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:08 smithi033 ceph-mon[32390]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T13:35:09.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:09 smithi145 ceph-mon[35646]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:35:09.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:09 smithi145 ceph-mon[35646]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T13:35:09.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:09 smithi050 ceph-mon[35524]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:35:09.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:09 smithi050 ceph-mon[35524]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T13:35:10.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:09 smithi033 ceph-mon[32390]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:35:10.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:09 smithi033 ceph-mon[32390]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T13:35:10.949 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:10 smithi145 ceph-mon[35646]: from='client.? 172.21.15.145:0/1442300029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b33c049-e9a7-4f62-a514-4cc24a0c4267"}]: dispatch 2024-04-02T13:35:10.949 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:10 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b33c049-e9a7-4f62-a514-4cc24a0c4267"}]: dispatch 2024-04-02T13:35:10.950 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:10 smithi145 ceph-mon[35646]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b33c049-e9a7-4f62-a514-4cc24a0c4267"}]': finished 2024-04-02T13:35:10.950 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:10 smithi145 ceph-mon[35646]: osdmap e78: 12 total, 11 up, 12 in 2024-04-02T13:35:10.950 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:10 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:10.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:10 smithi050 ceph-mon[35524]: from='client.? 172.21.15.145:0/1442300029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b33c049-e9a7-4f62-a514-4cc24a0c4267"}]: dispatch 2024-04-02T13:35:10.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:10 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b33c049-e9a7-4f62-a514-4cc24a0c4267"}]: dispatch 2024-04-02T13:35:10.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:10 smithi050 ceph-mon[35524]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b33c049-e9a7-4f62-a514-4cc24a0c4267"}]': finished 2024-04-02T13:35:10.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:10 smithi050 ceph-mon[35524]: osdmap e78: 12 total, 11 up, 12 in 2024-04-02T13:35:10.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:10 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:11.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:10 smithi033 ceph-mon[32390]: from='client.? 172.21.15.145:0/1442300029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b33c049-e9a7-4f62-a514-4cc24a0c4267"}]: dispatch 2024-04-02T13:35:11.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:10 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b33c049-e9a7-4f62-a514-4cc24a0c4267"}]: dispatch 2024-04-02T13:35:11.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:10 smithi033 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b33c049-e9a7-4f62-a514-4cc24a0c4267"}]': finished 2024-04-02T13:35:11.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:10 smithi033 ceph-mon[32390]: osdmap e78: 12 total, 11 up, 12 in 2024-04-02T13:35:11.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:10 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:11.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:11 smithi145 ceph-mon[35646]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-02T13:35:11.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:11 smithi145 ceph-mon[35646]: from='client.? 172.21.15.145:0/3871260245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:35:11.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:11 smithi050 ceph-mon[35524]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-02T13:35:11.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:11 smithi050 ceph-mon[35524]: from='client.? 172.21.15.145:0/3871260245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:35:12.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:11 smithi033 ceph-mon[32390]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-02T13:35:12.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:11 smithi033 ceph-mon[32390]: from='client.? 172.21.15.145:0/3871260245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:35:13.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:13 smithi145 ceph-mon[35646]: pgmap v246: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-02T13:35:13.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:13 smithi050 ceph-mon[35524]: pgmap v246: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-02T13:35:14.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:13 smithi033 ceph-mon[32390]: pgmap v246: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-02T13:35:15.810 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:15 smithi145 ceph-mon[35646]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:15.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:15 smithi050 ceph-mon[35524]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:16.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:15 smithi033 ceph-mon[32390]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:17.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:17 smithi145 ceph-mon[35646]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:17.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:17 smithi050 ceph-mon[35524]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:18.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:17 smithi033 ceph-mon[32390]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:18.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T13:35:18.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:18.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T13:35:18.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:19.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T13:35:19.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:19.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:19 smithi145 ceph-mon[35646]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:19.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:19 smithi145 ceph-mon[35646]: Deploying daemon osd.11 on smithi145 2024-04-02T13:35:19.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:19 smithi050 ceph-mon[35524]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:19.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:19 smithi050 ceph-mon[35524]: Deploying daemon osd.11 on smithi145 2024-04-02T13:35:20.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:19 smithi033 ceph-mon[32390]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:20.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:19 smithi033 ceph-mon[32390]: Deploying daemon osd.11 on smithi145 2024-04-02T13:35:21.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:21 smithi145 ceph-mon[35646]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:21.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:21 smithi050 ceph-mon[35524]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:22.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:21 smithi033 ceph-mon[32390]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:23.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:23 smithi145 ceph-mon[35646]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:23.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:23 smithi050 ceph-mon[35524]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:24.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:23 smithi033 ceph-mon[32390]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:25.302 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:25 smithi145 ceph-mon[35646]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:25.302 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:25 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:25.302 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:25 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:25.302 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:25 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:25.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:25 smithi050 ceph-mon[35524]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:25.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:25 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:25.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:25 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:25.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:25 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:25.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:25 smithi033 ceph-mon[32390]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:25.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:25 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:25.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:25 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:25.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:25 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:27.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:27 smithi145 ceph-mon[35646]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:27.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:27 smithi050 ceph-mon[35524]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:27.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:27 smithi033 ceph-mon[32390]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:28.133 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 11 on host 'smithi145' 2024-04-02T13:35:28.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:28.129+0000 7f5f7cff9700 1 -- 172.21.15.145:0/4157219712 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5f80075f40 con 0x7f5f680a3c40 2024-04-02T13:35:28.135 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:28.132+0000 7f5f85aec700 1 -- 172.21.15.145:0/4157219712 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5f680a3c40 msgr2=0x7f5f680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:28.135 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:28.132+0000 7f5f85aec700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5f680a3c40 0x7f5f680a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f80103670 tx=0x7f5f70009190 comp rx=0 tx=0).stop 2024-04-02T13:35:28.135 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:28.132+0000 7f5f85aec700 1 -- 172.21.15.145:0/4157219712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f801041b0 msgr2=0x7f5f801a2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:28.135 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:28.132+0000 7f5f85aec700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f801041b0 0x7f5f801a2040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009850 tx=0x7f5f7800bdf0 comp rx=0 tx=0).stop 2024-04-02T13:35:28.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:28.132+0000 7f5f85aec700 1 -- 172.21.15.145:0/4157219712 shutdown_connections 2024-04-02T13:35:28.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:28.132+0000 7f5f85aec700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5f680a3c40 0x7f5f680a6100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:28.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:28.132+0000 7f5f85aec700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f801041b0 0x7f5f801a2040 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:28.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:28.132+0000 7f5f85aec700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f80102610 0x7f5f8019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:28.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:28.132+0000 7f5f85aec700 1 --2- 172.21.15.145:0/4157219712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f80103810 0x7f5f8019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:28.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:28.132+0000 7f5f85aec700 1 -- 172.21.15.145:0/4157219712 >> 172.21.15.145:0/4157219712 conn(0x7f5f800fdbd0 msgr2=0x7f5f80074820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:28.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:28.132+0000 7f5f85aec700 1 -- 172.21.15.145:0/4157219712 shutdown_connections 2024-04-02T13:35:28.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:28.132+0000 7f5f85aec700 1 -- 172.21.15.145:0/4157219712 wait complete. 2024-04-02T13:35:28.454 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:28 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.454 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:28 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.455 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:28 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:28.455 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:28 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:28.455 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:28 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.455 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:28 smithi145 ceph-mon[35646]: from='osd.11 [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:35:28.455 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:28 smithi145 ceph-mon[35646]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:35:28.455 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:28 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:28.455 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:28 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.456 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:28 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:28 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:28 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:28 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:28.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:28 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:28.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:28 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:28 smithi050 ceph-mon[35524]: from='osd.11 [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:35:28.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:28 smithi050 ceph-mon[35524]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:35:28.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:28 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:28.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:28 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:28 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:28.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:28.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:28 smithi033 ceph-mon[32390]: from='osd.11 [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:35:28.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:28 smithi033 ceph-mon[32390]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:35:28.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:28.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:28 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:28.776 DEBUG:teuthology.orchestra.run.smithi145:osd.11> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.11.service 2024-04-02T13:35:28.778 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-02T13:35:28.779 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-04-02T13:35:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:35:29.468 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:29 smithi033 ceph-mon[32390]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:29.468 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:29 smithi033 ceph-mon[32390]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T13:35:29.468 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:29 smithi033 ceph-mon[32390]: osdmap e79: 12 total, 11 up, 12 in 2024-04-02T13:35:29.468 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:29 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:29.468 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:29 smithi033 ceph-mon[32390]: from='osd.11 [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:35:29.468 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:29 smithi033 ceph-mon[32390]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:35:29.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:29 smithi145 ceph-mon[35646]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:29.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:29 smithi145 ceph-mon[35646]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T13:35:29.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:29 smithi145 ceph-mon[35646]: osdmap e79: 12 total, 11 up, 12 in 2024-04-02T13:35:29.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:29 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:29.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:29 smithi145 ceph-mon[35646]: from='osd.11 [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:35:29.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:29 smithi145 ceph-mon[35646]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:35:29.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:29 smithi050 ceph-mon[35524]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:29.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:29 smithi050 ceph-mon[35524]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T13:35:29.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:29 smithi050 ceph-mon[35524]: osdmap e79: 12 total, 11 up, 12 in 2024-04-02T13:35:29.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:29 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:29.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:29 smithi050 ceph-mon[35524]: from='osd.11 [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:35:29.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:29 smithi050 ceph-mon[35524]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-04-02T13:35:29.979 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:35:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:35:29.637+0000 7fc5421e0700 -1 osd.11 0 waiting for initial osdmap 2024-04-02T13:35:29.979 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:35:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:35:29.649+0000 7fc5397ca700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:35:30.309 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:30 smithi145 ceph-mon[35646]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-04-02T13:35:30.309 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:30 smithi145 ceph-mon[35646]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T13:35:30.309 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:30.309 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:30 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:30.309 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:30 smithi145 ceph-mon[35646]: from='osd.11 ' entity='osd.11' 2024-04-02T13:35:30.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:30 smithi033 ceph-mon[32390]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-04-02T13:35:30.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:30 smithi033 ceph-mon[32390]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T13:35:30.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:30.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:30 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:30.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:30 smithi033 ceph-mon[32390]: from='osd.11 ' entity='osd.11' 2024-04-02T13:35:30.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:30 smithi050 ceph-mon[35524]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-04-02T13:35:30.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:30 smithi050 ceph-mon[35524]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T13:35:30.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:30.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:30 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:30.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:30 smithi050 ceph-mon[35524]: from='osd.11 ' entity='osd.11' 2024-04-02T13:35:30.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.996+0000 7f084993f700 1 -- 172.21.15.33:0/1098106302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844102620 msgr2=0x7f0844102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:30.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.996+0000 7f084993f700 1 --2- 172.21.15.33:0/1098106302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844102620 0x7f0844102a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f083400b0d0 tx=0x7f083400b3e0 comp rx=0 tx=0).stop 2024-04-02T13:35:30.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.996+0000 7f084993f700 1 -- 172.21.15.33:0/1098106302 shutdown_connections 2024-04-02T13:35:30.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.996+0000 7f084993f700 1 --2- 172.21.15.33:0/1098106302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08441041c0 0x7f084410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:30.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.996+0000 7f084993f700 1 --2- 172.21.15.33:0/1098106302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844102620 0x7f0844102a40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:30.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.996+0000 7f084993f700 1 --2- 172.21.15.33:0/1098106302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0844103820 0x7f0844103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:30.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.996+0000 7f084993f700 1 -- 172.21.15.33:0/1098106302 >> 172.21.15.33:0/1098106302 conn(0x7f08440fdc00 msgr2=0x7f0844100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:30.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.996+0000 7f084993f700 1 -- 172.21.15.33:0/1098106302 shutdown_connections 2024-04-02T13:35:30.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.996+0000 7f084993f700 1 -- 172.21.15.33:0/1098106302 wait complete. 2024-04-02T13:35:31.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.996+0000 7f084993f700 1 Processor -- start 2024-04-02T13:35:31.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.996+0000 7f084993f700 1 -- start start 2024-04-02T13:35:31.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f084993f700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0844102620 0x7f0844198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:31.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f084993f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0844103820 0x7f0844198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:31.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f084993f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08441041c0 0x7f084419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:31.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f084993f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f084410e240 con 0x7f08441041c0 2024-04-02T13:35:31.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f084993f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f084410e0c0 con 0x7f0844103820 2024-04-02T13:35:31.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f084993f700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f084410e3c0 con 0x7f0844102620 2024-04-02T13:35:31.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f0842ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0844102620 0x7f0844198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:31.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f08437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08441041c0 0x7f084419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f0842ffd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0844102620 0x7f0844198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:36564/0 (socket says 172.21.15.33:36564) 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f08437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08441041c0 0x7f084419fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34190/0 (socket says 172.21.15.33:34190) 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f0842ffd700 1 -- 172.21.15.33:0/2848977565 learned_addr learned my addr 172.21.15.33:0/2848977565 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f08427fc700 1 --2- 172.21.15.33:0/2848977565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0844103820 0x7f0844198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f0842ffd700 1 -- 172.21.15.33:0/2848977565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0844103820 msgr2=0x7f0844198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f0842ffd700 1 --2- 172.21.15.33:0/2848977565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0844103820 0x7f0844198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f0842ffd700 1 -- 172.21.15.33:0/2848977565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08441041c0 msgr2=0x7f084419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f0842ffd700 1 --2- 172.21.15.33:0/2848977565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08441041c0 0x7f084419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f0842ffd700 1 -- 172.21.15.33:0/2848977565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08441a04e0 con 0x7f0844102620 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.997+0000 7f0842ffd700 1 --2- 172.21.15.33:0/2848977565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0844102620 0x7f0844198880 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0834009850 tx=0x7f0834007a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.998+0000 7f084893d700 1 -- 172.21.15.33:0/2848977565 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834003c00 con 0x7f0844102620 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.998+0000 7f084993f700 1 -- 172.21.15.33:0/2848977565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08441a0770 con 0x7f0844102620 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.998+0000 7f084893d700 1 -- 172.21.15.33:0/2848977565 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0834022070 con 0x7f0844102620 2024-04-02T13:35:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.998+0000 7f084893d700 1 -- 172.21.15.33:0/2848977565 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083401b680 con 0x7f0844102620 2024-04-02T13:35:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.998+0000 7f084993f700 1 -- 172.21.15.33:0/2848977565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08441a0c20 con 0x7f0844102620 2024-04-02T13:35:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:30.999+0000 7f084993f700 1 -- 172.21.15.33:0/2848977565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08440692b0 con 0x7f0844102620 2024-04-02T13:35:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.000+0000 7f084893d700 1 -- 172.21.15.33:0/2848977565 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f083401d9d0 con 0x7f0844102620 2024-04-02T13:35:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.000+0000 7f084893d700 1 --2- 172.21.15.33:0/2848977565 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f08300a3c40 0x7f08300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.000+0000 7f08427fc700 1 --2- 172.21.15.33:0/2848977565 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f08300a3c40 0x7f08300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.000+0000 7f084893d700 1 -- 172.21.15.33:0/2848977565 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f083400f020 con 0x7f0844102620 2024-04-02T13:35:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.001+0000 7f08427fc700 1 --2- 172.21.15.33:0/2848977565 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f08300a3c40 0x7f08300a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f082c009c60 tx=0x7f082c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:31.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.004+0000 7f084893d700 1 -- 172.21.15.33:0/2848977565 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f083409db80 con 0x7f0844102620 2024-04-02T13:35:31.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.213+0000 7f084993f700 1 -- 172.21.15.33:0/2848977565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f084402dd30 con 0x7f0844102620 2024-04-02T13:35:31.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.214+0000 7f084893d700 1 -- 172.21.15.33:0/2848977565 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f083409d480 con 0x7f0844102620 2024-04-02T13:35:31.217 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:35:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.216+0000 7f084993f700 1 -- 172.21.15.33:0/2848977565 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f08300a3c40 msgr2=0x7f08300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.216+0000 7f084993f700 1 --2- 172.21.15.33:0/2848977565 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f08300a3c40 0x7f08300a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f082c009c60 tx=0x7f082c009260 comp rx=0 tx=0).stop 2024-04-02T13:35:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.216+0000 7f084993f700 1 -- 172.21.15.33:0/2848977565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0844102620 msgr2=0x7f0844198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.216+0000 7f084993f700 1 --2- 172.21.15.33:0/2848977565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0844102620 0x7f0844198880 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0834009850 tx=0x7f0834007a20 comp rx=0 tx=0).stop 2024-04-02T13:35:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.216+0000 7f084993f700 1 -- 172.21.15.33:0/2848977565 shutdown_connections 2024-04-02T13:35:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.216+0000 7f084993f700 1 --2- 172.21.15.33:0/2848977565 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f08300a3c40 0x7f08300a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.216+0000 7f084993f700 1 --2- 172.21.15.33:0/2848977565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08441041c0 0x7f084419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.216+0000 7f084993f700 1 --2- 172.21.15.33:0/2848977565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0844102620 0x7f0844198880 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.216+0000 7f084993f700 1 --2- 172.21.15.33:0/2848977565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0844103820 0x7f0844198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.216+0000 7f084993f700 1 -- 172.21.15.33:0/2848977565 >> 172.21.15.33:0/2848977565 conn(0x7f08440fdc00 msgr2=0x7f0844106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.217+0000 7f084993f700 1 -- 172.21.15.33:0/2848977565 shutdown_connections 2024-04-02T13:35:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:31.217+0000 7f084993f700 1 -- 172.21.15.33:0/2848977565 wait complete. 2024-04-02T13:35:31.472 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:31 smithi145 ceph-mon[35646]: purged_snaps scrub starts 2024-04-02T13:35:31.472 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:31 smithi145 ceph-mon[35646]: purged_snaps scrub ok 2024-04-02T13:35:31.472 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:31 smithi145 ceph-mon[35646]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:31.472 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:31 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:31.472 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:31 smithi145 ceph-mon[35646]: osd.11 [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123] boot 2024-04-02T13:35:31.473 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:31 smithi145 ceph-mon[35646]: osdmap e81: 12 total, 12 up, 12 in 2024-04-02T13:35:31.473 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:31 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:31.473 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:31 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2848977565' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T13:35:31.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:31 smithi033 ceph-mon[32390]: purged_snaps scrub starts 2024-04-02T13:35:31.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:31 smithi033 ceph-mon[32390]: purged_snaps scrub ok 2024-04-02T13:35:31.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:31 smithi033 ceph-mon[32390]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:31.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:31 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:31.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:31 smithi033 ceph-mon[32390]: osd.11 [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123] boot 2024-04-02T13:35:31.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:31 smithi033 ceph-mon[32390]: osdmap e81: 12 total, 12 up, 12 in 2024-04-02T13:35:31.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:31 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:31.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:31 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2848977565' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T13:35:31.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:31 smithi050 ceph-mon[35524]: purged_snaps scrub starts 2024-04-02T13:35:31.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:31 smithi050 ceph-mon[35524]: purged_snaps scrub ok 2024-04-02T13:35:31.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:31 smithi050 ceph-mon[35524]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:35:31.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:31 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:31.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:31 smithi050 ceph-mon[35524]: osd.11 [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123] boot 2024-04-02T13:35:31.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:31 smithi050 ceph-mon[35524]: osdmap e81: 12 total, 12 up, 12 in 2024-04-02T13:35:31.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:31 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:31.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:31 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2848977565' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T13:35:32.118 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1712064930,"num_in_osds":12,"osd_in_since":1712064910,"num_remapped_pgs":0} 2024-04-02T13:35:32.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T13:35:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:35:32.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:32 smithi033 ceph-mon[32390]: osdmap e82: 12 total, 12 up, 12 in 2024-04-02T13:35:32.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:32.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:32.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:32.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:32.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:32.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:32.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:32.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:32.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:32.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:32 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:32 smithi145 ceph-mon[35646]: osdmap e82: 12 total, 12 up, 12 in 2024-04-02T13:35:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:32.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:32.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:32.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:32.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:32.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:32.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:32.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:32 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:32 smithi050 ceph-mon[35524]: osdmap e82: 12 total, 12 up, 12 in 2024-04-02T13:35:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:32.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:32 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:33.854 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:33 smithi033 ceph-mon[32390]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:33.854 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:33 smithi033 ceph-mon[32390]: Detected new or changed devices on smithi145 2024-04-02T13:35:33.854 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:33 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi145 to 4272M 2024-04-02T13:35:33.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:33 smithi145 ceph-mon[35646]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:33.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:33 smithi145 ceph-mon[35646]: Detected new or changed devices on smithi145 2024-04-02T13:35:33.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:33 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi145 to 4272M 2024-04-02T13:35:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:33 smithi050 ceph-mon[35524]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:33 smithi050 ceph-mon[35524]: Detected new or changed devices on smithi145 2024-04-02T13:35:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:33 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi145 to 4272M 2024-04-02T13:35:34.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.101+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1913996086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bf0108630 msgr2=0x7f1bf0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:34.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.101+0000 7f1bf4b93700 1 --2- 172.21.15.33:0/1913996086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bf0108630 0x7f1bf0108a10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009990 tx=0x7f1be8009ca0 comp rx=0 tx=0).stop 2024-04-02T13:35:34.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.101+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1913996086 shutdown_connections 2024-04-02T13:35:34.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.101+0000 7f1bf4b93700 1 --2- 172.21.15.33:0/1913996086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bf006b4f0 0x7f1bf010f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:34.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.101+0000 7f1bf4b93700 1 --2- 172.21.15.33:0/1913996086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bf0108630 0x7f1bf0108a10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:34.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.101+0000 7f1bf4b93700 1 --2- 172.21.15.33:0/1913996086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bf006ab30 0x7f1bf006afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:34.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.101+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1913996086 >> 172.21.15.33:0/1913996086 conn(0x7f1bf0077540 msgr2=0x7f1bf0077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:34.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.101+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1913996086 shutdown_connections 2024-04-02T13:35:34.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.101+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1913996086 wait complete. 2024-04-02T13:35:34.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.102+0000 7f1bf4b93700 1 Processor -- start 2024-04-02T13:35:34.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.102+0000 7f1bf4b93700 1 -- start start 2024-04-02T13:35:34.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.102+0000 7f1bf4b93700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bf006ab30 0x7f1bf019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:34.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.102+0000 7f1bf4b93700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bf006b4f0 0x7f1bf019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:34.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.102+0000 7f1bf4b93700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bf0108630 0x7f1bf0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:34.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.102+0000 7f1bf4b93700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf01128a0 con 0x7f1bf006b4f0 2024-04-02T13:35:34.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.102+0000 7f1bf4b93700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf0112720 con 0x7f1bf0108630 2024-04-02T13:35:34.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.102+0000 7f1bf4b93700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf0112a20 con 0x7f1bf006ab30 2024-04-02T13:35:34.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.102+0000 7f1bedd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bf006b4f0 0x7f1bf019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:34.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.102+0000 7f1bedd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bf006b4f0 0x7f1bf019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57560/0 (socket says 172.21.15.33:57560) 2024-04-02T13:35:34.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.102+0000 7f1bedd9b700 1 -- 172.21.15.33:0/1998027338 learned_addr learned my addr 172.21.15.33:0/1998027338 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:35:34.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.102+0000 7f1beed9d700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bf0108630 0x7f1bf0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:34.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.103+0000 7f1bee59c700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bf006ab30 0x7f1bf019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:34.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.103+0000 7f1bedd9b700 1 -- 172.21.15.33:0/1998027338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bf006ab30 msgr2=0x7f1bf019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:34.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.103+0000 7f1bedd9b700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bf006ab30 0x7f1bf019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:34.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.103+0000 7f1bedd9b700 1 -- 172.21.15.33:0/1998027338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bf0108630 msgr2=0x7f1bf0198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:34.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.103+0000 7f1bedd9b700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bf0108630 0x7f1bf0198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:34.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.103+0000 7f1bedd9b700 1 -- 172.21.15.33:0/1998027338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf0199460 con 0x7f1bf006b4f0 2024-04-02T13:35:34.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.103+0000 7f1bee59c700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bf006ab30 0x7f1bf019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:34.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.103+0000 7f1bedd9b700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bf006b4f0 0x7f1bf019efa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800d950 tx=0x7f1bd800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:34.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.103+0000 7f1be77fe700 1 -- 172.21.15.33:0/1998027338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8016070 con 0x7f1bf006b4f0 2024-04-02T13:35:34.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.103+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1998027338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf0199750 con 0x7f1bf006b4f0 2024-04-02T13:35:34.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.103+0000 7f1be77fe700 1 -- 172.21.15.33:0/1998027338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1bd8009e10 con 0x7f1bf006b4f0 2024-04-02T13:35:34.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.103+0000 7f1be77fe700 1 -- 172.21.15.33:0/1998027338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd800f620 con 0x7f1bf006b4f0 2024-04-02T13:35:34.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.103+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1998027338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf01a59e0 con 0x7f1bf006b4f0 2024-04-02T13:35:34.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.104+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1998027338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf0103d00 con 0x7f1bf006b4f0 2024-04-02T13:35:34.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.105+0000 7f1be77fe700 1 -- 172.21.15.33:0/1998027338 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1bd8019070 con 0x7f1bf006b4f0 2024-04-02T13:35:34.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.106+0000 7f1be77fe700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1bdc0a3c40 0x7f1bdc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:34.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.106+0000 7f1bee59c700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1bdc0a3c40 0x7f1bdc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:34.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.106+0000 7f1be77fe700 1 -- 172.21.15.33:0/1998027338 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1bd80cfb20 con 0x7f1bf006b4f0 2024-04-02T13:35:34.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.107+0000 7f1bee59c700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1bdc0a3c40 0x7f1bdc0a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1be8006010 tx=0x7f1be800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:34.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.111+0000 7f1be77fe700 1 -- 172.21.15.33:0/1998027338 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1bd809c9a0 con 0x7f1bf006b4f0 2024-04-02T13:35:34.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.317+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1998027338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bf0199ff0 con 0x7f1bf006b4f0 2024-04-02T13:35:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.318+0000 7f1be77fe700 1 -- 172.21.15.33:0/1998027338 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18385 (secure 0 0 0) 0x7f1bd809c2a0 con 0x7f1bf006b4f0 2024-04-02T13:35:34.322 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:35:34.322 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:35:31.648681+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:35:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.320+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1998027338 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1bdc0a3c40 msgr2=0x7f1bdc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.320+0000 7f1bf4b93700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1bdc0a3c40 0x7f1bdc0a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1be8006010 tx=0x7f1be800b540 comp rx=0 tx=0).stop 2024-04-02T13:35:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.320+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1998027338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bf006b4f0 msgr2=0x7f1bf019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.320+0000 7f1bf4b93700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bf006b4f0 0x7f1bf019efa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800d950 tx=0x7f1bd800dc60 comp rx=0 tx=0).stop 2024-04-02T13:35:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.320+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1998027338 shutdown_connections 2024-04-02T13:35:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.320+0000 7f1bf4b93700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1bdc0a3c40 0x7f1bdc0a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.320+0000 7f1bf4b93700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bf0108630 0x7f1bf0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.320+0000 7f1bf4b93700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bf006ab30 0x7f1bf019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.320+0000 7f1bf4b93700 1 --2- 172.21.15.33:0/1998027338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bf006b4f0 0x7f1bf019efa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.320+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1998027338 >> 172.21.15.33:0/1998027338 conn(0x7f1bf0077540 msgr2=0x7f1bf01001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.320+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1998027338 shutdown_connections 2024-04-02T13:35:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:34.320+0000 7f1bf4b93700 1 -- 172.21.15.33:0/1998027338 wait complete. 2024-04-02T13:35:34.801 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:34 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1998027338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:34.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:34 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1998027338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:34 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1998027338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:35.153 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T13:35:35.154 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd pool get .mgr pg_num 2024-04-02T13:35:35.402 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:35:35.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:35 smithi033 ceph-mon[32390]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:35.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:35 smithi145 ceph-mon[35646]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:35 smithi050 ceph-mon[35524]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.272+0000 7fafa21a6700 1 -- 172.21.15.33:0/521630326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf9c103060 msgr2=0x7faf9c10f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:37.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.272+0000 7fafa21a6700 1 --2- 172.21.15.33:0/521630326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf9c103060 0x7faf9c10f450 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf9400b0d0 tx=0x7faf9400b3e0 comp rx=0 tx=0).stop 2024-04-02T13:35:37.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.272+0000 7fafa21a6700 1 -- 172.21.15.33:0/521630326 shutdown_connections 2024-04-02T13:35:37.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.272+0000 7fafa21a6700 1 --2- 172.21.15.33:0/521630326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf9c103060 0x7faf9c10f450 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:37.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.272+0000 7fafa21a6700 1 --2- 172.21.15.33:0/521630326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c1086f0 0x7faf9c108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:37.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.272+0000 7fafa21a6700 1 --2- 172.21.15.33:0/521630326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf9c1026a0 0x7faf9c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:37.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.272+0000 7fafa21a6700 1 -- 172.21.15.33:0/521630326 >> 172.21.15.33:0/521630326 conn(0x7faf9c0796c0 msgr2=0x7faf9c079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:37.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.272+0000 7fafa21a6700 1 -- 172.21.15.33:0/521630326 shutdown_connections 2024-04-02T13:35:37.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.272+0000 7fafa21a6700 1 -- 172.21.15.33:0/521630326 wait complete. 2024-04-02T13:35:37.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.272+0000 7fafa21a6700 1 Processor -- start 2024-04-02T13:35:37.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.272+0000 7fafa21a6700 1 -- start start 2024-04-02T13:35:37.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7fafa21a6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf9c1026a0 0x7faf9c0748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:37.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7fafa21a6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf9c103060 0x7faf9c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:37.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7fafa21a6700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c1086f0 0x7faf9c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:37.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7fafa21a6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c112900 con 0x7faf9c1026a0 2024-04-02T13:35:37.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7fafa21a6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c112780 con 0x7faf9c103060 2024-04-02T13:35:37.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7fafa21a6700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c112a80 con 0x7faf9c1086f0 2024-04-02T13:35:37.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7faf9b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf9c1026a0 0x7faf9c0748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:37.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7faf9b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf9c1026a0 0x7faf9c0748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57596/0 (socket says 172.21.15.33:57596) 2024-04-02T13:35:37.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7faf9b7fe700 1 -- 172.21.15.33:0/70264763 learned_addr learned my addr 172.21.15.33:0/70264763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7faf9affd700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf9c103060 0x7faf9c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7faf9bfff700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c1086f0 0x7faf9c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7faf9b7fe700 1 -- 172.21.15.33:0/70264763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c1086f0 msgr2=0x7faf9c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7faf9b7fe700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c1086f0 0x7faf9c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7faf9b7fe700 1 -- 172.21.15.33:0/70264763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf9c103060 msgr2=0x7faf9c074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7faf9b7fe700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf9c103060 0x7faf9c074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7faf9b7fe700 1 -- 172.21.15.33:0/70264763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9c1a71b0 con 0x7faf9c1026a0 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.273+0000 7faf9b7fe700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf9c1026a0 0x7faf9c0748f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faf9000c970 tx=0x7faf9000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.274+0000 7faf98ff9700 1 -- 172.21.15.33:0/70264763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf900078d0 con 0x7faf9c1026a0 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.274+0000 7fafa21a6700 1 -- 172.21.15.33:0/70264763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9c1a74a0 con 0x7faf9c1026a0 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.274+0000 7faf98ff9700 1 -- 172.21.15.33:0/70264763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7faf90018480 con 0x7faf9c1026a0 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.274+0000 7faf98ff9700 1 -- 172.21.15.33:0/70264763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9000f6b0 con 0x7faf9c1026a0 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.274+0000 7fafa21a6700 1 -- 172.21.15.33:0/70264763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9c1a79e0 con 0x7faf9c1026a0 2024-04-02T13:35:37.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.275+0000 7fafa21a6700 1 -- 172.21.15.33:0/70264763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf9c0692b0 con 0x7faf9c1026a0 2024-04-02T13:35:37.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.276+0000 7faf98ff9700 1 -- 172.21.15.33:0/70264763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faf90018630 con 0x7faf9c1026a0 2024-04-02T13:35:37.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.276+0000 7faf98ff9700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7faf840a3c40 0x7faf840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:37.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.276+0000 7faf9affd700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7faf840a3c40 0x7faf840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:37.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.276+0000 7faf98ff9700 1 -- 172.21.15.33:0/70264763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7faf900cffc0 con 0x7faf9c1026a0 2024-04-02T13:35:37.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.277+0000 7faf9affd700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7faf840a3c40 0x7faf840a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faf9c079ca0 tx=0x7faf8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:37.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.281+0000 7faf98ff9700 1 -- 172.21.15.33:0/70264763 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7faf9009ced0 con 0x7faf9c1026a0 2024-04-02T13:35:37.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.485+0000 7fafa21a6700 1 -- 172.21.15.33:0/70264763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faf9c075dd0 con 0x7faf9c1026a0 2024-04-02T13:35:37.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.485+0000 7faf98ff9700 1 -- 172.21.15.33:0/70264763 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7faf9009c7d0 con 0x7faf9c1026a0 2024-04-02T13:35:37.978 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:35:37.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.487+0000 7fafa21a6700 1 -- 172.21.15.33:0/70264763 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7faf840a3c40 msgr2=0x7faf840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:37.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.487+0000 7fafa21a6700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7faf840a3c40 0x7faf840a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faf9c079ca0 tx=0x7faf8c009260 comp rx=0 tx=0).stop 2024-04-02T13:35:37.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.487+0000 7fafa21a6700 1 -- 172.21.15.33:0/70264763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf9c1026a0 msgr2=0x7faf9c0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:37.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.487+0000 7fafa21a6700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf9c1026a0 0x7faf9c0748f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faf9000c970 tx=0x7faf9000cc80 comp rx=0 tx=0).stop 2024-04-02T13:35:37.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.487+0000 7fafa21a6700 1 -- 172.21.15.33:0/70264763 shutdown_connections 2024-04-02T13:35:37.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.487+0000 7fafa21a6700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7faf840a3c40 0x7faf840a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:37.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.487+0000 7fafa21a6700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf9c1086f0 0x7faf9c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:37.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.487+0000 7fafa21a6700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf9c1026a0 0x7faf9c0748f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:37.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.487+0000 7fafa21a6700 1 --2- 172.21.15.33:0/70264763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf9c103060 0x7faf9c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:37.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.487+0000 7fafa21a6700 1 -- 172.21.15.33:0/70264763 >> 172.21.15.33:0/70264763 conn(0x7faf9c0796c0 msgr2=0x7faf9c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:37.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.488+0000 7fafa21a6700 1 -- 172.21.15.33:0/70264763 shutdown_connections 2024-04-02T13:35:37.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:37.488+0000 7fafa21a6700 1 -- 172.21.15.33:0/70264763 wait complete. 2024-04-02T13:35:37.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:37 smithi145 ceph-mon[35646]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:37.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:37 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/70264763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:37.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:37 smithi050 ceph-mon[35524]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:37.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:37 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/70264763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:38.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:37 smithi033 ceph-mon[32390]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:38.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:37 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/70264763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:38.323 INFO:tasks.cephadm:Adding mds.a on smithi033 2024-04-02T13:35:38.323 INFO:tasks.cephadm:Adding mds.d on smithi033 2024-04-02T13:35:38.323 INFO:tasks.cephadm:Adding mds.g on smithi033 2024-04-02T13:35:38.323 INFO:tasks.cephadm:Adding mds.j on smithi033 2024-04-02T13:35:38.323 INFO:tasks.cephadm:Adding mds.b on smithi050 2024-04-02T13:35:38.323 INFO:tasks.cephadm:Adding mds.e on smithi050 2024-04-02T13:35:38.323 INFO:tasks.cephadm:Adding mds.h on smithi050 2024-04-02T13:35:38.323 INFO:tasks.cephadm:Adding mds.k on smithi050 2024-04-02T13:35:38.323 INFO:tasks.cephadm:Adding mds.c on smithi145 2024-04-02T13:35:38.323 INFO:tasks.cephadm:Adding mds.f on smithi145 2024-04-02T13:35:38.323 INFO:tasks.cephadm:Adding mds.i on smithi145 2024-04-02T13:35:38.323 INFO:tasks.cephadm:Adding mds.l on smithi145 2024-04-02T13:35:38.323 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch apply mds all '12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l' 2024-04-02T13:35:38.576 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:35:39.933 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:39 smithi145 ceph-mon[35646]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:39.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:39 smithi050 ceph-mon[35524]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:40.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:39 smithi033 ceph-mon[32390]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:40.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.253+0000 7fa142637700 1 -- 172.21.15.145:0/505748531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa13c102610 msgr2=0x7fa13c102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:40.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.253+0000 7fa142637700 1 --2- 172.21.15.145:0/505748531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa13c102610 0x7fa13c102a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa130009990 tx=0x7fa130009ca0 comp rx=0 tx=0).stop 2024-04-02T13:35:40.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.253+0000 7fa142637700 1 -- 172.21.15.145:0/505748531 shutdown_connections 2024-04-02T13:35:40.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.253+0000 7fa142637700 1 --2- 172.21.15.145:0/505748531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c1041b0 0x7fa13c10aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:40.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.253+0000 7fa142637700 1 --2- 172.21.15.145:0/505748531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa13c102610 0x7fa13c102a30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:40.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.253+0000 7fa142637700 1 --2- 172.21.15.145:0/505748531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa13c103810 0x7fa13c103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:40.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.253+0000 7fa142637700 1 -- 172.21.15.145:0/505748531 >> 172.21.15.145:0/505748531 conn(0x7fa13c0fdbd0 msgr2=0x7fa13c0ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:40.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.253+0000 7fa142637700 1 -- 172.21.15.145:0/505748531 shutdown_connections 2024-04-02T13:35:40.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.253+0000 7fa142637700 1 -- 172.21.15.145:0/505748531 wait complete. 2024-04-02T13:35:40.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.254+0000 7fa142637700 1 Processor -- start 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.254+0000 7fa142637700 1 -- start start 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.254+0000 7fa142637700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa13c102610 0x7fa13c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.254+0000 7fa142637700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa13c103810 0x7fa13c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.254+0000 7fa142637700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c1041b0 0x7fa13c19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.254+0000 7fa142637700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c10e1e0 con 0x7fa13c103810 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.254+0000 7fa142637700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c10e060 con 0x7fa13c1041b0 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.254+0000 7fa142637700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c10e360 con 0x7fa13c102610 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.254+0000 7fa13bfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa13c102610 0x7fa13c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.254+0000 7fa13bfff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa13c102610 0x7fa13c198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43986/0 (socket says 172.21.15.145:43986) 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.254+0000 7fa13bfff700 1 -- 172.21.15.145:0/343310766 learned_addr learned my addr 172.21.15.145:0/343310766 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.254+0000 7fa13b7fe700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa13c103810 0x7fa13c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.254+0000 7fa140bd4700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c1041b0 0x7fa13c19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.255+0000 7fa13bfff700 1 -- 172.21.15.145:0/343310766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c1041b0 msgr2=0x7fa13c19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:40.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.255+0000 7fa13bfff700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c1041b0 0x7fa13c19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.255+0000 7fa13bfff700 1 -- 172.21.15.145:0/343310766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa13c103810 msgr2=0x7fa13c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.255+0000 7fa13bfff700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa13c103810 0x7fa13c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.255+0000 7fa13bfff700 1 -- 172.21.15.145:0/343310766 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c1a04b0 con 0x7fa13c102610 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.255+0000 7fa13bfff700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa13c102610 0x7fa13c198850 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa130000c00 tx=0x7fa1300057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.255+0000 7fa1397fa700 1 -- 172.21.15.145:0/343310766 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130027070 con 0x7fa13c102610 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.255+0000 7fa1397fa700 1 -- 172.21.15.145:0/343310766 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa1300041b0 con 0x7fa13c102610 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.255+0000 7fa142637700 1 -- 172.21.15.145:0/343310766 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13c1a0740 con 0x7fa13c102610 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.255+0000 7fa1397fa700 1 -- 172.21.15.145:0/343310766 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13000f460 con 0x7fa13c102610 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.255+0000 7fa142637700 1 -- 172.21.15.145:0/343310766 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13c1a0b90 con 0x7fa13c102610 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.256+0000 7fa142637700 1 -- 172.21.15.145:0/343310766 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13c0692b0 con 0x7fa13c102610 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.257+0000 7fa1397fa700 1 -- 172.21.15.145:0/343310766 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa13002a070 con 0x7fa13c102610 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.258+0000 7fa1397fa700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa1240a3c40 0x7fa1240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.258+0000 7fa1397fa700 1 -- 172.21.15.145:0/343310766 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa1300d0be0 con 0x7fa13c102610 2024-04-02T13:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.258+0000 7fa13b7fe700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa1240a3c40 0x7fa1240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:40.264 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.258+0000 7fa13b7fe700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa1240a3c40 0x7fa1240a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa12c005950 tx=0x7fa12c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:40.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.262+0000 7fa1397fa700 1 -- 172.21.15.145:0/343310766 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa130013070 con 0x7fa13c102610 2024-04-02T13:35:40.474 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.471+0000 7fa142637700 1 -- 172.21.15.145:0/343310766 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}) -- 0x7fa13c062cf0 con 0x7fa1240a3c40 2024-04-02T13:35:40.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.479+0000 7fa1397fa700 1 -- 172.21.15.145:0/343310766 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fa13c062cf0 con 0x7fa1240a3c40 2024-04-02T13:35:40.482 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mds.all update... 2024-04-02T13:35:40.485 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.482+0000 7fa142637700 1 -- 172.21.15.145:0/343310766 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa1240a3c40 msgr2=0x7fa1240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:40.485 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.482+0000 7fa142637700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa1240a3c40 0x7fa1240a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa12c005950 tx=0x7fa12c00b410 comp rx=0 tx=0).stop 2024-04-02T13:35:40.485 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.482+0000 7fa142637700 1 -- 172.21.15.145:0/343310766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa13c102610 msgr2=0x7fa13c198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:40.485 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.482+0000 7fa142637700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa13c102610 0x7fa13c198850 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa130000c00 tx=0x7fa1300057c0 comp rx=0 tx=0).stop 2024-04-02T13:35:40.485 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.482+0000 7fa142637700 1 -- 172.21.15.145:0/343310766 shutdown_connections 2024-04-02T13:35:40.485 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.482+0000 7fa142637700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa1240a3c40 0x7fa1240a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:40.485 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.482+0000 7fa142637700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13c1041b0 0x7fa13c19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:40.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.482+0000 7fa142637700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa13c102610 0x7fa13c198850 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:40.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.482+0000 7fa142637700 1 --2- 172.21.15.145:0/343310766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa13c103810 0x7fa13c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:40.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.482+0000 7fa142637700 1 -- 172.21.15.145:0/343310766 >> 172.21.15.145:0/343310766 conn(0x7fa13c0fdbd0 msgr2=0x7fa13c106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:40.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.482+0000 7fa142637700 1 -- 172.21.15.145:0/343310766 shutdown_connections 2024-04-02T13:35:40.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-04-02T13:35:40.482+0000 7fa142637700 1 -- 172.21.15.145:0/343310766 wait complete. 2024-04-02T13:35:41.135 DEBUG:teuthology.orchestra.run.smithi033:mds.a> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.a.service 2024-04-02T13:35:41.138 DEBUG:teuthology.orchestra.run.smithi033:mds.d> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.d.service 2024-04-02T13:35:41.181 DEBUG:teuthology.orchestra.run.smithi033:mds.g> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.g.service 2024-04-02T13:35:41.225 DEBUG:teuthology.orchestra.run.smithi033:mds.j> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.j.service 2024-04-02T13:35:41.269 DEBUG:teuthology.orchestra.run.smithi050:mds.b> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.b.service 2024-04-02T13:35:41.271 DEBUG:teuthology.orchestra.run.smithi050:mds.e> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.e.service 2024-04-02T13:35:41.314 DEBUG:teuthology.orchestra.run.smithi050:mds.h> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.h.service 2024-04-02T13:35:41.357 DEBUG:teuthology.orchestra.run.smithi050:mds.k> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.k.service 2024-04-02T13:35:41.400 DEBUG:teuthology.orchestra.run.smithi145:mds.c> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.c.service 2024-04-02T13:35:41.403 DEBUG:teuthology.orchestra.run.smithi145:mds.f> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.f.service 2024-04-02T13:35:41.446 DEBUG:teuthology.orchestra.run.smithi145:mds.i> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.i.service 2024-04-02T13:35:41.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:41 smithi033 ceph-mon[32390]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:41.489 DEBUG:teuthology.orchestra.run.smithi145:mds.l> sudo journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.l.service 2024-04-02T13:35:41.532 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-02T13:35:41.532 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:35:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.640+0000 7f9f6b715640 1 -- 172.21.15.33:0/3877088776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f64102d00 msgr2=0x7f9f64105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.640+0000 7f9f6b715640 1 --2- 172.21.15.33:0/3877088776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f64102d00 0x7f9f64105170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009900 tx=0x7f9f5402bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.640+0000 7f9f6b715640 1 -- 172.21.15.33:0/3877088776 shutdown_connections 2024-04-02T13:35:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.640+0000 7f9f6b715640 1 --2- 172.21.15.33:0/3877088776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f641056b0 0x7f9f64105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.640+0000 7f9f6b715640 1 --2- 172.21.15.33:0/3877088776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f64102d00 0x7f9f64105170 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.640+0000 7f9f6b715640 1 --2- 172.21.15.33:0/3877088776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f64100390 0x7f9f641027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.640+0000 7f9f6b715640 1 -- 172.21.15.33:0/3877088776 >> 172.21.15.33:0/3877088776 conn(0x7f9f640fbf90 msgr2=0x7f9f640fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f6b715640 1 -- 172.21.15.33:0/3877088776 shutdown_connections 2024-04-02T13:35:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f6b715640 1 -- 172.21.15.33:0/3877088776 wait complete. 2024-04-02T13:35:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f6b715640 1 Processor -- start 2024-04-02T13:35:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f6b715640 1 -- start start 2024-04-02T13:35:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f6b715640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f64100390 0x7f9f6419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f6b715640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f64102d00 0x7f9f6419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f6b715640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f641056b0 0x7f9f641a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f6b715640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f64113de0 con 0x7f9f64102d00 2024-04-02T13:35:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f6b715640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9f64113c60 con 0x7f9f641056b0 2024-04-02T13:35:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f6b715640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9f64113f60 con 0x7f9f64100390 2024-04-02T13:35:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f68cb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f64102d00 0x7f9f6419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f68cb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f64102d00 0x7f9f6419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57620/0 (socket says 172.21.15.33:57620) 2024-04-02T13:35:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f68cb0640 1 -- 172.21.15.33:0/980473320 learned_addr learned my addr 172.21.15.33:0/980473320 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:35:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.641+0000 7f9f694b1640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f64100390 0x7f9f6419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f69cb2640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f641056b0 0x7f9f641a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f68cb0640 1 -- 172.21.15.33:0/980473320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f64100390 msgr2=0x7f9f6419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f68cb0640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f64100390 0x7f9f6419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f68cb0640 1 -- 172.21.15.33:0/980473320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f641056b0 msgr2=0x7f9f641a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f68cb0640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f641056b0 0x7f9f641a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f68cb0640 1 -- 172.21.15.33:0/980473320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f641a6010 con 0x7f9f64102d00 2024-04-02T13:35:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f69cb2640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f641056b0 0x7f9f641a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f694b1640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f64100390 0x7f9f6419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f68cb0640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f64102d00 0x7f9f6419e930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9f54000c00 tx=0x7f9f54004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f5a7fc640 1 -- 172.21.15.33:0/980473320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54043020 con 0x7f9f64102d00 2024-04-02T13:35:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f5a7fc640 1 -- 172.21.15.33:0/980473320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9f540054d0 con 0x7f9f64102d00 2024-04-02T13:35:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f6b715640 1 -- 172.21.15.33:0/980473320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f641a62a0 con 0x7f9f64102d00 2024-04-02T13:35:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f5a7fc640 1 -- 172.21.15.33:0/980473320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54003d40 con 0x7f9f64102d00 2024-04-02T13:35:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.642+0000 7f9f6b715640 1 -- 172.21.15.33:0/980473320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f641a67f0 con 0x7f9f64102d00 2024-04-02T13:35:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.643+0000 7f9f6b715640 1 -- 172.21.15.33:0/980473320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f640fda80 con 0x7f9f64102d00 2024-04-02T13:35:41.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.644+0000 7f9f5a7fc640 1 -- 172.21.15.33:0/980473320 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9f54046020 con 0x7f9f64102d00 2024-04-02T13:35:41.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.645+0000 7f9f5a7fc640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9f500a3bb0 0x7f9f500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:41.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.645+0000 7f9f694b1640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9f500a3bb0 0x7f9f500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:41.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.645+0000 7f9f5a7fc640 1 -- 172.21.15.33:0/980473320 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9f540f2760 con 0x7f9f64102d00 2024-04-02T13:35:41.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.645+0000 7f9f694b1640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9f500a3bb0 0x7f9f500a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9f60005d00 tx=0x7f9f60005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:41.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.648+0000 7f9f5a7fc640 1 -- 172.21.15.33:0/980473320 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9f5404d020 con 0x7f9f64102d00 2024-04-02T13:35:41.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:41 smithi145 ceph-mon[35646]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:41 smithi145 ceph-mon[35646]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:41 smithi145 ceph-mon[35646]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-04-02T13:35:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:41.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:41 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:41.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:41 smithi050 ceph-mon[35524]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:41.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:41 smithi050 ceph-mon[35524]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:41.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:41 smithi050 ceph-mon[35524]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-04-02T13:35:41.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:41.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:41.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:41.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:41.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:41.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:41.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:41.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:41 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:41.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:41 smithi033 ceph-mon[32390]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:41 smithi033 ceph-mon[32390]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-04-02T13:35:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:41 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.838+0000 7f9f6b715640 1 -- 172.21.15.33:0/980473320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f6402d5c0 con 0x7f9f64102d00 2024-04-02T13:35:41.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.839+0000 7f9f5a7fc640 1 -- 172.21.15.33:0/980473320 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18385 (secure 0 0 0) 0x7f9f540bf700 con 0x7f9f64102d00 2024-04-02T13:35:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:35:41.843 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:35:31.648681+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:35:41.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.841+0000 7f9f6b715640 1 -- 172.21.15.33:0/980473320 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9f500a3bb0 msgr2=0x7f9f500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:41.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.841+0000 7f9f6b715640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9f500a3bb0 0x7f9f500a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9f60005d00 tx=0x7f9f60005c70 comp rx=0 tx=0).stop 2024-04-02T13:35:41.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.841+0000 7f9f6b715640 1 -- 172.21.15.33:0/980473320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f64102d00 msgr2=0x7f9f6419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:41.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.841+0000 7f9f6b715640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f64102d00 0x7f9f6419e930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9f54000c00 tx=0x7f9f54004f10 comp rx=0 tx=0).stop 2024-04-02T13:35:41.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.841+0000 7f9f6b715640 1 -- 172.21.15.33:0/980473320 shutdown_connections 2024-04-02T13:35:41.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.841+0000 7f9f6b715640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9f500a3bb0 0x7f9f500a6070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.841+0000 7f9f6b715640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f641056b0 0x7f9f641a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.841+0000 7f9f6b715640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f64102d00 0x7f9f6419e930 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.841+0000 7f9f6b715640 1 --2- 172.21.15.33:0/980473320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f64100390 0x7f9f6419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.841+0000 7f9f6b715640 1 -- 172.21.15.33:0/980473320 >> 172.21.15.33:0/980473320 conn(0x7f9f640fbf90 msgr2=0x7f9f64100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:41.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.841+0000 7f9f6b715640 1 -- 172.21.15.33:0/980473320 shutdown_connections 2024-04-02T13:35:41.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.841+0000 7f9f6b715640 1 -- 172.21.15.33:0/980473320 wait complete. 2024-04-02T13:35:41.856 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T13:35:41.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:35:41.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.960+0000 7f98d522b640 1 -- 172.21.15.33:0/2984759797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d0101bf0 msgr2=0x7f98d0102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:41.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.960+0000 7f98d522b640 1 --2- 172.21.15.33:0/2984759797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d0101bf0 0x7f98d0102070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009900 tx=0x7f98c402bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:41.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.960+0000 7f98d522b640 1 -- 172.21.15.33:0/2984759797 shutdown_connections 2024-04-02T13:35:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.960+0000 7f98d522b640 1 --2- 172.21.15.33:0/2984759797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98d01025b0 0x7f98d0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.960+0000 7f98d522b640 1 --2- 172.21.15.33:0/2984759797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d0101bf0 0x7f98d0102070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.960+0000 7f98d522b640 1 --2- 172.21.15.33:0/2984759797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98d0100a00 0x7f98d0100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.960+0000 7f98d522b640 1 -- 172.21.15.33:0/2984759797 >> 172.21.15.33:0/2984759797 conn(0x7f98d00fbf90 msgr2=0x7f98d00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.960+0000 7f98d522b640 1 -- 172.21.15.33:0/2984759797 shutdown_connections 2024-04-02T13:35:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.960+0000 7f98d522b640 1 -- 172.21.15.33:0/2984759797 wait complete. 2024-04-02T13:35:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.961+0000 7f98d522b640 1 Processor -- start 2024-04-02T13:35:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.961+0000 7f98d522b640 1 -- start start 2024-04-02T13:35:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.961+0000 7f98d522b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98d0100a00 0x7f98d0196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.961+0000 7f98d522b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d0101bf0 0x7f98d0197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.961+0000 7f98d522b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98d01025b0 0x7f98d019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.961+0000 7f98d522b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98d010c660 con 0x7f98d0101bf0 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.961+0000 7f98d522b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f98d010c4e0 con 0x7f98d01025b0 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.961+0000 7f98d522b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98d010c7e0 con 0x7f98d0100a00 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.961+0000 7f98ce59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d0101bf0 0x7f98d0197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.961+0000 7f98ce59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d0101bf0 0x7f98d0197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57666/0 (socket says 172.21.15.33:57666) 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.961+0000 7f98ce59c640 1 -- 172.21.15.33:0/3908744530 learned_addr learned my addr 172.21.15.33:0/3908744530 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.961+0000 7f98cf59e640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98d01025b0 0x7f98d019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.962+0000 7f98ced9d640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98d0100a00 0x7f98d0196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.962+0000 7f98ce59c640 1 -- 172.21.15.33:0/3908744530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98d0100a00 msgr2=0x7f98d0196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.962+0000 7f98ce59c640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98d0100a00 0x7f98d0196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.962+0000 7f98ce59c640 1 -- 172.21.15.33:0/3908744530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98d01025b0 msgr2=0x7f98d019e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.962+0000 7f98ce59c640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98d01025b0 0x7f98d019e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.962+0000 7f98ce59c640 1 -- 172.21.15.33:0/3908744530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d019e950 con 0x7f98d0101bf0 2024-04-02T13:35:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.962+0000 7f98ced9d640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98d0100a00 0x7f98d0196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.962+0000 7f98ce59c640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d0101bf0 0x7f98d0197280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009230 tx=0x7f98c4004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.962+0000 7f98b7fff640 1 -- 172.21.15.33:0/3908744530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4043020 con 0x7f98d0101bf0 2024-04-02T13:35:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.962+0000 7f98d522b640 1 -- 172.21.15.33:0/3908744530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d019ebe0 con 0x7f98d0101bf0 2024-04-02T13:35:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.962+0000 7f98b7fff640 1 -- 172.21.15.33:0/3908744530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f98c4004350 con 0x7f98d0101bf0 2024-04-02T13:35:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.962+0000 7f98b7fff640 1 -- 172.21.15.33:0/3908744530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4003ca0 con 0x7f98d0101bf0 2024-04-02T13:35:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.962+0000 7f98d522b640 1 -- 172.21.15.33:0/3908744530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d019f130 con 0x7f98d0101bf0 2024-04-02T13:35:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.963+0000 7f98d522b640 1 -- 172.21.15.33:0/3908744530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d0107da0 con 0x7f98d0101bf0 2024-04-02T13:35:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.964+0000 7f98b7fff640 1 -- 172.21.15.33:0/3908744530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f98c4046020 con 0x7f98d0101bf0 2024-04-02T13:35:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.964+0000 7f98b7fff640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f98b80a3bb0 0x7f98b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.965+0000 7f98b7fff640 1 -- 172.21.15.33:0/3908744530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f98c40f1d80 con 0x7f98d0101bf0 2024-04-02T13:35:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.965+0000 7f98ced9d640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f98b80a3bb0 0x7f98b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.965+0000 7f98ced9d640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f98b80a3bb0 0x7f98b80a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98c8005d00 tx=0x7f98c8005f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:41.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:41.969+0000 7f98b7fff640 1 -- 172.21.15.33:0/3908744530 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f98c404d070 con 0x7f98d0101bf0 2024-04-02T13:35:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.156+0000 7f98d522b640 1 -- 172.21.15.33:0/3908744530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98d002d590 con 0x7f98d0101bf0 2024-04-02T13:35:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.157+0000 7f98b7fff640 1 -- 172.21.15.33:0/3908744530 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f98c40bec00 con 0x7f98d0101bf0 2024-04-02T13:35:42.160 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:35:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.159+0000 7f98d522b640 1 -- 172.21.15.33:0/3908744530 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f98b80a3bb0 msgr2=0x7f98b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.159+0000 7f98d522b640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f98b80a3bb0 0x7f98b80a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98c8005d00 tx=0x7f98c8005f10 comp rx=0 tx=0).stop 2024-04-02T13:35:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.159+0000 7f98d522b640 1 -- 172.21.15.33:0/3908744530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d0101bf0 msgr2=0x7f98d0197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.159+0000 7f98d522b640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d0101bf0 0x7f98d0197280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009230 tx=0x7f98c4004ea0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.159+0000 7f98d522b640 1 -- 172.21.15.33:0/3908744530 shutdown_connections 2024-04-02T13:35:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.159+0000 7f98d522b640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f98b80a3bb0 0x7f98b80a6070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.159+0000 7f98d522b640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98d01025b0 0x7f98d019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.159+0000 7f98d522b640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d0101bf0 0x7f98d0197280 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.159+0000 7f98d522b640 1 --2- 172.21.15.33:0/3908744530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98d0100a00 0x7f98d0196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.159+0000 7f98d522b640 1 -- 172.21.15.33:0/3908744530 >> 172.21.15.33:0/3908744530 conn(0x7f98d00fbf90 msgr2=0x7f98d0104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.159+0000 7f98d522b640 1 -- 172.21.15.33:0/3908744530 shutdown_connections 2024-04-02T13:35:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.159+0000 7f98d522b640 1 -- 172.21.15.33:0/3908744530 wait complete. 2024-04-02T13:35:42.174 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:35:42.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 -- 172.21.15.33:0/3867679584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff738100a20 msgr2=0x7ff738100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:42.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 --2- 172.21.15.33:0/3867679584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff738100a20 0x7ff738100e40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff734009900 tx=0x7ff73402bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:42.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 -- 172.21.15.33:0/3867679584 shutdown_connections 2024-04-02T13:35:42.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 --2- 172.21.15.33:0/3867679584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7381025d0 0x7ff738108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 --2- 172.21.15.33:0/3867679584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff738101c10 0x7ff738102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 --2- 172.21.15.33:0/3867679584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff738100a20 0x7ff738100e40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 -- 172.21.15.33:0/3867679584 >> 172.21.15.33:0/3867679584 conn(0x7ff7380fbf90 msgr2=0x7ff7380fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:42.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 -- 172.21.15.33:0/3867679584 shutdown_connections 2024-04-02T13:35:42.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 -- 172.21.15.33:0/3867679584 wait complete. 2024-04-02T13:35:42.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 Processor -- start 2024-04-02T13:35:42.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 -- start start 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff738100a20 0x7ff738197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff738101c10 0x7ff7381984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7381025d0 0x7ff73819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff73810c6e0 con 0x7ff7381025d0 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff73810c560 con 0x7ff738100a20 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.273+0000 7ff73fdcd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff73810c860 con 0x7ff738101c10 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73e36a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7381025d0 0x7ff73819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73e36a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7381025d0 0x7ff73819f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57706/0 (socket says 172.21.15.33:57706) 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73e36a640 1 -- 172.21.15.33:0/4088408436 learned_addr learned my addr 172.21.15.33:0/4088408436 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73db69640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff738100a20 0x7ff738197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73d368640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff738101c10 0x7ff7381984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73e36a640 1 -- 172.21.15.33:0/4088408436 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff738101c10 msgr2=0x7ff7381984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73e36a640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff738101c10 0x7ff7381984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73e36a640 1 -- 172.21.15.33:0/4088408436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff738100a20 msgr2=0x7ff738197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:42.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73e36a640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff738100a20 0x7ff738197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73e36a640 1 -- 172.21.15.33:0/4088408436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73819fb40 con 0x7ff7381025d0 2024-04-02T13:35:42.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73d368640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff738101c10 0x7ff7381984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:42.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73e36a640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7381025d0 0x7ff73819f4d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff73000eaa0 tx=0x7ff73000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:42.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff72effd640 1 -- 172.21.15.33:0/4088408436 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73000cdb0 con 0x7ff7381025d0 2024-04-02T13:35:42.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73fdcd640 1 -- 172.21.15.33:0/4088408436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73819fe30 con 0x7ff7381025d0 2024-04-02T13:35:42.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff72effd640 1 -- 172.21.15.33:0/4088408436 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff730005020 con 0x7ff7381025d0 2024-04-02T13:35:42.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff72effd640 1 -- 172.21.15.33:0/4088408436 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7300116b0 con 0x7ff7381025d0 2024-04-02T13:35:42.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.274+0000 7ff73fdcd640 1 -- 172.21.15.33:0/4088408436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7381a0320 con 0x7ff7381025d0 2024-04-02T13:35:42.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.275+0000 7ff73fdcd640 1 -- 172.21.15.33:0/4088408436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7381a05e0 con 0x7ff7381025d0 2024-04-02T13:35:42.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.276+0000 7ff72effd640 1 -- 172.21.15.33:0/4088408436 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff7300051d0 con 0x7ff7381025d0 2024-04-02T13:35:42.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.277+0000 7ff72effd640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff7240a3bb0 0x7ff7240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:42.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.277+0000 7ff73db69640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff7240a3bb0 0x7ff7240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:42.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.277+0000 7ff72effd640 1 -- 172.21.15.33:0/4088408436 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff7300d01a0 con 0x7ff7381025d0 2024-04-02T13:35:42.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.278+0000 7ff73db69640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff7240a3bb0 0x7ff7240a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff734000c00 tx=0x7ff734005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:42.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.282+0000 7ff72effd640 1 -- 172.21.15.33:0/4088408436 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff730016030 con 0x7ff7381025d0 2024-04-02T13:35:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.507+0000 7ff73fdcd640 1 -- 172.21.15.33:0/4088408436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7381a07b0 con 0x7ff7381025d0 2024-04-02T13:35:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.508+0000 7ff72effd640 1 -- 172.21.15.33:0/4088408436 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7ff73009d020 con 0x7ff7381025d0 2024-04-02T13:35:42.512 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:35:42.512 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-02T13:35:42.512 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 1 2024-04-02T13:35:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.510+0000 7ff73fdcd640 1 -- 172.21.15.33:0/4088408436 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff7240a3bb0 msgr2=0x7ff7240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.510+0000 7ff73fdcd640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff7240a3bb0 0x7ff7240a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff734000c00 tx=0x7ff734005fd0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.510+0000 7ff73fdcd640 1 -- 172.21.15.33:0/4088408436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7381025d0 msgr2=0x7ff73819f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.510+0000 7ff73fdcd640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7381025d0 0x7ff73819f4d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff73000eaa0 tx=0x7ff73000ef60 comp rx=0 tx=0).stop 2024-04-02T13:35:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.510+0000 7ff73fdcd640 1 -- 172.21.15.33:0/4088408436 shutdown_connections 2024-04-02T13:35:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.510+0000 7ff73fdcd640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff7240a3bb0 0x7ff7240a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.510+0000 7ff73fdcd640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7381025d0 0x7ff73819f4d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.510+0000 7ff73fdcd640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff738101c10 0x7ff7381984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.510+0000 7ff73fdcd640 1 --2- 172.21.15.33:0/4088408436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff738100a20 0x7ff738197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:42.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.510+0000 7ff73fdcd640 1 -- 172.21.15.33:0/4088408436 >> 172.21.15.33:0/4088408436 conn(0x7ff7380fbf90 msgr2=0x7ff738068d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:42.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.510+0000 7ff73fdcd640 1 -- 172.21.15.33:0/4088408436 shutdown_connections 2024-04-02T13:35:42.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:42.510+0000 7ff73fdcd640 1 -- 172.21.15.33:0/4088408436 wait complete. 2024-04-02T13:35:42.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:42 smithi050 ceph-mon[35524]: Deploying daemon mds.l on smithi145 2024-04-02T13:35:42.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:42 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/980473320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:42.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:42 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3908744530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:42.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:42 smithi033 ceph-mon[32390]: Deploying daemon mds.l on smithi145 2024-04-02T13:35:42.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:42 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/980473320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:42.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:42 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3908744530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:42.837 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:42 smithi145 ceph-mon[35646]: Deploying daemon mds.l on smithi145 2024-04-02T13:35:42.837 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:42 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/980473320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:42.837 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:42 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3908744530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:43.479 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:35:43 smithi145 systemd[1]: Starting Ceph mds.l for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:35:43.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:43 smithi050 ceph-mon[35524]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:43.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:43 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4088408436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:43.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:43 smithi033 ceph-mon[32390]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:43.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:43 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4088408436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:43.795 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:43 smithi145 ceph-mon[35646]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:43.796 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:43 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4088408436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:43.796 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:35:43 smithi145 podman[64978]: 2024-04-02 13:35:43.439764459 +0000 UTC m=+0.018100049 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:35:43.796 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:35:43 smithi145 podman[64978]: 2024-04-02 13:35:43.549486145 +0000 UTC m=+0.127821764 container create cb3d3318fbc9a13bb3805b4f839c42d1e1f1ef8144f0087ea89e0a5f407fe5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-l, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326) 2024-04-02T13:35:43.796 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:35:43 smithi145 podman[64978]: 2024-04-02 13:35:43.793779432 +0000 UTC m=+0.372115023 container init cb3d3318fbc9a13bb3805b4f839c42d1e1f1ef8144f0087ea89e0a5f407fe5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-l, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:35:44.229 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:35:43 smithi145 podman[64978]: 2024-04-02 13:35:43.796742257 +0000 UTC m=+0.375077840 container start cb3d3318fbc9a13bb3805b4f839c42d1e1f1ef8144f0087ea89e0a5f407fe5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-l, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:35:44.229 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:35:43 smithi145 ceph-mds[64997]: -- 172.21.15.145:0/1189467808 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562134e87a40 con 0x562134e5b000 2024-04-02T13:35:44.229 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:35:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-l[64993]: starting mds.l at 2024-04-02T13:35:44.229 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:35:43 smithi145 bash[64978]: cb3d3318fbc9a13bb3805b4f839c42d1e1f1ef8144f0087ea89e0a5f407fe5e8 2024-04-02T13:35:44.229 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:35:43 smithi145 systemd[1]: Started Ceph mds.l for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:35:44.526 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:35:44.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.638+0000 7fe929960640 1 -- 172.21.15.33:0/4219244522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe924107800 msgr2=0x7fe924109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:44.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.638+0000 7fe929960640 1 --2- 172.21.15.33:0/4219244522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe924107800 0x7fe924109c50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009900 tx=0x7fe91c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:44.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.638+0000 7fe929960640 1 -- 172.21.15.33:0/4219244522 shutdown_connections 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.638+0000 7fe929960640 1 --2- 172.21.15.33:0/4219244522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe924107800 0x7fe924109c50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.638+0000 7fe929960640 1 --2- 172.21.15.33:0/4219244522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe924104ed0 0x7fe9241072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.638+0000 7fe929960640 1 --2- 172.21.15.33:0/4219244522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9241025a0 0x7fe924104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.638+0000 7fe929960640 1 -- 172.21.15.33:0/4219244522 >> 172.21.15.33:0/4219244522 conn(0x7fe9240fbf90 msgr2=0x7fe9240fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.638+0000 7fe929960640 1 -- 172.21.15.33:0/4219244522 shutdown_connections 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.638+0000 7fe929960640 1 -- 172.21.15.33:0/4219244522 wait complete. 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.639+0000 7fe929960640 1 Processor -- start 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.639+0000 7fe929960640 1 -- start start 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.639+0000 7fe929960640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9241025a0 0x7fe924197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.639+0000 7fe929960640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe924104ed0 0x7fe924198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.639+0000 7fe929960640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe924107800 0x7fe92419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.639+0000 7fe929960640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe92410d7a0 con 0x7fe924107800 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.639+0000 7fe929960640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe92410d620 con 0x7fe9241025a0 2024-04-02T13:35:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.639+0000 7fe929960640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe92410d920 con 0x7fe924104ed0 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe922ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9241025a0 0x7fe924197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe9237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe924107800 0x7fe92419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe922ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9241025a0 0x7fe924197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:35686/0 (socket says 172.21.15.33:35686) 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe9237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe924107800 0x7fe92419f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39228/0 (socket says 172.21.15.33:39228) 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe922ffd640 1 -- 172.21.15.33:0/2525873940 learned_addr learned my addr 172.21.15.33:0/2525873940 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe9227fc640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe924104ed0 0x7fe924198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe9237fe640 1 -- 172.21.15.33:0/2525873940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe924104ed0 msgr2=0x7fe924198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe9237fe640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe924104ed0 0x7fe924198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe9237fe640 1 -- 172.21.15.33:0/2525873940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9241025a0 msgr2=0x7fe924197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe9237fe640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9241025a0 0x7fe924197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe9237fe640 1 -- 172.21.15.33:0/2525873940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92419fa00 con 0x7fe924107800 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe922ffd640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9241025a0 0x7fe924197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe9237fe640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe924107800 0x7fe92419f390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe91c02c170 tx=0x7fe91c005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe92895e640 1 -- 172.21.15.33:0/2525873940 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c043020 con 0x7fe924107800 2024-04-02T13:35:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe92895e640 1 -- 172.21.15.33:0/2525873940 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe91c0054b0 con 0x7fe924107800 2024-04-02T13:35:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe929960640 1 -- 172.21.15.33:0/2525873940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92419fc90 con 0x7fe924107800 2024-04-02T13:35:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe92895e640 1 -- 172.21.15.33:0/2525873940 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c003b30 con 0x7fe924107800 2024-04-02T13:35:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.640+0000 7fe929960640 1 -- 172.21.15.33:0/2525873940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9241a0190 con 0x7fe924107800 2024-04-02T13:35:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.641+0000 7fe929960640 1 -- 172.21.15.33:0/2525873940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9240fda80 con 0x7fe924107800 2024-04-02T13:35:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.642+0000 7fe92895e640 1 -- 172.21.15.33:0/2525873940 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe91c046020 con 0x7fe924107800 2024-04-02T13:35:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.643+0000 7fe92895e640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe9100a3bb0 0x7fe9100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.643+0000 7fe922ffd640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe9100a3bb0 0x7fe9100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.643+0000 7fe92895e640 1 -- 172.21.15.33:0/2525873940 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe91c0f20b0 con 0x7fe924107800 2024-04-02T13:35:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.644+0000 7fe922ffd640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe9100a3bb0 0x7fe9100a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe914009fd0 tx=0x7fe914009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:44.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.647+0000 7fe92895e640 1 -- 172.21.15.33:0/2525873940 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe91c04d070 con 0x7fe924107800 2024-04-02T13:35:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.872+0000 7fe929960640 1 -- 172.21.15.33:0/2525873940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe92402d370 con 0x7fe924107800 2024-04-02T13:35:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.872+0000 7fe92895e640 1 -- 172.21.15.33:0/2525873940 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fe91c0bef30 con 0x7fe924107800 2024-04-02T13:35:44.876 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:35:44.876 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-02T13:35:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 2 2024-04-02T13:35:44.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.874+0000 7fe929960640 1 -- 172.21.15.33:0/2525873940 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe9100a3bb0 msgr2=0x7fe9100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.874+0000 7fe929960640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe9100a3bb0 0x7fe9100a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe914009fd0 tx=0x7fe914009700 comp rx=0 tx=0).stop 2024-04-02T13:35:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.874+0000 7fe929960640 1 -- 172.21.15.33:0/2525873940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe924107800 msgr2=0x7fe92419f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.874+0000 7fe929960640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe924107800 0x7fe92419f390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe91c02c170 tx=0x7fe91c005de0 comp rx=0 tx=0).stop 2024-04-02T13:35:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.874+0000 7fe929960640 1 -- 172.21.15.33:0/2525873940 shutdown_connections 2024-04-02T13:35:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.874+0000 7fe929960640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe9100a3bb0 0x7fe9100a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.874+0000 7fe929960640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe924107800 0x7fe92419f390 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.874+0000 7fe929960640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe924104ed0 0x7fe924198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.874+0000 7fe929960640 1 --2- 172.21.15.33:0/2525873940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9241025a0 0x7fe924197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.874+0000 7fe929960640 1 -- 172.21.15.33:0/2525873940 >> 172.21.15.33:0/2525873940 conn(0x7fe9240fbf90 msgr2=0x7fe924102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.875+0000 7fe929960640 1 -- 172.21.15.33:0/2525873940 shutdown_connections 2024-04-02T13:35:44.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:44.875+0000 7fe929960640 1 -- 172.21.15.33:0/2525873940 wait complete. 2024-04-02T13:35:44.979 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:44 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.145:6832/3754674798,v1:172.21.15.145:6833/3754674798] up:boot 2024-04-02T13:35:44.979 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:44 smithi050 ceph-mon[35524]: fsmap 1 up:standby 2024-04-02T13:35:44.979 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:35:44.979 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:44 smithi050 ceph-mon[35524]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:44.979 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:44.979 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:44.979 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:44.979 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:44.979 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:44.979 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:44 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:44.979 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:44 smithi050 ceph-mon[35524]: Deploying daemon mds.k on smithi050 2024-04-02T13:35:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:44 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.145:6832/3754674798,v1:172.21.15.145:6833/3754674798] up:boot 2024-04-02T13:35:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:44 smithi145 ceph-mon[35646]: fsmap 1 up:standby 2024-04-02T13:35:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:35:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:44 smithi145 ceph-mon[35646]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:44 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:45.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:44 smithi145 ceph-mon[35646]: Deploying daemon mds.k on smithi050 2024-04-02T13:35:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:44 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.145:6832/3754674798,v1:172.21.15.145:6833/3754674798] up:boot 2024-04-02T13:35:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:44 smithi033 ceph-mon[32390]: fsmap 1 up:standby 2024-04-02T13:35:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:35:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:44 smithi033 ceph-mon[32390]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:45.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:44 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:45.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:44 smithi033 ceph-mon[32390]: Deploying daemon mds.k on smithi050 2024-04-02T13:35:46.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:45 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2525873940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:46.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:45 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2525873940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:46.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:45 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2525873940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:46.494 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:35:46 smithi050 systemd[1]: Starting Ceph mds.k for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:35:46.783 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:35:46 smithi050 podman[65360]: 2024-04-02 13:35:46.423973239 +0000 UTC m=+0.017807751 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:35:46.783 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:35:46 smithi050 podman[65360]: 2024-04-02 13:35:46.526528238 +0000 UTC m=+0.120362779 container create a9da2e12226d724711d396934a9bb65d8c1a0c8fe41431fae04af5877ac7fa8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-02T13:35:46.783 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:35:46 smithi050 podman[65360]: 2024-04-02 13:35:46.754743953 +0000 UTC m=+0.348578452 container init a9da2e12226d724711d396934a9bb65d8c1a0c8fe41431fae04af5877ac7fa8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-k, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5) 2024-04-02T13:35:46.783 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:35:46 smithi050 podman[65360]: 2024-04-02 13:35:46.757343031 +0000 UTC m=+0.351177536 container start a9da2e12226d724711d396934a9bb65d8c1a0c8fe41431fae04af5877ac7fa8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-k, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:35:46.783 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:35:46 smithi050 ceph-mds[65379]: -- 172.21.15.50:0/112614586 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55decc8fda40 con 0x55decc8d0800 2024-04-02T13:35:46.890 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:35:47.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.004+0000 7f856b711640 1 -- 172.21.15.33:0/2574534963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8564101b40 msgr2=0x7f8564101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.004+0000 7f856b711640 1 --2- 172.21.15.33:0/2574534963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8564101b40 0x7f8564101f60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8560009900 tx=0x7f856002bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:47.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.004+0000 7f856b711640 1 -- 172.21.15.33:0/2574534963 shutdown_connections 2024-04-02T13:35:47.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.004+0000 7f856b711640 1 --2- 172.21.15.33:0/2574534963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85641036f0 0x7f8564109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.004+0000 7f856b711640 1 --2- 172.21.15.33:0/2574534963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8564102d30 0x7f85641031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.004+0000 7f856b711640 1 --2- 172.21.15.33:0/2574534963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8564101b40 0x7f8564101f60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.004+0000 7f856b711640 1 -- 172.21.15.33:0/2574534963 >> 172.21.15.33:0/2574534963 conn(0x7f85640fd0b0 msgr2=0x7f85640ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.004+0000 7f856b711640 1 -- 172.21.15.33:0/2574534963 shutdown_connections 2024-04-02T13:35:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.004+0000 7f856b711640 1 -- 172.21.15.33:0/2574534963 wait complete. 2024-04-02T13:35:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.005+0000 7f856b711640 1 Processor -- start 2024-04-02T13:35:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.005+0000 7f856b711640 1 -- start start 2024-04-02T13:35:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.005+0000 7f856b711640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8564101b40 0x7f85640758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.005+0000 7f856b711640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8564102d30 0x7f8564073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.005+0000 7f856b711640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85641036f0 0x7f8564074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.005+0000 7f856b711640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f856410d790 con 0x7f8564102d30 2024-04-02T13:35:47.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.005+0000 7f856b711640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f856410d610 con 0x7f8564101b40 2024-04-02T13:35:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.005+0000 7f856b711640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f856410d910 con 0x7f85641036f0 2024-04-02T13:35:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.005+0000 7f8568cac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8564102d30 0x7f8564073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.005+0000 7f8568cac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8564102d30 0x7f8564073ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39244/0 (socket says 172.21.15.33:39244) 2024-04-02T13:35:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.005+0000 7f8568cac640 1 -- 172.21.15.33:0/2427317512 learned_addr learned my addr 172.21.15.33:0/2427317512 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:35:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.005+0000 7f85694ad640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8564101b40 0x7f85640758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.005+0000 7f8569cae640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85641036f0 0x7f8564074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.006+0000 7f8568cac640 1 -- 172.21.15.33:0/2427317512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85641036f0 msgr2=0x7f8564074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.006+0000 7f8568cac640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85641036f0 0x7f8564074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.006+0000 7f8568cac640 1 -- 172.21.15.33:0/2427317512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8564101b40 msgr2=0x7f85640758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.006+0000 7f8568cac640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8564101b40 0x7f85640758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.006+0000 7f8568cac640 1 -- 172.21.15.33:0/2427317512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8564074cc0 con 0x7f8564102d30 2024-04-02T13:35:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.006+0000 7f8569cae640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85641036f0 0x7f8564074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:47.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.006+0000 7f8568cac640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8564102d30 0x7f8564073ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f855400e9f0 tx=0x7f855400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:47.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.006+0000 7f855a7fc640 1 -- 172.21.15.33:0/2427317512 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855400cd80 con 0x7f8564102d30 2024-04-02T13:35:47.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.006+0000 7f856b711640 1 -- 172.21.15.33:0/2427317512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85641a1f60 con 0x7f8564102d30 2024-04-02T13:35:47.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.006+0000 7f855a7fc640 1 -- 172.21.15.33:0/2427317512 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8554010900 con 0x7f8564102d30 2024-04-02T13:35:47.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.006+0000 7f855a7fc640 1 -- 172.21.15.33:0/2427317512 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85540189e0 con 0x7f8564102d30 2024-04-02T13:35:47.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.006+0000 7f856b711640 1 -- 172.21.15.33:0/2427317512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85641a23d0 con 0x7f8564102d30 2024-04-02T13:35:47.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.007+0000 7f856b711640 1 -- 172.21.15.33:0/2427317512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8564108ee0 con 0x7f8564102d30 2024-04-02T13:35:47.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.008+0000 7f855a7fc640 1 -- 172.21.15.33:0/2427317512 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8554010ab0 con 0x7f8564102d30 2024-04-02T13:35:47.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.009+0000 7f855a7fc640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f85500a3bb0 0x7f85500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.009+0000 7f85694ad640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f85500a3bb0 0x7f85500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.009+0000 7f855a7fc640 1 -- 172.21.15.33:0/2427317512 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f85540d0780 con 0x7f8564102d30 2024-04-02T13:35:47.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.009+0000 7f85694ad640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f85500a3bb0 0x7f85500a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8560000c00 tx=0x7f8560005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:47.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.013+0000 7f855a7fc640 1 -- 172.21.15.33:0/2427317512 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f855409d600 con 0x7f8564102d30 2024-04-02T13:35:47.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:46 smithi145 ceph-mon[35646]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:47.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.240+0000 7f856b711640 1 -- 172.21.15.33:0/2427317512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f856402d370 con 0x7f8564102d30 2024-04-02T13:35:47.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.241+0000 7f855a7fc640 1 -- 172.21.15.33:0/2427317512 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f855409cf00 con 0x7f8564102d30 2024-04-02T13:35:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:35:47.243 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T13:35:47.244 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2024-04-02T13:35:47.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:46 smithi050 ceph-mon[35524]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:47.244 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:35:46 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-k[65375]: starting mds.k at 2024-04-02T13:35:47.244 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:35:46 smithi050 bash[65360]: a9da2e12226d724711d396934a9bb65d8c1a0c8fe41431fae04af5877ac7fa8a 2024-04-02T13:35:47.244 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:35:46 smithi050 systemd[1]: Started Ceph mds.k for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:35:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.243+0000 7f856b711640 1 -- 172.21.15.33:0/2427317512 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f85500a3bb0 msgr2=0x7f85500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.243+0000 7f856b711640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f85500a3bb0 0x7f85500a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8560000c00 tx=0x7f8560005fd0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.243+0000 7f856b711640 1 -- 172.21.15.33:0/2427317512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8564102d30 msgr2=0x7f8564073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.243+0000 7f856b711640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8564102d30 0x7f8564073ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f855400e9f0 tx=0x7f855400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.243+0000 7f856b711640 1 -- 172.21.15.33:0/2427317512 shutdown_connections 2024-04-02T13:35:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.243+0000 7f856b711640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f85500a3bb0 0x7f85500a6070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.243+0000 7f856b711640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85641036f0 0x7f8564074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.243+0000 7f856b711640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8564102d30 0x7f8564073ef0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.243+0000 7f856b711640 1 --2- 172.21.15.33:0/2427317512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8564101b40 0x7f85640758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.243+0000 7f856b711640 1 -- 172.21.15.33:0/2427317512 >> 172.21.15.33:0/2427317512 conn(0x7f85640fd0b0 msgr2=0x7f85641060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.243+0000 7f856b711640 1 -- 172.21.15.33:0/2427317512 shutdown_connections 2024-04-02T13:35:47.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:47.243+0000 7f856b711640 1 -- 172.21.15.33:0/2427317512 wait complete. 2024-04-02T13:35:47.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:46 smithi033 ceph-mon[32390]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:47.877 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:47 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.50:6832/1584719197,v1:172.21.15.50:6833/1584719197] up:boot 2024-04-02T13:35:47.877 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:47 smithi033 ceph-mon[32390]: fsmap 2 up:standby 2024-04-02T13:35:47.877 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:35:47.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:47.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:47.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:47.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:47.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:47.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:47 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:47.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:47 smithi033 ceph-mon[32390]: Deploying daemon mds.j on smithi033 2024-04-02T13:35:47.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:47 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2427317512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:48.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:47 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.50:6832/1584719197,v1:172.21.15.50:6833/1584719197] up:boot 2024-04-02T13:35:48.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:47 smithi145 ceph-mon[35646]: fsmap 2 up:standby 2024-04-02T13:35:48.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:47 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:35:48.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:47 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:48.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:47 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:48.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:47 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:48.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:47 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:48.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:47 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:48.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:47 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:48.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:47 smithi145 ceph-mon[35646]: Deploying daemon mds.j on smithi033 2024-04-02T13:35:48.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:47 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2427317512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:48.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:47 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.50:6832/1584719197,v1:172.21.15.50:6833/1584719197] up:boot 2024-04-02T13:35:48.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:47 smithi050 ceph-mon[35524]: fsmap 2 up:standby 2024-04-02T13:35:48.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:47 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:35:48.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:47 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:48.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:47 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:48.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:47 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:48.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:47 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:48.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:47 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:48.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:47 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:48.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:47 smithi050 ceph-mon[35524]: Deploying daemon mds.j on smithi033 2024-04-02T13:35:48.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:47 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2427317512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:49.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:48 smithi033 ceph-mon[32390]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:49.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:48 smithi050 ceph-mon[35524]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:49.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:48 smithi145 ceph-mon[35646]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:49.258 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:35:49.287 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:35:49 smithi033 systemd[1]: Starting Ceph mds.j for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:35:49.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.382+0000 7ff49a60d640 1 -- 172.21.15.33:0/868481833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff494100c50 msgr2=0x7ff4941010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:49.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.382+0000 7ff49a60d640 1 --2- 172.21.15.33:0/868481833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff494100c50 0x7ff4941010d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff488009900 tx=0x7ff48802bc80 comp rx=0 tx=0).stop 2024-04-02T13:35:49.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.382+0000 7ff49a60d640 1 -- 172.21.15.33:0/868481833 shutdown_connections 2024-04-02T13:35:49.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.382+0000 7ff49a60d640 1 --2- 172.21.15.33:0/868481833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff494101610 0x7ff494108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:49.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.382+0000 7ff49a60d640 1 --2- 172.21.15.33:0/868481833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff494100c50 0x7ff4941010d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.382+0000 7ff49a60d640 1 --2- 172.21.15.33:0/868481833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff494102090 0x7ff4941024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.382+0000 7ff49a60d640 1 -- 172.21.15.33:0/868481833 >> 172.21.15.33:0/868481833 conn(0x7ff4940fb700 msgr2=0x7ff4940fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.382+0000 7ff49a60d640 1 -- 172.21.15.33:0/868481833 shutdown_connections 2024-04-02T13:35:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.382+0000 7ff49a60d640 1 -- 172.21.15.33:0/868481833 wait complete. 2024-04-02T13:35:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.382+0000 7ff49a60d640 1 Processor -- start 2024-04-02T13:35:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff49a60d640 1 -- start start 2024-04-02T13:35:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff49a60d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff494100c50 0x7ff494196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff49a60d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff494101610 0x7ff4941972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff49a60d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff494102090 0x7ff49419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff49a60d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff49410c6b0 con 0x7ff494102090 2024-04-02T13:35:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff49a60d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff49410c530 con 0x7ff494100c50 2024-04-02T13:35:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff49a60d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff49410c830 con 0x7ff494101610 2024-04-02T13:35:49.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff498baa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff494102090 0x7ff49419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff498baa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff494102090 0x7ff49419e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39272/0 (socket says 172.21.15.33:39272) 2024-04-02T13:35:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff493fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff494100c50 0x7ff494196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff498baa640 1 -- 172.21.15.33:0/3625907640 learned_addr learned my addr 172.21.15.33:0/3625907640 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:35:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff4937fe640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff494101610 0x7ff4941972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff498baa640 1 -- 172.21.15.33:0/3625907640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff494101610 msgr2=0x7ff4941972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff498baa640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff494101610 0x7ff4941972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff498baa640 1 -- 172.21.15.33:0/3625907640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff494100c50 msgr2=0x7ff494196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff498baa640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff494100c50 0x7ff494196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff498baa640 1 -- 172.21.15.33:0/3625907640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49419e980 con 0x7ff494102090 2024-04-02T13:35:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.383+0000 7ff493fff640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff494100c50 0x7ff494196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.384+0000 7ff498baa640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff494102090 0x7ff49419e310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00eb10 tx=0x7ff48c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.384+0000 7ff4917fa640 1 -- 172.21.15.33:0/3625907640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c004b20 con 0x7ff494102090 2024-04-02T13:35:49.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.384+0000 7ff4917fa640 1 -- 172.21.15.33:0/3625907640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff48c004cc0 con 0x7ff494102090 2024-04-02T13:35:49.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.384+0000 7ff49a60d640 1 -- 172.21.15.33:0/3625907640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4940ffba0 con 0x7ff494102090 2024-04-02T13:35:49.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.384+0000 7ff4917fa640 1 -- 172.21.15.33:0/3625907640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c010700 con 0x7ff494102090 2024-04-02T13:35:49.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.384+0000 7ff49a60d640 1 -- 172.21.15.33:0/3625907640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff494100070 con 0x7ff494102090 2024-04-02T13:35:49.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.385+0000 7ff49a60d640 1 -- 172.21.15.33:0/3625907640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff494068060 con 0x7ff494102090 2024-04-02T13:35:49.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.387+0000 7ff4917fa640 1 -- 172.21.15.33:0/3625907640 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff48c004e70 con 0x7ff494102090 2024-04-02T13:35:49.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.387+0000 7ff4917fa640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff47c0a3bb0 0x7ff47c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:49.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.387+0000 7ff493fff640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff47c0a3bb0 0x7ff47c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:49.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.387+0000 7ff4917fa640 1 -- 172.21.15.33:0/3625907640 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff48c0d05c0 con 0x7ff494102090 2024-04-02T13:35:49.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.388+0000 7ff493fff640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff47c0a3bb0 0x7ff47c0a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff484005fd0 tx=0x7ff48400d5a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.392+0000 7ff4917fa640 1 -- 172.21.15.33:0/3625907640 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff48c09d420 con 0x7ff494102090 2024-04-02T13:35:49.616 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:35:49 smithi033 podman[70593]: 2024-04-02 13:35:49.373967173 +0000 UTC m=+0.019875715 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:35:49.616 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:35:49 smithi033 podman[70593]: 2024-04-02 13:35:49.492527573 +0000 UTC m=+0.138436099 container create e135d8098509da5639a4aeaecf1105f5c454f6315646b931dee895280408dea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-j, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:35:49.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.620+0000 7ff49a60d640 1 -- 172.21.15.33:0/3625907640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff494107a90 con 0x7ff494102090 2024-04-02T13:35:49.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.621+0000 7ff4917fa640 1 -- 172.21.15.33:0/3625907640 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7ff48c09cd20 con 0x7ff494102090 2024-04-02T13:35:49.624 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:35:49.624 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T13:35:49.624 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2024-04-02T13:35:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.623+0000 7ff49a60d640 1 -- 172.21.15.33:0/3625907640 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff47c0a3bb0 msgr2=0x7ff47c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.623+0000 7ff49a60d640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff47c0a3bb0 0x7ff47c0a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff484005fd0 tx=0x7ff48400d5a0 comp rx=0 tx=0).stop 2024-04-02T13:35:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.623+0000 7ff49a60d640 1 -- 172.21.15.33:0/3625907640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff494102090 msgr2=0x7ff49419e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.623+0000 7ff49a60d640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff494102090 0x7ff49419e310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00eb10 tx=0x7ff48c00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:35:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.623+0000 7ff49a60d640 1 -- 172.21.15.33:0/3625907640 shutdown_connections 2024-04-02T13:35:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.623+0000 7ff49a60d640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff47c0a3bb0 0x7ff47c0a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.623+0000 7ff49a60d640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff494102090 0x7ff49419e310 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.623+0000 7ff49a60d640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff494101610 0x7ff4941972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.623+0000 7ff49a60d640 1 --2- 172.21.15.33:0/3625907640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff494100c50 0x7ff494196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.623+0000 7ff49a60d640 1 -- 172.21.15.33:0/3625907640 >> 172.21.15.33:0/3625907640 conn(0x7ff4940fb700 msgr2=0x7ff494104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.623+0000 7ff49a60d640 1 -- 172.21.15.33:0/3625907640 shutdown_connections 2024-04-02T13:35:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:49.623+0000 7ff49a60d640 1 -- 172.21.15.33:0/3625907640 wait complete. 2024-04-02T13:35:50.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:49 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3625907640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:50.004 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:35:49 smithi033 podman[70593]: 2024-04-02 13:35:49.748745148 +0000 UTC m=+0.394653685 container init e135d8098509da5639a4aeaecf1105f5c454f6315646b931dee895280408dea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-j, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326) 2024-04-02T13:35:50.004 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:35:49 smithi033 podman[70593]: 2024-04-02 13:35:49.751623514 +0000 UTC m=+0.397532049 container start e135d8098509da5639a4aeaecf1105f5c454f6315646b931dee895280408dea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-j, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-02T13:35:50.004 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:35:49 smithi033 ceph-mds[70642]: -- 172.21.15.33:0/2670409236 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec6f67da40 con 0x55ec6f651000 2024-04-02T13:35:50.004 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:35:49 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-j[70634]: starting mds.j at 2024-04-02T13:35:50.004 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:35:49 smithi033 bash[70593]: e135d8098509da5639a4aeaecf1105f5c454f6315646b931dee895280408dea7 2024-04-02T13:35:50.004 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:35:49 smithi033 systemd[1]: Started Ceph mds.j for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:35:50.183 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:49 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3625907640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:50.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:49 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3625907640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:51.221 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:50 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.33:6834/1918898100,v1:172.21.15.33:6835/1918898100] up:boot 2024-04-02T13:35:51.221 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:50 smithi145 ceph-mon[35646]: fsmap 3 up:standby 2024-04-02T13:35:51.221 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:50 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:35:51.221 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:50 smithi145 ceph-mon[35646]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:51.221 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:50 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:51.221 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:50 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:51.222 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:50 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:51.222 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:50 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:51.222 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:50 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:51.222 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:50 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:51.222 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:50 smithi145 ceph-mon[35646]: Deploying daemon mds.d on smithi033 2024-04-02T13:35:51.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:50 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.33:6834/1918898100,v1:172.21.15.33:6835/1918898100] up:boot 2024-04-02T13:35:51.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:50 smithi050 ceph-mon[35524]: fsmap 3 up:standby 2024-04-02T13:35:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:50 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:35:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:50 smithi050 ceph-mon[35524]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:50 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:50 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:50 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:50 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:50 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:50 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:50 smithi050 ceph-mon[35524]: Deploying daemon mds.d on smithi033 2024-04-02T13:35:51.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:50 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.33:6834/1918898100,v1:172.21.15.33:6835/1918898100] up:boot 2024-04-02T13:35:51.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:50 smithi033 ceph-mon[32390]: fsmap 3 up:standby 2024-04-02T13:35:51.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:50 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:35:51.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:50 smithi033 ceph-mon[32390]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:51.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:50 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:51.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:50 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:51.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:50 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:51.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:50 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:51.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:50 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:51.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:50 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:51.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:50 smithi033 ceph-mon[32390]: Deploying daemon mds.d on smithi033 2024-04-02T13:35:51.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:35:51.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.755+0000 7fdb0067b640 1 -- 172.21.15.33:0/682085433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8101c10 msgr2=0x7fdaf8102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:51.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.755+0000 7fdb0067b640 1 --2- 172.21.15.33:0/682085433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8101c10 0x7fdaf8102090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009900 tx=0x7fdaec02bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:51.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.755+0000 7fdb0067b640 1 -- 172.21.15.33:0/682085433 shutdown_connections 2024-04-02T13:35:51.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.755+0000 7fdb0067b640 1 --2- 172.21.15.33:0/682085433 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdaf81025d0 0x7fdaf8108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:51.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.755+0000 7fdb0067b640 1 --2- 172.21.15.33:0/682085433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8101c10 0x7fdaf8102090 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.755+0000 7fdb0067b640 1 --2- 172.21.15.33:0/682085433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdaf8100a20 0x7fdaf8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.755+0000 7fdb0067b640 1 -- 172.21.15.33:0/682085433 >> 172.21.15.33:0/682085433 conn(0x7fdaf80fbf90 msgr2=0x7fdaf80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.755+0000 7fdb0067b640 1 -- 172.21.15.33:0/682085433 shutdown_connections 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.755+0000 7fdb0067b640 1 -- 172.21.15.33:0/682085433 wait complete. 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.755+0000 7fdb0067b640 1 Processor -- start 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.755+0000 7fdb0067b640 1 -- start start 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdb0067b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdaf8100a20 0x7fdaf8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdb0067b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdaf8101c10 0x7fdaf8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdb0067b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf81025d0 0x7fdaf819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdb0067b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf810d790 con 0x7fdaf81025d0 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdb0067b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf810d610 con 0x7fdaf8100a20 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdb0067b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf810d910 con 0x7fdaf8101c10 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdafec18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf81025d0 0x7fdaf819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdafec18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf81025d0 0x7fdaf819f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39314/0 (socket says 172.21.15.33:39314) 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdafec18640 1 -- 172.21.15.33:0/1740429265 learned_addr learned my addr 172.21.15.33:0/1740429265 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdafdc16640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdaf8101c10 0x7fdaf8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdafe417640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdaf8100a20 0x7fdaf8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdafec18640 1 -- 172.21.15.33:0/1740429265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdaf8101c10 msgr2=0x7fdaf8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdafec18640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdaf8101c10 0x7fdaf8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdafec18640 1 -- 172.21.15.33:0/1740429265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdaf8100a20 msgr2=0x7fdaf8197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdafec18640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdaf8100a20 0x7fdaf8197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdafec18640 1 -- 172.21.15.33:0/1740429265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf819fa60 con 0x7fdaf81025d0 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdafec18640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf81025d0 0x7fdaf819f3f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdae800d750 tx=0x7fdae800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdaf77fe640 1 -- 172.21.15.33:0/1740429265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8017070 con 0x7fdaf81025d0 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdb0067b640 1 -- 172.21.15.33:0/1740429265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf819fd50 con 0x7fdaf81025d0 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdaf77fe640 1 -- 172.21.15.33:0/1740429265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdae80054f0 con 0x7fdaf81025d0 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdaf77fe640 1 -- 172.21.15.33:0/1740429265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8004cf0 con 0x7fdaf81025d0 2024-04-02T13:35:51.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.756+0000 7fdb0067b640 1 -- 172.21.15.33:0/1740429265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf81a01c0 con 0x7fdaf81025d0 2024-04-02T13:35:51.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.757+0000 7fdb0067b640 1 -- 172.21.15.33:0/1740429265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf8107d90 con 0x7fdaf81025d0 2024-04-02T13:35:51.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.758+0000 7fdaf77fe640 1 -- 172.21.15.33:0/1740429265 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdae8004e90 con 0x7fdaf81025d0 2024-04-02T13:35:51.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.759+0000 7fdaf77fe640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fdae00a3bb0 0x7fdae00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:51.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.759+0000 7fdafe417640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fdae00a3bb0 0x7fdae00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:51.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.759+0000 7fdaf77fe640 1 -- 172.21.15.33:0/1740429265 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdae80cf9d0 con 0x7fdaf81025d0 2024-04-02T13:35:51.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.760+0000 7fdafe417640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fdae00a3bb0 0x7fdae00a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000a300 tx=0x7fdaf000a290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:51.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:51.764+0000 7fdaf77fe640 1 -- 172.21.15.33:0/1740429265 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdae809c7b0 con 0x7fdaf81025d0 2024-04-02T13:35:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.004+0000 7fdb0067b640 1 -- 172.21.15.33:0/1740429265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaf80627a0 con 0x7fdaf81025d0 2024-04-02T13:35:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.005+0000 7fdaf77fe640 1 -- 172.21.15.33:0/1740429265 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fdae809c0b0 con 0x7fdaf81025d0 2024-04-02T13:35:52.007 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:35:52.007 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T13:35:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 4 2024-04-02T13:35:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.007+0000 7fdb0067b640 1 -- 172.21.15.33:0/1740429265 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fdae00a3bb0 msgr2=0x7fdae00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.007+0000 7fdb0067b640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fdae00a3bb0 0x7fdae00a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000a300 tx=0x7fdaf000a290 comp rx=0 tx=0).stop 2024-04-02T13:35:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.007+0000 7fdb0067b640 1 -- 172.21.15.33:0/1740429265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf81025d0 msgr2=0x7fdaf819f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.007+0000 7fdb0067b640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf81025d0 0x7fdaf819f3f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdae800d750 tx=0x7fdae800dc10 comp rx=0 tx=0).stop 2024-04-02T13:35:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.007+0000 7fdb0067b640 1 -- 172.21.15.33:0/1740429265 shutdown_connections 2024-04-02T13:35:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.007+0000 7fdb0067b640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fdae00a3bb0 0x7fdae00a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.007+0000 7fdb0067b640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf81025d0 0x7fdaf819f3f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.007+0000 7fdb0067b640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdaf8101c10 0x7fdaf8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.007+0000 7fdb0067b640 1 --2- 172.21.15.33:0/1740429265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdaf8100a20 0x7fdaf8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.007+0000 7fdb0067b640 1 -- 172.21.15.33:0/1740429265 >> 172.21.15.33:0/1740429265 conn(0x7fdaf80fbf90 msgr2=0x7fdaf8104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.007+0000 7fdb0067b640 1 -- 172.21.15.33:0/1740429265 shutdown_connections 2024-04-02T13:35:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:52.007+0000 7fdb0067b640 1 -- 172.21.15.33:0/1740429265 wait complete. 2024-04-02T13:35:52.397 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:35:52 smithi033 systemd[1]: Starting Ceph mds.d for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:35:52.754 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:35:52 smithi033 podman[70974]: 2024-04-02 13:35:52.297129846 +0000 UTC m=+0.017648608 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:35:52.754 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:35:52 smithi033 podman[70974]: 2024-04-02 13:35:52.418719318 +0000 UTC m=+0.139238060 container create 34b9deba6824cbc2314f52e10e4a07805d7c6f053beae312bcf34fe4b97d5a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-d, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:35:53.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:52 smithi145 ceph-mon[35646]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:53.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:52 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1740429265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:53.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:52 smithi050 ceph-mon[35524]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:53.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:52 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1740429265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:53.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:52 smithi033 ceph-mon[32390]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:53.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:52 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1740429265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:53.254 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:35:52 smithi033 podman[70974]: 2024-04-02 13:35:52.796431103 +0000 UTC m=+0.516949848 container init 34b9deba6824cbc2314f52e10e4a07805d7c6f053beae312bcf34fe4b97d5a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:35:53.254 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:35:52 smithi033 podman[70974]: 2024-04-02 13:35:52.799127658 +0000 UTC m=+0.519646401 container start 34b9deba6824cbc2314f52e10e4a07805d7c6f053beae312bcf34fe4b97d5a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:35:53.254 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:35:52 smithi033 ceph-mds[70993]: -- 172.21.15.33:0/2964752115 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5599b8b59a40 con 0x5599b8b2cc00 2024-04-02T13:35:53.254 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:35:52 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-d[70989]: starting mds.d at 2024-04-02T13:35:53.254 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:35:52 smithi033 bash[70974]: 34b9deba6824cbc2314f52e10e4a07805d7c6f053beae312bcf34fe4b97d5a80 2024-04-02T13:35:53.254 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:35:52 smithi033 systemd[1]: Started Ceph mds.d for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:35:53.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:53 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.33:6836/1083419876,v1:172.21.15.33:6837/1083419876] up:boot 2024-04-02T13:35:53.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:53 smithi050 ceph-mon[35524]: fsmap 4 up:standby 2024-04-02T13:35:53.945 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:53 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:35:53.945 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:53 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:53.945 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:53 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:53.945 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:53 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:53.945 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:53 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:53.945 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:53 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:53.945 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:53 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:53.945 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:53 smithi050 ceph-mon[35524]: Deploying daemon mds.e on smithi050 2024-04-02T13:35:54.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:35:54.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.138+0000 7f3788550640 1 -- 172.21.15.33:0/264937148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780100df0 msgr2=0x7f3780101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:54.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.138+0000 7f3788550640 1 --2- 172.21.15.33:0/264937148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780100df0 0x7f3780101250 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3770009900 tx=0x7f377002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:35:54.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 -- 172.21.15.33:0/264937148 shutdown_connections 2024-04-02T13:35:54.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 --2- 172.21.15.33:0/264937148 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3780101790 0x7f3780101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:54.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 --2- 172.21.15.33:0/264937148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780100df0 0x7f3780101250 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:54.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 --2- 172.21.15.33:0/264937148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3780100490 0x7f37801008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:54.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 -- 172.21.15.33:0/264937148 >> 172.21.15.33:0/264937148 conn(0x7f37800fc030 msgr2=0x7f37800fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:54.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 -- 172.21.15.33:0/264937148 shutdown_connections 2024-04-02T13:35:54.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 -- 172.21.15.33:0/264937148 wait complete. 2024-04-02T13:35:54.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 Processor -- start 2024-04-02T13:35:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 -- start start 2024-04-02T13:35:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3780100490 0x7f378019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780100df0 0x7f378019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3780101790 0x7f37801a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f378010f980 con 0x7f3780100df0 2024-04-02T13:35:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f378010f800 con 0x7f3780100490 2024-04-02T13:35:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3788550640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f378010fb00 con 0x7f3780101790 2024-04-02T13:35:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3785aeb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780100df0 0x7f378019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3785aeb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780100df0 0x7f378019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38772/0 (socket says 172.21.15.33:38772) 2024-04-02T13:35:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.139+0000 7f3785aeb640 1 -- 172.21.15.33:0/3312152657 learned_addr learned my addr 172.21.15.33:0/3312152657 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:35:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f37862ec640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3780100490 0x7f378019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f3786aed640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3780101790 0x7f37801a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f3785aeb640 1 -- 172.21.15.33:0/3312152657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3780101790 msgr2=0x7f37801a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f3785aeb640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3780101790 0x7f37801a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f3785aeb640 1 -- 172.21.15.33:0/3312152657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3780100490 msgr2=0x7f378019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f3785aeb640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3780100490 0x7f378019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f3785aeb640 1 -- 172.21.15.33:0/3312152657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37801a1c20 con 0x7f3780100df0 2024-04-02T13:35:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f3786aed640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3780101790 0x7f37801a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f37862ec640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3780100490 0x7f378019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f3785aeb640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780100df0 0x7f378019a540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3770009230 tx=0x7f3770005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f37777fe640 1 -- 172.21.15.33:0/3312152657 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770043020 con 0x7f3780100df0 2024-04-02T13:35:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f37777fe640 1 -- 172.21.15.33:0/3312152657 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3770005470 con 0x7f3780100df0 2024-04-02T13:35:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f3788550640 1 -- 172.21.15.33:0/3312152657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37801a1eb0 con 0x7f3780100df0 2024-04-02T13:35:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f37777fe640 1 -- 172.21.15.33:0/3312152657 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770003b80 con 0x7f3780100df0 2024-04-02T13:35:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.140+0000 7f3788550640 1 -- 172.21.15.33:0/3312152657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37801a2370 con 0x7f3780100df0 2024-04-02T13:35:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.141+0000 7f3788550640 1 -- 172.21.15.33:0/3312152657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f378010b1e0 con 0x7f3780100df0 2024-04-02T13:35:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.142+0000 7f37777fe640 1 -- 172.21.15.33:0/3312152657 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3770046020 con 0x7f3780100df0 2024-04-02T13:35:54.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.143+0000 7f37777fe640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f376c0a3bb0 0x7f376c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:54.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.143+0000 7f37862ec640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f376c0a3bb0 0x7f376c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:54.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.143+0000 7f37777fe640 1 -- 172.21.15.33:0/3312152657 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f37700f2630 con 0x7f3780100df0 2024-04-02T13:35:54.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.143+0000 7f37862ec640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f376c0a3bb0 0x7f376c0a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f37780058c0 tx=0x7f3778005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:54.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.146+0000 7f37777fe640 1 -- 172.21.15.33:0/3312152657 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f377004d020 con 0x7f3780100df0 2024-04-02T13:35:54.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:53 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.33:6836/1083419876,v1:172.21.15.33:6837/1083419876] up:boot 2024-04-02T13:35:54.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:53 smithi145 ceph-mon[35646]: fsmap 4 up:standby 2024-04-02T13:35:54.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:53 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:35:54.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:53 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:54.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:53 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:54.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:53 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:54.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:53 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:54.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:53 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:54.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:53 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:54.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:53 smithi145 ceph-mon[35646]: Deploying daemon mds.e on smithi050 2024-04-02T13:35:54.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:53 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.33:6836/1083419876,v1:172.21.15.33:6837/1083419876] up:boot 2024-04-02T13:35:54.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:53 smithi033 ceph-mon[32390]: fsmap 4 up:standby 2024-04-02T13:35:54.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:35:54.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:54.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:54.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:54.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:54.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:54.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:53 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:54.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:53 smithi033 ceph-mon[32390]: Deploying daemon mds.e on smithi050 2024-04-02T13:35:54.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.371+0000 7f3788550640 1 -- 172.21.15.33:0/3312152657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f378002d410 con 0x7f3780100df0 2024-04-02T13:35:54.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.372+0000 7f37777fe640 1 -- 172.21.15.33:0/3312152657 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f37700bf4b0 con 0x7f3780100df0 2024-04-02T13:35:54.375 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:35:54.375 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T13:35:54.375 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 5 2024-04-02T13:35:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.374+0000 7f3788550640 1 -- 172.21.15.33:0/3312152657 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f376c0a3bb0 msgr2=0x7f376c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.374+0000 7f3788550640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f376c0a3bb0 0x7f376c0a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f37780058c0 tx=0x7f3778005850 comp rx=0 tx=0).stop 2024-04-02T13:35:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.374+0000 7f3788550640 1 -- 172.21.15.33:0/3312152657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780100df0 msgr2=0x7f378019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.374+0000 7f3788550640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780100df0 0x7f378019a540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3770009230 tx=0x7f3770005e60 comp rx=0 tx=0).stop 2024-04-02T13:35:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.374+0000 7f3788550640 1 -- 172.21.15.33:0/3312152657 shutdown_connections 2024-04-02T13:35:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.374+0000 7f3788550640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f376c0a3bb0 0x7f376c0a6070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.374+0000 7f3788550640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3780101790 0x7f37801a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.374+0000 7f3788550640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780100df0 0x7f378019a540 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.374+0000 7f3788550640 1 --2- 172.21.15.33:0/3312152657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3780100490 0x7f378019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.374+0000 7f3788550640 1 -- 172.21.15.33:0/3312152657 >> 172.21.15.33:0/3312152657 conn(0x7f37800fc030 msgr2=0x7f37800fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:54.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.374+0000 7f3788550640 1 -- 172.21.15.33:0/3312152657 shutdown_connections 2024-04-02T13:35:54.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:54.374+0000 7f3788550640 1 -- 172.21.15.33:0/3312152657 wait complete. 2024-04-02T13:35:55.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:54 smithi050 ceph-mon[35524]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:55.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:54 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3312152657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:55.245 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:35:55 smithi050 systemd[1]: Starting Ceph mds.e for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:35:55.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:54 smithi033 ceph-mon[32390]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:55.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:54 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3312152657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:55.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:54 smithi145 ceph-mon[35646]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:55.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:54 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3312152657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:55.689 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:35:55 smithi050 podman[65674]: 2024-04-02 13:35:55.31920102 +0000 UTC m=+0.017638588 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:35:55.690 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:35:55 smithi050 podman[65674]: 2024-04-02 13:35:55.433498957 +0000 UTC m=+0.131936496 container create 985dba5ca7f63f0e2567ae6d5f68c438b8938b77aa69cc7a53a5b8981a635d6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-e, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True) 2024-04-02T13:35:55.690 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:35:55 smithi050 podman[65674]: 2024-04-02 13:35:55.661477517 +0000 UTC m=+0.359915058 container init 985dba5ca7f63f0e2567ae6d5f68c438b8938b77aa69cc7a53a5b8981a635d6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-02T13:35:55.690 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:35:55 smithi050 podman[65674]: 2024-04-02 13:35:55.664317186 +0000 UTC m=+0.362754733 container start 985dba5ca7f63f0e2567ae6d5f68c438b8938b77aa69cc7a53a5b8981a635d6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-04-02T13:35:55.994 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:35:55 smithi050 ceph-mds[65694]: -- 172.21.15.50:0/1868248896 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a0d829a40 con 0x559a0d7fcc00 2024-04-02T13:35:55.995 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:35:55 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-e[65690]: starting mds.e at 2024-04-02T13:35:55.995 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:35:55 smithi050 bash[65674]: 985dba5ca7f63f0e2567ae6d5f68c438b8938b77aa69cc7a53a5b8981a635d6c 2024-04-02T13:35:55.995 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:35:55 smithi050 systemd[1]: Started Ceph mds.e for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:35:56.388 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:35:56.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.501+0000 7fc721849640 1 -- 172.21.15.33:0/2831800022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc71c100a20 msgr2=0x7fc71c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:56.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.501+0000 7fc721849640 1 --2- 172.21.15.33:0/2831800022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc71c100a20 0x7fc71c100e40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc714009900 tx=0x7fc71402bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:56.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.501+0000 7fc721849640 1 -- 172.21.15.33:0/2831800022 shutdown_connections 2024-04-02T13:35:56.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.501+0000 7fc721849640 1 --2- 172.21.15.33:0/2831800022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc71c1025d0 0x7fc71c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:56.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.501+0000 7fc721849640 1 --2- 172.21.15.33:0/2831800022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc71c101c10 0x7fc71c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:56.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.501+0000 7fc721849640 1 --2- 172.21.15.33:0/2831800022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc71c100a20 0x7fc71c100e40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:56.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.501+0000 7fc721849640 1 -- 172.21.15.33:0/2831800022 >> 172.21.15.33:0/2831800022 conn(0x7fc71c0fbf90 msgr2=0x7fc71c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:56.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.502+0000 7fc721849640 1 -- 172.21.15.33:0/2831800022 shutdown_connections 2024-04-02T13:35:56.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.502+0000 7fc721849640 1 -- 172.21.15.33:0/2831800022 wait complete. 2024-04-02T13:35:56.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.502+0000 7fc721849640 1 Processor -- start 2024-04-02T13:35:56.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.502+0000 7fc721849640 1 -- start start 2024-04-02T13:35:56.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.502+0000 7fc721849640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc71c100a20 0x7fc71c10b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:56.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.502+0000 7fc721849640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc71c101c10 0x7fc71c1099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.502+0000 7fc721849640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc71c1025d0 0x7fc71c109f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.502+0000 7fc721849640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c10d7d0 con 0x7fc71c100a20 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.502+0000 7fc721849640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c10d650 con 0x7fc71c1025d0 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.502+0000 7fc721849640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c10d950 con 0x7fc71c101c10 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc71affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc71c100a20 0x7fc71c10b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc71affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc71c100a20 0x7fc71c10b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38794/0 (socket says 172.21.15.33:38794) 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc71affd640 1 -- 172.21.15.33:0/937887019 learned_addr learned my addr 172.21.15.33:0/937887019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc71a7fc640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc71c101c10 0x7fc71c1099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc71b7fe640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc71c1025d0 0x7fc71c109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc71affd640 1 -- 172.21.15.33:0/937887019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc71c101c10 msgr2=0x7fc71c1099e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc71affd640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc71c101c10 0x7fc71c1099e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc71affd640 1 -- 172.21.15.33:0/937887019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc71c1025d0 msgr2=0x7fc71c109f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc71affd640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc71c1025d0 0x7fc71c109f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc71affd640 1 -- 172.21.15.33:0/937887019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c10a7b0 con 0x7fc71c100a20 2024-04-02T13:35:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc71affd640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc71c100a20 0x7fc71c10b3a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc714000c00 tx=0x7fc714004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:56.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc720847640 1 -- 172.21.15.33:0/937887019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714043020 con 0x7fc71c100a20 2024-04-02T13:35:56.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc720847640 1 -- 172.21.15.33:0/937887019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc714005460 con 0x7fc71c100a20 2024-04-02T13:35:56.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc721849640 1 -- 172.21.15.33:0/937887019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c1a1f90 con 0x7fc71c100a20 2024-04-02T13:35:56.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc720847640 1 -- 172.21.15.33:0/937887019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714003d00 con 0x7fc71c100a20 2024-04-02T13:35:56.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.503+0000 7fc721849640 1 -- 172.21.15.33:0/937887019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c1a23f0 con 0x7fc71c100a20 2024-04-02T13:35:56.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.504+0000 7fc721849640 1 -- 172.21.15.33:0/937887019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71c107d40 con 0x7fc71c100a20 2024-04-02T13:35:56.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.505+0000 7fc720847640 1 -- 172.21.15.33:0/937887019 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc714046020 con 0x7fc71c100a20 2024-04-02T13:35:56.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.506+0000 7fc720847640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc7080a3c30 0x7fc7080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:56.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.506+0000 7fc71a7fc640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc7080a3c30 0x7fc7080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:56.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.506+0000 7fc720847640 1 -- 172.21.15.33:0/937887019 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc7140f2030 con 0x7fc71c100a20 2024-04-02T13:35:56.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.506+0000 7fc71a7fc640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc7080a3c30 0x7fc7080a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc71c10b110 tx=0x7fc704005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:56.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.511+0000 7fc720847640 1 -- 172.21.15.33:0/937887019 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc71404d020 con 0x7fc71c100a20 2024-04-02T13:35:56.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.737+0000 7fc721849640 1 -- 172.21.15.33:0/937887019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc71c02d5c0 con 0x7fc71c100a20 2024-04-02T13:35:56.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.737+0000 7fc720847640 1 -- 172.21.15.33:0/937887019 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7fc7140beeb0 con 0x7fc71c100a20 2024-04-02T13:35:56.740 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:35:56.740 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-02T13:35:56.741 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 6 2024-04-02T13:35:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.739+0000 7fc721849640 1 -- 172.21.15.33:0/937887019 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc7080a3c30 msgr2=0x7fc7080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.739+0000 7fc721849640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc7080a3c30 0x7fc7080a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc71c10b110 tx=0x7fc704005f90 comp rx=0 tx=0).stop 2024-04-02T13:35:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.739+0000 7fc721849640 1 -- 172.21.15.33:0/937887019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc71c100a20 msgr2=0x7fc71c10b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:56.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.739+0000 7fc721849640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc71c100a20 0x7fc71c10b3a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc714000c00 tx=0x7fc714004e20 comp rx=0 tx=0).stop 2024-04-02T13:35:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.739+0000 7fc721849640 1 -- 172.21.15.33:0/937887019 shutdown_connections 2024-04-02T13:35:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.739+0000 7fc721849640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc7080a3c30 0x7fc7080a60f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.739+0000 7fc721849640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc71c1025d0 0x7fc71c109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.739+0000 7fc721849640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc71c101c10 0x7fc71c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.739+0000 7fc721849640 1 --2- 172.21.15.33:0/937887019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc71c100a20 0x7fc71c10b3a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.739+0000 7fc721849640 1 -- 172.21.15.33:0/937887019 >> 172.21.15.33:0/937887019 conn(0x7fc71c0fbf90 msgr2=0x7fc71c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.740+0000 7fc721849640 1 -- 172.21.15.33:0/937887019 shutdown_connections 2024-04-02T13:35:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:56.740+0000 7fc721849640 1 -- 172.21.15.33:0/937887019 wait complete. 2024-04-02T13:35:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:56 smithi033 ceph-mon[32390]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:56 smithi033 ceph-mon[32390]: Deploying daemon mds.f on smithi145 2024-04-02T13:35:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:56 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] up:boot 2024-04-02T13:35:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:56 smithi033 ceph-mon[32390]: fsmap 5 up:standby 2024-04-02T13:35:57.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:35:57.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:56 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/937887019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:57.123 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:56 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:57.124 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:56 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:57.124 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:56 smithi145 ceph-mon[35646]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:57.124 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:56 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:57.124 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:56 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:57.124 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:56 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:57.124 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:56 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:57.124 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:56 smithi145 ceph-mon[35646]: Deploying daemon mds.f on smithi145 2024-04-02T13:35:57.124 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:56 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] up:boot 2024-04-02T13:35:57.124 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:56 smithi145 ceph-mon[35646]: fsmap 5 up:standby 2024-04-02T13:35:57.124 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:56 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:35:57.124 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:56 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/937887019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:57.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:56 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:57.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:56 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:57.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:56 smithi050 ceph-mon[35524]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:57.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:56 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:57.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:56 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:57.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:56 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:57.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:56 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:57.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:56 smithi050 ceph-mon[35524]: Deploying daemon mds.f on smithi145 2024-04-02T13:35:57.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:56 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] up:boot 2024-04-02T13:35:57.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:56 smithi050 ceph-mon[35524]: fsmap 5 up:standby 2024-04-02T13:35:57.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:56 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:35:57.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:56 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/937887019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:58.227 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:35:58 smithi145 systemd[1]: Starting Ceph mds.f for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:35:58.479 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:35:58 smithi145 podman[65294]: 2024-04-02 13:35:58.12774168 +0000 UTC m=+0.017529264 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:35:58.479 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:35:58 smithi145 podman[65294]: 2024-04-02 13:35:58.237537595 +0000 UTC m=+0.127325166 container create feb4f52be8ac04c85cb29bf8691834ec5110b720f636e591037ddbe3b1dbd7d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-f, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:35:58.754 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:35:58.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.864+0000 7fc9a262a640 1 -- 172.21.15.33:0/1078232302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc99c100a20 msgr2=0x7fc99c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:58.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.864+0000 7fc9a262a640 1 --2- 172.21.15.33:0/1078232302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc99c100a20 0x7fc99c100e40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009900 tx=0x7fc98c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:58.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.864+0000 7fc9a262a640 1 -- 172.21.15.33:0/1078232302 shutdown_connections 2024-04-02T13:35:58.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.864+0000 7fc9a262a640 1 --2- 172.21.15.33:0/1078232302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc99c1025d0 0x7fc99c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:58.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.864+0000 7fc9a262a640 1 --2- 172.21.15.33:0/1078232302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc99c101c10 0x7fc99c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:58.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.864+0000 7fc9a262a640 1 --2- 172.21.15.33:0/1078232302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc99c100a20 0x7fc99c100e40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:58.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.864+0000 7fc9a262a640 1 -- 172.21.15.33:0/1078232302 >> 172.21.15.33:0/1078232302 conn(0x7fc99c0fbfb0 msgr2=0x7fc99c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.864+0000 7fc9a262a640 1 -- 172.21.15.33:0/1078232302 shutdown_connections 2024-04-02T13:35:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.864+0000 7fc9a262a640 1 -- 172.21.15.33:0/1078232302 wait complete. 2024-04-02T13:35:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.865+0000 7fc9a262a640 1 Processor -- start 2024-04-02T13:35:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.865+0000 7fc9a262a640 1 -- start start 2024-04-02T13:35:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.865+0000 7fc9a262a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc99c100a20 0x7fc99c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.865+0000 7fc9a262a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc99c101c10 0x7fc99c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.865+0000 7fc9a262a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc99c1025d0 0x7fc99c19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.865+0000 7fc9a262a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c10c680 con 0x7fc99c1025d0 2024-04-02T13:35:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.865+0000 7fc9a262a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c10c500 con 0x7fc99c101c10 2024-04-02T13:35:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.865+0000 7fc9a262a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c10c800 con 0x7fc99c100a20 2024-04-02T13:35:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.865+0000 7fc9a0bc7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc99c1025d0 0x7fc99c19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.865+0000 7fc99bfff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc99c100a20 0x7fc99c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.865+0000 7fc9a0bc7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc99c1025d0 0x7fc99c19f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38814/0 (socket says 172.21.15.33:38814) 2024-04-02T13:35:58.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.865+0000 7fc9a0bc7640 1 -- 172.21.15.33:0/902309148 learned_addr learned my addr 172.21.15.33:0/902309148 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:35:58.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.866+0000 7fc99b7fe640 1 --2- 172.21.15.33:0/902309148 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc99c101c10 0x7fc99c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:58.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.866+0000 7fc9a0bc7640 1 -- 172.21.15.33:0/902309148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc99c100a20 msgr2=0x7fc99c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:58.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.866+0000 7fc9a0bc7640 1 --2- 172.21.15.33:0/902309148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc99c100a20 0x7fc99c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:58.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.866+0000 7fc9a0bc7640 1 -- 172.21.15.33:0/902309148 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc99c101c10 msgr2=0x7fc99c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:58.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.866+0000 7fc9a0bc7640 1 --2- 172.21.15.33:0/902309148 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc99c101c10 0x7fc99c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:58.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.866+0000 7fc9a0bc7640 1 -- 172.21.15.33:0/902309148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c19fac0 con 0x7fc99c1025d0 2024-04-02T13:35:58.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.866+0000 7fc9a0bc7640 1 --2- 172.21.15.33:0/902309148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc99c1025d0 0x7fc99c19f450 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc99400eb10 tx=0x7fc99400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:58.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.866+0000 7fc9997fa640 1 -- 172.21.15.33:0/902309148 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994004b20 con 0x7fc99c1025d0 2024-04-02T13:35:58.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.866+0000 7fc9a262a640 1 -- 172.21.15.33:0/902309148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99c19fdb0 con 0x7fc99c1025d0 2024-04-02T13:35:58.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.866+0000 7fc9997fa640 1 -- 172.21.15.33:0/902309148 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc994004cc0 con 0x7fc99c1025d0 2024-04-02T13:35:58.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.866+0000 7fc9997fa640 1 -- 172.21.15.33:0/902309148 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994010700 con 0x7fc99c1025d0 2024-04-02T13:35:58.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.866+0000 7fc9a262a640 1 -- 172.21.15.33:0/902309148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99c1a0220 con 0x7fc99c1025d0 2024-04-02T13:35:58.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.867+0000 7fc9a262a640 1 -- 172.21.15.33:0/902309148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99c1a04e0 con 0x7fc99c1025d0 2024-04-02T13:35:58.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.868+0000 7fc9997fa640 1 -- 172.21.15.33:0/902309148 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc994004e70 con 0x7fc99c1025d0 2024-04-02T13:35:58.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.869+0000 7fc9997fa640 1 --2- 172.21.15.33:0/902309148 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc9840a3bb0 0x7fc9840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:58.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.869+0000 7fc99bfff640 1 --2- 172.21.15.33:0/902309148 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc9840a3bb0 0x7fc9840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:58.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.869+0000 7fc9997fa640 1 -- 172.21.15.33:0/902309148 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc9940d0500 con 0x7fc99c1025d0 2024-04-02T13:35:58.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.869+0000 7fc99bfff640 1 --2- 172.21.15.33:0/902309148 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc9840a3bb0 0x7fc9840a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc98c000c00 tx=0x7fc98c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:58.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:58.873+0000 7fc9997fa640 1 -- 172.21.15.33:0/902309148 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc99409d380 con 0x7fc99c1025d0 2024-04-02T13:35:58.979 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:35:58 smithi145 podman[65294]: 2024-04-02 13:35:58.49882109 +0000 UTC m=+0.388608663 container init feb4f52be8ac04c85cb29bf8691834ec5110b720f636e591037ddbe3b1dbd7d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-04-02T13:35:58.979 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:35:58 smithi145 podman[65294]: 2024-04-02 13:35:58.501644345 +0000 UTC m=+0.391431926 container start feb4f52be8ac04c85cb29bf8691834ec5110b720f636e591037ddbe3b1dbd7d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_CLEAN=True, ceph=True) 2024-04-02T13:35:58.979 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:35:58 smithi145 ceph-mds[65313]: -- 172.21.15.145:0/3901883661 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557809241a40 con 0x557809214c00 2024-04-02T13:35:58.979 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:35:58 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-f[65309]: starting mds.f at 2024-04-02T13:35:58.979 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:35:58 smithi145 bash[65294]: feb4f52be8ac04c85cb29bf8691834ec5110b720f636e591037ddbe3b1dbd7d0 2024-04-02T13:35:58.979 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:35:58 smithi145 systemd[1]: Started Ceph mds.f for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.099+0000 7fc9a262a640 1 -- 172.21.15.33:0/902309148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc99c1a0840 con 0x7fc99c1025d0 2024-04-02T13:35:59.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.100+0000 7fc9997fa640 1 -- 172.21.15.33:0/902309148 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7fc99409cc80 con 0x7fc99c1025d0 2024-04-02T13:35:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:35:59.103 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-02T13:35:59.103 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2024-04-02T13:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.102+0000 7fc9a262a640 1 -- 172.21.15.33:0/902309148 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc9840a3bb0 msgr2=0x7fc9840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.102+0000 7fc9a262a640 1 --2- 172.21.15.33:0/902309148 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc9840a3bb0 0x7fc9840a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc98c000c00 tx=0x7fc98c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.102+0000 7fc9a262a640 1 -- 172.21.15.33:0/902309148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc99c1025d0 msgr2=0x7fc99c19f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.102+0000 7fc9a262a640 1 --2- 172.21.15.33:0/902309148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc99c1025d0 0x7fc99c19f450 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc99400eb10 tx=0x7fc99400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.102+0000 7fc9a262a640 1 -- 172.21.15.33:0/902309148 shutdown_connections 2024-04-02T13:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.102+0000 7fc9a262a640 1 --2- 172.21.15.33:0/902309148 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc9840a3bb0 0x7fc9840a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.102+0000 7fc9a262a640 1 --2- 172.21.15.33:0/902309148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc99c1025d0 0x7fc99c19f450 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.102+0000 7fc9a262a640 1 --2- 172.21.15.33:0/902309148 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc99c101c10 0x7fc99c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.102+0000 7fc9a262a640 1 --2- 172.21.15.33:0/902309148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc99c100a20 0x7fc99c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.102+0000 7fc9a262a640 1 -- 172.21.15.33:0/902309148 >> 172.21.15.33:0/902309148 conn(0x7fc99c0fbfb0 msgr2=0x7fc99c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.102+0000 7fc9a262a640 1 -- 172.21.15.33:0/902309148 shutdown_connections 2024-04-02T13:35:59.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:35:59.102+0000 7fc9a262a640 1 -- 172.21.15.33:0/902309148 wait complete. 2024-04-02T13:35:59.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:59 smithi050 ceph-mon[35524]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:59.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:59 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:59.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:59 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:59.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:59 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:59.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:59 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:59.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:59 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:59.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:59 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:59.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:59 smithi050 ceph-mon[35524]: Deploying daemon mds.c on smithi145 2024-04-02T13:35:59.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:59 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.145:6834/4110737670,v1:172.21.15.145:6835/4110737670] up:boot 2024-04-02T13:35:59.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:59 smithi050 ceph-mon[35524]: fsmap 6 up:standby 2024-04-02T13:35:59.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:59 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:35:59.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:35:59 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/902309148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:59 smithi145 ceph-mon[35646]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:59 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:59 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:59 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:35:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:59 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:35:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:59 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:35:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:59 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:59 smithi145 ceph-mon[35646]: Deploying daemon mds.c on smithi145 2024-04-02T13:35:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:59 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.145:6834/4110737670,v1:172.21.15.145:6835/4110737670] up:boot 2024-04-02T13:35:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:59 smithi145 ceph-mon[35646]: fsmap 6 up:standby 2024-04-02T13:35:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:59 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:35:59.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:35:59 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/902309148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:00.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:59 smithi033 ceph-mon[32390]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:59 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:59 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:59 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:59 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:59 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:59 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:59 smithi033 ceph-mon[32390]: Deploying daemon mds.c on smithi145 2024-04-02T13:36:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:59 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.145:6834/4110737670,v1:172.21.15.145:6835/4110737670] up:boot 2024-04-02T13:36:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:59 smithi033 ceph-mon[32390]: fsmap 6 up:standby 2024-04-02T13:36:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:59 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:36:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:35:59 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/902309148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:00.979 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:36:00 smithi145 systemd[1]: Starting Ceph mds.c for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:36:01.117 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:01.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.229+0000 7fe883086640 1 -- 172.21.15.33:0/2845557411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87c101c10 msgr2=0x7fe87c102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:01.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.229+0000 7fe883086640 1 --2- 172.21.15.33:0/2845557411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87c101c10 0x7fe87c102090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe868009900 tx=0x7fe86802bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:01.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 -- 172.21.15.33:0/2845557411 shutdown_connections 2024-04-02T13:36:01.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 --2- 172.21.15.33:0/2845557411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87c1025d0 0x7fe87c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:01.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 --2- 172.21.15.33:0/2845557411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87c101c10 0x7fe87c102090 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 --2- 172.21.15.33:0/2845557411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe87c100a20 0x7fe87c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 -- 172.21.15.33:0/2845557411 >> 172.21.15.33:0/2845557411 conn(0x7fe87c0fbf90 msgr2=0x7fe87c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 -- 172.21.15.33:0/2845557411 shutdown_connections 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 -- 172.21.15.33:0/2845557411 wait complete. 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 Processor -- start 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 -- start start 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87c100a20 0x7fe87c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87c101c10 0x7fe87c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe87c1025d0 0x7fe87c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c10d7f0 con 0x7fe87c101c10 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c10d670 con 0x7fe87c100a20 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe883086640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c10d970 con 0x7fe87c1025d0 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.230+0000 7fe87bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87c101c10 0x7fe87c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:01.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe880e22640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87c100a20 0x7fe87c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe87bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87c101c10 0x7fe87c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38826/0 (socket says 172.21.15.33:38826) 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe87bfff640 1 -- 172.21.15.33:0/3789634390 learned_addr learned my addr 172.21.15.33:0/3789634390 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe881623640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe87c1025d0 0x7fe87c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe880e22640 1 -- 172.21.15.33:0/3789634390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe87c1025d0 msgr2=0x7fe87c19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe880e22640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe87c1025d0 0x7fe87c19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe880e22640 1 -- 172.21.15.33:0/3789634390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87c101c10 msgr2=0x7fe87c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe880e22640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87c101c10 0x7fe87c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe880e22640 1 -- 172.21.15.33:0/3789634390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87c19fa60 con 0x7fe87c100a20 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe87bfff640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87c101c10 0x7fe87c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe880e22640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87c100a20 0x7fe87c197e40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe87c101a70 tx=0x7fe87000dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe879ffb640 1 -- 172.21.15.33:0/3789634390 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870017070 con 0x7fe87c100a20 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe879ffb640 1 -- 172.21.15.33:0/3789634390 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe870010460 con 0x7fe87c100a20 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe879ffb640 1 -- 172.21.15.33:0/3789634390 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87000f620 con 0x7fe87c100a20 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe883086640 1 -- 172.21.15.33:0/3789634390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe87c19fd50 con 0x7fe87c100a20 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.231+0000 7fe883086640 1 -- 172.21.15.33:0/3789634390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe87c1a0190 con 0x7fe87c100a20 2024-04-02T13:36:01.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.232+0000 7fe883086640 1 -- 172.21.15.33:0/3789634390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe87c107d40 con 0x7fe87c100a20 2024-04-02T13:36:01.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.234+0000 7fe879ffb640 1 -- 172.21.15.33:0/3789634390 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe87001a070 con 0x7fe87c100a20 2024-04-02T13:36:01.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.234+0000 7fe879ffb640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe86c0a3bb0 0x7fe86c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:01.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.234+0000 7fe87bfff640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe86c0a3bb0 0x7fe86c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:01.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.234+0000 7fe879ffb640 1 -- 172.21.15.33:0/3789634390 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe8700cf7d0 con 0x7fe87c100a20 2024-04-02T13:36:01.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.235+0000 7fe87bfff640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe86c0a3bb0 0x7fe86c0a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe86802c710 tx=0x7fe86802c820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:01.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.238+0000 7fe879ffb640 1 -- 172.21.15.33:0/3789634390 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe87009c000 con 0x7fe87c100a20 2024-04-02T13:36:01.332 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:36:01 smithi145 podman[65604]: 2024-04-02 13:36:00.961749785 +0000 UTC m=+0.018172102 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:36:01.333 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:36:01 smithi145 podman[65604]: 2024-04-02 13:36:01.060211558 +0000 UTC m=+0.116633914 container create 21a7f37a992bf39db3c07f28ddc005c7350a84a310ffda028da624fedf019ef6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-c, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-02T13:36:01.333 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:36:01 smithi145 podman[65604]: 2024-04-02 13:36:01.304539936 +0000 UTC m=+0.360962246 container init 21a7f37a992bf39db3c07f28ddc005c7350a84a310ffda028da624fedf019ef6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-c, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-02T13:36:01.333 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:36:01 smithi145 podman[65604]: 2024-04-02 13:36:01.307088097 +0000 UTC m=+0.363510410 container start 21a7f37a992bf39db3c07f28ddc005c7350a84a310ffda028da624fedf019ef6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-c, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-02T13:36:01.333 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:36:01 smithi145 ceph-mds[65623]: -- 172.21.15.145:0/2410574997 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56355b41da40 con 0x56355b3f0c00 2024-04-02T13:36:01.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.465+0000 7fe883086640 1 -- 172.21.15.33:0/3789634390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe87c02d370 con 0x7fe87c100a20 2024-04-02T13:36:01.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.466+0000 7fe879ffb640 1 -- 172.21.15.33:0/3789634390 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7fe8700a0300 con 0x7fe87c100a20 2024-04-02T13:36:01.469 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:01.469 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-02T13:36:01.469 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2024-04-02T13:36:01.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.469+0000 7fe883086640 1 -- 172.21.15.33:0/3789634390 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe86c0a3bb0 msgr2=0x7fe86c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:01.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.469+0000 7fe883086640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe86c0a3bb0 0x7fe86c0a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe86802c710 tx=0x7fe86802c820 comp rx=0 tx=0).stop 2024-04-02T13:36:01.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.469+0000 7fe883086640 1 -- 172.21.15.33:0/3789634390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87c100a20 msgr2=0x7fe87c197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:01.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.469+0000 7fe883086640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87c100a20 0x7fe87c197e40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe87c101a70 tx=0x7fe87000dbb0 comp rx=0 tx=0).stop 2024-04-02T13:36:01.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.469+0000 7fe883086640 1 -- 172.21.15.33:0/3789634390 shutdown_connections 2024-04-02T13:36:01.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.469+0000 7fe883086640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe86c0a3bb0 0x7fe86c0a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:01.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.469+0000 7fe883086640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe87c1025d0 0x7fe87c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:01.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.469+0000 7fe883086640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87c101c10 0x7fe87c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:01.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.469+0000 7fe883086640 1 --2- 172.21.15.33:0/3789634390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe87c100a20 0x7fe87c197e40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:01.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.469+0000 7fe883086640 1 -- 172.21.15.33:0/3789634390 >> 172.21.15.33:0/3789634390 conn(0x7fe87c0fbf90 msgr2=0x7fe87c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:01.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.469+0000 7fe883086640 1 -- 172.21.15.33:0/3789634390 shutdown_connections 2024-04-02T13:36:01.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:01.469+0000 7fe883086640 1 -- 172.21.15.33:0/3789634390 wait complete. 2024-04-02T13:36:01.656 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:36:01 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-c[65619]: starting mds.c at 2024-04-02T13:36:01.656 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:36:01 smithi145 bash[65604]: 21a7f37a992bf39db3c07f28ddc005c7350a84a310ffda028da624fedf019ef6 2024-04-02T13:36:01.656 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:36:01 smithi145 systemd[1]: Started Ceph mds.c for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:36:01.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:01 smithi050 ceph-mon[35524]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:01.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:01 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:01.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:01 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:01.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:01 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3789634390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:01.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:01 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:01.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:01 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:01.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:01 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:01.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:01 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:01.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:01 smithi033 ceph-mon[32390]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:01.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:01 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:01.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:01 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:01.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:01 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3789634390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:01.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:01 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:01.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:01 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:01.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:01 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:01.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:01 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:01.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:01 smithi145 ceph-mon[35646]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:01.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:01 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:01.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:01 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:01.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:01 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3789634390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:01.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:01 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:01.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:01 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:01.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:01 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:01.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:01 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:02.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:02 smithi050 ceph-mon[35524]: Deploying daemon mds.h on smithi050 2024-04-02T13:36:02.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:02 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.145:6836/4234446304,v1:172.21.15.145:6837/4234446304] up:boot 2024-04-02T13:36:02.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:02 smithi050 ceph-mon[35524]: fsmap 7 up:standby 2024-04-02T13:36:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:02 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:36:02.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:02 smithi145 ceph-mon[35646]: Deploying daemon mds.h on smithi050 2024-04-02T13:36:02.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:02 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.145:6836/4234446304,v1:172.21.15.145:6837/4234446304] up:boot 2024-04-02T13:36:02.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:02 smithi145 ceph-mon[35646]: fsmap 7 up:standby 2024-04-02T13:36:02.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:02 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:36:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:02 smithi033 ceph-mon[32390]: Deploying daemon mds.h on smithi050 2024-04-02T13:36:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:02 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.145:6836/4234446304,v1:172.21.15.145:6837/4234446304] up:boot 2024-04-02T13:36:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:02 smithi033 ceph-mon[32390]: fsmap 7 up:standby 2024-04-02T13:36:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:02 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:36:03.485 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.598+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1476282032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e8101c30 msgr2=0x7f01e81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.598+0000 7f01ed2eb640 1 --2- 172.21.15.33:0/1476282032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e8101c30 0x7f01e81020b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009900 tx=0x7f01dc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.598+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1476282032 shutdown_connections 2024-04-02T13:36:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.598+0000 7f01ed2eb640 1 --2- 172.21.15.33:0/1476282032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e81025f0 0x7f01e8108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.598+0000 7f01ed2eb640 1 --2- 172.21.15.33:0/1476282032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e8101c30 0x7f01e81020b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.598+0000 7f01ed2eb640 1 --2- 172.21.15.33:0/1476282032 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01e8100a40 0x7f01e8100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.598+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1476282032 >> 172.21.15.33:0/1476282032 conn(0x7f01e80fbfb0 msgr2=0x7f01e80fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.598+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1476282032 shutdown_connections 2024-04-02T13:36:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.598+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1476282032 wait complete. 2024-04-02T13:36:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.598+0000 7f01ed2eb640 1 Processor -- start 2024-04-02T13:36:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01ed2eb640 1 -- start start 2024-04-02T13:36:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01ed2eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e8100a40 0x7f01e8197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01ed2eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e8101c30 0x7f01e8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01ed2eb640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01e81025f0 0x7f01e819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01ed2eb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01e810c6a0 con 0x7f01e8101c30 2024-04-02T13:36:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01ed2eb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f01e810c520 con 0x7f01e8100a40 2024-04-02T13:36:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01ed2eb640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f01e810c820 con 0x7f01e81025f0 2024-04-02T13:36:03.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01e67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e8101c30 0x7f01e8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:03.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01e67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e8101c30 0x7f01e8198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46508/0 (socket says 172.21.15.33:46508) 2024-04-02T13:36:03.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01e67fc640 1 -- 172.21.15.33:0/1898472465 learned_addr learned my addr 172.21.15.33:0/1898472465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:03.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01e6ffd640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e8100a40 0x7f01e8197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:03.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01e77fe640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01e81025f0 0x7f01e819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:03.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01e67fc640 1 -- 172.21.15.33:0/1898472465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01e81025f0 msgr2=0x7f01e819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01e67fc640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01e81025f0 0x7f01e819f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01e67fc640 1 -- 172.21.15.33:0/1898472465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e8100a40 msgr2=0x7f01e8197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01e67fc640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e8100a40 0x7f01e8197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01e67fc640 1 -- 172.21.15.33:0/1898472465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e819fa30 con 0x7f01e8101c30 2024-04-02T13:36:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01e6ffd640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e8100a40 0x7f01e8197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.599+0000 7f01e67fc640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e8101c30 0x7f01e8198350 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009230 tx=0x7f01dc004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.600+0000 7f01cffff640 1 -- 172.21.15.33:0/1898472465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc043020 con 0x7f01e8101c30 2024-04-02T13:36:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.600+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1898472465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e819fcc0 con 0x7f01e8101c30 2024-04-02T13:36:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.600+0000 7f01cffff640 1 -- 172.21.15.33:0/1898472465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f01dc005460 con 0x7f01e8101c30 2024-04-02T13:36:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.600+0000 7f01cffff640 1 -- 172.21.15.33:0/1898472465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc003c60 con 0x7f01e8101c30 2024-04-02T13:36:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.600+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1898472465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e81a0200 con 0x7f01e8101c30 2024-04-02T13:36:03.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.601+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1898472465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e81a0430 con 0x7f01e8101c30 2024-04-02T13:36:03.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.602+0000 7f01cffff640 1 -- 172.21.15.33:0/1898472465 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f01dc046020 con 0x7f01e8101c30 2024-04-02T13:36:03.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.602+0000 7f01cffff640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f01d00a3bb0 0x7f01d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.602+0000 7f01e6ffd640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f01d00a3bb0 0x7f01d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.603+0000 7f01cffff640 1 -- 172.21.15.33:0/1898472465 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f01dc0f2440 con 0x7f01e8101c30 2024-04-02T13:36:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.603+0000 7f01e6ffd640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f01d00a3bb0 0x7f01d00a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f01e8101a90 tx=0x7f01e000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.606+0000 7f01cffff640 1 -- 172.21.15.33:0/1898472465 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f01dc04d070 con 0x7f01e8101c30 2024-04-02T13:36:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.833+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1898472465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01e81a07c0 con 0x7f01e8101c30 2024-04-02T13:36:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.833+0000 7f01cffff640 1 -- 172.21.15.33:0/1898472465 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f01dc0bf220 con 0x7f01e8101c30 2024-04-02T13:36:03.836 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:03.836 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-02T13:36:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-04-02T13:36:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.835+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1898472465 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f01d00a3bb0 msgr2=0x7f01d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.835+0000 7f01ed2eb640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f01d00a3bb0 0x7f01d00a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f01e8101a90 tx=0x7f01e000a2d0 comp rx=0 tx=0).stop 2024-04-02T13:36:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.835+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1898472465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e8101c30 msgr2=0x7f01e8198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.835+0000 7f01ed2eb640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e8101c30 0x7f01e8198350 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009230 tx=0x7f01dc004e20 comp rx=0 tx=0).stop 2024-04-02T13:36:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.835+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1898472465 shutdown_connections 2024-04-02T13:36:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.835+0000 7f01ed2eb640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f01d00a3bb0 0x7f01d00a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.835+0000 7f01ed2eb640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f01e81025f0 0x7f01e819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.835+0000 7f01ed2eb640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e8101c30 0x7f01e8198350 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.835+0000 7f01ed2eb640 1 --2- 172.21.15.33:0/1898472465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f01e8100a40 0x7f01e8197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.835+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1898472465 >> 172.21.15.33:0/1898472465 conn(0x7f01e80fbfb0 msgr2=0x7f01e80fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.835+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1898472465 shutdown_connections 2024-04-02T13:36:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:03.835+0000 7f01ed2eb640 1 -- 172.21.15.33:0/1898472465 wait complete. 2024-04-02T13:36:03.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:03 smithi033 ceph-mon[32390]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:03.930 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:03 smithi050 ceph-mon[35524]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:03.930 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:36:03 smithi050 systemd[1]: Starting Ceph mds.h for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:36:03.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:03 smithi145 ceph-mon[35646]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:04.187 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:36:03 smithi050 podman[66000]: 2024-04-02 13:36:03.829822948 +0000 UTC m=+0.017553543 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:36:04.188 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:36:03 smithi050 podman[66000]: 2024-04-02 13:36:03.932060555 +0000 UTC m=+0.119791128 container create 398fcfd4da0bb586cd61fdea485b5367c3f6372e26d1bd343cf95967d4824f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:36:04.494 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:36:04 smithi050 podman[66000]: 2024-04-02 13:36:04.185084951 +0000 UTC m=+0.372815541 container init 398fcfd4da0bb586cd61fdea485b5367c3f6372e26d1bd343cf95967d4824f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-h, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-02T13:36:04.495 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:36:04 smithi050 podman[66000]: 2024-04-02 13:36:04.187853774 +0000 UTC m=+0.375584357 container start 398fcfd4da0bb586cd61fdea485b5367c3f6372e26d1bd343cf95967d4824f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-02T13:36:04.495 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:36:04 smithi050 ceph-mds[66019]: -- 172.21.15.50:0/2402309147 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2d5cafa40 con 0x55c2d5c82c00 2024-04-02T13:36:04.495 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:36:04 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-h[66015]: starting mds.h at 2024-04-02T13:36:04.495 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:36:04 smithi050 bash[66000]: 398fcfd4da0bb586cd61fdea485b5367c3f6372e26d1bd343cf95967d4824f49 2024-04-02T13:36:04.495 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:36:04 smithi050 systemd[1]: Started Ceph mds.h for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:36:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:04 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1898472465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:04.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:04 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:04.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:04 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1898472465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:04.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:04.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:04.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:04.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:04.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:04.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:04 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:05.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:04 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1898472465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:05.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:04 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:05.850 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:05.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.964+0000 7fd16b03c640 1 -- 172.21.15.33:0/2919150454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd164100d10 msgr2=0x7fd164101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:05.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.964+0000 7fd16b03c640 1 --2- 172.21.15.33:0/2919150454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd164100d10 0x7fd164101190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd150009900 tx=0x7fd15002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:05.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.964+0000 7fd16b03c640 1 -- 172.21.15.33:0/2919150454 shutdown_connections 2024-04-02T13:36:05.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.964+0000 7fd16b03c640 1 --2- 172.21.15.33:0/2919150454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1641016d0 0x7fd164108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:05.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.964+0000 7fd16b03c640 1 --2- 172.21.15.33:0/2919150454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd164100d10 0x7fd164101190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:05.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.964+0000 7fd16b03c640 1 --2- 172.21.15.33:0/2919150454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd164101fe0 0x7fd164102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.964+0000 7fd16b03c640 1 -- 172.21.15.33:0/2919150454 >> 172.21.15.33:0/2919150454 conn(0x7fd1640fb740 msgr2=0x7fd1640fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.964+0000 7fd16b03c640 1 -- 172.21.15.33:0/2919150454 shutdown_connections 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.964+0000 7fd16b03c640 1 -- 172.21.15.33:0/2919150454 wait complete. 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd16b03c640 1 Processor -- start 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd16b03c640 1 -- start start 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd16b03c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd164100d10 0x7fd164197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd16b03c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1641016d0 0x7fd164198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd16b03c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd164101fe0 0x7fd16419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd16b03c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd16410c6c0 con 0x7fd164101fe0 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd16b03c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd16410c540 con 0x7fd1641016d0 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd16b03c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd16410c840 con 0x7fd164100d10 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd1695d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd164101fe0 0x7fd16419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd1695d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd164101fe0 0x7fd16419f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46530/0 (socket says 172.21.15.33:46530) 2024-04-02T13:36:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd1695d9640 1 -- 172.21.15.33:0/40349032 learned_addr learned my addr 172.21.15.33:0/40349032 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd163fff640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1641016d0 0x7fd164198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd168dd8640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd164100d10 0x7fd164197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd1695d9640 1 -- 172.21.15.33:0/40349032 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd164100d10 msgr2=0x7fd164197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd1695d9640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd164100d10 0x7fd164197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd1695d9640 1 -- 172.21.15.33:0/40349032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1641016d0 msgr2=0x7fd164198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd1695d9640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1641016d0 0x7fd164198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.965+0000 7fd1695d9640 1 -- 172.21.15.33:0/40349032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16419fa50 con 0x7fd164101fe0 2024-04-02T13:36:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.966+0000 7fd1695d9640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd164101fe0 0x7fd16419f3e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00cce0 tx=0x7fd15c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.966+0000 7fd161ffb640 1 -- 172.21.15.33:0/40349032 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00f450 con 0x7fd164101fe0 2024-04-02T13:36:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.966+0000 7fd161ffb640 1 -- 172.21.15.33:0/40349032 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd15c00fac0 con 0x7fd164101fe0 2024-04-02T13:36:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.966+0000 7fd16b03c640 1 -- 172.21.15.33:0/40349032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16419fd40 con 0x7fd164101fe0 2024-04-02T13:36:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.966+0000 7fd161ffb640 1 -- 172.21.15.33:0/40349032 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c018be0 con 0x7fd164101fe0 2024-04-02T13:36:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.966+0000 7fd16b03c640 1 -- 172.21.15.33:0/40349032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1641a01b0 con 0x7fd164101fe0 2024-04-02T13:36:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.967+0000 7fd16b03c640 1 -- 172.21.15.33:0/40349032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1641a0440 con 0x7fd164101fe0 2024-04-02T13:36:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.968+0000 7fd161ffb640 1 -- 172.21.15.33:0/40349032 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd15c00fc70 con 0x7fd164101fe0 2024-04-02T13:36:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.969+0000 7fd161ffb640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd1540a3bb0 0x7fd1540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.969+0000 7fd168dd8640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd1540a3bb0 0x7fd1540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.969+0000 7fd161ffb640 1 -- 172.21.15.33:0/40349032 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd15c0d06d0 con 0x7fd164101fe0 2024-04-02T13:36:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.969+0000 7fd168dd8640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd1540a3bb0 0x7fd1540a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd158005fd0 tx=0x7fd15800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:05.974+0000 7fd161ffb640 1 -- 172.21.15.33:0/40349032 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd15c014030 con 0x7fd164101fe0 2024-04-02T13:36:05.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:05 smithi145 ceph-mon[35646]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:05.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:05 smithi145 ceph-mon[35646]: Deploying daemon mds.i on smithi145 2024-04-02T13:36:05.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:05 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] up:boot 2024-04-02T13:36:05.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:05 smithi145 ceph-mon[35646]: fsmap 8 up:standby 2024-04-02T13:36:05.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:05 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:36:05.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:05 smithi050 ceph-mon[35524]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:05.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:05 smithi050 ceph-mon[35524]: Deploying daemon mds.i on smithi145 2024-04-02T13:36:05.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:05 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] up:boot 2024-04-02T13:36:05.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:05 smithi050 ceph-mon[35524]: fsmap 8 up:standby 2024-04-02T13:36:05.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:05 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:36:06.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:05 smithi033 ceph-mon[32390]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:06.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:05 smithi033 ceph-mon[32390]: Deploying daemon mds.i on smithi145 2024-04-02T13:36:06.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:05 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] up:boot 2024-04-02T13:36:06.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:05 smithi033 ceph-mon[32390]: fsmap 8 up:standby 2024-04-02T13:36:06.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:05 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:36:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.198+0000 7fd16b03c640 1 -- 172.21.15.33:0/40349032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1641a0610 con 0x7fd164101fe0 2024-04-02T13:36:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.199+0000 7fd161ffb640 1 -- 172.21.15.33:0/40349032 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7fd15c09d4b0 con 0x7fd164101fe0 2024-04-02T13:36:06.202 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:06.202 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-02T13:36:06.202 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-04-02T13:36:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.201+0000 7fd16b03c640 1 -- 172.21.15.33:0/40349032 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd1540a3bb0 msgr2=0x7fd1540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.201+0000 7fd16b03c640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd1540a3bb0 0x7fd1540a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd158005fd0 tx=0x7fd15800b5b0 comp rx=0 tx=0).stop 2024-04-02T13:36:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.201+0000 7fd16b03c640 1 -- 172.21.15.33:0/40349032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd164101fe0 msgr2=0x7fd16419f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.201+0000 7fd16b03c640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd164101fe0 0x7fd16419f3e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00cce0 tx=0x7fd15c00a620 comp rx=0 tx=0).stop 2024-04-02T13:36:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.201+0000 7fd16b03c640 1 -- 172.21.15.33:0/40349032 shutdown_connections 2024-04-02T13:36:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.201+0000 7fd16b03c640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd1540a3bb0 0x7fd1540a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.201+0000 7fd16b03c640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd164101fe0 0x7fd16419f3e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.201+0000 7fd16b03c640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1641016d0 0x7fd164198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.201+0000 7fd16b03c640 1 --2- 172.21.15.33:0/40349032 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd164100d10 0x7fd164197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.201+0000 7fd16b03c640 1 -- 172.21.15.33:0/40349032 >> 172.21.15.33:0/40349032 conn(0x7fd1640fb740 msgr2=0x7fd1640fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.201+0000 7fd16b03c640 1 -- 172.21.15.33:0/40349032 shutdown_connections 2024-04-02T13:36:06.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:06.201+0000 7fd16b03c640 1 -- 172.21.15.33:0/40349032 wait complete. 2024-04-02T13:36:06.616 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:36:06 smithi145 systemd[1]: Starting Ceph mds.i for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:36:06.907 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:06 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/40349032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:06.907 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:36:06 smithi145 podman[65909]: 2024-04-02 13:36:06.663261542 +0000 UTC m=+0.017785372 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:36:06.907 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:36:06 smithi145 podman[65909]: 2024-04-02 13:36:06.782442166 +0000 UTC m=+0.136966023 container create df105ad19076c883d511eeb3f9766328f637701e164620d48143e84c5b6c5de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:36:06.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:06 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/40349032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:07.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:06 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/40349032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:07.479 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:36:07 smithi145 podman[65909]: 2024-04-02 13:36:07.002041806 +0000 UTC m=+0.356565622 container init df105ad19076c883d511eeb3f9766328f637701e164620d48143e84c5b6c5de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-i, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-02T13:36:07.479 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:36:07 smithi145 podman[65909]: 2024-04-02 13:36:07.005026474 +0000 UTC m=+0.359550298 container start df105ad19076c883d511eeb3f9766328f637701e164620d48143e84c5b6c5de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-i, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-02T13:36:07.479 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:36:07 smithi145 ceph-mds[65928]: -- 172.21.15.145:0/1571043382 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555aee499a40 con 0x555aee46c800 2024-04-02T13:36:07.479 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:36:07 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-i[65924]: starting mds.i at 2024-04-02T13:36:07.479 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:36:07 smithi145 bash[65909]: df105ad19076c883d511eeb3f9766328f637701e164620d48143e84c5b6c5de3 2024-04-02T13:36:07.479 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:36:07 smithi145 systemd[1]: Started Ceph mds.i for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:36:07.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:07 smithi145 ceph-mon[35646]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:07.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:07.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:07.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:07.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:07.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:07.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:07 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:07.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:07 smithi050 ceph-mon[35524]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:07.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:07.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:07.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:07.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:07.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:07.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:07 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:08.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:07 smithi033 ceph-mon[32390]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:08.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:08.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:08.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:08.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:08.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:08.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:07 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:08.216 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.321+0000 7f4ae839d640 1 -- 172.21.15.33:0/4154387205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae0101c90 msgr2=0x7f4ae0102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.321+0000 7f4ae839d640 1 --2- 172.21.15.33:0/4154387205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae0101c90 0x7f4ae0102110 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009900 tx=0x7f4adc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.321+0000 7f4ae839d640 1 -- 172.21.15.33:0/4154387205 shutdown_connections 2024-04-02T13:36:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.321+0000 7f4ae839d640 1 --2- 172.21.15.33:0/4154387205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ae0102650 0x7f4ae0108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.321+0000 7f4ae839d640 1 --2- 172.21.15.33:0/4154387205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae0101c90 0x7f4ae0102110 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.321+0000 7f4ae839d640 1 --2- 172.21.15.33:0/4154387205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae0100aa0 0x7f4ae0100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.321+0000 7f4ae839d640 1 -- 172.21.15.33:0/4154387205 >> 172.21.15.33:0/4154387205 conn(0x7f4ae00fc030 msgr2=0x7f4ae00fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.321+0000 7f4ae839d640 1 -- 172.21.15.33:0/4154387205 shutdown_connections 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.321+0000 7f4ae839d640 1 -- 172.21.15.33:0/4154387205 wait complete. 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae839d640 1 Processor -- start 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae839d640 1 -- start start 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae839d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ae0100aa0 0x7f4ae0197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae839d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae0101c90 0x7f4ae0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae839d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae0102650 0x7f4ae019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae839d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae010d790 con 0x7f4ae0102650 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae839d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae010d610 con 0x7f4ae0101c90 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae839d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae010d910 con 0x7f4ae0100aa0 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae693a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae0102650 0x7f4ae019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae693a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae0102650 0x7f4ae019f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46550/0 (socket says 172.21.15.33:46550) 2024-04-02T13:36:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae693a640 1 -- 172.21.15.33:0/1099739211 learned_addr learned my addr 172.21.15.33:0/1099739211 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae6139640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ae0100aa0 0x7f4ae0197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae693a640 1 -- 172.21.15.33:0/1099739211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ae0100aa0 msgr2=0x7f4ae0197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae693a640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ae0100aa0 0x7f4ae0197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae693a640 1 -- 172.21.15.33:0/1099739211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae0101c90 msgr2=0x7f4ae0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae693a640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae0101c90 0x7f4ae0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae693a640 1 -- 172.21.15.33:0/1099739211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae019fa20 con 0x7f4ae0102650 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae6139640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ae0100aa0 0x7f4ae0197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ae693a640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae0102650 0x7f4ae019f3b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800eaa0 tx=0x7f4ad800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.322+0000 7f4ad77fe640 1 -- 172.21.15.33:0/1099739211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800cdb0 con 0x7f4ae0102650 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.323+0000 7f4ad77fe640 1 -- 172.21.15.33:0/1099739211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4ad8005020 con 0x7f4ae0102650 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.323+0000 7f4ae839d640 1 -- 172.21.15.33:0/1099739211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae019fd10 con 0x7f4ae0102650 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.323+0000 7f4ad77fe640 1 -- 172.21.15.33:0/1099739211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80116b0 con 0x7f4ae0102650 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.323+0000 7f4ae839d640 1 -- 172.21.15.33:0/1099739211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae01a02b0 con 0x7f4ae0102650 2024-04-02T13:36:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.323+0000 7f4ae839d640 1 -- 172.21.15.33:0/1099739211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae0107dc0 con 0x7f4ae0102650 2024-04-02T13:36:08.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.325+0000 7f4ad77fe640 1 -- 172.21.15.33:0/1099739211 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4ad80051d0 con 0x7f4ae0102650 2024-04-02T13:36:08.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.325+0000 7f4ad77fe640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f4acc0a3c30 0x7f4acc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:08.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.325+0000 7f4ad77fe640 1 -- 172.21.15.33:0/1099739211 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4ad80d01a0 con 0x7f4ae0102650 2024-04-02T13:36:08.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.325+0000 7f4ae6139640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f4acc0a3c30 0x7f4acc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:08.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.326+0000 7f4ae6139640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f4acc0a3c30 0x7f4acc0a60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0101af0 tx=0x7f4ad0005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.330+0000 7f4ad77fe640 1 -- 172.21.15.33:0/1099739211 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4ad8016030 con 0x7f4ae0102650 2024-04-02T13:36:08.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.556+0000 7f4ae839d640 1 -- 172.21.15.33:0/1099739211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ae002d590 con 0x7f4ae0102650 2024-04-02T13:36:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.556+0000 7f4ad77fe640 1 -- 172.21.15.33:0/1099739211 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f4ad809d020 con 0x7f4ae0102650 2024-04-02T13:36:08.766 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:08.766 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-02T13:36:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 10 2024-04-02T13:36:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.558+0000 7f4ae839d640 1 -- 172.21.15.33:0/1099739211 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f4acc0a3c30 msgr2=0x7f4acc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.558+0000 7f4ae839d640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f4acc0a3c30 0x7f4acc0a60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0101af0 tx=0x7f4ad0005d30 comp rx=0 tx=0).stop 2024-04-02T13:36:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.558+0000 7f4ae839d640 1 -- 172.21.15.33:0/1099739211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae0102650 msgr2=0x7f4ae019f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.558+0000 7f4ae839d640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae0102650 0x7f4ae019f3b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800eaa0 tx=0x7f4ad800ef60 comp rx=0 tx=0).stop 2024-04-02T13:36:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.558+0000 7f4ae839d640 1 -- 172.21.15.33:0/1099739211 shutdown_connections 2024-04-02T13:36:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.558+0000 7f4ae839d640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f4acc0a3c30 0x7f4acc0a60f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.558+0000 7f4ae839d640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ae0102650 0x7f4ae019f3b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.558+0000 7f4ae839d640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ae0101c90 0x7f4ae0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.558+0000 7f4ae839d640 1 --2- 172.21.15.33:0/1099739211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ae0100aa0 0x7f4ae0197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.558+0000 7f4ae839d640 1 -- 172.21.15.33:0/1099739211 >> 172.21.15.33:0/1099739211 conn(0x7f4ae00fc030 msgr2=0x7f4ae0104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.558+0000 7f4ae839d640 1 -- 172.21.15.33:0/1099739211 shutdown_connections 2024-04-02T13:36:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:08.558+0000 7f4ae839d640 1 -- 172.21.15.33:0/1099739211 wait complete. 2024-04-02T13:36:08.869 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:08 smithi033 ceph-mon[32390]: Deploying daemon mds.g on smithi033 2024-04-02T13:36:08.869 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:08 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.145:6838/3484016588,v1:172.21.15.145:6839/3484016588] up:boot 2024-04-02T13:36:08.869 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:08 smithi033 ceph-mon[32390]: fsmap 9 up:standby 2024-04-02T13:36:08.869 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:08 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:36:08.869 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:08 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1099739211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:08.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:08 smithi145 ceph-mon[35646]: Deploying daemon mds.g on smithi033 2024-04-02T13:36:08.983 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:08 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.145:6838/3484016588,v1:172.21.15.145:6839/3484016588] up:boot 2024-04-02T13:36:08.983 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:08 smithi145 ceph-mon[35646]: fsmap 9 up:standby 2024-04-02T13:36:08.983 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:08 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:36:08.983 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:08 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1099739211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:08.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:08 smithi050 ceph-mon[35524]: Deploying daemon mds.g on smithi033 2024-04-02T13:36:08.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:08 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.145:6838/3484016588,v1:172.21.15.145:6839/3484016588] up:boot 2024-04-02T13:36:08.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:08 smithi050 ceph-mon[35524]: fsmap 9 up:standby 2024-04-02T13:36:08.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:08 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:36:08.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:08 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1099739211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:09.504 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:36:09 smithi033 systemd[1]: Starting Ceph mds.g for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:36:09.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:09 smithi033 ceph-mon[32390]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:09.829 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:36:09 smithi033 podman[71568]: 2024-04-02 13:36:09.458917462 +0000 UTC m=+0.016968796 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:36:09.829 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:36:09 smithi033 podman[71568]: 2024-04-02 13:36:09.566111823 +0000 UTC m=+0.124163179 container create ed7fe4339c384e2243f6a211b87ef99923fa59606126bad3e4fda72fa90e6391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-g, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:36:09.829 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:36:09 smithi033 podman[71568]: 2024-04-02 13:36:09.827318284 +0000 UTC m=+0.385369601 container init ed7fe4339c384e2243f6a211b87ef99923fa59606126bad3e4fda72fa90e6391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:36:09.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:09 smithi145 ceph-mon[35646]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:09.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:09 smithi050 ceph-mon[35524]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:10.254 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:36:09 smithi033 podman[71568]: 2024-04-02 13:36:09.829949523 +0000 UTC m=+0.388000837 container start ed7fe4339c384e2243f6a211b87ef99923fa59606126bad3e4fda72fa90e6391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-g, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-02T13:36:10.254 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:36:09 smithi033 ceph-mds[71587]: -- 172.21.15.33:0/2741966028 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558858c27a40 con 0x558858bfac00 2024-04-02T13:36:10.254 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:36:09 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-g[71583]: starting mds.g at 2024-04-02T13:36:10.254 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:36:09 smithi033 bash[71568]: ed7fe4339c384e2243f6a211b87ef99923fa59606126bad3e4fda72fa90e6391 2024-04-02T13:36:10.254 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:36:09 smithi033 systemd[1]: Started Ceph mds.g for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:36:10.769 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:10.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.872+0000 7f53f658d640 1 -- 172.21.15.33:0/1882142843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f01025e0 msgr2=0x7f53f0108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:10.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.872+0000 7f53f658d640 1 --2- 172.21.15.33:0/1882142843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f01025e0 0x7f53f0108e70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009ca0 tx=0x7f53e802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:10.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.872+0000 7f53f658d640 1 -- 172.21.15.33:0/1882142843 shutdown_connections 2024-04-02T13:36:10.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.872+0000 7f53f658d640 1 --2- 172.21.15.33:0/1882142843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f01025e0 0x7f53f0108e70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.872+0000 7f53f658d640 1 --2- 172.21.15.33:0/1882142843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53f0101c20 0x7f53f01020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.872+0000 7f53f658d640 1 --2- 172.21.15.33:0/1882142843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53f0100a30 0x7f53f0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.872+0000 7f53f658d640 1 -- 172.21.15.33:0/1882142843 >> 172.21.15.33:0/1882142843 conn(0x7f53f00fbfe0 msgr2=0x7f53f00fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.872+0000 7f53f658d640 1 -- 172.21.15.33:0/1882142843 shutdown_connections 2024-04-02T13:36:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.872+0000 7f53f658d640 1 -- 172.21.15.33:0/1882142843 wait complete. 2024-04-02T13:36:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.873+0000 7f53f658d640 1 Processor -- start 2024-04-02T13:36:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.873+0000 7f53f658d640 1 -- start start 2024-04-02T13:36:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.873+0000 7f53f658d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53f0100a30 0x7f53f006ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.873+0000 7f53f658d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53f0101c20 0x7f53f019cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.873+0000 7f53f658d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f01025e0 0x7f53f019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.873+0000 7f53f658d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f53f010c690 con 0x7f53f01025e0 2024-04-02T13:36:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.873+0000 7f53f658d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f53f010c510 con 0x7f53f0100a30 2024-04-02T13:36:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.873+0000 7f53f658d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f53f010c810 con 0x7f53f0101c20 2024-04-02T13:36:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53f4b2a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f01025e0 0x7f53f019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53f4b2a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f01025e0 0x7f53f019f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46588/0 (socket says 172.21.15.33:46588) 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53f4b2a640 1 -- 172.21.15.33:0/27602870 learned_addr learned my addr 172.21.15.33:0/27602870 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53effff640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53f0100a30 0x7f53f006ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53ef7fe640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53f0101c20 0x7f53f019cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53f4b2a640 1 -- 172.21.15.33:0/27602870 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53f0101c20 msgr2=0x7f53f019cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53f4b2a640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53f0101c20 0x7f53f019cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53f4b2a640 1 -- 172.21.15.33:0/27602870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53f0100a30 msgr2=0x7f53f006ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53f4b2a640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53f0100a30 0x7f53f006ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53f4b2a640 1 -- 172.21.15.33:0/27602870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f019fa50 con 0x7f53f01025e0 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53ef7fe640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53f0101c20 0x7f53f019cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53f4b2a640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f01025e0 0x7f53f019f140 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f53e802c630 tx=0x7f53e8032900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53ed7fa640 1 -- 172.21.15.33:0/27602870 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8039050 con 0x7f53f01025e0 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53f658d640 1 -- 172.21.15.33:0/27602870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f019fc80 con 0x7f53f01025e0 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53ed7fa640 1 -- 172.21.15.33:0/27602870 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f53e803a040 con 0x7f53f01025e0 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53f658d640 1 -- 172.21.15.33:0/27602870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f01a01c0 con 0x7f53f01025e0 2024-04-02T13:36:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.874+0000 7f53ed7fa640 1 -- 172.21.15.33:0/27602870 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8040450 con 0x7f53f01025e0 2024-04-02T13:36:10.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.876+0000 7f53f658d640 1 -- 172.21.15.33:0/27602870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53dc0052c0 con 0x7f53f01025e0 2024-04-02T13:36:10.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.877+0000 7f53ed7fa640 1 -- 172.21.15.33:0/27602870 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f53e8003b70 con 0x7f53f01025e0 2024-04-02T13:36:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.877+0000 7f53ed7fa640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f53d80a3bb0 0x7f53d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.878+0000 7f53effff640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f53d80a3bb0 0x7f53d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.878+0000 7f53ed7fa640 1 -- 172.21.15.33:0/27602870 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f53e80f2810 con 0x7f53f01025e0 2024-04-02T13:36:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.878+0000 7f53effff640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f53d80a3bb0 0x7f53d80a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f53e00058c0 tx=0x7f53e000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:10.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:10.882+0000 7f53ed7fa640 1 -- 172.21.15.33:0/27602870 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f53e804e020 con 0x7f53f01025e0 2024-04-02T13:36:11.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.113+0000 7f53f658d640 1 -- 172.21.15.33:0/27602870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53dc005880 con 0x7f53f01025e0 2024-04-02T13:36:11.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.113+0000 7f53ed7fa640 1 -- 172.21.15.33:0/27602870 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f53e80bf690 con 0x7f53f01025e0 2024-04-02T13:36:11.116 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:11.116 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-02T13:36:11.117 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 11 2024-04-02T13:36:11.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.115+0000 7f53f658d640 1 -- 172.21.15.33:0/27602870 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f53d80a3bb0 msgr2=0x7f53d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:11.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.115+0000 7f53f658d640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f53d80a3bb0 0x7f53d80a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f53e00058c0 tx=0x7f53e000a2d0 comp rx=0 tx=0).stop 2024-04-02T13:36:11.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.115+0000 7f53f658d640 1 -- 172.21.15.33:0/27602870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f01025e0 msgr2=0x7f53f019f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:11.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.115+0000 7f53f658d640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f01025e0 0x7f53f019f140 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f53e802c630 tx=0x7f53e8032900 comp rx=0 tx=0).stop 2024-04-02T13:36:11.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.115+0000 7f53f658d640 1 -- 172.21.15.33:0/27602870 shutdown_connections 2024-04-02T13:36:11.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.115+0000 7f53f658d640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f53d80a3bb0 0x7f53d80a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:11.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.115+0000 7f53f658d640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53f01025e0 0x7f53f019f140 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:11.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.115+0000 7f53f658d640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f53f0101c20 0x7f53f019cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:11.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.115+0000 7f53f658d640 1 --2- 172.21.15.33:0/27602870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53f0100a30 0x7f53f006ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:11.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.115+0000 7f53f658d640 1 -- 172.21.15.33:0/27602870 >> 172.21.15.33:0/27602870 conn(0x7f53f00fbfe0 msgr2=0x7f53f0104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:11.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.116+0000 7f53f658d640 1 -- 172.21.15.33:0/27602870 shutdown_connections 2024-04-02T13:36:11.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:11.116+0000 7f53f658d640 1 -- 172.21.15.33:0/27602870 wait complete. 2024-04-02T13:36:11.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:10 smithi050 ceph-mon[35524]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:11.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:10 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:11.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:10 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:11.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:10 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:11.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:10 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:11.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:10 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:11.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:10 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:11.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:10 smithi050 ceph-mon[35524]: Deploying daemon mds.a on smithi033 2024-04-02T13:36:11.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:10 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.33:6838/3175531919,v1:172.21.15.33:6839/3175531919] up:boot 2024-04-02T13:36:11.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:10 smithi050 ceph-mon[35524]: fsmap 10 up:standby 2024-04-02T13:36:11.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:10 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:36:11.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:10 smithi033 ceph-mon[32390]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:11.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:10 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:11.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:10 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:11.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:10 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:11.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:10 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:11.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:10 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:11.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:10 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:11.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:10 smithi033 ceph-mon[32390]: Deploying daemon mds.a on smithi033 2024-04-02T13:36:11.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:10 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.33:6838/3175531919,v1:172.21.15.33:6839/3175531919] up:boot 2024-04-02T13:36:11.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:10 smithi033 ceph-mon[32390]: fsmap 10 up:standby 2024-04-02T13:36:11.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:10 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:36:11.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:10 smithi145 ceph-mon[35646]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:10 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:10 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:10 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:10 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:10 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:10 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:10 smithi145 ceph-mon[35646]: Deploying daemon mds.a on smithi033 2024-04-02T13:36:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:10 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.33:6838/3175531919,v1:172.21.15.33:6839/3175531919] up:boot 2024-04-02T13:36:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:10 smithi145 ceph-mon[35646]: fsmap 10 up:standby 2024-04-02T13:36:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:10 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:36:12.246 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:11 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/27602870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:12.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:11 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/27602870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:12.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:11 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/27602870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:12.504 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 02 13:36:12 smithi033 podman[71928]: 2024-04-02 13:36:12.490144522 +0000 UTC m=+0.139539377 container create 213ccd759d4c0e7fce24f9767fbb358f990dfbf344546f85dbbaad464135ec10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-a, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-02T13:36:12.996 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 02 13:36:12 smithi033 podman[71928]: 2024-04-02 13:36:12.742983414 +0000 UTC m=+0.392378280 container init 213ccd759d4c0e7fce24f9767fbb358f990dfbf344546f85dbbaad464135ec10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-a, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-02T13:36:12.996 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 02 13:36:12 smithi033 podman[71928]: 2024-04-02 13:36:12.745937672 +0000 UTC m=+0.395332533 container start 213ccd759d4c0e7fce24f9767fbb358f990dfbf344546f85dbbaad464135ec10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-02T13:36:12.996 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 02 13:36:12 smithi033 ceph-mds[71948]: -- 172.21.15.33:0/1057450295 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bf2d21da40 con 0x55bf2d1f1000 2024-04-02T13:36:12.996 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 02 13:36:12 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-a[71944]: starting mds.a at 2024-04-02T13:36:12.996 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 02 13:36:12 smithi033 bash[71928]: 213ccd759d4c0e7fce24f9767fbb358f990dfbf344546f85dbbaad464135ec10 2024-04-02T13:36:12.996 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 02 13:36:12 smithi033 systemd[1]: Started Ceph mds.a for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:36:13.130 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:13.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.234+0000 7f1b60693640 1 -- 172.21.15.33:0/4286965260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b581025d0 msgr2=0x7f1b58108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:13.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.234+0000 7f1b60693640 1 --2- 172.21.15.33:0/4286965260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b581025d0 0x7f1b58108e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009900 tx=0x7f1b4802bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:13.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.234+0000 7f1b60693640 1 -- 172.21.15.33:0/4286965260 shutdown_connections 2024-04-02T13:36:13.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.234+0000 7f1b60693640 1 --2- 172.21.15.33:0/4286965260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b581025d0 0x7f1b58108e60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:13.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.234+0000 7f1b60693640 1 --2- 172.21.15.33:0/4286965260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b58101c10 0x7f1b58102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:13.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.234+0000 7f1b60693640 1 --2- 172.21.15.33:0/4286965260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b58100a20 0x7f1b58100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:13.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.234+0000 7f1b60693640 1 -- 172.21.15.33:0/4286965260 >> 172.21.15.33:0/4286965260 conn(0x7f1b580fbf90 msgr2=0x7f1b580fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:13.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.234+0000 7f1b60693640 1 -- 172.21.15.33:0/4286965260 shutdown_connections 2024-04-02T13:36:13.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.234+0000 7f1b60693640 1 -- 172.21.15.33:0/4286965260 wait complete. 2024-04-02T13:36:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.235+0000 7f1b60693640 1 Processor -- start 2024-04-02T13:36:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.235+0000 7f1b60693640 1 -- start start 2024-04-02T13:36:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.235+0000 7f1b60693640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b58100a20 0x7f1b58196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.235+0000 7f1b60693640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b58101c10 0x7f1b58197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.235+0000 7f1b60693640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b581025d0 0x7f1b5819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.235+0000 7f1b60693640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5810c670 con 0x7f1b58101c10 2024-04-02T13:36:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.235+0000 7f1b60693640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5810c4f0 con 0x7f1b58100a20 2024-04-02T13:36:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.235+0000 7f1b60693640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5810c7f0 con 0x7f1b581025d0 2024-04-02T13:36:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.235+0000 7f1b5dc2e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b58101c10 0x7f1b58197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.235+0000 7f1b5dc2e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b58101c10 0x7f1b58197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46636/0 (socket says 172.21.15.33:46636) 2024-04-02T13:36:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.235+0000 7f1b5dc2e640 1 -- 172.21.15.33:0/2645650764 learned_addr learned my addr 172.21.15.33:0/2645650764 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.235+0000 7f1b5e42f640 1 --2- 172.21.15.33:0/2645650764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b58100a20 0x7f1b58196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.236+0000 7f1b5dc2e640 1 -- 172.21.15.33:0/2645650764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b581025d0 msgr2=0x7f1b5819e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.236+0000 7f1b5dc2e640 1 --2- 172.21.15.33:0/2645650764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b581025d0 0x7f1b5819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.236+0000 7f1b5dc2e640 1 -- 172.21.15.33:0/2645650764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b58100a20 msgr2=0x7f1b58196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.236+0000 7f1b5dc2e640 1 --2- 172.21.15.33:0/2645650764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b58100a20 0x7f1b58196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.236+0000 7f1b5dc2e640 1 -- 172.21.15.33:0/2645650764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5819e900 con 0x7f1b58101c10 2024-04-02T13:36:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.236+0000 7f1b5dc2e640 1 --2- 172.21.15.33:0/2645650764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b58101c10 0x7f1b58197200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00d950 tx=0x7f1b4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.236+0000 7f1b577fe640 1 -- 172.21.15.33:0/2645650764 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c017070 con 0x7f1b58101c10 2024-04-02T13:36:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.236+0000 7f1b577fe640 1 -- 172.21.15.33:0/2645650764 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1b4c009c30 con 0x7f1b58101c10 2024-04-02T13:36:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.236+0000 7f1b577fe640 1 -- 172.21.15.33:0/2645650764 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c00f6b0 con 0x7f1b58101c10 2024-04-02T13:36:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.236+0000 7f1b60693640 1 -- 172.21.15.33:0/2645650764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5819ebf0 con 0x7f1b58101c10 2024-04-02T13:36:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.236+0000 7f1b60693640 1 -- 172.21.15.33:0/2645650764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5819f140 con 0x7f1b58101c10 2024-04-02T13:36:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.237+0000 7f1b60693640 1 -- 172.21.15.33:0/2645650764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b58107d40 con 0x7f1b58101c10 2024-04-02T13:36:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.238+0000 7f1b577fe640 1 -- 172.21.15.33:0/2645650764 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1b4c010a50 con 0x7f1b58101c10 2024-04-02T13:36:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.239+0000 7f1b577fe640 1 --2- 172.21.15.33:0/2645650764 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b400a3bb0 0x7f1b400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.239+0000 7f1b577fe640 1 -- 172.21.15.33:0/2645650764 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1b4c0cfb40 con 0x7f1b58101c10 2024-04-02T13:36:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.239+0000 7f1b5e42f640 1 --2- 172.21.15.33:0/2645650764 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b400a3bb0 0x7f1b400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.239+0000 7f1b5e42f640 1 --2- 172.21.15.33:0/2645650764 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b400a3bb0 0x7f1b400a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1b58197b90 tx=0x7f1b50009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:13.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.243+0000 7f1b577fe640 1 -- 172.21.15.33:0/2645650764 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1b4c009de0 con 0x7f1b58101c10 2024-04-02T13:36:13.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:12 smithi033 ceph-mon[32390]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:13.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:12 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:13.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:12 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:13.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:12 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:13.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:12 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:13.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:12 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:13.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:12 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:13.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.472+0000 7f1b60693640 1 -- 172.21.15.33:0/2645650764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b5802d370 con 0x7f1b58101c10 2024-04-02T13:36:13.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.473+0000 7f1b577fe640 1 -- 172.21.15.33:0/2645650764 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f1b4c09c9c0 con 0x7f1b58101c10 2024-04-02T13:36:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:13.476 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-02T13:36:13.476 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 12 2024-04-02T13:36:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.475+0000 7f1b60693640 1 -- 172.21.15.33:0/2645650764 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b400a3bb0 msgr2=0x7f1b400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.476+0000 7f1b60693640 1 --2- 172.21.15.33:0/2645650764 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b400a3bb0 0x7f1b400a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1b58197b90 tx=0x7f1b50009500 comp rx=0 tx=0).stop 2024-04-02T13:36:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.476+0000 7f1b60693640 1 -- 172.21.15.33:0/2645650764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b58101c10 msgr2=0x7f1b58197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.476+0000 7f1b60693640 1 --2- 172.21.15.33:0/2645650764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b58101c10 0x7f1b58197200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00d950 tx=0x7f1b4c00de10 comp rx=0 tx=0).stop 2024-04-02T13:36:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.476+0000 7f1b60693640 1 -- 172.21.15.33:0/2645650764 shutdown_connections 2024-04-02T13:36:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.476+0000 7f1b60693640 1 --2- 172.21.15.33:0/2645650764 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b400a3bb0 0x7f1b400a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.476+0000 7f1b60693640 1 --2- 172.21.15.33:0/2645650764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b581025d0 0x7f1b5819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.476+0000 7f1b60693640 1 --2- 172.21.15.33:0/2645650764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b58101c10 0x7f1b58197200 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.476+0000 7f1b60693640 1 --2- 172.21.15.33:0/2645650764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b58100a20 0x7f1b58196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.476+0000 7f1b60693640 1 -- 172.21.15.33:0/2645650764 >> 172.21.15.33:0/2645650764 conn(0x7f1b580fbf90 msgr2=0x7f1b58104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.476+0000 7f1b60693640 1 -- 172.21.15.33:0/2645650764 shutdown_connections 2024-04-02T13:36:13.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:13.476+0000 7f1b60693640 1 -- 172.21.15.33:0/2645650764 wait complete. 2024-04-02T13:36:13.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:12 smithi145 ceph-mon[35646]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:13.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:12 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:13.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:12 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:13.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:12 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:13.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:12 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:13.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:12 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:13.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:12 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:13.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:12 smithi050 ceph-mon[35524]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:13.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:12 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:13.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:12 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:13.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:12 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:13.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:12 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:36:13.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:12 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:36:13.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:12 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:14 smithi050 ceph-mon[35524]: Deploying daemon mds.b on smithi050 2024-04-02T13:36:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:14 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.33:6840/2274071500,v1:172.21.15.33:6841/2274071500] up:boot 2024-04-02T13:36:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:14 smithi050 ceph-mon[35524]: fsmap 11 up:standby 2024-04-02T13:36:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:14 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:36:14.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:14 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2645650764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:14.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:14 smithi145 ceph-mon[35646]: Deploying daemon mds.b on smithi050 2024-04-02T13:36:14.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:14 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.33:6840/2274071500,v1:172.21.15.33:6841/2274071500] up:boot 2024-04-02T13:36:14.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:14 smithi145 ceph-mon[35646]: fsmap 11 up:standby 2024-04-02T13:36:14.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:14 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:36:14.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:14 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2645650764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:14.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:14 smithi033 ceph-mon[32390]: Deploying daemon mds.b on smithi050 2024-04-02T13:36:14.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:14 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.33:6840/2274071500,v1:172.21.15.33:6841/2274071500] up:boot 2024-04-02T13:36:14.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:14 smithi033 ceph-mon[32390]: fsmap 11 up:standby 2024-04-02T13:36:14.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:14 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:36:14.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:14 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2645650764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:15.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:15 smithi050 ceph-mon[35524]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:15.245 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:36:15 smithi050 systemd[1]: Starting Ceph mds.b for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:36:15.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:15 smithi145 ceph-mon[35646]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:15.491 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:15.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:15 smithi033 ceph-mon[32390]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:15.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.603+0000 7fbc705af640 1 -- 172.21.15.33:0/2645356338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68107860 msgr2=0x7fbc68109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:15.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.603+0000 7fbc705af640 1 --2- 172.21.15.33:0/2645356338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68107860 0x7fbc68109c50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009ca0 tx=0x7fbc6002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:15.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.603+0000 7fbc705af640 1 -- 172.21.15.33:0/2645356338 shutdown_connections 2024-04-02T13:36:15.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.603+0000 7fbc705af640 1 --2- 172.21.15.33:0/2645356338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68107860 0x7fbc68109c50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:15.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.603+0000 7fbc705af640 1 --2- 172.21.15.33:0/2645356338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc68104f30 0x7fbc68107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.603+0000 7fbc705af640 1 --2- 172.21.15.33:0/2645356338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc68102600 0x7fbc681049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.603+0000 7fbc705af640 1 -- 172.21.15.33:0/2645356338 >> 172.21.15.33:0/2645356338 conn(0x7fbc680fbf90 msgr2=0x7fbc680fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.603+0000 7fbc705af640 1 -- 172.21.15.33:0/2645356338 shutdown_connections 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.603+0000 7fbc705af640 1 -- 172.21.15.33:0/2645356338 wait complete. 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.604+0000 7fbc705af640 1 Processor -- start 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.604+0000 7fbc705af640 1 -- start start 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.604+0000 7fbc705af640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc68102600 0x7fbc68197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.604+0000 7fbc705af640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc68104f30 0x7fbc68198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.604+0000 7fbc705af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68107860 0x7fbc6819f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.604+0000 7fbc705af640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6810d7a0 con 0x7fbc68107860 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.604+0000 7fbc705af640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6810d620 con 0x7fbc68104f30 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.604+0000 7fbc705af640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6810d920 con 0x7fbc68102600 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.604+0000 7fbc6eb4c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68107860 0x7fbc6819f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.604+0000 7fbc6eb4c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68107860 0x7fbc6819f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40054/0 (socket says 172.21.15.33:40054) 2024-04-02T13:36:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.604+0000 7fbc6eb4c640 1 -- 172.21.15.33:0/1157729906 learned_addr learned my addr 172.21.15.33:0/1157729906 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:15.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.604+0000 7fbc6db4a640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc68104f30 0x7fbc68198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:15.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.604+0000 7fbc6e34b640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc68102600 0x7fbc68197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:15.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.605+0000 7fbc6eb4c640 1 -- 172.21.15.33:0/1157729906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc68102600 msgr2=0x7fbc68197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:15.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.605+0000 7fbc6eb4c640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc68102600 0x7fbc68197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:15.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.605+0000 7fbc6eb4c640 1 -- 172.21.15.33:0/1157729906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc68104f30 msgr2=0x7fbc68198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:15.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.605+0000 7fbc6eb4c640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc68104f30 0x7fbc68198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:15.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.605+0000 7fbc6eb4c640 1 -- 172.21.15.33:0/1157729906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6819f9e0 con 0x7fbc68107860 2024-04-02T13:36:15.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.605+0000 7fbc6eb4c640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68107860 0x7fbc6819f370 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc6002c630 tx=0x7fbc60032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:15.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.605+0000 7fbc5f7fe640 1 -- 172.21.15.33:0/1157729906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60039050 con 0x7fbc68107860 2024-04-02T13:36:15.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.605+0000 7fbc5f7fe640 1 -- 172.21.15.33:0/1157729906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc6003a040 con 0x7fbc68107860 2024-04-02T13:36:15.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.605+0000 7fbc5f7fe640 1 -- 172.21.15.33:0/1157729906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60040450 con 0x7fbc68107860 2024-04-02T13:36:15.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.605+0000 7fbc705af640 1 -- 172.21.15.33:0/1157729906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6819fc70 con 0x7fbc68107860 2024-04-02T13:36:15.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.605+0000 7fbc705af640 1 -- 172.21.15.33:0/1157729906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc681a01b0 con 0x7fbc68107860 2024-04-02T13:36:15.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.606+0000 7fbc705af640 1 -- 172.21.15.33:0/1157729906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc680fda80 con 0x7fbc68107860 2024-04-02T13:36:15.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.607+0000 7fbc5f7fe640 1 -- 172.21.15.33:0/1157729906 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbc60003c70 con 0x7fbc68107860 2024-04-02T13:36:15.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.608+0000 7fbc5f7fe640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbc540a3c30 0x7fbc540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:15.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.608+0000 7fbc6e34b640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbc540a3c30 0x7fbc540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:15.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.608+0000 7fbc5f7fe640 1 -- 172.21.15.33:0/1157729906 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbc600f36e0 con 0x7fbc68107860 2024-04-02T13:36:15.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.609+0000 7fbc6e34b640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbc540a3c30 0x7fbc540a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400bbe0 tx=0x7fbc6400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:15.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.612+0000 7fbc5f7fe640 1 -- 172.21.15.33:0/1157729906 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbc600c0680 con 0x7fbc68107860 2024-04-02T13:36:15.661 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:36:15 smithi050 podman[66309]: 2024-04-02 13:36:15.305490768 +0000 UTC m=+0.017922271 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:36:15.661 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:36:15 smithi050 podman[66309]: 2024-04-02 13:36:15.414182537 +0000 UTC m=+0.126614038 container create a6141b466874b9569b0927ffa20a80b8b868e6cbea229503438c084ad1251725 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-04-02T13:36:15.661 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:36:15 smithi050 podman[66309]: 2024-04-02 13:36:15.633471803 +0000 UTC m=+0.345903319 container init a6141b466874b9569b0927ffa20a80b8b868e6cbea229503438c084ad1251725 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-b, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:36:15.661 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:36:15 smithi050 podman[66309]: 2024-04-02 13:36:15.636014916 +0000 UTC m=+0.348446419 container start a6141b466874b9569b0927ffa20a80b8b868e6cbea229503438c084ad1251725 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:36:15.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.842+0000 7fbc705af640 1 -- 172.21.15.33:0/1157729906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc6802d5c0 con 0x7fbc68107860 2024-04-02T13:36:15.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.843+0000 7fbc5f7fe640 1 -- 172.21.15.33:0/1157729906 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7fbc600bff80 con 0x7fbc68107860 2024-04-02T13:36:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:15.846 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-02T13:36:15.846 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 12 2024-04-02T13:36:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.845+0000 7fbc705af640 1 -- 172.21.15.33:0/1157729906 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbc540a3c30 msgr2=0x7fbc540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.845+0000 7fbc705af640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbc540a3c30 0x7fbc540a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400bbe0 tx=0x7fbc6400b390 comp rx=0 tx=0).stop 2024-04-02T13:36:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.845+0000 7fbc705af640 1 -- 172.21.15.33:0/1157729906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68107860 msgr2=0x7fbc6819f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.845+0000 7fbc705af640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68107860 0x7fbc6819f370 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc6002c630 tx=0x7fbc60032980 comp rx=0 tx=0).stop 2024-04-02T13:36:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.845+0000 7fbc705af640 1 -- 172.21.15.33:0/1157729906 shutdown_connections 2024-04-02T13:36:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.845+0000 7fbc705af640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbc540a3c30 0x7fbc540a60f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.845+0000 7fbc705af640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68107860 0x7fbc6819f370 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.845+0000 7fbc705af640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc68104f30 0x7fbc68198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.845+0000 7fbc705af640 1 --2- 172.21.15.33:0/1157729906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc68102600 0x7fbc68197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.845+0000 7fbc705af640 1 -- 172.21.15.33:0/1157729906 >> 172.21.15.33:0/1157729906 conn(0x7fbc680fbf90 msgr2=0x7fbc68102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.845+0000 7fbc705af640 1 -- 172.21.15.33:0/1157729906 shutdown_connections 2024-04-02T13:36:15.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:15.845+0000 7fbc705af640 1 -- 172.21.15.33:0/1157729906 wait complete. 2024-04-02T13:36:15.994 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:36:15 smithi050 ceph-mds[66329]: -- 172.21.15.50:0/3588619 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f520b17a40 con 0x55f520aeac00 2024-04-02T13:36:15.995 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:36:15 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-b[66325]: starting mds.b at 2024-04-02T13:36:15.995 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:36:15 smithi050 bash[66309]: a6141b466874b9569b0927ffa20a80b8b868e6cbea229503438c084ad1251725 2024-04-02T13:36:15.995 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:36:15 smithi050 systemd[1]: Started Ceph mds.b for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:36:17.032 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.032 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.032 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.032 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.032 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.032 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:16 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1157729906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:17.032 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.032 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:36:17.032 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:16 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] up:boot 2024-04-02T13:36:17.032 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:16 smithi050 ceph-mon[35524]: fsmap 12 up:standby 2024-04-02T13:36:17.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:36:17.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:16 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1157729906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:17.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:36:17.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:16 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] up:boot 2024-04-02T13:36:17.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:16 smithi033 ceph-mon[32390]: fsmap 12 up:standby 2024-04-02T13:36:17.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:36:17.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:16 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1157729906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:17.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:17.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:36:17.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:16 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] up:boot 2024-04-02T13:36:17.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:16 smithi145 ceph-mon[35646]: fsmap 12 up:standby 2024-04-02T13:36:17.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:36:17.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:17.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.963+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3091388668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfdc103780 msgr2=0x7fcfdc10a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.963+0000 7fcfe2df6640 1 --2- 172.21.15.33:0/3091388668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfdc103780 0x7fcfdc10a010 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009930 tx=0x7fcfc402bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.963+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3091388668 shutdown_connections 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.963+0000 7fcfe2df6640 1 --2- 172.21.15.33:0/3091388668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfdc103780 0x7fcfdc10a010 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.963+0000 7fcfe2df6640 1 --2- 172.21.15.33:0/3091388668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfdc102dc0 0x7fcfdc103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.963+0000 7fcfe2df6640 1 --2- 172.21.15.33:0/3091388668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfdc0ffdf0 0x7fcfdc100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.963+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3091388668 >> 172.21.15.33:0/3091388668 conn(0x7fcfdc0fb720 msgr2=0x7fcfdc0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.963+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3091388668 shutdown_connections 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.963+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3091388668 wait complete. 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.963+0000 7fcfe2df6640 1 Processor -- start 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.963+0000 7fcfe2df6640 1 -- start start 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfe2df6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfdc0ffdf0 0x7fcfdc197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfe2df6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfdc102dc0 0x7fcfdc198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfe2df6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfdc103780 0x7fcfdc19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:17.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfe2df6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc10d7b0 con 0x7fcfdc102dc0 2024-04-02T13:36:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfe2df6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc10d630 con 0x7fcfdc103780 2024-04-02T13:36:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfe2df6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc10d930 con 0x7fcfdc0ffdf0 2024-04-02T13:36:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfdc102dc0 0x7fcfdc198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfdc102dc0 0x7fcfdc198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40076/0 (socket says 172.21.15.33:40076) 2024-04-02T13:36:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfdbfff640 1 -- 172.21.15.33:0/3907011194 learned_addr learned my addr 172.21.15.33:0/3907011194 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfe0b92640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfdc0ffdf0 0x7fcfdc197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfe1393640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfdc103780 0x7fcfdc19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfdbfff640 1 -- 172.21.15.33:0/3907011194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfdc0ffdf0 msgr2=0x7fcfdc197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfdbfff640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfdc0ffdf0 0x7fcfdc197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfdbfff640 1 -- 172.21.15.33:0/3907011194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfdc103780 msgr2=0x7fcfdc19f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfdbfff640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfdc103780 0x7fcfdc19f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfdbfff640 1 -- 172.21.15.33:0/3907011194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc19fa10 con 0x7fcfdc102dc0 2024-04-02T13:36:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfe1393640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfdc103780 0x7fcfdc19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfdbfff640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfdc102dc0 0x7fcfdc198330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00d770 tx=0x7fcfcc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfd9ffb640 1 -- 172.21.15.33:0/3907011194 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc017070 con 0x7fcfdc102dc0 2024-04-02T13:36:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfd9ffb640 1 -- 172.21.15.33:0/3907011194 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcfcc0054f0 con 0x7fcfdc102dc0 2024-04-02T13:36:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3907011194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc19fd00 con 0x7fcfdc102dc0 2024-04-02T13:36:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.964+0000 7fcfd9ffb640 1 -- 172.21.15.33:0/3907011194 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc004d60 con 0x7fcfdc102dc0 2024-04-02T13:36:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.965+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3907011194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc1a01a0 con 0x7fcfdc102dc0 2024-04-02T13:36:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.965+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3907011194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfdc108ef0 con 0x7fcfdc102dc0 2024-04-02T13:36:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.966+0000 7fcfd9ffb640 1 -- 172.21.15.33:0/3907011194 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcfcc004f00 con 0x7fcfdc102dc0 2024-04-02T13:36:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.967+0000 7fcfd9ffb640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fcfc80a3bb0 0x7fcfc80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.967+0000 7fcfe0b92640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fcfc80a3bb0 0x7fcfc80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.967+0000 7fcfd9ffb640 1 -- 172.21.15.33:0/3907011194 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcfcc0cfa20 con 0x7fcfdc102dc0 2024-04-02T13:36:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.967+0000 7fcfe0b92640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fcfc80a3bb0 0x7fcfc80a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc069fb0 tx=0x7fcfd400a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:17.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:17.972+0000 7fcfd9ffb640 1 -- 172.21.15.33:0/3907011194 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcfcc09c880 con 0x7fcfdc102dc0 2024-04-02T13:36:18.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.202+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3907011194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfdc02d370 con 0x7fcfdc102dc0 2024-04-02T13:36:18.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.202+0000 7fcfd9ffb640 1 -- 172.21.15.33:0/3907011194 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7fcfcc09c180 con 0x7fcfdc102dc0 2024-04-02T13:36:18.205 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:18.205 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-02T13:36:18.206 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 13 2024-04-02T13:36:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.205+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3907011194 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fcfc80a3bb0 msgr2=0x7fcfc80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.205+0000 7fcfe2df6640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fcfc80a3bb0 0x7fcfc80a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc069fb0 tx=0x7fcfd400a590 comp rx=0 tx=0).stop 2024-04-02T13:36:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.205+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3907011194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfdc102dc0 msgr2=0x7fcfdc198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.205+0000 7fcfe2df6640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfdc102dc0 0x7fcfdc198330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00d770 tx=0x7fcfcc00dc30 comp rx=0 tx=0).stop 2024-04-02T13:36:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.205+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3907011194 shutdown_connections 2024-04-02T13:36:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.205+0000 7fcfe2df6640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fcfc80a3bb0 0x7fcfc80a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.205+0000 7fcfe2df6640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfdc103780 0x7fcfdc19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.205+0000 7fcfe2df6640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfdc102dc0 0x7fcfdc198330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.205+0000 7fcfe2df6640 1 --2- 172.21.15.33:0/3907011194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcfdc0ffdf0 0x7fcfdc197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.205+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3907011194 >> 172.21.15.33:0/3907011194 conn(0x7fcfdc0fb720 msgr2=0x7fcfdc1060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.205+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3907011194 shutdown_connections 2024-04-02T13:36:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.205+0000 7fcfe2df6640 1 -- 172.21.15.33:0/3907011194 wait complete. 2024-04-02T13:36:18.218 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:17 smithi033 ceph-mon[32390]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:18.218 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.218 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.218 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.218 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.218 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.218 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.220 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:36:18.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:17 smithi145 ceph-mon[35646]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:18.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:17 smithi050 ceph-mon[35524]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:18.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.321+0000 7fd653a95640 1 -- 172.21.15.33:0/1079414318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd64c100a20 msgr2=0x7fd64c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.321+0000 7fd653a95640 1 --2- 172.21.15.33:0/1079414318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd64c100a20 0x7fd64c100e40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd648009900 tx=0x7fd64802bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.321+0000 7fd653a95640 1 -- 172.21.15.33:0/1079414318 shutdown_connections 2024-04-02T13:36:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.321+0000 7fd653a95640 1 --2- 172.21.15.33:0/1079414318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd64c1025d0 0x7fd64c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.321+0000 7fd653a95640 1 --2- 172.21.15.33:0/1079414318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd64c101c10 0x7fd64c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.321+0000 7fd653a95640 1 --2- 172.21.15.33:0/1079414318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd64c100a20 0x7fd64c100e40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.321+0000 7fd653a95640 1 -- 172.21.15.33:0/1079414318 >> 172.21.15.33:0/1079414318 conn(0x7fd64c0fbfb0 msgr2=0x7fd64c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.321+0000 7fd653a95640 1 -- 172.21.15.33:0/1079414318 shutdown_connections 2024-04-02T13:36:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.321+0000 7fd653a95640 1 -- 172.21.15.33:0/1079414318 wait complete. 2024-04-02T13:36:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd653a95640 1 Processor -- start 2024-04-02T13:36:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd653a95640 1 -- start start 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd653a95640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd64c100a20 0x7fd64c1049b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd653a95640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd64c101c10 0x7fd64c102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd653a95640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd64c1025d0 0x7fd64c103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd653a95640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c10c680 con 0x7fd64c101c10 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd653a95640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c10c500 con 0x7fd64c100a20 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd653a95640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c10c800 con 0x7fd64c1025d0 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd651030640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd64c101c10 0x7fd64c102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd651030640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd64c101c10 0x7fd64c102ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40082/0 (socket says 172.21.15.33:40082) 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd651030640 1 -- 172.21.15.33:0/3723466404 learned_addr learned my addr 172.21.15.33:0/3723466404 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd651831640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd64c100a20 0x7fd64c1049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd651030640 1 -- 172.21.15.33:0/3723466404 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd64c1025d0 msgr2=0x7fd64c103530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd651030640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd64c1025d0 0x7fd64c103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd651030640 1 -- 172.21.15.33:0/3723466404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd64c100a20 msgr2=0x7fd64c1049b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd651030640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd64c100a20 0x7fd64c1049b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd651030640 1 -- 172.21.15.33:0/3723466404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64c103dc0 con 0x7fd64c101c10 2024-04-02T13:36:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd651831640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd64c100a20 0x7fd64c1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd651030640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd64c101c10 0x7fd64c102ff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00d770 tx=0x7fd63c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd642ffd640 1 -- 172.21.15.33:0/3723466404 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c017070 con 0x7fd64c101c10 2024-04-02T13:36:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd642ffd640 1 -- 172.21.15.33:0/3723466404 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd63c0054f0 con 0x7fd64c101c10 2024-04-02T13:36:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.322+0000 7fd653a95640 1 -- 172.21.15.33:0/3723466404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64c1a1fc0 con 0x7fd64c101c10 2024-04-02T13:36:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.323+0000 7fd642ffd640 1 -- 172.21.15.33:0/3723466404 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c004d60 con 0x7fd64c101c10 2024-04-02T13:36:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.323+0000 7fd653a95640 1 -- 172.21.15.33:0/3723466404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd64c1a2420 con 0x7fd64c101c10 2024-04-02T13:36:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.324+0000 7fd653a95640 1 -- 172.21.15.33:0/3723466404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd64c1a26e0 con 0x7fd64c101c10 2024-04-02T13:36:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.324+0000 7fd642ffd640 1 -- 172.21.15.33:0/3723466404 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd63c004f00 con 0x7fd64c101c10 2024-04-02T13:36:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.325+0000 7fd642ffd640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd6380a3c30 0x7fd6380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.325+0000 7fd651831640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd6380a3c30 0x7fd6380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.325+0000 7fd642ffd640 1 -- 172.21.15.33:0/3723466404 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd63c0cfa00 con 0x7fd64c101c10 2024-04-02T13:36:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.325+0000 7fd651831640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd6380a3c30 0x7fd6380a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd648006010 tx=0x7fd648000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.330+0000 7fd642ffd640 1 -- 172.21.15.33:0/3723466404 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd63c09c880 con 0x7fd64c101c10 2024-04-02T13:36:18.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.518+0000 7fd653a95640 1 -- 172.21.15.33:0/3723466404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd64c1a2a70 con 0x7fd64c101c10 2024-04-02T13:36:18.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.519+0000 7fd642ffd640 1 -- 172.21.15.33:0/3723466404 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18385 (secure 0 0 0) 0x7fd63c09c180 con 0x7fd64c101c10 2024-04-02T13:36:18.522 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:18.522 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:35:31.648681+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:36:18.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.521+0000 7fd653a95640 1 -- 172.21.15.33:0/3723466404 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd6380a3c30 msgr2=0x7fd6380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.521+0000 7fd653a95640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd6380a3c30 0x7fd6380a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd648006010 tx=0x7fd648000bc0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.521+0000 7fd653a95640 1 -- 172.21.15.33:0/3723466404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd64c101c10 msgr2=0x7fd64c102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.521+0000 7fd653a95640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd64c101c10 0x7fd64c102ff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00d770 tx=0x7fd63c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:36:18.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.522+0000 7fd653a95640 1 -- 172.21.15.33:0/3723466404 shutdown_connections 2024-04-02T13:36:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.522+0000 7fd653a95640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd6380a3c30 0x7fd6380a60f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.522+0000 7fd653a95640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd64c1025d0 0x7fd64c103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.522+0000 7fd653a95640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd64c101c10 0x7fd64c102ff0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.522+0000 7fd653a95640 1 --2- 172.21.15.33:0/3723466404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd64c100a20 0x7fd64c1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.522+0000 7fd653a95640 1 -- 172.21.15.33:0/3723466404 >> 172.21.15.33:0/3723466404 conn(0x7fd64c0fbfb0 msgr2=0x7fd64c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.522+0000 7fd653a95640 1 -- 172.21.15.33:0/3723466404 shutdown_connections 2024-04-02T13:36:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.522+0000 7fd653a95640 1 -- 172.21.15.33:0/3723466404 wait complete. 2024-04-02T13:36:18.536 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T13:36:18.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:36:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.641+0000 7fa70c886640 1 -- 172.21.15.33:0/2151348494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7081003f0 msgr2=0x7fa708100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.641+0000 7fa70c886640 1 --2- 172.21.15.33:0/2151348494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7081003f0 0x7fa708100810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009900 tx=0x7fa6fc02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.641+0000 7fa70c886640 1 -- 172.21.15.33:0/2151348494 shutdown_connections 2024-04-02T13:36:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.641+0000 7fa70c886640 1 --2- 172.21.15.33:0/2151348494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7081016f0 0x7fa708101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.641+0000 7fa70c886640 1 --2- 172.21.15.33:0/2151348494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa708100d50 0x7fa7081011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.641+0000 7fa70c886640 1 --2- 172.21.15.33:0/2151348494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7081003f0 0x7fa708100810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.641+0000 7fa70c886640 1 -- 172.21.15.33:0/2151348494 >> 172.21.15.33:0/2151348494 conn(0x7fa7080fbf90 msgr2=0x7fa7080fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.641+0000 7fa70c886640 1 -- 172.21.15.33:0/2151348494 shutdown_connections 2024-04-02T13:36:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.641+0000 7fa70c886640 1 -- 172.21.15.33:0/2151348494 wait complete. 2024-04-02T13:36:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa70c886640 1 Processor -- start 2024-04-02T13:36:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa70c886640 1 -- start start 2024-04-02T13:36:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa70c886640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7081003f0 0x7fa70819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa70c886640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708100d50 0x7fa70819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa70c886640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7081016f0 0x7fa7081a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa70c886640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa70810f960 con 0x7fa708100d50 2024-04-02T13:36:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa70c886640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa70810f7e0 con 0x7fa7081016f0 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa70c886640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa70810fae0 con 0x7fa7081003f0 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa705d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708100d50 0x7fa70819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa705d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708100d50 0x7fa70819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40092/0 (socket says 172.21.15.33:40092) 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa705d9b640 1 -- 172.21.15.33:0/1141469890 learned_addr learned my addr 172.21.15.33:0/1141469890 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa706d9d640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7081016f0 0x7fa7081a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa70659c640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7081003f0 0x7fa70819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa705d9b640 1 -- 172.21.15.33:0/1141469890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7081003f0 msgr2=0x7fa70819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa705d9b640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7081003f0 0x7fa70819a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa705d9b640 1 -- 172.21.15.33:0/1141469890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7081016f0 msgr2=0x7fa7081a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa705d9b640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7081016f0 0x7fa7081a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.642+0000 7fa705d9b640 1 -- 172.21.15.33:0/1141469890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7081a1c80 con 0x7fa708100d50 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.643+0000 7fa705d9b640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708100d50 0x7fa70819a5a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800e9f0 tx=0x7fa6f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.643+0000 7fa6f77fe640 1 -- 172.21.15.33:0/1141469890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f800cd80 con 0x7fa708100d50 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.643+0000 7fa6f77fe640 1 -- 172.21.15.33:0/1141469890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa6f80054f0 con 0x7fa708100d50 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.643+0000 7fa70c886640 1 -- 172.21.15.33:0/1141469890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7081a1f70 con 0x7fa708100d50 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.643+0000 7fa6f77fe640 1 -- 172.21.15.33:0/1141469890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8010690 con 0x7fa708100d50 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.643+0000 7fa70c886640 1 -- 172.21.15.33:0/1141469890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7081a23e0 con 0x7fa708100d50 2024-04-02T13:36:18.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.644+0000 7fa70c886640 1 -- 172.21.15.33:0/1141469890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa70810b370 con 0x7fa708100d50 2024-04-02T13:36:18.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.645+0000 7fa6f77fe640 1 -- 172.21.15.33:0/1141469890 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa6f8004f20 con 0x7fa708100d50 2024-04-02T13:36:18.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.645+0000 7fa6f77fe640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa6f00a3bb0 0x7fa6f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.646+0000 7fa6f77fe640 1 -- 172.21.15.33:0/1141469890 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa6f80d0340 con 0x7fa708100d50 2024-04-02T13:36:18.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.646+0000 7fa70659c640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa6f00a3bb0 0x7fa6f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.646+0000 7fa70659c640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa6f00a3bb0 0x7fa6f00a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc006010 tx=0x7fa6fc02c6d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.650+0000 7fa6f77fe640 1 -- 172.21.15.33:0/1141469890 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa6f809d1c0 con 0x7fa708100d50 2024-04-02T13:36:18.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:18 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3907011194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:18.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:18.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:36:18.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:18.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:36:18.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:18 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3723466404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.838+0000 7fa70c886640 1 -- 172.21.15.33:0/1141469890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa70802d520 con 0x7fa708100d50 2024-04-02T13:36:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.838+0000 7fa6f77fe640 1 -- 172.21.15.33:0/1141469890 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fa6f809cac0 con 0x7fa708100d50 2024-04-02T13:36:18.841 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:36:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.840+0000 7fa70c886640 1 -- 172.21.15.33:0/1141469890 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa6f00a3bb0 msgr2=0x7fa6f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.840+0000 7fa70c886640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa6f00a3bb0 0x7fa6f00a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc006010 tx=0x7fa6fc02c6d0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.840+0000 7fa70c886640 1 -- 172.21.15.33:0/1141469890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708100d50 msgr2=0x7fa70819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.840+0000 7fa70c886640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708100d50 0x7fa70819a5a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800e9f0 tx=0x7fa6f800edf0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.840+0000 7fa70c886640 1 -- 172.21.15.33:0/1141469890 shutdown_connections 2024-04-02T13:36:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.840+0000 7fa70c886640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa6f00a3bb0 0x7fa6f00a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.840+0000 7fa70c886640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7081016f0 0x7fa7081a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.840+0000 7fa70c886640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa708100d50 0x7fa70819a5a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.840+0000 7fa70c886640 1 --2- 172.21.15.33:0/1141469890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7081003f0 0x7fa70819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.840+0000 7fa70c886640 1 -- 172.21.15.33:0/1141469890 >> 172.21.15.33:0/1141469890 conn(0x7fa7080fbf90 msgr2=0x7fa7080fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:18.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.840+0000 7fa70c886640 1 -- 172.21.15.33:0/1141469890 shutdown_connections 2024-04-02T13:36:18.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.840+0000 7fa70c886640 1 -- 172.21.15.33:0/1141469890 wait complete. 2024-04-02T13:36:18.854 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-02T13:36:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.969+0000 7f676ade4640 1 -- 172.21.15.33:0/2113065285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6764107820 msgr2=0x7f6764109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.969+0000 7f676ade4640 1 --2- 172.21.15.33:0/2113065285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6764107820 0x7f6764109c70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f675c009ca0 tx=0x7f675c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.969+0000 7f676ade4640 1 -- 172.21.15.33:0/2113065285 shutdown_connections 2024-04-02T13:36:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.969+0000 7f676ade4640 1 --2- 172.21.15.33:0/2113065285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6764107820 0x7f6764109c70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.969+0000 7f676ade4640 1 --2- 172.21.15.33:0/2113065285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6764104ef0 0x7f67641072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.969+0000 7f676ade4640 1 --2- 172.21.15.33:0/2113065285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67641025c0 0x7f67641049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.969+0000 7f676ade4640 1 -- 172.21.15.33:0/2113065285 >> 172.21.15.33:0/2113065285 conn(0x7f67640fbfb0 msgr2=0x7f67640fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.969+0000 7f676ade4640 1 -- 172.21.15.33:0/2113065285 shutdown_connections 2024-04-02T13:36:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.969+0000 7f676ade4640 1 -- 172.21.15.33:0/2113065285 wait complete. 2024-04-02T13:36:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.969+0000 7f676ade4640 1 Processor -- start 2024-04-02T13:36:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.969+0000 7f676ade4640 1 -- start start 2024-04-02T13:36:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f676ade4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67641025c0 0x7f6764101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f676ade4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6764104ef0 0x7f6764100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f676ade4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6764107820 0x7f6764100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f676ade4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f676410d7c0 con 0x7f6764104ef0 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f676ade4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f676410d640 con 0x7f6764107820 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f676ade4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f676410d940 con 0x7f67641025c0 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f6763fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6764104ef0 0x7f6764100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f6763fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6764104ef0 0x7f6764100570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40114/0 (socket says 172.21.15.33:40114) 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f6763fff640 1 -- 172.21.15.33:0/2008995633 learned_addr learned my addr 172.21.15.33:0/2008995633 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f6769381640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6764107820 0x7f6764100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f6768b80640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67641025c0 0x7f6764101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f6763fff640 1 -- 172.21.15.33:0/2008995633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67641025c0 msgr2=0x7f6764101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f6763fff640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67641025c0 0x7f6764101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f6763fff640 1 -- 172.21.15.33:0/2008995633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6764107820 msgr2=0x7f6764100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f6763fff640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6764107820 0x7f6764100ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f6763fff640 1 -- 172.21.15.33:0/2008995633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6764101340 con 0x7f6764104ef0 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f6769381640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6764107820 0x7f6764100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:18.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f6763fff640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6764104ef0 0x7f6764100570 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f675000e9f0 tx=0x7f675000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.970+0000 7f6761ffb640 1 -- 172.21.15.33:0/2008995633 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675000ccc0 con 0x7f6764104ef0 2024-04-02T13:36:18.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.971+0000 7f6761ffb640 1 -- 172.21.15.33:0/2008995633 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f67500054f0 con 0x7f6764104ef0 2024-04-02T13:36:18.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.971+0000 7f676ade4640 1 -- 172.21.15.33:0/2008995633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67641a1f50 con 0x7f6764104ef0 2024-04-02T13:36:18.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.971+0000 7f6761ffb640 1 -- 172.21.15.33:0/2008995633 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750010690 con 0x7f6764104ef0 2024-04-02T13:36:18.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.971+0000 7f676ade4640 1 -- 172.21.15.33:0/2008995633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67641a23b0 con 0x7f6764104ef0 2024-04-02T13:36:18.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.972+0000 7f676ade4640 1 -- 172.21.15.33:0/2008995633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67640fdaa0 con 0x7f6764104ef0 2024-04-02T13:36:18.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.973+0000 7f6761ffb640 1 -- 172.21.15.33:0/2008995633 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6750004e80 con 0x7f6764104ef0 2024-04-02T13:36:18.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.974+0000 7f6761ffb640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f67540a3c30 0x7f67540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.974+0000 7f6761ffb640 1 -- 172.21.15.33:0/2008995633 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f67500d0260 con 0x7f6764104ef0 2024-04-02T13:36:18.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.974+0000 7f6768b80640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f67540a3c30 0x7f67540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.975+0000 7f6768b80640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f67540a3c30 0x7f67540a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6758005d00 tx=0x7f675800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:18.979+0000 7f6761ffb640 1 -- 172.21.15.33:0/2008995633 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f675009d040 con 0x7f6764104ef0 2024-04-02T13:36:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.173+0000 7f676ade4640 1 -- 172.21.15.33:0/2008995633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f67640627a0 con 0x7f6764104ef0 2024-04-02T13:36:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.173+0000 7f6761ffb640 1 -- 172.21.15.33:0/2008995633 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f675009c940 con 0x7f6764104ef0 2024-04-02T13:36:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:19.176 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2024-04-02T13:36:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:36:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2024-04-02T13:36:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2024-04-02T13:36:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:36:19.177 INFO:teuthology.orchestra.run.smithi033.stdout:] 2024-04-02T13:36:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.175+0000 7f676ade4640 1 -- 172.21.15.33:0/2008995633 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f67540a3c30 msgr2=0x7f67540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.175+0000 7f676ade4640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f67540a3c30 0x7f67540a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6758005d00 tx=0x7f675800b410 comp rx=0 tx=0).stop 2024-04-02T13:36:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.175+0000 7f676ade4640 1 -- 172.21.15.33:0/2008995633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6764104ef0 msgr2=0x7f6764100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.175+0000 7f676ade4640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6764104ef0 0x7f6764100570 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f675000e9f0 tx=0x7f675000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.175+0000 7f676ade4640 1 -- 172.21.15.33:0/2008995633 shutdown_connections 2024-04-02T13:36:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.175+0000 7f676ade4640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f67540a3c30 0x7f67540a60f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.175+0000 7f676ade4640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6764107820 0x7f6764100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.175+0000 7f676ade4640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6764104ef0 0x7f6764100570 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.175+0000 7f676ade4640 1 --2- 172.21.15.33:0/2008995633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67641025c0 0x7f6764101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.175+0000 7f676ade4640 1 -- 172.21.15.33:0/2008995633 >> 172.21.15.33:0/2008995633 conn(0x7f67640fbfb0 msgr2=0x7f6764102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.175+0000 7f676ade4640 1 -- 172.21.15.33:0/2008995633 shutdown_connections 2024-04-02T13:36:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.175+0000 7f676ade4640 1 -- 172.21.15.33:0/2008995633 wait complete. 2024-04-02T13:36:19.189 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-02T13:36:19.189 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-02T13:36:19.192 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:18 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3907011194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:19.192 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:19.192 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:36:19.192 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:19.193 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:36:19.193 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:18 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3723466404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:19.193 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:18 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3907011194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:19.193 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:19.193 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:36:19.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:19.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:36:19.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:18 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3723466404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.298+0000 7f8c3b663640 1 -- 172.21.15.33:0/2616977579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c34101bf0 msgr2=0x7f8c34102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.298+0000 7f8c3b663640 1 --2- 172.21.15.33:0/2616977579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c34101bf0 0x7f8c34102070 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009900 tx=0x7f8c3002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.298+0000 7f8c3b663640 1 -- 172.21.15.33:0/2616977579 shutdown_connections 2024-04-02T13:36:19.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.298+0000 7f8c3b663640 1 --2- 172.21.15.33:0/2616977579 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c341025b0 0x7f8c34108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.298+0000 7f8c3b663640 1 --2- 172.21.15.33:0/2616977579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c34101bf0 0x7f8c34102070 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.298+0000 7f8c3b663640 1 --2- 172.21.15.33:0/2616977579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c34100a00 0x7f8c34100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.298+0000 7f8c3b663640 1 -- 172.21.15.33:0/2616977579 >> 172.21.15.33:0/2616977579 conn(0x7f8c340fbf90 msgr2=0x7f8c340fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.298+0000 7f8c3b663640 1 -- 172.21.15.33:0/2616977579 shutdown_connections 2024-04-02T13:36:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.298+0000 7f8c3b663640 1 -- 172.21.15.33:0/2616977579 wait complete. 2024-04-02T13:36:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c3b663640 1 Processor -- start 2024-04-02T13:36:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c3b663640 1 -- start start 2024-04-02T13:36:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c3b663640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c34100a00 0x7f8c340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c3b663640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c34101bf0 0x7f8c3419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c3b663640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c341025b0 0x7f8c3419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c3b663640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3410c660 con 0x7f8c34100a00 2024-04-02T13:36:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c3b663640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3410c4e0 con 0x7f8c34101bf0 2024-04-02T13:36:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c3b663640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3410c7e0 con 0x7f8c341025b0 2024-04-02T13:36:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c393ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c34100a00 0x7f8c340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c393ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c34100a00 0x7f8c340697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40136/0 (socket says 172.21.15.33:40136) 2024-04-02T13:36:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c393ff640 1 -- 172.21.15.33:0/528844437 learned_addr learned my addr 172.21.15.33:0/528844437 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c38bfe640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c34101bf0 0x7f8c3419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c39c00640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c341025b0 0x7f8c3419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c393ff640 1 -- 172.21.15.33:0/528844437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c341025b0 msgr2=0x7f8c3419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c393ff640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c341025b0 0x7f8c3419f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c393ff640 1 -- 172.21.15.33:0/528844437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c34101bf0 msgr2=0x7f8c3419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c393ff640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c34101bf0 0x7f8c3419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c393ff640 1 -- 172.21.15.33:0/528844437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3419fb00 con 0x7f8c34100a00 2024-04-02T13:36:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.299+0000 7f8c393ff640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c34100a00 0x7f8c340697d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400d950 tx=0x7f8c2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.300+0000 7f8c2a7fc640 1 -- 172.21.15.33:0/528844437 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24017070 con 0x7f8c34100a00 2024-04-02T13:36:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.300+0000 7f8c2a7fc640 1 -- 172.21.15.33:0/528844437 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8c24009c30 con 0x7f8c34100a00 2024-04-02T13:36:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.300+0000 7f8c3b663640 1 -- 172.21.15.33:0/528844437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3419fd90 con 0x7f8c34100a00 2024-04-02T13:36:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.300+0000 7f8c2a7fc640 1 -- 172.21.15.33:0/528844437 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2400f620 con 0x7f8c34100a00 2024-04-02T13:36:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.300+0000 7f8c3b663640 1 -- 172.21.15.33:0/528844437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c341a01a0 con 0x7f8c34100a00 2024-04-02T13:36:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.301+0000 7f8c3b663640 1 -- 172.21.15.33:0/528844437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c34107e40 con 0x7f8c34100a00 2024-04-02T13:36:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.302+0000 7f8c2a7fc640 1 -- 172.21.15.33:0/528844437 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8c2401a070 con 0x7f8c34100a00 2024-04-02T13:36:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.303+0000 7f8c2a7fc640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8c200a3bb0 0x7f8c200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.303+0000 7f8c38bfe640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8c200a3bb0 0x7f8c200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.303+0000 7f8c2a7fc640 1 -- 172.21.15.33:0/528844437 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8c240cf8a0 con 0x7f8c34100a00 2024-04-02T13:36:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.303+0000 7f8c38bfe640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8c200a3bb0 0x7f8c200a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8c30005ab0 tx=0x7f8c30005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.307+0000 7f8c2a7fc640 1 -- 172.21.15.33:0/528844437 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8c24009de0 con 0x7f8c34100a00 2024-04-02T13:36:19.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.496+0000 7f8c3b663640 1 -- 172.21.15.33:0/528844437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8c3402d370 con 0x7f8c34100a00 2024-04-02T13:36:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.805+0000 7f8c2a7fc640 1 -- 172.21.15.33:0/528844437 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f8c2409c0d0 con 0x7f8c34100a00 2024-04-02T13:36:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2024-04-02T13:36:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.808+0000 7f8c3b663640 1 -- 172.21.15.33:0/528844437 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8c200a3bb0 msgr2=0x7f8c200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.808+0000 7f8c3b663640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8c200a3bb0 0x7f8c200a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8c30005ab0 tx=0x7f8c30005fd0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.808+0000 7f8c3b663640 1 -- 172.21.15.33:0/528844437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c34100a00 msgr2=0x7f8c340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.808+0000 7f8c3b663640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c34100a00 0x7f8c340697d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400d950 tx=0x7f8c2400de10 comp rx=0 tx=0).stop 2024-04-02T13:36:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.808+0000 7f8c3b663640 1 -- 172.21.15.33:0/528844437 shutdown_connections 2024-04-02T13:36:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.808+0000 7f8c3b663640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8c200a3bb0 0x7f8c200a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.808+0000 7f8c3b663640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c341025b0 0x7f8c3419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.808+0000 7f8c3b663640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c34101bf0 0x7f8c3419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.808+0000 7f8c3b663640 1 --2- 172.21.15.33:0/528844437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c34100a00 0x7f8c340697d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.808+0000 7f8c3b663640 1 -- 172.21.15.33:0/528844437 >> 172.21.15.33:0/528844437 conn(0x7f8c340fbf90 msgr2=0x7f8c34105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.808+0000 7f8c3b663640 1 -- 172.21.15.33:0/528844437 shutdown_connections 2024-04-02T13:36:19.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.808+0000 7f8c3b663640 1 -- 172.21.15.33:0/528844437 wait complete. 2024-04-02T13:36:19.827 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.942+0000 7f74a17cd640 1 -- 172.21.15.33:0/2146342949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f749c1025c0 msgr2=0x7f749c1049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.942+0000 7f74a17cd640 1 --2- 172.21.15.33:0/2146342949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f749c1025c0 0x7f749c1049b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7490009900 tx=0x7f749002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.942+0000 7f74a17cd640 1 -- 172.21.15.33:0/2146342949 shutdown_connections 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.942+0000 7f74a17cd640 1 --2- 172.21.15.33:0/2146342949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f749c107820 0x7f749c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.942+0000 7f74a17cd640 1 --2- 172.21.15.33:0/2146342949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f749c104ef0 0x7f749c1072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.942+0000 7f74a17cd640 1 --2- 172.21.15.33:0/2146342949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f749c1025c0 0x7f749c1049b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.942+0000 7f74a17cd640 1 -- 172.21.15.33:0/2146342949 >> 172.21.15.33:0/2146342949 conn(0x7f749c0fbfb0 msgr2=0x7f749c0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.942+0000 7f74a17cd640 1 -- 172.21.15.33:0/2146342949 shutdown_connections 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.942+0000 7f74a17cd640 1 -- 172.21.15.33:0/2146342949 wait complete. 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.942+0000 7f74a17cd640 1 Processor -- start 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f74a17cd640 1 -- start start 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f74a17cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f749c1025c0 0x7f749c101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f74a17cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f749c104ef0 0x7f749c100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f74a17cd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f749c107820 0x7f749c100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f74a17cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f749c10d7a0 con 0x7f749c104ef0 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f74a17cd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f749c10d620 con 0x7f749c107820 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f74a17cd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f749c10d920 con 0x7f749c1025c0 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f749a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f749c104ef0 0x7f749c100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f749a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f749c104ef0 0x7f749c100570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40142/0 (socket says 172.21.15.33:40142) 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f749a7fc640 1 -- 172.21.15.33:0/2691584720 learned_addr learned my addr 172.21.15.33:0/2691584720 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f749affd640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f749c1025c0 0x7f749c101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f749b7fe640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f749c107820 0x7f749c100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f749a7fc640 1 -- 172.21.15.33:0/2691584720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f749c1025c0 msgr2=0x7f749c101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f749a7fc640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f749c1025c0 0x7f749c101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f749a7fc640 1 -- 172.21.15.33:0/2691584720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f749c107820 msgr2=0x7f749c100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f749a7fc640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f749c107820 0x7f749c100ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f749a7fc640 1 -- 172.21.15.33:0/2691584720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749c101340 con 0x7f749c104ef0 2024-04-02T13:36:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f749affd640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f749c1025c0 0x7f749c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.943+0000 7f749a7fc640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f749c104ef0 0x7f749c100570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f748c00e9c0 tx=0x7f748c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.944+0000 7f7483fff640 1 -- 172.21.15.33:0/2691584720 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c00cd80 con 0x7f749c104ef0 2024-04-02T13:36:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.944+0000 7f74a17cd640 1 -- 172.21.15.33:0/2691584720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749c1a1fc0 con 0x7f749c104ef0 2024-04-02T13:36:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.944+0000 7f7483fff640 1 -- 172.21.15.33:0/2691584720 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f748c0054f0 con 0x7f749c104ef0 2024-04-02T13:36:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.944+0000 7f7483fff640 1 -- 172.21.15.33:0/2691584720 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c010690 con 0x7f749c104ef0 2024-04-02T13:36:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.944+0000 7f74a17cd640 1 -- 172.21.15.33:0/2691584720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f749c1a2480 con 0x7f749c104ef0 2024-04-02T13:36:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.945+0000 7f74a17cd640 1 -- 172.21.15.33:0/2691584720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f749c0fdaa0 con 0x7f749c104ef0 2024-04-02T13:36:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.946+0000 7f7483fff640 1 -- 172.21.15.33:0/2691584720 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f748c004e10 con 0x7f749c104ef0 2024-04-02T13:36:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.947+0000 7f7483fff640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f74840a3bb0 0x7f74840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.947+0000 7f749affd640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f74840a3bb0 0x7f74840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.947+0000 7f7483fff640 1 -- 172.21.15.33:0/2691584720 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f748c0d0470 con 0x7f749c104ef0 2024-04-02T13:36:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.947+0000 7f749affd640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f74840a3bb0 0x7f74840a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7490005ab0 tx=0x7f7490005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:19.952+0000 7f7483fff640 1 -- 172.21.15.33:0/2691584720 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f748c09d180 con 0x7f749c104ef0 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.157+0000 7f74a17cd640 1 -- 172.21.15.33:0/2691584720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f749c02d370 con 0x7f749c104ef0 2024-04-02T13:36:20.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:19 smithi145 ceph-mon[35646]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:20.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:19 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1141469890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:36:20.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:19 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2008995633' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T13:36:20.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:19 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/528844437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T13:36:20.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:19 smithi050 ceph-mon[35524]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:20.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:19 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1141469890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:36:20.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:19 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2008995633' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T13:36:20.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:19 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/528844437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T13:36:20.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:19 smithi033 ceph-mon[32390]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:20.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:19 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1141469890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:36:20.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:19 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2008995633' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T13:36:20.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:19 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/528844437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T13:36:20.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.807+0000 7f7483fff640 1 -- 172.21.15.33:0/2691584720 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7f748c0bd0c0 con 0x7f749c104ef0 2024-04-02T13:36:20.811 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2024-04-02T13:36:20.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.813+0000 7f74a17cd640 1 -- 172.21.15.33:0/2691584720 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f74840a3bb0 msgr2=0x7f74840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.813+0000 7f74a17cd640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f74840a3bb0 0x7f74840a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7490005ab0 tx=0x7f7490005fd0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.813+0000 7f74a17cd640 1 -- 172.21.15.33:0/2691584720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f749c104ef0 msgr2=0x7f749c100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.813+0000 7f74a17cd640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f749c104ef0 0x7f749c100570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f748c00e9c0 tx=0x7f748c00ee80 comp rx=0 tx=0).stop 2024-04-02T13:36:20.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.813+0000 7f74a17cd640 1 -- 172.21.15.33:0/2691584720 shutdown_connections 2024-04-02T13:36:20.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.813+0000 7f74a17cd640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f74840a3bb0 0x7f74840a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.813+0000 7f74a17cd640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f749c107820 0x7f749c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.813+0000 7f74a17cd640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f749c104ef0 0x7f749c100570 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.813+0000 7f74a17cd640 1 --2- 172.21.15.33:0/2691584720 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f749c1025c0 0x7f749c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.813+0000 7f74a17cd640 1 -- 172.21.15.33:0/2691584720 >> 172.21.15.33:0/2691584720 conn(0x7f749c0fbfb0 msgr2=0x7f749c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:20.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.814+0000 7f74a17cd640 1 -- 172.21.15.33:0/2691584720 shutdown_connections 2024-04-02T13:36:20.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.814+0000 7f74a17cd640 1 -- 172.21.15.33:0/2691584720 wait complete. 2024-04-02T13:36:20.839 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-02T13:36:20.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.948+0000 7fefe583c640 1 -- 172.21.15.33:0/968018514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe01036c0 msgr2=0x7fefe0109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.948+0000 7fefe583c640 1 --2- 172.21.15.33:0/968018514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe01036c0 0x7fefe0109f50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009ca0 tx=0x7fefd802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.948+0000 7fefe583c640 1 -- 172.21.15.33:0/968018514 shutdown_connections 2024-04-02T13:36:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.948+0000 7fefe583c640 1 --2- 172.21.15.33:0/968018514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe01036c0 0x7fefe0109f50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.948+0000 7fefe583c640 1 --2- 172.21.15.33:0/968018514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fefe0102d00 0x7fefe0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.948+0000 7fefe583c640 1 --2- 172.21.15.33:0/968018514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefe0101b10 0x7fefe0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.948+0000 7fefe583c640 1 -- 172.21.15.33:0/968018514 >> 172.21.15.33:0/968018514 conn(0x7fefe00fd0a0 msgr2=0x7fefe00ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.948+0000 7fefe583c640 1 -- 172.21.15.33:0/968018514 shutdown_connections 2024-04-02T13:36:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.948+0000 7fefe583c640 1 -- 172.21.15.33:0/968018514 wait complete. 2024-04-02T13:36:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.948+0000 7fefe583c640 1 Processor -- start 2024-04-02T13:36:20.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.948+0000 7fefe583c640 1 -- start start 2024-04-02T13:36:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefe583c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefe0101b10 0x7fefe0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefe583c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe0102d00 0x7fefe0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefe583c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fefe01036c0 0x7fefe019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefe583c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefe010d770 con 0x7fefe0102d00 2024-04-02T13:36:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefe583c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fefe010d5f0 con 0x7fefe0101b10 2024-04-02T13:36:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefe583c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fefe010d8f0 con 0x7fefe01036c0 2024-04-02T13:36:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe0102d00 0x7fefe0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe0102d00 0x7fefe0198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40148/0 (socket says 172.21.15.33:40148) 2024-04-02T13:36:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefd7fff640 1 -- 172.21.15.33:0/3310644906 learned_addr learned my addr 172.21.15.33:0/3310644906 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefdeffd640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefe0101b10 0x7fefe0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefdf7fe640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fefe01036c0 0x7fefe019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefd7fff640 1 -- 172.21.15.33:0/3310644906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fefe01036c0 msgr2=0x7fefe019f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefd7fff640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fefe01036c0 0x7fefe019f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefd7fff640 1 -- 172.21.15.33:0/3310644906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefe0101b10 msgr2=0x7fefe0197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefd7fff640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefe0101b10 0x7fefe0197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefd7fff640 1 -- 172.21.15.33:0/3310644906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe019f9e0 con 0x7fefe0102d00 2024-04-02T13:36:20.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefd7fff640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe0102d00 0x7fefe0198300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fefc800d770 tx=0x7fefc800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:20.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefe483a640 1 -- 172.21.15.33:0/3310644906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8017070 con 0x7fefe0102d00 2024-04-02T13:36:20.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefe483a640 1 -- 172.21.15.33:0/3310644906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fefc80054f0 con 0x7fefe0102d00 2024-04-02T13:36:20.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefe583c640 1 -- 172.21.15.33:0/3310644906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe019fcd0 con 0x7fefe0102d00 2024-04-02T13:36:20.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefe483a640 1 -- 172.21.15.33:0/3310644906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8004d60 con 0x7fefe0102d00 2024-04-02T13:36:20.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.949+0000 7fefe583c640 1 -- 172.21.15.33:0/3310644906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe01a01d0 con 0x7fefe0102d00 2024-04-02T13:36:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.951+0000 7fefe583c640 1 -- 172.21.15.33:0/3310644906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe0108e30 con 0x7fefe0102d00 2024-04-02T13:36:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.952+0000 7fefe483a640 1 -- 172.21.15.33:0/3310644906 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fefc8004f00 con 0x7fefe0102d00 2024-04-02T13:36:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.952+0000 7fefe483a640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fefcc0a3c30 0x7fefcc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.952+0000 7fefdeffd640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fefcc0a3c30 0x7fefcc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.952+0000 7fefe483a640 1 -- 172.21.15.33:0/3310644906 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fefc80d0010 con 0x7fefe0102d00 2024-04-02T13:36:20.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.953+0000 7fefdeffd640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fefcc0a3c30 0x7fefcc0a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fefd0005d00 tx=0x7fefd0005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:20.956+0000 7fefe483a640 1 -- 172.21.15.33:0/3310644906 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fefc809cc20 con 0x7fefe0102d00 2024-04-02T13:36:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.183+0000 7fefe583c640 1 -- 172.21.15.33:0/3310644906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fefe002d370 con 0x7fefe0102d00 2024-04-02T13:36:21.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/528844437' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T13:36:21.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T13:36:21.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: pgmap v285: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:21.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2691584720' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-02T13:36:21.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:21.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:36:21.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2691584720' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-02T13:36:21.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:20 smithi145 ceph-mon[35646]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T13:36:21.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/528844437' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T13:36:21.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T13:36:21.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: pgmap v285: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:21.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2691584720' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-02T13:36:21.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:21.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:36:21.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2691584720' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-02T13:36:21.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:20 smithi050 ceph-mon[35524]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T13:36:21.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/528844437' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T13:36:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T13:36:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: pgmap v285: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2691584720' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-02T13:36:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:36:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:36:21.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2691584720' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-02T13:36:21.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:20 smithi033 ceph-mon[32390]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T13:36:21.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.838+0000 7fefe483a640 1 -- 172.21.15.33:0/3310644906 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fefc809c520 con 0x7fefe0102d00 2024-04-02T13:36:21.841 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 2 and data pool 3 2024-04-02T13:36:21.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.840+0000 7fefe583c640 1 -- 172.21.15.33:0/3310644906 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fefcc0a3c30 msgr2=0x7fefcc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.840+0000 7fefe583c640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fefcc0a3c30 0x7fefcc0a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fefd0005d00 tx=0x7fefd0005c50 comp rx=0 tx=0).stop 2024-04-02T13:36:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.840+0000 7fefe583c640 1 -- 172.21.15.33:0/3310644906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe0102d00 msgr2=0x7fefe0198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.840+0000 7fefe583c640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe0102d00 0x7fefe0198300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fefc800d770 tx=0x7fefc800dc30 comp rx=0 tx=0).stop 2024-04-02T13:36:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.840+0000 7fefe583c640 1 -- 172.21.15.33:0/3310644906 shutdown_connections 2024-04-02T13:36:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.840+0000 7fefe583c640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fefcc0a3c30 0x7fefcc0a60f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.840+0000 7fefe583c640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fefe01036c0 0x7fefe019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.840+0000 7fefe583c640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe0102d00 0x7fefe0198300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.840+0000 7fefe583c640 1 --2- 172.21.15.33:0/3310644906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefe0101b10 0x7fefe0197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.841+0000 7fefe583c640 1 -- 172.21.15.33:0/3310644906 >> 172.21.15.33:0/3310644906 conn(0x7fefe00fd0a0 msgr2=0x7fefe0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.841+0000 7fefe583c640 1 -- 172.21.15.33:0/3310644906 shutdown_connections 2024-04-02T13:36:21.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.841+0000 7fefe583c640 1 -- 172.21.15.33:0/3310644906 wait complete. 2024-04-02T13:36:21.859 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-02T13:36:21.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-02T13:36:21.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.971+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/682078367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8e4102d10 msgr2=0x7fc8e4103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.971+0000 7fc8ec7cd640 1 --2- 172.21.15.33:0/682078367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8e4102d10 0x7fc8e4103190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009900 tx=0x7fc8d402bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:21.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.971+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/682078367 shutdown_connections 2024-04-02T13:36:21.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.971+0000 7fc8ec7cd640 1 --2- 172.21.15.33:0/682078367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e41036d0 0x7fc8e4109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.971+0000 7fc8ec7cd640 1 --2- 172.21.15.33:0/682078367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8e4102d10 0x7fc8e4103190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.971+0000 7fc8ec7cd640 1 --2- 172.21.15.33:0/682078367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8e4101b20 0x7fc8e4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.971+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/682078367 >> 172.21.15.33:0/682078367 conn(0x7fc8e40fd070 msgr2=0x7fc8e40ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.971+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/682078367 shutdown_connections 2024-04-02T13:36:21.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.971+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/682078367 wait complete. 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ec7cd640 1 Processor -- start 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ec7cd640 1 -- start start 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ec7cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8e4101b20 0x7fc8e4197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ec7cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8e4102d10 0x7fc8e41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ec7cd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e41036d0 0x7fc8e419f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ec7cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e410d7e0 con 0x7fc8e4101b20 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ec7cd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e410d660 con 0x7fc8e41036d0 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ec7cd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e410d960 con 0x7fc8e4102d10 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ea569640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8e4101b20 0x7fc8e4197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ea569640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8e4101b20 0x7fc8e4197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40158/0 (socket says 172.21.15.33:40158) 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ea569640 1 -- 172.21.15.33:0/985371210 learned_addr learned my addr 172.21.15.33:0/985371210 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8e9d68640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8e4102d10 0x7fc8e41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ead6a640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e41036d0 0x7fc8e419f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ea569640 1 -- 172.21.15.33:0/985371210 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8e4102d10 msgr2=0x7fc8e41983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ea569640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8e4102d10 0x7fc8e41983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ea569640 1 -- 172.21.15.33:0/985371210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e41036d0 msgr2=0x7fc8e419f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ea569640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e41036d0 0x7fc8e419f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ea569640 1 -- 172.21.15.33:0/985371210 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e419faa0 con 0x7fc8e4101b20 2024-04-02T13:36:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ead6a640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e41036d0 0x7fc8e419f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.972+0000 7fc8ea569640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8e4101b20 0x7fc8e4197e80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000c970 tx=0x7fc8e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.973+0000 7fc8db7fe640 1 -- 172.21.15.33:0/985371210 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0004220 con 0x7fc8e4101b20 2024-04-02T13:36:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.973+0000 7fc8db7fe640 1 -- 172.21.15.33:0/985371210 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc8e00043c0 con 0x7fc8e4101b20 2024-04-02T13:36:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.973+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/985371210 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e419fd90 con 0x7fc8e4101b20 2024-04-02T13:36:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.973+0000 7fc8db7fe640 1 -- 172.21.15.33:0/985371210 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e000f450 con 0x7fc8e4101b20 2024-04-02T13:36:21.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.973+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/985371210 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e41a0270 con 0x7fc8e4101b20 2024-04-02T13:36:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.974+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/985371210 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e4108f60 con 0x7fc8e4101b20 2024-04-02T13:36:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.975+0000 7fc8db7fe640 1 -- 172.21.15.33:0/985371210 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc8e00079b0 con 0x7fc8e4101b20 2024-04-02T13:36:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.976+0000 7fc8db7fe640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc8d00a3bb0 0x7fc8d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.976+0000 7fc8e9d68640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc8d00a3bb0 0x7fc8d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.976+0000 7fc8db7fe640 1 -- 172.21.15.33:0/985371210 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc8e00d0400 con 0x7fc8e4101b20 2024-04-02T13:36:21.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.976+0000 7fc8e9d68640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc8d00a3bb0 0x7fc8d00a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc8d402c880 tx=0x7fc8d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:21.980+0000 7fc8db7fe640 1 -- 172.21.15.33:0/985371210 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc8e0014030 con 0x7fc8e4101b20 2024-04-02T13:36:22.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.173+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/985371210 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fc8e402d370 con 0x7fc8e4101b20 2024-04-02T13:36:22.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:21 smithi145 ceph-mon[35646]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:36:22.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:21 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3310644906' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T13:36:22.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:21 smithi145 ceph-mon[35646]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T13:36:22.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:21 smithi145 ceph-mon[35646]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T13:36:22.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:21 smithi050 ceph-mon[35524]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:36:22.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:21 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3310644906' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T13:36:22.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:21 smithi050 ceph-mon[35524]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T13:36:22.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:21 smithi050 ceph-mon[35524]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T13:36:22.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:21 smithi033 ceph-mon[32390]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:36:22.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:21 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3310644906' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T13:36:22.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:21 smithi033 ceph-mon[32390]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T13:36:22.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:21 smithi033 ceph-mon[32390]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T13:36:22.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.471+0000 7fc8db7fe640 1 -- 172.21.15.33:0/985371210 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7fc8e009cf30 con 0x7fc8e4101b20 2024-04-02T13:36:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.474+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/985371210 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc8d00a3bb0 msgr2=0x7fc8d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.474+0000 7fc8ec7cd640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc8d00a3bb0 0x7fc8d00a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc8d402c880 tx=0x7fc8d4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.474+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/985371210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8e4101b20 msgr2=0x7fc8e4197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.474+0000 7fc8ec7cd640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8e4101b20 0x7fc8e4197e80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000c970 tx=0x7fc8e000ce30 comp rx=0 tx=0).stop 2024-04-02T13:36:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.474+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/985371210 shutdown_connections 2024-04-02T13:36:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.474+0000 7fc8ec7cd640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc8d00a3bb0 0x7fc8d00a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.474+0000 7fc8ec7cd640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8e41036d0 0x7fc8e419f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.474+0000 7fc8ec7cd640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8e4102d10 0x7fc8e41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.474+0000 7fc8ec7cd640 1 --2- 172.21.15.33:0/985371210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8e4101b20 0x7fc8e4197e80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.474+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/985371210 >> 172.21.15.33:0/985371210 conn(0x7fc8e40fd070 msgr2=0x7fc8e4106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.474+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/985371210 shutdown_connections 2024-04-02T13:36:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.474+0000 7fc8ec7cd640 1 -- 172.21.15.33:0/985371210 wait complete. 2024-04-02T13:36:22.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-02T13:36:22.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.598+0000 7fd2e992b640 1 -- 172.21.15.33:0/450228268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e4100a20 msgr2=0x7fd2e4100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.598+0000 7fd2e992b640 1 --2- 172.21.15.33:0/450228268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e4100a20 0x7fd2e4100e40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009ca0 tx=0x7fd2d402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.598+0000 7fd2e992b640 1 -- 172.21.15.33:0/450228268 shutdown_connections 2024-04-02T13:36:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.598+0000 7fd2e992b640 1 --2- 172.21.15.33:0/450228268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2e41025d0 0x7fd2e4108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.598+0000 7fd2e992b640 1 --2- 172.21.15.33:0/450228268 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2e4101c10 0x7fd2e4102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.598+0000 7fd2e992b640 1 --2- 172.21.15.33:0/450228268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e4100a20 0x7fd2e4100e40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.598+0000 7fd2e992b640 1 -- 172.21.15.33:0/450228268 >> 172.21.15.33:0/450228268 conn(0x7fd2e40fbf90 msgr2=0x7fd2e40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.598+0000 7fd2e992b640 1 -- 172.21.15.33:0/450228268 shutdown_connections 2024-04-02T13:36:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.598+0000 7fd2e992b640 1 -- 172.21.15.33:0/450228268 wait complete. 2024-04-02T13:36:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e992b640 1 Processor -- start 2024-04-02T13:36:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e992b640 1 -- start start 2024-04-02T13:36:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e992b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2e4100a20 0x7fd2e410b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e992b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2e4101c10 0x7fd2e41099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e992b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e41025d0 0x7fd2e4109f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e992b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e410d870 con 0x7fd2e41025d0 2024-04-02T13:36:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e992b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e410d6f0 con 0x7fd2e4100a20 2024-04-02T13:36:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e992b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e410d9f0 con 0x7fd2e4101c10 2024-04-02T13:36:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e41025d0 0x7fd2e4109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e41025d0 0x7fd2e4109f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40172/0 (socket says 172.21.15.33:40172) 2024-04-02T13:36:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e37fe640 1 -- 172.21.15.33:0/508171528 learned_addr learned my addr 172.21.15.33:0/508171528 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e27fc640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2e4101c10 0x7fd2e41099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e2ffd640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2e4100a20 0x7fd2e410b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e37fe640 1 -- 172.21.15.33:0/508171528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2e4101c10 msgr2=0x7fd2e41099e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e37fe640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2e4101c10 0x7fd2e41099e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e37fe640 1 -- 172.21.15.33:0/508171528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2e4100a20 msgr2=0x7fd2e410b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e37fe640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2e4100a20 0x7fd2e410b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e37fe640 1 -- 172.21.15.33:0/508171528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e410a7b0 con 0x7fd2e41025d0 2024-04-02T13:36:22.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.599+0000 7fd2e37fe640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e41025d0 0x7fd2e4109f20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009fd0 tx=0x7fd2dc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.600+0000 7fd2e8929640 1 -- 172.21.15.33:0/508171528 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc011830 con 0x7fd2e41025d0 2024-04-02T13:36:22.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.600+0000 7fd2e8929640 1 -- 172.21.15.33:0/508171528 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd2dc016070 con 0x7fd2e41025d0 2024-04-02T13:36:22.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.600+0000 7fd2e992b640 1 -- 172.21.15.33:0/508171528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e41a20c0 con 0x7fd2e41025d0 2024-04-02T13:36:22.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.600+0000 7fd2e8929640 1 -- 172.21.15.33:0/508171528 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc00f2e0 con 0x7fd2e41025d0 2024-04-02T13:36:22.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.600+0000 7fd2e992b640 1 -- 172.21.15.33:0/508171528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e41a2640 con 0x7fd2e41025d0 2024-04-02T13:36:22.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.601+0000 7fd2e992b640 1 -- 172.21.15.33:0/508171528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e4107e10 con 0x7fd2e41025d0 2024-04-02T13:36:22.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.602+0000 7fd2e8929640 1 -- 172.21.15.33:0/508171528 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd2dc0119d0 con 0x7fd2e41025d0 2024-04-02T13:36:22.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.602+0000 7fd2e8929640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd2cc0a3bb0 0x7fd2cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.602+0000 7fd2e2ffd640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd2cc0a3bb0 0x7fd2cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.602+0000 7fd2e8929640 1 -- 172.21.15.33:0/508171528 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7fd2dc0cfc60 con 0x7fd2e41025d0 2024-04-02T13:36:22.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.603+0000 7fd2e2ffd640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd2cc0a3bb0 0x7fd2cc0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd2d402c700 tx=0x7fd2d40323d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.607+0000 7fd2e8929640 1 -- 172.21.15.33:0/508171528 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd2dc09c610 con 0x7fd2e41025d0 2024-04-02T13:36:22.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:22.794+0000 7fd2e992b640 1 -- 172.21.15.33:0/508171528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd2e4062700 con 0x7fd2e41025d0 2024-04-02T13:36:23.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:22 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3310644906' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T13:36:23.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:22 smithi145 ceph-mon[35646]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T13:36:23.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:22 smithi145 ceph-mon[35646]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T13:36:23.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:22 smithi145 ceph-mon[35646]: pgmap v288: 97 pgs: 14 creating+peering, 23 active+clean, 60 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:23.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:22 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/985371210' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-02T13:36:23.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:22 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/985371210' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-02T13:36:23.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:22 smithi145 ceph-mon[35646]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T13:36:23.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:22 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/508171528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T13:36:23.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:22 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3310644906' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T13:36:23.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:22 smithi050 ceph-mon[35524]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T13:36:23.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:22 smithi050 ceph-mon[35524]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T13:36:23.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:22 smithi050 ceph-mon[35524]: pgmap v288: 97 pgs: 14 creating+peering, 23 active+clean, 60 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:23.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:22 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/985371210' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-02T13:36:23.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:22 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/985371210' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-02T13:36:23.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:22 smithi050 ceph-mon[35524]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T13:36:23.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:22 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/508171528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T13:36:23.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:22 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3310644906' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T13:36:23.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:22 smithi033 ceph-mon[32390]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T13:36:23.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:22 smithi033 ceph-mon[32390]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T13:36:23.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:22 smithi033 ceph-mon[32390]: pgmap v288: 97 pgs: 14 creating+peering, 23 active+clean, 60 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:23.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:22 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/985371210' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-02T13:36:23.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:22 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/985371210' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-02T13:36:23.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:22 smithi033 ceph-mon[32390]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T13:36:23.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:22 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/508171528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T13:36:24.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:23 smithi145 ceph-mon[35646]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T13:36:24.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:23 smithi145 ceph-mon[35646]: Cluster is now healthy 2024-04-02T13:36:24.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:23 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] up:active 2024-04-02T13:36:24.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:23 smithi145 ceph-mon[35646]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:24.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:23 smithi145 ceph-mon[35646]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:24.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:23 smithi145 ceph-mon[35646]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T13:36:24.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:23 smithi050 ceph-mon[35524]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T13:36:24.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:23 smithi050 ceph-mon[35524]: Cluster is now healthy 2024-04-02T13:36:24.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:23 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] up:active 2024-04-02T13:36:24.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:23 smithi050 ceph-mon[35524]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:24.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:23 smithi050 ceph-mon[35524]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:24.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:23 smithi050 ceph-mon[35524]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T13:36:24.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:23 smithi033 ceph-mon[32390]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T13:36:24.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:23 smithi033 ceph-mon[32390]: Cluster is now healthy 2024-04-02T13:36:24.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:23 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] up:active 2024-04-02T13:36:24.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:23 smithi033 ceph-mon[32390]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:24.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:23 smithi033 ceph-mon[32390]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:24.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:23 smithi033 ceph-mon[32390]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T13:36:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.475+0000 7fd2e8929640 1 -- 172.21.15.33:0/508171528 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7fd2dc09bf10 con 0x7fd2e41025d0 2024-04-02T13:36:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data_ec' created 2024-04-02T13:36:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.478+0000 7fd2e992b640 1 -- 172.21.15.33:0/508171528 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd2cc0a3bb0 msgr2=0x7fd2cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.478+0000 7fd2e992b640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd2cc0a3bb0 0x7fd2cc0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd2d402c700 tx=0x7fd2d40323d0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.478+0000 7fd2e992b640 1 -- 172.21.15.33:0/508171528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e41025d0 msgr2=0x7fd2e4109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.478+0000 7fd2e992b640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e41025d0 0x7fd2e4109f20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009fd0 tx=0x7fd2dc00b8e0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.478+0000 7fd2e992b640 1 -- 172.21.15.33:0/508171528 shutdown_connections 2024-04-02T13:36:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.478+0000 7fd2e992b640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd2cc0a3bb0 0x7fd2cc0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.478+0000 7fd2e992b640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2e41025d0 0x7fd2e4109f20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.478+0000 7fd2e992b640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2e4101c10 0x7fd2e41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.478+0000 7fd2e992b640 1 --2- 172.21.15.33:0/508171528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2e4100a20 0x7fd2e410b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.478+0000 7fd2e992b640 1 -- 172.21.15.33:0/508171528 >> 172.21.15.33:0/508171528 conn(0x7fd2e40fbf90 msgr2=0x7fd2e4104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.479+0000 7fd2e992b640 1 -- 172.21.15.33:0/508171528 shutdown_connections 2024-04-02T13:36:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.479+0000 7fd2e992b640 1 -- 172.21.15.33:0/508171528 wait complete. 2024-04-02T13:36:24.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-02T13:36:24.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 -- 172.21.15.33:0/2784150164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f445c1036b0 msgr2=0x7f445c109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 --2- 172.21.15.33:0/2784150164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f445c1036b0 0x7f445c109f40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f444c009900 tx=0x7f444c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:24.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 -- 172.21.15.33:0/2784150164 shutdown_connections 2024-04-02T13:36:24.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 --2- 172.21.15.33:0/2784150164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f445c1036b0 0x7f445c109f40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 --2- 172.21.15.33:0/2784150164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c102cf0 0x7f445c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 --2- 172.21.15.33:0/2784150164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f445c101b00 0x7f445c101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 -- 172.21.15.33:0/2784150164 >> 172.21.15.33:0/2784150164 conn(0x7f445c0fd070 msgr2=0x7f445c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 -- 172.21.15.33:0/2784150164 shutdown_connections 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 -- 172.21.15.33:0/2784150164 wait complete. 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 Processor -- start 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 -- start start 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f445c101b00 0x7f445c197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c102cf0 0x7f445c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f445c1036b0 0x7f445c19f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f445c10d760 con 0x7f445c101b00 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f445c10d5e0 con 0x7f445c1036b0 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4462d7b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f445c10d8e0 con 0x7f445c102cf0 2024-04-02T13:36:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.611+0000 7f4460b17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f445c101b00 0x7f445c197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4460b17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f445c101b00 0x7f445c197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53450/0 (socket says 172.21.15.33:53450) 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4460b17640 1 -- 172.21.15.33:0/948669454 learned_addr learned my addr 172.21.15.33:0/948669454 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4460b17640 1 -- 172.21.15.33:0/948669454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c102cf0 msgr2=0x7f445c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4460b17640 1 --2- 172.21.15.33:0/948669454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c102cf0 0x7f445c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4460b17640 1 -- 172.21.15.33:0/948669454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f445c1036b0 msgr2=0x7f445c19f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4461318640 1 --2- 172.21.15.33:0/948669454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f445c1036b0 0x7f445c19f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4460b17640 1 --2- 172.21.15.33:0/948669454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f445c1036b0 0x7f445c19f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4460b17640 1 -- 172.21.15.33:0/948669454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445c19f9d0 con 0x7f445c101b00 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4460b17640 1 --2- 172.21.15.33:0/948669454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f445c101b00 0x7f445c197db0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f445400ccb0 tx=0x7f445400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4459ffb640 1 -- 172.21.15.33:0/948669454 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454003990 con 0x7f445c101b00 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4462d7b640 1 -- 172.21.15.33:0/948669454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445c19fcc0 con 0x7f445c101b00 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4459ffb640 1 -- 172.21.15.33:0/948669454 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4454004020 con 0x7f445c101b00 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4459ffb640 1 -- 172.21.15.33:0/948669454 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445400f780 con 0x7f445c101b00 2024-04-02T13:36:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.612+0000 7f4462d7b640 1 -- 172.21.15.33:0/948669454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445c1a0200 con 0x7f445c101b00 2024-04-02T13:36:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.613+0000 7f4462d7b640 1 -- 172.21.15.33:0/948669454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f445c108e20 con 0x7f445c101b00 2024-04-02T13:36:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.614+0000 7f4459ffb640 1 -- 172.21.15.33:0/948669454 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f44540041d0 con 0x7f445c101b00 2024-04-02T13:36:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.615+0000 7f4459ffb640 1 --2- 172.21.15.33:0/948669454 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f44480a3bb0 0x7f44480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.615+0000 7f445bfff640 1 --2- 172.21.15.33:0/948669454 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f44480a3bb0 0x7f44480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.615+0000 7f4459ffb640 1 -- 172.21.15.33:0/948669454 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f44540d0b10 con 0x7f445c101b00 2024-04-02T13:36:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.615+0000 7f445bfff640 1 --2- 172.21.15.33:0/948669454 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f44480a3bb0 0x7f44480a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4444005fd0 tx=0x7f4444005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:24.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.620+0000 7f4459ffb640 1 -- 172.21.15.33:0/948669454 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4454014030 con 0x7f445c101b00 2024-04-02T13:36:24.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:24.806+0000 7f4462d7b640 1 -- 172.21.15.33:0/948669454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f445c02d370 con 0x7f445c101b00 2024-04-02T13:36:25.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.477+0000 7f4459ffb640 1 -- 172.21.15.33:0/948669454 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7f445409d2d0 con 0x7f445c101b00 2024-04-02T13:36:25.480 INFO:teuthology.orchestra.run.smithi033.stderr:set pool 4 allow_ec_overwrites to true 2024-04-02T13:36:25.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.481+0000 7f4462d7b640 1 -- 172.21.15.33:0/948669454 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f44480a3bb0 msgr2=0x7f44480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:25.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.481+0000 7f4462d7b640 1 --2- 172.21.15.33:0/948669454 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f44480a3bb0 0x7f44480a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4444005fd0 tx=0x7f4444005850 comp rx=0 tx=0).stop 2024-04-02T13:36:25.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.481+0000 7f4462d7b640 1 -- 172.21.15.33:0/948669454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f445c101b00 msgr2=0x7f445c197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:25.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.481+0000 7f4462d7b640 1 --2- 172.21.15.33:0/948669454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f445c101b00 0x7f445c197db0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f445400ccb0 tx=0x7f445400a620 comp rx=0 tx=0).stop 2024-04-02T13:36:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.483+0000 7f4462d7b640 1 -- 172.21.15.33:0/948669454 shutdown_connections 2024-04-02T13:36:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.483+0000 7f4462d7b640 1 --2- 172.21.15.33:0/948669454 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f44480a3bb0 0x7f44480a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.483+0000 7f4462d7b640 1 --2- 172.21.15.33:0/948669454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f445c1036b0 0x7f445c19f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.483+0000 7f4462d7b640 1 --2- 172.21.15.33:0/948669454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f445c102cf0 0x7f445c1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.483+0000 7f4462d7b640 1 --2- 172.21.15.33:0/948669454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f445c101b00 0x7f445c197db0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.483+0000 7f4462d7b640 1 -- 172.21.15.33:0/948669454 >> 172.21.15.33:0/948669454 conn(0x7f445c0fd070 msgr2=0x7f445c105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:25.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.486+0000 7f4462d7b640 1 -- 172.21.15.33:0/948669454 shutdown_connections 2024-04-02T13:36:25.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.486+0000 7f4462d7b640 1 -- 172.21.15.33:0/948669454 wait complete. 2024-04-02T13:36:25.512 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-02T13:36:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.620+0000 7f0dfb80f640 1 -- 172.21.15.33:0/2187276981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df41025e0 msgr2=0x7f0df41049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.620+0000 7f0dfb80f640 1 --2- 172.21.15.33:0/2187276981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df41025e0 0x7f0df41049d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009900 tx=0x7f0dec02bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.620+0000 7f0dfb80f640 1 -- 172.21.15.33:0/2187276981 shutdown_connections 2024-04-02T13:36:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.620+0000 7f0dfb80f640 1 --2- 172.21.15.33:0/2187276981 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0df4107840 0x7f0df4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.620+0000 7f0dfb80f640 1 --2- 172.21.15.33:0/2187276981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df4104f10 0x7f0df4107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.620+0000 7f0dfb80f640 1 --2- 172.21.15.33:0/2187276981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df41025e0 0x7f0df41049d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.620+0000 7f0dfb80f640 1 -- 172.21.15.33:0/2187276981 >> 172.21.15.33:0/2187276981 conn(0x7f0df40fbf90 msgr2=0x7f0df40fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.620+0000 7f0dfb80f640 1 -- 172.21.15.33:0/2187276981 shutdown_connections 2024-04-02T13:36:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.620+0000 7f0dfb80f640 1 -- 172.21.15.33:0/2187276981 wait complete. 2024-04-02T13:36:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.620+0000 7f0dfb80f640 1 Processor -- start 2024-04-02T13:36:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0dfb80f640 1 -- start start 2024-04-02T13:36:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0dfb80f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0df41025e0 0x7f0df4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0dfb80f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df4104f10 0x7f0df4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0dfb80f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df4107840 0x7f0df419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0dfb80f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0df410d7e0 con 0x7f0df4107840 2024-04-02T13:36:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0dfb80f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0df410d660 con 0x7f0df4104f10 2024-04-02T13:36:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0dfb80f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0df410d960 con 0x7f0df41025e0 2024-04-02T13:36:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0df9dac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df4107840 0x7f0df419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0df9dac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df4107840 0x7f0df419f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53476/0 (socket says 172.21.15.33:53476) 2024-04-02T13:36:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0df9dac640 1 -- 172.21.15.33:0/911934596 learned_addr learned my addr 172.21.15.33:0/911934596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0df95ab640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0df41025e0 0x7f0df4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0df8daa640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df4104f10 0x7f0df4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0df9dac640 1 -- 172.21.15.33:0/911934596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0df41025e0 msgr2=0x7f0df4197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0df9dac640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0df41025e0 0x7f0df4197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0df9dac640 1 -- 172.21.15.33:0/911934596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df4104f10 msgr2=0x7f0df4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0df9dac640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df4104f10 0x7f0df4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0df9dac640 1 -- 172.21.15.33:0/911934596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df419fa40 con 0x7f0df4107840 2024-04-02T13:36:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0df9dac640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df4107840 0x7f0df419f3d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0de400b970 tx=0x7f0de400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0df27fc640 1 -- 172.21.15.33:0/911934596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4019070 con 0x7f0df4107840 2024-04-02T13:36:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0df27fc640 1 -- 172.21.15.33:0/911934596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0de4014440 con 0x7f0df4107840 2024-04-02T13:36:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.621+0000 7f0dfb80f640 1 -- 172.21.15.33:0/911934596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df419fd30 con 0x7f0df4107840 2024-04-02T13:36:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.622+0000 7f0df27fc640 1 -- 172.21.15.33:0/911934596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de40075d0 con 0x7f0df4107840 2024-04-02T13:36:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.622+0000 7f0dfb80f640 1 -- 172.21.15.33:0/911934596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df41a02a0 con 0x7f0df4107840 2024-04-02T13:36:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.622+0000 7f0dfb80f640 1 -- 172.21.15.33:0/911934596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df40fda80 con 0x7f0df4107840 2024-04-02T13:36:25.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.623+0000 7f0df27fc640 1 -- 172.21.15.33:0/911934596 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0de401c070 con 0x7f0df4107840 2024-04-02T13:36:25.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.624+0000 7f0df27fc640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0ddc0a3bb0 0x7f0ddc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:25.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.624+0000 7f0df95ab640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0ddc0a3bb0 0x7f0ddc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:25.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.624+0000 7f0df27fc640 1 -- 172.21.15.33:0/911934596 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f0de40cff10 con 0x7f0df4107840 2024-04-02T13:36:25.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.624+0000 7f0df95ab640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0ddc0a3bb0 0x7f0ddc0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0dec02c880 tx=0x7f0dec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.628+0000 7f0df27fc640 1 -- 172.21.15.33:0/911934596 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0de409c080 con 0x7f0df4107840 2024-04-02T13:36:25.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:25 smithi050 ceph-mon[35524]: pgmap v291: 97 pgs: 14 creating+peering, 49 active+clean, 34 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:25.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:25 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/508171528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T13:36:25.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:25 smithi050 ceph-mon[35524]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T13:36:25.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:25 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/948669454' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-02T13:36:25.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:25 smithi033 ceph-mon[32390]: pgmap v291: 97 pgs: 14 creating+peering, 49 active+clean, 34 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:25.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:25 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/508171528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T13:36:25.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:25 smithi033 ceph-mon[32390]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T13:36:25.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:25 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/948669454' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-02T13:36:25.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:25.855+0000 7f0dfb80f640 1 -- 172.21.15.33:0/911934596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f0df402d370 con 0x7f0df4107840 2024-04-02T13:36:25.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:25 smithi145 ceph-mon[35646]: pgmap v291: 97 pgs: 14 creating+peering, 49 active+clean, 34 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:36:25.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:25 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/508171528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T13:36:25.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:25 smithi145 ceph-mon[35646]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T13:36:25.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:25 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/948669454' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-02T13:36:26.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.488+0000 7f0df27fc640 1 -- 172.21.15.33:0/911934596 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f0de40a0380 con 0x7f0df4107840 2024-04-02T13:36:26.491 INFO:teuthology.orchestra.run.smithi033.stderr:added data pool 4 to fsmap 2024-04-02T13:36:26.492 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:36:26 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-b[66325]: 2024-04-02T13:36:26.011+0000 7f9385417700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-02T13:36:26.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.491+0000 7f0dfb80f640 1 -- 172.21.15.33:0/911934596 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0ddc0a3bb0 msgr2=0x7f0ddc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:26.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.491+0000 7f0dfb80f640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0ddc0a3bb0 0x7f0ddc0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0dec02c880 tx=0x7f0dec005fd0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.491+0000 7f0dfb80f640 1 -- 172.21.15.33:0/911934596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df4107840 msgr2=0x7f0df419f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:26.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.491+0000 7f0dfb80f640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df4107840 0x7f0df419f3d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0de400b970 tx=0x7f0de400bd70 comp rx=0 tx=0).stop 2024-04-02T13:36:26.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.491+0000 7f0dfb80f640 1 -- 172.21.15.33:0/911934596 shutdown_connections 2024-04-02T13:36:26.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.491+0000 7f0dfb80f640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0ddc0a3bb0 0x7f0ddc0a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.491+0000 7f0dfb80f640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df4107840 0x7f0df419f3d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.491+0000 7f0dfb80f640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0df4104f10 0x7f0df4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.491+0000 7f0dfb80f640 1 --2- 172.21.15.33:0/911934596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0df41025e0 0x7f0df4197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.491+0000 7f0dfb80f640 1 -- 172.21.15.33:0/911934596 >> 172.21.15.33:0/911934596 conn(0x7f0df40fbf90 msgr2=0x7f0df4102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:26.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.491+0000 7f0dfb80f640 1 -- 172.21.15.33:0/911934596 shutdown_connections 2024-04-02T13:36:26.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.492+0000 7f0dfb80f640 1 -- 172.21.15.33:0/911934596 wait complete. 2024-04-02T13:36:26.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.614+0000 7f73d739d640 1 -- 172.21.15.33:0/1863516319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73d01025e0 msgr2=0x7f73d0108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.614+0000 7f73d739d640 1 --2- 172.21.15.33:0/1863516319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73d01025e0 0x7f73d0108e70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009ca0 tx=0x7f73c802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.614+0000 7f73d739d640 1 -- 172.21.15.33:0/1863516319 shutdown_connections 2024-04-02T13:36:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.614+0000 7f73d739d640 1 --2- 172.21.15.33:0/1863516319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73d01025e0 0x7f73d0108e70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.614+0000 7f73d739d640 1 --2- 172.21.15.33:0/1863516319 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0101c20 0x7f73d01020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.614+0000 7f73d739d640 1 --2- 172.21.15.33:0/1863516319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73d0100a30 0x7f73d0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.614+0000 7f73d739d640 1 -- 172.21.15.33:0/1863516319 >> 172.21.15.33:0/1863516319 conn(0x7f73d00fbfe0 msgr2=0x7f73d00fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.614+0000 7f73d739d640 1 -- 172.21.15.33:0/1863516319 shutdown_connections 2024-04-02T13:36:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.614+0000 7f73d739d640 1 -- 172.21.15.33:0/1863516319 wait complete. 2024-04-02T13:36:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d739d640 1 Processor -- start 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d739d640 1 -- start start 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d739d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73d0100a30 0x7f73d01049c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d739d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0101c20 0x7f73d0103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d739d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73d01025e0 0x7f73d0103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d739d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f73d010c690 con 0x7f73d01025e0 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d739d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f73d010c510 con 0x7f73d0100a30 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d739d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f73d010c810 con 0x7f73d0101c20 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d593a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73d01025e0 0x7f73d0103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d593a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73d01025e0 0x7f73d0103540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53490/0 (socket says 172.21.15.33:53490) 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d593a640 1 -- 172.21.15.33:0/166739439 learned_addr learned my addr 172.21.15.33:0/166739439 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d5139640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73d0100a30 0x7f73d01049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d4938640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0101c20 0x7f73d0103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d593a640 1 -- 172.21.15.33:0/166739439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0101c20 msgr2=0x7f73d0103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d593a640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0101c20 0x7f73d0103000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d593a640 1 -- 172.21.15.33:0/166739439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73d0100a30 msgr2=0x7f73d01049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d593a640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73d0100a30 0x7f73d01049c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d593a640 1 -- 172.21.15.33:0/166739439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d0103dd0 con 0x7f73d01025e0 2024-04-02T13:36:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.615+0000 7f73d5139640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73d0100a30 0x7f73d01049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.616+0000 7f73d593a640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73d01025e0 0x7f73d0103540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f73c802c630 tx=0x7f73c8032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.616+0000 7f73c67fc640 1 -- 172.21.15.33:0/166739439 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8039050 con 0x7f73d01025e0 2024-04-02T13:36:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.616+0000 7f73c67fc640 1 -- 172.21.15.33:0/166739439 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f73c803a040 con 0x7f73d01025e0 2024-04-02T13:36:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.616+0000 7f73d739d640 1 -- 172.21.15.33:0/166739439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d01a1ee0 con 0x7f73d01025e0 2024-04-02T13:36:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.616+0000 7f73c67fc640 1 -- 172.21.15.33:0/166739439 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8040450 con 0x7f73d01025e0 2024-04-02T13:36:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.616+0000 7f73d739d640 1 -- 172.21.15.33:0/166739439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d01a2340 con 0x7f73d01025e0 2024-04-02T13:36:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.617+0000 7f73d739d640 1 -- 172.21.15.33:0/166739439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d01a2600 con 0x7f73d01025e0 2024-04-02T13:36:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.618+0000 7f73c67fc640 1 -- 172.21.15.33:0/166739439 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f73c8003c70 con 0x7f73d01025e0 2024-04-02T13:36:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.618+0000 7f73c67fc640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f73bc0a3bb0 0x7f73bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.619+0000 7f73d5139640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f73bc0a3bb0 0x7f73bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.619+0000 7f73c67fc640 1 -- 172.21.15.33:0/166739439 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f73c80f3390 con 0x7f73d01025e0 2024-04-02T13:36:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.619+0000 7f73d5139640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f73bc0a3bb0 0x7f73bc0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0058c0 tx=0x7f73cc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.623+0000 7f73c67fc640 1 -- 172.21.15.33:0/166739439 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f73c80bfad0 con 0x7f73d01025e0 2024-04-02T13:36:26.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:26 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/948669454' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-02T13:36:26.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:26 smithi050 ceph-mon[35524]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T13:36:26.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:26 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/911934596' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-02T13:36:26.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:26 smithi050 ceph-mon[35524]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T13:36:26.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:26 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/948669454' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-02T13:36:26.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:26 smithi033 ceph-mon[32390]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T13:36:26.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:26 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/911934596' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-02T13:36:26.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:26 smithi033 ceph-mon[32390]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T13:36:26.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.851+0000 7f73d739d640 1 -- 172.21.15.33:0/166739439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73d01a2960 con 0x7f73d01025e0 2024-04-02T13:36:26.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.851+0000 7f73c67fc640 1 -- 172.21.15.33:0/166739439 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11800 (secure 0 0 0) 0x7f73c80bf3d0 con 0x7f73d01025e0 2024-04-02T13:36:26.854 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:26.854 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:36:25.856615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24551,"qdb_cluster":[24551]},"id":1}]} 2024-04-02T13:36:26.855 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-04-02T13:36:26.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.853+0000 7f73d739d640 1 -- 172.21.15.33:0/166739439 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f73bc0a3bb0 msgr2=0x7f73bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:26.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.853+0000 7f73d739d640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f73bc0a3bb0 0x7f73bc0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0058c0 tx=0x7f73cc005850 comp rx=0 tx=0).stop 2024-04-02T13:36:26.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.853+0000 7f73d739d640 1 -- 172.21.15.33:0/166739439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73d01025e0 msgr2=0x7f73d0103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:26.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.853+0000 7f73d739d640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73d01025e0 0x7f73d0103540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f73c802c630 tx=0x7f73c8032980 comp rx=0 tx=0).stop 2024-04-02T13:36:26.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.853+0000 7f73d739d640 1 -- 172.21.15.33:0/166739439 shutdown_connections 2024-04-02T13:36:26.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.853+0000 7f73d739d640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f73bc0a3bb0 0x7f73bc0a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.853+0000 7f73d739d640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73d01025e0 0x7f73d0103540 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.853+0000 7f73d739d640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73d0101c20 0x7f73d0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.853+0000 7f73d739d640 1 --2- 172.21.15.33:0/166739439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73d0100a30 0x7f73d01049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.853+0000 7f73d739d640 1 -- 172.21.15.33:0/166739439 >> 172.21.15.33:0/166739439 conn(0x7f73d00fbfe0 msgr2=0x7f73d00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:26.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.853+0000 7f73d739d640 1 -- 172.21.15.33:0/166739439 shutdown_connections 2024-04-02T13:36:26.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.853+0000 7f73d739d640 1 -- 172.21.15.33:0/166739439 wait complete. 2024-04-02T13:36:26.868 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:36:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.968+0000 7f9231d4a640 1 -- 172.21.15.33:0/351922452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f922c100a20 msgr2=0x7f922c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.968+0000 7f9231d4a640 1 --2- 172.21.15.33:0/351922452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f922c100a20 0x7f922c100e40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f921c009900 tx=0x7f921c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.968+0000 7f9231d4a640 1 -- 172.21.15.33:0/351922452 shutdown_connections 2024-04-02T13:36:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.968+0000 7f9231d4a640 1 --2- 172.21.15.33:0/351922452 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f922c1025d0 0x7f922c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.968+0000 7f9231d4a640 1 --2- 172.21.15.33:0/351922452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922c101c10 0x7f922c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.968+0000 7f9231d4a640 1 --2- 172.21.15.33:0/351922452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f922c100a20 0x7f922c100e40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.968+0000 7f9231d4a640 1 -- 172.21.15.33:0/351922452 >> 172.21.15.33:0/351922452 conn(0x7f922c0fbfb0 msgr2=0x7f922c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.968+0000 7f9231d4a640 1 -- 172.21.15.33:0/351922452 shutdown_connections 2024-04-02T13:36:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.968+0000 7f9231d4a640 1 -- 172.21.15.33:0/351922452 wait complete. 2024-04-02T13:36:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.968+0000 7f9231d4a640 1 Processor -- start 2024-04-02T13:36:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f9231d4a640 1 -- start start 2024-04-02T13:36:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f9231d4a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922c100a20 0x7f922c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f9231d4a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f922c101c10 0x7f922c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f9231d4a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f922c1025d0 0x7f922c19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f9231d4a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f922c10d790 con 0x7f922c1025d0 2024-04-02T13:36:26.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f9231d4a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f922c10d610 con 0x7f922c100a20 2024-04-02T13:36:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f9231d4a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f922c10d910 con 0x7f922c101c10 2024-04-02T13:36:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f922bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f922c1025d0 0x7f922c19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f922bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f922c1025d0 0x7f922c19f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53498/0 (socket says 172.21.15.33:53498) 2024-04-02T13:36:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f922bfff640 1 -- 172.21.15.33:0/727467179 learned_addr learned my addr 172.21.15.33:0/727467179 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f922b7fe640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922c100a20 0x7f922c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f922affd640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f922c101c10 0x7f922c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f922bfff640 1 -- 172.21.15.33:0/727467179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f922c101c10 msgr2=0x7f922c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f922bfff640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f922c101c10 0x7f922c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f922bfff640 1 -- 172.21.15.33:0/727467179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922c100a20 msgr2=0x7f922c197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f922bfff640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922c100a20 0x7f922c197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f922bfff640 1 -- 172.21.15.33:0/727467179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922c19fa80 con 0x7f922c1025d0 2024-04-02T13:36:26.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f922b7fe640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922c100a20 0x7f922c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.969+0000 7f922bfff640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f922c1025d0 0x7f922c19f410 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f922400eaa0 tx=0x7f922400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.970+0000 7f9228ff9640 1 -- 172.21.15.33:0/727467179 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922400cdb0 con 0x7f922c1025d0 2024-04-02T13:36:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.970+0000 7f9228ff9640 1 -- 172.21.15.33:0/727467179 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9224005020 con 0x7f922c1025d0 2024-04-02T13:36:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.970+0000 7f9231d4a640 1 -- 172.21.15.33:0/727467179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922c19fd70 con 0x7f922c1025d0 2024-04-02T13:36:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.970+0000 7f9228ff9640 1 -- 172.21.15.33:0/727467179 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240116b0 con 0x7f922c1025d0 2024-04-02T13:36:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.970+0000 7f9231d4a640 1 -- 172.21.15.33:0/727467179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f922c1a0280 con 0x7f922c1025d0 2024-04-02T13:36:26.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.971+0000 7f9231d4a640 1 -- 172.21.15.33:0/727467179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f922c107d90 con 0x7f922c1025d0 2024-04-02T13:36:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.972+0000 7f9228ff9640 1 -- 172.21.15.33:0/727467179 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f92240051d0 con 0x7f922c1025d0 2024-04-02T13:36:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.972+0000 7f9228ff9640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f92140a3c30 0x7f92140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.972+0000 7f922b7fe640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f92140a3c30 0x7f92140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.973+0000 7f9228ff9640 1 -- 172.21.15.33:0/727467179 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f92240d0880 con 0x7f922c1025d0 2024-04-02T13:36:26.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.973+0000 7f922b7fe640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f92140a3c30 0x7f92140a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f921c005c40 tx=0x7f921c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:26.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:26 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/948669454' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-02T13:36:26.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:26 smithi145 ceph-mon[35646]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T13:36:26.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:26 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/911934596' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-02T13:36:26.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:26 smithi145 ceph-mon[35646]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T13:36:26.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:26.977+0000 7f9228ff9640 1 -- 172.21.15.33:0/727467179 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9224016030 con 0x7f922c1025d0 2024-04-02T13:36:27.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.163+0000 7f9231d4a640 1 -- 172.21.15.33:0/727467179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f922c0627a0 con 0x7f922c1025d0 2024-04-02T13:36:27.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.164+0000 7f9228ff9640 1 -- 172.21.15.33:0/727467179 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24057 (secure 0 0 0) 0x7f922409d020 con 0x7f922c1025d0 2024-04-02T13:36:27.168 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:27.168 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:36:25.856437+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:36:27.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.166+0000 7f9231d4a640 1 -- 172.21.15.33:0/727467179 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f92140a3c30 msgr2=0x7f92140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.166+0000 7f9231d4a640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f92140a3c30 0x7f92140a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f921c005c40 tx=0x7f921c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.166+0000 7f9231d4a640 1 -- 172.21.15.33:0/727467179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f922c1025d0 msgr2=0x7f922c19f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.166+0000 7f9231d4a640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f922c1025d0 0x7f922c19f410 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f922400eaa0 tx=0x7f922400ef60 comp rx=0 tx=0).stop 2024-04-02T13:36:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.167+0000 7f9231d4a640 1 -- 172.21.15.33:0/727467179 shutdown_connections 2024-04-02T13:36:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.167+0000 7f9231d4a640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f92140a3c30 0x7f92140a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.167+0000 7f9231d4a640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f922c1025d0 0x7f922c19f410 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.167+0000 7f9231d4a640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f922c101c10 0x7f922c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.167+0000 7f9231d4a640 1 --2- 172.21.15.33:0/727467179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f922c100a20 0x7f922c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.167+0000 7f9231d4a640 1 -- 172.21.15.33:0/727467179 >> 172.21.15.33:0/727467179 conn(0x7f922c0fbfb0 msgr2=0x7f922c104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.167+0000 7f9231d4a640 1 -- 172.21.15.33:0/727467179 shutdown_connections 2024-04-02T13:36:27.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.167+0000 7f9231d4a640 1 -- 172.21.15.33:0/727467179 wait complete. 2024-04-02T13:36:27.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:36:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.282+0000 7f5bc47a8640 1 -- 172.21.15.33:0/3933581028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bbc101c00 msgr2=0x7f5bbc102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.282+0000 7f5bc47a8640 1 --2- 172.21.15.33:0/3933581028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bbc101c00 0x7f5bbc102080 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009900 tx=0x7f5bb002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.282+0000 7f5bc47a8640 1 -- 172.21.15.33:0/3933581028 shutdown_connections 2024-04-02T13:36:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.282+0000 7f5bc47a8640 1 --2- 172.21.15.33:0/3933581028 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bbc1025c0 0x7f5bbc108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.282+0000 7f5bc47a8640 1 --2- 172.21.15.33:0/3933581028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bbc101c00 0x7f5bbc102080 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.282+0000 7f5bc47a8640 1 --2- 172.21.15.33:0/3933581028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bbc100a10 0x7f5bbc100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.282+0000 7f5bc47a8640 1 -- 172.21.15.33:0/3933581028 >> 172.21.15.33:0/3933581028 conn(0x7f5bbc0fbf80 msgr2=0x7f5bbc0fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.282+0000 7f5bc47a8640 1 -- 172.21.15.33:0/3933581028 shutdown_connections 2024-04-02T13:36:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.282+0000 7f5bc47a8640 1 -- 172.21.15.33:0/3933581028 wait complete. 2024-04-02T13:36:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.282+0000 7f5bc47a8640 1 Processor -- start 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc47a8640 1 -- start start 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc47a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bbc100a10 0x7f5bbc196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc47a8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bbc101c00 0x7f5bbc197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc47a8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bbc1025c0 0x7f5bbc19e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc47a8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc10c670 con 0x7f5bbc100a10 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc47a8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc10c4f0 con 0x7f5bbc1025c0 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc47a8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc10c7f0 con 0x7f5bbc101c00 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc2544640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bbc100a10 0x7f5bbc196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc2544640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bbc100a10 0x7f5bbc196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53512/0 (socket says 172.21.15.33:53512) 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc2544640 1 -- 172.21.15.33:0/1665146048 learned_addr learned my addr 172.21.15.33:0/1665146048 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc1d43640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bbc101c00 0x7f5bbc197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc2d45640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bbc1025c0 0x7f5bbc19e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc2544640 1 -- 172.21.15.33:0/1665146048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bbc101c00 msgr2=0x7f5bbc197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc2544640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bbc101c00 0x7f5bbc197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc2544640 1 -- 172.21.15.33:0/1665146048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bbc1025c0 msgr2=0x7f5bbc19e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc2544640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bbc1025c0 0x7f5bbc19e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc2544640 1 -- 172.21.15.33:0/1665146048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc19e940 con 0x7f5bbc100a10 2024-04-02T13:36:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc2544640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bbc100a10 0x7f5bbc196d30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400c970 tx=0x7f5bb400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bbb7fe640 1 -- 172.21.15.33:0/1665146048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4004220 con 0x7f5bbc100a10 2024-04-02T13:36:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bbb7fe640 1 -- 172.21.15.33:0/1665146048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5bb40043c0 con 0x7f5bbc100a10 2024-04-02T13:36:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc47a8640 1 -- 172.21.15.33:0/1665146048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bbc19ec30 con 0x7f5bbc100a10 2024-04-02T13:36:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bbb7fe640 1 -- 172.21.15.33:0/1665146048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400f450 con 0x7f5bbc100a10 2024-04-02T13:36:27.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.283+0000 7f5bc47a8640 1 -- 172.21.15.33:0/1665146048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bbc19f180 con 0x7f5bbc100a10 2024-04-02T13:36:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.285+0000 7f5bc47a8640 1 -- 172.21.15.33:0/1665146048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bbc107d30 con 0x7f5bbc100a10 2024-04-02T13:36:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.285+0000 7f5bbb7fe640 1 -- 172.21.15.33:0/1665146048 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5bb40079b0 con 0x7f5bbc100a10 2024-04-02T13:36:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.286+0000 7f5bbb7fe640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5ba40a3bb0 0x7f5ba40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.286+0000 7f5bc1d43640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5ba40a3bb0 0x7f5ba40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.286+0000 7f5bbb7fe640 1 -- 172.21.15.33:0/1665146048 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5bb40d04b0 con 0x7f5bbc100a10 2024-04-02T13:36:27.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.287+0000 7f5bc1d43640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5ba40a3bb0 0x7f5ba40a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005ab0 tx=0x7f5bb0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:27.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.291+0000 7f5bbb7fe640 1 -- 172.21.15.33:0/1665146048 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5bb4014030 con 0x7f5bbc100a10 2024-04-02T13:36:27.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.479+0000 7f5bc47a8640 1 -- 172.21.15.33:0/1665146048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bbc0627a0 con 0x7f5bbc100a10 2024-04-02T13:36:27.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.481+0000 7f5bbb7fe640 1 -- 172.21.15.33:0/1665146048 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24057 (secure 0 0 0) 0x7f5bb409cbf0 con 0x7f5bbc100a10 2024-04-02T13:36:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:27.484 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:36:25.856437+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:36:27.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.483+0000 7f5bc47a8640 1 -- 172.21.15.33:0/1665146048 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5ba40a3bb0 msgr2=0x7f5ba40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.483+0000 7f5bc47a8640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5ba40a3bb0 0x7f5ba40a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005ab0 tx=0x7f5bb0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.483+0000 7f5bc47a8640 1 -- 172.21.15.33:0/1665146048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bbc100a10 msgr2=0x7f5bbc196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.483+0000 7f5bc47a8640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bbc100a10 0x7f5bbc196d30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400c970 tx=0x7f5bb400ce30 comp rx=0 tx=0).stop 2024-04-02T13:36:27.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.483+0000 7f5bc47a8640 1 -- 172.21.15.33:0/1665146048 shutdown_connections 2024-04-02T13:36:27.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.483+0000 7f5bc47a8640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5ba40a3bb0 0x7f5ba40a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.483+0000 7f5bc47a8640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bbc1025c0 0x7f5bbc19e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.483+0000 7f5bc47a8640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bbc101c00 0x7f5bbc197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.483+0000 7f5bc47a8640 1 --2- 172.21.15.33:0/1665146048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bbc100a10 0x7f5bbc196d30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.483+0000 7f5bc47a8640 1 -- 172.21.15.33:0/1665146048 >> 172.21.15.33:0/1665146048 conn(0x7f5bbc0fbf80 msgr2=0x7f5bbc104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:27.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.483+0000 7f5bc47a8640 1 -- 172.21.15.33:0/1665146048 shutdown_connections 2024-04-02T13:36:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.483+0000 7f5bc47a8640 1 -- 172.21.15.33:0/1665146048 wait complete. 2024-04-02T13:36:27.512 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:27.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.612+0000 7f578c055640 1 -- 172.21.15.33:0/3882155697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5784100d10 msgr2=0x7f5784101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.612+0000 7f578c055640 1 --2- 172.21.15.33:0/3882155697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5784100d10 0x7f5784101190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5774009900 tx=0x7f577402bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:27.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.612+0000 7f578c055640 1 -- 172.21.15.33:0/3882155697 shutdown_connections 2024-04-02T13:36:27.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.612+0000 7f578c055640 1 --2- 172.21.15.33:0/3882155697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57841016d0 0x7f5784108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.612+0000 7f578c055640 1 --2- 172.21.15.33:0/3882155697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5784100d10 0x7f5784101190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.612+0000 7f578c055640 1 --2- 172.21.15.33:0/3882155697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5784101fe0 0x7f5784102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.612+0000 7f578c055640 1 -- 172.21.15.33:0/3882155697 >> 172.21.15.33:0/3882155697 conn(0x7f57840fb740 msgr2=0x7f57840fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.612+0000 7f578c055640 1 -- 172.21.15.33:0/3882155697 shutdown_connections 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.612+0000 7f578c055640 1 -- 172.21.15.33:0/3882155697 wait complete. 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f578c055640 1 Processor -- start 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f578c055640 1 -- start start 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f578c055640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5784100d10 0x7f5784197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f578c055640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57841016d0 0x7f5784198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f578c055640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5784101fe0 0x7f578419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f578c055640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f578410c6c0 con 0x7f57841016d0 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f578c055640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f578410c540 con 0x7f5784100d10 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f578c055640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f578410c840 con 0x7f5784101fe0 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f57895f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57841016d0 0x7f5784198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f57895f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57841016d0 0x7f5784198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53536/0 (socket says 172.21.15.33:53536) 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f57895f0640 1 -- 172.21.15.33:0/329174097 learned_addr learned my addr 172.21.15.33:0/329174097 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f578a5f2640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5784101fe0 0x7f578419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f5789df1640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5784100d10 0x7f5784197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f57895f0640 1 -- 172.21.15.33:0/329174097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5784101fe0 msgr2=0x7f578419f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f57895f0640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5784101fe0 0x7f578419f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f57895f0640 1 -- 172.21.15.33:0/329174097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5784100d10 msgr2=0x7f5784197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f57895f0640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5784100d10 0x7f5784197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f57895f0640 1 -- 172.21.15.33:0/329174097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578419fb20 con 0x7f57841016d0 2024-04-02T13:36:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f57895f0640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57841016d0 0x7f5784198440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5774009230 tx=0x7f5774005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.613+0000 7f577affd640 1 -- 172.21.15.33:0/329174097 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774043020 con 0x7f57841016d0 2024-04-02T13:36:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.614+0000 7f578c055640 1 -- 172.21.15.33:0/329174097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578419fdb0 con 0x7f57841016d0 2024-04-02T13:36:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.614+0000 7f577affd640 1 -- 172.21.15.33:0/329174097 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f57740054f0 con 0x7f57841016d0 2024-04-02T13:36:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.614+0000 7f577affd640 1 -- 172.21.15.33:0/329174097 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774003cd0 con 0x7f57841016d0 2024-04-02T13:36:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.614+0000 7f578c055640 1 -- 172.21.15.33:0/329174097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57841a0280 con 0x7f57841016d0 2024-04-02T13:36:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.615+0000 7f578c055640 1 -- 172.21.15.33:0/329174097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57841a04b0 con 0x7f57841016d0 2024-04-02T13:36:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.616+0000 7f577affd640 1 -- 172.21.15.33:0/329174097 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5774046020 con 0x7f57841016d0 2024-04-02T13:36:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.616+0000 7f577affd640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f57700a3bb0 0x7f57700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.616+0000 7f5789df1640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f57700a3bb0 0x7f57700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.616+0000 7f577affd640 1 -- 172.21.15.33:0/329174097 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f57740f2a10 con 0x7f57841016d0 2024-04-02T13:36:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.617+0000 7f5789df1640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f57700a3bb0 0x7f57700a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5780005fd0 tx=0x7f5780005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:27.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.620+0000 7f577affd640 1 -- 172.21.15.33:0/329174097 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f577404d070 con 0x7f57841016d0 2024-04-02T13:36:27.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:27 smithi145 ceph-mon[35646]: pgmap v294: 129 pgs: 17 creating+peering, 93 active+clean, 19 unknown; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-02T13:36:27.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:27 smithi145 ceph-mon[35646]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:36:27.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:27 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/911934596' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-02T13:36:27.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:27 smithi145 ceph-mon[35646]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:27.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:27 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/166739439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:27.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:27 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/727467179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:27.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:27 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1665146048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:27.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:27 smithi050 ceph-mon[35524]: pgmap v294: 129 pgs: 17 creating+peering, 93 active+clean, 19 unknown; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-02T13:36:27.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:27 smithi050 ceph-mon[35524]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:36:27.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:27 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/911934596' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-02T13:36:27.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:27 smithi050 ceph-mon[35524]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:27.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:27 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/166739439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:27.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:27 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/727467179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:27.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:27 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1665146048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:27.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:27 smithi033 ceph-mon[32390]: pgmap v294: 129 pgs: 17 creating+peering, 93 active+clean, 19 unknown; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-02T13:36:27.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:27 smithi033 ceph-mon[32390]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:36:27.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:27 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/911934596' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-02T13:36:27.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:27 smithi033 ceph-mon[32390]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:27.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:27 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/166739439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:27.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:27 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/727467179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:27.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:27 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1665146048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:27.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.848+0000 7f578c055640 1 -- 172.21.15.33:0/329174097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57841a07a0 con 0x7f57841016d0 2024-04-02T13:36:27.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.848+0000 7f577affd640 1 -- 172.21.15.33:0/329174097 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11800 (secure 0 0 0) 0x7f57740bf1b0 con 0x7f57841016d0 2024-04-02T13:36:27.851 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:27.851 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:36:25.856615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24551,"qdb_cluster":[24551]},"id":1}]} 2024-04-02T13:36:27.851 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-04-02T13:36:27.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.850+0000 7f578c055640 1 -- 172.21.15.33:0/329174097 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f57700a3bb0 msgr2=0x7f57700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.850+0000 7f578c055640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f57700a3bb0 0x7f57700a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5780005fd0 tx=0x7f5780005850 comp rx=0 tx=0).stop 2024-04-02T13:36:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.850+0000 7f578c055640 1 -- 172.21.15.33:0/329174097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57841016d0 msgr2=0x7f5784198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.850+0000 7f578c055640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57841016d0 0x7f5784198440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5774009230 tx=0x7f5774005e40 comp rx=0 tx=0).stop 2024-04-02T13:36:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.851+0000 7f578c055640 1 -- 172.21.15.33:0/329174097 shutdown_connections 2024-04-02T13:36:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.851+0000 7f578c055640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f57700a3bb0 0x7f57700a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.851+0000 7f578c055640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5784101fe0 0x7f578419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.851+0000 7f578c055640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57841016d0 0x7f5784198440 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.851+0000 7f578c055640 1 --2- 172.21.15.33:0/329174097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5784100d10 0x7f5784197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.851+0000 7f578c055640 1 -- 172.21.15.33:0/329174097 >> 172.21.15.33:0/329174097 conn(0x7f57840fb740 msgr2=0x7f57840fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.851+0000 7f578c055640 1 -- 172.21.15.33:0/329174097 shutdown_connections 2024-04-02T13:36:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.851+0000 7f578c055640 1 -- 172.21.15.33:0/329174097 wait complete. 2024-04-02T13:36:27.864 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:36:27.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.965+0000 7f2eb422b640 1 -- 172.21.15.33:0/248020943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eac101b20 msgr2=0x7f2eac101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.965+0000 7f2eb422b640 1 --2- 172.21.15.33:0/248020943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eac101b20 0x7f2eac101f40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4009900 tx=0x7f2ea402bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:27.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.965+0000 7f2eb422b640 1 -- 172.21.15.33:0/248020943 shutdown_connections 2024-04-02T13:36:27.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.965+0000 7f2eb422b640 1 --2- 172.21.15.33:0/248020943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eac1036d0 0x7f2eac109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.965+0000 7f2eb422b640 1 --2- 172.21.15.33:0/248020943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2eac102d10 0x7f2eac103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.965+0000 7f2eb422b640 1 --2- 172.21.15.33:0/248020943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eac101b20 0x7f2eac101f40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.965+0000 7f2eb422b640 1 -- 172.21.15.33:0/248020943 >> 172.21.15.33:0/248020943 conn(0x7f2eac0fd0b0 msgr2=0x7f2eac0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:27.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.965+0000 7f2eb422b640 1 -- 172.21.15.33:0/248020943 shutdown_connections 2024-04-02T13:36:27.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.965+0000 7f2eb422b640 1 -- 172.21.15.33:0/248020943 wait complete. 2024-04-02T13:36:27.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.965+0000 7f2eb422b640 1 Processor -- start 2024-04-02T13:36:27.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb422b640 1 -- start start 2024-04-02T13:36:27.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb422b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eac101b20 0x7f2eac0758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb422b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2eac102d10 0x7f2eac073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb422b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eac1036d0 0x7f2eac074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb422b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac10d780 con 0x7f2eac1036d0 2024-04-02T13:36:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb422b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac10d600 con 0x7f2eac101b20 2024-04-02T13:36:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb422b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac10d900 con 0x7f2eac102d10 2024-04-02T13:36:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb27c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eac1036d0 0x7f2eac074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb27c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eac1036d0 0x7f2eac074430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53564/0 (socket says 172.21.15.33:53564) 2024-04-02T13:36:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb27c8640 1 -- 172.21.15.33:0/27915236 learned_addr learned my addr 172.21.15.33:0/27915236 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb17c6640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2eac102d10 0x7f2eac073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb1fc7640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eac101b20 0x7f2eac0758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb27c8640 1 -- 172.21.15.33:0/27915236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2eac102d10 msgr2=0x7f2eac073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb27c8640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2eac102d10 0x7f2eac073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb27c8640 1 -- 172.21.15.33:0/27915236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eac101b20 msgr2=0x7f2eac0758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb27c8640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eac101b20 0x7f2eac0758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb27c8640 1 -- 172.21.15.33:0/27915236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eac074cc0 con 0x7f2eac1036d0 2024-04-02T13:36:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb17c6640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2eac102d10 0x7f2eac073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb27c8640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eac1036d0 0x7f2eac074430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009fd0 tx=0x7f2e9c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:27.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eaaffd640 1 -- 172.21.15.33:0/27915236 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c017070 con 0x7f2eac1036d0 2024-04-02T13:36:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eaaffd640 1 -- 172.21.15.33:0/27915236 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2e9c003e40 con 0x7f2eac1036d0 2024-04-02T13:36:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eb422b640 1 -- 172.21.15.33:0/27915236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eac1a1fa0 con 0x7f2eac1036d0 2024-04-02T13:36:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.966+0000 7f2eaaffd640 1 -- 172.21.15.33:0/27915236 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0106c0 con 0x7f2eac1036d0 2024-04-02T13:36:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.967+0000 7f2eb422b640 1 -- 172.21.15.33:0/27915236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eac1a23b0 con 0x7f2eac1036d0 2024-04-02T13:36:27.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.967+0000 7f2eb422b640 1 -- 172.21.15.33:0/27915236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eac108ec0 con 0x7f2eac1036d0 2024-04-02T13:36:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.968+0000 7f2eaaffd640 1 -- 172.21.15.33:0/27915236 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2e9c003970 con 0x7f2eac1036d0 2024-04-02T13:36:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.969+0000 7f2eaaffd640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2e940a3bb0 0x7f2e940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.969+0000 7f2eaaffd640 1 -- 172.21.15.33:0/27915236 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2e9c0d0340 con 0x7f2eac1036d0 2024-04-02T13:36:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.969+0000 7f2eb1fc7640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2e940a3bb0 0x7f2e940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.969+0000 7f2eb1fc7640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2e940a3bb0 0x7f2e940a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4000c00 tx=0x7f2ea4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:27.974+0000 7f2eaaffd640 1 -- 172.21.15.33:0/27915236 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2e9c003c60 con 0x7f2eac1036d0 2024-04-02T13:36:28.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.160+0000 7f2eb422b640 1 -- 172.21.15.33:0/27915236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2eac0627a0 con 0x7f2eac1036d0 2024-04-02T13:36:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.162+0000 7f2eaaffd640 1 -- 172.21.15.33:0/27915236 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24057 (secure 0 0 0) 0x7f2e9c09ca80 con 0x7f2eac1036d0 2024-04-02T13:36:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:28.165 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:36:25.856437+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:36:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.164+0000 7f2eb422b640 1 -- 172.21.15.33:0/27915236 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2e940a3bb0 msgr2=0x7f2e940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.164+0000 7f2eb422b640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2e940a3bb0 0x7f2e940a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4000c00 tx=0x7f2ea4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.164+0000 7f2eb422b640 1 -- 172.21.15.33:0/27915236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eac1036d0 msgr2=0x7f2eac074430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.164+0000 7f2eb422b640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eac1036d0 0x7f2eac074430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009fd0 tx=0x7f2e9c00b260 comp rx=0 tx=0).stop 2024-04-02T13:36:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.164+0000 7f2eb422b640 1 -- 172.21.15.33:0/27915236 shutdown_connections 2024-04-02T13:36:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.164+0000 7f2eb422b640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2e940a3bb0 0x7f2e940a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.164+0000 7f2eb422b640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2eac1036d0 0x7f2eac074430 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.164+0000 7f2eb422b640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2eac102d10 0x7f2eac073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.164+0000 7f2eb422b640 1 --2- 172.21.15.33:0/27915236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eac101b20 0x7f2eac0758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.164+0000 7f2eb422b640 1 -- 172.21.15.33:0/27915236 >> 172.21.15.33:0/27915236 conn(0x7f2eac0fd0b0 msgr2=0x7f2eac106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.164+0000 7f2eb422b640 1 -- 172.21.15.33:0/27915236 shutdown_connections 2024-04-02T13:36:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.164+0000 7f2eb422b640 1 -- 172.21.15.33:0/27915236 wait complete. 2024-04-02T13:36:28.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-02T13:36:21.184650+0000', 'modified': '2024-04-02T13:36:25.856615+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24551}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24551': {'gid': 24551, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6839/932333028', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6838', 'nonce': 932333028}, {'type': 'v1', 'addr': '172.21.15.50:6839', 'nonce': 932333028}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24551, 'qdb_cluster': [24551]} 2024-04-02T13:36:28.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-02T13:36:28.179 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T13:36:28.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.280+0000 7f878e915640 1 -- 172.21.15.33:0/2123419019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87881025c0 msgr2=0x7f8788108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.280+0000 7f878e915640 1 --2- 172.21.15.33:0/2123419019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87881025c0 0x7f8788108e50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8780009ca0 tx=0x7f878002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.280+0000 7f878e915640 1 -- 172.21.15.33:0/2123419019 shutdown_connections 2024-04-02T13:36:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.280+0000 7f878e915640 1 --2- 172.21.15.33:0/2123419019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87881025c0 0x7f8788108e50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.280+0000 7f878e915640 1 --2- 172.21.15.33:0/2123419019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8788101c00 0x7f8788102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.280+0000 7f878e915640 1 --2- 172.21.15.33:0/2123419019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8788100a10 0x7f8788100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.280+0000 7f878e915640 1 -- 172.21.15.33:0/2123419019 >> 172.21.15.33:0/2123419019 conn(0x7f87880fbfa0 msgr2=0x7f87880fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.280+0000 7f878e915640 1 -- 172.21.15.33:0/2123419019 shutdown_connections 2024-04-02T13:36:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.280+0000 7f878e915640 1 -- 172.21.15.33:0/2123419019 wait complete. 2024-04-02T13:36:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.280+0000 7f878e915640 1 Processor -- start 2024-04-02T13:36:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f878e915640 1 -- start start 2024-04-02T13:36:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f878e915640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8788100a10 0x7f87881049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f878e915640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788101c00 0x7f8788102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f878e915640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87881025c0 0x7f8788103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f878e915640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f878810c670 con 0x7f8788101c00 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f878e915640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f878810c4f0 con 0x7f87881025c0 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f878e915640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f878810c7f0 con 0x7f8788100a10 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f87877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788101c00 0x7f8788102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f87877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788101c00 0x7f8788102fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53584/0 (socket says 172.21.15.33:53584) 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f87877fe640 1 -- 172.21.15.33:0/919298827 learned_addr learned my addr 172.21.15.33:0/919298827 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f8787fff640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8788100a10 0x7f87881049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f878ceb2640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87881025c0 0x7f8788103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f87877fe640 1 -- 172.21.15.33:0/919298827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8788100a10 msgr2=0x7f87881049a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f87877fe640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8788100a10 0x7f87881049a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f87877fe640 1 -- 172.21.15.33:0/919298827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87881025c0 msgr2=0x7f8788103520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f87877fe640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87881025c0 0x7f8788103520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f87877fe640 1 -- 172.21.15.33:0/919298827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8788103db0 con 0x7f8788101c00 2024-04-02T13:36:28.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.281+0000 7f87877fe640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788101c00 0x7f8788102fe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f877c00d950 tx=0x7f877c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:28.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.282+0000 7f87857fa640 1 -- 172.21.15.33:0/919298827 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c017070 con 0x7f8788101c00 2024-04-02T13:36:28.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.282+0000 7f878e915640 1 -- 172.21.15.33:0/919298827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87881a1f00 con 0x7f8788101c00 2024-04-02T13:36:28.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.282+0000 7f87857fa640 1 -- 172.21.15.33:0/919298827 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f877c009c30 con 0x7f8788101c00 2024-04-02T13:36:28.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.282+0000 7f87857fa640 1 -- 172.21.15.33:0/919298827 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c00f6b0 con 0x7f8788101c00 2024-04-02T13:36:28.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.282+0000 7f878e915640 1 -- 172.21.15.33:0/919298827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87881a2360 con 0x7f8788101c00 2024-04-02T13:36:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f87857fa640 1 -- 172.21.15.33:0/919298827 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f877c010a50 con 0x7f8788101c00 2024-04-02T13:36:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f87915d8740 1 Processor -- start 2024-04-02T13:36:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f87915d8740 1 -- start start 2024-04-02T13:36:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f87915d8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ae0 0x564df2616f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f87915d8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df26174d0 0x564df261bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f87915d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 0x564df261ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f87915d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564df257b060 con 0x564df261c640 2024-04-02T13:36:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f87915d8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564df2523650 con 0x564df26174d0 2024-04-02T13:36:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f87915d8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564df2577010 con 0x564df2616ae0 2024-04-02T13:36:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f87877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 0x564df261ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f87877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 0x564df261ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53600/0 (socket says 172.21.15.33:53600) 2024-04-02T13:36:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f87877fe640 1 -- 172.21.15.33:0/3769676866 learned_addr learned my addr 172.21.15.33:0/3769676866 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f8787fff640 1 --2- 172.21.15.33:0/3769676866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df26174d0 0x564df261bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.284+0000 7f87857fa640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f87700a3bb0 0x7f87700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f878ceb2640 1 --2- 172.21.15.33:0/3769676866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ae0 0x564df2616f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f878ceb2640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f87700a3bb0 0x7f87700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f87857fa640 1 -- 172.21.15.33:0/919298827 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f877c0d00a0 con 0x7f8788101c00 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f87877fe640 1 -- 172.21.15.33:0/3769676866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ae0 msgr2=0x564df2616f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f87877fe640 1 --2- 172.21.15.33:0/3769676866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ae0 0x564df2616f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f87877fe640 1 -- 172.21.15.33:0/3769676866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df26174d0 msgr2=0x564df261bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f87877fe640 1 --2- 172.21.15.33:0/3769676866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df26174d0 0x564df261bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f87877fe640 1 -- 172.21.15.33:0/3769676866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564df228f2d0 con 0x564df261c640 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f8787fff640 1 --2- 172.21.15.33:0/3769676866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df26174d0 0x564df261bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f87877fe640 1 --2- 172.21.15.33:0/3769676866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 0x564df261ead0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f877c0a4880 tx=0x7f877c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d405eaa1d0ca22cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f878ceb2640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f87700a3bb0 0x7f87700a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8780005e30 tx=0x7f87800325d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f876e7fc640 1 -- 172.21.15.33:0/3769676866 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c020070 con 0x564df261c640 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f876e7fc640 1 -- 172.21.15.33:0/3769676866 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f877c009de0 con 0x564df261c640 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.285+0000 7f876e7fc640 1 -- 172.21.15.33:0/3769676866 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c0d2370 con 0x564df261c640 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.286+0000 7f87915d8740 1 -- 172.21.15.33:0/3769676866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 msgr2=0x564df261ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.286+0000 7f87915d8740 1 --2- 172.21.15.33:0/3769676866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 0x564df261ead0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f877c0a4880 tx=0x7f877c000bc0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.286+0000 7f87915d8740 1 -- 172.21.15.33:0/3769676866 shutdown_connections 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.286+0000 7f87915d8740 1 --2- 172.21.15.33:0/3769676866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 0x564df261ead0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.286+0000 7f87915d8740 1 --2- 172.21.15.33:0/3769676866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df26174d0 0x564df261bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.286+0000 7f87915d8740 1 --2- 172.21.15.33:0/3769676866 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ae0 0x564df2616f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.286+0000 7f87915d8740 1 -- 172.21.15.33:0/3769676866 >> 172.21.15.33:0/3769676866 conn(0x564df2612050 msgr2=0x564df2612450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.286+0000 7f87915d8740 1 -- 172.21.15.33:0/3769676866 shutdown_connections 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.286+0000 7f87915d8740 1 -- 172.21.15.33:0/3769676866 wait complete. 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.286+0000 7f87915d8740 1 -- start start 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.286+0000 7f87915d8740 1 Processor -- start 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87915d8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ae0 0x564df2620ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87915d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df26174d0 0x564df2621530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87915d8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df261c640 0x564df2628570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87915d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564df261f590 con 0x564df26174d0 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87915d8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564df2536a10 con 0x564df2616ae0 2024-04-02T13:36:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87915d8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564df261f710 con 0x564df261c640 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87915d8740 10 client.0 _ll_register_callbacks cb 0x564df25730b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df26174d0 0x564df2621530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df26174d0 0x564df2621530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53602/0 (socket says 172.21.15.33:53602) 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87877fe640 1 -- 172.21.15.33:0/1370689709 learned_addr learned my addr 172.21.15.33:0/1370689709 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f878ceb2640 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df261c640 0x564df2628570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f8787fff640 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ae0 0x564df2620ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87877fe640 1 -- 172.21.15.33:0/1370689709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df261c640 msgr2=0x564df2628570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87877fe640 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df261c640 0x564df2628570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87877fe640 1 -- 172.21.15.33:0/1370689709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ae0 msgr2=0x564df2620ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87877fe640 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ae0 0x564df2620ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87877fe640 1 -- 172.21.15.33:0/1370689709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564df23596b0 con 0x564df26174d0 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f8787fff640 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ae0 0x564df2620ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87877fe640 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df26174d0 0x564df2621530 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f877c0cd9b0 tx=0x7f877c098b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c82e7469a57ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87677fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:36:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87677fe640 1 -- 172.21.15.33:0/1370689709 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c020070 con 0x564df26174d0 2024-04-02T13:36:28.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87677fe640 1 -- 172.21.15.33:0/1370689709 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f877c09b640 con 0x564df26174d0 2024-04-02T13:36:28.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87915d8740 1 -- 172.21.15.33:0/1370689709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f877c009de0 con 0x564df26174d0 2024-04-02T13:36:28.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87677fe640 1 -- 172.21.15.33:0/1370689709 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f877c09b840 con 0x564df26174d0 2024-04-02T13:36:28.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87915d8740 10 client.14751 fetch_fsmap learned FSMap version 17 2024-04-02T13:36:28.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.287+0000 7f87915d8740 1 -- 172.21.15.33:0/1370689709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x564df262b8e0 con 0x564df26174d0 2024-04-02T13:36:28.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.288+0000 7f87677fe640 1 -- 172.21.15.33:0/1370689709 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f877c009de0 con 0x564df26174d0 2024-04-02T13:36:28.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.288+0000 7f87677fe640 1 -- 172.21.15.33:0/1370689709 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f877c0a1680 con 0x564df26174d0 2024-04-02T13:36:28.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.288+0000 7f87915d8740 10 client.14751 fetch_fsmap finished waiting for FSMap version 17 2024-04-02T13:36:28.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.288+0000 7f87915d8740 10 client.14751 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:36:28.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.288+0000 7f87915d8740 20 client.14751 populate_metadata read hostname 'smithi033' 2024-04-02T13:36:28.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.288+0000 7f87915d8740 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564df262d9e0 0x564df262ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.288+0000 7f87915d8740 4 client.14751 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:28.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.288+0000 7f87915d8740 1 -- 172.21.15.33:0/1370689709 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564df2522cf0 con 0x564df262d9e0 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.288+0000 7f8787fff640 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564df262d9e0 0x564df262ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.289+0000 7f87677fe640 10 client.14751 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.289+0000 7f8787fff640 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564df262d9e0 0x564df262ee50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5023664547b9b5a server_cookie=2401d24dd7251b22 in_seq=0 out_seq=0 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87677fe640 1 -- 172.21.15.33:0/1370689709 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564df2522cf0 con 0x564df262d9e0 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87677fe640 10 client.14751 handle_command_reply: tid=0 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87915d8740 1 client.14751 shutdown 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87915d8740 1 -- 172.21.15.33:0/1370689709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df26174d0 msgr2=0x564df2621530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87915d8740 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df26174d0 0x564df2621530 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f877c0cd9b0 tx=0x7f877c098b30 comp rx=0 tx=0).stop 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87915d8740 1 -- 172.21.15.33:0/1370689709 shutdown_connections 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87915d8740 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564df262d9e0 0x564df262ee50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87915d8740 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df261c640 0x564df2628570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87915d8740 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df26174d0 0x564df2621530 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87677fe640 0 client.14751 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87915d8740 1 --2- 172.21.15.33:0/1370689709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ae0 0x564df2620ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87915d8740 1 -- 172.21.15.33:0/1370689709 >> 172.21.15.33:0/1370689709 conn(0x564df2612050 msgr2=0x564df2611190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87915d8740 1 -- 172.21.15.33:0/1370689709 shutdown_connections 2024-04-02T13:36:28.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87915d8740 1 -- 172.21.15.33:0/1370689709 wait complete. 2024-04-02T13:36:28.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.290+0000 7f87915d8740 20 client.14751 trim_cache size 0 max 16384 2024-04-02T13:36:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.308+0000 7f87915d8740 1 Processor -- start 2024-04-02T13:36:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.308+0000 7f87915d8740 1 -- start start 2024-04-02T13:36:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.308+0000 7f87915d8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ae0 0x564df262cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.308+0000 7f87915d8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df26174d0 0x564df2628530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.308+0000 7f87915d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 0x564df2611010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.308+0000 7f87915d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564df262d190 con 0x564df261c640 2024-04-02T13:36:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.308+0000 7f87915d8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564df262d860 con 0x564df26174d0 2024-04-02T13:36:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.308+0000 7f87915d8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564df262df30 con 0x564df2616ae0 2024-04-02T13:36:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.308+0000 7f878ceb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 0x564df2611010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f878ceb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 0x564df2611010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53604/0 (socket says 172.21.15.33:53604) 2024-04-02T13:36:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f878ceb2640 1 -- 172.21.15.33:0/3526829828 learned_addr learned my addr 172.21.15.33:0/3526829828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f8787fff640 1 --2- 172.21.15.33:0/3526829828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ae0 0x564df262cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f87877fe640 1 --2- 172.21.15.33:0/3526829828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df26174d0 0x564df2628530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f878ceb2640 1 -- 172.21.15.33:0/3526829828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ae0 msgr2=0x564df262cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f878ceb2640 1 --2- 172.21.15.33:0/3526829828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ae0 0x564df262cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f878ceb2640 1 -- 172.21.15.33:0/3526829828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df26174d0 msgr2=0x564df2628530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f878ceb2640 1 --2- 172.21.15.33:0/3526829828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df26174d0 0x564df2628530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f878ceb2640 1 -- 172.21.15.33:0/3526829828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564df261bca0 con 0x564df261c640 2024-04-02T13:36:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f8787fff640 1 --2- 172.21.15.33:0/3526829828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ae0 0x564df262cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f87877fe640 1 --2- 172.21.15.33:0/3526829828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df26174d0 0x564df2628530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f878ceb2640 1 --2- 172.21.15.33:0/3526829828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 0x564df2611010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8780009ca0 tx=0x7f878003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f0f54b2f2d45fb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f876dffb640 1 -- 172.21.15.33:0/3526829828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780055070 con 0x564df261c640 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f876dffb640 1 -- 172.21.15.33:0/3526829828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8780048040 con 0x564df261c640 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.309+0000 7f876dffb640 1 -- 172.21.15.33:0/3526829828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780039030 con 0x564df261c640 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 -- 172.21.15.33:0/3526829828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 msgr2=0x564df2611010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 --2- 172.21.15.33:0/3526829828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 0x564df2611010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8780009ca0 tx=0x7f878003b000 comp rx=0 tx=0).stop 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 -- 172.21.15.33:0/3526829828 shutdown_connections 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 --2- 172.21.15.33:0/3526829828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df261c640 0x564df2611010 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 --2- 172.21.15.33:0/3526829828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df26174d0 0x564df2628530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 --2- 172.21.15.33:0/3526829828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ae0 0x564df262cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 -- 172.21.15.33:0/3526829828 >> 172.21.15.33:0/3526829828 conn(0x564df2612050 msgr2=0x564df2610900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 -- 172.21.15.33:0/3526829828 shutdown_connections 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 -- 172.21.15.33:0/3526829828 wait complete. 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 -- start start 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 Processor -- start 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ae0 0x564df2627d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df26174d0 0x564df2628280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df261c640 0x564df2624830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564df262ecd0 con 0x564df2616ae0 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564df262e600 con 0x564df261c640 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.310+0000 7f87915d8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564df262f3a0 con 0x564df26174d0 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f87915d8740 10 client.0 _ll_register_callbacks cb 0x7f8778004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f8787fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ae0 0x564df2627d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f8787fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ae0 0x564df2627d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53612/0 (socket says 172.21.15.33:53612) 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f8787fff640 1 -- 172.21.15.33:0/2579847443 learned_addr learned my addr 172.21.15.33:0/2579847443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f87877fe640 1 --2- 172.21.15.33:0/2579847443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df26174d0 0x564df2628280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f878ceb2640 1 --2- 172.21.15.33:0/2579847443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df261c640 0x564df2624830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f8787fff640 1 -- 172.21.15.33:0/2579847443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df26174d0 msgr2=0x564df2628280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f8787fff640 1 --2- 172.21.15.33:0/2579847443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df26174d0 0x564df2628280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f8787fff640 1 -- 172.21.15.33:0/2579847443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df261c640 msgr2=0x564df2624830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f8787fff640 1 --2- 172.21.15.33:0/2579847443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df261c640 0x564df2624830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f8787fff640 1 -- 172.21.15.33:0/2579847443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564df2624e20 con 0x564df2616ae0 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f8787fff640 1 --2- 172.21.15.33:0/2579847443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ae0 0x564df2627d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x564df2617f40 tx=0x7f877800bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6050e47d3db4e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f87677fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f87677fe640 1 -- 172.21.15.33:0/2579847443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778003a00 con 0x564df2616ae0 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f87677fe640 1 -- 172.21.15.33:0/2579847443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8778003ba0 con 0x564df2616ae0 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f87915d8740 1 -- 172.21.15.33:0/2579847443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8780048040 con 0x564df2616ae0 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f87677fe640 1 -- 172.21.15.33:0/2579847443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f877802f050 con 0x564df2616ae0 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f87915d8740 10 client.14757 fetch_fsmap learned FSMap version 17 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.311+0000 7f87915d8740 1 -- 172.21.15.33:0/2579847443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x564df2694510 con 0x564df2616ae0 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.312+0000 7f87677fe640 1 -- 172.21.15.33:0/2579847443 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8780048040 con 0x564df2616ae0 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.312+0000 7f87677fe640 1 -- 172.21.15.33:0/2579847443 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f877801d400 con 0x564df2616ae0 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.312+0000 7f87915d8740 10 client.14757 fetch_fsmap finished waiting for FSMap version 17 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.312+0000 7f87915d8740 10 client.14757 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.312+0000 7f87915d8740 20 client.14757 populate_metadata read hostname 'smithi033' 2024-04-02T13:36:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.312+0000 7f87915d8740 1 --2- 172.21.15.33:0/2579847443 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564df2699120 0x564df2697e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.312+0000 7f87915d8740 4 client.14757 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:36:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.312+0000 7f87915d8740 1 -- 172.21.15.33:0/2579847443 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "status"}) -- 0x564df25885a0 con 0x564df2699120 2024-04-02T13:36:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.312+0000 7f87877fe640 1 --2- 172.21.15.33:0/2579847443 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564df2699120 0x564df2697e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.312+0000 7f87877fe640 1 --2- 172.21.15.33:0/2579847443 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564df2699120 0x564df2697e30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dfae419ad40b4ed server_cookie=202a8b7e91aeb25c in_seq=0 out_seq=0 2024-04-02T13:36:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.312+0000 7f87677fe640 10 client.14757 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:36:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87677fe640 1 -- 172.21.15.33:0/2579847443 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x564df25885a0 con 0x564df2699120 2024-04-02T13:36:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87677fe640 10 client.14757 handle_command_reply: tid=0 2024-04-02T13:36:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87915d8740 1 client.14757 shutdown 2024-04-02T13:36:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87915d8740 1 -- 172.21.15.33:0/2579847443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ae0 msgr2=0x564df2627d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87915d8740 1 --2- 172.21.15.33:0/2579847443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ae0 0x564df2627d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x564df2617f40 tx=0x7f877800bd90 comp rx=0 tx=0).stop 2024-04-02T13:36:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87915d8740 1 -- 172.21.15.33:0/2579847443 shutdown_connections 2024-04-02T13:36:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87915d8740 1 --2- 172.21.15.33:0/2579847443 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564df2699120 0x564df2697e30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87915d8740 1 --2- 172.21.15.33:0/2579847443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df261c640 0x564df2624830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87915d8740 1 --2- 172.21.15.33:0/2579847443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df26174d0 0x564df2628280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87677fe640 0 client.14757 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:36:28.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87915d8740 1 --2- 172.21.15.33:0/2579847443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ae0 0x564df2627d40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87915d8740 1 -- 172.21.15.33:0/2579847443 >> 172.21.15.33:0/2579847443 conn(0x564df2612050 msgr2=0x564df2611190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:28.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87915d8740 1 -- 172.21.15.33:0/2579847443 shutdown_connections 2024-04-02T13:36:28.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87915d8740 1 -- 172.21.15.33:0/2579847443 wait complete. 2024-04-02T13:36:28.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f87915d8740 20 client.14757 trim_cache size 0 max 16384 2024-04-02T13:36:28.323 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:36:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "b5fd982e-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:36:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-04-02T13:36:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24551, 2024-04-02T13:36:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-02T13:36:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-02T13:36:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-02T13:36:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.473658962, 2024-04-02T13:36:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 17, 2024-04-02T13:36:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 90, 2024-04-02T13:36:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:36:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.652233067999999 2024-04-02T13:36:28.324 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:36:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f878e915640 1 -- 172.21.15.33:0/919298827 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f87700a3bb0 msgr2=0x7f87700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f878e915640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f87700a3bb0 0x7f87700a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8780005e30 tx=0x7f87800325d0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f878e915640 1 -- 172.21.15.33:0/919298827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788101c00 msgr2=0x7f8788102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f878e915640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788101c00 0x7f8788102fe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f877c00d950 tx=0x7f877c00de10 comp rx=0 tx=0).stop 2024-04-02T13:36:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.313+0000 7f878e915640 1 -- 172.21.15.33:0/919298827 shutdown_connections 2024-04-02T13:36:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.314+0000 7f878e915640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f87700a3bb0 0x7f87700a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.314+0000 7f878e915640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87881025c0 0x7f8788103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.314+0000 7f878e915640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788101c00 0x7f8788102fe0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.314+0000 7f878e915640 1 --2- 172.21.15.33:0/919298827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8788100a10 0x7f87881049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.314+0000 7f878e915640 1 -- 172.21.15.33:0/919298827 >> 172.21.15.33:0/919298827 conn(0x7f87880fbfa0 msgr2=0x7f87880fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:28.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.314+0000 7f878e915640 1 -- 172.21.15.33:0/919298827 shutdown_connections 2024-04-02T13:36:28.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:28.314+0000 7f878e915640 1 -- 172.21.15.33:0/919298827 wait complete. 2024-04-02T13:36:28.327 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T13:36:28.327 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:36:28.327 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-04-02T13:36:28.569 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:36:28.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:28 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/329174097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:28.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:28 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/27915236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:28.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:28 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/329174097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:28.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:28 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/27915236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:28.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:28 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/329174097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:28.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:28 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/27915236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:29.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:29 smithi033 ceph-mon[32390]: pgmap v296: 129 pgs: 6 creating+peering, 119 active+clean, 4 unknown; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2024-04-02T13:36:29.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:29 smithi033 ceph-mon[32390]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T13:36:29.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:29 smithi033 ceph-mon[32390]: Cluster is now healthy 2024-04-02T13:36:29.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:29 smithi050 ceph-mon[35524]: pgmap v296: 129 pgs: 6 creating+peering, 119 active+clean, 4 unknown; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2024-04-02T13:36:29.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:29 smithi050 ceph-mon[35524]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T13:36:29.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:29 smithi050 ceph-mon[35524]: Cluster is now healthy 2024-04-02T13:36:29.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:29 smithi145 ceph-mon[35646]: pgmap v296: 129 pgs: 6 creating+peering, 119 active+clean, 4 unknown; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 10 op/s 2024-04-02T13:36:29.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:29 smithi145 ceph-mon[35646]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T13:36:29.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:29 smithi145 ceph-mon[35646]: Cluster is now healthy 2024-04-02T13:36:30.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.243+0000 7f8e0b43c700 1 -- 172.21.15.33:0/260598898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e04102640 msgr2=0x7f8e04102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:30.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.243+0000 7f8e0b43c700 1 --2- 172.21.15.33:0/260598898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e04102640 0x7f8e04102aa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009990 tx=0x7f8df4009ca0 comp rx=0 tx=0).stop 2024-04-02T13:36:30.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.243+0000 7f8e0b43c700 1 -- 172.21.15.33:0/260598898 shutdown_connections 2024-04-02T13:36:30.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.243+0000 7f8e0b43c700 1 --2- 172.21.15.33:0/260598898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e04102fe0 0x7f8e0410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:30.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.243+0000 7f8e0b43c700 1 --2- 172.21.15.33:0/260598898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e04107520 0x7f8e04107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:30.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.243+0000 7f8e0b43c700 1 --2- 172.21.15.33:0/260598898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e04102640 0x7f8e04102aa0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:30.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.243+0000 7f8e0b43c700 1 -- 172.21.15.33:0/260598898 >> 172.21.15.33:0/260598898 conn(0x7f8e04079720 msgr2=0x7f8e04079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:30.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.243+0000 7f8e0b43c700 1 -- 172.21.15.33:0/260598898 shutdown_connections 2024-04-02T13:36:30.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.243+0000 7f8e0b43c700 1 -- 172.21.15.33:0/260598898 wait complete. 2024-04-02T13:36:30.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e0b43c700 1 Processor -- start 2024-04-02T13:36:30.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e0b43c700 1 -- start start 2024-04-02T13:36:30.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e0b43c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e04102640 0x7f8e04198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:30.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e0b43c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e04102fe0 0x7f8e04199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:30.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e0b43c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e04107520 0x7f8e0419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:30.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e0b43c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e04112920 con 0x7f8e04102fe0 2024-04-02T13:36:30.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e0b43c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8e041127a0 con 0x7f8e04102640 2024-04-02T13:36:30.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e0b43c700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8e04112aa0 con 0x7f8e04107520 2024-04-02T13:36:30.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e089d7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e04102fe0 0x7f8e04199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e089d7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e04102fe0 0x7f8e04199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53632/0 (socket says 172.21.15.33:53632) 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e089d7700 1 -- 172.21.15.33:0/2142594581 learned_addr learned my addr 172.21.15.33:0/2142594581 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e091d8700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e04102640 0x7f8e04198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e099d9700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e04107520 0x7f8e0419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e089d7700 1 -- 172.21.15.33:0/2142594581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e04107520 msgr2=0x7f8e0419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e089d7700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e04107520 0x7f8e0419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e089d7700 1 -- 172.21.15.33:0/2142594581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e04102640 msgr2=0x7f8e04198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e089d7700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e04102640 0x7f8e04198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e089d7700 1 -- 172.21.15.33:0/2142594581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0419dd60 con 0x7f8e04102fe0 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e099d9700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e04107520 0x7f8e0419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.244+0000 7f8e091d8700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e04102640 0x7f8e04198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.245+0000 7f8e089d7700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e04102fe0 0x7f8e04199290 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8df40057c0 tx=0x7f8df40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.245+0000 7f8dfa7fc700 1 -- 172.21.15.33:0/2142594581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4026070 con 0x7f8e04102fe0 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.245+0000 7f8e0b43c700 1 -- 172.21.15.33:0/2142594581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0419dff0 con 0x7f8e04102fe0 2024-04-02T13:36:30.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.245+0000 7f8dfa7fc700 1 -- 172.21.15.33:0/2142594581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8df400fb40 con 0x7f8e04102fe0 2024-04-02T13:36:30.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.245+0000 7f8dfa7fc700 1 -- 172.21.15.33:0/2142594581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4017650 con 0x7f8e04102fe0 2024-04-02T13:36:30.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.245+0000 7f8e0b43c700 1 -- 172.21.15.33:0/2142594581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e041a58d0 con 0x7f8e04102fe0 2024-04-02T13:36:30.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.246+0000 7f8e0b43c700 1 -- 172.21.15.33:0/2142594581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e040692b0 con 0x7f8e04102fe0 2024-04-02T13:36:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.247+0000 7f8dfa7fc700 1 -- 172.21.15.33:0/2142594581 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8df4029070 con 0x7f8e04102fe0 2024-04-02T13:36:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.247+0000 7f8dfa7fc700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8df00a3c40 0x7f8df00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.248+0000 7f8e091d8700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8df00a3c40 0x7f8df00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.248+0000 7f8dfa7fc700 1 -- 172.21.15.33:0/2142594581 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8df40d12f0 con 0x7f8e04102fe0 2024-04-02T13:36:30.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.248+0000 7f8e091d8700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8df00a3c40 0x7f8df00a6100 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8e041036d0 tx=0x7f8e0000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:30.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.252+0000 7f8dfa7fc700 1 -- 172.21.15.33:0/2142594581 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8df4013070 con 0x7f8e04102fe0 2024-04-02T13:36:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.511+0000 7f8e0b43c700 1 -- 172.21.15.33:0/2142594581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8e0419a0b0 con 0x7f8e04102fe0 2024-04-02T13:36:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.511+0000 7f8dfa7fc700 1 -- 172.21.15.33:0/2142594581 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f8df4021e20 con 0x7f8e04102fe0 2024-04-02T13:36:30.514 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-02T13:36:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.513+0000 7f8e0b43c700 1 -- 172.21.15.33:0/2142594581 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8df00a3c40 msgr2=0x7f8df00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.513+0000 7f8e0b43c700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8df00a3c40 0x7f8df00a6100 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8e041036d0 tx=0x7f8e0000b410 comp rx=0 tx=0).stop 2024-04-02T13:36:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.513+0000 7f8e0b43c700 1 -- 172.21.15.33:0/2142594581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e04102fe0 msgr2=0x7f8e04199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.513+0000 7f8e0b43c700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e04102fe0 0x7f8e04199290 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8df40057c0 tx=0x7f8df40057f0 comp rx=0 tx=0).stop 2024-04-02T13:36:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.513+0000 7f8e0b43c700 1 -- 172.21.15.33:0/2142594581 shutdown_connections 2024-04-02T13:36:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.513+0000 7f8e0b43c700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8df00a3c40 0x7f8df00a6100 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.513+0000 7f8e0b43c700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e04107520 0x7f8e0419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.513+0000 7f8e0b43c700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e04102640 0x7f8e04198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.513+0000 7f8e0b43c700 1 --2- 172.21.15.33:0/2142594581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e04102fe0 0x7f8e04199290 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.513+0000 7f8e0b43c700 1 -- 172.21.15.33:0/2142594581 >> 172.21.15.33:0/2142594581 conn(0x7f8e04079720 msgr2=0x7f8e041006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.514+0000 7f8e0b43c700 1 -- 172.21.15.33:0/2142594581 shutdown_connections 2024-04-02T13:36:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:30.514+0000 7f8e0b43c700 1 -- 172.21.15.33:0/2142594581 wait complete. 2024-04-02T13:36:31.151 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-02T13:36:31.152 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T13:36:31.152 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-04-02T13:36:31.152 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-02T13:36:31.152 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:36:31.152 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-02T13:36:31.152 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T13:36:31.152 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T13:36:31.152 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-02T13:36:31.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:31.152 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T13:36:31.169 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:36:31.170 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-04-02T13:36:31.170 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-02T13:36:31.170 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42227sec preferred_lft 42227sec 2024-04-02T13:36:31.170 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-04-02T13:36:31.170 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:36:31.170 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:36:31.170 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:36:31.170 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-04-02T13:36:31.171 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T13:36:31.171 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:36:31.171 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-02T13:36:31.171 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-04-02T13:36:31.171 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-04-02T13:36:31.171 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-04-02T13:36:31.171 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T13:36:31.171 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-02T13:36:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:36:31.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:36:31.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:31.371 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T13:36:31.398 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-04-02T13:36:31.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:31.400 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-04-02T13:36:31.468 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-04-02T13:36:31.468 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T13:36:31.468 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-02T13:36:31.469 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-02T13:36:31.469 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T13:36:31.469 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:36:31.469 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-02T13:36:31.469 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-02T13:36:31.469 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-02T13:36:31.469 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-02T13:36:31.469 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-02T13:36:31.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:36:31.715 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:31 smithi033 ceph-mon[32390]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-02T13:36:31.715 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:31 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2142594581' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:36:31.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:36:31.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:31.835 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-02T13:36:31.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:31.849 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-04-02T13:36:31.904 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:36:31.905 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-02T13:36:31.905 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:36:31.905 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-02T13:36:31.905 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-02T13:36:31.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:31 smithi145 ceph-mon[35646]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-02T13:36:31.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:31 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2142594581' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:36:31.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:36:31.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:31 smithi050 ceph-mon[35524]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-02T13:36:31.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:31 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2142594581' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:36:32.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:36:32.027 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-02T13:36:32.027 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:36:32.027 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-02T13:36:32.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-02T13:36:32.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T13:36:32.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-02T13:36:32.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-02T13:36:32.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-02T13:36:32.027 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-02T13:36:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:36:32.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:36:32.161 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:36:32.161 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-02T13:36:32.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-04-02T13:36:32.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T13:36:32.161 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-02T13:36:32.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:36:32.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:36:32.244 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-02T13:36:32.244 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T13:36:32.245 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:36:32.259 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-02T13:36:32.260 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T13:36:32.260 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:36:32.314 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-02T13:36:32.381 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T13:36:32.437 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-02T13:36:32.437 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-02T13:36:32.437 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-02T13:36:32.437 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-02T13:36:32.437 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-02T13:36:32.437 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-02T13:36:32.437 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-02T13:36:32.437 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-02T13:36:32.437 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-02T13:36:32.437 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-02T13:36:32.437 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-02T13:36:32.438 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-04-02T13:36:32.438 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:36:32.438 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-02T13:36:32.438 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-02T13:36:32.438 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-02T13:36:32.438 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-02T13:36:32.438 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T13:36:32.438 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-02T13:36:32.438 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-02T13:36:32.438 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T13:36:32.438 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T13:36:32.438 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T13:36:32.438 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/0ebeb3da0dd34f8d94007fc151b6d2588192fff9ab353aa3f4acfb0cf084df55/merged 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/dfe8c37ae7135c0a7f78a5cf8699396447f2dad62742b90e685d6e7739afaa7b/merged 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/6670f8ed89d9e5fff5c9aa1fae6de93f2f45287d5c03ec0031f40bc22622da28/merged 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/a917a17da40ebb0aa0673f46fa16bbc2efca845d0b3e85aaf3134a25576b1412/merged 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/ad73ca7d4949a816f571ffc36b08e4c009dc9dab17c6a4e83b390814ebb235e3/merged 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/fca44f73029ac4cc1fb49198880e18a9e1bda9a2b2fae3463330f27a42d2e2ac/merged 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/9847228aaa43b9471a6980db4b10398714757c25c8061d6017166911e5ff8d02/merged 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/3ed9f8d0604af5def8f2cf080564de09304b6b24f4e8095b605ccf6e91ad89c2/merged 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/ff71119cf0c70afd63a8e6548b8092b9ff819d7decc1df28d7e9d5dc532a5757/merged 2024-04-02T13:36:32.439 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/5150cff3add9666036b2b9b04dd935d8b0e27d025b4d2ad90a004d6451c7eaab/merged 2024-04-02T13:36:32.440 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-02T13:36:32.440 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:36:32.440 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:36:32.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:32.440 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-02T13:36:32.491 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-02T13:36:32.492 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-02T13:36:32.534 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-02T13:36:32.560 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T13:36:32.616 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-02T13:36:32.616 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-02T13:36:32.616 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-02T13:36:32.616 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-02T13:36:32.616 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-02T13:36:32.616 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-02T13:36:32.617 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/0ebeb3da0dd34f8d94007fc151b6d2588192fff9ab353aa3f4acfb0cf084df55/merged 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/dfe8c37ae7135c0a7f78a5cf8699396447f2dad62742b90e685d6e7739afaa7b/merged 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/6670f8ed89d9e5fff5c9aa1fae6de93f2f45287d5c03ec0031f40bc22622da28/merged 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/a917a17da40ebb0aa0673f46fa16bbc2efca845d0b3e85aaf3134a25576b1412/merged 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/ad73ca7d4949a816f571ffc36b08e4c009dc9dab17c6a4e83b390814ebb235e3/merged 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/fca44f73029ac4cc1fb49198880e18a9e1bda9a2b2fae3463330f27a42d2e2ac/merged 2024-04-02T13:36:32.618 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/9847228aaa43b9471a6980db4b10398714757c25c8061d6017166911e5ff8d02/merged 2024-04-02T13:36:32.619 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/3ed9f8d0604af5def8f2cf080564de09304b6b24f4e8095b605ccf6e91ad89c2/merged 2024-04-02T13:36:32.619 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/ff71119cf0c70afd63a8e6548b8092b9ff819d7decc1df28d7e9d5dc532a5757/merged 2024-04-02T13:36:32.619 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/5150cff3add9666036b2b9b04dd935d8b0e27d025b4d2ad90a004d6451c7eaab/merged 2024-04-02T13:36:32.619 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-02T13:36:32.619 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:36:32.619 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:36:32.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:32.619 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-02T13:36:32.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:2024-04-02T13:36:32.629+0000 7f54e3e43480 -1 init, newargv = 0x560186b29420 newargc=15 2024-04-02T13:36:32.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[74096]: starting ceph client 2024-04-02T13:36:32.638 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[74096]: starting fuse 2024-04-02T13:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout:119 2024-04-02T13:36:32.647 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-04-02T13:36:32.647 DEBUG:teuthology.orchestra.run.smithi033:> sudo stdin-killer -- python3 -c ' 2024-04-02T13:36:32.647 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-04-02T13:36:32.647 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-04-02T13:36:32.647 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-04-02T13:36:32.647 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-04-02T13:36:32.647 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-02T13:36:32.647 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-04-02T13:36:32.647 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-02T13:36:32.648 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-02T13:36:32.649 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.admin")) 2024-04-02T13:36:32.649 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-04-02T13:36:32.768 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.admin.74096.asok 2024-04-02T13:36:32.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:36:32.778 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74096.asok 2024-04-02T13:36:32.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:32.779 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74096.asok status 2024-04-02T13:36:32.916 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:36:32.916 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-04-02T13:36:32.916 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "191d963ac9de7322da8aff20b567b3fa458e2ef6", 2024-04-02T13:36:32.916 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "admin", 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "74096", 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 14769, 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 14769 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-02T13:36:32.917 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-02T13:36:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 405798380 2024-04-02T13:36:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:36:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:36:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-02T13:36:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-02T13:36:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-02T13:36:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 405798380 2024-04-02T13:36:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:36:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.14769 192.168.0.1:0/405798380", 2024-04-02T13:36:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/405798380", 2024-04-02T13:36:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2024-04-02T13:36:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 17, 2024-04-02T13:36:32.919 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 90, 2024-04-02T13:36:32.919 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-04-02T13:36:32.919 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-04-02T13:36:32.919 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-04-02T13:36:32.919 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:36:32.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:32.925 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-04-02T13:36:32.925 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:36:32.940 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-04-02T13:36:32.940 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:36:32.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:32.940 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:36:33.008 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-02T13:36:33.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T13:36:33.117 INFO:teuthology.orchestra.run.smithi033.stdout:# file: . 2024-04-02T13:36:33.118 INFO:teuthology.orchestra.run.smithi033.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-02T13:36:33.118 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:33.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:36:33.124 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:36:33.124 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T13:36:33.140 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:36:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.279+0000 7f1be4115640 1 -- 172.21.15.33:0/1601363887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bdc100d70 msgr2=0x7f1bdc1011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.279+0000 7f1be4115640 1 --2- 172.21.15.33:0/1601363887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bdc100d70 0x7f1bdc1011d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009900 tx=0x7f1bcc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.279+0000 7f1be4115640 1 -- 172.21.15.33:0/1601363887 shutdown_connections 2024-04-02T13:36:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.279+0000 7f1be4115640 1 --2- 172.21.15.33:0/1601363887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bdc101710 0x7f1bdc101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.279+0000 7f1be4115640 1 --2- 172.21.15.33:0/1601363887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bdc100d70 0x7f1bdc1011d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.279+0000 7f1be4115640 1 --2- 172.21.15.33:0/1601363887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bdc100410 0x7f1bdc100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.279+0000 7f1be4115640 1 -- 172.21.15.33:0/1601363887 >> 172.21.15.33:0/1601363887 conn(0x7f1bdc0fc030 msgr2=0x7f1bdc0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.279+0000 7f1be4115640 1 -- 172.21.15.33:0/1601363887 shutdown_connections 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.279+0000 7f1be4115640 1 -- 172.21.15.33:0/1601363887 wait complete. 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be4115640 1 Processor -- start 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be4115640 1 -- start start 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be4115640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bdc100410 0x7f1bdc199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be4115640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bdc100d70 0x7f1bdc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be4115640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bdc101710 0x7f1bdc1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be4115640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc10f960 con 0x7f1bdc100d70 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be4115640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc10f7e0 con 0x7f1bdc101710 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be4115640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc10fae0 con 0x7f1bdc100410 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be16b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bdc100d70 0x7f1bdc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be16b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bdc100d70 0x7f1bdc19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53650/0 (socket says 172.21.15.33:53650) 2024-04-02T13:36:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be16b0640 1 -- 172.21.15.33:0/3266987951 learned_addr learned my addr 172.21.15.33:0/3266987951 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:33.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be1eb1640 1 --2- 172.21.15.33:0/3266987951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bdc100410 0x7f1bdc199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:33.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be16b0640 1 -- 172.21.15.33:0/3266987951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bdc100410 msgr2=0x7f1bdc199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:33.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be16b0640 1 --2- 172.21.15.33:0/3266987951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bdc100410 0x7f1bdc199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be16b0640 1 -- 172.21.15.33:0/3266987951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bdc101710 msgr2=0x7f1bdc1a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:33.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be16b0640 1 --2- 172.21.15.33:0/3266987951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bdc101710 0x7f1bdc1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.280+0000 7f1be16b0640 1 -- 172.21.15.33:0/3266987951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc1a1bc0 con 0x7f1bdc100d70 2024-04-02T13:36:33.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.281+0000 7f1be16b0640 1 --2- 172.21.15.33:0/3266987951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bdc100d70 0x7f1bdc19a4e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009230 tx=0x7f1bcc004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:33.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.281+0000 7f1bd2ffd640 1 -- 172.21.15.33:0/3266987951 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc043020 con 0x7f1bdc100d70 2024-04-02T13:36:33.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.281+0000 7f1bd2ffd640 1 -- 172.21.15.33:0/3266987951 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1bcc004410 con 0x7f1bdc100d70 2024-04-02T13:36:33.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.281+0000 7f1be4115640 1 -- 172.21.15.33:0/3266987951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bdc1a1e50 con 0x7f1bdc100d70 2024-04-02T13:36:33.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.281+0000 7f1bd2ffd640 1 -- 172.21.15.33:0/3266987951 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc003e50 con 0x7f1bdc100d70 2024-04-02T13:36:33.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.281+0000 7f1be4115640 1 -- 172.21.15.33:0/3266987951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bdc1a2310 con 0x7f1bdc100d70 2024-04-02T13:36:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.282+0000 7f1be4115640 1 -- 172.21.15.33:0/3266987951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bdc10b160 con 0x7f1bdc100d70 2024-04-02T13:36:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.284+0000 7f1bd2ffd640 1 -- 172.21.15.33:0/3266987951 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1bcc046020 con 0x7f1bdc100d70 2024-04-02T13:36:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.284+0000 7f1bd2ffd640 1 --2- 172.21.15.33:0/3266987951 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1bc80a3d50 0x7f1bc80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.284+0000 7f1be1eb1640 1 --2- 172.21.15.33:0/3266987951 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1bc80a3d50 0x7f1bc80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.284+0000 7f1bd2ffd640 1 -- 172.21.15.33:0/3266987951 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1bcc0f2ba0 con 0x7f1bdc100d70 2024-04-02T13:36:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.285+0000 7f1be1eb1640 1 --2- 172.21.15.33:0/3266987951 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1bc80a3d50 0x7f1bc80a6210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4005e10 tx=0x7f1bd4005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:33.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.288+0000 7f1bd2ffd640 1 -- 172.21.15.33:0/3266987951 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1bcc04d070 con 0x7f1bdc100d70 2024-04-02T13:36:33.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.476+0000 7f1be4115640 1 -- 172.21.15.33:0/3266987951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bdc02d600 con 0x7f1bdc100d70 2024-04-02T13:36:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.477+0000 7f1bd2ffd640 1 -- 172.21.15.33:0/3266987951 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24057 (secure 0 0 0) 0x7f1bcc0473b0 con 0x7f1bdc100d70 2024-04-02T13:36:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:33.480 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:36:25.856437+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:36:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.479+0000 7f1be4115640 1 -- 172.21.15.33:0/3266987951 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1bc80a3d50 msgr2=0x7f1bc80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.479+0000 7f1be4115640 1 --2- 172.21.15.33:0/3266987951 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1bc80a3d50 0x7f1bc80a6210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4005e10 tx=0x7f1bd4005da0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.479+0000 7f1be4115640 1 -- 172.21.15.33:0/3266987951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bdc100d70 msgr2=0x7f1bdc19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.479+0000 7f1be4115640 1 --2- 172.21.15.33:0/3266987951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bdc100d70 0x7f1bdc19a4e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009230 tx=0x7f1bcc004f10 comp rx=0 tx=0).stop 2024-04-02T13:36:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.479+0000 7f1be4115640 1 -- 172.21.15.33:0/3266987951 shutdown_connections 2024-04-02T13:36:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.479+0000 7f1be4115640 1 --2- 172.21.15.33:0/3266987951 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1bc80a3d50 0x7f1bc80a6210 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.479+0000 7f1be4115640 1 --2- 172.21.15.33:0/3266987951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bdc101710 0x7f1bdc1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.479+0000 7f1be4115640 1 --2- 172.21.15.33:0/3266987951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bdc100d70 0x7f1bdc19a4e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.479+0000 7f1be4115640 1 --2- 172.21.15.33:0/3266987951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1bdc100410 0x7f1bdc199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.479+0000 7f1be4115640 1 -- 172.21.15.33:0/3266987951 >> 172.21.15.33:0/3266987951 conn(0x7f1bdc0fc030 msgr2=0x7f1bdc0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:33.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.479+0000 7f1be4115640 1 -- 172.21.15.33:0/3266987951 shutdown_connections 2024-04-02T13:36:33.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.479+0000 7f1be4115640 1 -- 172.21.15.33:0/3266987951 wait complete. 2024-04-02T13:36:33.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:36:19.501121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:36:20.161569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:36:23.482134+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:36:33.493 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:36:33.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.599+0000 7f71e0ce3640 1 -- 172.21.15.33:0/720564230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc1036c0 msgr2=0x7f71dc109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:33.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.599+0000 7f71e0ce3640 1 --2- 172.21.15.33:0/720564230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc1036c0 0x7f71dc109f50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009ca0 tx=0x7f71d402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.599+0000 7f71e0ce3640 1 -- 172.21.15.33:0/720564230 shutdown_connections 2024-04-02T13:36:33.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.599+0000 7f71e0ce3640 1 --2- 172.21.15.33:0/720564230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc1036c0 0x7f71dc109f50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.599+0000 7f71e0ce3640 1 --2- 172.21.15.33:0/720564230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71dc102d00 0x7f71dc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.599+0000 7f71e0ce3640 1 --2- 172.21.15.33:0/720564230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71dc101b10 0x7f71dc101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.599+0000 7f71e0ce3640 1 -- 172.21.15.33:0/720564230 >> 172.21.15.33:0/720564230 conn(0x7f71dc0fd0a0 msgr2=0x7f71dc0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.599+0000 7f71e0ce3640 1 -- 172.21.15.33:0/720564230 shutdown_connections 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.599+0000 7f71e0ce3640 1 -- 172.21.15.33:0/720564230 wait complete. 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71e0ce3640 1 Processor -- start 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71e0ce3640 1 -- start start 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71e0ce3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc101b10 0x7f71dc0758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71e0ce3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71dc102d00 0x7f71dc073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71e0ce3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71dc1036c0 0x7f71dc074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71e0ce3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10d760 con 0x7f71dc101b10 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71e0ce3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10d5e0 con 0x7f71dc1036c0 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71e0ce3640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10d8e0 con 0x7f71dc102d00 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71da59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc101b10 0x7f71dc0758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71da59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc101b10 0x7f71dc0758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54930/0 (socket says 172.21.15.33:54930) 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71da59c640 1 -- 172.21.15.33:0/3565156540 learned_addr learned my addr 172.21.15.33:0/3565156540 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71dad9d640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71dc1036c0 0x7f71dc074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71d1d9b640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71dc102d00 0x7f71dc073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71da59c640 1 -- 172.21.15.33:0/3565156540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71dc102d00 msgr2=0x7f71dc073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71da59c640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71dc102d00 0x7f71dc073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71da59c640 1 -- 172.21.15.33:0/3565156540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71dc1036c0 msgr2=0x7f71dc074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71da59c640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71dc1036c0 0x7f71dc074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71da59c640 1 -- 172.21.15.33:0/3565156540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc074cc0 con 0x7f71dc101b10 2024-04-02T13:36:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.600+0000 7f71d1d9b640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71dc102d00 0x7f71dc073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.601+0000 7f71da59c640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc101b10 0x7f71dc0758b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00d770 tx=0x7f71cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.601+0000 7f71d3fff640 1 -- 172.21.15.33:0/3565156540 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc017070 con 0x7f71dc101b10 2024-04-02T13:36:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.601+0000 7f71d3fff640 1 -- 172.21.15.33:0/3565156540 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f71cc009c30 con 0x7f71dc101b10 2024-04-02T13:36:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.601+0000 7f71e0ce3640 1 -- 172.21.15.33:0/3565156540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71dc1a1ed0 con 0x7f71dc101b10 2024-04-02T13:36:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.601+0000 7f71d3fff640 1 -- 172.21.15.33:0/3565156540 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc003e10 con 0x7f71dc101b10 2024-04-02T13:36:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.601+0000 7f71e0ce3640 1 -- 172.21.15.33:0/3565156540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71dc1a22e0 con 0x7f71dc101b10 2024-04-02T13:36:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.602+0000 7f71e0ce3640 1 -- 172.21.15.33:0/3565156540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71dc108e30 con 0x7f71dc101b10 2024-04-02T13:36:33.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.603+0000 7f71d3fff640 1 -- 172.21.15.33:0/3565156540 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f71cc004020 con 0x7f71dc101b10 2024-04-02T13:36:33.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.604+0000 7f71d3fff640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f71c80a3bb0 0x7f71c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:33.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.604+0000 7f71d1d9b640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f71c80a3bb0 0x7f71c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:33.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.604+0000 7f71d3fff640 1 -- 172.21.15.33:0/3565156540 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f71cc0d0340 con 0x7f71dc101b10 2024-04-02T13:36:33.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.604+0000 7f71d1d9b640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f71c80a3bb0 0x7f71c80a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f71c40058c0 tx=0x7f71c4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:33.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.608+0000 7f71d3fff640 1 -- 172.21.15.33:0/3565156540 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f71cc09cae0 con 0x7f71dc101b10 2024-04-02T13:36:33.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:33 smithi033 ceph-mon[32390]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-02T13:36:33.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:33 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3266987951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:33.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.794+0000 7f71e0ce3640 1 -- 172.21.15.33:0/3565156540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f71dc061f40 con 0x7f71dc101b10 2024-04-02T13:36:33.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.794+0000 7f71d3fff640 1 -- 172.21.15.33:0/3565156540 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f71cc09c3e0 con 0x7f71dc101b10 2024-04-02T13:36:33.797 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:36:33.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.796+0000 7f71e0ce3640 1 -- 172.21.15.33:0/3565156540 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f71c80a3bb0 msgr2=0x7f71c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:33.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.796+0000 7f71e0ce3640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f71c80a3bb0 0x7f71c80a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f71c40058c0 tx=0x7f71c4005850 comp rx=0 tx=0).stop 2024-04-02T13:36:33.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.796+0000 7f71e0ce3640 1 -- 172.21.15.33:0/3565156540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc101b10 msgr2=0x7f71dc0758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:33.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.796+0000 7f71e0ce3640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc101b10 0x7f71dc0758b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00d770 tx=0x7f71cc00dc30 comp rx=0 tx=0).stop 2024-04-02T13:36:33.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.796+0000 7f71e0ce3640 1 -- 172.21.15.33:0/3565156540 shutdown_connections 2024-04-02T13:36:33.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.796+0000 7f71e0ce3640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f71c80a3bb0 0x7f71c80a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.796+0000 7f71e0ce3640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71dc1036c0 0x7f71dc074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.796+0000 7f71e0ce3640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71dc102d00 0x7f71dc073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.796+0000 7f71e0ce3640 1 --2- 172.21.15.33:0/3565156540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc101b10 0x7f71dc0758b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.796+0000 7f71e0ce3640 1 -- 172.21.15.33:0/3565156540 >> 172.21.15.33:0/3565156540 conn(0x7f71dc0fd0a0 msgr2=0x7f71dc106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:33.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.796+0000 7f71e0ce3640 1 -- 172.21.15.33:0/3565156540 shutdown_connections 2024-04-02T13:36:33.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.796+0000 7f71e0ce3640 1 -- 172.21.15.33:0/3565156540 wait complete. 2024-04-02T13:36:33.810 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:36:33.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.914+0000 7f019f2e4640 1 -- 172.21.15.33:0/1411190963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01981025d0 msgr2=0x7f0198108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:33.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.914+0000 7f019f2e4640 1 --2- 172.21.15.33:0/1411190963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01981025d0 0x7f0198108e60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0190009ca0 tx=0x7f019002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.914+0000 7f019f2e4640 1 -- 172.21.15.33:0/1411190963 shutdown_connections 2024-04-02T13:36:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.914+0000 7f019f2e4640 1 --2- 172.21.15.33:0/1411190963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01981025d0 0x7f0198108e60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.914+0000 7f019f2e4640 1 --2- 172.21.15.33:0/1411190963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0198101c10 0x7f0198102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.914+0000 7f019f2e4640 1 --2- 172.21.15.33:0/1411190963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0198100a20 0x7f0198100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.914+0000 7f019f2e4640 1 -- 172.21.15.33:0/1411190963 >> 172.21.15.33:0/1411190963 conn(0x7f01980fbfb0 msgr2=0x7f01980fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.914+0000 7f019f2e4640 1 -- 172.21.15.33:0/1411190963 shutdown_connections 2024-04-02T13:36:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.914+0000 7f019f2e4640 1 -- 172.21.15.33:0/1411190963 wait complete. 2024-04-02T13:36:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.914+0000 7f019f2e4640 1 Processor -- start 2024-04-02T13:36:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.914+0000 7f019f2e4640 1 -- start start 2024-04-02T13:36:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019f2e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0198100a20 0x7f019810b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019f2e4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0198101c10 0x7f01981099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019f2e4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01981025d0 0x7f0198109f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019f2e4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f019810d790 con 0x7f01981025d0 2024-04-02T13:36:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019f2e4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f019810d610 con 0x7f0198100a20 2024-04-02T13:36:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019f2e4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f019810d910 con 0x7f0198101c10 2024-04-02T13:36:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019d881640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01981025d0 0x7f0198109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019d881640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01981025d0 0x7f0198109f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54956/0 (socket says 172.21.15.33:54956) 2024-04-02T13:36:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019d881640 1 -- 172.21.15.33:0/191465460 learned_addr learned my addr 172.21.15.33:0/191465460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019c87f640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0198101c10 0x7f01981099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019d080640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0198100a20 0x7f019810b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019d881640 1 -- 172.21.15.33:0/191465460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0198101c10 msgr2=0x7f01981099e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019d881640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0198101c10 0x7f01981099e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019d881640 1 -- 172.21.15.33:0/191465460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0198100a20 msgr2=0x7f019810b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019d881640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0198100a20 0x7f019810b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019d881640 1 -- 172.21.15.33:0/191465460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019810a7b0 con 0x7f01981025d0 2024-04-02T13:36:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f019d881640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01981025d0 0x7f0198109f20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f019002c630 tx=0x7f0190032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f018e7fc640 1 -- 172.21.15.33:0/191465460 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190039050 con 0x7f01981025d0 2024-04-02T13:36:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.915+0000 7f018e7fc640 1 -- 172.21.15.33:0/191465460 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f019003a040 con 0x7f01981025d0 2024-04-02T13:36:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.916+0000 7f019f2e4640 1 -- 172.21.15.33:0/191465460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01981a1ec0 con 0x7f01981025d0 2024-04-02T13:36:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.916+0000 7f018e7fc640 1 -- 172.21.15.33:0/191465460 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190040450 con 0x7f01981025d0 2024-04-02T13:36:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.916+0000 7f019f2e4640 1 -- 172.21.15.33:0/191465460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01981a2320 con 0x7f01981025d0 2024-04-02T13:36:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.917+0000 7f019f2e4640 1 -- 172.21.15.33:0/191465460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0198107d40 con 0x7f01981025d0 2024-04-02T13:36:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.918+0000 7f018e7fc640 1 -- 172.21.15.33:0/191465460 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0190003c70 con 0x7f01981025d0 2024-04-02T13:36:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.918+0000 7f018e7fc640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f01840a3bb0 0x7f01840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.919+0000 7f019d080640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f01840a3bb0 0x7f01840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.919+0000 7f018e7fc640 1 -- 172.21.15.33:0/191465460 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f01900f3200 con 0x7f01981025d0 2024-04-02T13:36:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.919+0000 7f019d080640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f01840a3bb0 0x7f01840a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0194005fd0 tx=0x7f019400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:33.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:33.923+0000 7f018e7fc640 1 -- 172.21.15.33:0/191465460 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f01900bf9a0 con 0x7f01981025d0 2024-04-02T13:36:33.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:33 smithi145 ceph-mon[35646]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-02T13:36:33.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:33 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3266987951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:33 smithi050 ceph-mon[35524]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-02T13:36:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:33 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3266987951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:34.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.110+0000 7f019f2e4640 1 -- 172.21.15.33:0/191465460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f01980627a0 con 0x7f01981025d0 2024-04-02T13:36:34.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.110+0000 7f018e7fc640 1 -- 172.21.15.33:0/191465460 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f01900bf2a0 con 0x7f01981025d0 2024-04-02T13:36:34.113 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:36:34.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.112+0000 7f019f2e4640 1 -- 172.21.15.33:0/191465460 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f01840a3bb0 msgr2=0x7f01840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.112+0000 7f019f2e4640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f01840a3bb0 0x7f01840a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0194005fd0 tx=0x7f019400b410 comp rx=0 tx=0).stop 2024-04-02T13:36:34.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.112+0000 7f019f2e4640 1 -- 172.21.15.33:0/191465460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01981025d0 msgr2=0x7f0198109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.112+0000 7f019f2e4640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01981025d0 0x7f0198109f20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f019002c630 tx=0x7f0190032980 comp rx=0 tx=0).stop 2024-04-02T13:36:34.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.112+0000 7f019f2e4640 1 -- 172.21.15.33:0/191465460 shutdown_connections 2024-04-02T13:36:34.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.112+0000 7f019f2e4640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f01840a3bb0 0x7f01840a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.112+0000 7f019f2e4640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01981025d0 0x7f0198109f20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.112+0000 7f019f2e4640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0198101c10 0x7f01981099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.112+0000 7f019f2e4640 1 --2- 172.21.15.33:0/191465460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0198100a20 0x7f019810b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.112+0000 7f019f2e4640 1 -- 172.21.15.33:0/191465460 >> 172.21.15.33:0/191465460 conn(0x7f01980fbfb0 msgr2=0x7f0198104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:34.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.112+0000 7f019f2e4640 1 -- 172.21.15.33:0/191465460 shutdown_connections 2024-04-02T13:36:34.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.112+0000 7f019f2e4640 1 -- 172.21.15.33:0/191465460 wait complete. 2024-04-02T13:36:34.126 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:36:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.229+0000 7f20b130e640 1 -- 172.21.15.33:0/1742007996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac102d10 msgr2=0x7f20ac103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.229+0000 7f20b130e640 1 --2- 172.21.15.33:0/1742007996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac102d10 0x7f20ac103190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2094009900 tx=0x7f209402bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.229+0000 7f20b130e640 1 -- 172.21.15.33:0/1742007996 shutdown_connections 2024-04-02T13:36:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.229+0000 7f20b130e640 1 --2- 172.21.15.33:0/1742007996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20ac1036d0 0x7f20ac109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.229+0000 7f20b130e640 1 --2- 172.21.15.33:0/1742007996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac102d10 0x7f20ac103190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.229+0000 7f20b130e640 1 --2- 172.21.15.33:0/1742007996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20ac101b20 0x7f20ac101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.229+0000 7f20b130e640 1 -- 172.21.15.33:0/1742007996 >> 172.21.15.33:0/1742007996 conn(0x7f20ac0fd090 msgr2=0x7f20ac0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.229+0000 7f20b130e640 1 -- 172.21.15.33:0/1742007996 shutdown_connections 2024-04-02T13:36:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.229+0000 7f20b130e640 1 -- 172.21.15.33:0/1742007996 wait complete. 2024-04-02T13:36:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20b130e640 1 Processor -- start 2024-04-02T13:36:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20b130e640 1 -- start start 2024-04-02T13:36:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20b130e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20ac101b20 0x7f20ac197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20b130e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20ac102d10 0x7f20ac198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20b130e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac1036d0 0x7f20ac19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20b130e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac10d780 con 0x7f20ac1036d0 2024-04-02T13:36:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20b130e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac10d600 con 0x7f20ac101b20 2024-04-02T13:36:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20b130e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac10d900 con 0x7f20ac102d10 2024-04-02T13:36:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac1036d0 0x7f20ac19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac1036d0 0x7f20ac19f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54978/0 (socket says 172.21.15.33:54978) 2024-04-02T13:36:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20ab7fe640 1 -- 172.21.15.33:0/3902167169 learned_addr learned my addr 172.21.15.33:0/3902167169 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20aaffd640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20ac101b20 0x7f20ac197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20a3fff640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20ac102d10 0x7f20ac198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20ab7fe640 1 -- 172.21.15.33:0/3902167169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20ac102d10 msgr2=0x7f20ac198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20ab7fe640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20ac102d10 0x7f20ac198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20ab7fe640 1 -- 172.21.15.33:0/3902167169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20ac101b20 msgr2=0x7f20ac197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20ab7fe640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20ac101b20 0x7f20ac197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20ab7fe640 1 -- 172.21.15.33:0/3902167169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ac19fa20 con 0x7f20ac1036d0 2024-04-02T13:36:34.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20a3fff640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20ac102d10 0x7f20ac198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:34.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.230+0000 7f20ab7fe640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac1036d0 0x7f20ac19f3b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f20a400cad0 tx=0x7f20a400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:34.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.231+0000 7f20a8ff9640 1 -- 172.21.15.33:0/3902167169 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a400ae20 con 0x7f20ac1036d0 2024-04-02T13:36:34.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.231+0000 7f20a8ff9640 1 -- 172.21.15.33:0/3902167169 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f20a4005020 con 0x7f20ac1036d0 2024-04-02T13:36:34.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.231+0000 7f20b130e640 1 -- 172.21.15.33:0/3902167169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ac19fd10 con 0x7f20ac1036d0 2024-04-02T13:36:34.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.231+0000 7f20a8ff9640 1 -- 172.21.15.33:0/3902167169 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a400f690 con 0x7f20ac1036d0 2024-04-02T13:36:34.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.231+0000 7f20b130e640 1 -- 172.21.15.33:0/3902167169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ac1a0250 con 0x7f20ac1036d0 2024-04-02T13:36:34.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.232+0000 7f20b130e640 1 -- 172.21.15.33:0/3902167169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20ac108f10 con 0x7f20ac1036d0 2024-04-02T13:36:34.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.233+0000 7f20a8ff9640 1 -- 172.21.15.33:0/3902167169 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f20a40051d0 con 0x7f20ac1036d0 2024-04-02T13:36:34.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.233+0000 7f20a8ff9640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f20980a3c30 0x7f20980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:34.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.234+0000 7f20aaffd640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f20980a3c30 0x7f20980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:34.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.234+0000 7f20a8ff9640 1 -- 172.21.15.33:0/3902167169 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f20a40d0650 con 0x7f20ac1036d0 2024-04-02T13:36:34.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.234+0000 7f20aaffd640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f20980a3c30 0x7f20980a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f209c005fd0 tx=0x7f209c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:34.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.238+0000 7f20a8ff9640 1 -- 172.21.15.33:0/3902167169 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f20a4014030 con 0x7f20ac1036d0 2024-04-02T13:36:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.425+0000 7f20b130e640 1 -- 172.21.15.33:0/3902167169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f20ac0627a0 con 0x7f20ac1036d0 2024-04-02T13:36:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.426+0000 7f20a8ff9640 1 -- 172.21.15.33:0/3902167169 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f20a409cdf0 con 0x7f20ac1036d0 2024-04-02T13:36:34.429 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-02T13:36:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.427+0000 7f20b130e640 1 -- 172.21.15.33:0/3902167169 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f20980a3c30 msgr2=0x7f20980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.427+0000 7f20b130e640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f20980a3c30 0x7f20980a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f209c005fd0 tx=0x7f209c005850 comp rx=0 tx=0).stop 2024-04-02T13:36:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.427+0000 7f20b130e640 1 -- 172.21.15.33:0/3902167169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac1036d0 msgr2=0x7f20ac19f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.427+0000 7f20b130e640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac1036d0 0x7f20ac19f3b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f20a400cad0 tx=0x7f20a400cf90 comp rx=0 tx=0).stop 2024-04-02T13:36:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.428+0000 7f20b130e640 1 -- 172.21.15.33:0/3902167169 shutdown_connections 2024-04-02T13:36:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.428+0000 7f20b130e640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f20980a3c30 0x7f20980a60f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.428+0000 7f20b130e640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac1036d0 0x7f20ac19f3b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.428+0000 7f20b130e640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20ac102d10 0x7f20ac198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.428+0000 7f20b130e640 1 --2- 172.21.15.33:0/3902167169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20ac101b20 0x7f20ac197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.428+0000 7f20b130e640 1 -- 172.21.15.33:0/3902167169 >> 172.21.15.33:0/3902167169 conn(0x7f20ac0fd090 msgr2=0x7f20ac1060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:34.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.428+0000 7f20b130e640 1 -- 172.21.15.33:0/3902167169 shutdown_connections 2024-04-02T13:36:34.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.428+0000 7f20b130e640 1 -- 172.21.15.33:0/3902167169 wait complete. 2024-04-02T13:36:34.441 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:36:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.545+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1201166651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b41025a0 msgr2=0x7fc2b4104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.545+0000 7fc2bc3cd640 1 --2- 172.21.15.33:0/1201166651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b41025a0 0x7fc2b4104990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009900 tx=0x7fc2b002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.545+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1201166651 shutdown_connections 2024-04-02T13:36:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.545+0000 7fc2bc3cd640 1 --2- 172.21.15.33:0/1201166651 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b4107800 0x7fc2b4109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.545+0000 7fc2bc3cd640 1 --2- 172.21.15.33:0/1201166651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2b4104ed0 0x7fc2b41072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.545+0000 7fc2bc3cd640 1 --2- 172.21.15.33:0/1201166651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b41025a0 0x7fc2b4104990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.545+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1201166651 >> 172.21.15.33:0/1201166651 conn(0x7fc2b40fbfb0 msgr2=0x7fc2b40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.545+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1201166651 shutdown_connections 2024-04-02T13:36:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.545+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1201166651 wait complete. 2024-04-02T13:36:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2bc3cd640 1 Processor -- start 2024-04-02T13:36:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2bc3cd640 1 -- start start 2024-04-02T13:36:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2bc3cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b41025a0 0x7fc2b4197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2bc3cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b4104ed0 0x7fc2b4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2bc3cd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2b4107800 0x7fc2b419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2bc3cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b410d7a0 con 0x7fc2b4104ed0 2024-04-02T13:36:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2bc3cd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b410d620 con 0x7fc2b4107800 2024-04-02T13:36:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2bc3cd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b410d920 con 0x7fc2b41025a0 2024-04-02T13:36:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2b9968640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b4104ed0 0x7fc2b4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2b9968640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b4104ed0 0x7fc2b4198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54992/0 (socket says 172.21.15.33:54992) 2024-04-02T13:36:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2b9968640 1 -- 172.21.15.33:0/1777200323 learned_addr learned my addr 172.21.15.33:0/1777200323 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2ba96a640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2b4107800 0x7fc2b419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2ba169640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b41025a0 0x7fc2b4197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2b9968640 1 -- 172.21.15.33:0/1777200323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b41025a0 msgr2=0x7fc2b4197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2b9968640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b41025a0 0x7fc2b4197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2b9968640 1 -- 172.21.15.33:0/1777200323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2b4107800 msgr2=0x7fc2b419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2b9968640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2b4107800 0x7fc2b419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2b9968640 1 -- 172.21.15.33:0/1777200323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b419fa60 con 0x7fc2b4104ed0 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.546+0000 7fc2ba169640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b41025a0 0x7fc2b4197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.547+0000 7fc2b9968640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b4104ed0 0x7fc2b4198380 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4005fd0 tx=0x7fc2a400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.547+0000 7fc2ab7fe640 1 -- 172.21.15.33:0/1777200323 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4017070 con 0x7fc2b4104ed0 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.547+0000 7fc2ab7fe640 1 -- 172.21.15.33:0/1777200323 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc2a4009c30 con 0x7fc2b4104ed0 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.547+0000 7fc2ab7fe640 1 -- 172.21.15.33:0/1777200323 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4003e20 con 0x7fc2b4104ed0 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.547+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1777200323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b419fd50 con 0x7fc2b4104ed0 2024-04-02T13:36:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.547+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1777200323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b41a0240 con 0x7fc2b4104ed0 2024-04-02T13:36:34.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.548+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1777200323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b40fdaa0 con 0x7fc2b4104ed0 2024-04-02T13:36:34.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.549+0000 7fc2ab7fe640 1 -- 172.21.15.33:0/1777200323 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc2a4003fe0 con 0x7fc2b4104ed0 2024-04-02T13:36:34.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.550+0000 7fc2ab7fe640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc2a00a3bb0 0x7fc2a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:34.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.550+0000 7fc2ba169640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc2a00a3bb0 0x7fc2a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:34.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.550+0000 7fc2ab7fe640 1 -- 172.21.15.33:0/1777200323 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc2a40d0b30 con 0x7fc2b4104ed0 2024-04-02T13:36:34.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.550+0000 7fc2ba169640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc2a00a3bb0 0x7fc2a00a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0005c40 tx=0x7fc2b0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:34.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.554+0000 7fc2ab7fe640 1 -- 172.21.15.33:0/1777200323 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc2a4020030 con 0x7fc2b4104ed0 2024-04-02T13:36:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.738+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1777200323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc2b402d5c0 con 0x7fc2b4104ed0 2024-04-02T13:36:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.738+0000 7fc2ab7fe640 1 -- 172.21.15.33:0/1777200323 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fc2a409d2b0 con 0x7fc2b4104ed0 2024-04-02T13:36:34.742 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:36:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.740+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1777200323 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc2a00a3bb0 msgr2=0x7fc2a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.740+0000 7fc2bc3cd640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc2a00a3bb0 0x7fc2a00a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0005c40 tx=0x7fc2b0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.740+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1777200323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b4104ed0 msgr2=0x7fc2b4198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.740+0000 7fc2bc3cd640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b4104ed0 0x7fc2b4198380 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4005fd0 tx=0x7fc2a400dde0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.740+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1777200323 shutdown_connections 2024-04-02T13:36:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.740+0000 7fc2bc3cd640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc2a00a3bb0 0x7fc2a00a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.740+0000 7fc2bc3cd640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2b4107800 0x7fc2b419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.740+0000 7fc2bc3cd640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b4104ed0 0x7fc2b4198380 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.740+0000 7fc2bc3cd640 1 --2- 172.21.15.33:0/1777200323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2b41025a0 0x7fc2b4197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.740+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1777200323 >> 172.21.15.33:0/1777200323 conn(0x7fc2b40fbfb0 msgr2=0x7fc2b4102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.740+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1777200323 shutdown_connections 2024-04-02T13:36:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.740+0000 7fc2bc3cd640 1 -- 172.21.15.33:0/1777200323 wait complete. 2024-04-02T13:36:34.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:34 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3565156540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:36:34.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:34 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/191465460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:36:34.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:34 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3902167169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:36:34.754 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T13:36:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.855+0000 7fa73c374640 1 -- 172.21.15.33:0/4103125833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7341077f0 msgr2=0x7fa734109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.855+0000 7fa73c374640 1 --2- 172.21.15.33:0/4103125833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7341077f0 0x7fa734109c40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009ca0 tx=0x7fa72c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.855+0000 7fa73c374640 1 -- 172.21.15.33:0/4103125833 shutdown_connections 2024-04-02T13:36:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.855+0000 7fa73c374640 1 --2- 172.21.15.33:0/4103125833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7341077f0 0x7fa734109c40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.855+0000 7fa73c374640 1 --2- 172.21.15.33:0/4103125833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa734104ec0 0x7fa7341072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.855+0000 7fa73c374640 1 --2- 172.21.15.33:0/4103125833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa734102590 0x7fa734104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.855+0000 7fa73c374640 1 -- 172.21.15.33:0/4103125833 >> 172.21.15.33:0/4103125833 conn(0x7fa7340fbfa0 msgr2=0x7fa7340fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.855+0000 7fa73c374640 1 -- 172.21.15.33:0/4103125833 shutdown_connections 2024-04-02T13:36:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.855+0000 7fa73c374640 1 -- 172.21.15.33:0/4103125833 wait complete. 2024-04-02T13:36:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73c374640 1 Processor -- start 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73c374640 1 -- start start 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73c374640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa734102590 0x7fa734197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73c374640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa734104ec0 0x7fa7341982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73c374640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7341077f0 0x7fa73419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73c374640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa73410d770 con 0x7fa7341077f0 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73c374640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa73410d5f0 con 0x7fa734104ec0 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73c374640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa73410d8f0 con 0x7fa734102590 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73a911640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7341077f0 0x7fa73419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73a911640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7341077f0 0x7fa73419f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55012/0 (socket says 172.21.15.33:55012) 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73a911640 1 -- 172.21.15.33:0/3830924287 learned_addr learned my addr 172.21.15.33:0/3830924287 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73a911640 1 -- 172.21.15.33:0/3830924287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa734102590 msgr2=0x7fa734197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73a911640 1 --2- 172.21.15.33:0/3830924287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa734102590 0x7fa734197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73a911640 1 -- 172.21.15.33:0/3830924287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa734104ec0 msgr2=0x7fa7341982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73a911640 1 --2- 172.21.15.33:0/3830924287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa734104ec0 0x7fa7341982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.856+0000 7fa73a911640 1 -- 172.21.15.33:0/3830924287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa73419f9b0 con 0x7fa7341077f0 2024-04-02T13:36:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.857+0000 7fa73a911640 1 --2- 172.21.15.33:0/3830924287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7341077f0 0x7fa73419f310 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa72c02c630 tx=0x7fa72c032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.857+0000 7fa72b7fe640 1 -- 172.21.15.33:0/3830924287 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c039050 con 0x7fa7341077f0 2024-04-02T13:36:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.857+0000 7fa72b7fe640 1 -- 172.21.15.33:0/3830924287 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa72c03a040 con 0x7fa7341077f0 2024-04-02T13:36:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.857+0000 7fa72b7fe640 1 -- 172.21.15.33:0/3830924287 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c040450 con 0x7fa7341077f0 2024-04-02T13:36:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.857+0000 7fa73c374640 1 -- 172.21.15.33:0/3830924287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa73419fc40 con 0x7fa7341077f0 2024-04-02T13:36:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.857+0000 7fa73c374640 1 -- 172.21.15.33:0/3830924287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7341a0180 con 0x7fa7341077f0 2024-04-02T13:36:34.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.858+0000 7fa73c374640 1 -- 172.21.15.33:0/3830924287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7340fda90 con 0x7fa7341077f0 2024-04-02T13:36:34.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.859+0000 7fa72b7fe640 1 -- 172.21.15.33:0/3830924287 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa72c003c70 con 0x7fa7341077f0 2024-04-02T13:36:34.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.859+0000 7fa72b7fe640 1 --2- 172.21.15.33:0/3830924287 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa7200a3c30 0x7fa7200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:34.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.860+0000 7fa73a110640 1 --2- 172.21.15.33:0/3830924287 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa7200a3c30 0x7fa7200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:34.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.860+0000 7fa72b7fe640 1 -- 172.21.15.33:0/3830924287 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa72c0f33f0 con 0x7fa7341077f0 2024-04-02T13:36:34.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.860+0000 7fa73a110640 1 --2- 172.21.15.33:0/3830924287 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa7200a3c30 0x7fa7200a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa730005fd0 tx=0x7fa730005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:34.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:34.864+0000 7fa72b7fe640 1 -- 172.21.15.33:0/3830924287 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa72c0bfb90 con 0x7fa7341077f0 2024-04-02T13:36:34.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:34 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3565156540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:36:34.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:34 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/191465460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:36:34.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:34 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3902167169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:36:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:34 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3565156540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:36:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:34 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/191465460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:36:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:34 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3902167169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:36:35.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.062+0000 7fa73c374640 1 -- 172.21.15.33:0/3830924287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa73402d370 con 0x7fa7341077f0 2024-04-02T13:36:35.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.062+0000 7fa72b7fe640 1 -- 172.21.15.33:0/3830924287 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7fa72c0bf490 con 0x7fa7341077f0 2024-04-02T13:36:35.065 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1117299953 2024-04-03T13:29:15.778774+0000 2024-04-02T13:36:35.065 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2272183007 2024-04-03T13:29:15.778774+0000 2024-04-02T13:36:35.065 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2666001000 2024-04-03T13:29:15.778774+0000 2024-04-02T13:36:35.065 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2666001000 2024-04-03T13:29:15.778774+0000 2024-04-02T13:36:35.065 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2869265288 2024-04-03T13:28:48.425104+0000 2024-04-02T13:36:35.065 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3041616812 2024-04-03T13:28:48.425104+0000 2024-04-02T13:36:35.066 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2869265288 2024-04-03T13:28:48.425104+0000 2024-04-02T13:36:35.066 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3449207963 2024-04-03T13:28:48.425104+0000 2024-04-02T13:36:35.066 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/917705077 2024-04-03T13:28:48.425104+0000 2024-04-02T13:36:35.066 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1792579346 2024-04-03T13:29:15.778774+0000 2024-04-02T13:36:35.066 INFO:teuthology.orchestra.run.smithi033.stderr:listed 10 entries 2024-04-02T13:36:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.064+0000 7fa73c374640 1 -- 172.21.15.33:0/3830924287 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa7200a3c30 msgr2=0x7fa7200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.064+0000 7fa73c374640 1 --2- 172.21.15.33:0/3830924287 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa7200a3c30 0x7fa7200a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa730005fd0 tx=0x7fa730005850 comp rx=0 tx=0).stop 2024-04-02T13:36:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.064+0000 7fa73c374640 1 -- 172.21.15.33:0/3830924287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7341077f0 msgr2=0x7fa73419f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.064+0000 7fa73c374640 1 --2- 172.21.15.33:0/3830924287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7341077f0 0x7fa73419f310 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa72c02c630 tx=0x7fa72c032980 comp rx=0 tx=0).stop 2024-04-02T13:36:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.064+0000 7fa73c374640 1 -- 172.21.15.33:0/3830924287 shutdown_connections 2024-04-02T13:36:35.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.064+0000 7fa73c374640 1 --2- 172.21.15.33:0/3830924287 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa7200a3c30 0x7fa7200a60f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.064+0000 7fa73c374640 1 --2- 172.21.15.33:0/3830924287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7341077f0 0x7fa73419f310 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.064+0000 7fa73c374640 1 --2- 172.21.15.33:0/3830924287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa734104ec0 0x7fa7341982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.064+0000 7fa73c374640 1 --2- 172.21.15.33:0/3830924287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa734102590 0x7fa734197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.064+0000 7fa73c374640 1 -- 172.21.15.33:0/3830924287 >> 172.21.15.33:0/3830924287 conn(0x7fa7340fbfa0 msgr2=0x7fa734102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:35.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.065+0000 7fa73c374640 1 -- 172.21.15.33:0/3830924287 shutdown_connections 2024-04-02T13:36:35.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.065+0000 7fa73c374640 1 -- 172.21.15.33:0/3830924287 wait complete. 2024-04-02T13:36:35.078 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:36:35.078 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T13:36:35.093 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:36:35.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.232+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/1075307281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4101b20 msgr2=0x7fa1d4101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.232+0000 7fa1dbdcd640 1 --2- 172.21.15.33:0/1075307281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4101b20 0x7fa1d4101f40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009900 tx=0x7fa1d002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/1075307281 shutdown_connections 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 --2- 172.21.15.33:0/1075307281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d41036d0 0x7fa1d4109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 --2- 172.21.15.33:0/1075307281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d4102d10 0x7fa1d4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 --2- 172.21.15.33:0/1075307281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4101b20 0x7fa1d4101f40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/1075307281 >> 172.21.15.33:0/1075307281 conn(0x7fa1d40fd0b0 msgr2=0x7fa1d40ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/1075307281 shutdown_connections 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/1075307281 wait complete. 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 Processor -- start 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 -- start start 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d4101b20 0x7fa1d4197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4102d10 0x7fa1d4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d41036d0 0x7fa1d419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d410d7e0 con 0x7fa1d4102d10 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d410d660 con 0x7fa1d4101b20 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1dbdcd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d410d960 con 0x7fa1d41036d0 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1d9368640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4102d10 0x7fa1d4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1d9368640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4102d10 0x7fa1d4198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55032/0 (socket says 172.21.15.33:55032) 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.233+0000 7fa1d9368640 1 -- 172.21.15.33:0/396404654 learned_addr learned my addr 172.21.15.33:0/396404654 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.234+0000 7fa1d9b69640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d4101b20 0x7fa1d4197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.234+0000 7fa1da36a640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d41036d0 0x7fa1d419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.234+0000 7fa1d9368640 1 -- 172.21.15.33:0/396404654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d41036d0 msgr2=0x7fa1d419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.234+0000 7fa1d9368640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d41036d0 0x7fa1d419f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.234+0000 7fa1d9368640 1 -- 172.21.15.33:0/396404654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d4101b20 msgr2=0x7fa1d4197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.234+0000 7fa1d9368640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d4101b20 0x7fa1d4197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.234+0000 7fa1d9368640 1 -- 172.21.15.33:0/396404654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d419fb00 con 0x7fa1d4102d10 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.234+0000 7fa1d9368640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4102d10 0x7fa1d4198420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400b4a0 tx=0x7fa1c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.234+0000 7fa1caffd640 1 -- 172.21.15.33:0/396404654 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4014070 con 0x7fa1d4102d10 2024-04-02T13:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.234+0000 7fa1caffd640 1 -- 172.21.15.33:0/396404654 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa1c4010ae0 con 0x7fa1d4102d10 2024-04-02T13:36:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.234+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/396404654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d419fdf0 con 0x7fa1d4102d10 2024-04-02T13:36:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.234+0000 7fa1caffd640 1 -- 172.21.15.33:0/396404654 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4017c70 con 0x7fa1d4102d10 2024-04-02T13:36:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.234+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/396404654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d41a0330 con 0x7fa1d4102d10 2024-04-02T13:36:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.235+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/396404654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d4108ec0 con 0x7fa1d4102d10 2024-04-02T13:36:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.237+0000 7fa1caffd640 1 -- 172.21.15.33:0/396404654 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa1c4017070 con 0x7fa1d4102d10 2024-04-02T13:36:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.237+0000 7fa1caffd640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa1c00a3bb0 0x7fa1c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.237+0000 7fa1d9b69640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa1c00a3bb0 0x7fa1c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.237+0000 7fa1caffd640 1 -- 172.21.15.33:0/396404654 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa1c40d0090 con 0x7fa1d4102d10 2024-04-02T13:36:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.238+0000 7fa1d9b69640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa1c00a3bb0 0x7fa1c00a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0000c00 tx=0x7fa1d0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:35.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.241+0000 7fa1caffd640 1 -- 172.21.15.33:0/396404654 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa1c409c830 con 0x7fa1d4102d10 2024-04-02T13:36:35.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.428+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/396404654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1d402d570 con 0x7fa1d4102d10 2024-04-02T13:36:35.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.429+0000 7fa1caffd640 1 -- 172.21.15.33:0/396404654 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24057 (secure 0 0 0) 0x7fa1c409c130 con 0x7fa1d4102d10 2024-04-02T13:36:35.433 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:35.434 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:36:25.856437+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:36:35.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.431+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/396404654 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa1c00a3bb0 msgr2=0x7fa1c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.431+0000 7fa1dbdcd640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa1c00a3bb0 0x7fa1c00a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0000c00 tx=0x7fa1d0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.431+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/396404654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4102d10 msgr2=0x7fa1d4198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.431+0000 7fa1dbdcd640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4102d10 0x7fa1d4198420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400b4a0 tx=0x7fa1c400b960 comp rx=0 tx=0).stop 2024-04-02T13:36:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.432+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/396404654 shutdown_connections 2024-04-02T13:36:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.432+0000 7fa1dbdcd640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa1c00a3bb0 0x7fa1c00a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.432+0000 7fa1dbdcd640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1d41036d0 0x7fa1d419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.432+0000 7fa1dbdcd640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d4102d10 0x7fa1d4198420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.432+0000 7fa1dbdcd640 1 --2- 172.21.15.33:0/396404654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa1d4101b20 0x7fa1d4197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.432+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/396404654 >> 172.21.15.33:0/396404654 conn(0x7fa1d40fd0b0 msgr2=0x7fa1d4106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.432+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/396404654 shutdown_connections 2024-04-02T13:36:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.432+0000 7fa1dbdcd640 1 -- 172.21.15.33:0/396404654 wait complete. 2024-04-02T13:36:35.446 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:36:19.501121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:36:20.161569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:36:23.482134+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:36:35.447 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:36:35.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.549+0000 7fa6ce58d640 1 -- 172.21.15.33:0/975031203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c81036c0 msgr2=0x7fa6c8109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.549+0000 7fa6ce58d640 1 --2- 172.21.15.33:0/975031203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c81036c0 0x7fa6c8109f50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009ca0 tx=0x7fa6c002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.549+0000 7fa6ce58d640 1 -- 172.21.15.33:0/975031203 shutdown_connections 2024-04-02T13:36:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.549+0000 7fa6ce58d640 1 --2- 172.21.15.33:0/975031203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c81036c0 0x7fa6c8109f50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.549+0000 7fa6ce58d640 1 --2- 172.21.15.33:0/975031203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6c8102d00 0x7fa6c8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.549+0000 7fa6ce58d640 1 --2- 172.21.15.33:0/975031203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6c8101b10 0x7fa6c8101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.549+0000 7fa6ce58d640 1 -- 172.21.15.33:0/975031203 >> 172.21.15.33:0/975031203 conn(0x7fa6c80fd0a0 msgr2=0x7fa6c80ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.549+0000 7fa6ce58d640 1 -- 172.21.15.33:0/975031203 shutdown_connections 2024-04-02T13:36:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.549+0000 7fa6ce58d640 1 -- 172.21.15.33:0/975031203 wait complete. 2024-04-02T13:36:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.550+0000 7fa6ce58d640 1 Processor -- start 2024-04-02T13:36:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.550+0000 7fa6ce58d640 1 -- start start 2024-04-02T13:36:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.550+0000 7fa6ce58d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c8101b10 0x7fa6c80758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.550+0000 7fa6ce58d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6c8102d00 0x7fa6c8073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.550+0000 7fa6ce58d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6c81036c0 0x7fa6c8074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.550+0000 7fa6ce58d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c810d770 con 0x7fa6c8101b10 2024-04-02T13:36:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.550+0000 7fa6ce58d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c810d5f0 con 0x7fa6c81036c0 2024-04-02T13:36:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.550+0000 7fa6ce58d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c810d8f0 con 0x7fa6c8102d00 2024-04-02T13:36:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.550+0000 7fa6c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c8101b10 0x7fa6c80758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.550+0000 7fa6c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c8101b10 0x7fa6c80758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55056/0 (socket says 172.21.15.33:55056) 2024-04-02T13:36:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.550+0000 7fa6c7fff640 1 -- 172.21.15.33:0/2049198989 learned_addr learned my addr 172.21.15.33:0/2049198989 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:35.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.550+0000 7fa6ccb2a640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6c81036c0 0x7fa6c8074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:35.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.551+0000 7fa6c77fe640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6c8102d00 0x7fa6c8073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:35.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.551+0000 7fa6c7fff640 1 -- 172.21.15.33:0/2049198989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6c8102d00 msgr2=0x7fa6c8073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.551+0000 7fa6c7fff640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6c8102d00 0x7fa6c8073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.551+0000 7fa6c7fff640 1 -- 172.21.15.33:0/2049198989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6c81036c0 msgr2=0x7fa6c8074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.551+0000 7fa6c7fff640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6c81036c0 0x7fa6c8074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.551+0000 7fa6c7fff640 1 -- 172.21.15.33:0/2049198989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c8074cc0 con 0x7fa6c8101b10 2024-04-02T13:36:35.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.551+0000 7fa6c7fff640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c8101b10 0x7fa6c80758b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800d770 tx=0x7fa6b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:35.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.551+0000 7fa6c57fa640 1 -- 172.21.15.33:0/2049198989 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8017070 con 0x7fa6c8101b10 2024-04-02T13:36:35.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.551+0000 7fa6c57fa640 1 -- 172.21.15.33:0/2049198989 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa6b8009c30 con 0x7fa6c8101b10 2024-04-02T13:36:35.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.551+0000 7fa6ce58d640 1 -- 172.21.15.33:0/2049198989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c81a1eb0 con 0x7fa6c8101b10 2024-04-02T13:36:35.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.551+0000 7fa6c57fa640 1 -- 172.21.15.33:0/2049198989 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8003e10 con 0x7fa6c8101b10 2024-04-02T13:36:35.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.551+0000 7fa6ce58d640 1 -- 172.21.15.33:0/2049198989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c81a22c0 con 0x7fa6c8101b10 2024-04-02T13:36:35.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.552+0000 7fa6ce58d640 1 -- 172.21.15.33:0/2049198989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c8108eb0 con 0x7fa6c8101b10 2024-04-02T13:36:35.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.553+0000 7fa6c57fa640 1 -- 172.21.15.33:0/2049198989 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa6b8003fb0 con 0x7fa6c8101b10 2024-04-02T13:36:35.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.554+0000 7fa6c57fa640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa6b00a3bb0 0x7fa6b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:35.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.554+0000 7fa6c57fa640 1 -- 172.21.15.33:0/2049198989 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa6b80d0040 con 0x7fa6c8101b10 2024-04-02T13:36:35.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.554+0000 7fa6c77fe640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa6b00a3bb0 0x7fa6b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:35.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.554+0000 7fa6c77fe640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa6b00a3bb0 0x7fa6b00a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005fd0 tx=0x7fa6bc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:35.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.558+0000 7fa6c57fa640 1 -- 172.21.15.33:0/2049198989 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa6b809c780 con 0x7fa6c8101b10 2024-04-02T13:36:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.744+0000 7fa6ce58d640 1 -- 172.21.15.33:0/2049198989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa6c8061f40 con 0x7fa6c8101b10 2024-04-02T13:36:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.744+0000 7fa6c57fa640 1 -- 172.21.15.33:0/2049198989 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fa6b809c080 con 0x7fa6c8101b10 2024-04-02T13:36:35.748 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:36:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.746+0000 7fa6ce58d640 1 -- 172.21.15.33:0/2049198989 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa6b00a3bb0 msgr2=0x7fa6b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.746+0000 7fa6ce58d640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa6b00a3bb0 0x7fa6b00a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005fd0 tx=0x7fa6bc005850 comp rx=0 tx=0).stop 2024-04-02T13:36:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.746+0000 7fa6ce58d640 1 -- 172.21.15.33:0/2049198989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c8101b10 msgr2=0x7fa6c80758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.746+0000 7fa6ce58d640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c8101b10 0x7fa6c80758b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800d770 tx=0x7fa6b800dc30 comp rx=0 tx=0).stop 2024-04-02T13:36:35.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.746+0000 7fa6ce58d640 1 -- 172.21.15.33:0/2049198989 shutdown_connections 2024-04-02T13:36:35.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.746+0000 7fa6ce58d640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa6b00a3bb0 0x7fa6b00a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.746+0000 7fa6ce58d640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6c81036c0 0x7fa6c8074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.746+0000 7fa6ce58d640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6c8102d00 0x7fa6c8073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.746+0000 7fa6ce58d640 1 --2- 172.21.15.33:0/2049198989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6c8101b10 0x7fa6c80758b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.746+0000 7fa6ce58d640 1 -- 172.21.15.33:0/2049198989 >> 172.21.15.33:0/2049198989 conn(0x7fa6c80fd0a0 msgr2=0x7fa6c8106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:35.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.746+0000 7fa6ce58d640 1 -- 172.21.15.33:0/2049198989 shutdown_connections 2024-04-02T13:36:35.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.746+0000 7fa6ce58d640 1 -- 172.21.15.33:0/2049198989 wait complete. 2024-04-02T13:36:35.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:35 smithi033 ceph-mon[32390]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s wr, 3 op/s 2024-04-02T13:36:35.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:35 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1777200323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:36:35.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:35 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3830924287' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:36:35.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:35 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/396404654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:35.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:36:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.863+0000 7f1827f8f640 1 -- 172.21.15.33:0/1052326551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820107860 msgr2=0x7f1820109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.863+0000 7f1827f8f640 1 --2- 172.21.15.33:0/1052326551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820107860 0x7f1820109c50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1810009900 tx=0x7f181002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.863+0000 7f1827f8f640 1 -- 172.21.15.33:0/1052326551 shutdown_connections 2024-04-02T13:36:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.863+0000 7f1827f8f640 1 --2- 172.21.15.33:0/1052326551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820107860 0x7f1820109c50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.863+0000 7f1827f8f640 1 --2- 172.21.15.33:0/1052326551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1820104f30 0x7f1820107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.863+0000 7f1827f8f640 1 --2- 172.21.15.33:0/1052326551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1820102600 0x7f18201049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.863+0000 7f1827f8f640 1 -- 172.21.15.33:0/1052326551 >> 172.21.15.33:0/1052326551 conn(0x7f18200fbf90 msgr2=0x7f18200fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.864+0000 7f1827f8f640 1 -- 172.21.15.33:0/1052326551 shutdown_connections 2024-04-02T13:36:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.864+0000 7f1827f8f640 1 -- 172.21.15.33:0/1052326551 wait complete. 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.864+0000 7f1827f8f640 1 Processor -- start 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.864+0000 7f1827f8f640 1 -- start start 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.864+0000 7f1827f8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820102600 0x7f1820101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.864+0000 7f1827f8f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1820104f30 0x7f18201005b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.864+0000 7f1827f8f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1820107860 0x7f1820100af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.864+0000 7f1827f8f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f182010d780 con 0x7f1820102600 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.864+0000 7f1827f8f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f182010d600 con 0x7f1820104f30 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.864+0000 7f1827f8f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f182010d900 con 0x7f1820107860 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.864+0000 7f1825d2b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820102600 0x7f1820101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.864+0000 7f1825d2b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820102600 0x7f1820101f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55072/0 (socket says 172.21.15.33:55072) 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.864+0000 7f1825d2b640 1 -- 172.21.15.33:0/668048936 learned_addr learned my addr 172.21.15.33:0/668048936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f182552a640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1820104f30 0x7f18201005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f182652c640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1820107860 0x7f1820100af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f1825d2b640 1 -- 172.21.15.33:0/668048936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1820107860 msgr2=0x7f1820100af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f1825d2b640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1820107860 0x7f1820100af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f1825d2b640 1 -- 172.21.15.33:0/668048936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1820104f30 msgr2=0x7f18201005b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f1825d2b640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1820104f30 0x7f18201005b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f1825d2b640 1 -- 172.21.15.33:0/668048936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1820101380 con 0x7f1820102600 2024-04-02T13:36:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f182552a640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1820104f30 0x7f18201005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f1825d2b640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820102600 0x7f1820101f70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f181800ea80 tx=0x7f181800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f181effd640 1 -- 172.21.15.33:0/668048936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181800ce40 con 0x7f1820102600 2024-04-02T13:36:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f181effd640 1 -- 172.21.15.33:0/668048936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1818005020 con 0x7f1820102600 2024-04-02T13:36:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f1827f8f640 1 -- 172.21.15.33:0/668048936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18201a1f50 con 0x7f1820102600 2024-04-02T13:36:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f181effd640 1 -- 172.21.15.33:0/668048936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18180116d0 con 0x7f1820102600 2024-04-02T13:36:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.865+0000 7f1827f8f640 1 -- 172.21.15.33:0/668048936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18201a2360 con 0x7f1820102600 2024-04-02T13:36:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.866+0000 7f1827f8f640 1 -- 172.21.15.33:0/668048936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18200fda80 con 0x7f1820102600 2024-04-02T13:36:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.867+0000 7f181effd640 1 -- 172.21.15.33:0/668048936 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f18180051d0 con 0x7f1820102600 2024-04-02T13:36:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.868+0000 7f181effd640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f18080a3c30 0x7f18080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.868+0000 7f182552a640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f18080a3c30 0x7f18080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.868+0000 7f181effd640 1 -- 172.21.15.33:0/668048936 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f18180d0a00 con 0x7f1820102600 2024-04-02T13:36:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.868+0000 7f182552a640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f18080a3c30 0x7f18080a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1820101ce0 tx=0x7f181400b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:35.872+0000 7f181effd640 1 -- 172.21.15.33:0/668048936 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1818016030 con 0x7f1820102600 2024-04-02T13:36:35.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:35 smithi145 ceph-mon[35646]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s wr, 3 op/s 2024-04-02T13:36:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:35 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1777200323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:36:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:35 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3830924287' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:36:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:35 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/396404654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:35 smithi050 ceph-mon[35524]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s wr, 3 op/s 2024-04-02T13:36:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:35 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1777200323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:36:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:35 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3830924287' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:36:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:35 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/396404654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:36.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.060+0000 7f1827f8f640 1 -- 172.21.15.33:0/668048936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1820061f40 con 0x7f1820102600 2024-04-02T13:36:36.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.060+0000 7f181effd640 1 -- 172.21.15.33:0/668048936 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f181809d140 con 0x7f1820102600 2024-04-02T13:36:36.064 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:36:36.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.062+0000 7f1827f8f640 1 -- 172.21.15.33:0/668048936 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f18080a3c30 msgr2=0x7f18080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.062+0000 7f1827f8f640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f18080a3c30 0x7f18080a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1820101ce0 tx=0x7f181400b5f0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.062+0000 7f1827f8f640 1 -- 172.21.15.33:0/668048936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820102600 msgr2=0x7f1820101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.062+0000 7f1827f8f640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820102600 0x7f1820101f70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f181800ea80 tx=0x7f181800ef40 comp rx=0 tx=0).stop 2024-04-02T13:36:36.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.062+0000 7f1827f8f640 1 -- 172.21.15.33:0/668048936 shutdown_connections 2024-04-02T13:36:36.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.062+0000 7f1827f8f640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f18080a3c30 0x7f18080a60f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.062+0000 7f1827f8f640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1820107860 0x7f1820100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.062+0000 7f1827f8f640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1820104f30 0x7f18201005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.062+0000 7f1827f8f640 1 --2- 172.21.15.33:0/668048936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820102600 0x7f1820101f70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.062+0000 7f1827f8f640 1 -- 172.21.15.33:0/668048936 >> 172.21.15.33:0/668048936 conn(0x7f18200fbf90 msgr2=0x7f18201030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:36.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.062+0000 7f1827f8f640 1 -- 172.21.15.33:0/668048936 shutdown_connections 2024-04-02T13:36:36.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.062+0000 7f1827f8f640 1 -- 172.21.15.33:0/668048936 wait complete. 2024-04-02T13:36:36.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:36:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.179+0000 7f876c350640 1 -- 172.21.15.33:0/3305170670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87641025b0 msgr2=0x7f8764108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.179+0000 7f876c350640 1 --2- 172.21.15.33:0/3305170670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87641025b0 0x7f8764108e40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8754009ca0 tx=0x7f875402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.179+0000 7f876c350640 1 -- 172.21.15.33:0/3305170670 shutdown_connections 2024-04-02T13:36:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.179+0000 7f876c350640 1 --2- 172.21.15.33:0/3305170670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87641025b0 0x7f8764108e40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.179+0000 7f876c350640 1 --2- 172.21.15.33:0/3305170670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8764101bf0 0x7f8764102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.179+0000 7f876c350640 1 --2- 172.21.15.33:0/3305170670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8764100a00 0x7f8764100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.179+0000 7f876c350640 1 -- 172.21.15.33:0/3305170670 >> 172.21.15.33:0/3305170670 conn(0x7f87640fbf90 msgr2=0x7f87640fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876c350640 1 -- 172.21.15.33:0/3305170670 shutdown_connections 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876c350640 1 -- 172.21.15.33:0/3305170670 wait complete. 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876c350640 1 Processor -- start 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876c350640 1 -- start start 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876c350640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8764100a00 0x7f87640697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876c350640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8764101bf0 0x7f876419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876c350640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87641025b0 0x7f876419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876c350640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f876410c660 con 0x7f8764100a00 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876c350640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f876410c4e0 con 0x7f87641025b0 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876c350640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f876410c7e0 con 0x7f8764101bf0 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876a0ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8764100a00 0x7f87640697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876a0ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8764100a00 0x7f87640697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55078/0 (socket says 172.21.15.33:55078) 2024-04-02T13:36:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876a0ec640 1 -- 172.21.15.33:0/1510250728 learned_addr learned my addr 172.21.15.33:0/1510250728 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.180+0000 7f876a8ed640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87641025b0 0x7f876419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.181+0000 7f876a0ec640 1 -- 172.21.15.33:0/1510250728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8764101bf0 msgr2=0x7f876419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.181+0000 7f876a0ec640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8764101bf0 0x7f876419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.181+0000 7f876a0ec640 1 -- 172.21.15.33:0/1510250728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87641025b0 msgr2=0x7f876419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.181+0000 7f876a0ec640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87641025b0 0x7f876419f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.181+0000 7f876a0ec640 1 -- 172.21.15.33:0/1510250728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876419fa20 con 0x7f8764100a00 2024-04-02T13:36:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.181+0000 7f876a8ed640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87641025b0 0x7f876419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.181+0000 7f876a0ec640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8764100a00 0x7f87640697f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f875c00d950 tx=0x7f875c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.181+0000 7f87637fe640 1 -- 172.21.15.33:0/1510250728 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c017070 con 0x7f8764100a00 2024-04-02T13:36:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.181+0000 7f87637fe640 1 -- 172.21.15.33:0/1510250728 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f875c009c30 con 0x7f8764100a00 2024-04-02T13:36:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.181+0000 7f876c350640 1 -- 172.21.15.33:0/1510250728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876419fd10 con 0x7f8764100a00 2024-04-02T13:36:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.181+0000 7f87637fe640 1 -- 172.21.15.33:0/1510250728 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c00f620 con 0x7f8764100a00 2024-04-02T13:36:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.181+0000 7f876c350640 1 -- 172.21.15.33:0/1510250728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87641a01b0 con 0x7f8764100a00 2024-04-02T13:36:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.182+0000 7f876c350640 1 -- 172.21.15.33:0/1510250728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8764107dc0 con 0x7f8764100a00 2024-04-02T13:36:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.183+0000 7f87637fe640 1 -- 172.21.15.33:0/1510250728 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f875c01a070 con 0x7f8764100a00 2024-04-02T13:36:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.183+0000 7f87637fe640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f874c0a3bb0 0x7f874c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.184+0000 7f87698eb640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f874c0a3bb0 0x7f874c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.184+0000 7f87637fe640 1 -- 172.21.15.33:0/1510250728 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f875c0d01f0 con 0x7f8764100a00 2024-04-02T13:36:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.184+0000 7f87698eb640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f874c0a3bb0 0x7f874c0a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f87580058c0 tx=0x7f8758005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:36.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.188+0000 7f87637fe640 1 -- 172.21.15.33:0/1510250728 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f875c009de0 con 0x7f8764100a00 2024-04-02T13:36:36.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.372+0000 7f876c350640 1 -- 172.21.15.33:0/1510250728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f87640627a0 con 0x7f8764100a00 2024-04-02T13:36:36.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.373+0000 7f87637fe640 1 -- 172.21.15.33:0/1510250728 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f875c09c930 con 0x7f8764100a00 2024-04-02T13:36:36.377 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-02T13:36:36.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.375+0000 7f876c350640 1 -- 172.21.15.33:0/1510250728 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f874c0a3bb0 msgr2=0x7f874c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.375+0000 7f876c350640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f874c0a3bb0 0x7f874c0a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f87580058c0 tx=0x7f8758005850 comp rx=0 tx=0).stop 2024-04-02T13:36:36.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.375+0000 7f876c350640 1 -- 172.21.15.33:0/1510250728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8764100a00 msgr2=0x7f87640697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.375+0000 7f876c350640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8764100a00 0x7f87640697f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f875c00d950 tx=0x7f875c00de10 comp rx=0 tx=0).stop 2024-04-02T13:36:36.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.375+0000 7f876c350640 1 -- 172.21.15.33:0/1510250728 shutdown_connections 2024-04-02T13:36:36.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.375+0000 7f876c350640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f874c0a3bb0 0x7f874c0a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.375+0000 7f876c350640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87641025b0 0x7f876419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.375+0000 7f876c350640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8764101bf0 0x7f876419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.375+0000 7f876c350640 1 --2- 172.21.15.33:0/1510250728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8764100a00 0x7f87640697f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.375+0000 7f876c350640 1 -- 172.21.15.33:0/1510250728 >> 172.21.15.33:0/1510250728 conn(0x7f87640fbf90 msgr2=0x7f8764104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:36.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.375+0000 7f876c350640 1 -- 172.21.15.33:0/1510250728 shutdown_connections 2024-04-02T13:36:36.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.375+0000 7f876c350640 1 -- 172.21.15.33:0/1510250728 wait complete. 2024-04-02T13:36:36.390 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:36:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 -- 172.21.15.33:0/2917870210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c0101c10 msgr2=0x7fb0c0102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 --2- 172.21.15.33:0/2917870210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c0101c10 0x7fb0c0102090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009900 tx=0x7fb0a802bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 -- 172.21.15.33:0/2917870210 shutdown_connections 2024-04-02T13:36:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 --2- 172.21.15.33:0/2917870210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c01025d0 0x7fb0c0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 --2- 172.21.15.33:0/2917870210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c0101c10 0x7fb0c0102090 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 --2- 172.21.15.33:0/2917870210 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0c0100a20 0x7fb0c0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 -- 172.21.15.33:0/2917870210 >> 172.21.15.33:0/2917870210 conn(0x7fb0c00fbfb0 msgr2=0x7fb0c00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 -- 172.21.15.33:0/2917870210 shutdown_connections 2024-04-02T13:36:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 -- 172.21.15.33:0/2917870210 wait complete. 2024-04-02T13:36:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 Processor -- start 2024-04-02T13:36:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 -- start start 2024-04-02T13:36:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0c0100a20 0x7fb0c0196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c0101c10 0x7fb0c0197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c01025d0 0x7fb0c019e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c010c6d0 con 0x7fb0c01025d0 2024-04-02T13:36:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c010c550 con 0x7fb0c0101c10 2024-04-02T13:36:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.492+0000 7fb0c58d3640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c010c850 con 0x7fb0c0100a20 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0bf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c01025d0 0x7fb0c019e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0bf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c01025d0 0x7fb0c019e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55086/0 (socket says 172.21.15.33:55086) 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0bf7fe640 1 -- 172.21.15.33:0/4044835487 learned_addr learned my addr 172.21.15.33:0/4044835487 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0b7fff640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c0101c10 0x7fb0c0197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0beffd640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0c0100a20 0x7fb0c0196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0bf7fe640 1 -- 172.21.15.33:0/4044835487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0c0100a20 msgr2=0x7fb0c0196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0bf7fe640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0c0100a20 0x7fb0c0196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0bf7fe640 1 -- 172.21.15.33:0/4044835487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c0101c10 msgr2=0x7fb0c0197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0bf7fe640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c0101c10 0x7fb0c0197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0bf7fe640 1 -- 172.21.15.33:0/4044835487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c019e940 con 0x7fb0c01025d0 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0beffd640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0c0100a20 0x7fb0c0196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0bf7fe640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c01025d0 0x7fb0c019e2a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800cce0 tx=0x7fb0b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0c48d1640 1 -- 172.21.15.33:0/4044835487 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800f450 con 0x7fb0c01025d0 2024-04-02T13:36:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0c48d1640 1 -- 172.21.15.33:0/4044835487 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb0b800fac0 con 0x7fb0c01025d0 2024-04-02T13:36:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0c58d3640 1 -- 172.21.15.33:0/4044835487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c006a130 con 0x7fb0c01025d0 2024-04-02T13:36:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0c48d1640 1 -- 172.21.15.33:0/4044835487 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8018be0 con 0x7fb0c01025d0 2024-04-02T13:36:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.493+0000 7fb0c58d3640 1 -- 172.21.15.33:0/4044835487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c006a630 con 0x7fb0c01025d0 2024-04-02T13:36:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.494+0000 7fb0c58d3640 1 -- 172.21.15.33:0/4044835487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c0068d40 con 0x7fb0c01025d0 2024-04-02T13:36:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.495+0000 7fb0c48d1640 1 -- 172.21.15.33:0/4044835487 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb0b800fc70 con 0x7fb0c01025d0 2024-04-02T13:36:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.496+0000 7fb0c48d1640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb0ac0a3bb0 0x7fb0ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.496+0000 7fb0beffd640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb0ac0a3bb0 0x7fb0ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.496+0000 7fb0c48d1640 1 -- 172.21.15.33:0/4044835487 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb0b80d0e30 con 0x7fb0c01025d0 2024-04-02T13:36:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.497+0000 7fb0beffd640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb0ac0a3bb0 0x7fb0ac0a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0005fd0 tx=0x7fb0b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.500+0000 7fb0c48d1640 1 -- 172.21.15.33:0/4044835487 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb0b8014030 con 0x7fb0c01025d0 2024-04-02T13:36:36.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.688+0000 7fb0c58d3640 1 -- 172.21.15.33:0/4044835487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb0c0107900 con 0x7fb0c01025d0 2024-04-02T13:36:36.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.688+0000 7fb0c48d1640 1 -- 172.21.15.33:0/4044835487 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fb0b809d5d0 con 0x7fb0c01025d0 2024-04-02T13:36:36.692 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:36:36.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.690+0000 7fb0c58d3640 1 -- 172.21.15.33:0/4044835487 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb0ac0a3bb0 msgr2=0x7fb0ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.690+0000 7fb0c58d3640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb0ac0a3bb0 0x7fb0ac0a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0005fd0 tx=0x7fb0b0005850 comp rx=0 tx=0).stop 2024-04-02T13:36:36.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.690+0000 7fb0c58d3640 1 -- 172.21.15.33:0/4044835487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c01025d0 msgr2=0x7fb0c019e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.690+0000 7fb0c58d3640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c01025d0 0x7fb0c019e2a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800cce0 tx=0x7fb0b800a620 comp rx=0 tx=0).stop 2024-04-02T13:36:36.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.690+0000 7fb0c58d3640 1 -- 172.21.15.33:0/4044835487 shutdown_connections 2024-04-02T13:36:36.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.690+0000 7fb0c58d3640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb0ac0a3bb0 0x7fb0ac0a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.690+0000 7fb0c58d3640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c01025d0 0x7fb0c019e2a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.690+0000 7fb0c58d3640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0c0101c10 0x7fb0c0197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.690+0000 7fb0c58d3640 1 --2- 172.21.15.33:0/4044835487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0c0100a20 0x7fb0c0196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.690+0000 7fb0c58d3640 1 -- 172.21.15.33:0/4044835487 >> 172.21.15.33:0/4044835487 conn(0x7fb0c00fbfb0 msgr2=0x7fb0c0104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:36.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.690+0000 7fb0c58d3640 1 -- 172.21.15.33:0/4044835487 shutdown_connections 2024-04-02T13:36:36.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.690+0000 7fb0c58d3640 1 -- 172.21.15.33:0/4044835487 wait complete. 2024-04-02T13:36:36.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:36 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2049198989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:36:36.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:36 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/668048936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:36:36.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:36 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1510250728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:36:36.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T13:36:36.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.805+0000 7fb788350640 1 -- 172.21.15.33:0/2224365376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb780101c10 msgr2=0x7fb780102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.805+0000 7fb788350640 1 --2- 172.21.15.33:0/2224365376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb780101c10 0x7fb780102090 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb770009900 tx=0x7fb77002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:36.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.805+0000 7fb788350640 1 -- 172.21.15.33:0/2224365376 shutdown_connections 2024-04-02T13:36:36.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.805+0000 7fb788350640 1 --2- 172.21.15.33:0/2224365376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7801025d0 0x7fb780108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.805+0000 7fb788350640 1 --2- 172.21.15.33:0/2224365376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb780101c10 0x7fb780102090 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.805+0000 7fb788350640 1 --2- 172.21.15.33:0/2224365376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb780100a20 0x7fb780100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.805+0000 7fb788350640 1 -- 172.21.15.33:0/2224365376 >> 172.21.15.33:0/2224365376 conn(0x7fb7800fbfb0 msgr2=0x7fb7800fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:36.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.805+0000 7fb788350640 1 -- 172.21.15.33:0/2224365376 shutdown_connections 2024-04-02T13:36:36.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.805+0000 7fb788350640 1 -- 172.21.15.33:0/2224365376 wait complete. 2024-04-02T13:36:36.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb788350640 1 Processor -- start 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb788350640 1 -- start start 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb788350640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb780100a20 0x7fb78010b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb788350640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb780101c10 0x7fb7801099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb788350640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7801025d0 0x7fb780109f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb788350640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb78010d7d0 con 0x7fb7801025d0 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb788350640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb78010d650 con 0x7fb780100a20 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb788350640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb78010d950 con 0x7fb780101c10 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb7868ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7801025d0 0x7fb780109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb7868ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7801025d0 0x7fb780109f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55102/0 (socket says 172.21.15.33:55102) 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb7868ed640 1 -- 172.21.15.33:0/4097527075 learned_addr learned my addr 172.21.15.33:0/4097527075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb7860ec640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb780100a20 0x7fb78010b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb7858eb640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb780101c10 0x7fb7801099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb7868ed640 1 -- 172.21.15.33:0/4097527075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb780101c10 msgr2=0x7fb7801099e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb7868ed640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb780101c10 0x7fb7801099e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb7868ed640 1 -- 172.21.15.33:0/4097527075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb780100a20 msgr2=0x7fb78010b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb7868ed640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb780100a20 0x7fb78010b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb7868ed640 1 -- 172.21.15.33:0/4097527075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78010a7b0 con 0x7fb7801025d0 2024-04-02T13:36:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb7860ec640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb780100a20 0x7fb78010b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.806+0000 7fb7868ed640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7801025d0 0x7fb780109f20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb77800cad0 tx=0x7fb77800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.807+0000 7fb7777fe640 1 -- 172.21.15.33:0/4097527075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77800ae20 con 0x7fb7801025d0 2024-04-02T13:36:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.807+0000 7fb788350640 1 -- 172.21.15.33:0/4097527075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7801a1f90 con 0x7fb7801025d0 2024-04-02T13:36:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.807+0000 7fb7777fe640 1 -- 172.21.15.33:0/4097527075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb778005020 con 0x7fb7801025d0 2024-04-02T13:36:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.807+0000 7fb7777fe640 1 -- 172.21.15.33:0/4097527075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77800f690 con 0x7fb7801025d0 2024-04-02T13:36:36.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.807+0000 7fb788350640 1 -- 172.21.15.33:0/4097527075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7801a23a0 con 0x7fb7801025d0 2024-04-02T13:36:36.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.808+0000 7fb788350640 1 -- 172.21.15.33:0/4097527075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb780107dc0 con 0x7fb7801025d0 2024-04-02T13:36:36.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.809+0000 7fb7777fe640 1 -- 172.21.15.33:0/4097527075 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb7780051d0 con 0x7fb7801025d0 2024-04-02T13:36:36.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.809+0000 7fb7777fe640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb76c0a3bb0 0x7fb76c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:36.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.809+0000 7fb7860ec640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb76c0a3bb0 0x7fb76c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:36.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.809+0000 7fb7777fe640 1 -- 172.21.15.33:0/4097527075 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb7780d0720 con 0x7fb7801025d0 2024-04-02T13:36:36.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.810+0000 7fb7860ec640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb76c0a3bb0 0x7fb76c0a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0058c0 tx=0x7fb77c005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:36.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:36.814+0000 7fb7777fe640 1 -- 172.21.15.33:0/4097527075 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb778014030 con 0x7fb7801025d0 2024-04-02T13:36:36.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:36 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2049198989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:36:36.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:36 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/668048936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:36:36.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:36 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1510250728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:36:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:36 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2049198989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:36:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:36 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/668048936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:36:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:36 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1510250728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:36:37.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.013+0000 7fb788350640 1 -- 172.21.15.33:0/4097527075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb78002d370 con 0x7fb7801025d0 2024-04-02T13:36:37.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.013+0000 7fb7777fe640 1 -- 172.21.15.33:0/4097527075 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7fb77809cec0 con 0x7fb7801025d0 2024-04-02T13:36:37.017 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1117299953 2024-04-03T13:29:15.778774+0000 2024-04-02T13:36:37.017 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2272183007 2024-04-03T13:29:15.778774+0000 2024-04-02T13:36:37.017 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2666001000 2024-04-03T13:29:15.778774+0000 2024-04-02T13:36:37.017 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2666001000 2024-04-03T13:29:15.778774+0000 2024-04-02T13:36:37.017 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2869265288 2024-04-03T13:28:48.425104+0000 2024-04-02T13:36:37.017 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3041616812 2024-04-03T13:28:48.425104+0000 2024-04-02T13:36:37.017 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2869265288 2024-04-03T13:28:48.425104+0000 2024-04-02T13:36:37.017 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3449207963 2024-04-03T13:28:48.425104+0000 2024-04-02T13:36:37.018 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/917705077 2024-04-03T13:28:48.425104+0000 2024-04-02T13:36:37.018 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1792579346 2024-04-03T13:29:15.778774+0000 2024-04-02T13:36:37.018 INFO:teuthology.orchestra.run.smithi033.stderr:listed 10 entries 2024-04-02T13:36:37.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.015+0000 7fb788350640 1 -- 172.21.15.33:0/4097527075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb76c0a3bb0 msgr2=0x7fb76c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:37.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.015+0000 7fb788350640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb76c0a3bb0 0x7fb76c0a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0058c0 tx=0x7fb77c005ec0 comp rx=0 tx=0).stop 2024-04-02T13:36:37.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.015+0000 7fb788350640 1 -- 172.21.15.33:0/4097527075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7801025d0 msgr2=0x7fb780109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:37.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.015+0000 7fb788350640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7801025d0 0x7fb780109f20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb77800cad0 tx=0x7fb77800cf90 comp rx=0 tx=0).stop 2024-04-02T13:36:37.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.015+0000 7fb788350640 1 -- 172.21.15.33:0/4097527075 shutdown_connections 2024-04-02T13:36:37.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.015+0000 7fb788350640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb76c0a3bb0 0x7fb76c0a6070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:37.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.015+0000 7fb788350640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7801025d0 0x7fb780109f20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:37.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.015+0000 7fb788350640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb780101c10 0x7fb7801099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:37.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.015+0000 7fb788350640 1 --2- 172.21.15.33:0/4097527075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb780100a20 0x7fb78010b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:37.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.015+0000 7fb788350640 1 -- 172.21.15.33:0/4097527075 >> 172.21.15.33:0/4097527075 conn(0x7fb7800fbfb0 msgr2=0x7fb780104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:37.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.015+0000 7fb788350640 1 -- 172.21.15.33:0/4097527075 shutdown_connections 2024-04-02T13:36:37.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:37.015+0000 7fb788350640 1 -- 172.21.15.33:0/4097527075 wait complete. 2024-04-02T13:36:37.030 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2024-04-02T13:36:37.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:37.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:36:37.055 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[74096]: fuse finished with error 0 and tester_r 0 2024-04-02T13:36:37.057 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T13:36:37.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:37 smithi145 ceph-mon[35646]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 2 op/s 2024-04-02T13:36:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:37 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4044835487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:36:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:37 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4097527075' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:36:37.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:37 smithi050 ceph-mon[35524]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 2 op/s 2024-04-02T13:36:37.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:37 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4044835487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:36:37.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:37 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4097527075' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:36:38.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:37 smithi033 ceph-mon[32390]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 2 op/s 2024-04-02T13:36:38.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:37 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4044835487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:36:38.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:37 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4097527075' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:36:39.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:39 smithi145 ceph-mon[35646]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 359 B/s wr, 2 op/s 2024-04-02T13:36:39.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:39 smithi050 ceph-mon[35524]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 359 B/s wr, 2 op/s 2024-04-02T13:36:40.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:39 smithi033 ceph-mon[32390]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 359 B/s wr, 2 op/s 2024-04-02T13:36:41.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:41 smithi033 ceph-mon[32390]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:36:41.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:41 smithi145 ceph-mon[35646]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:36:41.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:41 smithi050 ceph-mon[35524]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:36:43.158 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-04-02T13:36:43.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:43.159 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-02T13:36:43.189 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:36:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.340+0000 7f2c992b1640 1 -- 172.21.15.33:0/488797461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c941025d0 msgr2=0x7f2c94108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.340+0000 7f2c992b1640 1 --2- 172.21.15.33:0/488797461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c941025d0 0x7f2c94108e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009ca0 tx=0x7f2c8c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.340+0000 7f2c992b1640 1 -- 172.21.15.33:0/488797461 shutdown_connections 2024-04-02T13:36:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.340+0000 7f2c992b1640 1 --2- 172.21.15.33:0/488797461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c941025d0 0x7f2c94108e60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.340+0000 7f2c992b1640 1 --2- 172.21.15.33:0/488797461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c94101c10 0x7f2c94102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.340+0000 7f2c992b1640 1 --2- 172.21.15.33:0/488797461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c94100a20 0x7f2c94100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.340+0000 7f2c992b1640 1 -- 172.21.15.33:0/488797461 >> 172.21.15.33:0/488797461 conn(0x7f2c940fbf90 msgr2=0x7f2c940fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.340+0000 7f2c992b1640 1 -- 172.21.15.33:0/488797461 shutdown_connections 2024-04-02T13:36:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.340+0000 7f2c992b1640 1 -- 172.21.15.33:0/488797461 wait complete. 2024-04-02T13:36:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.341+0000 7f2c992b1640 1 Processor -- start 2024-04-02T13:36:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.341+0000 7f2c992b1640 1 -- start start 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.341+0000 7f2c992b1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c94100a20 0x7f2c941049b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.341+0000 7f2c992b1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c94101c10 0x7f2c94102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.341+0000 7f2c992b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c941025d0 0x7f2c94103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.341+0000 7f2c992b1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9410c670 con 0x7f2c941025d0 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.341+0000 7f2c992b1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9410c4f0 con 0x7f2c94101c10 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.341+0000 7f2c992b1640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9410c7f0 con 0x7f2c94100a20 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.341+0000 7f2c937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c941025d0 0x7f2c94103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.341+0000 7f2c937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c941025d0 0x7f2c94103530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55118/0 (socket says 172.21.15.33:55118) 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.341+0000 7f2c937fe640 1 -- 172.21.15.33:0/2793308489 learned_addr learned my addr 172.21.15.33:0/2793308489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.341+0000 7f2c8a7fc640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c94101c10 0x7f2c94102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.341+0000 7f2c92ffd640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c94100a20 0x7f2c941049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.342+0000 7f2c937fe640 1 -- 172.21.15.33:0/2793308489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c94100a20 msgr2=0x7f2c941049b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.342+0000 7f2c937fe640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c94100a20 0x7f2c941049b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.342+0000 7f2c937fe640 1 -- 172.21.15.33:0/2793308489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c94101c10 msgr2=0x7f2c94102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.342+0000 7f2c937fe640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c94101c10 0x7f2c94102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.342+0000 7f2c937fe640 1 -- 172.21.15.33:0/2793308489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c94103dc0 con 0x7f2c941025d0 2024-04-02T13:36:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.342+0000 7f2c937fe640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c941025d0 0x7f2c94103530 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c02c630 tx=0x7f2c8c032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.342+0000 7f2c90ff9640 1 -- 172.21.15.33:0/2793308489 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c039050 con 0x7f2c941025d0 2024-04-02T13:36:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.342+0000 7f2c90ff9640 1 -- 172.21.15.33:0/2793308489 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2c8c03a040 con 0x7f2c941025d0 2024-04-02T13:36:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.342+0000 7f2c992b1640 1 -- 172.21.15.33:0/2793308489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c941a1ec0 con 0x7f2c941025d0 2024-04-02T13:36:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.342+0000 7f2c90ff9640 1 -- 172.21.15.33:0/2793308489 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c040450 con 0x7f2c941025d0 2024-04-02T13:36:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.342+0000 7f2c992b1640 1 -- 172.21.15.33:0/2793308489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c941a2320 con 0x7f2c941025d0 2024-04-02T13:36:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.343+0000 7f2c992b1640 1 -- 172.21.15.33:0/2793308489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c941a25e0 con 0x7f2c941025d0 2024-04-02T13:36:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.345+0000 7f2c90ff9640 1 -- 172.21.15.33:0/2793308489 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2c8c004040 con 0x7f2c941025d0 2024-04-02T13:36:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.345+0000 7f2c90ff9640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2c800a3bb0 0x7f2c800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.346+0000 7f2c92ffd640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2c800a3bb0 0x7f2c800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.346+0000 7f2c90ff9640 1 -- 172.21.15.33:0/2793308489 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2c8c0f3cf0 con 0x7f2c941025d0 2024-04-02T13:36:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.346+0000 7f2c92ffd640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2c800a3bb0 0x7f2c800a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005d00 tx=0x7f2c84005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.349+0000 7f2c90ff9640 1 -- 172.21.15.33:0/2793308489 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2c8c004330 con 0x7f2c941025d0 2024-04-02T13:36:43.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.537+0000 7f2c992b1640 1 -- 172.21.15.33:0/2793308489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c941a2970 con 0x7f2c941025d0 2024-04-02T13:36:43.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.538+0000 7f2c90ff9640 1 -- 172.21.15.33:0/2793308489 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24057 (secure 0 0 0) 0x7f2c8c0c0490 con 0x7f2c941025d0 2024-04-02T13:36:43.542 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:43.543 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:36:25.856437+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:36:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.540+0000 7f2c992b1640 1 -- 172.21.15.33:0/2793308489 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2c800a3bb0 msgr2=0x7f2c800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.540+0000 7f2c992b1640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2c800a3bb0 0x7f2c800a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005d00 tx=0x7f2c84005c50 comp rx=0 tx=0).stop 2024-04-02T13:36:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.540+0000 7f2c992b1640 1 -- 172.21.15.33:0/2793308489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c941025d0 msgr2=0x7f2c94103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.540+0000 7f2c992b1640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c941025d0 0x7f2c94103530 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c02c630 tx=0x7f2c8c032980 comp rx=0 tx=0).stop 2024-04-02T13:36:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.541+0000 7f2c992b1640 1 -- 172.21.15.33:0/2793308489 shutdown_connections 2024-04-02T13:36:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.541+0000 7f2c992b1640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2c800a3bb0 0x7f2c800a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.541+0000 7f2c992b1640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c941025d0 0x7f2c94103530 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.541+0000 7f2c992b1640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c94101c10 0x7f2c94102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.541+0000 7f2c992b1640 1 --2- 172.21.15.33:0/2793308489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c94100a20 0x7f2c941049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.541+0000 7f2c992b1640 1 -- 172.21.15.33:0/2793308489 >> 172.21.15.33:0/2793308489 conn(0x7f2c940fbf90 msgr2=0x7f2c940fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.541+0000 7f2c992b1640 1 -- 172.21.15.33:0/2793308489 shutdown_connections 2024-04-02T13:36:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.541+0000 7f2c992b1640 1 -- 172.21.15.33:0/2793308489 wait complete. 2024-04-02T13:36:43.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:36:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.655+0000 7f2c03b67640 1 -- 172.21.15.33:0/305027005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bfc101b20 msgr2=0x7f2bfc101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:43.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.655+0000 7f2c03b67640 1 --2- 172.21.15.33:0/305027005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bfc101b20 0x7f2bfc101f40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009900 tx=0x7f2bf802bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.655+0000 7f2c03b67640 1 -- 172.21.15.33:0/305027005 shutdown_connections 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.655+0000 7f2c03b67640 1 --2- 172.21.15.33:0/305027005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bfc1036d0 0x7f2bfc109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.655+0000 7f2c03b67640 1 --2- 172.21.15.33:0/305027005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bfc102d10 0x7f2bfc103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.655+0000 7f2c03b67640 1 --2- 172.21.15.33:0/305027005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bfc101b20 0x7f2bfc101f40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.655+0000 7f2c03b67640 1 -- 172.21.15.33:0/305027005 >> 172.21.15.33:0/305027005 conn(0x7f2bfc0fd0b0 msgr2=0x7f2bfc0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.655+0000 7f2c03b67640 1 -- 172.21.15.33:0/305027005 shutdown_connections 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.655+0000 7f2c03b67640 1 -- 172.21.15.33:0/305027005 wait complete. 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.655+0000 7f2c03b67640 1 Processor -- start 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.655+0000 7f2c03b67640 1 -- start start 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c03b67640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bfc101b20 0x7f2bfc197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c03b67640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bfc102d10 0x7f2bfc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c03b67640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bfc1036d0 0x7f2bfc19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c03b67640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc10d7e0 con 0x7f2bfc101b20 2024-04-02T13:36:43.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c03b67640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc10d660 con 0x7f2bfc1036d0 2024-04-02T13:36:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c03b67640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc10d960 con 0x7f2bfc102d10 2024-04-02T13:36:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c01903640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bfc101b20 0x7f2bfc197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c01903640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bfc101b20 0x7f2bfc197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35364/0 (socket says 172.21.15.33:35364) 2024-04-02T13:36:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c01903640 1 -- 172.21.15.33:0/3892821995 learned_addr learned my addr 172.21.15.33:0/3892821995 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c02104640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bfc1036d0 0x7f2bfc19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c01102640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bfc102d10 0x7f2bfc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c01903640 1 -- 172.21.15.33:0/3892821995 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bfc102d10 msgr2=0x7f2bfc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c01903640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bfc102d10 0x7f2bfc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c01903640 1 -- 172.21.15.33:0/3892821995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bfc1036d0 msgr2=0x7f2bfc19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c01903640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bfc1036d0 0x7f2bfc19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c01903640 1 -- 172.21.15.33:0/3892821995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bfc19fa80 con 0x7f2bfc101b20 2024-04-02T13:36:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c01903640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bfc101b20 0x7f2bfc197e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8000c00 tx=0x7f2bf8004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:43.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2bf2ffd640 1 -- 172.21.15.33:0/3892821995 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8043020 con 0x7f2bfc101b20 2024-04-02T13:36:43.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2bf2ffd640 1 -- 172.21.15.33:0/3892821995 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2bf80054f0 con 0x7f2bfc101b20 2024-04-02T13:36:43.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c03b67640 1 -- 172.21.15.33:0/3892821995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bfc19fd10 con 0x7f2bfc101b20 2024-04-02T13:36:43.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2bf2ffd640 1 -- 172.21.15.33:0/3892821995 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8003d70 con 0x7f2bfc101b20 2024-04-02T13:36:43.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.656+0000 7f2c03b67640 1 -- 172.21.15.33:0/3892821995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bfc1a0230 con 0x7f2bfc101b20 2024-04-02T13:36:43.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.658+0000 7f2c03b67640 1 -- 172.21.15.33:0/3892821995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bfc108e40 con 0x7f2bfc101b20 2024-04-02T13:36:43.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.659+0000 7f2bf2ffd640 1 -- 172.21.15.33:0/3892821995 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2bf8046020 con 0x7f2bfc101b20 2024-04-02T13:36:43.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.659+0000 7f2bf2ffd640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2be80a3bb0 0x7f2be80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:43.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.659+0000 7f2c01102640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2be80a3bb0 0x7f2be80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:43.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.659+0000 7f2bf2ffd640 1 -- 172.21.15.33:0/3892821995 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2bf80f2410 con 0x7f2bfc101b20 2024-04-02T13:36:43.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.660+0000 7f2c01102640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2be80a3bb0 0x7f2be80a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc199370 tx=0x7f2bec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:43.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.664+0000 7f2bf2ffd640 1 -- 172.21.15.33:0/3892821995 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2bf804d070 con 0x7f2bfc101b20 2024-04-02T13:36:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.854+0000 7f2c03b67640 1 -- 172.21.15.33:0/3892821995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2bfc0627a0 con 0x7f2bfc101b20 2024-04-02T13:36:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.856+0000 7f2bf2ffd640 1 -- 172.21.15.33:0/3892821995 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24057 (secure 0 0 0) 0x7f2bf80bebb0 con 0x7f2bfc101b20 2024-04-02T13:36:43.860 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:43.860 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:36:25.856437+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:36:43.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.858+0000 7f2c03b67640 1 -- 172.21.15.33:0/3892821995 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2be80a3bb0 msgr2=0x7f2be80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.858+0000 7f2c03b67640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2be80a3bb0 0x7f2be80a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc199370 tx=0x7f2bec008040 comp rx=0 tx=0).stop 2024-04-02T13:36:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.858+0000 7f2c03b67640 1 -- 172.21.15.33:0/3892821995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bfc101b20 msgr2=0x7f2bfc197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.858+0000 7f2c03b67640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bfc101b20 0x7f2bfc197e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8000c00 tx=0x7f2bf8004e90 comp rx=0 tx=0).stop 2024-04-02T13:36:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.858+0000 7f2c03b67640 1 -- 172.21.15.33:0/3892821995 shutdown_connections 2024-04-02T13:36:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.858+0000 7f2c03b67640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2be80a3bb0 0x7f2be80a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.858+0000 7f2c03b67640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bfc1036d0 0x7f2bfc19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.858+0000 7f2c03b67640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bfc102d10 0x7f2bfc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.858+0000 7f2c03b67640 1 --2- 172.21.15.33:0/3892821995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bfc101b20 0x7f2bfc197e60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.858+0000 7f2c03b67640 1 -- 172.21.15.33:0/3892821995 >> 172.21.15.33:0/3892821995 conn(0x7f2bfc0fd0b0 msgr2=0x7f2bfc106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.858+0000 7f2c03b67640 1 -- 172.21.15.33:0/3892821995 shutdown_connections 2024-04-02T13:36:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.858+0000 7f2c03b67640 1 -- 172.21.15.33:0/3892821995 wait complete. 2024-04-02T13:36:43.871 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:43 smithi033 ceph-mon[32390]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:36:43.873 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-02T13:36:43.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:43 smithi145 ceph-mon[35646]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:36:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.975+0000 7f8967270640 1 -- 172.21.15.33:0/2452983780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8960104ed0 msgr2=0x7f89601072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.975+0000 7f8967270640 1 --2- 172.21.15.33:0/2452983780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8960104ed0 0x7f89601072c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8950009900 tx=0x7f895002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.975+0000 7f8967270640 1 -- 172.21.15.33:0/2452983780 shutdown_connections 2024-04-02T13:36:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.975+0000 7f8967270640 1 --2- 172.21.15.33:0/2452983780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8960107800 0x7f8960109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.975+0000 7f8967270640 1 --2- 172.21.15.33:0/2452983780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8960104ed0 0x7f89601072c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.975+0000 7f8967270640 1 --2- 172.21.15.33:0/2452983780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601025a0 0x7f8960104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.975+0000 7f8967270640 1 -- 172.21.15.33:0/2452983780 >> 172.21.15.33:0/2452983780 conn(0x7f89600fbfd0 msgr2=0x7f89600fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.975+0000 7f8967270640 1 -- 172.21.15.33:0/2452983780 shutdown_connections 2024-04-02T13:36:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.975+0000 7f8967270640 1 -- 172.21.15.33:0/2452983780 wait complete. 2024-04-02T13:36:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f8967270640 1 Processor -- start 2024-04-02T13:36:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f8967270640 1 -- start start 2024-04-02T13:36:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f8967270640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601025a0 0x7f8960197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f8967270640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8960104ed0 0x7f8960198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f8967270640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8960107800 0x7f896019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f8967270640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f896010d760 con 0x7f8960107800 2024-04-02T13:36:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f8967270640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f896010d5e0 con 0x7f8960104ed0 2024-04-02T13:36:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f8967270640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f896010d8e0 con 0x7f89601025a0 2024-04-02T13:36:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f896580d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8960107800 0x7f896019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f896580d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8960107800 0x7f896019f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35386/0 (socket says 172.21.15.33:35386) 2024-04-02T13:36:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f896580d640 1 -- 172.21.15.33:0/3907040092 learned_addr learned my addr 172.21.15.33:0/3907040092 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f896500c640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601025a0 0x7f8960197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f896480b640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8960104ed0 0x7f8960198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f896580d640 1 -- 172.21.15.33:0/3907040092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601025a0 msgr2=0x7f8960197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f896580d640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601025a0 0x7f8960197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f896580d640 1 -- 172.21.15.33:0/3907040092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8960104ed0 msgr2=0x7f8960198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f896580d640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8960104ed0 0x7f8960198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f896580d640 1 -- 172.21.15.33:0/3907040092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896019f950 con 0x7f8960107800 2024-04-02T13:36:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.976+0000 7f896480b640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8960104ed0 0x7f8960198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.977+0000 7f896580d640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8960107800 0x7f896019f2b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f895800cce0 tx=0x7f895800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.977+0000 7f89567fc640 1 -- 172.21.15.33:0/3907040092 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895800f450 con 0x7f8960107800 2024-04-02T13:36:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.977+0000 7f89567fc640 1 -- 172.21.15.33:0/3907040092 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f895800fac0 con 0x7f8960107800 2024-04-02T13:36:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.977+0000 7f8967270640 1 -- 172.21.15.33:0/3907040092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896019fc40 con 0x7f8960107800 2024-04-02T13:36:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.977+0000 7f89567fc640 1 -- 172.21.15.33:0/3907040092 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958018b10 con 0x7f8960107800 2024-04-02T13:36:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.977+0000 7f8967270640 1 -- 172.21.15.33:0/3907040092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89601a0140 con 0x7f8960107800 2024-04-02T13:36:43.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.978+0000 7f8967270640 1 -- 172.21.15.33:0/3907040092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89600fdac0 con 0x7f8960107800 2024-04-02T13:36:43.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.979+0000 7f89567fc640 1 -- 172.21.15.33:0/3907040092 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f895800fc70 con 0x7f8960107800 2024-04-02T13:36:43.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.979+0000 7f89567fc640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f894c0a3bb0 0x7f894c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:43.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.980+0000 7f896500c640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f894c0a3bb0 0x7f894c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:43.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.980+0000 7f89567fc640 1 -- 172.21.15.33:0/3907040092 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f89580d0be0 con 0x7f8960107800 2024-04-02T13:36:43.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.980+0000 7f896500c640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f894c0a3bb0 0x7f894c0a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8960199210 tx=0x7f895c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:43.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:43.984+0000 7f89567fc640 1 -- 172.21.15.33:0/3907040092 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8958014030 con 0x7f8960107800 2024-04-02T13:36:43.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:43 smithi050 ceph-mon[35524]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:36:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.209+0000 7f8967270640 1 -- 172.21.15.33:0/3907040092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8960062730 con 0x7f8960107800 2024-04-02T13:36:44.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.527+0000 7f89567fc640 1 -- 172.21.15.33:0/3907040092 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f895809d320 con 0x7f8960107800 2024-04-02T13:36:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.530+0000 7f8967270640 1 -- 172.21.15.33:0/3907040092 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f894c0a3bb0 msgr2=0x7f894c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.530+0000 7f8967270640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f894c0a3bb0 0x7f894c0a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8960199210 tx=0x7f895c00a2d0 comp rx=0 tx=0).stop 2024-04-02T13:36:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.530+0000 7f8967270640 1 -- 172.21.15.33:0/3907040092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8960107800 msgr2=0x7f896019f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.530+0000 7f8967270640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8960107800 0x7f896019f2b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f895800cce0 tx=0x7f895800a620 comp rx=0 tx=0).stop 2024-04-02T13:36:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.530+0000 7f8967270640 1 -- 172.21.15.33:0/3907040092 shutdown_connections 2024-04-02T13:36:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.530+0000 7f8967270640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f894c0a3bb0 0x7f894c0a6070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.530+0000 7f8967270640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8960107800 0x7f896019f2b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.530+0000 7f8967270640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8960104ed0 0x7f8960198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:44.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.530+0000 7f8967270640 1 --2- 172.21.15.33:0/3907040092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89601025a0 0x7f8960197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.530+0000 7f8967270640 1 -- 172.21.15.33:0/3907040092 >> 172.21.15.33:0/3907040092 conn(0x7f89600fbfd0 msgr2=0x7f8960102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.530+0000 7f8967270640 1 -- 172.21.15.33:0/3907040092 shutdown_connections 2024-04-02T13:36:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.530+0000 7f8967270640 1 -- 172.21.15.33:0/3907040092 wait complete. 2024-04-02T13:36:44.545 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-02T13:36:44.546 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-02T13:36:44.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.649+0000 7f0e497af640 1 -- 172.21.15.33:0/1256707756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e44100a00 msgr2=0x7f0e44100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:44.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.649+0000 7f0e497af640 1 --2- 172.21.15.33:0/1256707756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e44100a00 0x7f0e44100e20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009900 tx=0x7f0e3402bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:44.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.649+0000 7f0e497af640 1 -- 172.21.15.33:0/1256707756 shutdown_connections 2024-04-02T13:36:44.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.649+0000 7f0e497af640 1 --2- 172.21.15.33:0/1256707756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e441025b0 0x7f0e44108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:44.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.649+0000 7f0e497af640 1 --2- 172.21.15.33:0/1256707756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e44101bf0 0x7f0e44102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:44.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.649+0000 7f0e497af640 1 --2- 172.21.15.33:0/1256707756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e44100a00 0x7f0e44100e20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:44.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.649+0000 7f0e497af640 1 -- 172.21.15.33:0/1256707756 >> 172.21.15.33:0/1256707756 conn(0x7f0e440fbf90 msgr2=0x7f0e440fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:44.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.649+0000 7f0e497af640 1 -- 172.21.15.33:0/1256707756 shutdown_connections 2024-04-02T13:36:44.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.649+0000 7f0e497af640 1 -- 172.21.15.33:0/1256707756 wait complete. 2024-04-02T13:36:44.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e497af640 1 Processor -- start 2024-04-02T13:36:44.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e497af640 1 -- start start 2024-04-02T13:36:44.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e497af640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e44100a00 0x7f0e44196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e497af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e44101bf0 0x7f0e44197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e497af640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e441025b0 0x7f0e4419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e497af640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4410c660 con 0x7f0e44101bf0 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e497af640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4410c4e0 con 0x7f0e441025b0 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e497af640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4410c7e0 con 0x7f0e44100a00 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e3bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e44101bf0 0x7f0e44197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e3bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e44101bf0 0x7f0e44197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35410/0 (socket says 172.21.15.33:35410) 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e3bfff640 1 -- 172.21.15.33:0/690038005 learned_addr learned my addr 172.21.15.33:0/690038005 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e437fe640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e441025b0 0x7f0e4419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e42ffd640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e44100a00 0x7f0e44196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e3bfff640 1 -- 172.21.15.33:0/690038005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e44100a00 msgr2=0x7f0e44196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e3bfff640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e44100a00 0x7f0e44196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e3bfff640 1 -- 172.21.15.33:0/690038005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e441025b0 msgr2=0x7f0e4419e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:44.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e3bfff640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e441025b0 0x7f0e4419e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:44.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e3bfff640 1 -- 172.21.15.33:0/690038005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4419e940 con 0x7f0e44101bf0 2024-04-02T13:36:44.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.650+0000 7f0e437fe640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e441025b0 0x7f0e4419e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:44.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.651+0000 7f0e3bfff640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e44101bf0 0x7f0e44197270 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00c920 tx=0x7f0e2c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:44.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.651+0000 7f0e40ff9640 1 -- 172.21.15.33:0/690038005 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c007c70 con 0x7f0e44101bf0 2024-04-02T13:36:44.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.651+0000 7f0e40ff9640 1 -- 172.21.15.33:0/690038005 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0e2c007e10 con 0x7f0e44101bf0 2024-04-02T13:36:44.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.651+0000 7f0e497af640 1 -- 172.21.15.33:0/690038005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4419ec30 con 0x7f0e44101bf0 2024-04-02T13:36:44.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.651+0000 7f0e40ff9640 1 -- 172.21.15.33:0/690038005 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c018850 con 0x7f0e44101bf0 2024-04-02T13:36:44.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.651+0000 7f0e497af640 1 -- 172.21.15.33:0/690038005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4419f180 con 0x7f0e44101bf0 2024-04-02T13:36:44.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.652+0000 7f0e497af640 1 -- 172.21.15.33:0/690038005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e44107cc0 con 0x7f0e44101bf0 2024-04-02T13:36:44.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.653+0000 7f0e40ff9640 1 -- 172.21.15.33:0/690038005 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0e2c018a30 con 0x7f0e44101bf0 2024-04-02T13:36:44.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.653+0000 7f0e40ff9640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0e300a3bb0 0x7f0e300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:44.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.653+0000 7f0e40ff9640 1 -- 172.21.15.33:0/690038005 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0e2c0d08d0 con 0x7f0e44101bf0 2024-04-02T13:36:44.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.653+0000 7f0e42ffd640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0e300a3bb0 0x7f0e300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:44.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.654+0000 7f0e42ffd640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0e300a3bb0 0x7f0e300a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e3402c710 tx=0x7f0e34005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:44.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.658+0000 7f0e40ff9640 1 -- 172.21.15.33:0/690038005 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0e2c014030 con 0x7f0e44101bf0 2024-04-02T13:36:44.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:44.888+0000 7f0e497af640 1 -- 172.21.15.33:0/690038005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f0e44062800 con 0x7f0e44101bf0 2024-04-02T13:36:44.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:44 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2793308489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:44.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:44 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3892821995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:44.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:44 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3907040092' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T13:36:44.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:44 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2793308489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:44.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:44 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3892821995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:44.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:44 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3907040092' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T13:36:45.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:44 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2793308489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:45.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:44 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3892821995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:45.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:44 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3907040092' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T13:36:45.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.529+0000 7f0e40ff9640 1 -- 172.21.15.33:0/690038005 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f0e2c01f300 con 0x7f0e44101bf0 2024-04-02T13:36:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.531+0000 7f0e497af640 1 -- 172.21.15.33:0/690038005 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0e300a3bb0 msgr2=0x7f0e300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.531+0000 7f0e497af640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0e300a3bb0 0x7f0e300a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e3402c710 tx=0x7f0e34005fd0 comp rx=0 tx=0).stop 2024-04-02T13:36:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.531+0000 7f0e497af640 1 -- 172.21.15.33:0/690038005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e44101bf0 msgr2=0x7f0e44197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.531+0000 7f0e497af640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e44101bf0 0x7f0e44197270 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00c920 tx=0x7f0e2c00cde0 comp rx=0 tx=0).stop 2024-04-02T13:36:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.532+0000 7f0e497af640 1 -- 172.21.15.33:0/690038005 shutdown_connections 2024-04-02T13:36:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.532+0000 7f0e497af640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0e300a3bb0 0x7f0e300a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.532+0000 7f0e497af640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e441025b0 0x7f0e4419e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.532+0000 7f0e497af640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e44101bf0 0x7f0e44197270 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.532+0000 7f0e497af640 1 --2- 172.21.15.33:0/690038005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e44100a00 0x7f0e44196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.532+0000 7f0e497af640 1 -- 172.21.15.33:0/690038005 >> 172.21.15.33:0/690038005 conn(0x7f0e440fbf90 msgr2=0x7f0e44104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.532+0000 7f0e497af640 1 -- 172.21.15.33:0/690038005 shutdown_connections 2024-04-02T13:36:45.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.532+0000 7f0e497af640 1 -- 172.21.15.33:0/690038005 wait complete. 2024-04-02T13:36:45.552 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-02T13:36:45.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.660+0000 7f2dfe504640 1 -- 172.21.15.33:0/4260544489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2df81025b0 msgr2=0x7f2df8108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:45.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.660+0000 7f2dfe504640 1 --2- 172.21.15.33:0/4260544489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2df81025b0 0x7f2df8108e40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009ca0 tx=0x7f2df002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:45.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.660+0000 7f2dfe504640 1 -- 172.21.15.33:0/4260544489 shutdown_connections 2024-04-02T13:36:45.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.660+0000 7f2dfe504640 1 --2- 172.21.15.33:0/4260544489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2df81025b0 0x7f2df8108e40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:45.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.660+0000 7f2dfe504640 1 --2- 172.21.15.33:0/4260544489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2df8101bf0 0x7f2df8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:45.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.660+0000 7f2dfe504640 1 --2- 172.21.15.33:0/4260544489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2df8100a00 0x7f2df8100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:45.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.660+0000 7f2dfe504640 1 -- 172.21.15.33:0/4260544489 >> 172.21.15.33:0/4260544489 conn(0x7f2df80fbf90 msgr2=0x7f2df80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:45.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.660+0000 7f2dfe504640 1 -- 172.21.15.33:0/4260544489 shutdown_connections 2024-04-02T13:36:45.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.660+0000 7f2dfe504640 1 -- 172.21.15.33:0/4260544489 wait complete. 2024-04-02T13:36:45.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.660+0000 7f2dfe504640 1 Processor -- start 2024-04-02T13:36:45.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.660+0000 7f2dfe504640 1 -- start start 2024-04-02T13:36:45.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2dfe504640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2df8100a00 0x7f2df819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:45.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2dfe504640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2df8101bf0 0x7f2df819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:45.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2dfe504640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2df81025b0 0x7f2df81a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:45.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2dfe504640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2df810c6c0 con 0x7f2df8101bf0 2024-04-02T13:36:45.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2dfe504640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2df810c540 con 0x7f2df81025b0 2024-04-02T13:36:45.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2dfe504640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2df810c840 con 0x7f2df8100a00 2024-04-02T13:36:45.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2df77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2df8101bf0 0x7f2df819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:45.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2df77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2df8101bf0 0x7f2df819a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35438/0 (socket says 172.21.15.33:35438) 2024-04-02T13:36:45.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2df77fe640 1 -- 172.21.15.33:0/2202157851 learned_addr learned my addr 172.21.15.33:0/2202157851 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:45.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2df7fff640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2df8100a00 0x7f2df819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:45.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2dfcaa1640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2df81025b0 0x7f2df81a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:45.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2df77fe640 1 -- 172.21.15.33:0/2202157851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2df8100a00 msgr2=0x7f2df819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:45.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2df77fe640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2df8100a00 0x7f2df819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:45.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2df77fe640 1 -- 172.21.15.33:0/2202157851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2df81025b0 msgr2=0x7f2df81a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:45.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2df77fe640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2df81025b0 0x7f2df81a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:45.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2df77fe640 1 -- 172.21.15.33:0/2202157851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df81a1c50 con 0x7f2df8101bf0 2024-04-02T13:36:45.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.661+0000 7f2df77fe640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2df8101bf0 0x7f2df819a570 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00b4a0 tx=0x7f2dec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:45.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.662+0000 7f2df57fa640 1 -- 172.21.15.33:0/2202157851 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec014070 con 0x7f2df8101bf0 2024-04-02T13:36:45.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.662+0000 7f2df57fa640 1 -- 172.21.15.33:0/2202157851 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2dec010ae0 con 0x7f2df8101bf0 2024-04-02T13:36:45.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.662+0000 7f2dfe504640 1 -- 172.21.15.33:0/2202157851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df81a1f40 con 0x7f2df8101bf0 2024-04-02T13:36:45.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.662+0000 7f2df57fa640 1 -- 172.21.15.33:0/2202157851 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec017c70 con 0x7f2df8101bf0 2024-04-02T13:36:45.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.662+0000 7f2dfe504640 1 -- 172.21.15.33:0/2202157851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df81a24d0 con 0x7f2df8101bf0 2024-04-02T13:36:45.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.663+0000 7f2dfe504640 1 -- 172.21.15.33:0/2202157851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df81a2760 con 0x7f2df8101bf0 2024-04-02T13:36:45.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.664+0000 7f2df57fa640 1 -- 172.21.15.33:0/2202157851 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2dec017070 con 0x7f2df8101bf0 2024-04-02T13:36:45.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.664+0000 7f2df57fa640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2de00a3c30 0x7f2de00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:45.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.665+0000 7f2df7fff640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2de00a3c30 0x7f2de00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:45.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.665+0000 7f2df57fa640 1 -- 172.21.15.33:0/2202157851 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2dec0cf8b0 con 0x7f2df8101bf0 2024-04-02T13:36:45.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.665+0000 7f2df7fff640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2de00a3c30 0x7f2de00a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2df8101a50 tx=0x7f2de800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:45.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.669+0000 7f2df57fa640 1 -- 172.21.15.33:0/2202157851 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2dec09ba00 con 0x7f2df8101bf0 2024-04-02T13:36:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:45.896+0000 7f2dfe504640 1 -- 172.21.15.33:0/2202157851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f2df81a2930 con 0x7f2df8101bf0 2024-04-02T13:36:45.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:45 smithi145 ceph-mon[35646]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:36:45.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:45 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3907040092' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T13:36:45.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:45 smithi145 ceph-mon[35646]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:45.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:45 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/690038005' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-02T13:36:45.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:45 smithi050 ceph-mon[35524]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:36:45.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:45 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3907040092' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T13:36:45.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:45 smithi050 ceph-mon[35524]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:45.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:45 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/690038005' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-02T13:36:46.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:45 smithi033 ceph-mon[32390]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:36:46.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:45 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3907040092' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T13:36:46.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:45 smithi033 ceph-mon[32390]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:46.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:45 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/690038005' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-02T13:36:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.474+0000 7f2df57fa640 1 -- 172.21.15.33:0/2202157851 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f2dec09fd00 con 0x7f2df8101bf0 2024-04-02T13:36:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.478+0000 7f2dfe504640 1 -- 172.21.15.33:0/2202157851 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2de00a3c30 msgr2=0x7f2de00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.478+0000 7f2dfe504640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2de00a3c30 0x7f2de00a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2df8101a50 tx=0x7f2de800b390 comp rx=0 tx=0).stop 2024-04-02T13:36:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.478+0000 7f2dfe504640 1 -- 172.21.15.33:0/2202157851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2df8101bf0 msgr2=0x7f2df819a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.478+0000 7f2dfe504640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2df8101bf0 0x7f2df819a570 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00b4a0 tx=0x7f2dec00b960 comp rx=0 tx=0).stop 2024-04-02T13:36:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.478+0000 7f2dfe504640 1 -- 172.21.15.33:0/2202157851 shutdown_connections 2024-04-02T13:36:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.478+0000 7f2dfe504640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2de00a3c30 0x7f2de00a60f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.478+0000 7f2dfe504640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2df81025b0 0x7f2df81a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.478+0000 7f2dfe504640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2df8101bf0 0x7f2df819a570 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.478+0000 7f2dfe504640 1 --2- 172.21.15.33:0/2202157851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2df8100a00 0x7f2df819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:46.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.478+0000 7f2dfe504640 1 -- 172.21.15.33:0/2202157851 >> 172.21.15.33:0/2202157851 conn(0x7f2df80fbf90 msgr2=0x7f2df80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:46.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.478+0000 7f2dfe504640 1 -- 172.21.15.33:0/2202157851 shutdown_connections 2024-04-02T13:36:46.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.478+0000 7f2dfe504640 1 -- 172.21.15.33:0/2202157851 wait complete. 2024-04-02T13:36:46.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-02T13:36:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.610+0000 7fa335cd3640 1 -- 172.21.15.33:0/1325633423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3301077e0 msgr2=0x7fa330109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.610+0000 7fa335cd3640 1 --2- 172.21.15.33:0/1325633423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3301077e0 0x7fa330109c30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa328009900 tx=0x7fa32802bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.610+0000 7fa335cd3640 1 -- 172.21.15.33:0/1325633423 shutdown_connections 2024-04-02T13:36:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.610+0000 7fa335cd3640 1 --2- 172.21.15.33:0/1325633423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3301077e0 0x7fa330109c30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.610+0000 7fa335cd3640 1 --2- 172.21.15.33:0/1325633423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa330104eb0 0x7fa3301072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.610+0000 7fa335cd3640 1 --2- 172.21.15.33:0/1325633423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330102580 0x7fa330104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.610+0000 7fa335cd3640 1 -- 172.21.15.33:0/1325633423 >> 172.21.15.33:0/1325633423 conn(0x7fa3300fbf90 msgr2=0x7fa3300fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.611+0000 7fa335cd3640 1 -- 172.21.15.33:0/1325633423 shutdown_connections 2024-04-02T13:36:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.611+0000 7fa335cd3640 1 -- 172.21.15.33:0/1325633423 wait complete. 2024-04-02T13:36:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.611+0000 7fa335cd3640 1 Processor -- start 2024-04-02T13:36:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.611+0000 7fa335cd3640 1 -- start start 2024-04-02T13:36:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.611+0000 7fa335cd3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330102580 0x7fa330197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.611+0000 7fa335cd3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa330104eb0 0x7fa330198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.611+0000 7fa335cd3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3301077e0 0x7fa33019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.611+0000 7fa335cd3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa33010d780 con 0x7fa3301077e0 2024-04-02T13:36:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.611+0000 7fa335cd3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa33010d600 con 0x7fa330102580 2024-04-02T13:36:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.611+0000 7fa335cd3640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa33010d900 con 0x7fa330104eb0 2024-04-02T13:36:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3301077e0 0x7fa33019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330102580 0x7fa330197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3301077e0 0x7fa33019f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35464/0 (socket says 172.21.15.33:35464) 2024-04-02T13:36:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32ffff640 1 -- 172.21.15.33:0/305729841 learned_addr learned my addr 172.21.15.33:0/305729841 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32effd640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa330104eb0 0x7fa330198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32ffff640 1 -- 172.21.15.33:0/305729841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa330104eb0 msgr2=0x7fa330198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32ffff640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa330104eb0 0x7fa330198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32ffff640 1 -- 172.21.15.33:0/305729841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330102580 msgr2=0x7fa330197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32ffff640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330102580 0x7fa330197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32ffff640 1 -- 172.21.15.33:0/305729841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33019fa40 con 0x7fa3301077e0 2024-04-02T13:36:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32effd640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa330104eb0 0x7fa330198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32f7fe640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330102580 0x7fa330197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32ffff640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3301077e0 0x7fa33019f3d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa328036040 tx=0x7fa328005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32cff9640 1 -- 172.21.15.33:0/305729841 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328043020 con 0x7fa3301077e0 2024-04-02T13:36:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32cff9640 1 -- 172.21.15.33:0/305729841 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa328005470 con 0x7fa3301077e0 2024-04-02T13:36:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa335cd3640 1 -- 172.21.15.33:0/305729841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33019fcd0 con 0x7fa3301077e0 2024-04-02T13:36:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa32cff9640 1 -- 172.21.15.33:0/305729841 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328003b70 con 0x7fa3301077e0 2024-04-02T13:36:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.612+0000 7fa335cd3640 1 -- 172.21.15.33:0/305729841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3301a01d0 con 0x7fa3301077e0 2024-04-02T13:36:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.613+0000 7fa335cd3640 1 -- 172.21.15.33:0/305729841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3300fda80 con 0x7fa3301077e0 2024-04-02T13:36:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.614+0000 7fa32cff9640 1 -- 172.21.15.33:0/305729841 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa328046020 con 0x7fa3301077e0 2024-04-02T13:36:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.615+0000 7fa32cff9640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa3180a3bb0 0x7fa3180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.615+0000 7fa32f7fe640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa3180a3bb0 0x7fa3180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.615+0000 7fa32cff9640 1 -- 172.21.15.33:0/305729841 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa3280f2380 con 0x7fa3301077e0 2024-04-02T13:36:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.616+0000 7fa32f7fe640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa3180a3bb0 0x7fa3180a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa320005fd0 tx=0x7fa320009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.621+0000 7fa32cff9640 1 -- 172.21.15.33:0/305729841 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa32804d070 con 0x7fa3301077e0 2024-04-02T13:36:46.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T13:36:46.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/690038005' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-02T13:36:46.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T13:36:46.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:46.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-02T13:36:46.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-02T13:36:46.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2202157851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T13:36:46.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2202157851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T13:36:46.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] up:active 2024-04-02T13:36:46.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T13:36:46.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T13:36:46.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: Cluster is now healthy 2024-04-02T13:36:46.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2024-04-02T13:36:46.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2024-04-02T13:36:46.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:46 smithi033 ceph-mon[32390]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-02T13:36:46.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:46.846+0000 7fa335cd3640 1 -- 172.21.15.33:0/305729841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa330062700 con 0x7fa3301077e0 2024-04-02T13:36:46.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T13:36:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/690038005' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-02T13:36:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T13:36:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-02T13:36:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-02T13:36:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2202157851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T13:36:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2202157851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T13:36:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] up:active 2024-04-02T13:36:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T13:36:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T13:36:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: Cluster is now healthy 2024-04-02T13:36:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2024-04-02T13:36:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2024-04-02T13:36:46.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:46 smithi145 ceph-mon[35646]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-02T13:36:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T13:36:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/690038005' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-02T13:36:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T13:36:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:36:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-02T13:36:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-02T13:36:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2202157851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T13:36:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2202157851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T13:36:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] up:active 2024-04-02T13:36:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T13:36:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T13:36:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: Cluster is now healthy 2024-04-02T13:36:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2024-04-02T13:36:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2024-04-02T13:36:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:46 smithi050 ceph-mon[35524]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-02T13:36:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.485+0000 7fa32cff9640 1 -- 172.21.15.33:0/305729841 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7fa3280beb20 con 0x7fa3301077e0 2024-04-02T13:36:47.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.488+0000 7fa335cd3640 1 -- 172.21.15.33:0/305729841 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa3180a3bb0 msgr2=0x7fa3180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.488+0000 7fa335cd3640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa3180a3bb0 0x7fa3180a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa320005fd0 tx=0x7fa320009500 comp rx=0 tx=0).stop 2024-04-02T13:36:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.488+0000 7fa335cd3640 1 -- 172.21.15.33:0/305729841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3301077e0 msgr2=0x7fa33019f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.488+0000 7fa335cd3640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3301077e0 0x7fa33019f3d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa328036040 tx=0x7fa328005de0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.488+0000 7fa335cd3640 1 -- 172.21.15.33:0/305729841 shutdown_connections 2024-04-02T13:36:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.488+0000 7fa335cd3640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa3180a3bb0 0x7fa3180a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.488+0000 7fa335cd3640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3301077e0 0x7fa33019f3d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.488+0000 7fa335cd3640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa330104eb0 0x7fa330198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.488+0000 7fa335cd3640 1 --2- 172.21.15.33:0/305729841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa330102580 0x7fa330197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.488+0000 7fa335cd3640 1 -- 172.21.15.33:0/305729841 >> 172.21.15.33:0/305729841 conn(0x7fa3300fbf90 msgr2=0x7fa330102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.488+0000 7fa335cd3640 1 -- 172.21.15.33:0/305729841 shutdown_connections 2024-04-02T13:36:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.488+0000 7fa335cd3640 1 -- 172.21.15.33:0/305729841 wait complete. 2024-04-02T13:36:47.505 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-02T13:36:47.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-02T13:36:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.616+0000 7f9f326c8640 1 -- 172.21.15.33:0/3012118235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f2c101c00 msgr2=0x7f9f2c102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.617+0000 7f9f326c8640 1 --2- 172.21.15.33:0/3012118235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f2c101c00 0x7f9f2c102080 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009900 tx=0x7f9f2002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.617+0000 7f9f326c8640 1 -- 172.21.15.33:0/3012118235 shutdown_connections 2024-04-02T13:36:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.617+0000 7f9f326c8640 1 --2- 172.21.15.33:0/3012118235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f2c1025c0 0x7f9f2c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.617+0000 7f9f326c8640 1 --2- 172.21.15.33:0/3012118235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f2c101c00 0x7f9f2c102080 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.617+0000 7f9f326c8640 1 --2- 172.21.15.33:0/3012118235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c100a10 0x7f9f2c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.617+0000 7f9f326c8640 1 -- 172.21.15.33:0/3012118235 >> 172.21.15.33:0/3012118235 conn(0x7f9f2c0fbfa0 msgr2=0x7f9f2c0fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.617+0000 7f9f326c8640 1 -- 172.21.15.33:0/3012118235 shutdown_connections 2024-04-02T13:36:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.617+0000 7f9f326c8640 1 -- 172.21.15.33:0/3012118235 wait complete. 2024-04-02T13:36:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.617+0000 7f9f326c8640 1 Processor -- start 2024-04-02T13:36:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.617+0000 7f9f326c8640 1 -- start start 2024-04-02T13:36:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.617+0000 7f9f326c8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f2c100a10 0x7f9f2c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.617+0000 7f9f326c8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c101c00 0x7f9f2c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f326c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f2c1025c0 0x7f9f2c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f326c8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c10d780 con 0x7f9f2c1025c0 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f326c8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c10d600 con 0x7f9f2c101c00 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f326c8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c10d900 con 0x7f9f2c100a10 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f30c65640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f2c1025c0 0x7f9f2c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f30c65640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f2c1025c0 0x7f9f2c19f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35482/0 (socket says 172.21.15.33:35482) 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f30c65640 1 -- 172.21.15.33:0/4012004760 learned_addr learned my addr 172.21.15.33:0/4012004760 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f2b7fe640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c101c00 0x7f9f2c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f2bfff640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f2c100a10 0x7f9f2c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f30c65640 1 -- 172.21.15.33:0/4012004760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f2c100a10 msgr2=0x7f9f2c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f30c65640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f2c100a10 0x7f9f2c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f30c65640 1 -- 172.21.15.33:0/4012004760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c101c00 msgr2=0x7f9f2c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f30c65640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c101c00 0x7f9f2c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f30c65640 1 -- 172.21.15.33:0/4012004760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c19faa0 con 0x7f9f2c1025c0 2024-04-02T13:36:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f2b7fe640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c101c00 0x7f9f2c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f30c65640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f2c1025c0 0x7f9f2c19f430 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400cad0 tx=0x7f9f2400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f297fa640 1 -- 172.21.15.33:0/4012004760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2400ae20 con 0x7f9f2c1025c0 2024-04-02T13:36:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f297fa640 1 -- 172.21.15.33:0/4012004760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9f24005020 con 0x7f9f2c1025c0 2024-04-02T13:36:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f326c8640 1 -- 172.21.15.33:0/4012004760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2c19fd90 con 0x7f9f2c1025c0 2024-04-02T13:36:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f297fa640 1 -- 172.21.15.33:0/4012004760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2400f690 con 0x7f9f2c1025c0 2024-04-02T13:36:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.618+0000 7f9f326c8640 1 -- 172.21.15.33:0/4012004760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2c1a0250 con 0x7f9f2c1025c0 2024-04-02T13:36:47.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.619+0000 7f9f326c8640 1 -- 172.21.15.33:0/4012004760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f2c107db0 con 0x7f9f2c1025c0 2024-04-02T13:36:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.620+0000 7f9f297fa640 1 -- 172.21.15.33:0/4012004760 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9f240051d0 con 0x7f9f2c1025c0 2024-04-02T13:36:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.621+0000 7f9f297fa640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9f140a3bb0 0x7f9f140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.621+0000 7f9f2bfff640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9f140a3bb0 0x7f9f140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.621+0000 7f9f297fa640 1 -- 172.21.15.33:0/4012004760 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9f240d0600 con 0x7f9f2c1025c0 2024-04-02T13:36:47.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.621+0000 7f9f2bfff640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9f140a3bb0 0x7f9f140a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c101a60 tx=0x7f9f1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.625+0000 7f9f297fa640 1 -- 172.21.15.33:0/4012004760 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9f24014030 con 0x7f9f2c1025c0 2024-04-02T13:36:47.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.842+0000 7f9f326c8640 1 -- 172.21.15.33:0/4012004760 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9f2c061f40 con 0x7f9f140a3bb0 2024-04-02T13:36:47.850 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:47 smithi033 ceph-mon[32390]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 0 op/s 2024-04-02T13:36:47.850 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:47 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/305729841' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T13:36:47.850 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:47 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/305729841' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T13:36:47.850 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:47 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] up:active 2024-04-02T13:36:47.850 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:47 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-04-02T13:36:47.850 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:47 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:36:47.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.868+0000 7f9f297fa640 1 -- 172.21.15.33:0/4012004760 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9f2c061f40 con 0x7f9f140a3bb0 2024-04-02T13:36:47.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.870+0000 7f9f326c8640 1 -- 172.21.15.33:0/4012004760 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9f140a3bb0 msgr2=0x7f9f140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:47.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.870+0000 7f9f326c8640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9f140a3bb0 0x7f9f140a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c101a60 tx=0x7f9f1c005f90 comp rx=0 tx=0).stop 2024-04-02T13:36:47.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.870+0000 7f9f326c8640 1 -- 172.21.15.33:0/4012004760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f2c1025c0 msgr2=0x7f9f2c19f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:47.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.870+0000 7f9f326c8640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f2c1025c0 0x7f9f2c19f430 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400cad0 tx=0x7f9f2400cf90 comp rx=0 tx=0).stop 2024-04-02T13:36:47.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.870+0000 7f9f326c8640 1 -- 172.21.15.33:0/4012004760 shutdown_connections 2024-04-02T13:36:47.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.870+0000 7f9f326c8640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9f140a3bb0 0x7f9f140a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.870+0000 7f9f326c8640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f2c1025c0 0x7f9f2c19f430 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.870+0000 7f9f326c8640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f2c101c00 0x7f9f2c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.870+0000 7f9f326c8640 1 --2- 172.21.15.33:0/4012004760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f2c100a10 0x7f9f2c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.870+0000 7f9f326c8640 1 -- 172.21.15.33:0/4012004760 >> 172.21.15.33:0/4012004760 conn(0x7f9f2c0fbfa0 msgr2=0x7f9f2c104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:47.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.870+0000 7f9f326c8640 1 -- 172.21.15.33:0/4012004760 shutdown_connections 2024-04-02T13:36:47.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.870+0000 7f9f326c8640 1 -- 172.21.15.33:0/4012004760 wait complete. 2024-04-02T13:36:47.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-02T13:36:47.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:47 smithi145 ceph-mon[35646]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 0 op/s 2024-04-02T13:36:47.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:47 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/305729841' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T13:36:47.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:47 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/305729841' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T13:36:47.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:47 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] up:active 2024-04-02T13:36:47.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:47 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-04-02T13:36:47.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:47 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:36:47.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.989+0000 7f7beafba640 1 -- 172.21.15.33:0/3906327882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be4100a10 msgr2=0x7f7be4100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:47.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.989+0000 7f7beafba640 1 --2- 172.21.15.33:0/3906327882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be4100a10 0x7f7be4100e30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009900 tx=0x7f7bd802bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:47.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.989+0000 7f7beafba640 1 -- 172.21.15.33:0/3906327882 shutdown_connections 2024-04-02T13:36:47.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.989+0000 7f7beafba640 1 --2- 172.21.15.33:0/3906327882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7be41025c0 0x7f7be4108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.989+0000 7f7beafba640 1 --2- 172.21.15.33:0/3906327882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7be4101c00 0x7f7be4102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.989+0000 7f7beafba640 1 --2- 172.21.15.33:0/3906327882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be4100a10 0x7f7be4100e30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.989+0000 7f7beafba640 1 -- 172.21.15.33:0/3906327882 >> 172.21.15.33:0/3906327882 conn(0x7f7be40fbfa0 msgr2=0x7f7be40fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:47.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.989+0000 7f7beafba640 1 -- 172.21.15.33:0/3906327882 shutdown_connections 2024-04-02T13:36:47.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.989+0000 7f7beafba640 1 -- 172.21.15.33:0/3906327882 wait complete. 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7beafba640 1 Processor -- start 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7beafba640 1 -- start start 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7beafba640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7be4100a10 0x7f7be406ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7beafba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be4101c00 0x7f7be419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7beafba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7be41025c0 0x7f7be419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7beafba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7be410c670 con 0x7f7be4101c00 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7beafba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7be410c4f0 con 0x7f7be41025c0 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7beafba640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7be410c7f0 con 0x7f7be4100a10 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7be3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be4101c00 0x7f7be419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7be3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be4101c00 0x7f7be419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35524/0 (socket says 172.21.15.33:35524) 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7be3fff640 1 -- 172.21.15.33:0/3936786703 learned_addr learned my addr 172.21.15.33:0/3936786703 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7be9557640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7be41025c0 0x7f7be419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7be8d56640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7be4100a10 0x7f7be406ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7be3fff640 1 -- 172.21.15.33:0/3936786703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7be4100a10 msgr2=0x7f7be406ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:47.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7be3fff640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7be4100a10 0x7f7be406ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7be3fff640 1 -- 172.21.15.33:0/3936786703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7be41025c0 msgr2=0x7f7be419f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:47.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7be3fff640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7be41025c0 0x7f7be419f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:47.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7be3fff640 1 -- 172.21.15.33:0/3936786703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be419fae0 con 0x7f7be4101c00 2024-04-02T13:36:47.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.990+0000 7f7be3fff640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be4101c00 0x7f7be419cbb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000d770 tx=0x7f7bd000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:47.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.991+0000 7f7be1ffb640 1 -- 172.21.15.33:0/3936786703 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0017070 con 0x7f7be4101c00 2024-04-02T13:36:47.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.991+0000 7f7beafba640 1 -- 172.21.15.33:0/3936786703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be419fd70 con 0x7f7be4101c00 2024-04-02T13:36:47.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.991+0000 7f7be1ffb640 1 -- 172.21.15.33:0/3936786703 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7bd00054f0 con 0x7f7be4101c00 2024-04-02T13:36:47.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.991+0000 7f7be1ffb640 1 -- 172.21.15.33:0/3936786703 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0004d60 con 0x7f7be4101c00 2024-04-02T13:36:47.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.991+0000 7f7beafba640 1 -- 172.21.15.33:0/3936786703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be41a0210 con 0x7f7be4101c00 2024-04-02T13:36:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:47 smithi050 ceph-mon[35524]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 0 op/s 2024-04-02T13:36:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:47 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/305729841' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T13:36:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:47 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/305729841' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T13:36:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:47 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] up:active 2024-04-02T13:36:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:47 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-04-02T13:36:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:47 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:36:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.992+0000 7f7beafba640 1 -- 172.21.15.33:0/3936786703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be4107e20 con 0x7f7be4101c00 2024-04-02T13:36:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.993+0000 7f7be1ffb640 1 -- 172.21.15.33:0/3936786703 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7bd0004f00 con 0x7f7be4101c00 2024-04-02T13:36:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.993+0000 7f7be1ffb640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bd40a3bb0 0x7f7bd40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.994+0000 7f7be1ffb640 1 -- 172.21.15.33:0/3936786703 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7bd00d0490 con 0x7f7be4101c00 2024-04-02T13:36:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.994+0000 7f7be8d56640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bd40a3bb0 0x7f7bd40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.994+0000 7f7be8d56640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bd40a3bb0 0x7f7bd40a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8006010 tx=0x7f7bd8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:48.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:47.997+0000 7f7be1ffb640 1 -- 172.21.15.33:0/3936786703 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7bd009cc30 con 0x7f7be4101c00 2024-04-02T13:36:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.211+0000 7f7beafba640 1 -- 172.21.15.33:0/3936786703 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7be4061fd0 con 0x7f7bd40a3bb0 2024-04-02T13:36:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.235+0000 7f7be1ffb640 1 -- 172.21.15.33:0/3936786703 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7be4061fd0 con 0x7f7bd40a3bb0 2024-04-02T13:36:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.237+0000 7f7beafba640 1 -- 172.21.15.33:0/3936786703 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bd40a3bb0 msgr2=0x7f7bd40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.237+0000 7f7beafba640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bd40a3bb0 0x7f7bd40a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8006010 tx=0x7f7bd8000bc0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.237+0000 7f7beafba640 1 -- 172.21.15.33:0/3936786703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be4101c00 msgr2=0x7f7be419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.237+0000 7f7beafba640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be4101c00 0x7f7be419cbb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000d770 tx=0x7f7bd000dc30 comp rx=0 tx=0).stop 2024-04-02T13:36:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.237+0000 7f7beafba640 1 -- 172.21.15.33:0/3936786703 shutdown_connections 2024-04-02T13:36:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.237+0000 7f7beafba640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bd40a3bb0 0x7f7bd40a6070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.237+0000 7f7beafba640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7be41025c0 0x7f7be419f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.237+0000 7f7beafba640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be4101c00 0x7f7be419cbb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.237+0000 7f7beafba640 1 --2- 172.21.15.33:0/3936786703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7be4100a10 0x7f7be406ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.237+0000 7f7beafba640 1 -- 172.21.15.33:0/3936786703 >> 172.21.15.33:0/3936786703 conn(0x7f7be40fbfa0 msgr2=0x7f7be4104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.237+0000 7f7beafba640 1 -- 172.21.15.33:0/3936786703 shutdown_connections 2024-04-02T13:36:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.237+0000 7f7beafba640 1 -- 172.21.15.33:0/3936786703 wait complete. 2024-04-02T13:36:48.250 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:36:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:36:47.846+0000 7f84ae749700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:36:48.250 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:36:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:36:47.846+0000 7f84ae749700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:36:48.250 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:36:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:36:47.846+0000 7f84ae749700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:36:48.250 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:36:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:36:47.846+0000 7f84ae749700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:36:48.250 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:36:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:36:47.846+0000 7f84ae749700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:36:48.251 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-02T13:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.356+0000 7f3cc7532640 1 -- 172.21.15.33:0/1287556779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc00ffe00 msgr2=0x7f3cc0100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.356+0000 7f3cc7532640 1 --2- 172.21.15.33:0/1287556779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc00ffe00 0x7f3cc0100220 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009930 tx=0x7f3cbc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.356+0000 7f3cc7532640 1 -- 172.21.15.33:0/1287556779 shutdown_connections 2024-04-02T13:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.356+0000 7f3cc7532640 1 --2- 172.21.15.33:0/1287556779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cc0103830 0x7f3cc010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.356+0000 7f3cc7532640 1 --2- 172.21.15.33:0/1287556779 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3cc0102e70 0x7f3cc01032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.356+0000 7f3cc7532640 1 --2- 172.21.15.33:0/1287556779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc00ffe00 0x7f3cc0100220 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.356+0000 7f3cc7532640 1 -- 172.21.15.33:0/1287556779 >> 172.21.15.33:0/1287556779 conn(0x7f3cc00fb6f0 msgr2=0x7f3cc00fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.356+0000 7f3cc7532640 1 -- 172.21.15.33:0/1287556779 shutdown_connections 2024-04-02T13:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.356+0000 7f3cc7532640 1 -- 172.21.15.33:0/1287556779 wait complete. 2024-04-02T13:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.356+0000 7f3cc7532640 1 Processor -- start 2024-04-02T13:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.356+0000 7f3cc7532640 1 -- start start 2024-04-02T13:36:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc7532640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc00ffe00 0x7f3cc0197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc7532640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3cc0102e70 0x7f3cc0198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc7532640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cc0103830 0x7f3cc019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc7532640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc010d7e0 con 0x7f3cc00ffe00 2024-04-02T13:36:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc7532640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc010d660 con 0x7f3cc0103830 2024-04-02T13:36:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc7532640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc010d960 con 0x7f3cc0102e70 2024-04-02T13:36:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc52ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc00ffe00 0x7f3cc0197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc52ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc00ffe00 0x7f3cc0197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35536/0 (socket says 172.21.15.33:35536) 2024-04-02T13:36:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc52ce640 1 -- 172.21.15.33:0/1232394106 learned_addr learned my addr 172.21.15.33:0/1232394106 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc5acf640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cc0103830 0x7f3cc019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc52ce640 1 -- 172.21.15.33:0/1232394106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3cc0102e70 msgr2=0x7f3cc0198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc4acd640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3cc0102e70 0x7f3cc0198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc52ce640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3cc0102e70 0x7f3cc0198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc52ce640 1 -- 172.21.15.33:0/1232394106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cc0103830 msgr2=0x7f3cc019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc52ce640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cc0103830 0x7f3cc019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc52ce640 1 -- 172.21.15.33:0/1232394106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc019f9f0 con 0x7f3cc00ffe00 2024-04-02T13:36:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc52ce640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc00ffe00 0x7f3cc0197dd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc000c00 tx=0x7f3cbc004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cb67fc640 1 -- 172.21.15.33:0/1232394106 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc043020 con 0x7f3cc00ffe00 2024-04-02T13:36:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc7532640 1 -- 172.21.15.33:0/1232394106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc019fc80 con 0x7f3cc00ffe00 2024-04-02T13:36:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.357+0000 7f3cc7532640 1 -- 172.21.15.33:0/1232394106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc01a01d0 con 0x7f3cc00ffe00 2024-04-02T13:36:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.358+0000 7f3cb67fc640 1 -- 172.21.15.33:0/1232394106 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3cbc004c70 con 0x7f3cc00ffe00 2024-04-02T13:36:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.358+0000 7f3cb67fc640 1 -- 172.21.15.33:0/1232394106 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc003850 con 0x7f3cc00ffe00 2024-04-02T13:36:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.358+0000 7f3cc7532640 1 -- 172.21.15.33:0/1232394106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca40052c0 con 0x7f3cc00ffe00 2024-04-02T13:36:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.359+0000 7f3cb67fc640 1 -- 172.21.15.33:0/1232394106 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3cbc046020 con 0x7f3cc00ffe00 2024-04-02T13:36:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.360+0000 7f3cb67fc640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3cac0a3c30 0x7f3cac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.360+0000 7f3cc4acd640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3cac0a3c30 0x7f3cac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.360+0000 7f3cb67fc640 1 -- 172.21.15.33:0/1232394106 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3cbc0f2720 con 0x7f3cc00ffe00 2024-04-02T13:36:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.361+0000 7f3cc4acd640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3cac0a3c30 0x7f3cac0a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3cc01992e0 tx=0x7f3cb0005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.364+0000 7f3cb67fc640 1 -- 172.21.15.33:0/1232394106 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3cbc04d030 con 0x7f3cc00ffe00 2024-04-02T13:36:48.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.562+0000 7f3cb67fc640 1 -- 172.21.15.33:0/1232394106 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3cbc0bae30 con 0x7f3cc00ffe00 2024-04-02T13:36:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.587+0000 7f3cc7532640 1 -- 172.21.15.33:0/1232394106 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f3ca4000bf0 con 0x7f3cac0a3c30 2024-04-02T13:36:48.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.594+0000 7f3cb67fc640 1 -- 172.21.15.33:0/1232394106 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3ca4000bf0 con 0x7f3cac0a3c30 2024-04-02T13:36:48.596 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_0/bd756cb6-7314-4c9e-8eb1-a4a1e6d486b4 2024-04-02T13:36:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.596+0000 7f3cc7532640 1 -- 172.21.15.33:0/1232394106 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3cac0a3c30 msgr2=0x7f3cac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.596+0000 7f3cc7532640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3cac0a3c30 0x7f3cac0a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3cc01992e0 tx=0x7f3cb0005cc0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.596+0000 7f3cc7532640 1 -- 172.21.15.33:0/1232394106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc00ffe00 msgr2=0x7f3cc0197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.596+0000 7f3cc7532640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc00ffe00 0x7f3cc0197dd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc000c00 tx=0x7f3cbc004e20 comp rx=0 tx=0).stop 2024-04-02T13:36:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.596+0000 7f3cc7532640 1 -- 172.21.15.33:0/1232394106 shutdown_connections 2024-04-02T13:36:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.596+0000 7f3cc7532640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3cac0a3c30 0x7f3cac0a60f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.596+0000 7f3cc7532640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3cc0103830 0x7f3cc019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.596+0000 7f3cc7532640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3cc0102e70 0x7f3cc0198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.596+0000 7f3cc7532640 1 --2- 172.21.15.33:0/1232394106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc00ffe00 0x7f3cc0197dd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.596+0000 7f3cc7532640 1 -- 172.21.15.33:0/1232394106 >> 172.21.15.33:0/1232394106 conn(0x7f3cc00fb6f0 msgr2=0x7f3cc0106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.596+0000 7f3cc7532640 1 -- 172.21.15.33:0/1232394106 shutdown_connections 2024-04-02T13:36:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.596+0000 7f3cc7532640 1 -- 172.21.15.33:0/1232394106 wait complete. 2024-04-02T13:36:48.609 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-02T13:36:48.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.715+0000 7f11539d4640 1 -- 172.21.15.33:0/1621096098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f114c101b10 msgr2=0x7f114c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:48.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.715+0000 7f11539d4640 1 --2- 172.21.15.33:0/1621096098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f114c101b10 0x7f114c101f30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1144009900 tx=0x7f114402bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:48.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.715+0000 7f11539d4640 1 -- 172.21.15.33:0/1621096098 shutdown_connections 2024-04-02T13:36:48.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.715+0000 7f11539d4640 1 --2- 172.21.15.33:0/1621096098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f114c1036c0 0x7f114c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.715+0000 7f11539d4640 1 --2- 172.21.15.33:0/1621096098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114c102d00 0x7f114c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.715+0000 7f11539d4640 1 --2- 172.21.15.33:0/1621096098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f114c101b10 0x7f114c101f30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.715+0000 7f11539d4640 1 -- 172.21.15.33:0/1621096098 >> 172.21.15.33:0/1621096098 conn(0x7f114c0fd0a0 msgr2=0x7f114c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.715+0000 7f11539d4640 1 -- 172.21.15.33:0/1621096098 shutdown_connections 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.715+0000 7f11539d4640 1 -- 172.21.15.33:0/1621096098 wait complete. 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f11539d4640 1 Processor -- start 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f11539d4640 1 -- start start 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f11539d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f114c101b10 0x7f114c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f11539d4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114c102d00 0x7f114c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f11539d4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f114c1036c0 0x7f114c19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f11539d4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f114c10d770 con 0x7f114c101b10 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f11539d4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f114c10d5f0 con 0x7f114c1036c0 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f11539d4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f114c10d8f0 con 0x7f114c102d00 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f1151770640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f114c101b10 0x7f114c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f1151770640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f114c101b10 0x7f114c197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35560/0 (socket says 172.21.15.33:35560) 2024-04-02T13:36:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f1151770640 1 -- 172.21.15.33:0/382982032 learned_addr learned my addr 172.21.15.33:0/382982032 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f1150f6f640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114c102d00 0x7f114c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f1151f71640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f114c1036c0 0x7f114c19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f1151770640 1 -- 172.21.15.33:0/382982032 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114c102d00 msgr2=0x7f114c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f1151770640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114c102d00 0x7f114c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f1151770640 1 -- 172.21.15.33:0/382982032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f114c1036c0 msgr2=0x7f114c19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f1151770640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f114c1036c0 0x7f114c19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f1151770640 1 -- 172.21.15.33:0/382982032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c19fa80 con 0x7f114c101b10 2024-04-02T13:36:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.716+0000 7f1151f71640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f114c1036c0 0x7f114c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.717+0000 7f1151770640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f114c101b10 0x7f114c197e60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1144005ab0 tx=0x7f1144004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.717+0000 7f114a7fc640 1 -- 172.21.15.33:0/382982032 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144043020 con 0x7f114c101b10 2024-04-02T13:36:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.717+0000 7f114a7fc640 1 -- 172.21.15.33:0/382982032 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f11440054f0 con 0x7f114c101b10 2024-04-02T13:36:48.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.717+0000 7f11539d4640 1 -- 172.21.15.33:0/382982032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114c19fd10 con 0x7f114c101b10 2024-04-02T13:36:48.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.717+0000 7f114a7fc640 1 -- 172.21.15.33:0/382982032 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144003d10 con 0x7f114c101b10 2024-04-02T13:36:48.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.717+0000 7f11539d4640 1 -- 172.21.15.33:0/382982032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114c1a01b0 con 0x7f114c101b10 2024-04-02T13:36:48.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.718+0000 7f11539d4640 1 -- 172.21.15.33:0/382982032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114c108f20 con 0x7f114c101b10 2024-04-02T13:36:48.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.719+0000 7f114a7fc640 1 -- 172.21.15.33:0/382982032 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1144046020 con 0x7f114c101b10 2024-04-02T13:36:48.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.720+0000 7f114a7fc640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f11340a3c40 0x7f11340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:48.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.720+0000 7f1150f6f640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f11340a3c40 0x7f11340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:48.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.720+0000 7f114a7fc640 1 -- 172.21.15.33:0/382982032 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f11440f23d0 con 0x7f114c101b10 2024-04-02T13:36:48.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.720+0000 7f1150f6f640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f11340a3c40 0x7f11340a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f114c199370 tx=0x7f114000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:48.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.724+0000 7f114a7fc640 1 -- 172.21.15.33:0/382982032 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f114404d070 con 0x7f114c101b10 2024-04-02T13:36:48.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.940+0000 7f11539d4640 1 -- 172.21.15.33:0/382982032 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f114c061f40 con 0x7f11340a3c40 2024-04-02T13:36:48.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.969+0000 7f114a7fc640 1 -- 172.21.15.33:0/382982032 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f114c061f40 con 0x7f11340a3c40 2024-04-02T13:36:48.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.971+0000 7f11539d4640 1 -- 172.21.15.33:0/382982032 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f11340a3c40 msgr2=0x7f11340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:48.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.971+0000 7f11539d4640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f11340a3c40 0x7f11340a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f114c199370 tx=0x7f114000b390 comp rx=0 tx=0).stop 2024-04-02T13:36:48.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.971+0000 7f11539d4640 1 -- 172.21.15.33:0/382982032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f114c101b10 msgr2=0x7f114c197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:48.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.971+0000 7f11539d4640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f114c101b10 0x7f114c197e60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1144005ab0 tx=0x7f1144004e90 comp rx=0 tx=0).stop 2024-04-02T13:36:48.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.971+0000 7f11539d4640 1 -- 172.21.15.33:0/382982032 shutdown_connections 2024-04-02T13:36:48.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.971+0000 7f11539d4640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f11340a3c40 0x7f11340a6100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.971+0000 7f11539d4640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f114c1036c0 0x7f114c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.971+0000 7f11539d4640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f114c102d00 0x7f114c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.971+0000 7f11539d4640 1 --2- 172.21.15.33:0/382982032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f114c101b10 0x7f114c197e60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:48.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.971+0000 7f11539d4640 1 -- 172.21.15.33:0/382982032 >> 172.21.15.33:0/382982032 conn(0x7f114c0fd0a0 msgr2=0x7f114c1060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:48.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.971+0000 7f11539d4640 1 -- 172.21.15.33:0/382982032 shutdown_connections 2024-04-02T13:36:48.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:48.971+0000 7f11539d4640 1 -- 172.21.15.33:0/382982032 wait complete. 2024-04-02T13:36:48.985 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-02T13:36:49.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.090+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1871679331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8101c00 msgr2=0x7fd8c8102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:49.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.090+0000 7fd8cee2c640 1 --2- 172.21.15.33:0/1871679331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8101c00 0x7fd8c8102080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009900 tx=0x7fd8b402bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.090+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1871679331 shutdown_connections 2024-04-02T13:36:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.090+0000 7fd8cee2c640 1 --2- 172.21.15.33:0/1871679331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c81025c0 0x7fd8c8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.090+0000 7fd8cee2c640 1 --2- 172.21.15.33:0/1871679331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8101c00 0x7fd8c8102080 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.090+0000 7fd8cee2c640 1 --2- 172.21.15.33:0/1871679331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8c8100a10 0x7fd8c8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.090+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1871679331 >> 172.21.15.33:0/1871679331 conn(0x7fd8c80fbfa0 msgr2=0x7fd8c80fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.091+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1871679331 shutdown_connections 2024-04-02T13:36:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.091+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1871679331 wait complete. 2024-04-02T13:36:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.091+0000 7fd8cee2c640 1 Processor -- start 2024-04-02T13:36:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.091+0000 7fd8cee2c640 1 -- start start 2024-04-02T13:36:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.091+0000 7fd8cee2c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c8100a10 0x7fd8c8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.091+0000 7fd8cee2c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8101c00 0x7fd8c8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.091+0000 7fd8cee2c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8c81025c0 0x7fd8c819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.091+0000 7fd8cee2c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c810c670 con 0x7fd8c8101c00 2024-04-02T13:36:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.091+0000 7fd8cee2c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c810c4f0 con 0x7fd8c81025c0 2024-04-02T13:36:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.091+0000 7fd8cee2c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c810c7f0 con 0x7fd8c8100a10 2024-04-02T13:36:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.091+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8101c00 0x7fd8c8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.091+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8101c00 0x7fd8c8198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35574/0 (socket says 172.21.15.33:35574) 2024-04-02T13:36:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.091+0000 7fd8c7fff640 1 -- 172.21.15.33:0/1314172834 learned_addr learned my addr 172.21.15.33:0/1314172834 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.092+0000 7fd8cd3c9640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8c81025c0 0x7fd8c819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.092+0000 7fd8ccbc8640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c8100a10 0x7fd8c8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.092+0000 7fd8c7fff640 1 -- 172.21.15.33:0/1314172834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c8100a10 msgr2=0x7fd8c8197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.092+0000 7fd8c7fff640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c8100a10 0x7fd8c8197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.092+0000 7fd8c7fff640 1 -- 172.21.15.33:0/1314172834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8c81025c0 msgr2=0x7fd8c819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.092+0000 7fd8c7fff640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8c81025c0 0x7fd8c819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.092+0000 7fd8c7fff640 1 -- 172.21.15.33:0/1314172834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c819fa60 con 0x7fd8c8101c00 2024-04-02T13:36:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.092+0000 7fd8c7fff640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8101c00 0x7fd8c8198380 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009230 tx=0x7fd8b4004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:49.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.092+0000 7fd8c5ffb640 1 -- 172.21.15.33:0/1314172834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4043020 con 0x7fd8c8101c00 2024-04-02T13:36:49.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.092+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1314172834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c819fcf0 con 0x7fd8c8101c00 2024-04-02T13:36:49.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.092+0000 7fd8c5ffb640 1 -- 172.21.15.33:0/1314172834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd8b4004350 con 0x7fd8c8101c00 2024-04-02T13:36:49.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.092+0000 7fd8c5ffb640 1 -- 172.21.15.33:0/1314172834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4003ca0 con 0x7fd8c8101c00 2024-04-02T13:36:49.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.092+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1314172834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c81a0230 con 0x7fd8c8101c00 2024-04-02T13:36:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.093+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1314172834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c81a0520 con 0x7fd8c8101c00 2024-04-02T13:36:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.094+0000 7fd8c5ffb640 1 -- 172.21.15.33:0/1314172834 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd8b4046020 con 0x7fd8c8101c00 2024-04-02T13:36:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.095+0000 7fd8c5ffb640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd8b80a3c40 0x7fd8b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.095+0000 7fd8ccbc8640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd8b80a3c40 0x7fd8b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.095+0000 7fd8c5ffb640 1 -- 172.21.15.33:0/1314172834 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd8b40f2520 con 0x7fd8c8101c00 2024-04-02T13:36:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.096+0000 7fd8ccbc8640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd8b80a3c40 0x7fd8b80a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc005d00 tx=0x7fd8bc005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:49.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.100+0000 7fd8c5ffb640 1 -- 172.21.15.33:0/1314172834 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd8b404d070 con 0x7fd8c8101c00 2024-04-02T13:36:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.315+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1314172834 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fd8c80fe1c0 con 0x7fd8b80a3c40 2024-04-02T13:36:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.320+0000 7fd8c5ffb640 1 -- 172.21.15.33:0/1314172834 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd8c80fe1c0 con 0x7fd8b80a3c40 2024-04-02T13:36:49.323 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_1/bca06313-b243-4fc4-a150-e7ecf39b6bb0 2024-04-02T13:36:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.322+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1314172834 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd8b80a3c40 msgr2=0x7fd8b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.322+0000 7fd8cee2c640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd8b80a3c40 0x7fd8b80a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc005d00 tx=0x7fd8bc005c50 comp rx=0 tx=0).stop 2024-04-02T13:36:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.322+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1314172834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8101c00 msgr2=0x7fd8c8198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.322+0000 7fd8cee2c640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8101c00 0x7fd8c8198380 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009230 tx=0x7fd8b4004ea0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.322+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1314172834 shutdown_connections 2024-04-02T13:36:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.322+0000 7fd8cee2c640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd8b80a3c40 0x7fd8b80a6100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.322+0000 7fd8cee2c640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd8c81025c0 0x7fd8c819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.322+0000 7fd8cee2c640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c8101c00 0x7fd8c8198380 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.322+0000 7fd8cee2c640 1 --2- 172.21.15.33:0/1314172834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8c8100a10 0x7fd8c8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.322+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1314172834 >> 172.21.15.33:0/1314172834 conn(0x7fd8c80fbfa0 msgr2=0x7fd8c80fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.322+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1314172834 shutdown_connections 2024-04-02T13:36:49.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.322+0000 7fd8cee2c640 1 -- 172.21.15.33:0/1314172834 wait complete. 2024-04-02T13:36:49.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:49.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.440+0000 7f26796d6640 1 -- 172.21.15.33:0/327461449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674102d10 msgr2=0x7f2674103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.440+0000 7f26796d6640 1 --2- 172.21.15.33:0/327461449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674102d10 0x7f2674103190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2664009900 tx=0x7f266402bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.440+0000 7f26796d6640 1 -- 172.21.15.33:0/327461449 shutdown_connections 2024-04-02T13:36:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.440+0000 7f26796d6640 1 --2- 172.21.15.33:0/327461449 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26741036d0 0x7f2674109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.440+0000 7f26796d6640 1 --2- 172.21.15.33:0/327461449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674102d10 0x7f2674103190 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.440+0000 7f26796d6640 1 --2- 172.21.15.33:0/327461449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2674101b20 0x7f2674101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.440+0000 7f26796d6640 1 -- 172.21.15.33:0/327461449 >> 172.21.15.33:0/327461449 conn(0x7f26740fd0b0 msgr2=0x7f26740ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.440+0000 7f26796d6640 1 -- 172.21.15.33:0/327461449 shutdown_connections 2024-04-02T13:36:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.440+0000 7f26796d6640 1 -- 172.21.15.33:0/327461449 wait complete. 2024-04-02T13:36:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.440+0000 7f26796d6640 1 Processor -- start 2024-04-02T13:36:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.440+0000 7f26796d6640 1 -- start start 2024-04-02T13:36:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26796d6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2674101b20 0x7f2674197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26796d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674102d10 0x7f2674198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:49.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26796d6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26741036d0 0x7f267419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26796d6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f267410d780 con 0x7f2674102d10 2024-04-02T13:36:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26796d6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f267410d600 con 0x7f26741036d0 2024-04-02T13:36:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26796d6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f267410d900 con 0x7f2674101b20 2024-04-02T13:36:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26727fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674102d10 0x7f2674198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26727fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674102d10 0x7f2674198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35604/0 (socket says 172.21.15.33:35604) 2024-04-02T13:36:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26727fc640 1 -- 172.21.15.33:0/4080371048 learned_addr learned my addr 172.21.15.33:0/4080371048 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26737fe640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26741036d0 0x7f267419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f2672ffd640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2674101b20 0x7f2674197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26727fc640 1 -- 172.21.15.33:0/4080371048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2674101b20 msgr2=0x7f2674197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26727fc640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2674101b20 0x7f2674197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26727fc640 1 -- 172.21.15.33:0/4080371048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26741036d0 msgr2=0x7f267419f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26727fc640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26741036d0 0x7f267419f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26727fc640 1 -- 172.21.15.33:0/4080371048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267419f9f0 con 0x7f2674102d10 2024-04-02T13:36:49.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f2672ffd640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2674101b20 0x7f2674197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:49.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26737fe640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26741036d0 0x7f267419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:49.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26727fc640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674102d10 0x7f2674198310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2664009230 tx=0x7f2664004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:49.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f266bfff640 1 -- 172.21.15.33:0/4080371048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664043020 con 0x7f2674102d10 2024-04-02T13:36:49.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f26796d6640 1 -- 172.21.15.33:0/4080371048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267419fc80 con 0x7f2674102d10 2024-04-02T13:36:49.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f266bfff640 1 -- 172.21.15.33:0/4080371048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2664004350 con 0x7f2674102d10 2024-04-02T13:36:49.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.441+0000 7f266bfff640 1 -- 172.21.15.33:0/4080371048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664003ca0 con 0x7f2674102d10 2024-04-02T13:36:49.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.442+0000 7f26796d6640 1 -- 172.21.15.33:0/4080371048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26741a01d0 con 0x7f2674102d10 2024-04-02T13:36:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.442+0000 7f26796d6640 1 -- 172.21.15.33:0/4080371048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2674108e60 con 0x7f2674102d10 2024-04-02T13:36:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.444+0000 7f266bfff640 1 -- 172.21.15.33:0/4080371048 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2664046020 con 0x7f2674102d10 2024-04-02T13:36:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.444+0000 7f266bfff640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f26600a3c40 0x7f26600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.445+0000 7f2672ffd640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f26600a3c40 0x7f26600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.445+0000 7f266bfff640 1 -- 172.21.15.33:0/4080371048 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f26640f2c20 con 0x7f2674102d10 2024-04-02T13:36:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.445+0000 7f2672ffd640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f26600a3c40 0x7f26600a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f265c005d00 tx=0x7f265c005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:49.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.449+0000 7f266bfff640 1 -- 172.21.15.33:0/4080371048 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f266404d070 con 0x7f2674102d10 2024-04-02T13:36:49.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.680+0000 7f26796d6640 1 -- 172.21.15.33:0/4080371048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f267402d540 con 0x7f2674102d10 2024-04-02T13:36:49.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.681+0000 7f266bfff640 1 -- 172.21.15.33:0/4080371048 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11873 (secure 0 0 0) 0x7f26640bf370 con 0x7f2674102d10 2024-04-02T13:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:36:47.489590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:36:49.684 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-02T13:36:49.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.683+0000 7f26796d6640 1 -- 172.21.15.33:0/4080371048 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f26600a3c40 msgr2=0x7f26600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:49.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.683+0000 7f26796d6640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f26600a3c40 0x7f26600a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f265c005d00 tx=0x7f265c005ef0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.683+0000 7f26796d6640 1 -- 172.21.15.33:0/4080371048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674102d10 msgr2=0x7f2674198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:49.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.683+0000 7f26796d6640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674102d10 0x7f2674198310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2664009230 tx=0x7f2664004ea0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.683+0000 7f26796d6640 1 -- 172.21.15.33:0/4080371048 shutdown_connections 2024-04-02T13:36:49.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.683+0000 7f26796d6640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f26600a3c40 0x7f26600a6100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.683+0000 7f26796d6640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26741036d0 0x7f267419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.683+0000 7f26796d6640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674102d10 0x7f2674198310 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.683+0000 7f26796d6640 1 --2- 172.21.15.33:0/4080371048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2674101b20 0x7f2674197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.683+0000 7f26796d6640 1 -- 172.21.15.33:0/4080371048 >> 172.21.15.33:0/4080371048 conn(0x7f26740fd0b0 msgr2=0x7f2674106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.683+0000 7f26796d6640 1 -- 172.21.15.33:0/4080371048 shutdown_connections 2024-04-02T13:36:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.683+0000 7f26796d6640 1 -- 172.21.15.33:0/4080371048 wait complete. 2024-04-02T13:36:49.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:36:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.800+0000 7f245c029640 1 -- 172.21.15.33:0/3401201182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454104f30 msgr2=0x7f2454107320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.800+0000 7f245c029640 1 --2- 172.21.15.33:0/3401201182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454104f30 0x7f2454107320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2444009900 tx=0x7f244402bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.800+0000 7f245c029640 1 -- 172.21.15.33:0/3401201182 shutdown_connections 2024-04-02T13:36:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.800+0000 7f245c029640 1 --2- 172.21.15.33:0/3401201182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2454107860 0x7f2454109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.800+0000 7f245c029640 1 --2- 172.21.15.33:0/3401201182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454104f30 0x7f2454107320 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.800+0000 7f245c029640 1 --2- 172.21.15.33:0/3401201182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2454102600 0x7f24541049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.800+0000 7f245c029640 1 -- 172.21.15.33:0/3401201182 >> 172.21.15.33:0/3401201182 conn(0x7f24540fbf90 msgr2=0x7f24540fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.800+0000 7f245c029640 1 -- 172.21.15.33:0/3401201182 shutdown_connections 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.800+0000 7f245c029640 1 -- 172.21.15.33:0/3401201182 wait complete. 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.801+0000 7f245c029640 1 Processor -- start 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.801+0000 7f245c029640 1 -- start start 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.801+0000 7f245c029640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2454102600 0x7f2454197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.801+0000 7f245c029640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454104f30 0x7f24541983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.801+0000 7f245c029640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2454107860 0x7f245419f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.801+0000 7f245c029640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f245410d7d0 con 0x7f2454104f30 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.801+0000 7f245c029640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f245410d650 con 0x7f2454102600 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.801+0000 7f245c029640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f245410d950 con 0x7f2454107860 2024-04-02T13:36:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.801+0000 7f24595c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454104f30 0x7f24541983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.801+0000 7f24595c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454104f30 0x7f24541983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35622/0 (socket says 172.21.15.33:35622) 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.801+0000 7f24595c4640 1 -- 172.21.15.33:0/4049193582 learned_addr learned my addr 172.21.15.33:0/4049193582 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.801+0000 7f2459dc5640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2454102600 0x7f2454197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.801+0000 7f245a5c6640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2454107860 0x7f245419f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.802+0000 7f24595c4640 1 -- 172.21.15.33:0/4049193582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2454107860 msgr2=0x7f245419f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.802+0000 7f24595c4640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2454107860 0x7f245419f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.802+0000 7f24595c4640 1 -- 172.21.15.33:0/4049193582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2454102600 msgr2=0x7f2454197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.802+0000 7f24595c4640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2454102600 0x7f2454197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.802+0000 7f24595c4640 1 -- 172.21.15.33:0/4049193582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245419fac0 con 0x7f2454104f30 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.802+0000 7f24595c4640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454104f30 0x7f24541983e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2444000c00 tx=0x7f2444004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.802+0000 7f244affd640 1 -- 172.21.15.33:0/4049193582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444043020 con 0x7f2454104f30 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.802+0000 7f245c029640 1 -- 172.21.15.33:0/4049193582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245419fd50 con 0x7f2454104f30 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.802+0000 7f244affd640 1 -- 172.21.15.33:0/4049193582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f24440054d0 con 0x7f2454104f30 2024-04-02T13:36:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.802+0000 7f244affd640 1 -- 172.21.15.33:0/4049193582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444003d40 con 0x7f2454104f30 2024-04-02T13:36:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.802+0000 7f245c029640 1 -- 172.21.15.33:0/4049193582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24541a0240 con 0x7f2454104f30 2024-04-02T13:36:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.803+0000 7f245c029640 1 -- 172.21.15.33:0/4049193582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24540fda80 con 0x7f2454104f30 2024-04-02T13:36:49.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.805+0000 7f244affd640 1 -- 172.21.15.33:0/4049193582 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2444046020 con 0x7f2454104f30 2024-04-02T13:36:49.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.806+0000 7f244affd640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f24400a3c40 0x7f24400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:49.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.806+0000 7f2459dc5640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f24400a3c40 0x7f24400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:49.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.806+0000 7f244affd640 1 -- 172.21.15.33:0/4049193582 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f24440f2940 con 0x7f2454104f30 2024-04-02T13:36:49.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.806+0000 7f2459dc5640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f24400a3c40 0x7f24400a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2450005fd0 tx=0x7f245000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:49.811+0000 7f244affd640 1 -- 172.21.15.33:0/4049193582 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f244404d020 con 0x7f2454104f30 2024-04-02T13:36:49.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:49 smithi145 ceph-mon[35646]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:49.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:49 smithi145 ceph-mon[35646]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-04-02T13:36:49.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:49 smithi145 ceph-mon[35646]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:49.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:49 smithi145 ceph-mon[35646]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-02T13:36:49.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:49 smithi145 ceph-mon[35646]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:49.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:49 smithi050 ceph-mon[35524]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:49.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:49 smithi050 ceph-mon[35524]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-04-02T13:36:49.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:49 smithi050 ceph-mon[35524]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:49.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:49 smithi050 ceph-mon[35524]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-02T13:36:49.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:49 smithi050 ceph-mon[35524]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:50.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.000+0000 7f245c029640 1 -- 172.21.15.33:0/4049193582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f245402d5c0 con 0x7f2454104f30 2024-04-02T13:36:50.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:49 smithi033 ceph-mon[32390]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:50.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:49 smithi033 ceph-mon[32390]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-04-02T13:36:50.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:49 smithi033 ceph-mon[32390]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:50.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:49 smithi033 ceph-mon[32390]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-02T13:36:50.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:49 smithi033 ceph-mon[32390]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:50.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.002+0000 7f244affd640 1 -- 172.21.15.33:0/4049193582 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24057 (secure 0 0 0) 0x7f24440bf030 con 0x7f2454104f30 2024-04-02T13:36:50.005 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:50.005 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:36:25.856437+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:36:50.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.004+0000 7f245c029640 1 -- 172.21.15.33:0/4049193582 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f24400a3c40 msgr2=0x7f24400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:50.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.004+0000 7f245c029640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f24400a3c40 0x7f24400a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2450005fd0 tx=0x7f245000b390 comp rx=0 tx=0).stop 2024-04-02T13:36:50.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.004+0000 7f245c029640 1 -- 172.21.15.33:0/4049193582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454104f30 msgr2=0x7f24541983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:50.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.004+0000 7f245c029640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454104f30 0x7f24541983e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2444000c00 tx=0x7f2444004f10 comp rx=0 tx=0).stop 2024-04-02T13:36:50.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.004+0000 7f245c029640 1 -- 172.21.15.33:0/4049193582 shutdown_connections 2024-04-02T13:36:50.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.004+0000 7f245c029640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f24400a3c40 0x7f24400a6100 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.004+0000 7f245c029640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2454107860 0x7f245419f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.004+0000 7f245c029640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454104f30 0x7f24541983e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.004+0000 7f245c029640 1 --2- 172.21.15.33:0/4049193582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2454102600 0x7f2454197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.004+0000 7f245c029640 1 -- 172.21.15.33:0/4049193582 >> 172.21.15.33:0/4049193582 conn(0x7f24540fbf90 msgr2=0x7f2454103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:50.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.004+0000 7f245c029640 1 -- 172.21.15.33:0/4049193582 shutdown_connections 2024-04-02T13:36:50.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.004+0000 7f245c029640 1 -- 172.21.15.33:0/4049193582 wait complete. 2024-04-02T13:36:50.018 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-02T13:36:50.018 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:36:50.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.119+0000 7f86d5863640 1 -- 172.21.15.33:0/1935585969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d0101b50 msgr2=0x7f86d0101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:50.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.119+0000 7f86d5863640 1 --2- 172.21.15.33:0/1935585969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d0101b50 0x7f86d0101f70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009900 tx=0x7f86c002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:50.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.119+0000 7f86d5863640 1 -- 172.21.15.33:0/1935585969 shutdown_connections 2024-04-02T13:36:50.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.119+0000 7f86d5863640 1 --2- 172.21.15.33:0/1935585969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d0103700 0x7f86d0109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.119+0000 7f86d5863640 1 --2- 172.21.15.33:0/1935585969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86d0102d40 0x7f86d01031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.119+0000 7f86d5863640 1 --2- 172.21.15.33:0/1935585969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d0101b50 0x7f86d0101f70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.119+0000 7f86d5863640 1 -- 172.21.15.33:0/1935585969 >> 172.21.15.33:0/1935585969 conn(0x7f86d00fd0a0 msgr2=0x7f86d00ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.120+0000 7f86d5863640 1 -- 172.21.15.33:0/1935585969 shutdown_connections 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.120+0000 7f86d5863640 1 -- 172.21.15.33:0/1935585969 wait complete. 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.120+0000 7f86d5863640 1 Processor -- start 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.120+0000 7f86d5863640 1 -- start start 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.120+0000 7f86d5863640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86d0101b50 0x7f86d0197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.120+0000 7f86d5863640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d0102d40 0x7f86d01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.120+0000 7f86d5863640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d0103700 0x7f86d019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.120+0000 7f86d5863640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010d7b0 con 0x7f86d0103700 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.120+0000 7f86d5863640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010d630 con 0x7f86d0102d40 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.120+0000 7f86d5863640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010d930 con 0x7f86d0101b50 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.120+0000 7f86cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d0103700 0x7f86d019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d0103700 0x7f86d019f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35646/0 (socket says 172.21.15.33:35646) 2024-04-02T13:36:50.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86cf7fe640 1 -- 172.21.15.33:0/2644024727 learned_addr learned my addr 172.21.15.33:0/2644024727 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:50.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86ce7fc640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d0102d40 0x7f86d01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:50.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86ceffd640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86d0101b50 0x7f86d0197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:50.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86cf7fe640 1 -- 172.21.15.33:0/2644024727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86d0101b50 msgr2=0x7f86d0197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:50.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86cf7fe640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86d0101b50 0x7f86d0197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86cf7fe640 1 -- 172.21.15.33:0/2644024727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d0102d40 msgr2=0x7f86d01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:50.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86cf7fe640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d0102d40 0x7f86d01983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86cf7fe640 1 -- 172.21.15.33:0/2644024727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d019faa0 con 0x7f86d0103700 2024-04-02T13:36:50.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86cf7fe640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d0103700 0x7f86d019f430 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f86c800eb10 tx=0x7f86c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:50.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86d4861640 1 -- 172.21.15.33:0/2644024727 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8004b20 con 0x7f86d0103700 2024-04-02T13:36:50.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86d5863640 1 -- 172.21.15.33:0/2644024727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d019fd90 con 0x7f86d0103700 2024-04-02T13:36:50.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86d4861640 1 -- 172.21.15.33:0/2644024727 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f86c8004cc0 con 0x7f86d0103700 2024-04-02T13:36:50.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86d4861640 1 -- 172.21.15.33:0/2644024727 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8010700 con 0x7f86d0103700 2024-04-02T13:36:50.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.121+0000 7f86d5863640 1 -- 172.21.15.33:0/2644024727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d01a0280 con 0x7f86d0103700 2024-04-02T13:36:50.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.122+0000 7f86d5863640 1 -- 172.21.15.33:0/2644024727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d0108f10 con 0x7f86d0103700 2024-04-02T13:36:50.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.123+0000 7f86d4861640 1 -- 172.21.15.33:0/2644024727 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f86c8004e70 con 0x7f86d0103700 2024-04-02T13:36:50.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.124+0000 7f86d4861640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f86b80a3cc0 0x7f86b80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:50.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.124+0000 7f86ceffd640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f86b80a3cc0 0x7f86b80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:50.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.124+0000 7f86d4861640 1 -- 172.21.15.33:0/2644024727 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f86c80d0d90 con 0x7f86d0103700 2024-04-02T13:36:50.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.124+0000 7f86ceffd640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f86b80a3cc0 0x7f86b80a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f86c0006010 tx=0x7f86c0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.128+0000 7f86d4861640 1 -- 172.21.15.33:0/2644024727 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f86c80d4030 con 0x7f86d0103700 2024-04-02T13:36:50.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.312+0000 7f86d5863640 1 -- 172.21.15.33:0/2644024727 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86d0061f40 con 0x7f86b80a3cc0 2024-04-02T13:36:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.322+0000 7f86d4861640 1 -- 172.21.15.33:0/2644024727 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340589 (secure 0 0 0) 0x7f86d0061f40 con 0x7f86b80a3cc0 2024-04-02T13:36:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-02T13:36:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.325+0000 7f86d5863640 1 -- 172.21.15.33:0/2644024727 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f86b80a3cc0 msgr2=0x7f86b80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.325+0000 7f86d5863640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f86b80a3cc0 0x7f86b80a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f86c0006010 tx=0x7f86c0000bc0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.325+0000 7f86d5863640 1 -- 172.21.15.33:0/2644024727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d0103700 msgr2=0x7f86d019f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.325+0000 7f86d5863640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d0103700 0x7f86d019f430 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f86c800eb10 tx=0x7f86c800c2d0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.325+0000 7f86d5863640 1 -- 172.21.15.33:0/2644024727 shutdown_connections 2024-04-02T13:36:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.325+0000 7f86d5863640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f86b80a3cc0 0x7f86b80a6180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.325+0000 7f86d5863640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d0103700 0x7f86d019f430 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.325+0000 7f86d5863640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d0102d40 0x7f86d01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.325+0000 7f86d5863640 1 --2- 172.21.15.33:0/2644024727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f86d0101b50 0x7f86d0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.325+0000 7f86d5863640 1 -- 172.21.15.33:0/2644024727 >> 172.21.15.33:0/2644024727 conn(0x7f86d00fd0a0 msgr2=0x7f86d01060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:50.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.326+0000 7f86d5863640 1 -- 172.21.15.33:0/2644024727 shutdown_connections 2024-04-02T13:36:50.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.326+0000 7f86d5863640 1 -- 172.21.15.33:0/2644024727 wait complete. 2024-04-02T13:36:50.339 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-04-02T13:36:49.868304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633636,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":166,"num_read_kb":178,"num_write":312,"num_write_kb":3020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373388,"kb_used_data":11744,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568436,"statfs":{"total":1151940427776,"available":1151558078464,"internally_reserved":0,"allocated":12025856,"data_stored":7598251,"data_compressed":284964,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38769,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":30,"num_read_kb":59,"num_write":84,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003697"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433434+0000","last_change":"2024-04-02T13:36:21.492436+0000","last_active":"2024-04-02T13:36:27.433434+0000","last_peered":"2024-04-02T13:36:27.433434+0000","last_clean":"2024-04-02T13:36:27.433434+0000","last_became_active":"2024-04-02T13:36:21.491831+0000","last_became_peered":"2024-04-02T13:36:21.491831+0000","last_unstale":"2024-04-02T13:36:27.433434+0000","last_undegraded":"2024-04-02T13:36:27.433434+0000","last_fullsized":"2024-04-02T13:36:27.433434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:50:57.919680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726252+0000","last_change":"2024-04-02T13:36:21.498089+0000","last_active":"2024-04-02T13:36:26.726252+0000","last_peered":"2024-04-02T13:36:26.726252+0000","last_clean":"2024-04-02T13:36:26.726252+0000","last_became_active":"2024-04-02T13:36:21.497784+0000","last_became_peered":"2024-04-02T13:36:21.497784+0000","last_unstale":"2024-04-02T13:36:26.726252+0000","last_undegraded":"2024-04-02T13:36:26.726252+0000","last_fullsized":"2024-04-02T13:36:26.726252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:42.137396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726079+0000","last_change":"2024-04-02T13:36:21.492207+0000","last_active":"2024-04-02T13:36:26.726079+0000","last_peered":"2024-04-02T13:36:26.726079+0000","last_clean":"2024-04-02T13:36:26.726079+0000","last_became_active":"2024-04-02T13:36:21.490605+0000","last_became_peered":"2024-04-02T13:36:21.490605+0000","last_unstale":"2024-04-02T13:36:26.726079+0000","last_undegraded":"2024-04-02T13:36:26.726079+0000","last_fullsized":"2024-04-02T13:36:26.726079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:03.984226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.728073+0000","last_change":"2024-04-02T13:36:21.491225+0000","last_active":"2024-04-02T13:36:26.728073+0000","last_peered":"2024-04-02T13:36:26.728073+0000","last_clean":"2024-04-02T13:36:26.728073+0000","last_became_active":"2024-04-02T13:36:21.490886+0000","last_became_peered":"2024-04-02T13:36:21.490886+0000","last_unstale":"2024-04-02T13:36:26.728073+0000","last_undegraded":"2024-04-02T13:36:26.728073+0000","last_fullsized":"2024-04-02T13:36:26.728073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:55:53.316243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725546+0000","last_change":"2024-04-02T13:36:21.495808+0000","last_active":"2024-04-02T13:36:26.725546+0000","last_peered":"2024-04-02T13:36:26.725546+0000","last_clean":"2024-04-02T13:36:26.725546+0000","last_became_active":"2024-04-02T13:36:21.495468+0000","last_became_peered":"2024-04-02T13:36:21.495468+0000","last_unstale":"2024-04-02T13:36:26.725546+0000","last_undegraded":"2024-04-02T13:36:26.725546+0000","last_fullsized":"2024-04-02T13:36:26.725546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:17.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726485+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:36:26.726485+0000","last_peered":"2024-04-02T13:36:26.726485+0000","last_clean":"2024-04-02T13:36:26.726485+0000","last_became_active":"2024-04-02T13:36:21.499200+0000","last_became_peered":"2024-04-02T13:36:21.499200+0000","last_unstale":"2024-04-02T13:36:26.726485+0000","last_undegraded":"2024-04-02T13:36:26.726485+0000","last_fullsized":"2024-04-02T13:36:26.726485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:54:52.113844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727241+0000","last_change":"2024-04-02T13:36:21.493429+0000","last_active":"2024-04-02T13:36:26.727241+0000","last_peered":"2024-04-02T13:36:26.727241+0000","last_clean":"2024-04-02T13:36:26.727241+0000","last_became_active":"2024-04-02T13:36:21.493097+0000","last_became_peered":"2024-04-02T13:36:21.493097+0000","last_unstale":"2024-04-02T13:36:26.727241+0000","last_undegraded":"2024-04-02T13:36:26.727241+0000","last_fullsized":"2024-04-02T13:36:26.727241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:47:03.330448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727575+0000","last_change":"2024-04-02T13:36:21.498261+0000","last_active":"2024-04-02T13:36:26.727575+0000","last_peered":"2024-04-02T13:36:26.727575+0000","last_clean":"2024-04-02T13:36:26.727575+0000","last_became_active":"2024-04-02T13:36:21.497951+0000","last_became_peered":"2024-04-02T13:36:21.497951+0000","last_unstale":"2024-04-02T13:36:26.727575+0000","last_undegraded":"2024-04-02T13:36:26.727575+0000","last_fullsized":"2024-04-02T13:36:26.727575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:37:57.070013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.474040+0000","last_change":"2024-04-02T13:36:21.490623+0000","last_active":"2024-04-02T13:36:26.474040+0000","last_peered":"2024-04-02T13:36:26.474040+0000","last_clean":"2024-04-02T13:36:26.474040+0000","last_became_active":"2024-04-02T13:36:21.490274+0000","last_became_peered":"2024-04-02T13:36:21.490274+0000","last_unstale":"2024-04-02T13:36:26.474040+0000","last_undegraded":"2024-04-02T13:36:26.474040+0000","last_fullsized":"2024-04-02T13:36:26.474040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:59:35.220430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726952+0000","last_change":"2024-04-02T13:36:21.503185+0000","last_active":"2024-04-02T13:36:26.726952+0000","last_peered":"2024-04-02T13:36:26.726952+0000","last_clean":"2024-04-02T13:36:26.726952+0000","last_became_active":"2024-04-02T13:36:21.502815+0000","last_became_peered":"2024-04-02T13:36:21.502815+0000","last_unstale":"2024-04-02T13:36:26.726952+0000","last_undegraded":"2024-04-02T13:36:26.726952+0000","last_fullsized":"2024-04-02T13:36:26.726952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:53:04.156860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727262+0000","last_change":"2024-04-02T13:36:21.492510+0000","last_active":"2024-04-02T13:36:26.727262+0000","last_peered":"2024-04-02T13:36:26.727262+0000","last_clean":"2024-04-02T13:36:26.727262+0000","last_became_active":"2024-04-02T13:36:21.492189+0000","last_became_peered":"2024-04-02T13:36:21.492189+0000","last_unstale":"2024-04-02T13:36:26.727262+0000","last_undegraded":"2024-04-02T13:36:26.727262+0000","last_fullsized":"2024-04-02T13:36:26.727262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:15.321621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727897+0000","last_change":"2024-04-02T13:36:21.498015+0000","last_active":"2024-04-02T13:36:26.727897+0000","last_peered":"2024-04-02T13:36:26.727897+0000","last_clean":"2024-04-02T13:36:26.727897+0000","last_became_active":"2024-04-02T13:36:21.497677+0000","last_became_peered":"2024-04-02T13:36:21.497677+0000","last_unstale":"2024-04-02T13:36:26.727897+0000","last_undegraded":"2024-04-02T13:36:26.727897+0000","last_fullsized":"2024-04-02T13:36:26.727897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:43:19.035902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473348+0000","last_change":"2024-04-02T13:36:21.497047+0000","last_active":"2024-04-02T13:36:26.473348+0000","last_peered":"2024-04-02T13:36:26.473348+0000","last_clean":"2024-04-02T13:36:26.473348+0000","last_became_active":"2024-04-02T13:36:21.496722+0000","last_became_peered":"2024-04-02T13:36:21.496722+0000","last_unstale":"2024-04-02T13:36:26.473348+0000","last_undegraded":"2024-04-02T13:36:26.473348+0000","last_fullsized":"2024-04-02T13:36:26.473348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:03:14.400687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725892+0000","last_change":"2024-04-02T13:36:21.496513+0000","last_active":"2024-04-02T13:36:26.725892+0000","last_peered":"2024-04-02T13:36:26.725892+0000","last_clean":"2024-04-02T13:36:26.725892+0000","last_became_active":"2024-04-02T13:36:21.496209+0000","last_became_peered":"2024-04-02T13:36:21.496209+0000","last_unstale":"2024-04-02T13:36:26.725892+0000","last_undegraded":"2024-04-02T13:36:26.725892+0000","last_fullsized":"2024-04-02T13:36:26.725892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:50.684206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725884+0000","last_change":"2024-04-02T13:36:21.493418+0000","last_active":"2024-04-02T13:36:26.725884+0000","last_peered":"2024-04-02T13:36:26.725884+0000","last_clean":"2024-04-02T13:36:26.725884+0000","last_became_active":"2024-04-02T13:36:21.492785+0000","last_became_peered":"2024-04-02T13:36:21.492785+0000","last_unstale":"2024-04-02T13:36:26.725884+0000","last_undegraded":"2024-04-02T13:36:26.725884+0000","last_fullsized":"2024-04-02T13:36:26.725884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:11.973683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473191+0000","last_change":"2024-04-02T13:36:21.490710+0000","last_active":"2024-04-02T13:36:26.473191+0000","last_peered":"2024-04-02T13:36:26.473191+0000","last_clean":"2024-04-02T13:36:26.473191+0000","last_became_active":"2024-04-02T13:36:21.489654+0000","last_became_peered":"2024-04-02T13:36:21.489654+0000","last_unstale":"2024-04-02T13:36:26.473191+0000","last_undegraded":"2024-04-02T13:36:26.473191+0000","last_fullsized":"2024-04-02T13:36:26.473191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:43:06.198540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433365+0000","last_change":"2024-04-02T13:36:21.493306+0000","last_active":"2024-04-02T13:36:27.433365+0000","last_peered":"2024-04-02T13:36:27.433365+0000","last_clean":"2024-04-02T13:36:27.433365+0000","last_became_active":"2024-04-02T13:36:21.492949+0000","last_became_peered":"2024-04-02T13:36:21.492949+0000","last_unstale":"2024-04-02T13:36:27.433365+0000","last_undegraded":"2024-04-02T13:36:27.433365+0000","last_fullsized":"2024-04-02T13:36:27.433365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:34.264790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726028+0000","last_change":"2024-04-02T13:36:21.496631+0000","last_active":"2024-04-02T13:36:26.726028+0000","last_peered":"2024-04-02T13:36:26.726028+0000","last_clean":"2024-04-02T13:36:26.726028+0000","last_became_active":"2024-04-02T13:36:21.496245+0000","last_became_peered":"2024-04-02T13:36:21.496245+0000","last_unstale":"2024-04-02T13:36:26.726028+0000","last_undegraded":"2024-04-02T13:36:26.726028+0000","last_fullsized":"2024-04-02T13:36:26.726028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:14:51.126381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725957+0000","last_change":"2024-04-02T13:36:21.491640+0000","last_active":"2024-04-02T13:36:26.725957+0000","last_peered":"2024-04-02T13:36:26.725957+0000","last_clean":"2024-04-02T13:36:26.725957+0000","last_became_active":"2024-04-02T13:36:21.487799+0000","last_became_peered":"2024-04-02T13:36:21.487799+0000","last_unstale":"2024-04-02T13:36:26.725957+0000","last_undegraded":"2024-04-02T13:36:26.725957+0000","last_fullsized":"2024-04-02T13:36:26.725957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:16.893295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725672+0000","last_change":"2024-04-02T13:36:25.499691+0000","last_active":"2024-04-02T13:36:26.725672+0000","last_peered":"2024-04-02T13:36:26.725672+0000","last_clean":"2024-04-02T13:36:26.725672+0000","last_became_active":"2024-04-02T13:36:25.499120+0000","last_became_peered":"2024-04-02T13:36:25.499120+0000","last_unstale":"2024-04-02T13:36:26.725672+0000","last_undegraded":"2024-04-02T13:36:26.725672+0000","last_fullsized":"2024-04-02T13:36:26.725672+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:47:53.823456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.497107+0000","last_change":"2024-04-02T13:36:20.828794+0000","last_active":"2024-04-02T13:36:46.497107+0000","last_peered":"2024-04-02T13:36:46.497107+0000","last_clean":"2024-04-02T13:36:46.497107+0000","last_became_active":"2024-04-02T13:36:20.828444+0000","last_became_peered":"2024-04-02T13:36:20.828444+0000","last_unstale":"2024-04-02T13:36:46.497107+0000","last_undegraded":"2024-04-02T13:36:46.497107+0000","last_fullsized":"2024-04-02T13:36:46.497107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:54.439185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725845+0000","last_change":"2024-04-02T13:36:21.499019+0000","last_active":"2024-04-02T13:36:26.725845+0000","last_peered":"2024-04-02T13:36:26.725845+0000","last_clean":"2024-04-02T13:36:26.725845+0000","last_became_active":"2024-04-02T13:36:21.498684+0000","last_became_peered":"2024-04-02T13:36:21.498684+0000","last_unstale":"2024-04-02T13:36:26.725845+0000","last_undegraded":"2024-04-02T13:36:26.725845+0000","last_fullsized":"2024-04-02T13:36:26.725845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:15.618382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727063+0000","last_change":"2024-04-02T13:36:25.498544+0000","last_active":"2024-04-02T13:36:26.727063+0000","last_peered":"2024-04-02T13:36:26.727063+0000","last_clean":"2024-04-02T13:36:26.727063+0000","last_became_active":"2024-04-02T13:36:25.498076+0000","last_became_peered":"2024-04-02T13:36:25.498076+0000","last_unstale":"2024-04-02T13:36:26.727063+0000","last_undegraded":"2024-04-02T13:36:26.727063+0000","last_fullsized":"2024-04-02T13:36:26.727063+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:04:47.311056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727364+0000","last_change":"2024-04-02T13:36:20.832649+0000","last_active":"2024-04-02T13:36:26.727364+0000","last_peered":"2024-04-02T13:36:26.727364+0000","last_clean":"2024-04-02T13:36:26.727364+0000","last_became_active":"2024-04-02T13:36:20.832184+0000","last_became_peered":"2024-04-02T13:36:20.832184+0000","last_unstale":"2024-04-02T13:36:26.727364+0000","last_undegraded":"2024-04-02T13:36:26.727364+0000","last_fullsized":"2024-04-02T13:36:26.727364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:37.639406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726209+0000","last_change":"2024-04-02T13:36:21.489988+0000","last_active":"2024-04-02T13:36:26.726209+0000","last_peered":"2024-04-02T13:36:26.726209+0000","last_clean":"2024-04-02T13:36:26.726209+0000","last_became_active":"2024-04-02T13:36:21.489636+0000","last_became_peered":"2024-04-02T13:36:21.489636+0000","last_unstale":"2024-04-02T13:36:26.726209+0000","last_undegraded":"2024-04-02T13:36:26.726209+0000","last_fullsized":"2024-04-02T13:36:26.726209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:52:36.670712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727600+0000","last_change":"2024-04-02T13:36:25.499441+0000","last_active":"2024-04-02T13:36:26.727600+0000","last_peered":"2024-04-02T13:36:26.727600+0000","last_clean":"2024-04-02T13:36:26.727600+0000","last_became_active":"2024-04-02T13:36:25.498512+0000","last_became_peered":"2024-04-02T13:36:25.498512+0000","last_unstale":"2024-04-02T13:36:26.727600+0000","last_undegraded":"2024-04-02T13:36:26.727600+0000","last_fullsized":"2024-04-02T13:36:26.727600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:48:59.291360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.494535+0000","last_change":"2024-04-02T13:36:20.830008+0000","last_active":"2024-04-02T13:36:46.494535+0000","last_peered":"2024-04-02T13:36:46.494535+0000","last_clean":"2024-04-02T13:36:46.494535+0000","last_became_active":"2024-04-02T13:36:20.829048+0000","last_became_peered":"2024-04-02T13:36:20.829048+0000","last_unstale":"2024-04-02T13:36:46.494535+0000","last_undegraded":"2024-04-02T13:36:46.494535+0000","last_fullsized":"2024-04-02T13:36:46.494535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:08:34.405351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473608+0000","last_change":"2024-04-02T13:36:21.497296+0000","last_active":"2024-04-02T13:36:26.473608+0000","last_peered":"2024-04-02T13:36:26.473608+0000","last_clean":"2024-04-02T13:36:26.473608+0000","last_became_active":"2024-04-02T13:36:21.496802+0000","last_became_peered":"2024-04-02T13:36:21.496802+0000","last_unstale":"2024-04-02T13:36:26.473608+0000","last_undegraded":"2024-04-02T13:36:26.473608+0000","last_fullsized":"2024-04-02T13:36:26.473608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:03.995029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727386+0000","last_change":"2024-04-02T13:36:25.498661+0000","last_active":"2024-04-02T13:36:26.727386+0000","last_peered":"2024-04-02T13:36:26.727386+0000","last_clean":"2024-04-02T13:36:26.727386+0000","last_became_active":"2024-04-02T13:36:25.497866+0000","last_became_peered":"2024-04-02T13:36:25.497866+0000","last_unstale":"2024-04-02T13:36:26.727386+0000","last_undegraded":"2024-04-02T13:36:26.727386+0000","last_fullsized":"2024-04-02T13:36:26.727386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:47:05.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725988+0000","last_change":"2024-04-02T13:36:20.824586+0000","last_active":"2024-04-02T13:36:26.725988+0000","last_peered":"2024-04-02T13:36:26.725988+0000","last_clean":"2024-04-02T13:36:26.725988+0000","last_became_active":"2024-04-02T13:36:20.824254+0000","last_became_peered":"2024-04-02T13:36:20.824254+0000","last_unstale":"2024-04-02T13:36:26.725988+0000","last_undegraded":"2024-04-02T13:36:26.725988+0000","last_fullsized":"2024-04-02T13:36:26.725988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:08.774361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.474250+0000","last_change":"2024-04-02T13:36:21.495125+0000","last_active":"2024-04-02T13:36:26.474250+0000","last_peered":"2024-04-02T13:36:26.474250+0000","last_clean":"2024-04-02T13:36:26.474250+0000","last_became_active":"2024-04-02T13:36:21.494724+0000","last_became_peered":"2024-04-02T13:36:21.494724+0000","last_unstale":"2024-04-02T13:36:26.474250+0000","last_undegraded":"2024-04-02T13:36:26.474250+0000","last_fullsized":"2024-04-02T13:36:26.474250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:27.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725928+0000","last_change":"2024-04-02T13:36:25.498708+0000","last_active":"2024-04-02T13:36:26.725928+0000","last_peered":"2024-04-02T13:36:26.725928+0000","last_clean":"2024-04-02T13:36:26.725928+0000","last_became_active":"2024-04-02T13:36:25.497834+0000","last_became_peered":"2024-04-02T13:36:25.497834+0000","last_unstale":"2024-04-02T13:36:26.725928+0000","last_undegraded":"2024-04-02T13:36:26.725928+0000","last_fullsized":"2024-04-02T13:36:26.725928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:14.524569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:48.488619+0000","last_change":"2024-04-02T13:36:20.824901+0000","last_active":"2024-04-02T13:36:48.488619+0000","last_peered":"2024-04-02T13:36:48.488619+0000","last_clean":"2024-04-02T13:36:48.488619+0000","last_became_active":"2024-04-02T13:36:20.823973+0000","last_became_peered":"2024-04-02T13:36:20.823973+0000","last_unstale":"2024-04-02T13:36:48.488619+0000","last_undegraded":"2024-04-02T13:36:48.488619+0000","last_fullsized":"2024-04-02T13:36:48.488619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:17.636527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725893+0000","last_change":"2024-04-02T13:36:21.493411+0000","last_active":"2024-04-02T13:36:26.725893+0000","last_peered":"2024-04-02T13:36:26.725893+0000","last_clean":"2024-04-02T13:36:26.725893+0000","last_became_active":"2024-04-02T13:36:21.493061+0000","last_became_peered":"2024-04-02T13:36:21.493061+0000","last_unstale":"2024-04-02T13:36:26.725893+0000","last_undegraded":"2024-04-02T13:36:26.725893+0000","last_fullsized":"2024-04-02T13:36:26.725893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:15.902656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726395+0000","last_change":"2024-04-02T13:36:25.494817+0000","last_active":"2024-04-02T13:36:26.726395+0000","last_peered":"2024-04-02T13:36:26.726395+0000","last_clean":"2024-04-02T13:36:26.726395+0000","last_became_active":"2024-04-02T13:36:25.494185+0000","last_became_peered":"2024-04-02T13:36:25.494185+0000","last_unstale":"2024-04-02T13:36:26.726395+0000","last_undegraded":"2024-04-02T13:36:26.726395+0000","last_fullsized":"2024-04-02T13:36:26.726395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:04:32.719985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.495135+0000","last_change":"2024-04-02T13:36:20.821981+0000","last_active":"2024-04-02T13:36:46.495135+0000","last_peered":"2024-04-02T13:36:46.495135+0000","last_clean":"2024-04-02T13:36:46.495135+0000","last_became_active":"2024-04-02T13:36:20.821074+0000","last_became_peered":"2024-04-02T13:36:20.821074+0000","last_unstale":"2024-04-02T13:36:46.495135+0000","last_undegraded":"2024-04-02T13:36:46.495135+0000","last_fullsized":"2024-04-02T13:36:46.495135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:22.094586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.474238+0000","last_change":"2024-04-02T13:36:21.490792+0000","last_active":"2024-04-02T13:36:26.474238+0000","last_peered":"2024-04-02T13:36:26.474238+0000","last_clean":"2024-04-02T13:36:26.474238+0000","last_became_active":"2024-04-02T13:36:21.490471+0000","last_became_peered":"2024-04-02T13:36:21.490471+0000","last_unstale":"2024-04-02T13:36:26.474238+0000","last_undegraded":"2024-04-02T13:36:26.474238+0000","last_fullsized":"2024-04-02T13:36:26.474238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:55.612316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433299+0000","last_change":"2024-04-02T13:36:25.503774+0000","last_active":"2024-04-02T13:36:27.433299+0000","last_peered":"2024-04-02T13:36:27.433299+0000","last_clean":"2024-04-02T13:36:27.433299+0000","last_became_active":"2024-04-02T13:36:25.503287+0000","last_became_peered":"2024-04-02T13:36:25.503287+0000","last_unstale":"2024-04-02T13:36:27.433299+0000","last_undegraded":"2024-04-02T13:36:27.433299+0000","last_fullsized":"2024-04-02T13:36:27.433299+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:57:29.939484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.495514+0000","last_change":"2024-04-02T13:36:20.816760+0000","last_active":"2024-04-02T13:36:46.495514+0000","last_peered":"2024-04-02T13:36:46.495514+0000","last_clean":"2024-04-02T13:36:46.495514+0000","last_became_active":"2024-04-02T13:36:20.816291+0000","last_became_peered":"2024-04-02T13:36:20.816291+0000","last_unstale":"2024-04-02T13:36:46.495514+0000","last_undegraded":"2024-04-02T13:36:46.495514+0000","last_fullsized":"2024-04-02T13:36:46.495514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:25.543946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727518+0000","last_change":"2024-04-02T13:36:21.498541+0000","last_active":"2024-04-02T13:36:26.727518+0000","last_peered":"2024-04-02T13:36:26.727518+0000","last_clean":"2024-04-02T13:36:26.727518+0000","last_became_active":"2024-04-02T13:36:21.497498+0000","last_became_peered":"2024-04-02T13:36:21.497498+0000","last_unstale":"2024-04-02T13:36:26.727518+0000","last_undegraded":"2024-04-02T13:36:26.727518+0000","last_fullsized":"2024-04-02T13:36:26.727518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:08.315938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726191+0000","last_change":"2024-04-02T13:36:25.500245+0000","last_active":"2024-04-02T13:36:26.726191+0000","last_peered":"2024-04-02T13:36:26.726191+0000","last_clean":"2024-04-02T13:36:26.726191+0000","last_became_active":"2024-04-02T13:36:25.499844+0000","last_became_peered":"2024-04-02T13:36:25.499844+0000","last_unstale":"2024-04-02T13:36:26.726191+0000","last_undegraded":"2024-04-02T13:36:26.726191+0000","last_fullsized":"2024-04-02T13:36:26.726191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:17:01.105757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473734+0000","last_change":"2024-04-02T13:36:20.826851+0000","last_active":"2024-04-02T13:36:26.473734+0000","last_peered":"2024-04-02T13:36:26.473734+0000","last_clean":"2024-04-02T13:36:26.473734+0000","last_became_active":"2024-04-02T13:36:20.826227+0000","last_became_peered":"2024-04-02T13:36:20.826227+0000","last_unstale":"2024-04-02T13:36:26.473734+0000","last_undegraded":"2024-04-02T13:36:26.473734+0000","last_fullsized":"2024-04-02T13:36:26.473734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:45.841966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725299+0000","last_change":"2024-04-02T13:36:21.496520+0000","last_active":"2024-04-02T13:36:26.725299+0000","last_peered":"2024-04-02T13:36:26.725299+0000","last_clean":"2024-04-02T13:36:26.725299+0000","last_became_active":"2024-04-02T13:36:21.496082+0000","last_became_peered":"2024-04-02T13:36:21.496082+0000","last_unstale":"2024-04-02T13:36:26.725299+0000","last_undegraded":"2024-04-02T13:36:26.725299+0000","last_fullsized":"2024-04-02T13:36:26.725299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:12:06.517806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727554+0000","last_change":"2024-04-02T13:36:25.496064+0000","last_active":"2024-04-02T13:36:26.727554+0000","last_peered":"2024-04-02T13:36:26.727554+0000","last_clean":"2024-04-02T13:36:26.727554+0000","last_became_active":"2024-04-02T13:36:25.495464+0000","last_became_peered":"2024-04-02T13:36:25.495464+0000","last_unstale":"2024-04-02T13:36:26.727554+0000","last_undegraded":"2024-04-02T13:36:26.727554+0000","last_fullsized":"2024-04-02T13:36:26.727554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:06:15.206135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:48.487543+0000","last_change":"2024-04-02T13:36:20.821898+0000","last_active":"2024-04-02T13:36:48.487543+0000","last_peered":"2024-04-02T13:36:48.487543+0000","last_clean":"2024-04-02T13:36:48.487543+0000","last_became_active":"2024-04-02T13:36:20.821374+0000","last_became_peered":"2024-04-02T13:36:20.821374+0000","last_unstale":"2024-04-02T13:36:48.487543+0000","last_undegraded":"2024-04-02T13:36:48.487543+0000","last_fullsized":"2024-04-02T13:36:48.487543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:21.568976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473920+0000","last_change":"2024-04-02T13:36:21.497343+0000","last_active":"2024-04-02T13:36:26.473920+0000","last_peered":"2024-04-02T13:36:26.473920+0000","last_clean":"2024-04-02T13:36:26.473920+0000","last_became_active":"2024-04-02T13:36:21.496733+0000","last_became_peered":"2024-04-02T13:36:21.496733+0000","last_unstale":"2024-04-02T13:36:26.473920+0000","last_undegraded":"2024-04-02T13:36:26.473920+0000","last_fullsized":"2024-04-02T13:36:26.473920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:26.068717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725306+0000","last_change":"2024-04-02T13:36:25.503011+0000","last_active":"2024-04-02T13:36:26.725306+0000","last_peered":"2024-04-02T13:36:26.725306+0000","last_clean":"2024-04-02T13:36:26.725306+0000","last_became_active":"2024-04-02T13:36:25.502571+0000","last_became_peered":"2024-04-02T13:36:25.502571+0000","last_unstale":"2024-04-02T13:36:26.725306+0000","last_undegraded":"2024-04-02T13:36:26.725306+0000","last_fullsized":"2024-04-02T13:36:26.725306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:48.672023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.546358+0000","last_change":"2024-04-02T13:36:20.827705+0000","last_active":"2024-04-02T13:36:45.546358+0000","last_peered":"2024-04-02T13:36:45.546358+0000","last_clean":"2024-04-02T13:36:45.546358+0000","last_became_active":"2024-04-02T13:36:20.827285+0000","last_became_peered":"2024-04-02T13:36:20.827285+0000","last_unstale":"2024-04-02T13:36:45.546358+0000","last_undegraded":"2024-04-02T13:36:45.546358+0000","last_fullsized":"2024-04-02T13:36:45.546358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:12.175523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726387+0000","last_change":"2024-04-02T13:36:21.493365+0000","last_active":"2024-04-02T13:36:26.726387+0000","last_peered":"2024-04-02T13:36:26.726387+0000","last_clean":"2024-04-02T13:36:26.726387+0000","last_became_active":"2024-04-02T13:36:21.493011+0000","last_became_peered":"2024-04-02T13:36:21.493011+0000","last_unstale":"2024-04-02T13:36:26.726387+0000","last_undegraded":"2024-04-02T13:36:26.726387+0000","last_fullsized":"2024-04-02T13:36:26.726387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:53.165244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727784+0000","last_change":"2024-04-02T13:36:25.499017+0000","last_active":"2024-04-02T13:36:26.727784+0000","last_peered":"2024-04-02T13:36:26.727784+0000","last_clean":"2024-04-02T13:36:26.727784+0000","last_became_active":"2024-04-02T13:36:25.498293+0000","last_became_peered":"2024-04-02T13:36:25.498293+0000","last_unstale":"2024-04-02T13:36:26.727784+0000","last_undegraded":"2024-04-02T13:36:26.727784+0000","last_fullsized":"2024-04-02T13:36:26.727784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:55.910943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.547606+0000","last_change":"2024-04-02T13:36:20.836345+0000","last_active":"2024-04-02T13:36:45.547606+0000","last_peered":"2024-04-02T13:36:45.547606+0000","last_clean":"2024-04-02T13:36:45.547606+0000","last_became_active":"2024-04-02T13:36:20.836037+0000","last_became_peered":"2024-04-02T13:36:20.836037+0000","last_unstale":"2024-04-02T13:36:45.547606+0000","last_undegraded":"2024-04-02T13:36:45.547606+0000","last_fullsized":"2024-04-02T13:36:45.547606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:47.070493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473839+0000","last_change":"2024-04-02T13:36:21.493209+0000","last_active":"2024-04-02T13:36:26.473839+0000","last_peered":"2024-04-02T13:36:26.473839+0000","last_clean":"2024-04-02T13:36:26.473839+0000","last_became_active":"2024-04-02T13:36:21.492894+0000","last_became_peered":"2024-04-02T13:36:21.492894+0000","last_unstale":"2024-04-02T13:36:26.473839+0000","last_undegraded":"2024-04-02T13:36:26.473839+0000","last_fullsized":"2024-04-02T13:36:26.473839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:47.162354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725737+0000","last_change":"2024-04-02T13:36:25.506752+0000","last_active":"2024-04-02T13:36:26.725737+0000","last_peered":"2024-04-02T13:36:26.725737+0000","last_clean":"2024-04-02T13:36:26.725737+0000","last_became_active":"2024-04-02T13:36:25.506363+0000","last_became_peered":"2024-04-02T13:36:25.506363+0000","last_unstale":"2024-04-02T13:36:26.725737+0000","last_undegraded":"2024-04-02T13:36:26.725737+0000","last_fullsized":"2024-04-02T13:36:26.725737+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:15:21.640418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'23","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:49.491435+0000","last_change":"2024-04-02T13:36:20.823563+0000","last_active":"2024-04-02T13:36:49.491435+0000","last_peered":"2024-04-02T13:36:49.491435+0000","last_clean":"2024-04-02T13:36:49.491435+0000","last_became_active":"2024-04-02T13:36:20.823186+0000","last_became_peered":"2024-04-02T13:36:20.823186+0000","last_unstale":"2024-04-02T13:36:49.491435+0000","last_undegraded":"2024-04-02T13:36:49.491435+0000","last_fullsized":"2024-04-02T13:36:49.491435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:50.983357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40471,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":45,"num_write":24,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726088+0000","last_change":"2024-04-02T13:36:21.500829+0000","last_active":"2024-04-02T13:36:26.726088+0000","last_peered":"2024-04-02T13:36:26.726088+0000","last_clean":"2024-04-02T13:36:26.726088+0000","last_became_active":"2024-04-02T13:36:21.500485+0000","last_became_peered":"2024-04-02T13:36:21.500485+0000","last_unstale":"2024-04-02T13:36:26.726088+0000","last_undegraded":"2024-04-02T13:36:26.726088+0000","last_fullsized":"2024-04-02T13:36:26.726088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:23:19.627565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.474668+0000","last_change":"2024-04-02T13:36:25.506695+0000","last_active":"2024-04-02T13:36:26.474668+0000","last_peered":"2024-04-02T13:36:26.474668+0000","last_clean":"2024-04-02T13:36:26.474668+0000","last_became_active":"2024-04-02T13:36:25.506254+0000","last_became_peered":"2024-04-02T13:36:25.506254+0000","last_unstale":"2024-04-02T13:36:26.474668+0000","last_undegraded":"2024-04-02T13:36:26.474668+0000","last_fullsized":"2024-04-02T13:36:26.474668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:57:00.315005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.547607+0000","last_change":"2024-04-02T13:36:20.825827+0000","last_active":"2024-04-02T13:36:45.547607+0000","last_peered":"2024-04-02T13:36:45.547607+0000","last_clean":"2024-04-02T13:36:45.547607+0000","last_became_active":"2024-04-02T13:36:20.825530+0000","last_became_peered":"2024-04-02T13:36:20.825530+0000","last_unstale":"2024-04-02T13:36:45.547607+0000","last_undegraded":"2024-04-02T13:36:45.547607+0000","last_fullsized":"2024-04-02T13:36:45.547607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:27:02.962513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473300+0000","last_change":"2024-04-02T13:36:21.486343+0000","last_active":"2024-04-02T13:36:26.473300+0000","last_peered":"2024-04-02T13:36:26.473300+0000","last_clean":"2024-04-02T13:36:26.473300+0000","last_became_active":"2024-04-02T13:36:21.486021+0000","last_became_peered":"2024-04-02T13:36:21.486021+0000","last_unstale":"2024-04-02T13:36:26.473300+0000","last_undegraded":"2024-04-02T13:36:26.473300+0000","last_fullsized":"2024-04-02T13:36:26.473300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:53:42.430173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433041+0000","last_change":"2024-04-02T13:36:25.502806+0000","last_active":"2024-04-02T13:36:27.433041+0000","last_peered":"2024-04-02T13:36:27.433041+0000","last_clean":"2024-04-02T13:36:27.433041+0000","last_became_active":"2024-04-02T13:36:25.502289+0000","last_became_peered":"2024-04-02T13:36:25.502289+0000","last_unstale":"2024-04-02T13:36:27.433041+0000","last_undegraded":"2024-04-02T13:36:27.433041+0000","last_fullsized":"2024-04-02T13:36:27.433041+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:03:42.637972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.546948+0000","last_change":"2024-04-02T13:36:20.829303+0000","last_active":"2024-04-02T13:36:45.546948+0000","last_peered":"2024-04-02T13:36:45.546948+0000","last_clean":"2024-04-02T13:36:45.546948+0000","last_became_active":"2024-04-02T13:36:20.828952+0000","last_became_peered":"2024-04-02T13:36:20.828952+0000","last_unstale":"2024-04-02T13:36:45.546948+0000","last_undegraded":"2024-04-02T13:36:45.546948+0000","last_fullsized":"2024-04-02T13:36:45.546948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:14:04.385178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726643+0000","last_change":"2024-04-02T13:36:21.491767+0000","last_active":"2024-04-02T13:36:26.726643+0000","last_peered":"2024-04-02T13:36:26.726643+0000","last_clean":"2024-04-02T13:36:26.726643+0000","last_became_active":"2024-04-02T13:36:21.491369+0000","last_became_peered":"2024-04-02T13:36:21.491369+0000","last_unstale":"2024-04-02T13:36:26.726643+0000","last_undegraded":"2024-04-02T13:36:26.726643+0000","last_fullsized":"2024-04-02T13:36:26.726643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:14:35.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725470+0000","last_change":"2024-04-02T13:36:25.503900+0000","last_active":"2024-04-02T13:36:26.725470+0000","last_peered":"2024-04-02T13:36:26.725470+0000","last_clean":"2024-04-02T13:36:26.725470+0000","last_became_active":"2024-04-02T13:36:25.502840+0000","last_became_peered":"2024-04-02T13:36:25.502840+0000","last_unstale":"2024-04-02T13:36:26.725470+0000","last_undegraded":"2024-04-02T13:36:26.725470+0000","last_fullsized":"2024-04-02T13:36:26.725470+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:31.106865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727405+0000","last_change":"2024-04-02T13:36:20.823753+0000","last_active":"2024-04-02T13:36:26.727405+0000","last_peered":"2024-04-02T13:36:26.727405+0000","last_clean":"2024-04-02T13:36:26.727405+0000","last_became_active":"2024-04-02T13:36:20.823394+0000","last_became_peered":"2024-04-02T13:36:20.823394+0000","last_unstale":"2024-04-02T13:36:26.727405+0000","last_undegraded":"2024-04-02T13:36:26.727405+0000","last_fullsized":"2024-04-02T13:36:26.727405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:13.023125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726359+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:36:26.726359+0000","last_peered":"2024-04-02T13:36:26.726359+0000","last_clean":"2024-04-02T13:36:26.726359+0000","last_became_active":"2024-04-02T13:36:21.499197+0000","last_became_peered":"2024-04-02T13:36:21.499197+0000","last_unstale":"2024-04-02T13:36:26.726359+0000","last_undegraded":"2024-04-02T13:36:26.726359+0000","last_fullsized":"2024-04-02T13:36:26.726359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:26.435832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473949+0000","last_change":"2024-04-02T13:36:25.504643+0000","last_active":"2024-04-02T13:36:26.473949+0000","last_peered":"2024-04-02T13:36:26.473949+0000","last_clean":"2024-04-02T13:36:26.473949+0000","last_became_active":"2024-04-02T13:36:25.504249+0000","last_became_peered":"2024-04-02T13:36:25.504249+0000","last_unstale":"2024-04-02T13:36:26.473949+0000","last_undegraded":"2024-04-02T13:36:26.473949+0000","last_fullsized":"2024-04-02T13:36:26.473949+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:15.818958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.496244+0000","last_change":"2024-04-02T13:36:20.830097+0000","last_active":"2024-04-02T13:36:46.496244+0000","last_peered":"2024-04-02T13:36:46.496244+0000","last_clean":"2024-04-02T13:36:46.496244+0000","last_became_active":"2024-04-02T13:36:20.829779+0000","last_became_peered":"2024-04-02T13:36:20.829779+0000","last_unstale":"2024-04-02T13:36:46.496244+0000","last_undegraded":"2024-04-02T13:36:46.496244+0000","last_fullsized":"2024-04-02T13:36:46.496244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:59.477730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473722+0000","last_change":"2024-04-02T13:36:21.497480+0000","last_active":"2024-04-02T13:36:26.473722+0000","last_peered":"2024-04-02T13:36:26.473722+0000","last_clean":"2024-04-02T13:36:26.473722+0000","last_became_active":"2024-04-02T13:36:21.497106+0000","last_became_peered":"2024-04-02T13:36:21.497106+0000","last_unstale":"2024-04-02T13:36:26.473722+0000","last_undegraded":"2024-04-02T13:36:26.473722+0000","last_fullsized":"2024-04-02T13:36:26.473722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:30.394968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473799+0000","last_change":"2024-04-02T13:36:25.500040+0000","last_active":"2024-04-02T13:36:26.473799+0000","last_peered":"2024-04-02T13:36:26.473799+0000","last_clean":"2024-04-02T13:36:26.473799+0000","last_became_active":"2024-04-02T13:36:25.499576+0000","last_became_peered":"2024-04-02T13:36:25.499576+0000","last_unstale":"2024-04-02T13:36:26.473799+0000","last_undegraded":"2024-04-02T13:36:26.473799+0000","last_fullsized":"2024-04-02T13:36:26.473799+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:15:47.475234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726162+0000","last_change":"2024-04-02T13:36:20.823225+0000","last_active":"2024-04-02T13:36:26.726162+0000","last_peered":"2024-04-02T13:36:26.726162+0000","last_clean":"2024-04-02T13:36:26.726162+0000","last_became_active":"2024-04-02T13:36:20.822865+0000","last_became_peered":"2024-04-02T13:36:20.822865+0000","last_unstale":"2024-04-02T13:36:26.726162+0000","last_undegraded":"2024-04-02T13:36:26.726162+0000","last_fullsized":"2024-04-02T13:36:26.726162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:00.488190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433213+0000","last_change":"2024-04-02T13:36:21.497692+0000","last_active":"2024-04-02T13:36:27.433213+0000","last_peered":"2024-04-02T13:36:27.433213+0000","last_clean":"2024-04-02T13:36:27.433213+0000","last_became_active":"2024-04-02T13:36:21.497322+0000","last_became_peered":"2024-04-02T13:36:21.497322+0000","last_unstale":"2024-04-02T13:36:27.433213+0000","last_undegraded":"2024-04-02T13:36:27.433213+0000","last_fullsized":"2024-04-02T13:36:27.433213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:17.590235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725823+0000","last_change":"2024-04-02T13:36:25.531063+0000","last_active":"2024-04-02T13:36:26.725823+0000","last_peered":"2024-04-02T13:36:26.725823+0000","last_clean":"2024-04-02T13:36:26.725823+0000","last_became_active":"2024-04-02T13:36:25.530608+0000","last_became_peered":"2024-04-02T13:36:25.530608+0000","last_unstale":"2024-04-02T13:36:26.725823+0000","last_undegraded":"2024-04-02T13:36:26.725823+0000","last_fullsized":"2024-04-02T13:36:26.725823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:51:27.710136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725580+0000","last_change":"2024-04-02T13:36:20.835277+0000","last_active":"2024-04-02T13:36:26.725580+0000","last_peered":"2024-04-02T13:36:26.725580+0000","last_clean":"2024-04-02T13:36:26.725580+0000","last_became_active":"2024-04-02T13:36:20.834911+0000","last_became_peered":"2024-04-02T13:36:20.834911+0000","last_unstale":"2024-04-02T13:36:26.725580+0000","last_undegraded":"2024-04-02T13:36:26.725580+0000","last_fullsized":"2024-04-02T13:36:26.725580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:20.965544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726693+0000","last_change":"2024-04-02T13:36:21.495274+0000","last_active":"2024-04-02T13:36:26.726693+0000","last_peered":"2024-04-02T13:36:26.726693+0000","last_clean":"2024-04-02T13:36:26.726693+0000","last_became_active":"2024-04-02T13:36:21.494841+0000","last_became_peered":"2024-04-02T13:36:21.494841+0000","last_unstale":"2024-04-02T13:36:26.726693+0000","last_undegraded":"2024-04-02T13:36:26.726693+0000","last_fullsized":"2024-04-02T13:36:26.726693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:20.096507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473699+0000","last_change":"2024-04-02T13:36:25.499970+0000","last_active":"2024-04-02T13:36:26.473699+0000","last_peered":"2024-04-02T13:36:26.473699+0000","last_clean":"2024-04-02T13:36:26.473699+0000","last_became_active":"2024-04-02T13:36:25.499539+0000","last_became_peered":"2024-04-02T13:36:25.499539+0000","last_unstale":"2024-04-02T13:36:26.473699+0000","last_undegraded":"2024-04-02T13:36:26.473699+0000","last_fullsized":"2024-04-02T13:36:26.473699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:43.136635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726360+0000","last_change":"2024-04-02T13:36:20.831150+0000","last_active":"2024-04-02T13:36:26.726360+0000","last_peered":"2024-04-02T13:36:26.726360+0000","last_clean":"2024-04-02T13:36:26.726360+0000","last_became_active":"2024-04-02T13:36:20.830838+0000","last_became_peered":"2024-04-02T13:36:20.830838+0000","last_unstale":"2024-04-02T13:36:26.726360+0000","last_undegraded":"2024-04-02T13:36:26.726360+0000","last_fullsized":"2024-04-02T13:36:26.726360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:23:09.566911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726275+0000","last_change":"2024-04-02T13:36:21.503128+0000","last_active":"2024-04-02T13:36:26.726275+0000","last_peered":"2024-04-02T13:36:26.726275+0000","last_clean":"2024-04-02T13:36:26.726275+0000","last_became_active":"2024-04-02T13:36:21.502695+0000","last_became_peered":"2024-04-02T13:36:21.502695+0000","last_unstale":"2024-04-02T13:36:26.726275+0000","last_undegraded":"2024-04-02T13:36:26.726275+0000","last_fullsized":"2024-04-02T13:36:26.726275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:38.261472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473948+0000","last_change":"2024-04-02T13:36:25.530911+0000","last_active":"2024-04-02T13:36:26.473948+0000","last_peered":"2024-04-02T13:36:26.473948+0000","last_clean":"2024-04-02T13:36:26.473948+0000","last_became_active":"2024-04-02T13:36:25.530428+0000","last_became_peered":"2024-04-02T13:36:25.530428+0000","last_unstale":"2024-04-02T13:36:26.473948+0000","last_undegraded":"2024-04-02T13:36:26.473948+0000","last_fullsized":"2024-04-02T13:36:26.473948+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:49.185987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473842+0000","last_change":"2024-04-02T13:36:21.492251+0000","last_active":"2024-04-02T13:36:26.473842+0000","last_peered":"2024-04-02T13:36:26.473842+0000","last_clean":"2024-04-02T13:36:26.473842+0000","last_became_active":"2024-04-02T13:36:21.491929+0000","last_became_peered":"2024-04-02T13:36:21.491929+0000","last_unstale":"2024-04-02T13:36:26.473842+0000","last_undegraded":"2024-04-02T13:36:26.473842+0000","last_fullsized":"2024-04-02T13:36:26.473842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:59.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.497149+0000","last_change":"2024-04-02T13:36:20.821431+0000","last_active":"2024-04-02T13:36:46.497149+0000","last_peered":"2024-04-02T13:36:46.497149+0000","last_clean":"2024-04-02T13:36:46.497149+0000","last_became_active":"2024-04-02T13:36:20.821067+0000","last_became_peered":"2024-04-02T13:36:20.821067+0000","last_unstale":"2024-04-02T13:36:46.497149+0000","last_undegraded":"2024-04-02T13:36:46.497149+0000","last_fullsized":"2024-04-02T13:36:46.497149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:52.043907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727189+0000","last_change":"2024-04-02T13:36:25.496992+0000","last_active":"2024-04-02T13:36:26.727189+0000","last_peered":"2024-04-02T13:36:26.727189+0000","last_clean":"2024-04-02T13:36:26.727189+0000","last_became_active":"2024-04-02T13:36:25.496467+0000","last_became_peered":"2024-04-02T13:36:25.496467+0000","last_unstale":"2024-04-02T13:36:26.727189+0000","last_undegraded":"2024-04-02T13:36:26.727189+0000","last_fullsized":"2024-04-02T13:36:26.727189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:13:00.747716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726003+0000","last_change":"2024-04-02T13:36:21.491399+0000","last_active":"2024-04-02T13:36:26.726003+0000","last_peered":"2024-04-02T13:36:26.726003+0000","last_clean":"2024-04-02T13:36:26.726003+0000","last_became_active":"2024-04-02T13:36:21.491038+0000","last_became_peered":"2024-04-02T13:36:21.491038+0000","last_unstale":"2024-04-02T13:36:26.726003+0000","last_undegraded":"2024-04-02T13:36:26.726003+0000","last_fullsized":"2024-04-02T13:36:26.726003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:25.603086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.547804+0000","last_change":"2024-04-02T13:36:20.822796+0000","last_active":"2024-04-02T13:36:45.547804+0000","last_peered":"2024-04-02T13:36:45.547804+0000","last_clean":"2024-04-02T13:36:45.547804+0000","last_became_active":"2024-04-02T13:36:20.822168+0000","last_became_peered":"2024-04-02T13:36:20.822168+0000","last_unstale":"2024-04-02T13:36:45.547804+0000","last_undegraded":"2024-04-02T13:36:45.547804+0000","last_fullsized":"2024-04-02T13:36:45.547804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:11.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726022+0000","last_change":"2024-04-02T13:35:08.678967+0000","last_active":"2024-04-02T13:36:26.726022+0000","last_peered":"2024-04-02T13:36:26.726022+0000","last_clean":"2024-04-02T13:36:26.726022+0000","last_became_active":"2024-04-02T13:35:08.678408+0000","last_became_peered":"2024-04-02T13:35:08.678408+0000","last_unstale":"2024-04-02T13:36:26.726022+0000","last_undegraded":"2024-04-02T13:36:26.726022+0000","last_fullsized":"2024-04-02T13:36:26.726022+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_clean_scrub_stamp":"2024-04-02T13:31:42.453938+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:30:20.020809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.474405+0000","last_change":"2024-04-02T13:36:25.532723+0000","last_active":"2024-04-02T13:36:26.474405+0000","last_peered":"2024-04-02T13:36:26.474405+0000","last_clean":"2024-04-02T13:36:26.474405+0000","last_became_active":"2024-04-02T13:36:25.532341+0000","last_became_peered":"2024-04-02T13:36:25.532341+0000","last_unstale":"2024-04-02T13:36:26.474405+0000","last_undegraded":"2024-04-02T13:36:26.474405+0000","last_fullsized":"2024-04-02T13:36:26.474405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:53:43.190339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473026+0000","last_change":"2024-04-02T13:36:21.488983+0000","last_active":"2024-04-02T13:36:26.473026+0000","last_peered":"2024-04-02T13:36:26.473026+0000","last_clean":"2024-04-02T13:36:26.473026+0000","last_became_active":"2024-04-02T13:36:21.488580+0000","last_became_peered":"2024-04-02T13:36:21.488580+0000","last_unstale":"2024-04-02T13:36:26.473026+0000","last_undegraded":"2024-04-02T13:36:26.473026+0000","last_fullsized":"2024-04-02T13:36:26.473026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:39.382233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.495546+0000","last_change":"2024-04-02T13:36:20.833438+0000","last_active":"2024-04-02T13:36:46.495546+0000","last_peered":"2024-04-02T13:36:46.495546+0000","last_clean":"2024-04-02T13:36:46.495546+0000","last_became_active":"2024-04-02T13:36:20.833080+0000","last_became_peered":"2024-04-02T13:36:20.833080+0000","last_unstale":"2024-04-02T13:36:46.495546+0000","last_undegraded":"2024-04-02T13:36:46.495546+0000","last_fullsized":"2024-04-02T13:36:46.495546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:07:11.575387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433053+0000","last_change":"2024-04-02T13:36:25.502814+0000","last_active":"2024-04-02T13:36:27.433053+0000","last_peered":"2024-04-02T13:36:27.433053+0000","last_clean":"2024-04-02T13:36:27.433053+0000","last_became_active":"2024-04-02T13:36:25.502288+0000","last_became_peered":"2024-04-02T13:36:25.502288+0000","last_unstale":"2024-04-02T13:36:27.433053+0000","last_undegraded":"2024-04-02T13:36:27.433053+0000","last_fullsized":"2024-04-02T13:36:27.433053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:48.073286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.546013+0000","last_change":"2024-04-02T13:36:20.830512+0000","last_active":"2024-04-02T13:36:45.546013+0000","last_peered":"2024-04-02T13:36:45.546013+0000","last_clean":"2024-04-02T13:36:45.546013+0000","last_became_active":"2024-04-02T13:36:20.830086+0000","last_became_peered":"2024-04-02T13:36:20.830086+0000","last_unstale":"2024-04-02T13:36:45.546013+0000","last_undegraded":"2024-04-02T13:36:45.546013+0000","last_fullsized":"2024-04-02T13:36:45.546013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:07.428363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473676+0000","last_change":"2024-04-02T13:36:21.478218+0000","last_active":"2024-04-02T13:36:26.473676+0000","last_peered":"2024-04-02T13:36:26.473676+0000","last_clean":"2024-04-02T13:36:26.473676+0000","last_became_active":"2024-04-02T13:36:21.477856+0000","last_became_peered":"2024-04-02T13:36:21.477856+0000","last_unstale":"2024-04-02T13:36:26.473676+0000","last_undegraded":"2024-04-02T13:36:26.473676+0000","last_fullsized":"2024-04-02T13:36:26.473676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:04.397606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727473+0000","last_change":"2024-04-02T13:36:25.530915+0000","last_active":"2024-04-02T13:36:26.727473+0000","last_peered":"2024-04-02T13:36:26.727473+0000","last_clean":"2024-04-02T13:36:26.727473+0000","last_became_active":"2024-04-02T13:36:25.530343+0000","last_became_peered":"2024-04-02T13:36:25.530343+0000","last_unstale":"2024-04-02T13:36:26.727473+0000","last_undegraded":"2024-04-02T13:36:26.727473+0000","last_fullsized":"2024-04-02T13:36:26.727473+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:32:41.181485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.496602+0000","last_change":"2024-04-02T13:36:20.828527+0000","last_active":"2024-04-02T13:36:46.496602+0000","last_peered":"2024-04-02T13:36:46.496602+0000","last_clean":"2024-04-02T13:36:46.496602+0000","last_became_active":"2024-04-02T13:36:20.828136+0000","last_became_peered":"2024-04-02T13:36:20.828136+0000","last_unstale":"2024-04-02T13:36:46.496602+0000","last_undegraded":"2024-04-02T13:36:46.496602+0000","last_fullsized":"2024-04-02T13:36:46.496602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:57.709400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727334+0000","last_change":"2024-04-02T13:36:21.492035+0000","last_active":"2024-04-02T13:36:26.727334+0000","last_peered":"2024-04-02T13:36:26.727334+0000","last_clean":"2024-04-02T13:36:26.727334+0000","last_became_active":"2024-04-02T13:36:21.491682+0000","last_became_peered":"2024-04-02T13:36:21.491682+0000","last_unstale":"2024-04-02T13:36:26.727334+0000","last_undegraded":"2024-04-02T13:36:26.727334+0000","last_fullsized":"2024-04-02T13:36:26.727334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:09:40.600616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725438+0000","last_change":"2024-04-02T13:36:25.493301+0000","last_active":"2024-04-02T13:36:26.725438+0000","last_peered":"2024-04-02T13:36:26.725438+0000","last_clean":"2024-04-02T13:36:26.725438+0000","last_became_active":"2024-04-02T13:36:25.492846+0000","last_became_peered":"2024-04-02T13:36:25.492846+0000","last_unstale":"2024-04-02T13:36:26.725438+0000","last_undegraded":"2024-04-02T13:36:26.725438+0000","last_fullsized":"2024-04-02T13:36:26.725438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:12:53.350261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.545842+0000","last_change":"2024-04-02T13:36:20.823559+0000","last_active":"2024-04-02T13:36:45.545842+0000","last_peered":"2024-04-02T13:36:45.545842+0000","last_clean":"2024-04-02T13:36:45.545842+0000","last_became_active":"2024-04-02T13:36:20.823178+0000","last_became_peered":"2024-04-02T13:36:20.823178+0000","last_unstale":"2024-04-02T13:36:45.545842+0000","last_undegraded":"2024-04-02T13:36:45.545842+0000","last_fullsized":"2024-04-02T13:36:45.545842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:42.628855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726253+0000","last_change":"2024-04-02T13:36:21.497258+0000","last_active":"2024-04-02T13:36:26.726253+0000","last_peered":"2024-04-02T13:36:26.726253+0000","last_clean":"2024-04-02T13:36:26.726253+0000","last_became_active":"2024-04-02T13:36:21.496943+0000","last_became_peered":"2024-04-02T13:36:21.496943+0000","last_unstale":"2024-04-02T13:36:26.726253+0000","last_undegraded":"2024-04-02T13:36:26.726253+0000","last_fullsized":"2024-04-02T13:36:26.726253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:51.989025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727971+0000","last_change":"2024-04-02T13:36:25.531251+0000","last_active":"2024-04-02T13:36:26.727971+0000","last_peered":"2024-04-02T13:36:26.727971+0000","last_clean":"2024-04-02T13:36:26.727971+0000","last_became_active":"2024-04-02T13:36:25.530505+0000","last_became_peered":"2024-04-02T13:36:25.530505+0000","last_unstale":"2024-04-02T13:36:26.727971+0000","last_undegraded":"2024-04-02T13:36:26.727971+0000","last_fullsized":"2024-04-02T13:36:26.727971+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:02:10.711138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.547180+0000","last_change":"2024-04-02T13:36:20.837865+0000","last_active":"2024-04-02T13:36:45.547180+0000","last_peered":"2024-04-02T13:36:45.547180+0000","last_clean":"2024-04-02T13:36:45.547180+0000","last_became_active":"2024-04-02T13:36:20.837567+0000","last_became_peered":"2024-04-02T13:36:20.837567+0000","last_unstale":"2024-04-02T13:36:45.547180+0000","last_undegraded":"2024-04-02T13:36:45.547180+0000","last_fullsized":"2024-04-02T13:36:45.547180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:11:59.365592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725420+0000","last_change":"2024-04-02T13:36:21.500085+0000","last_active":"2024-04-02T13:36:26.725420+0000","last_peered":"2024-04-02T13:36:26.725420+0000","last_clean":"2024-04-02T13:36:26.725420+0000","last_became_active":"2024-04-02T13:36:21.499706+0000","last_became_peered":"2024-04-02T13:36:21.499706+0000","last_unstale":"2024-04-02T13:36:26.725420+0000","last_undegraded":"2024-04-02T13:36:26.725420+0000","last_fullsized":"2024-04-02T13:36:26.725420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.378226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725732+0000","last_change":"2024-04-02T13:36:25.505797+0000","last_active":"2024-04-02T13:36:26.725732+0000","last_peered":"2024-04-02T13:36:26.725732+0000","last_clean":"2024-04-02T13:36:26.725732+0000","last_became_active":"2024-04-02T13:36:25.505409+0000","last_became_peered":"2024-04-02T13:36:25.505409+0000","last_unstale":"2024-04-02T13:36:26.725732+0000","last_undegraded":"2024-04-02T13:36:26.725732+0000","last_fullsized":"2024-04-02T13:36:26.725732+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:46:31.723248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.548579+0000","last_change":"2024-04-02T13:36:20.821572+0000","last_active":"2024-04-02T13:36:45.548579+0000","last_peered":"2024-04-02T13:36:45.548579+0000","last_clean":"2024-04-02T13:36:45.548579+0000","last_became_active":"2024-04-02T13:36:20.821235+0000","last_became_peered":"2024-04-02T13:36:20.821235+0000","last_unstale":"2024-04-02T13:36:45.548579+0000","last_undegraded":"2024-04-02T13:36:45.548579+0000","last_fullsized":"2024-04-02T13:36:45.548579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:02.304145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.432872+0000","last_change":"2024-04-02T13:36:21.485255+0000","last_active":"2024-04-02T13:36:27.432872+0000","last_peered":"2024-04-02T13:36:27.432872+0000","last_clean":"2024-04-02T13:36:27.432872+0000","last_became_active":"2024-04-02T13:36:21.484870+0000","last_became_peered":"2024-04-02T13:36:21.484870+0000","last_unstale":"2024-04-02T13:36:27.432872+0000","last_undegraded":"2024-04-02T13:36:27.432872+0000","last_fullsized":"2024-04-02T13:36:27.432872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:03.851161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727821+0000","last_change":"2024-04-02T13:36:25.494161+0000","last_active":"2024-04-02T13:36:26.727821+0000","last_peered":"2024-04-02T13:36:26.727821+0000","last_clean":"2024-04-02T13:36:26.727821+0000","last_became_active":"2024-04-02T13:36:25.493667+0000","last_became_peered":"2024-04-02T13:36:25.493667+0000","last_unstale":"2024-04-02T13:36:26.727821+0000","last_undegraded":"2024-04-02T13:36:26.727821+0000","last_fullsized":"2024-04-02T13:36:26.727821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:35:45.703988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725751+0000","last_change":"2024-04-02T13:36:20.825219+0000","last_active":"2024-04-02T13:36:26.725751+0000","last_peered":"2024-04-02T13:36:26.725751+0000","last_clean":"2024-04-02T13:36:26.725751+0000","last_became_active":"2024-04-02T13:36:20.824878+0000","last_became_peered":"2024-04-02T13:36:20.824878+0000","last_unstale":"2024-04-02T13:36:26.725751+0000","last_undegraded":"2024-04-02T13:36:26.725751+0000","last_fullsized":"2024-04-02T13:36:26.725751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:12.579433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726487+0000","last_change":"2024-04-02T13:36:21.498640+0000","last_active":"2024-04-02T13:36:26.726487+0000","last_peered":"2024-04-02T13:36:26.726487+0000","last_clean":"2024-04-02T13:36:26.726487+0000","last_became_active":"2024-04-02T13:36:21.497996+0000","last_became_peered":"2024-04-02T13:36:21.497996+0000","last_unstale":"2024-04-02T13:36:26.726487+0000","last_undegraded":"2024-04-02T13:36:26.726487+0000","last_fullsized":"2024-04-02T13:36:26.726487+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:56:45.748945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.728067+0000","last_change":"2024-04-02T13:36:25.531119+0000","last_active":"2024-04-02T13:36:26.728067+0000","last_peered":"2024-04-02T13:36:26.728067+0000","last_clean":"2024-04-02T13:36:26.728067+0000","last_became_active":"2024-04-02T13:36:25.530307+0000","last_became_peered":"2024-04-02T13:36:25.530307+0000","last_unstale":"2024-04-02T13:36:26.728067+0000","last_undegraded":"2024-04-02T13:36:26.728067+0000","last_fullsized":"2024-04-02T13:36:26.728067+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:58:50.543386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.546092+0000","last_change":"2024-04-02T13:36:20.829843+0000","last_active":"2024-04-02T13:36:45.546092+0000","last_peered":"2024-04-02T13:36:45.546092+0000","last_clean":"2024-04-02T13:36:45.546092+0000","last_became_active":"2024-04-02T13:36:20.829502+0000","last_became_peered":"2024-04-02T13:36:20.829502+0000","last_unstale":"2024-04-02T13:36:45.546092+0000","last_undegraded":"2024-04-02T13:36:45.546092+0000","last_fullsized":"2024-04-02T13:36:45.546092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:36:45.016174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726425+0000","last_change":"2024-04-02T13:36:21.491579+0000","last_active":"2024-04-02T13:36:26.726425+0000","last_peered":"2024-04-02T13:36:26.726425+0000","last_clean":"2024-04-02T13:36:26.726425+0000","last_became_active":"2024-04-02T13:36:21.490992+0000","last_became_peered":"2024-04-02T13:36:21.490992+0000","last_unstale":"2024-04-02T13:36:26.726425+0000","last_undegraded":"2024-04-02T13:36:26.726425+0000","last_fullsized":"2024-04-02T13:36:26.726425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:41.899766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473493+0000","last_change":"2024-04-02T13:36:25.502658+0000","last_active":"2024-04-02T13:36:26.473493+0000","last_peered":"2024-04-02T13:36:26.473493+0000","last_clean":"2024-04-02T13:36:26.473493+0000","last_became_active":"2024-04-02T13:36:25.502219+0000","last_became_peered":"2024-04-02T13:36:25.502219+0000","last_unstale":"2024-04-02T13:36:26.473493+0000","last_undegraded":"2024-04-02T13:36:26.473493+0000","last_fullsized":"2024-04-02T13:36:26.473493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:34.189381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726564+0000","last_change":"2024-04-02T13:36:20.835176+0000","last_active":"2024-04-02T13:36:26.726564+0000","last_peered":"2024-04-02T13:36:26.726564+0000","last_clean":"2024-04-02T13:36:26.726564+0000","last_became_active":"2024-04-02T13:36:20.834871+0000","last_became_peered":"2024-04-02T13:36:20.834871+0000","last_unstale":"2024-04-02T13:36:26.726564+0000","last_undegraded":"2024-04-02T13:36:26.726564+0000","last_fullsized":"2024-04-02T13:36:26.726564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:17:14.148762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727352+0000","last_change":"2024-04-02T13:36:21.490336+0000","last_active":"2024-04-02T13:36:26.727352+0000","last_peered":"2024-04-02T13:36:26.727352+0000","last_clean":"2024-04-02T13:36:26.727352+0000","last_became_active":"2024-04-02T13:36:21.489869+0000","last_became_peered":"2024-04-02T13:36:21.489869+0000","last_unstale":"2024-04-02T13:36:26.727352+0000","last_undegraded":"2024-04-02T13:36:26.727352+0000","last_fullsized":"2024-04-02T13:36:26.727352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:42.243209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725578+0000","last_change":"2024-04-02T13:36:25.503124+0000","last_active":"2024-04-02T13:36:26.725578+0000","last_peered":"2024-04-02T13:36:26.725578+0000","last_clean":"2024-04-02T13:36:26.725578+0000","last_became_active":"2024-04-02T13:36:25.502737+0000","last_became_peered":"2024-04-02T13:36:25.502737+0000","last_unstale":"2024-04-02T13:36:26.725578+0000","last_undegraded":"2024-04-02T13:36:26.725578+0000","last_fullsized":"2024-04-02T13:36:26.725578+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:02:56.830746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.474433+0000","last_change":"2024-04-02T13:36:20.828597+0000","last_active":"2024-04-02T13:36:26.474433+0000","last_peered":"2024-04-02T13:36:26.474433+0000","last_clean":"2024-04-02T13:36:26.474433+0000","last_became_active":"2024-04-02T13:36:20.828138+0000","last_became_peered":"2024-04-02T13:36:20.828138+0000","last_unstale":"2024-04-02T13:36:26.474433+0000","last_undegraded":"2024-04-02T13:36:26.474433+0000","last_fullsized":"2024-04-02T13:36:26.474433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:05.171448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726691+0000","last_change":"2024-04-02T13:36:21.498498+0000","last_active":"2024-04-02T13:36:26.726691+0000","last_peered":"2024-04-02T13:36:26.726691+0000","last_clean":"2024-04-02T13:36:26.726691+0000","last_became_active":"2024-04-02T13:36:21.498119+0000","last_became_peered":"2024-04-02T13:36:21.498119+0000","last_unstale":"2024-04-02T13:36:26.726691+0000","last_undegraded":"2024-04-02T13:36:26.726691+0000","last_fullsized":"2024-04-02T13:36:26.726691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:01.528228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473582+0000","last_change":"2024-04-02T13:36:25.497189+0000","last_active":"2024-04-02T13:36:26.473582+0000","last_peered":"2024-04-02T13:36:26.473582+0000","last_clean":"2024-04-02T13:36:26.473582+0000","last_became_active":"2024-04-02T13:36:25.496711+0000","last_became_peered":"2024-04-02T13:36:25.496711+0000","last_unstale":"2024-04-02T13:36:26.473582+0000","last_undegraded":"2024-04-02T13:36:26.473582+0000","last_fullsized":"2024-04-02T13:36:26.473582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:33.452746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473694+0000","last_change":"2024-04-02T13:36:20.823053+0000","last_active":"2024-04-02T13:36:26.473694+0000","last_peered":"2024-04-02T13:36:26.473694+0000","last_clean":"2024-04-02T13:36:26.473694+0000","last_became_active":"2024-04-02T13:36:20.822665+0000","last_became_peered":"2024-04-02T13:36:20.822665+0000","last_unstale":"2024-04-02T13:36:26.473694+0000","last_undegraded":"2024-04-02T13:36:26.473694+0000","last_fullsized":"2024-04-02T13:36:26.473694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:43:51.148843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726766+0000","last_change":"2024-04-02T13:36:21.494069+0000","last_active":"2024-04-02T13:36:26.726766+0000","last_peered":"2024-04-02T13:36:26.726766+0000","last_clean":"2024-04-02T13:36:26.726766+0000","last_became_active":"2024-04-02T13:36:21.493751+0000","last_became_peered":"2024-04-02T13:36:21.493751+0000","last_unstale":"2024-04-02T13:36:26.726766+0000","last_undegraded":"2024-04-02T13:36:26.726766+0000","last_fullsized":"2024-04-02T13:36:26.726766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:24:39.648318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.432946+0000","last_change":"2024-04-02T13:36:21.494280+0000","last_active":"2024-04-02T13:36:27.432946+0000","last_peered":"2024-04-02T13:36:27.432946+0000","last_clean":"2024-04-02T13:36:27.432946+0000","last_became_active":"2024-04-02T13:36:21.491262+0000","last_became_peered":"2024-04-02T13:36:21.491262+0000","last_unstale":"2024-04-02T13:36:27.432946+0000","last_undegraded":"2024-04-02T13:36:27.432946+0000","last_fullsized":"2024-04-02T13:36:27.432946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:08:16.728646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725685+0000","last_change":"2024-04-02T13:36:21.497103+0000","last_active":"2024-04-02T13:36:26.725685+0000","last_peered":"2024-04-02T13:36:26.725685+0000","last_clean":"2024-04-02T13:36:26.725685+0000","last_became_active":"2024-04-02T13:36:21.496784+0000","last_became_peered":"2024-04-02T13:36:21.496784+0000","last_unstale":"2024-04-02T13:36:26.725685+0000","last_undegraded":"2024-04-02T13:36:26.725685+0000","last_fullsized":"2024-04-02T13:36:26.725685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:38:21.903362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725610+0000","last_change":"2024-04-02T13:36:21.491853+0000","last_active":"2024-04-02T13:36:26.725610+0000","last_peered":"2024-04-02T13:36:26.725610+0000","last_clean":"2024-04-02T13:36:26.725610+0000","last_became_active":"2024-04-02T13:36:21.487986+0000","last_became_peered":"2024-04-02T13:36:21.487986+0000","last_unstale":"2024-04-02T13:36:26.725610+0000","last_undegraded":"2024-04-02T13:36:26.725610+0000","last_fullsized":"2024-04-02T13:36:26.725610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:37:47.098681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725813+0000","last_change":"2024-04-02T13:36:21.493453+0000","last_active":"2024-04-02T13:36:26.725813+0000","last_peered":"2024-04-02T13:36:26.725813+0000","last_clean":"2024-04-02T13:36:26.725813+0000","last_became_active":"2024-04-02T13:36:21.492789+0000","last_became_peered":"2024-04-02T13:36:21.492789+0000","last_unstale":"2024-04-02T13:36:26.725813+0000","last_undegraded":"2024-04-02T13:36:26.725813+0000","last_fullsized":"2024-04-02T13:36:26.725813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:12:27.157466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473764+0000","last_change":"2024-04-02T13:36:21.488723+0000","last_active":"2024-04-02T13:36:26.473764+0000","last_peered":"2024-04-02T13:36:26.473764+0000","last_clean":"2024-04-02T13:36:26.473764+0000","last_became_active":"2024-04-02T13:36:21.488391+0000","last_became_peered":"2024-04-02T13:36:21.488391+0000","last_unstale":"2024-04-02T13:36:26.473764+0000","last_undegraded":"2024-04-02T13:36:26.473764+0000","last_fullsized":"2024-04-02T13:36:26.473764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:23.768286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433285+0000","last_change":"2024-04-02T13:36:21.492676+0000","last_active":"2024-04-02T13:36:27.433285+0000","last_peered":"2024-04-02T13:36:27.433285+0000","last_clean":"2024-04-02T13:36:27.433285+0000","last_became_active":"2024-04-02T13:36:21.491663+0000","last_became_peered":"2024-04-02T13:36:21.491663+0000","last_unstale":"2024-04-02T13:36:27.433285+0000","last_undegraded":"2024-04-02T13:36:27.433285+0000","last_fullsized":"2024-04-02T13:36:27.433285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:00.131388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726355+0000","last_change":"2024-04-02T13:36:21.493380+0000","last_active":"2024-04-02T13:36:26.726355+0000","last_peered":"2024-04-02T13:36:26.726355+0000","last_clean":"2024-04-02T13:36:26.726355+0000","last_became_active":"2024-04-02T13:36:21.493073+0000","last_became_peered":"2024-04-02T13:36:21.493073+0000","last_unstale":"2024-04-02T13:36:26.726355+0000","last_undegraded":"2024-04-02T13:36:26.726355+0000","last_fullsized":"2024-04-02T13:36:26.726355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:59.412056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727455+0000","last_change":"2024-04-02T13:36:21.500881+0000","last_active":"2024-04-02T13:36:26.727455+0000","last_peered":"2024-04-02T13:36:26.727455+0000","last_clean":"2024-04-02T13:36:26.727455+0000","last_became_active":"2024-04-02T13:36:21.500516+0000","last_became_peered":"2024-04-02T13:36:21.500516+0000","last_unstale":"2024-04-02T13:36:26.727455+0000","last_undegraded":"2024-04-02T13:36:26.727455+0000","last_fullsized":"2024-04-02T13:36:26.727455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:33.842494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473733+0000","last_change":"2024-04-02T13:36:21.494558+0000","last_active":"2024-04-02T13:36:26.473733+0000","last_peered":"2024-04-02T13:36:26.473733+0000","last_clean":"2024-04-02T13:36:26.473733+0000","last_became_active":"2024-04-02T13:36:21.494180+0000","last_became_peered":"2024-04-02T13:36:21.494180+0000","last_unstale":"2024-04-02T13:36:26.473733+0000","last_undegraded":"2024-04-02T13:36:26.473733+0000","last_fullsized":"2024-04-02T13:36:26.473733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:07:54.363504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727621+0000","last_change":"2024-04-02T13:36:21.500406+0000","last_active":"2024-04-02T13:36:26.727621+0000","last_peered":"2024-04-02T13:36:26.727621+0000","last_clean":"2024-04-02T13:36:26.727621+0000","last_became_active":"2024-04-02T13:36:21.499927+0000","last_became_peered":"2024-04-02T13:36:21.499927+0000","last_unstale":"2024-04-02T13:36:26.727621+0000","last_undegraded":"2024-04-02T13:36:26.727621+0000","last_fullsized":"2024-04-02T13:36:26.727621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:23.702776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473578+0000","last_change":"2024-04-02T13:36:21.486166+0000","last_active":"2024-04-02T13:36:26.473578+0000","last_peered":"2024-04-02T13:36:26.473578+0000","last_clean":"2024-04-02T13:36:26.473578+0000","last_became_active":"2024-04-02T13:36:21.485282+0000","last_became_peered":"2024-04-02T13:36:21.485282+0000","last_unstale":"2024-04-02T13:36:26.473578+0000","last_undegraded":"2024-04-02T13:36:26.473578+0000","last_fullsized":"2024-04-02T13:36:26.473578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:24:07.007675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473390+0000","last_change":"2024-04-02T13:36:21.484909+0000","last_active":"2024-04-02T13:36:26.473390+0000","last_peered":"2024-04-02T13:36:26.473390+0000","last_clean":"2024-04-02T13:36:26.473390+0000","last_became_active":"2024-04-02T13:36:21.484559+0000","last_became_peered":"2024-04-02T13:36:21.484559+0000","last_unstale":"2024-04-02T13:36:26.473390+0000","last_undegraded":"2024-04-02T13:36:26.473390+0000","last_fullsized":"2024-04-02T13:36:26.473390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:59:45.848407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433123+0000","last_change":"2024-04-02T13:36:21.493961+0000","last_active":"2024-04-02T13:36:27.433123+0000","last_peered":"2024-04-02T13:36:27.433123+0000","last_clean":"2024-04-02T13:36:27.433123+0000","last_became_active":"2024-04-02T13:36:21.493490+0000","last_became_peered":"2024-04-02T13:36:21.493490+0000","last_unstale":"2024-04-02T13:36:27.433123+0000","last_undegraded":"2024-04-02T13:36:27.433123+0000","last_fullsized":"2024-04-02T13:36:27.433123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:26:42.032550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":213880,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43268,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":30,"num_read_kb":59,"num_write":131,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":57869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":480266,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612631,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":1113074,"data_compressed":50891,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776157,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":485180,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":480892,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299431,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":1071838,"data_compressed":50891,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":480356,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":484108,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":480960,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":484142,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":480550,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888117,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":480288,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724593,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1076597,"data_compressed":50891,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34556,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34556,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":42440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:36:50.346 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.480+0000 7f8642fa8640 1 -- 172.21.15.33:0/1997769514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c100a20 msgr2=0x7f863c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.480+0000 7f8642fa8640 1 --2- 172.21.15.33:0/1997769514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c100a20 0x7f863c100e40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8630009900 tx=0x7f863002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.480+0000 7f8642fa8640 1 -- 172.21.15.33:0/1997769514 shutdown_connections 2024-04-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.480+0000 7f8642fa8640 1 --2- 172.21.15.33:0/1997769514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f863c1025d0 0x7f863c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.480+0000 7f8642fa8640 1 --2- 172.21.15.33:0/1997769514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f863c101c10 0x7f863c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.480+0000 7f8642fa8640 1 --2- 172.21.15.33:0/1997769514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c100a20 0x7f863c100e40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.480+0000 7f8642fa8640 1 -- 172.21.15.33:0/1997769514 >> 172.21.15.33:0/1997769514 conn(0x7f863c0fbf90 msgr2=0x7f863c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.480+0000 7f8642fa8640 1 -- 172.21.15.33:0/1997769514 shutdown_connections 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.480+0000 7f8642fa8640 1 -- 172.21.15.33:0/1997769514 wait complete. 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.480+0000 7f8642fa8640 1 Processor -- start 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.480+0000 7f8642fa8640 1 -- start start 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.480+0000 7f8642fa8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f863c100a20 0x7f863c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.480+0000 7f8642fa8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f863c101c10 0x7f863c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8642fa8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1025d0 0x7f863c1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8642fa8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f863c10c680 con 0x7f863c1025d0 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8642fa8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f863c10c500 con 0x7f863c101c10 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8642fa8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f863c10c800 con 0x7f863c100a20 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8641545640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1025d0 0x7f863c1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8641545640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1025d0 0x7f863c1a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35676/0 (socket says 172.21.15.33:35676) 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8641545640 1 -- 172.21.15.33:0/65646388 learned_addr learned my addr 172.21.15.33:0/65646388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f863bfff640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f863c101c10 0x7f863c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8641545640 1 -- 172.21.15.33:0/65646388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f863c100a20 msgr2=0x7f863c19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8640d44640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f863c100a20 0x7f863c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8641545640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f863c100a20 0x7f863c19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8641545640 1 -- 172.21.15.33:0/65646388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f863c101c10 msgr2=0x7f863c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8641545640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f863c101c10 0x7f863c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8641545640 1 -- 172.21.15.33:0/65646388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c1a1cb0 con 0x7f863c1025d0 2024-04-02T13:36:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f863bfff640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f863c101c10 0x7f863c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8641545640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1025d0 0x7f863c1a1640 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f863400eaa0 tx=0x7f863400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8639ffb640 1 -- 172.21.15.33:0/65646388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863400cdb0 con 0x7f863c1025d0 2024-04-02T13:36:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8642fa8640 1 -- 172.21.15.33:0/65646388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863c1a1fa0 con 0x7f863c1025d0 2024-04-02T13:36:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8639ffb640 1 -- 172.21.15.33:0/65646388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8634005020 con 0x7f863c1025d0 2024-04-02T13:36:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.481+0000 7f8639ffb640 1 -- 172.21.15.33:0/65646388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86340116b0 con 0x7f863c1025d0 2024-04-02T13:36:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.482+0000 7f8642fa8640 1 -- 172.21.15.33:0/65646388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863c1a2410 con 0x7f863c1025d0 2024-04-02T13:36:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.483+0000 7f8642fa8640 1 -- 172.21.15.33:0/65646388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f863c1a26d0 con 0x7f863c1025d0 2024-04-02T13:36:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.483+0000 7f8639ffb640 1 -- 172.21.15.33:0/65646388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f86340051d0 con 0x7f863c1025d0 2024-04-02T13:36:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.484+0000 7f8639ffb640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f862c0a3c40 0x7f862c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.484+0000 7f8639ffb640 1 -- 172.21.15.33:0/65646388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f86340d0a70 con 0x7f863c1025d0 2024-04-02T13:36:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.484+0000 7f8640d44640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f862c0a3c40 0x7f862c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.485+0000 7f8640d44640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f862c0a3c40 0x7f862c0a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f863002c710 tx=0x7f8630005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:50.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.489+0000 7f8639ffb640 1 -- 172.21.15.33:0/65646388 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8634016030 con 0x7f863c1025d0 2024-04-02T13:36:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.672+0000 7f8642fa8640 1 -- 172.21.15.33:0/65646388 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f863c1a28a0 con 0x7f862c0a3c40 2024-04-02T13:36:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.682+0000 7f8639ffb640 1 -- 172.21.15.33:0/65646388 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340589 (secure 0 0 0) 0x7f863c1a28a0 con 0x7f862c0a3c40 2024-04-02T13:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-02T13:36:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.685+0000 7f8642fa8640 1 -- 172.21.15.33:0/65646388 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f862c0a3c40 msgr2=0x7f862c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.685+0000 7f8642fa8640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f862c0a3c40 0x7f862c0a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f863002c710 tx=0x7f8630005fd0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.685+0000 7f8642fa8640 1 -- 172.21.15.33:0/65646388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1025d0 msgr2=0x7f863c1a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.685+0000 7f8642fa8640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1025d0 0x7f863c1a1640 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f863400eaa0 tx=0x7f863400ef60 comp rx=0 tx=0).stop 2024-04-02T13:36:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.685+0000 7f8642fa8640 1 -- 172.21.15.33:0/65646388 shutdown_connections 2024-04-02T13:36:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.685+0000 7f8642fa8640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f862c0a3c40 0x7f862c0a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.685+0000 7f8642fa8640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f863c1025d0 0x7f863c1a1640 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.685+0000 7f8642fa8640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f863c101c10 0x7f863c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.685+0000 7f8642fa8640 1 --2- 172.21.15.33:0/65646388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f863c100a20 0x7f863c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.685+0000 7f8642fa8640 1 -- 172.21.15.33:0/65646388 >> 172.21.15.33:0/65646388 conn(0x7f863c0fbf90 msgr2=0x7f863c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.685+0000 7f8642fa8640 1 -- 172.21.15.33:0/65646388 shutdown_connections 2024-04-02T13:36:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:50.685+0000 7f8642fa8640 1 -- 172.21.15.33:0/65646388 wait complete. 2024-04-02T13:36:50.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:50 smithi033 ceph-mon[32390]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:50.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:50 smithi033 ceph-mon[32390]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:50.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:50 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4080371048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:50.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:50 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4049193582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:50.699 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-04-02T13:36:49.868304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633636,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":166,"num_read_kb":178,"num_write":312,"num_write_kb":3020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373388,"kb_used_data":11744,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568436,"statfs":{"total":1151940427776,"available":1151558078464,"internally_reserved":0,"allocated":12025856,"data_stored":7598251,"data_compressed":284964,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38769,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":30,"num_read_kb":59,"num_write":84,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003697"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433434+0000","last_change":"2024-04-02T13:36:21.492436+0000","last_active":"2024-04-02T13:36:27.433434+0000","last_peered":"2024-04-02T13:36:27.433434+0000","last_clean":"2024-04-02T13:36:27.433434+0000","last_became_active":"2024-04-02T13:36:21.491831+0000","last_became_peered":"2024-04-02T13:36:21.491831+0000","last_unstale":"2024-04-02T13:36:27.433434+0000","last_undegraded":"2024-04-02T13:36:27.433434+0000","last_fullsized":"2024-04-02T13:36:27.433434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:50:57.919680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726252+0000","last_change":"2024-04-02T13:36:21.498089+0000","last_active":"2024-04-02T13:36:26.726252+0000","last_peered":"2024-04-02T13:36:26.726252+0000","last_clean":"2024-04-02T13:36:26.726252+0000","last_became_active":"2024-04-02T13:36:21.497784+0000","last_became_peered":"2024-04-02T13:36:21.497784+0000","last_unstale":"2024-04-02T13:36:26.726252+0000","last_undegraded":"2024-04-02T13:36:26.726252+0000","last_fullsized":"2024-04-02T13:36:26.726252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:42.137396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726079+0000","last_change":"2024-04-02T13:36:21.492207+0000","last_active":"2024-04-02T13:36:26.726079+0000","last_peered":"2024-04-02T13:36:26.726079+0000","last_clean":"2024-04-02T13:36:26.726079+0000","last_became_active":"2024-04-02T13:36:21.490605+0000","last_became_peered":"2024-04-02T13:36:21.490605+0000","last_unstale":"2024-04-02T13:36:26.726079+0000","last_undegraded":"2024-04-02T13:36:26.726079+0000","last_fullsized":"2024-04-02T13:36:26.726079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:03.984226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.728073+0000","last_change":"2024-04-02T13:36:21.491225+0000","last_active":"2024-04-02T13:36:26.728073+0000","last_peered":"2024-04-02T13:36:26.728073+0000","last_clean":"2024-04-02T13:36:26.728073+0000","last_became_active":"2024-04-02T13:36:21.490886+0000","last_became_peered":"2024-04-02T13:36:21.490886+0000","last_unstale":"2024-04-02T13:36:26.728073+0000","last_undegraded":"2024-04-02T13:36:26.728073+0000","last_fullsized":"2024-04-02T13:36:26.728073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:55:53.316243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725546+0000","last_change":"2024-04-02T13:36:21.495808+0000","last_active":"2024-04-02T13:36:26.725546+0000","last_peered":"2024-04-02T13:36:26.725546+0000","last_clean":"2024-04-02T13:36:26.725546+0000","last_became_active":"2024-04-02T13:36:21.495468+0000","last_became_peered":"2024-04-02T13:36:21.495468+0000","last_unstale":"2024-04-02T13:36:26.725546+0000","last_undegraded":"2024-04-02T13:36:26.725546+0000","last_fullsized":"2024-04-02T13:36:26.725546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:17.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726485+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:36:26.726485+0000","last_peered":"2024-04-02T13:36:26.726485+0000","last_clean":"2024-04-02T13:36:26.726485+0000","last_became_active":"2024-04-02T13:36:21.499200+0000","last_became_peered":"2024-04-02T13:36:21.499200+0000","last_unstale":"2024-04-02T13:36:26.726485+0000","last_undegraded":"2024-04-02T13:36:26.726485+0000","last_fullsized":"2024-04-02T13:36:26.726485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:54:52.113844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727241+0000","last_change":"2024-04-02T13:36:21.493429+0000","last_active":"2024-04-02T13:36:26.727241+0000","last_peered":"2024-04-02T13:36:26.727241+0000","last_clean":"2024-04-02T13:36:26.727241+0000","last_became_active":"2024-04-02T13:36:21.493097+0000","last_became_peered":"2024-04-02T13:36:21.493097+0000","last_unstale":"2024-04-02T13:36:26.727241+0000","last_undegraded":"2024-04-02T13:36:26.727241+0000","last_fullsized":"2024-04-02T13:36:26.727241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:47:03.330448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727575+0000","last_change":"2024-04-02T13:36:21.498261+0000","last_active":"2024-04-02T13:36:26.727575+0000","last_peered":"2024-04-02T13:36:26.727575+0000","last_clean":"2024-04-02T13:36:26.727575+0000","last_became_active":"2024-04-02T13:36:21.497951+0000","last_became_peered":"2024-04-02T13:36:21.497951+0000","last_unstale":"2024-04-02T13:36:26.727575+0000","last_undegraded":"2024-04-02T13:36:26.727575+0000","last_fullsized":"2024-04-02T13:36:26.727575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:37:57.070013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.474040+0000","last_change":"2024-04-02T13:36:21.490623+0000","last_active":"2024-04-02T13:36:26.474040+0000","last_peered":"2024-04-02T13:36:26.474040+0000","last_clean":"2024-04-02T13:36:26.474040+0000","last_became_active":"2024-04-02T13:36:21.490274+0000","last_became_peered":"2024-04-02T13:36:21.490274+0000","last_unstale":"2024-04-02T13:36:26.474040+0000","last_undegraded":"2024-04-02T13:36:26.474040+0000","last_fullsized":"2024-04-02T13:36:26.474040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:59:35.220430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726952+0000","last_change":"2024-04-02T13:36:21.503185+0000","last_active":"2024-04-02T13:36:26.726952+0000","last_peered":"2024-04-02T13:36:26.726952+0000","last_clean":"2024-04-02T13:36:26.726952+0000","last_became_active":"2024-04-02T13:36:21.502815+0000","last_became_peered":"2024-04-02T13:36:21.502815+0000","last_unstale":"2024-04-02T13:36:26.726952+0000","last_undegraded":"2024-04-02T13:36:26.726952+0000","last_fullsized":"2024-04-02T13:36:26.726952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:53:04.156860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727262+0000","last_change":"2024-04-02T13:36:21.492510+0000","last_active":"2024-04-02T13:36:26.727262+0000","last_peered":"2024-04-02T13:36:26.727262+0000","last_clean":"2024-04-02T13:36:26.727262+0000","last_became_active":"2024-04-02T13:36:21.492189+0000","last_became_peered":"2024-04-02T13:36:21.492189+0000","last_unstale":"2024-04-02T13:36:26.727262+0000","last_undegraded":"2024-04-02T13:36:26.727262+0000","last_fullsized":"2024-04-02T13:36:26.727262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:15.321621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727897+0000","last_change":"2024-04-02T13:36:21.498015+0000","last_active":"2024-04-02T13:36:26.727897+0000","last_peered":"2024-04-02T13:36:26.727897+0000","last_clean":"2024-04-02T13:36:26.727897+0000","last_became_active":"2024-04-02T13:36:21.497677+0000","last_became_peered":"2024-04-02T13:36:21.497677+0000","last_unstale":"2024-04-02T13:36:26.727897+0000","last_undegraded":"2024-04-02T13:36:26.727897+0000","last_fullsized":"2024-04-02T13:36:26.727897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:43:19.035902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473348+0000","last_change":"2024-04-02T13:36:21.497047+0000","last_active":"2024-04-02T13:36:26.473348+0000","last_peered":"2024-04-02T13:36:26.473348+0000","last_clean":"2024-04-02T13:36:26.473348+0000","last_became_active":"2024-04-02T13:36:21.496722+0000","last_became_peered":"2024-04-02T13:36:21.496722+0000","last_unstale":"2024-04-02T13:36:26.473348+0000","last_undegraded":"2024-04-02T13:36:26.473348+0000","last_fullsized":"2024-04-02T13:36:26.473348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:03:14.400687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725892+0000","last_change":"2024-04-02T13:36:21.496513+0000","last_active":"2024-04-02T13:36:26.725892+0000","last_peered":"2024-04-02T13:36:26.725892+0000","last_clean":"2024-04-02T13:36:26.725892+0000","last_became_active":"2024-04-02T13:36:21.496209+0000","last_became_peered":"2024-04-02T13:36:21.496209+0000","last_unstale":"2024-04-02T13:36:26.725892+0000","last_undegraded":"2024-04-02T13:36:26.725892+0000","last_fullsized":"2024-04-02T13:36:26.725892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:50.684206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725884+0000","last_change":"2024-04-02T13:36:21.493418+0000","last_active":"2024-04-02T13:36:26.725884+0000","last_peered":"2024-04-02T13:36:26.725884+0000","last_clean":"2024-04-02T13:36:26.725884+0000","last_became_active":"2024-04-02T13:36:21.492785+0000","last_became_peered":"2024-04-02T13:36:21.492785+0000","last_unstale":"2024-04-02T13:36:26.725884+0000","last_undegraded":"2024-04-02T13:36:26.725884+0000","last_fullsized":"2024-04-02T13:36:26.725884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:11.973683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473191+0000","last_change":"2024-04-02T13:36:21.490710+0000","last_active":"2024-04-02T13:36:26.473191+0000","last_peered":"2024-04-02T13:36:26.473191+0000","last_clean":"2024-04-02T13:36:26.473191+0000","last_became_active":"2024-04-02T13:36:21.489654+0000","last_became_peered":"2024-04-02T13:36:21.489654+0000","last_unstale":"2024-04-02T13:36:26.473191+0000","last_undegraded":"2024-04-02T13:36:26.473191+0000","last_fullsized":"2024-04-02T13:36:26.473191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:43:06.198540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433365+0000","last_change":"2024-04-02T13:36:21.493306+0000","last_active":"2024-04-02T13:36:27.433365+0000","last_peered":"2024-04-02T13:36:27.433365+0000","last_clean":"2024-04-02T13:36:27.433365+0000","last_became_active":"2024-04-02T13:36:21.492949+0000","last_became_peered":"2024-04-02T13:36:21.492949+0000","last_unstale":"2024-04-02T13:36:27.433365+0000","last_undegraded":"2024-04-02T13:36:27.433365+0000","last_fullsized":"2024-04-02T13:36:27.433365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:34.264790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726028+0000","last_change":"2024-04-02T13:36:21.496631+0000","last_active":"2024-04-02T13:36:26.726028+0000","last_peered":"2024-04-02T13:36:26.726028+0000","last_clean":"2024-04-02T13:36:26.726028+0000","last_became_active":"2024-04-02T13:36:21.496245+0000","last_became_peered":"2024-04-02T13:36:21.496245+0000","last_unstale":"2024-04-02T13:36:26.726028+0000","last_undegraded":"2024-04-02T13:36:26.726028+0000","last_fullsized":"2024-04-02T13:36:26.726028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:14:51.126381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725957+0000","last_change":"2024-04-02T13:36:21.491640+0000","last_active":"2024-04-02T13:36:26.725957+0000","last_peered":"2024-04-02T13:36:26.725957+0000","last_clean":"2024-04-02T13:36:26.725957+0000","last_became_active":"2024-04-02T13:36:21.487799+0000","last_became_peered":"2024-04-02T13:36:21.487799+0000","last_unstale":"2024-04-02T13:36:26.725957+0000","last_undegraded":"2024-04-02T13:36:26.725957+0000","last_fullsized":"2024-04-02T13:36:26.725957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:16.893295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725672+0000","last_change":"2024-04-02T13:36:25.499691+0000","last_active":"2024-04-02T13:36:26.725672+0000","last_peered":"2024-04-02T13:36:26.725672+0000","last_clean":"2024-04-02T13:36:26.725672+0000","last_became_active":"2024-04-02T13:36:25.499120+0000","last_became_peered":"2024-04-02T13:36:25.499120+0000","last_unstale":"2024-04-02T13:36:26.725672+0000","last_undegraded":"2024-04-02T13:36:26.725672+0000","last_fullsized":"2024-04-02T13:36:26.725672+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:47:53.823456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.497107+0000","last_change":"2024-04-02T13:36:20.828794+0000","last_active":"2024-04-02T13:36:46.497107+0000","last_peered":"2024-04-02T13:36:46.497107+0000","last_clean":"2024-04-02T13:36:46.497107+0000","last_became_active":"2024-04-02T13:36:20.828444+0000","last_became_peered":"2024-04-02T13:36:20.828444+0000","last_unstale":"2024-04-02T13:36:46.497107+0000","last_undegraded":"2024-04-02T13:36:46.497107+0000","last_fullsized":"2024-04-02T13:36:46.497107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:54.439185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725845+0000","last_change":"2024-04-02T13:36:21.499019+0000","last_active":"2024-04-02T13:36:26.725845+0000","last_peered":"2024-04-02T13:36:26.725845+0000","last_clean":"2024-04-02T13:36:26.725845+0000","last_became_active":"2024-04-02T13:36:21.498684+0000","last_became_peered":"2024-04-02T13:36:21.498684+0000","last_unstale":"2024-04-02T13:36:26.725845+0000","last_undegraded":"2024-04-02T13:36:26.725845+0000","last_fullsized":"2024-04-02T13:36:26.725845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:15.618382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727063+0000","last_change":"2024-04-02T13:36:25.498544+0000","last_active":"2024-04-02T13:36:26.727063+0000","last_peered":"2024-04-02T13:36:26.727063+0000","last_clean":"2024-04-02T13:36:26.727063+0000","last_became_active":"2024-04-02T13:36:25.498076+0000","last_became_peered":"2024-04-02T13:36:25.498076+0000","last_unstale":"2024-04-02T13:36:26.727063+0000","last_undegraded":"2024-04-02T13:36:26.727063+0000","last_fullsized":"2024-04-02T13:36:26.727063+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:04:47.311056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727364+0000","last_change":"2024-04-02T13:36:20.832649+0000","last_active":"2024-04-02T13:36:26.727364+0000","last_peered":"2024-04-02T13:36:26.727364+0000","last_clean":"2024-04-02T13:36:26.727364+0000","last_became_active":"2024-04-02T13:36:20.832184+0000","last_became_peered":"2024-04-02T13:36:20.832184+0000","last_unstale":"2024-04-02T13:36:26.727364+0000","last_undegraded":"2024-04-02T13:36:26.727364+0000","last_fullsized":"2024-04-02T13:36:26.727364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:37.639406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726209+0000","last_change":"2024-04-02T13:36:21.489988+0000","last_active":"2024-04-02T13:36:26.726209+0000","last_peered":"2024-04-02T13:36:26.726209+0000","last_clean":"2024-04-02T13:36:26.726209+0000","last_became_active":"2024-04-02T13:36:21.489636+0000","last_became_peered":"2024-04-02T13:36:21.489636+0000","last_unstale":"2024-04-02T13:36:26.726209+0000","last_undegraded":"2024-04-02T13:36:26.726209+0000","last_fullsized":"2024-04-02T13:36:26.726209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:52:36.670712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727600+0000","last_change":"2024-04-02T13:36:25.499441+0000","last_active":"2024-04-02T13:36:26.727600+0000","last_peered":"2024-04-02T13:36:26.727600+0000","last_clean":"2024-04-02T13:36:26.727600+0000","last_became_active":"2024-04-02T13:36:25.498512+0000","last_became_peered":"2024-04-02T13:36:25.498512+0000","last_unstale":"2024-04-02T13:36:26.727600+0000","last_undegraded":"2024-04-02T13:36:26.727600+0000","last_fullsized":"2024-04-02T13:36:26.727600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:48:59.291360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.494535+0000","last_change":"2024-04-02T13:36:20.830008+0000","last_active":"2024-04-02T13:36:46.494535+0000","last_peered":"2024-04-02T13:36:46.494535+0000","last_clean":"2024-04-02T13:36:46.494535+0000","last_became_active":"2024-04-02T13:36:20.829048+0000","last_became_peered":"2024-04-02T13:36:20.829048+0000","last_unstale":"2024-04-02T13:36:46.494535+0000","last_undegraded":"2024-04-02T13:36:46.494535+0000","last_fullsized":"2024-04-02T13:36:46.494535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:08:34.405351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473608+0000","last_change":"2024-04-02T13:36:21.497296+0000","last_active":"2024-04-02T13:36:26.473608+0000","last_peered":"2024-04-02T13:36:26.473608+0000","last_clean":"2024-04-02T13:36:26.473608+0000","last_became_active":"2024-04-02T13:36:21.496802+0000","last_became_peered":"2024-04-02T13:36:21.496802+0000","last_unstale":"2024-04-02T13:36:26.473608+0000","last_undegraded":"2024-04-02T13:36:26.473608+0000","last_fullsized":"2024-04-02T13:36:26.473608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:03.995029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727386+0000","last_change":"2024-04-02T13:36:25.498661+0000","last_active":"2024-04-02T13:36:26.727386+0000","last_peered":"2024-04-02T13:36:26.727386+0000","last_clean":"2024-04-02T13:36:26.727386+0000","last_became_active":"2024-04-02T13:36:25.497866+0000","last_became_peered":"2024-04-02T13:36:25.497866+0000","last_unstale":"2024-04-02T13:36:26.727386+0000","last_undegraded":"2024-04-02T13:36:26.727386+0000","last_fullsized":"2024-04-02T13:36:26.727386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:47:05.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725988+0000","last_change":"2024-04-02T13:36:20.824586+0000","last_active":"2024-04-02T13:36:26.725988+0000","last_peered":"2024-04-02T13:36:26.725988+0000","last_clean":"2024-04-02T13:36:26.725988+0000","last_became_active":"2024-04-02T13:36:20.824254+0000","last_became_peered":"2024-04-02T13:36:20.824254+0000","last_unstale":"2024-04-02T13:36:26.725988+0000","last_undegraded":"2024-04-02T13:36:26.725988+0000","last_fullsized":"2024-04-02T13:36:26.725988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:08.774361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.474250+0000","last_change":"2024-04-02T13:36:21.495125+0000","last_active":"2024-04-02T13:36:26.474250+0000","last_peered":"2024-04-02T13:36:26.474250+0000","last_clean":"2024-04-02T13:36:26.474250+0000","last_became_active":"2024-04-02T13:36:21.494724+0000","last_became_peered":"2024-04-02T13:36:21.494724+0000","last_unstale":"2024-04-02T13:36:26.474250+0000","last_undegraded":"2024-04-02T13:36:26.474250+0000","last_fullsized":"2024-04-02T13:36:26.474250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:27.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725928+0000","last_change":"2024-04-02T13:36:25.498708+0000","last_active":"2024-04-02T13:36:26.725928+0000","last_peered":"2024-04-02T13:36:26.725928+0000","last_clean":"2024-04-02T13:36:26.725928+0000","last_became_active":"2024-04-02T13:36:25.497834+0000","last_became_peered":"2024-04-02T13:36:25.497834+0000","last_unstale":"2024-04-02T13:36:26.725928+0000","last_undegraded":"2024-04-02T13:36:26.725928+0000","last_fullsized":"2024-04-02T13:36:26.725928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:14.524569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:48.488619+0000","last_change":"2024-04-02T13:36:20.824901+0000","last_active":"2024-04-02T13:36:48.488619+0000","last_peered":"2024-04-02T13:36:48.488619+0000","last_clean":"2024-04-02T13:36:48.488619+0000","last_became_active":"2024-04-02T13:36:20.823973+0000","last_became_peered":"2024-04-02T13:36:20.823973+0000","last_unstale":"2024-04-02T13:36:48.488619+0000","last_undegraded":"2024-04-02T13:36:48.488619+0000","last_fullsized":"2024-04-02T13:36:48.488619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:17.636527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725893+0000","last_change":"2024-04-02T13:36:21.493411+0000","last_active":"2024-04-02T13:36:26.725893+0000","last_peered":"2024-04-02T13:36:26.725893+0000","last_clean":"2024-04-02T13:36:26.725893+0000","last_became_active":"2024-04-02T13:36:21.493061+0000","last_became_peered":"2024-04-02T13:36:21.493061+0000","last_unstale":"2024-04-02T13:36:26.725893+0000","last_undegraded":"2024-04-02T13:36:26.725893+0000","last_fullsized":"2024-04-02T13:36:26.725893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:15.902656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726395+0000","last_change":"2024-04-02T13:36:25.494817+0000","last_active":"2024-04-02T13:36:26.726395+0000","last_peered":"2024-04-02T13:36:26.726395+0000","last_clean":"2024-04-02T13:36:26.726395+0000","last_became_active":"2024-04-02T13:36:25.494185+0000","last_became_peered":"2024-04-02T13:36:25.494185+0000","last_unstale":"2024-04-02T13:36:26.726395+0000","last_undegraded":"2024-04-02T13:36:26.726395+0000","last_fullsized":"2024-04-02T13:36:26.726395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:04:32.719985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.495135+0000","last_change":"2024-04-02T13:36:20.821981+0000","last_active":"2024-04-02T13:36:46.495135+0000","last_peered":"2024-04-02T13:36:46.495135+0000","last_clean":"2024-04-02T13:36:46.495135+0000","last_became_active":"2024-04-02T13:36:20.821074+0000","last_became_peered":"2024-04-02T13:36:20.821074+0000","last_unstale":"2024-04-02T13:36:46.495135+0000","last_undegraded":"2024-04-02T13:36:46.495135+0000","last_fullsized":"2024-04-02T13:36:46.495135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:22.094586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.474238+0000","last_change":"2024-04-02T13:36:21.490792+0000","last_active":"2024-04-02T13:36:26.474238+0000","last_peered":"2024-04-02T13:36:26.474238+0000","last_clean":"2024-04-02T13:36:26.474238+0000","last_became_active":"2024-04-02T13:36:21.490471+0000","last_became_peered":"2024-04-02T13:36:21.490471+0000","last_unstale":"2024-04-02T13:36:26.474238+0000","last_undegraded":"2024-04-02T13:36:26.474238+0000","last_fullsized":"2024-04-02T13:36:26.474238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:55.612316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433299+0000","last_change":"2024-04-02T13:36:25.503774+0000","last_active":"2024-04-02T13:36:27.433299+0000","last_peered":"2024-04-02T13:36:27.433299+0000","last_clean":"2024-04-02T13:36:27.433299+0000","last_became_active":"2024-04-02T13:36:25.503287+0000","last_became_peered":"2024-04-02T13:36:25.503287+0000","last_unstale":"2024-04-02T13:36:27.433299+0000","last_undegraded":"2024-04-02T13:36:27.433299+0000","last_fullsized":"2024-04-02T13:36:27.433299+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:57:29.939484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.495514+0000","last_change":"2024-04-02T13:36:20.816760+0000","last_active":"2024-04-02T13:36:46.495514+0000","last_peered":"2024-04-02T13:36:46.495514+0000","last_clean":"2024-04-02T13:36:46.495514+0000","last_became_active":"2024-04-02T13:36:20.816291+0000","last_became_peered":"2024-04-02T13:36:20.816291+0000","last_unstale":"2024-04-02T13:36:46.495514+0000","last_undegraded":"2024-04-02T13:36:46.495514+0000","last_fullsized":"2024-04-02T13:36:46.495514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:25.543946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727518+0000","last_change":"2024-04-02T13:36:21.498541+0000","last_active":"2024-04-02T13:36:26.727518+0000","last_peered":"2024-04-02T13:36:26.727518+0000","last_clean":"2024-04-02T13:36:26.727518+0000","last_became_active":"2024-04-02T13:36:21.497498+0000","last_became_peered":"2024-04-02T13:36:21.497498+0000","last_unstale":"2024-04-02T13:36:26.727518+0000","last_undegraded":"2024-04-02T13:36:26.727518+0000","last_fullsized":"2024-04-02T13:36:26.727518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:08.315938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726191+0000","last_change":"2024-04-02T13:36:25.500245+0000","last_active":"2024-04-02T13:36:26.726191+0000","last_peered":"2024-04-02T13:36:26.726191+0000","last_clean":"2024-04-02T13:36:26.726191+0000","last_became_active":"2024-04-02T13:36:25.499844+0000","last_became_peered":"2024-04-02T13:36:25.499844+0000","last_unstale":"2024-04-02T13:36:26.726191+0000","last_undegraded":"2024-04-02T13:36:26.726191+0000","last_fullsized":"2024-04-02T13:36:26.726191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:17:01.105757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473734+0000","last_change":"2024-04-02T13:36:20.826851+0000","last_active":"2024-04-02T13:36:26.473734+0000","last_peered":"2024-04-02T13:36:26.473734+0000","last_clean":"2024-04-02T13:36:26.473734+0000","last_became_active":"2024-04-02T13:36:20.826227+0000","last_became_peered":"2024-04-02T13:36:20.826227+0000","last_unstale":"2024-04-02T13:36:26.473734+0000","last_undegraded":"2024-04-02T13:36:26.473734+0000","last_fullsized":"2024-04-02T13:36:26.473734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:45.841966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725299+0000","last_change":"2024-04-02T13:36:21.496520+0000","last_active":"2024-04-02T13:36:26.725299+0000","last_peered":"2024-04-02T13:36:26.725299+0000","last_clean":"2024-04-02T13:36:26.725299+0000","last_became_active":"2024-04-02T13:36:21.496082+0000","last_became_peered":"2024-04-02T13:36:21.496082+0000","last_unstale":"2024-04-02T13:36:26.725299+0000","last_undegraded":"2024-04-02T13:36:26.725299+0000","last_fullsized":"2024-04-02T13:36:26.725299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:12:06.517806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727554+0000","last_change":"2024-04-02T13:36:25.496064+0000","last_active":"2024-04-02T13:36:26.727554+0000","last_peered":"2024-04-02T13:36:26.727554+0000","last_clean":"2024-04-02T13:36:26.727554+0000","last_became_active":"2024-04-02T13:36:25.495464+0000","last_became_peered":"2024-04-02T13:36:25.495464+0000","last_unstale":"2024-04-02T13:36:26.727554+0000","last_undegraded":"2024-04-02T13:36:26.727554+0000","last_fullsized":"2024-04-02T13:36:26.727554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:06:15.206135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:48.487543+0000","last_change":"2024-04-02T13:36:20.821898+0000","last_active":"2024-04-02T13:36:48.487543+0000","last_peered":"2024-04-02T13:36:48.487543+0000","last_clean":"2024-04-02T13:36:48.487543+0000","last_became_active":"2024-04-02T13:36:20.821374+0000","last_became_peered":"2024-04-02T13:36:20.821374+0000","last_unstale":"2024-04-02T13:36:48.487543+0000","last_undegraded":"2024-04-02T13:36:48.487543+0000","last_fullsized":"2024-04-02T13:36:48.487543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:21.568976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473920+0000","last_change":"2024-04-02T13:36:21.497343+0000","last_active":"2024-04-02T13:36:26.473920+0000","last_peered":"2024-04-02T13:36:26.473920+0000","last_clean":"2024-04-02T13:36:26.473920+0000","last_became_active":"2024-04-02T13:36:21.496733+0000","last_became_peered":"2024-04-02T13:36:21.496733+0000","last_unstale":"2024-04-02T13:36:26.473920+0000","last_undegraded":"2024-04-02T13:36:26.473920+0000","last_fullsized":"2024-04-02T13:36:26.473920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:26.068717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725306+0000","last_change":"2024-04-02T13:36:25.503011+0000","last_active":"2024-04-02T13:36:26.725306+0000","last_peered":"2024-04-02T13:36:26.725306+0000","last_clean":"2024-04-02T13:36:26.725306+0000","last_became_active":"2024-04-02T13:36:25.502571+0000","last_became_peered":"2024-04-02T13:36:25.502571+0000","last_unstale":"2024-04-02T13:36:26.725306+0000","last_undegraded":"2024-04-02T13:36:26.725306+0000","last_fullsized":"2024-04-02T13:36:26.725306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:48.672023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.546358+0000","last_change":"2024-04-02T13:36:20.827705+0000","last_active":"2024-04-02T13:36:45.546358+0000","last_peered":"2024-04-02T13:36:45.546358+0000","last_clean":"2024-04-02T13:36:45.546358+0000","last_became_active":"2024-04-02T13:36:20.827285+0000","last_became_peered":"2024-04-02T13:36:20.827285+0000","last_unstale":"2024-04-02T13:36:45.546358+0000","last_undegraded":"2024-04-02T13:36:45.546358+0000","last_fullsized":"2024-04-02T13:36:45.546358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:12.175523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726387+0000","last_change":"2024-04-02T13:36:21.493365+0000","last_active":"2024-04-02T13:36:26.726387+0000","last_peered":"2024-04-02T13:36:26.726387+0000","last_clean":"2024-04-02T13:36:26.726387+0000","last_became_active":"2024-04-02T13:36:21.493011+0000","last_became_peered":"2024-04-02T13:36:21.493011+0000","last_unstale":"2024-04-02T13:36:26.726387+0000","last_undegraded":"2024-04-02T13:36:26.726387+0000","last_fullsized":"2024-04-02T13:36:26.726387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:53.165244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727784+0000","last_change":"2024-04-02T13:36:25.499017+0000","last_active":"2024-04-02T13:36:26.727784+0000","last_peered":"2024-04-02T13:36:26.727784+0000","last_clean":"2024-04-02T13:36:26.727784+0000","last_became_active":"2024-04-02T13:36:25.498293+0000","last_became_peered":"2024-04-02T13:36:25.498293+0000","last_unstale":"2024-04-02T13:36:26.727784+0000","last_undegraded":"2024-04-02T13:36:26.727784+0000","last_fullsized":"2024-04-02T13:36:26.727784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:55.910943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.547606+0000","last_change":"2024-04-02T13:36:20.836345+0000","last_active":"2024-04-02T13:36:45.547606+0000","last_peered":"2024-04-02T13:36:45.547606+0000","last_clean":"2024-04-02T13:36:45.547606+0000","last_became_active":"2024-04-02T13:36:20.836037+0000","last_became_peered":"2024-04-02T13:36:20.836037+0000","last_unstale":"2024-04-02T13:36:45.547606+0000","last_undegraded":"2024-04-02T13:36:45.547606+0000","last_fullsized":"2024-04-02T13:36:45.547606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:47.070493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473839+0000","last_change":"2024-04-02T13:36:21.493209+0000","last_active":"2024-04-02T13:36:26.473839+0000","last_peered":"2024-04-02T13:36:26.473839+0000","last_clean":"2024-04-02T13:36:26.473839+0000","last_became_active":"2024-04-02T13:36:21.492894+0000","last_became_peered":"2024-04-02T13:36:21.492894+0000","last_unstale":"2024-04-02T13:36:26.473839+0000","last_undegraded":"2024-04-02T13:36:26.473839+0000","last_fullsized":"2024-04-02T13:36:26.473839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:47.162354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725737+0000","last_change":"2024-04-02T13:36:25.506752+0000","last_active":"2024-04-02T13:36:26.725737+0000","last_peered":"2024-04-02T13:36:26.725737+0000","last_clean":"2024-04-02T13:36:26.725737+0000","last_became_active":"2024-04-02T13:36:25.506363+0000","last_became_peered":"2024-04-02T13:36:25.506363+0000","last_unstale":"2024-04-02T13:36:26.725737+0000","last_undegraded":"2024-04-02T13:36:26.725737+0000","last_fullsized":"2024-04-02T13:36:26.725737+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:15:21.640418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'23","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:49.491435+0000","last_change":"2024-04-02T13:36:20.823563+0000","last_active":"2024-04-02T13:36:49.491435+0000","last_peered":"2024-04-02T13:36:49.491435+0000","last_clean":"2024-04-02T13:36:49.491435+0000","last_became_active":"2024-04-02T13:36:20.823186+0000","last_became_peered":"2024-04-02T13:36:20.823186+0000","last_unstale":"2024-04-02T13:36:49.491435+0000","last_undegraded":"2024-04-02T13:36:49.491435+0000","last_fullsized":"2024-04-02T13:36:49.491435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:50.983357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40471,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":45,"num_write":24,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726088+0000","last_change":"2024-04-02T13:36:21.500829+0000","last_active":"2024-04-02T13:36:26.726088+0000","last_peered":"2024-04-02T13:36:26.726088+0000","last_clean":"2024-04-02T13:36:26.726088+0000","last_became_active":"2024-04-02T13:36:21.500485+0000","last_became_peered":"2024-04-02T13:36:21.500485+0000","last_unstale":"2024-04-02T13:36:26.726088+0000","last_undegraded":"2024-04-02T13:36:26.726088+0000","last_fullsized":"2024-04-02T13:36:26.726088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:23:19.627565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.474668+0000","last_change":"2024-04-02T13:36:25.506695+0000","last_active":"2024-04-02T13:36:26.474668+0000","last_peered":"2024-04-02T13:36:26.474668+0000","last_clean":"2024-04-02T13:36:26.474668+0000","last_became_active":"2024-04-02T13:36:25.506254+0000","last_became_peered":"2024-04-02T13:36:25.506254+0000","last_unstale":"2024-04-02T13:36:26.474668+0000","last_undegraded":"2024-04-02T13:36:26.474668+0000","last_fullsized":"2024-04-02T13:36:26.474668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:57:00.315005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.547607+0000","last_change":"2024-04-02T13:36:20.825827+0000","last_active":"2024-04-02T13:36:45.547607+0000","last_peered":"2024-04-02T13:36:45.547607+0000","last_clean":"2024-04-02T13:36:45.547607+0000","last_became_active":"2024-04-02T13:36:20.825530+0000","last_became_peered":"2024-04-02T13:36:20.825530+0000","last_unstale":"2024-04-02T13:36:45.547607+0000","last_undegraded":"2024-04-02T13:36:45.547607+0000","last_fullsized":"2024-04-02T13:36:45.547607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:27:02.962513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473300+0000","last_change":"2024-04-02T13:36:21.486343+0000","last_active":"2024-04-02T13:36:26.473300+0000","last_peered":"2024-04-02T13:36:26.473300+0000","last_clean":"2024-04-02T13:36:26.473300+0000","last_became_active":"2024-04-02T13:36:21.486021+0000","last_became_peered":"2024-04-02T13:36:21.486021+0000","last_unstale":"2024-04-02T13:36:26.473300+0000","last_undegraded":"2024-04-02T13:36:26.473300+0000","last_fullsized":"2024-04-02T13:36:26.473300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:53:42.430173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433041+0000","last_change":"2024-04-02T13:36:25.502806+0000","last_active":"2024-04-02T13:36:27.433041+0000","last_peered":"2024-04-02T13:36:27.433041+0000","last_clean":"2024-04-02T13:36:27.433041+0000","last_became_active":"2024-04-02T13:36:25.502289+0000","last_became_peered":"2024-04-02T13:36:25.502289+0000","last_unstale":"2024-04-02T13:36:27.433041+0000","last_undegraded":"2024-04-02T13:36:27.433041+0000","last_fullsized":"2024-04-02T13:36:27.433041+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:03:42.637972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.546948+0000","last_change":"2024-04-02T13:36:20.829303+0000","last_active":"2024-04-02T13:36:45.546948+0000","last_peered":"2024-04-02T13:36:45.546948+0000","last_clean":"2024-04-02T13:36:45.546948+0000","last_became_active":"2024-04-02T13:36:20.828952+0000","last_became_peered":"2024-04-02T13:36:20.828952+0000","last_unstale":"2024-04-02T13:36:45.546948+0000","last_undegraded":"2024-04-02T13:36:45.546948+0000","last_fullsized":"2024-04-02T13:36:45.546948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:14:04.385178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726643+0000","last_change":"2024-04-02T13:36:21.491767+0000","last_active":"2024-04-02T13:36:26.726643+0000","last_peered":"2024-04-02T13:36:26.726643+0000","last_clean":"2024-04-02T13:36:26.726643+0000","last_became_active":"2024-04-02T13:36:21.491369+0000","last_became_peered":"2024-04-02T13:36:21.491369+0000","last_unstale":"2024-04-02T13:36:26.726643+0000","last_undegraded":"2024-04-02T13:36:26.726643+0000","last_fullsized":"2024-04-02T13:36:26.726643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:14:35.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725470+0000","last_change":"2024-04-02T13:36:25.503900+0000","last_active":"2024-04-02T13:36:26.725470+0000","last_peered":"2024-04-02T13:36:26.725470+0000","last_clean":"2024-04-02T13:36:26.725470+0000","last_became_active":"2024-04-02T13:36:25.502840+0000","last_became_peered":"2024-04-02T13:36:25.502840+0000","last_unstale":"2024-04-02T13:36:26.725470+0000","last_undegraded":"2024-04-02T13:36:26.725470+0000","last_fullsized":"2024-04-02T13:36:26.725470+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:31.106865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727405+0000","last_change":"2024-04-02T13:36:20.823753+0000","last_active":"2024-04-02T13:36:26.727405+0000","last_peered":"2024-04-02T13:36:26.727405+0000","last_clean":"2024-04-02T13:36:26.727405+0000","last_became_active":"2024-04-02T13:36:20.823394+0000","last_became_peered":"2024-04-02T13:36:20.823394+0000","last_unstale":"2024-04-02T13:36:26.727405+0000","last_undegraded":"2024-04-02T13:36:26.727405+0000","last_fullsized":"2024-04-02T13:36:26.727405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:13.023125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726359+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:36:26.726359+0000","last_peered":"2024-04-02T13:36:26.726359+0000","last_clean":"2024-04-02T13:36:26.726359+0000","last_became_active":"2024-04-02T13:36:21.499197+0000","last_became_peered":"2024-04-02T13:36:21.499197+0000","last_unstale":"2024-04-02T13:36:26.726359+0000","last_undegraded":"2024-04-02T13:36:26.726359+0000","last_fullsized":"2024-04-02T13:36:26.726359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:26.435832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473949+0000","last_change":"2024-04-02T13:36:25.504643+0000","last_active":"2024-04-02T13:36:26.473949+0000","last_peered":"2024-04-02T13:36:26.473949+0000","last_clean":"2024-04-02T13:36:26.473949+0000","last_became_active":"2024-04-02T13:36:25.504249+0000","last_became_peered":"2024-04-02T13:36:25.504249+0000","last_unstale":"2024-04-02T13:36:26.473949+0000","last_undegraded":"2024-04-02T13:36:26.473949+0000","last_fullsized":"2024-04-02T13:36:26.473949+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:15.818958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.496244+0000","last_change":"2024-04-02T13:36:20.830097+0000","last_active":"2024-04-02T13:36:46.496244+0000","last_peered":"2024-04-02T13:36:46.496244+0000","last_clean":"2024-04-02T13:36:46.496244+0000","last_became_active":"2024-04-02T13:36:20.829779+0000","last_became_peered":"2024-04-02T13:36:20.829779+0000","last_unstale":"2024-04-02T13:36:46.496244+0000","last_undegraded":"2024-04-02T13:36:46.496244+0000","last_fullsized":"2024-04-02T13:36:46.496244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:59.477730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473722+0000","last_change":"2024-04-02T13:36:21.497480+0000","last_active":"2024-04-02T13:36:26.473722+0000","last_peered":"2024-04-02T13:36:26.473722+0000","last_clean":"2024-04-02T13:36:26.473722+0000","last_became_active":"2024-04-02T13:36:21.497106+0000","last_became_peered":"2024-04-02T13:36:21.497106+0000","last_unstale":"2024-04-02T13:36:26.473722+0000","last_undegraded":"2024-04-02T13:36:26.473722+0000","last_fullsized":"2024-04-02T13:36:26.473722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:30.394968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473799+0000","last_change":"2024-04-02T13:36:25.500040+0000","last_active":"2024-04-02T13:36:26.473799+0000","last_peered":"2024-04-02T13:36:26.473799+0000","last_clean":"2024-04-02T13:36:26.473799+0000","last_became_active":"2024-04-02T13:36:25.499576+0000","last_became_peered":"2024-04-02T13:36:25.499576+0000","last_unstale":"2024-04-02T13:36:26.473799+0000","last_undegraded":"2024-04-02T13:36:26.473799+0000","last_fullsized":"2024-04-02T13:36:26.473799+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:15:47.475234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726162+0000","last_change":"2024-04-02T13:36:20.823225+0000","last_active":"2024-04-02T13:36:26.726162+0000","last_peered":"2024-04-02T13:36:26.726162+0000","last_clean":"2024-04-02T13:36:26.726162+0000","last_became_active":"2024-04-02T13:36:20.822865+0000","last_became_peered":"2024-04-02T13:36:20.822865+0000","last_unstale":"2024-04-02T13:36:26.726162+0000","last_undegraded":"2024-04-02T13:36:26.726162+0000","last_fullsized":"2024-04-02T13:36:26.726162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:00.488190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433213+0000","last_change":"2024-04-02T13:36:21.497692+0000","last_active":"2024-04-02T13:36:27.433213+0000","last_peered":"2024-04-02T13:36:27.433213+0000","last_clean":"2024-04-02T13:36:27.433213+0000","last_became_active":"2024-04-02T13:36:21.497322+0000","last_became_peered":"2024-04-02T13:36:21.497322+0000","last_unstale":"2024-04-02T13:36:27.433213+0000","last_undegraded":"2024-04-02T13:36:27.433213+0000","last_fullsized":"2024-04-02T13:36:27.433213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:17.590235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725823+0000","last_change":"2024-04-02T13:36:25.531063+0000","last_active":"2024-04-02T13:36:26.725823+0000","last_peered":"2024-04-02T13:36:26.725823+0000","last_clean":"2024-04-02T13:36:26.725823+0000","last_became_active":"2024-04-02T13:36:25.530608+0000","last_became_peered":"2024-04-02T13:36:25.530608+0000","last_unstale":"2024-04-02T13:36:26.725823+0000","last_undegraded":"2024-04-02T13:36:26.725823+0000","last_fullsized":"2024-04-02T13:36:26.725823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:51:27.710136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725580+0000","last_change":"2024-04-02T13:36:20.835277+0000","last_active":"2024-04-02T13:36:26.725580+0000","last_peered":"2024-04-02T13:36:26.725580+0000","last_clean":"2024-04-02T13:36:26.725580+0000","last_became_active":"2024-04-02T13:36:20.834911+0000","last_became_peered":"2024-04-02T13:36:20.834911+0000","last_unstale":"2024-04-02T13:36:26.725580+0000","last_undegraded":"2024-04-02T13:36:26.725580+0000","last_fullsized":"2024-04-02T13:36:26.725580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:20.965544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726693+0000","last_change":"2024-04-02T13:36:21.495274+0000","last_active":"2024-04-02T13:36:26.726693+0000","last_peered":"2024-04-02T13:36:26.726693+0000","last_clean":"2024-04-02T13:36:26.726693+0000","last_became_active":"2024-04-02T13:36:21.494841+0000","last_became_peered":"2024-04-02T13:36:21.494841+0000","last_unstale":"2024-04-02T13:36:26.726693+0000","last_undegraded":"2024-04-02T13:36:26.726693+0000","last_fullsized":"2024-04-02T13:36:26.726693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:20.096507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473699+0000","last_change":"2024-04-02T13:36:25.499970+0000","last_active":"2024-04-02T13:36:26.473699+0000","last_peered":"2024-04-02T13:36:26.473699+0000","last_clean":"2024-04-02T13:36:26.473699+0000","last_became_active":"2024-04-02T13:36:25.499539+0000","last_became_peered":"2024-04-02T13:36:25.499539+0000","last_unstale":"2024-04-02T13:36:26.473699+0000","last_undegraded":"2024-04-02T13:36:26.473699+0000","last_fullsized":"2024-04-02T13:36:26.473699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:43.136635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726360+0000","last_change":"2024-04-02T13:36:20.831150+0000","last_active":"2024-04-02T13:36:26.726360+0000","last_peered":"2024-04-02T13:36:26.726360+0000","last_clean":"2024-04-02T13:36:26.726360+0000","last_became_active":"2024-04-02T13:36:20.830838+0000","last_became_peered":"2024-04-02T13:36:20.830838+0000","last_unstale":"2024-04-02T13:36:26.726360+0000","last_undegraded":"2024-04-02T13:36:26.726360+0000","last_fullsized":"2024-04-02T13:36:26.726360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:23:09.566911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726275+0000","last_change":"2024-04-02T13:36:21.503128+0000","last_active":"2024-04-02T13:36:26.726275+0000","last_peered":"2024-04-02T13:36:26.726275+0000","last_clean":"2024-04-02T13:36:26.726275+0000","last_became_active":"2024-04-02T13:36:21.502695+0000","last_became_peered":"2024-04-02T13:36:21.502695+0000","last_unstale":"2024-04-02T13:36:26.726275+0000","last_undegraded":"2024-04-02T13:36:26.726275+0000","last_fullsized":"2024-04-02T13:36:26.726275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:38.261472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473948+0000","last_change":"2024-04-02T13:36:25.530911+0000","last_active":"2024-04-02T13:36:26.473948+0000","last_peered":"2024-04-02T13:36:26.473948+0000","last_clean":"2024-04-02T13:36:26.473948+0000","last_became_active":"2024-04-02T13:36:25.530428+0000","last_became_peered":"2024-04-02T13:36:25.530428+0000","last_unstale":"2024-04-02T13:36:26.473948+0000","last_undegraded":"2024-04-02T13:36:26.473948+0000","last_fullsized":"2024-04-02T13:36:26.473948+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:49.185987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473842+0000","last_change":"2024-04-02T13:36:21.492251+0000","last_active":"2024-04-02T13:36:26.473842+0000","last_peered":"2024-04-02T13:36:26.473842+0000","last_clean":"2024-04-02T13:36:26.473842+0000","last_became_active":"2024-04-02T13:36:21.491929+0000","last_became_peered":"2024-04-02T13:36:21.491929+0000","last_unstale":"2024-04-02T13:36:26.473842+0000","last_undegraded":"2024-04-02T13:36:26.473842+0000","last_fullsized":"2024-04-02T13:36:26.473842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:59.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.497149+0000","last_change":"2024-04-02T13:36:20.821431+0000","last_active":"2024-04-02T13:36:46.497149+0000","last_peered":"2024-04-02T13:36:46.497149+0000","last_clean":"2024-04-02T13:36:46.497149+0000","last_became_active":"2024-04-02T13:36:20.821067+0000","last_became_peered":"2024-04-02T13:36:20.821067+0000","last_unstale":"2024-04-02T13:36:46.497149+0000","last_undegraded":"2024-04-02T13:36:46.497149+0000","last_fullsized":"2024-04-02T13:36:46.497149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:52.043907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727189+0000","last_change":"2024-04-02T13:36:25.496992+0000","last_active":"2024-04-02T13:36:26.727189+0000","last_peered":"2024-04-02T13:36:26.727189+0000","last_clean":"2024-04-02T13:36:26.727189+0000","last_became_active":"2024-04-02T13:36:25.496467+0000","last_became_peered":"2024-04-02T13:36:25.496467+0000","last_unstale":"2024-04-02T13:36:26.727189+0000","last_undegraded":"2024-04-02T13:36:26.727189+0000","last_fullsized":"2024-04-02T13:36:26.727189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:13:00.747716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726003+0000","last_change":"2024-04-02T13:36:21.491399+0000","last_active":"2024-04-02T13:36:26.726003+0000","last_peered":"2024-04-02T13:36:26.726003+0000","last_clean":"2024-04-02T13:36:26.726003+0000","last_became_active":"2024-04-02T13:36:21.491038+0000","last_became_peered":"2024-04-02T13:36:21.491038+0000","last_unstale":"2024-04-02T13:36:26.726003+0000","last_undegraded":"2024-04-02T13:36:26.726003+0000","last_fullsized":"2024-04-02T13:36:26.726003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:25.603086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.547804+0000","last_change":"2024-04-02T13:36:20.822796+0000","last_active":"2024-04-02T13:36:45.547804+0000","last_peered":"2024-04-02T13:36:45.547804+0000","last_clean":"2024-04-02T13:36:45.547804+0000","last_became_active":"2024-04-02T13:36:20.822168+0000","last_became_peered":"2024-04-02T13:36:20.822168+0000","last_unstale":"2024-04-02T13:36:45.547804+0000","last_undegraded":"2024-04-02T13:36:45.547804+0000","last_fullsized":"2024-04-02T13:36:45.547804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:11.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726022+0000","last_change":"2024-04-02T13:35:08.678967+0000","last_active":"2024-04-02T13:36:26.726022+0000","last_peered":"2024-04-02T13:36:26.726022+0000","last_clean":"2024-04-02T13:36:26.726022+0000","last_became_active":"2024-04-02T13:35:08.678408+0000","last_became_peered":"2024-04-02T13:35:08.678408+0000","last_unstale":"2024-04-02T13:36:26.726022+0000","last_undegraded":"2024-04-02T13:36:26.726022+0000","last_fullsized":"2024-04-02T13:36:26.726022+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_clean_scrub_stamp":"2024-04-02T13:31:42.453938+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:30:20.020809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.474405+0000","last_change":"2024-04-02T13:36:25.532723+0000","last_active":"2024-04-02T13:36:26.474405+0000","last_peered":"2024-04-02T13:36:26.474405+0000","last_clean":"2024-04-02T13:36:26.474405+0000","last_became_active":"2024-04-02T13:36:25.532341+0000","last_became_peered":"2024-04-02T13:36:25.532341+0000","last_unstale":"2024-04-02T13:36:26.474405+0000","last_undegraded":"2024-04-02T13:36:26.474405+0000","last_fullsized":"2024-04-02T13:36:26.474405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:53:43.190339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473026+0000","last_change":"2024-04-02T13:36:21.488983+0000","last_active":"2024-04-02T13:36:26.473026+0000","last_peered":"2024-04-02T13:36:26.473026+0000","last_clean":"2024-04-02T13:36:26.473026+0000","last_became_active":"2024-04-02T13:36:21.488580+0000","last_became_peered":"2024-04-02T13:36:21.488580+0000","last_unstale":"2024-04-02T13:36:26.473026+0000","last_undegraded":"2024-04-02T13:36:26.473026+0000","last_fullsized":"2024-04-02T13:36:26.473026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:39.382233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.495546+0000","last_change":"2024-04-02T13:36:20.833438+0000","last_active":"2024-04-02T13:36:46.495546+0000","last_peered":"2024-04-02T13:36:46.495546+0000","last_clean":"2024-04-02T13:36:46.495546+0000","last_became_active":"2024-04-02T13:36:20.833080+0000","last_became_peered":"2024-04-02T13:36:20.833080+0000","last_unstale":"2024-04-02T13:36:46.495546+0000","last_undegraded":"2024-04-02T13:36:46.495546+0000","last_fullsized":"2024-04-02T13:36:46.495546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:07:11.575387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433053+0000","last_change":"2024-04-02T13:36:25.502814+0000","last_active":"2024-04-02T13:36:27.433053+0000","last_peered":"2024-04-02T13:36:27.433053+0000","last_clean":"2024-04-02T13:36:27.433053+0000","last_became_active":"2024-04-02T13:36:25.502288+0000","last_became_peered":"2024-04-02T13:36:25.502288+0000","last_unstale":"2024-04-02T13:36:27.433053+0000","last_undegraded":"2024-04-02T13:36:27.433053+0000","last_fullsized":"2024-04-02T13:36:27.433053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:48.073286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.546013+0000","last_change":"2024-04-02T13:36:20.830512+0000","last_active":"2024-04-02T13:36:45.546013+0000","last_peered":"2024-04-02T13:36:45.546013+0000","last_clean":"2024-04-02T13:36:45.546013+0000","last_became_active":"2024-04-02T13:36:20.830086+0000","last_became_peered":"2024-04-02T13:36:20.830086+0000","last_unstale":"2024-04-02T13:36:45.546013+0000","last_undegraded":"2024-04-02T13:36:45.546013+0000","last_fullsized":"2024-04-02T13:36:45.546013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:07.428363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473676+0000","last_change":"2024-04-02T13:36:21.478218+0000","last_active":"2024-04-02T13:36:26.473676+0000","last_peered":"2024-04-02T13:36:26.473676+0000","last_clean":"2024-04-02T13:36:26.473676+0000","last_became_active":"2024-04-02T13:36:21.477856+0000","last_became_peered":"2024-04-02T13:36:21.477856+0000","last_unstale":"2024-04-02T13:36:26.473676+0000","last_undegraded":"2024-04-02T13:36:26.473676+0000","last_fullsized":"2024-04-02T13:36:26.473676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:04.397606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727473+0000","last_change":"2024-04-02T13:36:25.530915+0000","last_active":"2024-04-02T13:36:26.727473+0000","last_peered":"2024-04-02T13:36:26.727473+0000","last_clean":"2024-04-02T13:36:26.727473+0000","last_became_active":"2024-04-02T13:36:25.530343+0000","last_became_peered":"2024-04-02T13:36:25.530343+0000","last_unstale":"2024-04-02T13:36:26.727473+0000","last_undegraded":"2024-04-02T13:36:26.727473+0000","last_fullsized":"2024-04-02T13:36:26.727473+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:32:41.181485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:46.496602+0000","last_change":"2024-04-02T13:36:20.828527+0000","last_active":"2024-04-02T13:36:46.496602+0000","last_peered":"2024-04-02T13:36:46.496602+0000","last_clean":"2024-04-02T13:36:46.496602+0000","last_became_active":"2024-04-02T13:36:20.828136+0000","last_became_peered":"2024-04-02T13:36:20.828136+0000","last_unstale":"2024-04-02T13:36:46.496602+0000","last_undegraded":"2024-04-02T13:36:46.496602+0000","last_fullsized":"2024-04-02T13:36:46.496602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:57.709400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727334+0000","last_change":"2024-04-02T13:36:21.492035+0000","last_active":"2024-04-02T13:36:26.727334+0000","last_peered":"2024-04-02T13:36:26.727334+0000","last_clean":"2024-04-02T13:36:26.727334+0000","last_became_active":"2024-04-02T13:36:21.491682+0000","last_became_peered":"2024-04-02T13:36:21.491682+0000","last_unstale":"2024-04-02T13:36:26.727334+0000","last_undegraded":"2024-04-02T13:36:26.727334+0000","last_fullsized":"2024-04-02T13:36:26.727334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:09:40.600616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725438+0000","last_change":"2024-04-02T13:36:25.493301+0000","last_active":"2024-04-02T13:36:26.725438+0000","last_peered":"2024-04-02T13:36:26.725438+0000","last_clean":"2024-04-02T13:36:26.725438+0000","last_became_active":"2024-04-02T13:36:25.492846+0000","last_became_peered":"2024-04-02T13:36:25.492846+0000","last_unstale":"2024-04-02T13:36:26.725438+0000","last_undegraded":"2024-04-02T13:36:26.725438+0000","last_fullsized":"2024-04-02T13:36:26.725438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:12:53.350261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.545842+0000","last_change":"2024-04-02T13:36:20.823559+0000","last_active":"2024-04-02T13:36:45.545842+0000","last_peered":"2024-04-02T13:36:45.545842+0000","last_clean":"2024-04-02T13:36:45.545842+0000","last_became_active":"2024-04-02T13:36:20.823178+0000","last_became_peered":"2024-04-02T13:36:20.823178+0000","last_unstale":"2024-04-02T13:36:45.545842+0000","last_undegraded":"2024-04-02T13:36:45.545842+0000","last_fullsized":"2024-04-02T13:36:45.545842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:42.628855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726253+0000","last_change":"2024-04-02T13:36:21.497258+0000","last_active":"2024-04-02T13:36:26.726253+0000","last_peered":"2024-04-02T13:36:26.726253+0000","last_clean":"2024-04-02T13:36:26.726253+0000","last_became_active":"2024-04-02T13:36:21.496943+0000","last_became_peered":"2024-04-02T13:36:21.496943+0000","last_unstale":"2024-04-02T13:36:26.726253+0000","last_undegraded":"2024-04-02T13:36:26.726253+0000","last_fullsized":"2024-04-02T13:36:26.726253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:51.989025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727971+0000","last_change":"2024-04-02T13:36:25.531251+0000","last_active":"2024-04-02T13:36:26.727971+0000","last_peered":"2024-04-02T13:36:26.727971+0000","last_clean":"2024-04-02T13:36:26.727971+0000","last_became_active":"2024-04-02T13:36:25.530505+0000","last_became_peered":"2024-04-02T13:36:25.530505+0000","last_unstale":"2024-04-02T13:36:26.727971+0000","last_undegraded":"2024-04-02T13:36:26.727971+0000","last_fullsized":"2024-04-02T13:36:26.727971+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:02:10.711138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.547180+0000","last_change":"2024-04-02T13:36:20.837865+0000","last_active":"2024-04-02T13:36:45.547180+0000","last_peered":"2024-04-02T13:36:45.547180+0000","last_clean":"2024-04-02T13:36:45.547180+0000","last_became_active":"2024-04-02T13:36:20.837567+0000","last_became_peered":"2024-04-02T13:36:20.837567+0000","last_unstale":"2024-04-02T13:36:45.547180+0000","last_undegraded":"2024-04-02T13:36:45.547180+0000","last_fullsized":"2024-04-02T13:36:45.547180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:11:59.365592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725420+0000","last_change":"2024-04-02T13:36:21.500085+0000","last_active":"2024-04-02T13:36:26.725420+0000","last_peered":"2024-04-02T13:36:26.725420+0000","last_clean":"2024-04-02T13:36:26.725420+0000","last_became_active":"2024-04-02T13:36:21.499706+0000","last_became_peered":"2024-04-02T13:36:21.499706+0000","last_unstale":"2024-04-02T13:36:26.725420+0000","last_undegraded":"2024-04-02T13:36:26.725420+0000","last_fullsized":"2024-04-02T13:36:26.725420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.378226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725732+0000","last_change":"2024-04-02T13:36:25.505797+0000","last_active":"2024-04-02T13:36:26.725732+0000","last_peered":"2024-04-02T13:36:26.725732+0000","last_clean":"2024-04-02T13:36:26.725732+0000","last_became_active":"2024-04-02T13:36:25.505409+0000","last_became_peered":"2024-04-02T13:36:25.505409+0000","last_unstale":"2024-04-02T13:36:26.725732+0000","last_undegraded":"2024-04-02T13:36:26.725732+0000","last_fullsized":"2024-04-02T13:36:26.725732+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:46:31.723248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.548579+0000","last_change":"2024-04-02T13:36:20.821572+0000","last_active":"2024-04-02T13:36:45.548579+0000","last_peered":"2024-04-02T13:36:45.548579+0000","last_clean":"2024-04-02T13:36:45.548579+0000","last_became_active":"2024-04-02T13:36:20.821235+0000","last_became_peered":"2024-04-02T13:36:20.821235+0000","last_unstale":"2024-04-02T13:36:45.548579+0000","last_undegraded":"2024-04-02T13:36:45.548579+0000","last_fullsized":"2024-04-02T13:36:45.548579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:02.304145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.432872+0000","last_change":"2024-04-02T13:36:21.485255+0000","last_active":"2024-04-02T13:36:27.432872+0000","last_peered":"2024-04-02T13:36:27.432872+0000","last_clean":"2024-04-02T13:36:27.432872+0000","last_became_active":"2024-04-02T13:36:21.484870+0000","last_became_peered":"2024-04-02T13:36:21.484870+0000","last_unstale":"2024-04-02T13:36:27.432872+0000","last_undegraded":"2024-04-02T13:36:27.432872+0000","last_fullsized":"2024-04-02T13:36:27.432872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:03.851161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727821+0000","last_change":"2024-04-02T13:36:25.494161+0000","last_active":"2024-04-02T13:36:26.727821+0000","last_peered":"2024-04-02T13:36:26.727821+0000","last_clean":"2024-04-02T13:36:26.727821+0000","last_became_active":"2024-04-02T13:36:25.493667+0000","last_became_peered":"2024-04-02T13:36:25.493667+0000","last_unstale":"2024-04-02T13:36:26.727821+0000","last_undegraded":"2024-04-02T13:36:26.727821+0000","last_fullsized":"2024-04-02T13:36:26.727821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:35:45.703988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725751+0000","last_change":"2024-04-02T13:36:20.825219+0000","last_active":"2024-04-02T13:36:26.725751+0000","last_peered":"2024-04-02T13:36:26.725751+0000","last_clean":"2024-04-02T13:36:26.725751+0000","last_became_active":"2024-04-02T13:36:20.824878+0000","last_became_peered":"2024-04-02T13:36:20.824878+0000","last_unstale":"2024-04-02T13:36:26.725751+0000","last_undegraded":"2024-04-02T13:36:26.725751+0000","last_fullsized":"2024-04-02T13:36:26.725751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:12.579433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726487+0000","last_change":"2024-04-02T13:36:21.498640+0000","last_active":"2024-04-02T13:36:26.726487+0000","last_peered":"2024-04-02T13:36:26.726487+0000","last_clean":"2024-04-02T13:36:26.726487+0000","last_became_active":"2024-04-02T13:36:21.497996+0000","last_became_peered":"2024-04-02T13:36:21.497996+0000","last_unstale":"2024-04-02T13:36:26.726487+0000","last_undegraded":"2024-04-02T13:36:26.726487+0000","last_fullsized":"2024-04-02T13:36:26.726487+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:56:45.748945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.728067+0000","last_change":"2024-04-02T13:36:25.531119+0000","last_active":"2024-04-02T13:36:26.728067+0000","last_peered":"2024-04-02T13:36:26.728067+0000","last_clean":"2024-04-02T13:36:26.728067+0000","last_became_active":"2024-04-02T13:36:25.530307+0000","last_became_peered":"2024-04-02T13:36:25.530307+0000","last_unstale":"2024-04-02T13:36:26.728067+0000","last_undegraded":"2024-04-02T13:36:26.728067+0000","last_fullsized":"2024-04-02T13:36:26.728067+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:58:50.543386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:45.546092+0000","last_change":"2024-04-02T13:36:20.829843+0000","last_active":"2024-04-02T13:36:45.546092+0000","last_peered":"2024-04-02T13:36:45.546092+0000","last_clean":"2024-04-02T13:36:45.546092+0000","last_became_active":"2024-04-02T13:36:20.829502+0000","last_became_peered":"2024-04-02T13:36:20.829502+0000","last_unstale":"2024-04-02T13:36:45.546092+0000","last_undegraded":"2024-04-02T13:36:45.546092+0000","last_fullsized":"2024-04-02T13:36:45.546092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:36:45.016174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726425+0000","last_change":"2024-04-02T13:36:21.491579+0000","last_active":"2024-04-02T13:36:26.726425+0000","last_peered":"2024-04-02T13:36:26.726425+0000","last_clean":"2024-04-02T13:36:26.726425+0000","last_became_active":"2024-04-02T13:36:21.490992+0000","last_became_peered":"2024-04-02T13:36:21.490992+0000","last_unstale":"2024-04-02T13:36:26.726425+0000","last_undegraded":"2024-04-02T13:36:26.726425+0000","last_fullsized":"2024-04-02T13:36:26.726425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:41.899766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473493+0000","last_change":"2024-04-02T13:36:25.502658+0000","last_active":"2024-04-02T13:36:26.473493+0000","last_peered":"2024-04-02T13:36:26.473493+0000","last_clean":"2024-04-02T13:36:26.473493+0000","last_became_active":"2024-04-02T13:36:25.502219+0000","last_became_peered":"2024-04-02T13:36:25.502219+0000","last_unstale":"2024-04-02T13:36:26.473493+0000","last_undegraded":"2024-04-02T13:36:26.473493+0000","last_fullsized":"2024-04-02T13:36:26.473493+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:34.189381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726564+0000","last_change":"2024-04-02T13:36:20.835176+0000","last_active":"2024-04-02T13:36:26.726564+0000","last_peered":"2024-04-02T13:36:26.726564+0000","last_clean":"2024-04-02T13:36:26.726564+0000","last_became_active":"2024-04-02T13:36:20.834871+0000","last_became_peered":"2024-04-02T13:36:20.834871+0000","last_unstale":"2024-04-02T13:36:26.726564+0000","last_undegraded":"2024-04-02T13:36:26.726564+0000","last_fullsized":"2024-04-02T13:36:26.726564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:17:14.148762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727352+0000","last_change":"2024-04-02T13:36:21.490336+0000","last_active":"2024-04-02T13:36:26.727352+0000","last_peered":"2024-04-02T13:36:26.727352+0000","last_clean":"2024-04-02T13:36:26.727352+0000","last_became_active":"2024-04-02T13:36:21.489869+0000","last_became_peered":"2024-04-02T13:36:21.489869+0000","last_unstale":"2024-04-02T13:36:26.727352+0000","last_undegraded":"2024-04-02T13:36:26.727352+0000","last_fullsized":"2024-04-02T13:36:26.727352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:42.243209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725578+0000","last_change":"2024-04-02T13:36:25.503124+0000","last_active":"2024-04-02T13:36:26.725578+0000","last_peered":"2024-04-02T13:36:26.725578+0000","last_clean":"2024-04-02T13:36:26.725578+0000","last_became_active":"2024-04-02T13:36:25.502737+0000","last_became_peered":"2024-04-02T13:36:25.502737+0000","last_unstale":"2024-04-02T13:36:26.725578+0000","last_undegraded":"2024-04-02T13:36:26.725578+0000","last_fullsized":"2024-04-02T13:36:26.725578+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:02:56.830746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.474433+0000","last_change":"2024-04-02T13:36:20.828597+0000","last_active":"2024-04-02T13:36:26.474433+0000","last_peered":"2024-04-02T13:36:26.474433+0000","last_clean":"2024-04-02T13:36:26.474433+0000","last_became_active":"2024-04-02T13:36:20.828138+0000","last_became_peered":"2024-04-02T13:36:20.828138+0000","last_unstale":"2024-04-02T13:36:26.474433+0000","last_undegraded":"2024-04-02T13:36:26.474433+0000","last_fullsized":"2024-04-02T13:36:26.474433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:05.171448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726691+0000","last_change":"2024-04-02T13:36:21.498498+0000","last_active":"2024-04-02T13:36:26.726691+0000","last_peered":"2024-04-02T13:36:26.726691+0000","last_clean":"2024-04-02T13:36:26.726691+0000","last_became_active":"2024-04-02T13:36:21.498119+0000","last_became_peered":"2024-04-02T13:36:21.498119+0000","last_unstale":"2024-04-02T13:36:26.726691+0000","last_undegraded":"2024-04-02T13:36:26.726691+0000","last_fullsized":"2024-04-02T13:36:26.726691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:01.528228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473582+0000","last_change":"2024-04-02T13:36:25.497189+0000","last_active":"2024-04-02T13:36:26.473582+0000","last_peered":"2024-04-02T13:36:26.473582+0000","last_clean":"2024-04-02T13:36:26.473582+0000","last_became_active":"2024-04-02T13:36:25.496711+0000","last_became_peered":"2024-04-02T13:36:25.496711+0000","last_unstale":"2024-04-02T13:36:26.473582+0000","last_undegraded":"2024-04-02T13:36:26.473582+0000","last_fullsized":"2024-04-02T13:36:26.473582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:33.452746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473694+0000","last_change":"2024-04-02T13:36:20.823053+0000","last_active":"2024-04-02T13:36:26.473694+0000","last_peered":"2024-04-02T13:36:26.473694+0000","last_clean":"2024-04-02T13:36:26.473694+0000","last_became_active":"2024-04-02T13:36:20.822665+0000","last_became_peered":"2024-04-02T13:36:20.822665+0000","last_unstale":"2024-04-02T13:36:26.473694+0000","last_undegraded":"2024-04-02T13:36:26.473694+0000","last_fullsized":"2024-04-02T13:36:26.473694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:43:51.148843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726766+0000","last_change":"2024-04-02T13:36:21.494069+0000","last_active":"2024-04-02T13:36:26.726766+0000","last_peered":"2024-04-02T13:36:26.726766+0000","last_clean":"2024-04-02T13:36:26.726766+0000","last_became_active":"2024-04-02T13:36:21.493751+0000","last_became_peered":"2024-04-02T13:36:21.493751+0000","last_unstale":"2024-04-02T13:36:26.726766+0000","last_undegraded":"2024-04-02T13:36:26.726766+0000","last_fullsized":"2024-04-02T13:36:26.726766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:24:39.648318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.432946+0000","last_change":"2024-04-02T13:36:21.494280+0000","last_active":"2024-04-02T13:36:27.432946+0000","last_peered":"2024-04-02T13:36:27.432946+0000","last_clean":"2024-04-02T13:36:27.432946+0000","last_became_active":"2024-04-02T13:36:21.491262+0000","last_became_peered":"2024-04-02T13:36:21.491262+0000","last_unstale":"2024-04-02T13:36:27.432946+0000","last_undegraded":"2024-04-02T13:36:27.432946+0000","last_fullsized":"2024-04-02T13:36:27.432946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:08:16.728646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725685+0000","last_change":"2024-04-02T13:36:21.497103+0000","last_active":"2024-04-02T13:36:26.725685+0000","last_peered":"2024-04-02T13:36:26.725685+0000","last_clean":"2024-04-02T13:36:26.725685+0000","last_became_active":"2024-04-02T13:36:21.496784+0000","last_became_peered":"2024-04-02T13:36:21.496784+0000","last_unstale":"2024-04-02T13:36:26.725685+0000","last_undegraded":"2024-04-02T13:36:26.725685+0000","last_fullsized":"2024-04-02T13:36:26.725685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:38:21.903362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725610+0000","last_change":"2024-04-02T13:36:21.491853+0000","last_active":"2024-04-02T13:36:26.725610+0000","last_peered":"2024-04-02T13:36:26.725610+0000","last_clean":"2024-04-02T13:36:26.725610+0000","last_became_active":"2024-04-02T13:36:21.487986+0000","last_became_peered":"2024-04-02T13:36:21.487986+0000","last_unstale":"2024-04-02T13:36:26.725610+0000","last_undegraded":"2024-04-02T13:36:26.725610+0000","last_fullsized":"2024-04-02T13:36:26.725610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:37:47.098681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.725813+0000","last_change":"2024-04-02T13:36:21.493453+0000","last_active":"2024-04-02T13:36:26.725813+0000","last_peered":"2024-04-02T13:36:26.725813+0000","last_clean":"2024-04-02T13:36:26.725813+0000","last_became_active":"2024-04-02T13:36:21.492789+0000","last_became_peered":"2024-04-02T13:36:21.492789+0000","last_unstale":"2024-04-02T13:36:26.725813+0000","last_undegraded":"2024-04-02T13:36:26.725813+0000","last_fullsized":"2024-04-02T13:36:26.725813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:12:27.157466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473764+0000","last_change":"2024-04-02T13:36:21.488723+0000","last_active":"2024-04-02T13:36:26.473764+0000","last_peered":"2024-04-02T13:36:26.473764+0000","last_clean":"2024-04-02T13:36:26.473764+0000","last_became_active":"2024-04-02T13:36:21.488391+0000","last_became_peered":"2024-04-02T13:36:21.488391+0000","last_unstale":"2024-04-02T13:36:26.473764+0000","last_undegraded":"2024-04-02T13:36:26.473764+0000","last_fullsized":"2024-04-02T13:36:26.473764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:23.768286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433285+0000","last_change":"2024-04-02T13:36:21.492676+0000","last_active":"2024-04-02T13:36:27.433285+0000","last_peered":"2024-04-02T13:36:27.433285+0000","last_clean":"2024-04-02T13:36:27.433285+0000","last_became_active":"2024-04-02T13:36:21.491663+0000","last_became_peered":"2024-04-02T13:36:21.491663+0000","last_unstale":"2024-04-02T13:36:27.433285+0000","last_undegraded":"2024-04-02T13:36:27.433285+0000","last_fullsized":"2024-04-02T13:36:27.433285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:00.131388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.726355+0000","last_change":"2024-04-02T13:36:21.493380+0000","last_active":"2024-04-02T13:36:26.726355+0000","last_peered":"2024-04-02T13:36:26.726355+0000","last_clean":"2024-04-02T13:36:26.726355+0000","last_became_active":"2024-04-02T13:36:21.493073+0000","last_became_peered":"2024-04-02T13:36:21.493073+0000","last_unstale":"2024-04-02T13:36:26.726355+0000","last_undegraded":"2024-04-02T13:36:26.726355+0000","last_fullsized":"2024-04-02T13:36:26.726355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:59.412056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727455+0000","last_change":"2024-04-02T13:36:21.500881+0000","last_active":"2024-04-02T13:36:26.727455+0000","last_peered":"2024-04-02T13:36:26.727455+0000","last_clean":"2024-04-02T13:36:26.727455+0000","last_became_active":"2024-04-02T13:36:21.500516+0000","last_became_peered":"2024-04-02T13:36:21.500516+0000","last_unstale":"2024-04-02T13:36:26.727455+0000","last_undegraded":"2024-04-02T13:36:26.727455+0000","last_fullsized":"2024-04-02T13:36:26.727455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:33.842494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473733+0000","last_change":"2024-04-02T13:36:21.494558+0000","last_active":"2024-04-02T13:36:26.473733+0000","last_peered":"2024-04-02T13:36:26.473733+0000","last_clean":"2024-04-02T13:36:26.473733+0000","last_became_active":"2024-04-02T13:36:21.494180+0000","last_became_peered":"2024-04-02T13:36:21.494180+0000","last_unstale":"2024-04-02T13:36:26.473733+0000","last_undegraded":"2024-04-02T13:36:26.473733+0000","last_fullsized":"2024-04-02T13:36:26.473733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:07:54.363504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.727621+0000","last_change":"2024-04-02T13:36:21.500406+0000","last_active":"2024-04-02T13:36:26.727621+0000","last_peered":"2024-04-02T13:36:26.727621+0000","last_clean":"2024-04-02T13:36:26.727621+0000","last_became_active":"2024-04-02T13:36:21.499927+0000","last_became_peered":"2024-04-02T13:36:21.499927+0000","last_unstale":"2024-04-02T13:36:26.727621+0000","last_undegraded":"2024-04-02T13:36:26.727621+0000","last_fullsized":"2024-04-02T13:36:26.727621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:23.702776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473578+0000","last_change":"2024-04-02T13:36:21.486166+0000","last_active":"2024-04-02T13:36:26.473578+0000","last_peered":"2024-04-02T13:36:26.473578+0000","last_clean":"2024-04-02T13:36:26.473578+0000","last_became_active":"2024-04-02T13:36:21.485282+0000","last_became_peered":"2024-04-02T13:36:21.485282+0000","last_unstale":"2024-04-02T13:36:26.473578+0000","last_undegraded":"2024-04-02T13:36:26.473578+0000","last_fullsized":"2024-04-02T13:36:26.473578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:24:07.007675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:26.473390+0000","last_change":"2024-04-02T13:36:21.484909+0000","last_active":"2024-04-02T13:36:26.473390+0000","last_peered":"2024-04-02T13:36:26.473390+0000","last_clean":"2024-04-02T13:36:26.473390+0000","last_became_active":"2024-04-02T13:36:21.484559+0000","last_became_peered":"2024-04-02T13:36:21.484559+0000","last_unstale":"2024-04-02T13:36:26.473390+0000","last_undegraded":"2024-04-02T13:36:26.473390+0000","last_fullsized":"2024-04-02T13:36:26.473390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:59:45.848407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:36:27.433123+0000","last_change":"2024-04-02T13:36:21.493961+0000","last_active":"2024-04-02T13:36:27.433123+0000","last_peered":"2024-04-02T13:36:27.433123+0000","last_clean":"2024-04-02T13:36:27.433123+0000","last_became_active":"2024-04-02T13:36:21.493490+0000","last_became_peered":"2024-04-02T13:36:21.493490+0000","last_unstale":"2024-04-02T13:36:27.433123+0000","last_undegraded":"2024-04-02T13:36:27.433123+0000","last_fullsized":"2024-04-02T13:36:27.433123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:26:42.032550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":213880,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43268,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":30,"num_read_kb":59,"num_write":131,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":57869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":480266,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612631,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":1113074,"data_compressed":50891,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776157,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":485180,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":480892,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299431,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":1071838,"data_compressed":50891,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":480356,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":484108,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":480960,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":484142,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":480550,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888117,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":480288,"data_compressed":14699,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724593,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1076597,"data_compressed":50891,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34556,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34556,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":42440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:36:50.704 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-02T13:36:50.704 INFO:tasks.cephadm:Setting up client nodes... 2024-04-02T13:36:50.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-02T13:36:50.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:50 smithi145 ceph-mon[35646]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:50.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:50 smithi145 ceph-mon[35646]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:50.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:50 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4080371048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:50.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:50 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4049193582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:50.985 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:36:50.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:50 smithi050 ceph-mon[35524]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:50.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:50 smithi050 ceph-mon[35524]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:36:50.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:50 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4080371048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:50.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:50 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4049193582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:51.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:51 smithi033 ceph-mon[32390]: pgmap v307: 129 pgs: 129 active+clean; 619 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-04-02T13:36:51.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:51 smithi033 ceph-mon[32390]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:36:51.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:51 smithi033 ceph-mon[32390]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:36:51.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:51 smithi145 ceph-mon[35646]: pgmap v307: 129 pgs: 129 active+clean; 619 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-04-02T13:36:51.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:51 smithi145 ceph-mon[35646]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:36:51.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:51 smithi145 ceph-mon[35646]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:36:51.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:51 smithi050 ceph-mon[35524]: pgmap v307: 129 pgs: 129 active+clean; 619 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-04-02T13:36:51.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:51 smithi050 ceph-mon[35524]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:36:51.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:51 smithi050 ceph-mon[35524]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:36:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 -- 172.21.15.33:0/1117390013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c01083b0 msgr2=0x7f59c010a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 --2- 172.21.15.33:0/1117390013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c01083b0 0x7f59c010a820 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f59b800b0a0 tx=0x7f59b800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:36:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 -- 172.21.15.33:0/1117390013 shutdown_connections 2024-04-02T13:36:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 --2- 172.21.15.33:0/1117390013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c01083b0 0x7f59c010a820 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 --2- 172.21.15.33:0/1117390013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59c0103150 0x7f59c0105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 --2- 172.21.15.33:0/1117390013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59c0105a80 0x7f59c0107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 -- 172.21.15.33:0/1117390013 >> 172.21.15.33:0/1117390013 conn(0x7f59c00fcb80 msgr2=0x7f59c00fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 -- 172.21.15.33:0/1117390013 shutdown_connections 2024-04-02T13:36:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 -- 172.21.15.33:0/1117390013 wait complete. 2024-04-02T13:36:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 Processor -- start 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 -- start start 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59c0103150 0x7f59c0198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0105a80 0x7f59c0198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59c01083b0 0x7f59c019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59c010e200 con 0x7f59c0105a80 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f59c010e080 con 0x7f59c0103150 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.771+0000 7f59c6b0e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f59c010e380 con 0x7f59c01083b0 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0105a80 0x7f59c0198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0105a80 0x7f59c0198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35696/0 (socket says 172.21.15.33:35696) 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59bffff700 1 -- 172.21.15.33:0/3175524492 learned_addr learned my addr 172.21.15.33:0/3175524492 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59c50ab700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59c01083b0 0x7f59c019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59c48aa700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59c0103150 0x7f59c0198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59bffff700 1 -- 172.21.15.33:0/3175524492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59c01083b0 msgr2=0x7f59c019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59bffff700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59c01083b0 0x7f59c019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59bffff700 1 -- 172.21.15.33:0/3175524492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59c0103150 msgr2=0x7f59c0198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59bffff700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59c0103150 0x7f59c0198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59bffff700 1 -- 172.21.15.33:0/3175524492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c01a02e0 con 0x7f59c0105a80 2024-04-02T13:36:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59bffff700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0105a80 0x7f59c0198bc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00d980 tx=0x7f59ac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59bdffb700 1 -- 172.21.15.33:0/3175524492 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac016070 con 0x7f59c0105a80 2024-04-02T13:36:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59c6b0e700 1 -- 172.21.15.33:0/3175524492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c01a05d0 con 0x7f59c0105a80 2024-04-02T13:36:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59bdffb700 1 -- 172.21.15.33:0/3175524492 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f59ac009b40 con 0x7f59c0105a80 2024-04-02T13:36:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.772+0000 7f59bdffb700 1 -- 172.21.15.33:0/3175524492 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00f3c0 con 0x7f59c0105a80 2024-04-02T13:36:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.773+0000 7f59c6b0e700 1 -- 172.21.15.33:0/3175524492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c01a0b10 con 0x7f59c0105a80 2024-04-02T13:36:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.774+0000 7f59c6b0e700 1 -- 172.21.15.33:0/3175524492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c0050b00 con 0x7f59c0105a80 2024-04-02T13:36:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.774+0000 7f59bdffb700 1 -- 172.21.15.33:0/3175524492 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f59ac019070 con 0x7f59c0105a80 2024-04-02T13:36:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.775+0000 7f59bdffb700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f59b00a3d80 0x7f59b00a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.775+0000 7f59c48aa700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f59b00a3d80 0x7f59b00a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.775+0000 7f59bdffb700 1 -- 172.21.15.33:0/3175524492 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f59ac0d03d0 con 0x7f59c0105a80 2024-04-02T13:36:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.776+0000 7f59c48aa700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f59b00a3d80 0x7f59b00a6240 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f59b4005ea0 tx=0x7f59b4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:52.780+0000 7f59bdffb700 1 -- 172.21.15.33:0/3175524492 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f59ac09ca10 con 0x7f59c0105a80 2024-04-02T13:36:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.072+0000 7f59c6b0e700 1 -- 172.21.15.33:0/3175524492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f59c0063200 con 0x7f59c0105a80 2024-04-02T13:36:53.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.075+0000 7f59bdffb700 1 -- 172.21.15.33:0/3175524492 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f59ac09c310 con 0x7f59c0105a80 2024-04-02T13:36:53.077 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-04-02T13:36:53.078 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQD1CQxm7lRsBBAAw+dhtq4/I1WFdBUxoLhuRA== 2024-04-02T13:36:53.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.077+0000 7f59c6b0e700 1 -- 172.21.15.33:0/3175524492 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f59b00a3d80 msgr2=0x7f59b00a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:53.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.077+0000 7f59c6b0e700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f59b00a3d80 0x7f59b00a6240 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f59b4005ea0 tx=0x7f59b4005e30 comp rx=0 tx=0).stop 2024-04-02T13:36:53.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.077+0000 7f59c6b0e700 1 -- 172.21.15.33:0/3175524492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0105a80 msgr2=0x7f59c0198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:53.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.077+0000 7f59c6b0e700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0105a80 0x7f59c0198bc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00d980 tx=0x7f59ac00dd40 comp rx=0 tx=0).stop 2024-04-02T13:36:53.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.077+0000 7f59c6b0e700 1 -- 172.21.15.33:0/3175524492 shutdown_connections 2024-04-02T13:36:53.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.077+0000 7f59c6b0e700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f59b00a3d80 0x7f59b00a6240 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:53.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.077+0000 7f59c6b0e700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59c01083b0 0x7f59c019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:53.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.077+0000 7f59c6b0e700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f59c0103150 0x7f59c0198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:53.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.077+0000 7f59c6b0e700 1 --2- 172.21.15.33:0/3175524492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0105a80 0x7f59c0198bc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:53.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.077+0000 7f59c6b0e700 1 -- 172.21.15.33:0/3175524492 >> 172.21.15.33:0/3175524492 conn(0x7f59c00fcb80 msgr2=0x7f59c0103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:53.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.077+0000 7f59c6b0e700 1 -- 172.21.15.33:0/3175524492 shutdown_connections 2024-04-02T13:36:53.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:53.077+0000 7f59c6b0e700 1 -- 172.21.15.33:0/3175524492 wait complete. 2024-04-02T13:36:53.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:53 smithi033 ceph-mon[32390]: pgmap v308: 129 pgs: 129 active+clean; 620 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-04-02T13:36:53.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:53 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3175524492' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T13:36:53.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:53 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3175524492' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T13:36:53.805 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:36:53.805 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-02T13:36:53.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-02T13:36:53.848 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-02T13:36:53.848 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-02T13:36:53.848 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph mgr dump --format=json 2024-04-02T13:36:53.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:53 smithi145 ceph-mon[35646]: pgmap v308: 129 pgs: 129 active+clean; 620 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-04-02T13:36:53.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:53 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3175524492' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T13:36:53.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:53 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3175524492' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T13:36:53.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:53 smithi050 ceph-mon[35524]: pgmap v308: 129 pgs: 129 active+clean; 620 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-04-02T13:36:53.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:53 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3175524492' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T13:36:53.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:53 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3175524492' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T13:36:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:36:55.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:55 smithi033 ceph-mon[32390]: pgmap v309: 129 pgs: 129 active+clean; 621 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-04-02T13:36:55.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:55 smithi145 ceph-mon[35646]: pgmap v309: 129 pgs: 129 active+clean; 621 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-04-02T13:36:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.991+0000 7fbebde2b700 1 -- 172.21.15.33:0/457364249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb8107500 msgr2=0x7fbeb81078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.991+0000 7fbebde2b700 1 --2- 172.21.15.33:0/457364249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb8107500 0x7fbeb81078e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009990 tx=0x7fbea8009ca0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.991+0000 7fbebde2b700 1 -- 172.21.15.33:0/457364249 shutdown_connections 2024-04-02T13:36:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.991+0000 7fbebde2b700 1 --2- 172.21.15.33:0/457364249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb8101eb0 0x7fbeb810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.991+0000 7fbebde2b700 1 --2- 172.21.15.33:0/457364249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb8107500 0x7fbeb81078e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.991+0000 7fbebde2b700 1 --2- 172.21.15.33:0/457364249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeb81014f0 0x7fbeb8101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.991+0000 7fbebde2b700 1 -- 172.21.15.33:0/457364249 >> 172.21.15.33:0/457364249 conn(0x7fbeb8078420 msgr2=0x7fbeb8078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.991+0000 7fbebde2b700 1 -- 172.21.15.33:0/457364249 shutdown_connections 2024-04-02T13:36:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.991+0000 7fbebde2b700 1 -- 172.21.15.33:0/457364249 wait complete. 2024-04-02T13:36:55.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:55 smithi050 ceph-mon[35524]: pgmap v309: 129 pgs: 129 active+clean; 621 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-04-02T13:36:55.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.992+0000 7fbebde2b700 1 Processor -- start 2024-04-02T13:36:55.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.992+0000 7fbebde2b700 1 -- start start 2024-04-02T13:36:55.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.992+0000 7fbebde2b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb81014f0 0x7fbeb8103130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.992+0000 7fbebde2b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeb8101eb0 0x7fbeb8103670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.992+0000 7fbebde2b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb8107500 0x7fbeb8103bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.992+0000 7fbebde2b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8111780 con 0x7fbeb81014f0 2024-04-02T13:36:55.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.992+0000 7fbebde2b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8111600 con 0x7fbeb8101eb0 2024-04-02T13:36:55.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.992+0000 7fbebde2b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8111900 con 0x7fbeb8107500 2024-04-02T13:36:55.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.992+0000 7fbeb77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb81014f0 0x7fbeb8103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.992+0000 7fbeb77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb81014f0 0x7fbeb8103130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50900/0 (socket says 172.21.15.33:50900) 2024-04-02T13:36:55.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.992+0000 7fbeb77fe700 1 -- 172.21.15.33:0/365246570 learned_addr learned my addr 172.21.15.33:0/365246570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.992+0000 7fbeb6ffd700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeb8101eb0 0x7fbeb8103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.992+0000 7fbeb7fff700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb8107500 0x7fbeb8103bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.993+0000 7fbeb77fe700 1 -- 172.21.15.33:0/365246570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb8107500 msgr2=0x7fbeb8103bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.993+0000 7fbeb77fe700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb8107500 0x7fbeb8103bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.993+0000 7fbeb77fe700 1 -- 172.21.15.33:0/365246570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeb8101eb0 msgr2=0x7fbeb8103670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.993+0000 7fbeb77fe700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeb8101eb0 0x7fbeb8103670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.993+0000 7fbeb77fe700 1 -- 172.21.15.33:0/365246570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb81a71e0 con 0x7fbeb81014f0 2024-04-02T13:36:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.993+0000 7fbeb7fff700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb8107500 0x7fbeb8103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.993+0000 7fbeb77fe700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb81014f0 0x7fbeb8103130 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbea80038b0 tx=0x7fbea8009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.993+0000 7fbeb4ff9700 1 -- 172.21.15.33:0/365246570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8027070 con 0x7fbeb81014f0 2024-04-02T13:36:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.993+0000 7fbebde2b700 1 -- 172.21.15.33:0/365246570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb81a7470 con 0x7fbeb81014f0 2024-04-02T13:36:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.993+0000 7fbeb4ff9700 1 -- 172.21.15.33:0/365246570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbea8004040 con 0x7fbeb81014f0 2024-04-02T13:36:55.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.993+0000 7fbeb4ff9700 1 -- 172.21.15.33:0/365246570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8022c80 con 0x7fbeb81014f0 2024-04-02T13:36:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.993+0000 7fbebde2b700 1 -- 172.21.15.33:0/365246570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb81a7950 con 0x7fbeb81014f0 2024-04-02T13:36:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.994+0000 7fbebde2b700 1 -- 172.21.15.33:0/365246570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb80692b0 con 0x7fbeb81014f0 2024-04-02T13:36:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.995+0000 7fbeb4ff9700 1 -- 172.21.15.33:0/365246570 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbea802a070 con 0x7fbeb81014f0 2024-04-02T13:36:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.996+0000 7fbeb4ff9700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbea00a3cd0 0x7fbea00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.996+0000 7fbeb6ffd700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbea00a3cd0 0x7fbea00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.996+0000 7fbeb4ff9700 1 -- 172.21.15.33:0/365246570 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbea80d1370 con 0x7fbeb81014f0 2024-04-02T13:36:56.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:55.997+0000 7fbeb6ffd700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbea00a3cd0 0x7fbea00a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbeb81062e0 tx=0x7fbeac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:56.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.001+0000 7fbeb4ff9700 1 -- 172.21.15.33:0/365246570 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbea8013070 con 0x7fbeb81014f0 2024-04-02T13:36:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.269+0000 7fbebde2b700 1 -- 172.21.15.33:0/365246570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fbeb81a7bb0 con 0x7fbeb81014f0 2024-04-02T13:36:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.274+0000 7fbeb4ff9700 1 -- 172.21.15.33:0/365246570 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264995 (secure 0 0 0) 0x7fbea8022e20 con 0x7fbeb81014f0 2024-04-02T13:36:56.283 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.277+0000 7fbebde2b700 1 -- 172.21.15.33:0/365246570 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbea00a3cd0 msgr2=0x7fbea00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.277+0000 7fbebde2b700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbea00a3cd0 0x7fbea00a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbeb81062e0 tx=0x7fbeac00b410 comp rx=0 tx=0).stop 2024-04-02T13:36:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.277+0000 7fbebde2b700 1 -- 172.21.15.33:0/365246570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb81014f0 msgr2=0x7fbeb8103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.277+0000 7fbebde2b700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb81014f0 0x7fbeb8103130 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbea80038b0 tx=0x7fbea8009ee0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.277+0000 7fbebde2b700 1 -- 172.21.15.33:0/365246570 shutdown_connections 2024-04-02T13:36:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.277+0000 7fbebde2b700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbea00a3cd0 0x7fbea00a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.277+0000 7fbebde2b700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb8107500 0x7fbeb8103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.277+0000 7fbebde2b700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb81014f0 0x7fbeb8103130 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.277+0000 7fbebde2b700 1 --2- 172.21.15.33:0/365246570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeb8101eb0 0x7fbeb8103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.277+0000 7fbebde2b700 1 -- 172.21.15.33:0/365246570 >> 172.21.15.33:0/365246570 conn(0x7fbeb8078420 msgr2=0x7fbeb8100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.277+0000 7fbebde2b700 1 -- 172.21.15.33:0/365246570 shutdown_connections 2024-04-02T13:36:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:56.277+0000 7fbebde2b700 1 -- 172.21.15.33:0/365246570 wait complete. 2024-04-02T13:36:56.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:56 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/365246570' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T13:36:56.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:56 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/365246570' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T13:36:57.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:56 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/365246570' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T13:36:57.041 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2562306694},{"type":"v1","addr":"172.21.15.33:6801","nonce":2562306694}]},"active_addr":"172.21.15.33:6801/2562306694","active_change":"2024-04-02T13:29:15.778942+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.33:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":3947066882}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":395956530}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1212609914}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":80287983}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2097011388}]}]} 2024-04-02T13:36:57.046 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-02T13:36:57.046 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-02T13:36:57.047 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T13:36:57.297 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:36:57.750 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:57 smithi033 ceph-mon[32390]: pgmap v310: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-02T13:36:57.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:57 smithi145 ceph-mon[35646]: pgmap v310: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-02T13:36:57.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:57 smithi050 ceph-mon[35524]: pgmap v310: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-02T13:36:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.933+0000 7fc1fb90a700 1 -- 172.21.15.33:0/1702768288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f41014f0 msgr2=0x7fc1f4101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.933+0000 7fc1fb90a700 1 --2- 172.21.15.33:0/1702768288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f41014f0 0x7fc1f4101970 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009990 tx=0x7fc1e4009ca0 comp rx=0 tx=0).stop 2024-04-02T13:36:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.933+0000 7fc1fb90a700 1 -- 172.21.15.33:0/1702768288 shutdown_connections 2024-04-02T13:36:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.933+0000 7fc1fb90a700 1 --2- 172.21.15.33:0/1702768288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f4101eb0 0x7fc1f410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.933+0000 7fc1fb90a700 1 --2- 172.21.15.33:0/1702768288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1f4107500 0x7fc1f41078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.933+0000 7fc1fb90a700 1 --2- 172.21.15.33:0/1702768288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f41014f0 0x7fc1f4101970 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.933+0000 7fc1fb90a700 1 -- 172.21.15.33:0/1702768288 >> 172.21.15.33:0/1702768288 conn(0x7fc1f4078420 msgr2=0x7fc1f4078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.933+0000 7fc1fb90a700 1 -- 172.21.15.33:0/1702768288 shutdown_connections 2024-04-02T13:36:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.933+0000 7fc1fb90a700 1 -- 172.21.15.33:0/1702768288 wait complete. 2024-04-02T13:36:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.934+0000 7fc1fb90a700 1 Processor -- start 2024-04-02T13:36:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.934+0000 7fc1fb90a700 1 -- start start 2024-04-02T13:36:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.934+0000 7fc1fb90a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f41014f0 0x7fc1f4198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.934+0000 7fc1fb90a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1f4101eb0 0x7fc1f41993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.934+0000 7fc1fb90a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f4107500 0x7fc1f419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.934+0000 7fc1fb90a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f4111860 con 0x7fc1f4107500 2024-04-02T13:36:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.934+0000 7fc1fb90a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f41116e0 con 0x7fc1f4101eb0 2024-04-02T13:36:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.934+0000 7fc1fb90a700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f41119e0 con 0x7fc1f41014f0 2024-04-02T13:36:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.934+0000 7fc1f9ea7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f4107500 0x7fc1f419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.934+0000 7fc1f9ea7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f4107500 0x7fc1f419d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50922/0 (socket says 172.21.15.33:50922) 2024-04-02T13:36:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.934+0000 7fc1f9ea7700 1 -- 172.21.15.33:0/738934517 learned_addr learned my addr 172.21.15.33:0/738934517 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:36:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.934+0000 7fc1f8ea5700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1f4101eb0 0x7fc1f41993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.934+0000 7fc1f96a6700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f41014f0 0x7fc1f4198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.935+0000 7fc1f9ea7700 1 -- 172.21.15.33:0/738934517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f41014f0 msgr2=0x7fc1f4198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.935+0000 7fc1f9ea7700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f41014f0 0x7fc1f4198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.935+0000 7fc1f9ea7700 1 -- 172.21.15.33:0/738934517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1f4101eb0 msgr2=0x7fc1f41993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.935+0000 7fc1f9ea7700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1f4101eb0 0x7fc1f41993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.935+0000 7fc1f9ea7700 1 -- 172.21.15.33:0/738934517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f419de70 con 0x7fc1f4107500 2024-04-02T13:36:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.935+0000 7fc1f9ea7700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f4107500 0x7fc1f419d730 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00b970 tx=0x7fc1ec00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.935+0000 7fc1ea7fc700 1 -- 172.21.15.33:0/738934517 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec019070 con 0x7fc1f4107500 2024-04-02T13:36:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.935+0000 7fc1fb90a700 1 -- 172.21.15.33:0/738934517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f419e160 con 0x7fc1f4107500 2024-04-02T13:36:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.935+0000 7fc1ea7fc700 1 -- 172.21.15.33:0/738934517 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc1ec00ce00 con 0x7fc1f4107500 2024-04-02T13:36:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.935+0000 7fc1ea7fc700 1 -- 172.21.15.33:0/738934517 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec007370 con 0x7fc1f4107500 2024-04-02T13:36:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.935+0000 7fc1fb90a700 1 -- 172.21.15.33:0/738934517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f41a59e0 con 0x7fc1f4107500 2024-04-02T13:36:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.936+0000 7fc1fb90a700 1 -- 172.21.15.33:0/738934517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f40692b0 con 0x7fc1f4107500 2024-04-02T13:36:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.937+0000 7fc1ea7fc700 1 -- 172.21.15.33:0/738934517 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc1ec01c070 con 0x7fc1f4107500 2024-04-02T13:36:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.938+0000 7fc1ea7fc700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc1e00a3cd0 0x7fc1e00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.938+0000 7fc1f96a6700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc1e00a3cd0 0x7fc1e00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.938+0000 7fc1ea7fc700 1 -- 172.21.15.33:0/738934517 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc1ec0cfea0 con 0x7fc1f4107500 2024-04-02T13:36:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.939+0000 7fc1f96a6700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc1e00a3cd0 0x7fc1e00a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0006fd0 tx=0x7fc1f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:58.942+0000 7fc1ea7fc700 1 -- 172.21.15.33:0/738934517 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc1ec09c5f0 con 0x7fc1f4107500 2024-04-02T13:36:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.150+0000 7fc1fb90a700 1 -- 172.21.15.33:0/738934517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1f41a5c70 con 0x7fc1f4107500 2024-04-02T13:36:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.151+0000 7fc1ea7fc700 1 -- 172.21.15.33:0/738934517 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24057 (secure 0 0 0) 0x7fc1ec09bef0 con 0x7fc1f4107500 2024-04-02T13:36:59.155 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:36:59.155 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:36:25.856437+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:36:59.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.153+0000 7fc1fb90a700 1 -- 172.21.15.33:0/738934517 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc1e00a3cd0 msgr2=0x7fc1e00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.153+0000 7fc1fb90a700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc1e00a3cd0 0x7fc1e00a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0006fd0 tx=0x7fc1f0008040 comp rx=0 tx=0).stop 2024-04-02T13:36:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.153+0000 7fc1fb90a700 1 -- 172.21.15.33:0/738934517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f4107500 msgr2=0x7fc1f419d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.153+0000 7fc1fb90a700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f4107500 0x7fc1f419d730 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00b970 tx=0x7fc1ec00bc80 comp rx=0 tx=0).stop 2024-04-02T13:36:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.154+0000 7fc1fb90a700 1 -- 172.21.15.33:0/738934517 shutdown_connections 2024-04-02T13:36:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.154+0000 7fc1fb90a700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc1e00a3cd0 0x7fc1e00a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.154+0000 7fc1fb90a700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1f4107500 0x7fc1f419d730 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.154+0000 7fc1fb90a700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f41014f0 0x7fc1f4198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.154+0000 7fc1fb90a700 1 --2- 172.21.15.33:0/738934517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1f4101eb0 0x7fc1f41993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.154+0000 7fc1fb90a700 1 -- 172.21.15.33:0/738934517 >> 172.21.15.33:0/738934517 conn(0x7fc1f4078420 msgr2=0x7fc1f41008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.154+0000 7fc1fb90a700 1 -- 172.21.15.33:0/738934517 shutdown_connections 2024-04-02T13:36:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:36:59.154+0000 7fc1fb90a700 1 -- 172.21.15.33:0/738934517 wait complete. 2024-04-02T13:36:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:59 smithi033 ceph-mon[32390]: pgmap v311: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-04-02T13:36:59.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:36:59 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/738934517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:59.943 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-02T13:36:59.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T13:36:59.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:59 smithi145 ceph-mon[35646]: pgmap v311: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-04-02T13:36:59.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:36:59 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/738934517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:36:59.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:59 smithi050 ceph-mon[35524]: pgmap v311: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-04-02T13:36:59.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:36:59 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/738934517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:37:00.190 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:01.749 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:01 smithi033 ceph-mon[32390]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-02T13:37:01.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:01 smithi145 ceph-mon[35646]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-02T13:37:01.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:01 smithi050 ceph-mon[35524]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-02T13:37:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.991+0000 7f4479b69700 1 -- 172.21.15.33:0/628359580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4474102640 msgr2=0x7f4474102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.991+0000 7f4479b69700 1 --2- 172.21.15.33:0/628359580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4474102640 0x7f4474102aa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4468009990 tx=0x7f4468009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.992+0000 7f4479b69700 1 -- 172.21.15.33:0/628359580 shutdown_connections 2024-04-02T13:37:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.992+0000 7f4479b69700 1 --2- 172.21.15.33:0/628359580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4474102fe0 0x7f447410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.992+0000 7f4479b69700 1 --2- 172.21.15.33:0/628359580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4474107520 0x7f4474107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.992+0000 7f4479b69700 1 --2- 172.21.15.33:0/628359580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4474102640 0x7f4474102aa0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.992+0000 7f4479b69700 1 -- 172.21.15.33:0/628359580 >> 172.21.15.33:0/628359580 conn(0x7f4474079720 msgr2=0x7f4474079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.992+0000 7f4479b69700 1 -- 172.21.15.33:0/628359580 shutdown_connections 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.992+0000 7f4479b69700 1 -- 172.21.15.33:0/628359580 wait complete. 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.992+0000 7f4479b69700 1 Processor -- start 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.992+0000 7f4479b69700 1 -- start start 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4479b69700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4474102640 0x7f4474198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4479b69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4474102fe0 0x7f4474199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4479b69700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4474107520 0x7f447419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4479b69700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44741129a0 con 0x7f4474102fe0 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4479b69700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4474112820 con 0x7f4474102640 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4479b69700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4474112b20 con 0x7f4474107520 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4472ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4474102fe0 0x7f4474199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4472ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4474102fe0 0x7f4474199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50938/0 (socket says 172.21.15.33:50938) 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4472ffd700 1 -- 172.21.15.33:0/2838460677 learned_addr learned my addr 172.21.15.33:0/2838460677 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f44737fe700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4474102640 0x7f4474198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4473fff700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4474107520 0x7f447419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4472ffd700 1 -- 172.21.15.33:0/2838460677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4474107520 msgr2=0x7f447419d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4472ffd700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4474107520 0x7f447419d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4472ffd700 1 -- 172.21.15.33:0/2838460677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4474102640 msgr2=0x7f4474198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4472ffd700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4474102640 0x7f4474198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4472ffd700 1 -- 172.21.15.33:0/2838460677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447419dee0 con 0x7f4474102fe0 2024-04-02T13:37:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4473fff700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4474107520 0x7f447419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4472ffd700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4474102fe0 0x7f4474199410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f446800bff0 tx=0x7f4468005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4470ff9700 1 -- 172.21.15.33:0/2838460677 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468027070 con 0x7f4474102fe0 2024-04-02T13:37:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4479b69700 1 -- 172.21.15.33:0/2838460677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447419e170 con 0x7f4474102fe0 2024-04-02T13:37:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4470ff9700 1 -- 172.21.15.33:0/2838460677 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f44680040c0 con 0x7f4474102fe0 2024-04-02T13:37:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.993+0000 7f4479b69700 1 -- 172.21.15.33:0/2838460677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44741a5a50 con 0x7f4474102fe0 2024-04-02T13:37:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.994+0000 7f4470ff9700 1 -- 172.21.15.33:0/2838460677 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468022e60 con 0x7f4474102fe0 2024-04-02T13:37:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.995+0000 7f4479b69700 1 -- 172.21.15.33:0/2838460677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447410ab30 con 0x7f4474102fe0 2024-04-02T13:37:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.996+0000 7f4470ff9700 1 -- 172.21.15.33:0/2838460677 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f446802a070 con 0x7f4474102fe0 2024-04-02T13:37:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.996+0000 7f4470ff9700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f445c0a3cd0 0x7f445c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.997+0000 7f44737fe700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f445c0a3cd0 0x7f445c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.997+0000 7f4470ff9700 1 -- 172.21.15.33:0/2838460677 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f44680d2150 con 0x7f4474102fe0 2024-04-02T13:37:02.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:01.997+0000 7f44737fe700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f445c0a3cd0 0x7f445c0a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f44741036d0 tx=0x7f4464009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:02.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.001+0000 7f4470ff9700 1 -- 172.21.15.33:0/2838460677 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4468031070 con 0x7f4474102fe0 2024-04-02T13:37:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.207+0000 7f4479b69700 1 -- 172.21.15.33:0/2838460677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f447419a230 con 0x7f4474102fe0 2024-04-02T13:37:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.209+0000 7f4470ff9700 1 -- 172.21.15.33:0/2838460677 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24057 (secure 0 0 0) 0x7f4468013070 con 0x7f4474102fe0 2024-04-02T13:37:02.212 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:37:02.212 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:36:25.856437+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:37:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.211+0000 7f4479b69700 1 -- 172.21.15.33:0/2838460677 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f445c0a3cd0 msgr2=0x7f445c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.211+0000 7f4479b69700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f445c0a3cd0 0x7f445c0a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f44741036d0 tx=0x7f4464009190 comp rx=0 tx=0).stop 2024-04-02T13:37:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.211+0000 7f4479b69700 1 -- 172.21.15.33:0/2838460677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4474102fe0 msgr2=0x7f4474199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.211+0000 7f4479b69700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4474102fe0 0x7f4474199410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f446800bff0 tx=0x7f4468005590 comp rx=0 tx=0).stop 2024-04-02T13:37:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.211+0000 7f4479b69700 1 -- 172.21.15.33:0/2838460677 shutdown_connections 2024-04-02T13:37:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.211+0000 7f4479b69700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f445c0a3cd0 0x7f445c0a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.211+0000 7f4479b69700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4474107520 0x7f447419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.211+0000 7f4479b69700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4474102640 0x7f4474198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.211+0000 7f4479b69700 1 --2- 172.21.15.33:0/2838460677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4474102fe0 0x7f4474199410 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.211+0000 7f4479b69700 1 -- 172.21.15.33:0/2838460677 >> 172.21.15.33:0/2838460677 conn(0x7f4474079720 msgr2=0x7f44741007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.211+0000 7f4479b69700 1 -- 172.21.15.33:0/2838460677 shutdown_connections 2024-04-02T13:37:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:02.211+0000 7f4479b69700 1 -- 172.21.15.33:0/2838460677 wait complete. 2024-04-02T13:37:02.877 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:02 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2838460677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:37:02.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.0 flush_pg_stats 2024-04-02T13:37:02.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.1 flush_pg_stats 2024-04-02T13:37:02.909 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.2 flush_pg_stats 2024-04-02T13:37:02.909 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.3 flush_pg_stats 2024-04-02T13:37:02.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.4 flush_pg_stats 2024-04-02T13:37:02.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.5 flush_pg_stats 2024-04-02T13:37:02.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.6 flush_pg_stats 2024-04-02T13:37:02.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.7 flush_pg_stats 2024-04-02T13:37:02.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.8 flush_pg_stats 2024-04-02T13:37:02.912 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.9 flush_pg_stats 2024-04-02T13:37:02.912 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.10 flush_pg_stats 2024-04-02T13:37:02.912 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.11 flush_pg_stats 2024-04-02T13:37:02.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:02 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2838460677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:37:02.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:02 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2838460677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:37:03.719 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:03.720 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:03.726 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:03.735 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:03.735 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:03.870 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:03 smithi033 ceph-mon[32390]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-02T13:37:03.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:03 smithi145 ceph-mon[35646]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-02T13:37:03.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:03 smithi050 ceph-mon[35524]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-02T13:37:04.822 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:04.822 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:04.823 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:04.823 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:05.886 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:05 smithi033 ceph-mon[32390]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-04-02T13:37:05.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:05 smithi145 ceph-mon[35646]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-04-02T13:37:05.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:05 smithi050 ceph-mon[35524]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-04-02T13:37:07.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:07 smithi145 ceph-mon[35646]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-02T13:37:07.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:07 smithi050 ceph-mon[35524]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-02T13:37:08.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:07 smithi033 ceph-mon[32390]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-02T13:37:09.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:09 smithi033 ceph-mon[32390]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-02T13:37:09.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:09 smithi145 ceph-mon[35646]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-02T13:37:09.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:09 smithi050 ceph-mon[35524]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-02T13:37:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.554+0000 7fbea23c9700 1 -- 172.21.15.33:0/4255051237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101190 msgr2=0x7fbe9c101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.554+0000 7fbea23c9700 1 --2- 172.21.15.33:0/4255051237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101190 0x7fbe9c101570 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009990 tx=0x7fbe8c009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.554+0000 7fbea23c9700 1 -- 172.21.15.33:0/4255051237 shutdown_connections 2024-04-02T13:37:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.554+0000 7fbea23c9700 1 --2- 172.21.15.33:0/4255051237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe9c106060 0x7fbe9c10dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.554+0000 7fbea23c9700 1 --2- 172.21.15.33:0/4255051237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101190 0x7fbe9c101570 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.554+0000 7fbea23c9700 1 --2- 172.21.15.33:0/4255051237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe9c101b40 0x7fbe9c105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.554+0000 7fbea23c9700 1 -- 172.21.15.33:0/4255051237 >> 172.21.15.33:0/4255051237 conn(0x7fbe9c076830 msgr2=0x7fbe9c076c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.554+0000 7fbea23c9700 1 -- 172.21.15.33:0/4255051237 shutdown_connections 2024-04-02T13:37:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.554+0000 7fbea23c9700 1 -- 172.21.15.33:0/4255051237 wait complete. 2024-04-02T13:37:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.555+0000 7fbea23c9700 1 Processor -- start 2024-04-02T13:37:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.555+0000 7fbea23c9700 1 -- start start 2024-04-02T13:37:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.555+0000 7fbea23c9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101190 0x7fbe9c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.555+0000 7fbea23c9700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe9c101b40 0x7fbe9c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.555+0000 7fbea23c9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe9c106060 0x7fbe9c1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.555+0000 7fbea23c9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c111840 con 0x7fbe9c101190 2024-04-02T13:37:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.555+0000 7fbea23c9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1116c0 con 0x7fbe9c106060 2024-04-02T13:37:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.555+0000 7fbea23c9700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1119c0 con 0x7fbe9c101b40 2024-04-02T13:37:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.555+0000 7fbe9bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101190 0x7fbe9c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbe9bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101190 0x7fbe9c199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34426/0 (socket says 172.21.15.33:34426) 2024-04-02T13:37:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbe9bfff700 1 -- 172.21.15.33:0/1862595044 learned_addr learned my addr 172.21.15.33:0/1862595044 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbea0966700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe9c106060 0x7fbe9c1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbe9b7fe700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe9c101b40 0x7fbe9c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbe9bfff700 1 -- 172.21.15.33:0/1862595044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe9c101b40 msgr2=0x7fbe9c19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbe9bfff700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe9c101b40 0x7fbe9c19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbe9bfff700 1 -- 172.21.15.33:0/1862595044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe9c106060 msgr2=0x7fbe9c1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbe9bfff700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe9c106060 0x7fbe9c1a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbe9bfff700 1 -- 172.21.15.33:0/1862595044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c1a5140 con 0x7fbe9c101190 2024-04-02T13:37:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbe9b7fe700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbe9c101b40 0x7fbe9c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbe9bfff700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101190 0x7fbe9c199b80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0057c0 tx=0x7fbe8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbe997fa700 1 -- 172.21.15.33:0/1862595044 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c026070 con 0x7fbe9c101190 2024-04-02T13:37:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbe997fa700 1 -- 172.21.15.33:0/1862595044 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbe8c00fb40 con 0x7fbe9c101190 2024-04-02T13:37:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbea23c9700 1 -- 172.21.15.33:0/1862595044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9c1a5310 con 0x7fbe9c101190 2024-04-02T13:37:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbe997fa700 1 -- 172.21.15.33:0/1862595044 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c017650 con 0x7fbe9c101190 2024-04-02T13:37:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.556+0000 7fbea23c9700 1 -- 172.21.15.33:0/1862595044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9c1a57c0 con 0x7fbe9c101190 2024-04-02T13:37:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.557+0000 7fbea23c9700 1 -- 172.21.15.33:0/1862595044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe9c062d80 con 0x7fbe9c101190 2024-04-02T13:37:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.558+0000 7fbe997fa700 1 -- 172.21.15.33:0/1862595044 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbe8c029070 con 0x7fbe9c101190 2024-04-02T13:37:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.559+0000 7fbe997fa700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbe840a3cd0 0x7fbe840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.559+0000 7fbe9b7fe700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbe840a3cd0 0x7fbe840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.559+0000 7fbe997fa700 1 -- 172.21.15.33:0/1862595044 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbe8c0d1580 con 0x7fbe9c101190 2024-04-02T13:37:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.559+0000 7fbe997fa700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676] conn(0x7fbe840ae990 0x7fbe840b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.559+0000 7fbe997fa700 1 -- 172.21.15.33:0/1862595044 --> [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe840b14a0 con 0x7fbe840ae990 2024-04-02T13:37:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.559+0000 7fbe997fa700 1 -- 172.21.15.33:0/1862595044 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fbe8c0d19d0 con 0x7fbe9c101190 2024-04-02T13:37:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.560+0000 7fbe9b7fe700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbe840a3cd0 0x7fbe840a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0ffa00 tx=0x7fbe90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.560+0000 7fbea0966700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676] conn(0x7fbe840ae990 0x7fbe840b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.560+0000 7fbea0966700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676] conn(0x7fbe840ae990 0x7fbe840b0dd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.561+0000 7fbe997fa700 1 -- 172.21.15.33:0/1862595044 <== osd.7 v2:172.21.15.50:6816/2798485676 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe840b14a0 con 0x7fbe840ae990 2024-04-02T13:37:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.577+0000 7fbea23c9700 1 -- 172.21.15.33:0/1862595044 --> [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe9c063180 con 0x7fbe840ae990 2024-04-02T13:37:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.578+0000 7fbe997fa700 1 -- 172.21.15.33:0/1862595044 <== osd.7 v2:172.21.15.50:6816/2798485676 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbe9c063180 con 0x7fbe840ae990 2024-04-02T13:37:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.578+0000 7fbea23c9700 1 -- 172.21.15.33:0/1862595044 >> [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676] conn(0x7fbe840ae990 msgr2=0x7fbe840b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.578+0000 7fbea23c9700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676] conn(0x7fbe840ae990 0x7fbe840b0dd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.578+0000 7fbea23c9700 1 -- 172.21.15.33:0/1862595044 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbe840a3cd0 msgr2=0x7fbe840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.578+0000 7fbea23c9700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbe840a3cd0 0x7fbe840a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0ffa00 tx=0x7fbe90009260 comp rx=0 tx=0).stop 2024-04-02T13:37:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.578+0000 7fbea23c9700 1 -- 172.21.15.33:0/1862595044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101190 msgr2=0x7fbe9c199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.578+0000 7fbea23c9700 1 --2- 172.21.15.33:0/1862595044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101190 0x7fbe9c199b80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0057c0 tx=0x7fbe8c0057f0 comp rx=0 tx=0).stop 2024-04-02T13:37:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.578+0000 7fbea0966700 1 -- 172.21.15.33:0/1862595044 reap_dead start 2024-04-02T13:37:10.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.578+0000 7fbea23c9700 1 -- 172.21.15.33:0/1862595044 shutdown_connections 2024-04-02T13:37:10.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.578+0000 7fbea23c9700 1 -- 172.21.15.33:0/1862595044 >> 172.21.15.33:0/1862595044 conn(0x7fbe9c076830 msgr2=0x7fbe9c107170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:10.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.579+0000 7fbea23c9700 1 -- 172.21.15.33:0/1862595044 shutdown_connections 2024-04-02T13:37:10.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:10.579+0000 7fbea23c9700 1 -- 172.21.15.33:0/1862595044 wait complete. 2024-04-02T13:37:11.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:11 smithi145 ceph-mon[35646]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-02T13:37:11.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:11 smithi050 ceph-mon[35524]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-02T13:37:12.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:11 smithi033 ceph-mon[32390]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-02T13:37:12.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.908+0000 7ff642f75700 1 -- 172.21.15.33:0/2349477318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c108660 msgr2=0x7ff63c108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:12.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.908+0000 7ff642f75700 1 --2- 172.21.15.33:0/2349477318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c108660 0x7ff63c108a40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff630009990 tx=0x7ff630009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:12.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.908+0000 7ff642f75700 1 -- 172.21.15.33:0/2349477318 shutdown_connections 2024-04-02T13:37:12.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.908+0000 7ff642f75700 1 --2- 172.21.15.33:0/2349477318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff63c102f90 0x7ff63c10f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:12.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.908+0000 7ff642f75700 1 --2- 172.21.15.33:0/2349477318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c108660 0x7ff63c108a40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:12.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.909+0000 7ff642f75700 1 --2- 172.21.15.33:0/2349477318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63c1025d0 0x7ff63c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:12.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.909+0000 7ff642f75700 1 -- 172.21.15.33:0/2349477318 >> 172.21.15.33:0/2349477318 conn(0x7ff63c0fe0a0 msgr2=0x7ff63c1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:12.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.909+0000 7ff642f75700 1 -- 172.21.15.33:0/2349477318 shutdown_connections 2024-04-02T13:37:12.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.909+0000 7ff642f75700 1 -- 172.21.15.33:0/2349477318 wait complete. 2024-04-02T13:37:12.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.909+0000 7ff642f75700 1 Processor -- start 2024-04-02T13:37:12.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.909+0000 7ff642f75700 1 -- start start 2024-04-02T13:37:12.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.909+0000 7ff642f75700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63c1025d0 0x7ff63c0748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:12.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.909+0000 7ff642f75700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff63c102f90 0x7ff63c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:12.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.909+0000 7ff642f75700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c108660 0x7ff63c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:12.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.909+0000 7ff642f75700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c1128e0 con 0x7ff63c108660 2024-04-02T13:37:12.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.909+0000 7ff642f75700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c112760 con 0x7ff63c102f90 2024-04-02T13:37:12.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.909+0000 7ff642f75700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c112a60 con 0x7ff63c1025d0 2024-04-02T13:37:12.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff641512700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c108660 0x7ff63c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:12.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff641512700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c108660 0x7ff63c075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34446/0 (socket says 172.21.15.33:34446) 2024-04-02T13:37:12.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff641512700 1 -- 172.21.15.33:0/1219932386 learned_addr learned my addr 172.21.15.33:0/1219932386 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:12.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff63bfff700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff63c102f90 0x7ff63c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:12.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff640d11700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63c1025d0 0x7ff63c0748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:12.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff641512700 1 -- 172.21.15.33:0/1219932386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63c1025d0 msgr2=0x7ff63c0748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:12.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff641512700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63c1025d0 0x7ff63c0748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:12.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff641512700 1 -- 172.21.15.33:0/1219932386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff63c102f90 msgr2=0x7ff63c074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:12.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff641512700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff63c102f90 0x7ff63c074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:12.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff641512700 1 -- 172.21.15.33:0/1219932386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c1a7210 con 0x7ff63c108660 2024-04-02T13:37:12.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff640d11700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff63c1025d0 0x7ff63c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:12.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff641512700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c108660 0x7ff63c075370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff634007d70 tx=0x7ff63400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:12.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff639ffb700 1 -- 172.21.15.33:0/1219932386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634016020 con 0x7ff63c108660 2024-04-02T13:37:12.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff639ffb700 1 -- 172.21.15.33:0/1219932386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff63400ad40 con 0x7ff63c108660 2024-04-02T13:37:12.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff642f75700 1 -- 172.21.15.33:0/1219932386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63c1a7500 con 0x7ff63c108660 2024-04-02T13:37:12.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff639ffb700 1 -- 172.21.15.33:0/1219932386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6340115e0 con 0x7ff63c108660 2024-04-02T13:37:12.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.910+0000 7ff642f75700 1 -- 172.21.15.33:0/1219932386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63c1a7a40 con 0x7ff63c108660 2024-04-02T13:37:12.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.911+0000 7ff642f75700 1 -- 172.21.15.33:0/1219932386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff63c062d00 con 0x7ff63c108660 2024-04-02T13:37:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.912+0000 7ff639ffb700 1 -- 172.21.15.33:0/1219932386 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff634019020 con 0x7ff63c108660 2024-04-02T13:37:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.913+0000 7ff639ffb700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff62c0a3d50 0x7ff62c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.913+0000 7ff640d11700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff62c0a3d50 0x7ff62c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.913+0000 7ff639ffb700 1 -- 172.21.15.33:0/1219932386 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff6340d0b40 con 0x7ff63c108660 2024-04-02T13:37:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.913+0000 7ff639ffb700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410] conn(0x7ff62c0aea10 0x7ff62c0b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.913+0000 7ff639ffb700 1 -- 172.21.15.33:0/1219932386 --> [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff62c0b1520 con 0x7ff62c0aea10 2024-04-02T13:37:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.913+0000 7ff639ffb700 1 -- 172.21.15.33:0/1219932386 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff6340d0f90 con 0x7ff63c108660 2024-04-02T13:37:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.913+0000 7ff640d11700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff62c0a3d50 0x7ff62c0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff630006010 tx=0x7ff63000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.913+0000 7ff63bfff700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410] conn(0x7ff62c0aea10 0x7ff62c0b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.914+0000 7ff63bfff700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410] conn(0x7ff62c0aea10 0x7ff62c0b0e50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:12.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.915+0000 7ff639ffb700 1 -- 172.21.15.33:0/1219932386 <== osd.10 v2:172.21.15.50:6824/1632195410 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff62c0b1520 con 0x7ff62c0aea10 2024-04-02T13:37:12.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.931+0000 7ff642f75700 1 -- 172.21.15.33:0/1219932386 --> [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff63c063180 con 0x7ff62c0aea10 2024-04-02T13:37:12.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.932+0000 7ff639ffb700 1 -- 172.21.15.33:0/1219932386 <== osd.10 v2:172.21.15.50:6824/1632195410 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff63c063180 con 0x7ff62c0aea10 2024-04-02T13:37:12.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.932+0000 7ff642f75700 1 -- 172.21.15.33:0/1219932386 >> [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410] conn(0x7ff62c0aea10 msgr2=0x7ff62c0b0e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:12.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.932+0000 7ff642f75700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410] conn(0x7ff62c0aea10 0x7ff62c0b0e50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:12.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.933+0000 7ff642f75700 1 -- 172.21.15.33:0/1219932386 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff62c0a3d50 msgr2=0x7ff62c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:12.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.933+0000 7ff642f75700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7ff62c0a3d50 0x7ff62c0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff630006010 tx=0x7ff63000b540 comp rx=0 tx=0).stop 2024-04-02T13:37:12.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.933+0000 7ff642f75700 1 -- 172.21.15.33:0/1219932386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c108660 msgr2=0x7ff63c075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:12.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.933+0000 7ff642f75700 1 --2- 172.21.15.33:0/1219932386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c108660 0x7ff63c075370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff634007d70 tx=0x7ff63400a340 comp rx=0 tx=0).stop 2024-04-02T13:37:12.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.933+0000 7ff641512700 1 -- 172.21.15.33:0/1219932386 reap_dead start 2024-04-02T13:37:12.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.933+0000 7ff642f75700 1 -- 172.21.15.33:0/1219932386 shutdown_connections 2024-04-02T13:37:12.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.933+0000 7ff642f75700 1 -- 172.21.15.33:0/1219932386 >> 172.21.15.33:0/1219932386 conn(0x7ff63c0fe0a0 msgr2=0x7ff63c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:12.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.933+0000 7ff642f75700 1 -- 172.21.15.33:0/1219932386 shutdown_connections 2024-04-02T13:37:12.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:12.933+0000 7ff642f75700 1 -- 172.21.15.33:0/1219932386 wait complete. 2024-04-02T13:37:13.385 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:13.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:13 smithi033 ceph-mon[32390]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-02T13:37:13.844 INFO:teuthology.orchestra.run.smithi033.stdout:223338299436 2024-04-02T13:37:13.845 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.7 2024-04-02T13:37:13.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:13 smithi145 ceph-mon[35646]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-02T13:37:13.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:13 smithi050 ceph-mon[35524]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-02T13:37:15.875 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:15 smithi033 ceph-mon[32390]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-02T13:37:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.879+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/3826719739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5dc06acd0 msgr2=0x7fe5dc06b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.879+0000 7fe5e3eb9700 1 --2- 172.21.15.33:0/3826719739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5dc06acd0 0x7fe5dc06b150 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00b0d0 tx=0x7fe5cc00b3e0 comp rx=0 tx=0).stop 2024-04-02T13:37:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.879+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/3826719739 shutdown_connections 2024-04-02T13:37:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.879+0000 7fe5e3eb9700 1 --2- 172.21.15.33:0/3826719739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5dc06b690 0x7fe5dc10f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.879+0000 7fe5e3eb9700 1 --2- 172.21.15.33:0/3826719739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5dc108650 0x7fe5dc108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.879+0000 7fe5e3eb9700 1 --2- 172.21.15.33:0/3826719739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5dc06acd0 0x7fe5dc06b150 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.879+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/3826719739 >> 172.21.15.33:0/3826719739 conn(0x7fe5dc077510 msgr2=0x7fe5dc077920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:15.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.879+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/3826719739 shutdown_connections 2024-04-02T13:37:15.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.879+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/3826719739 wait complete. 2024-04-02T13:37:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.880+0000 7fe5e3eb9700 1 Processor -- start 2024-04-02T13:37:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.880+0000 7fe5e3eb9700 1 -- start start 2024-04-02T13:37:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.880+0000 7fe5e3eb9700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5dc06acd0 0x7fe5dc198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.880+0000 7fe5e3eb9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5dc06b690 0x7fe5dc199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.880+0000 7fe5e3eb9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5dc108650 0x7fe5dc19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.880+0000 7fe5e3eb9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc112950 con 0x7fe5dc108650 2024-04-02T13:37:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.880+0000 7fe5e3eb9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1127d0 con 0x7fe5dc06b690 2024-04-02T13:37:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.880+0000 7fe5e3eb9700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc112ad0 con 0x7fe5dc06acd0 2024-04-02T13:37:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.880+0000 7fe5e2456700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5dc108650 0x7fe5dc19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.880+0000 7fe5e2456700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5dc108650 0x7fe5dc19d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49274/0 (socket says 172.21.15.33:49274) 2024-04-02T13:37:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.880+0000 7fe5e2456700 1 -- 172.21.15.33:0/2823441003 learned_addr learned my addr 172.21.15.33:0/2823441003 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.880+0000 7fe5e1454700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5dc06b690 0x7fe5dc199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.881+0000 7fe5e1c55700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5dc06acd0 0x7fe5dc198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.881+0000 7fe5e2456700 1 -- 172.21.15.33:0/2823441003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5dc06acd0 msgr2=0x7fe5dc198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.881+0000 7fe5e2456700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5dc06acd0 0x7fe5dc198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.881+0000 7fe5e2456700 1 -- 172.21.15.33:0/2823441003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5dc06b690 msgr2=0x7fe5dc199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.881+0000 7fe5e2456700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5dc06b690 0x7fe5dc199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.881+0000 7fe5e2456700 1 -- 172.21.15.33:0/2823441003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc19ddf0 con 0x7fe5dc108650 2024-04-02T13:37:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.881+0000 7fe5e2456700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5dc108650 0x7fe5dc19d6b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000b550 tx=0x7fe5d000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.881+0000 7fe5daffd700 1 -- 172.21.15.33:0/2823441003 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0015020 con 0x7fe5dc108650 2024-04-02T13:37:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.881+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/2823441003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc19e0e0 con 0x7fe5dc108650 2024-04-02T13:37:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.881+0000 7fe5daffd700 1 -- 172.21.15.33:0/2823441003 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe5d0011d00 con 0x7fe5dc108650 2024-04-02T13:37:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.881+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/2823441003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc1a5960 con 0x7fe5dc108650 2024-04-02T13:37:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.881+0000 7fe5daffd700 1 -- 172.21.15.33:0/2823441003 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0010d60 con 0x7fe5dc108650 2024-04-02T13:37:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.882+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/2823441003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe5c8001030 con 0x7fe5dc108650 2024-04-02T13:37:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.884+0000 7fe5daffd700 1 -- 172.21.15.33:0/2823441003 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe5d0018020 con 0x7fe5dc108650 2024-04-02T13:37:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.885+0000 7fe5daffd700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe5c40a3d50 0x7fe5c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.885+0000 7fe5e1c55700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe5c40a3d50 0x7fe5c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.885+0000 7fe5daffd700 1 -- 172.21.15.33:0/2823441003 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe5d00d02d0 con 0x7fe5dc108650 2024-04-02T13:37:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.885+0000 7fe5daffd700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265] conn(0x7fe5c40aea10 0x7fe5c40b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.885+0000 7fe5daffd700 1 -- 172.21.15.33:0/2823441003 --> [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5c40b1520 con 0x7fe5c40aea10 2024-04-02T13:37:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.885+0000 7fe5daffd700 1 -- 172.21.15.33:0/2823441003 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fe5d00d0720 con 0x7fe5dc108650 2024-04-02T13:37:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.885+0000 7fe5e1c55700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe5c40a3d50 0x7fe5c40a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009a10 tx=0x7fe5d4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.885+0000 7fe5e1454700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265] conn(0x7fe5c40aea10 0x7fe5c40b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:15.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.886+0000 7fe5e1454700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265] conn(0x7fe5c40aea10 0x7fe5c40b0e50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:15.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.887+0000 7fe5daffd700 1 -- 172.21.15.33:0/2823441003 <== osd.2 v2:172.21.15.145:6800/1898887265 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe5c40b1520 con 0x7fe5c40aea10 2024-04-02T13:37:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.903+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/2823441003 --> [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe5c8002de0 con 0x7fe5c40aea10 2024-04-02T13:37:15.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.904+0000 7fe5daffd700 1 -- 172.21.15.33:0/2823441003 <== osd.2 v2:172.21.15.145:6800/1898887265 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe5c8002de0 con 0x7fe5c40aea10 2024-04-02T13:37:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.904+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/2823441003 >> [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265] conn(0x7fe5c40aea10 msgr2=0x7fe5c40b0e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.904+0000 7fe5e3eb9700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265] conn(0x7fe5c40aea10 0x7fe5c40b0e50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.904+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/2823441003 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe5c40a3d50 msgr2=0x7fe5c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.904+0000 7fe5e3eb9700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fe5c40a3d50 0x7fe5c40a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009a10 tx=0x7fe5d4009190 comp rx=0 tx=0).stop 2024-04-02T13:37:15.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.904+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/2823441003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5dc108650 msgr2=0x7fe5dc19d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.904+0000 7fe5e3eb9700 1 --2- 172.21.15.33:0/2823441003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5dc108650 0x7fe5dc19d6b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000b550 tx=0x7fe5d000b860 comp rx=0 tx=0).stop 2024-04-02T13:37:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.904+0000 7fe5e2456700 1 -- 172.21.15.33:0/2823441003 reap_dead start 2024-04-02T13:37:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.904+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/2823441003 shutdown_connections 2024-04-02T13:37:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.904+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/2823441003 >> 172.21.15.33:0/2823441003 conn(0x7fe5dc077510 msgr2=0x7fe5dc100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.904+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/2823441003 shutdown_connections 2024-04-02T13:37:15.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:15.904+0000 7fe5e3eb9700 1 -- 172.21.15.33:0/2823441003 wait complete. 2024-04-02T13:37:15.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:15 smithi145 ceph-mon[35646]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-02T13:37:15.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:15 smithi050 ceph-mon[35524]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-02T13:37:16.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-02T13:37:16.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-02T13:37:16.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-02T13:37:16.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-02T13:37:16.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-04-02T13:37:16.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-02T13:37:16.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-02T13:37:16.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-04-02T13:37:16.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-02T13:37:16.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:16 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-02T13:37:16.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-02T13:37:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-02T13:37:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-02T13:37:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-02T13:37:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-04-02T13:37:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-02T13:37:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-02T13:37:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-04-02T13:37:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-02T13:37:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:16 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-02T13:37:16.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-02T13:37:16.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-02T13:37:16.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-02T13:37:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-02T13:37:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-04-02T13:37:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-02T13:37:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-02T13:37:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-04-02T13:37:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-02T13:37:16.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:16 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-02T13:37:17.771 INFO:teuthology.orchestra.run.smithi033.stdout:77309411398 2024-04-02T13:37:17.771 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.2 2024-04-02T13:37:17.891 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:17 smithi033 ceph-mon[32390]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-02T13:37:17.891 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-02T13:37:17.891 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-02T13:37:17.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-02T13:37:17.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-02T13:37:17.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-04-02T13:37:17.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-02T13:37:17.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-02T13:37:17.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-04-02T13:37:17.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-02T13:37:17.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-02T13:37:17.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:17 smithi033 ceph-mon[32390]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T13:37:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:17 smithi145 ceph-mon[35646]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-02T13:37:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-02T13:37:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-02T13:37:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-02T13:37:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-02T13:37:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-04-02T13:37:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-02T13:37:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-02T13:37:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-04-02T13:37:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-02T13:37:17.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-02T13:37:17.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:17 smithi145 ceph-mon[35646]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T13:37:17.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:17 smithi050 ceph-mon[35524]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-02T13:37:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-02T13:37:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-02T13:37:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-02T13:37:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-02T13:37:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-04-02T13:37:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-02T13:37:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-02T13:37:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-04-02T13:37:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-02T13:37:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-02T13:37:17.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:17 smithi050 ceph-mon[35524]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T13:37:18.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.055+0000 7f7032c35700 1 -- 172.21.15.33:0/1416840473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f702c108660 msgr2=0x7f702c108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:18.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.055+0000 7f7032c35700 1 --2- 172.21.15.33:0/1416840473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f702c108660 0x7f702c108a40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f701c009990 tx=0x7f701c009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.055+0000 7f7032c35700 1 -- 172.21.15.33:0/1416840473 shutdown_connections 2024-04-02T13:37:18.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.055+0000 7f7032c35700 1 --2- 172.21.15.33:0/1416840473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f702c102f90 0x7f702c10f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.055+0000 7f7032c35700 1 --2- 172.21.15.33:0/1416840473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f702c108660 0x7f702c108a40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.055+0000 7f7032c35700 1 --2- 172.21.15.33:0/1416840473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f702c1025d0 0x7f702c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.055+0000 7f7032c35700 1 -- 172.21.15.33:0/1416840473 >> 172.21.15.33:0/1416840473 conn(0x7f702c0fe0a0 msgr2=0x7f702c1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:18.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.055+0000 7f7032c35700 1 -- 172.21.15.33:0/1416840473 shutdown_connections 2024-04-02T13:37:18.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.055+0000 7f7032c35700 1 -- 172.21.15.33:0/1416840473 wait complete. 2024-04-02T13:37:18.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f7032c35700 1 Processor -- start 2024-04-02T13:37:18.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f7032c35700 1 -- start start 2024-04-02T13:37:18.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f7032c35700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f702c1025d0 0x7f702c0748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:18.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f7032c35700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f702c102f90 0x7f702c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f7032c35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f702c108660 0x7f702c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f7032c35700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1128e0 con 0x7f702c108660 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f7032c35700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f702c112760 con 0x7f702c102f90 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f7032c35700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f702c112a60 con 0x7f702c1025d0 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f70311d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f702c108660 0x7f702c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f70311d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f702c108660 0x7f702c075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49286/0 (socket says 172.21.15.33:49286) 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f70311d2700 1 -- 172.21.15.33:0/1264690984 learned_addr learned my addr 172.21.15.33:0/1264690984 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f70309d1700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f702c1025d0 0x7f702c0748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f702bfff700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f702c102f90 0x7f702c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f70311d2700 1 -- 172.21.15.33:0/1264690984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f702c1025d0 msgr2=0x7f702c0748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f70311d2700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f702c1025d0 0x7f702c0748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f70311d2700 1 -- 172.21.15.33:0/1264690984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f702c102f90 msgr2=0x7f702c074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f70311d2700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f702c102f90 0x7f702c074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.056+0000 7f70311d2700 1 -- 172.21.15.33:0/1264690984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c1a7220 con 0x7f702c108660 2024-04-02T13:37:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.057+0000 7f70309d1700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f702c1025d0 0x7f702c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.057+0000 7f702bfff700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f702c102f90 0x7f702c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.057+0000 7f70311d2700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f702c108660 0x7f702c075370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7024007d70 tx=0x7f702400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.057+0000 7f7029ffb700 1 -- 172.21.15.33:0/1264690984 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024016020 con 0x7f702c108660 2024-04-02T13:37:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.057+0000 7f7029ffb700 1 -- 172.21.15.33:0/1264690984 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f702400ad40 con 0x7f702c108660 2024-04-02T13:37:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.057+0000 7f7032c35700 1 -- 172.21.15.33:0/1264690984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702c1a7510 con 0x7f702c108660 2024-04-02T13:37:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.057+0000 7f7029ffb700 1 -- 172.21.15.33:0/1264690984 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024011680 con 0x7f702c108660 2024-04-02T13:37:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.057+0000 7f7032c35700 1 -- 172.21.15.33:0/1264690984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702c1a7a00 con 0x7f702c108660 2024-04-02T13:37:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.058+0000 7f7032c35700 1 -- 172.21.15.33:0/1264690984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f702c100020 con 0x7f702c108660 2024-04-02T13:37:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.059+0000 7f7029ffb700 1 -- 172.21.15.33:0/1264690984 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7024019020 con 0x7f702c108660 2024-04-02T13:37:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.060+0000 7f7029ffb700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f70180a3cd0 0x7f70180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.060+0000 7f70309d1700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f70180a3cd0 0x7f70180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.060+0000 7f7029ffb700 1 -- 172.21.15.33:0/1264690984 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f70240d0e50 con 0x7f702c108660 2024-04-02T13:37:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.060+0000 7f7029ffb700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141] conn(0x7f70180aee30 0x7f70180b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.060+0000 7f7029ffb700 1 -- 172.21.15.33:0/1264690984 --> [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70180b19a0 con 0x7f70180aee30 2024-04-02T13:37:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.060+0000 7f7029ffb700 1 -- 172.21.15.33:0/1264690984 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f70240d1200 con 0x7f702c108660 2024-04-02T13:37:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.060+0000 7f702bfff700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141] conn(0x7f70180aee30 0x7f70180b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.060+0000 7f70309d1700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f70180a3cd0 0x7f70180a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f701c006010 tx=0x7f701c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.060+0000 7f702bfff700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141] conn(0x7f70180aee30 0x7f70180b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.061+0000 7f7029ffb700 1 -- 172.21.15.33:0/1264690984 <== osd.3 v2:172.21.15.33:6810/1162388141 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f70180b19a0 con 0x7f70180aee30 2024-04-02T13:37:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.079+0000 7f7032c35700 1 -- 172.21.15.33:0/1264690984 --> [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f702c103d10 con 0x7f70180aee30 2024-04-02T13:37:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.080+0000 7f7029ffb700 1 -- 172.21.15.33:0/1264690984 <== osd.3 v2:172.21.15.33:6810/1162388141 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f702c103d10 con 0x7f70180aee30 2024-04-02T13:37:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.080+0000 7f7032c35700 1 -- 172.21.15.33:0/1264690984 >> [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141] conn(0x7f70180aee30 msgr2=0x7f70180b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.080+0000 7f7032c35700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141] conn(0x7f70180aee30 0x7f70180b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.080+0000 7f7032c35700 1 -- 172.21.15.33:0/1264690984 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f70180a3cd0 msgr2=0x7f70180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.080+0000 7f7032c35700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f70180a3cd0 0x7f70180a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f701c006010 tx=0x7f701c00b540 comp rx=0 tx=0).stop 2024-04-02T13:37:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.080+0000 7f7032c35700 1 -- 172.21.15.33:0/1264690984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f702c108660 msgr2=0x7f702c075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.080+0000 7f7032c35700 1 --2- 172.21.15.33:0/1264690984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f702c108660 0x7f702c075370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7024007d70 tx=0x7f702400a340 comp rx=0 tx=0).stop 2024-04-02T13:37:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.081+0000 7f70311d2700 1 -- 172.21.15.33:0/1264690984 reap_dead start 2024-04-02T13:37:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.081+0000 7f7032c35700 1 -- 172.21.15.33:0/1264690984 shutdown_connections 2024-04-02T13:37:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.081+0000 7f7032c35700 1 -- 172.21.15.33:0/1264690984 >> 172.21.15.33:0/1264690984 conn(0x7f702c0fe0a0 msgr2=0x7f702c0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.081+0000 7f7032c35700 1 -- 172.21.15.33:0/1264690984 shutdown_connections 2024-04-02T13:37:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.081+0000 7f7032c35700 1 -- 172.21.15.33:0/1264690984 wait complete. 2024-04-02T13:37:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.320+0000 7f1b56242700 1 -- 172.21.15.33:0/102228781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b501075f0 msgr2=0x7f1b501079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.320+0000 7f1b56242700 1 --2- 172.21.15.33:0/102228781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b501075f0 0x7f1b501079d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009990 tx=0x7f1b48009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.320+0000 7f1b56242700 1 -- 172.21.15.33:0/102228781 shutdown_connections 2024-04-02T13:37:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.320+0000 7f1b56242700 1 --2- 172.21.15.33:0/102228781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b50101ea0 0x7f1b5010e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.320+0000 7f1b56242700 1 --2- 172.21.15.33:0/102228781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b501075f0 0x7f1b501079d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.320+0000 7f1b56242700 1 --2- 172.21.15.33:0/102228781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b501014e0 0x7f1b50101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.320+0000 7f1b56242700 1 -- 172.21.15.33:0/102228781 >> 172.21.15.33:0/102228781 conn(0x7f1b50078460 msgr2=0x7f1b50078870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.320+0000 7f1b56242700 1 -- 172.21.15.33:0/102228781 shutdown_connections 2024-04-02T13:37:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.320+0000 7f1b56242700 1 -- 172.21.15.33:0/102228781 wait complete. 2024-04-02T13:37:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b56242700 1 Processor -- start 2024-04-02T13:37:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b56242700 1 -- start start 2024-04-02T13:37:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b56242700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b501014e0 0x7f1b50199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b56242700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b50101ea0 0x7f1b5019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b56242700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b501075f0 0x7f1b501a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b56242700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501117a0 con 0x7f1b501014e0 2024-04-02T13:37:18.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b56242700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50111620 con 0x7f1b50101ea0 2024-04-02T13:37:18.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b56242700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50111920 con 0x7f1b501075f0 2024-04-02T13:37:18.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b4f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b501014e0 0x7f1b50199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:18.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b4f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b501014e0 0x7f1b50199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49298/0 (socket says 172.21.15.33:49298) 2024-04-02T13:37:18.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b4f7fe700 1 -- 172.21.15.33:0/369043316 learned_addr learned my addr 172.21.15.33:0/369043316 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:18.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b4ffff700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b501075f0 0x7f1b501a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:18.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b4effd700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b50101ea0 0x7f1b5019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:18.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b4f7fe700 1 -- 172.21.15.33:0/369043316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b501075f0 msgr2=0x7f1b501a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:18.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b4f7fe700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b501075f0 0x7f1b501a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b4f7fe700 1 -- 172.21.15.33:0/369043316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b50101ea0 msgr2=0x7f1b5019a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:18.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b4f7fe700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b50101ea0 0x7f1b5019a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.321+0000 7f1b4f7fe700 1 -- 172.21.15.33:0/369043316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b501a5160 con 0x7f1b501014e0 2024-04-02T13:37:18.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.322+0000 7f1b4f7fe700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b501014e0 0x7f1b50199ba0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1b480038b0 tx=0x7f1b48009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:18.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.322+0000 7f1b4cff9700 1 -- 172.21.15.33:0/369043316 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48027070 con 0x7f1b501014e0 2024-04-02T13:37:18.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.322+0000 7f1b4cff9700 1 -- 172.21.15.33:0/369043316 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1b48004040 con 0x7f1b501014e0 2024-04-02T13:37:18.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.322+0000 7f1b56242700 1 -- 172.21.15.33:0/369043316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b501a5330 con 0x7f1b501014e0 2024-04-02T13:37:18.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.322+0000 7f1b4cff9700 1 -- 172.21.15.33:0/369043316 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48022df0 con 0x7f1b501014e0 2024-04-02T13:37:18.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.322+0000 7f1b56242700 1 -- 172.21.15.33:0/369043316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b501a5780 con 0x7f1b501014e0 2024-04-02T13:37:18.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.323+0000 7f1b56242700 1 -- 172.21.15.33:0/369043316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b50062d80 con 0x7f1b501014e0 2024-04-02T13:37:18.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.324+0000 7f1b4cff9700 1 -- 172.21.15.33:0/369043316 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1b4802a070 con 0x7f1b501014e0 2024-04-02T13:37:18.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.324+0000 7f1b4cff9700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b3c0a3cd0 0x7f1b3c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:18.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.325+0000 7f1b4effd700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b3c0a3cd0 0x7f1b3c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:18.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.325+0000 7f1b4cff9700 1 -- 172.21.15.33:0/369043316 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1b480d19d0 con 0x7f1b501014e0 2024-04-02T13:37:18.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.325+0000 7f1b4cff9700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754] conn(0x7f1b3c0aee30 0x7f1b3c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:18.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.325+0000 7f1b4cff9700 1 -- 172.21.15.33:0/369043316 --> [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b3c0b19a0 con 0x7f1b3c0aee30 2024-04-02T13:37:18.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.325+0000 7f1b4cff9700 1 -- 172.21.15.33:0/369043316 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f1b480d1d80 con 0x7f1b501014e0 2024-04-02T13:37:18.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.325+0000 7f1b4effd700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b3c0a3cd0 0x7f1b3c0a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1b38005950 tx=0x7f1b380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:18.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.325+0000 7f1b4ffff700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754] conn(0x7f1b3c0aee30 0x7f1b3c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:18.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.325+0000 7f1b4ffff700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754] conn(0x7f1b3c0aee30 0x7f1b3c0b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:18.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.326+0000 7f1b4cff9700 1 -- 172.21.15.33:0/369043316 <== osd.4 v2:172.21.15.50:6808/367423754 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b3c0b19a0 con 0x7f1b3c0aee30 2024-04-02T13:37:18.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.343+0000 7f1b56242700 1 -- 172.21.15.33:0/369043316 --> [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b50063180 con 0x7f1b3c0aee30 2024-04-02T13:37:18.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.344+0000 7f1b4cff9700 1 -- 172.21.15.33:0/369043316 <== osd.4 v2:172.21.15.50:6808/367423754 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1b50063180 con 0x7f1b3c0aee30 2024-04-02T13:37:18.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.344+0000 7f1b56242700 1 -- 172.21.15.33:0/369043316 >> [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754] conn(0x7f1b3c0aee30 msgr2=0x7f1b3c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:18.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.344+0000 7f1b56242700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754] conn(0x7f1b3c0aee30 0x7f1b3c0b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.344+0000 7f1b56242700 1 -- 172.21.15.33:0/369043316 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b3c0a3cd0 msgr2=0x7f1b3c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:18.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.344+0000 7f1b56242700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1b3c0a3cd0 0x7f1b3c0a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1b38005950 tx=0x7f1b380058e0 comp rx=0 tx=0).stop 2024-04-02T13:37:18.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.344+0000 7f1b56242700 1 -- 172.21.15.33:0/369043316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b501014e0 msgr2=0x7f1b50199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:18.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.344+0000 7f1b56242700 1 --2- 172.21.15.33:0/369043316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b501014e0 0x7f1b50199ba0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1b480038b0 tx=0x7f1b48009f90 comp rx=0 tx=0).stop 2024-04-02T13:37:18.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.344+0000 7f1b4ffff700 1 -- 172.21.15.33:0/369043316 reap_dead start 2024-04-02T13:37:18.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.344+0000 7f1b56242700 1 -- 172.21.15.33:0/369043316 shutdown_connections 2024-04-02T13:37:18.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.344+0000 7f1b56242700 1 -- 172.21.15.33:0/369043316 >> 172.21.15.33:0/369043316 conn(0x7f1b50078460 msgr2=0x7f1b50100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:18.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.344+0000 7f1b56242700 1 -- 172.21.15.33:0/369043316 shutdown_connections 2024-04-02T13:37:18.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:18.344+0000 7f1b56242700 1 -- 172.21.15.33:0/369043316 wait complete. 2024-04-02T13:37:18.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:18 smithi033 ceph-mon[32390]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T13:37:18.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:18 smithi145 ceph-mon[35646]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T13:37:18.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:18 smithi050 ceph-mon[35524]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T13:37:19.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.542+0000 7f6b75193700 1 -- 172.21.15.33:0/1621559051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7006acc0 msgr2=0x7f6b7006b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:19.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.542+0000 7f6b75193700 1 --2- 172.21.15.33:0/1621559051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7006acc0 0x7f6b7006b140 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6b580099c0 tx=0x7f6b58009cd0 comp rx=0 tx=0).stop 2024-04-02T13:37:19.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.542+0000 7f6b75193700 1 -- 172.21.15.33:0/1621559051 shutdown_connections 2024-04-02T13:37:19.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.542+0000 7f6b75193700 1 --2- 172.21.15.33:0/1621559051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b7006b680 0x7f6b7010f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:19.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.542+0000 7f6b75193700 1 --2- 172.21.15.33:0/1621559051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b70108620 0x7f6b70108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:19.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.542+0000 7f6b75193700 1 --2- 172.21.15.33:0/1621559051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b7006acc0 0x7f6b7006b140 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:19.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.542+0000 7f6b75193700 1 -- 172.21.15.33:0/1621559051 >> 172.21.15.33:0/1621559051 conn(0x7f6b70078460 msgr2=0x7f6b70078870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:19.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.542+0000 7f6b75193700 1 -- 172.21.15.33:0/1621559051 shutdown_connections 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.542+0000 7f6b75193700 1 -- 172.21.15.33:0/1621559051 wait complete. 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.543+0000 7f6b75193700 1 Processor -- start 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.543+0000 7f6b75193700 1 -- start start 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.543+0000 7f6b75193700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b7006acc0 0x7f6b701015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.543+0000 7f6b75193700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b7006b680 0x7f6b70101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.543+0000 7f6b75193700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b70108620 0x7f6b70102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.543+0000 7f6b75193700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b701128c0 con 0x7f6b70108620 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.543+0000 7f6b75193700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6b70112740 con 0x7f6b7006acc0 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.543+0000 7f6b75193700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6b70112a40 con 0x7f6b7006b680 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.543+0000 7f6b6f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b70108620 0x7f6b70102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.543+0000 7f6b6f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b70108620 0x7f6b70102030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49322/0 (socket says 172.21.15.33:49322) 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.543+0000 7f6b6f59e700 1 -- 172.21.15.33:0/370651730 learned_addr learned my addr 172.21.15.33:0/370651730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.543+0000 7f6b6ed9d700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b7006acc0 0x7f6b701015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:19.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.543+0000 7f6b6e59c700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b7006b680 0x7f6b70101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.544+0000 7f6b6f59e700 1 -- 172.21.15.33:0/370651730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b7006b680 msgr2=0x7f6b70101af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.544+0000 7f6b6f59e700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b7006b680 0x7f6b70101af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.544+0000 7f6b6f59e700 1 -- 172.21.15.33:0/370651730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b7006acc0 msgr2=0x7f6b701015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.544+0000 7f6b6f59e700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b7006acc0 0x7f6b701015b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.544+0000 7f6b6f59e700 1 -- 172.21.15.33:0/370651730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b701a7320 con 0x7f6b70108620 2024-04-02T13:37:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.544+0000 7f6b6e59c700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b7006b680 0x7f6b70101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.544+0000 7f6b6f59e700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b70108620 0x7f6b70102030 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800b550 tx=0x7f6b6800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.544+0000 7f6b67fff700 1 -- 172.21.15.33:0/370651730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68015020 con 0x7f6b70108620 2024-04-02T13:37:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.544+0000 7f6b67fff700 1 -- 172.21.15.33:0/370651730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6b68011d00 con 0x7f6b70108620 2024-04-02T13:37:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.544+0000 7f6b75193700 1 -- 172.21.15.33:0/370651730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b701a7610 con 0x7f6b70108620 2024-04-02T13:37:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.544+0000 7f6b67fff700 1 -- 172.21.15.33:0/370651730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68010e60 con 0x7f6b70108620 2024-04-02T13:37:19.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.544+0000 7f6b75193700 1 -- 172.21.15.33:0/370651730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b701a7b50 con 0x7f6b70108620 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.545+0000 7f6b75193700 1 -- 172.21.15.33:0/370651730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b70062d00 con 0x7f6b70108620 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.546+0000 7f6b67fff700 1 -- 172.21.15.33:0/370651730 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6b68018020 con 0x7f6b70108620 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.547+0000 7f6b67fff700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6b5c0a3cd0 0x7f6b5c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.547+0000 7f6b6ed9d700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6b5c0a3cd0 0x7f6b5c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.547+0000 7f6b67fff700 1 -- 172.21.15.33:0/370651730 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6b680d0660 con 0x7f6b70108620 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.547+0000 7f6b67fff700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637] conn(0x7f6b5c0aee30 0x7f6b5c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.547+0000 7f6b67fff700 1 -- 172.21.15.33:0/370651730 --> [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b68018ec0 con 0x7f6b5c0aee30 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.547+0000 7f6b67fff700 1 -- 172.21.15.33:0/370651730 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f6b680d0a10 con 0x7f6b70108620 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.547+0000 7f6b6e59c700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637] conn(0x7f6b5c0aee30 0x7f6b5c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.547+0000 7f6b6ed9d700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6b5c0a3cd0 0x7f6b5c0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b600060b0 tx=0x7f6b6000c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.547+0000 7f6b6e59c700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637] conn(0x7f6b5c0aee30 0x7f6b5c0b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.548+0000 7f6b67fff700 1 -- 172.21.15.33:0/370651730 <== osd.6 v2:172.21.15.33:6818/529207637 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b68018ec0 con 0x7f6b5c0aee30 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.565+0000 7f6b75193700 1 -- 172.21.15.33:0/370651730 --> [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6b70063180 con 0x7f6b5c0aee30 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.565+0000 7f6b67fff700 1 -- 172.21.15.33:0/370651730 <== osd.6 v2:172.21.15.33:6818/529207637 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6b70063180 con 0x7f6b5c0aee30 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.565+0000 7f6b75193700 1 -- 172.21.15.33:0/370651730 >> [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637] conn(0x7f6b5c0aee30 msgr2=0x7f6b5c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:19.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.565+0000 7f6b75193700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637] conn(0x7f6b5c0aee30 0x7f6b5c0b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:19.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.565+0000 7f6b75193700 1 -- 172.21.15.33:0/370651730 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6b5c0a3cd0 msgr2=0x7f6b5c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:19.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.565+0000 7f6b75193700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6b5c0a3cd0 0x7f6b5c0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b600060b0 tx=0x7f6b6000c040 comp rx=0 tx=0).stop 2024-04-02T13:37:19.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.565+0000 7f6b75193700 1 -- 172.21.15.33:0/370651730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b70108620 msgr2=0x7f6b70102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:19.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.565+0000 7f6b75193700 1 --2- 172.21.15.33:0/370651730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b70108620 0x7f6b70102030 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800b550 tx=0x7f6b6800b860 comp rx=0 tx=0).stop 2024-04-02T13:37:19.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.565+0000 7f6b6f59e700 1 -- 172.21.15.33:0/370651730 reap_dead start 2024-04-02T13:37:19.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.565+0000 7f6b75193700 1 -- 172.21.15.33:0/370651730 shutdown_connections 2024-04-02T13:37:19.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.565+0000 7f6b75193700 1 -- 172.21.15.33:0/370651730 >> 172.21.15.33:0/370651730 conn(0x7f6b70078460 msgr2=0x7f6b701002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:19.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.566+0000 7f6b75193700 1 -- 172.21.15.33:0/370651730 shutdown_connections 2024-04-02T13:37:19.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.566+0000 7f6b75193700 1 -- 172.21.15.33:0/370651730 wait complete. 2024-04-02T13:37:19.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:19 smithi033 ceph-mon[32390]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:37:19.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:19 smithi145 ceph-mon[35646]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:37:19.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:19 smithi050 ceph-mon[35524]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:37:20.487 INFO:teuthology.orchestra.run.smithi033.stdout:201863462963 2024-04-02T13:37:20.488 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.6 2024-04-02T13:37:20.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:20 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:37:20.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:20 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:37:20.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:20 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:37:21.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:21 smithi033 ceph-mon[32390]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 7 op/s; 72 B/s, 0 objects/s recovering 2024-04-02T13:37:21.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:21 smithi145 ceph-mon[35646]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 7 op/s; 72 B/s, 0 objects/s recovering 2024-04-02T13:37:21.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:21 smithi050 ceph-mon[35524]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 7 op/s; 72 B/s, 0 objects/s recovering 2024-04-02T13:37:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:22.734 INFO:teuthology.orchestra.run.smithi033.stdout:313532612636 2024-04-02T13:37:22.734 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.10 2024-04-02T13:37:23.143 INFO:teuthology.orchestra.run.smithi033.stdout:146028888124 2024-04-02T13:37:23.144 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.4 2024-04-02T13:37:23.326 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:23 smithi033 ceph-mon[32390]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s; 72 B/s, 0 objects/s recovering 2024-04-02T13:37:23.327 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:23.327 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:23.327 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:23.327 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:23 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:23.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:23 smithi145 ceph-mon[35646]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s; 72 B/s, 0 objects/s recovering 2024-04-02T13:37:23.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:23.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:23.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:23.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:23 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:23.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:23 smithi050 ceph-mon[35524]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s; 72 B/s, 0 objects/s recovering 2024-04-02T13:37:23.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:23.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:23.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:23.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:23 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.135+0000 7f5446dec700 1 -- 172.21.15.33:0/3027798602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f544006b6f0 msgr2=0x7f544010f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.135+0000 7f5446dec700 1 --2- 172.21.15.33:0/3027798602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f544006b6f0 0x7f544010f510 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5428009990 tx=0x7f5428009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.135+0000 7f5446dec700 1 -- 172.21.15.33:0/3027798602 shutdown_connections 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.135+0000 7f5446dec700 1 --2- 172.21.15.33:0/3027798602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f544006b6f0 0x7f544010f510 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.135+0000 7f5446dec700 1 --2- 172.21.15.33:0/3027798602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54401087d0 0x7f5440108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.135+0000 7f5446dec700 1 --2- 172.21.15.33:0/3027798602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f544006ad30 0x7f544006b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.135+0000 7f5446dec700 1 -- 172.21.15.33:0/3027798602 >> 172.21.15.33:0/3027798602 conn(0x7f5440078350 msgr2=0x7f5440078760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.136+0000 7f5446dec700 1 -- 172.21.15.33:0/3027798602 shutdown_connections 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.136+0000 7f5446dec700 1 -- 172.21.15.33:0/3027798602 wait complete. 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.136+0000 7f5446dec700 1 Processor -- start 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.136+0000 7f5446dec700 1 -- start start 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5446dec700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f544006ad30 0x7f5440198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5446dec700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f544006b6f0 0x7f54401993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5446dec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54401087d0 0x7f544019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:25.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5446dec700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5440112a10 con 0x7f54401087d0 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5446dec700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5440112890 con 0x7f544006ad30 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5446dec700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5440112b90 con 0x7f544006b6f0 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5445389700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54401087d0 0x7f544019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5445389700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54401087d0 0x7f544019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43232/0 (socket says 172.21.15.33:43232) 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5445389700 1 -- 172.21.15.33:0/2016627188 learned_addr learned my addr 172.21.15.33:0/2016627188 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5444b88700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f544006ad30 0x7f5440198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f543ffff700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f544006b6f0 0x7f54401993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5445389700 1 -- 172.21.15.33:0/2016627188 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f544006b6f0 msgr2=0x7f54401993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5445389700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f544006b6f0 0x7f54401993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5445389700 1 -- 172.21.15.33:0/2016627188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f544006ad30 msgr2=0x7f5440198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5445389700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f544006ad30 0x7f5440198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5445389700 1 -- 172.21.15.33:0/2016627188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544019de40 con 0x7f54401087d0 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f543ffff700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f544006b6f0 0x7f54401993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:25.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5445389700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54401087d0 0x7f544019d700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5428005910 tx=0x7f54280038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f543dffb700 1 -- 172.21.15.33:0/2016627188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428005240 con 0x7f54401087d0 2024-04-02T13:37:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f543dffb700 1 -- 172.21.15.33:0/2016627188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f54280053e0 con 0x7f54401087d0 2024-04-02T13:37:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f5446dec700 1 -- 172.21.15.33:0/2016627188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544019e0d0 con 0x7f54401087d0 2024-04-02T13:37:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.137+0000 7f543dffb700 1 -- 172.21.15.33:0/2016627188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428004900 con 0x7f54401087d0 2024-04-02T13:37:25.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.138+0000 7f5446dec700 1 -- 172.21.15.33:0/2016627188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54401a59b0 con 0x7f54401087d0 2024-04-02T13:37:25.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.138+0000 7f5446dec700 1 -- 172.21.15.33:0/2016627188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5440062cc0 con 0x7f54401087d0 2024-04-02T13:37:25.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.140+0000 7f543dffb700 1 -- 172.21.15.33:0/2016627188 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5428005590 con 0x7f54401087d0 2024-04-02T13:37:25.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.140+0000 7f543dffb700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f542c0a3cd0 0x7f542c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:25.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.140+0000 7f543dffb700 1 -- 172.21.15.33:0/2016627188 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f54280d1960 con 0x7f54401087d0 2024-04-02T13:37:25.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.140+0000 7f5444b88700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f542c0a3cd0 0x7f542c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:25.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.140+0000 7f543dffb700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839] conn(0x7f542c0aee30 0x7f542c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:25.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.140+0000 7f543dffb700 1 -- 172.21.15.33:0/2016627188 --> [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f542800b0b0 con 0x7f542c0aee30 2024-04-02T13:37:25.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.140+0000 7f543dffb700 1 -- 172.21.15.33:0/2016627188 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f54280d1db0 con 0x7f54401087d0 2024-04-02T13:37:25.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.141+0000 7f543ffff700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839] conn(0x7f542c0aee30 0x7f542c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:25.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.141+0000 7f5444b88700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f542c0a3cd0 0x7f542c0a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f543800a400 tx=0x7f543800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:25.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.141+0000 7f543ffff700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839] conn(0x7f542c0aee30 0x7f542c0b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:25.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.142+0000 7f543dffb700 1 -- 172.21.15.33:0/2016627188 <== osd.0 v2:172.21.15.33:6802/52311839 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f542800b0b0 con 0x7f542c0aee30 2024-04-02T13:37:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.163+0000 7f5446dec700 1 -- 172.21.15.33:0/2016627188 --> [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5440063140 con 0x7f542c0aee30 2024-04-02T13:37:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.164+0000 7f543dffb700 1 -- 172.21.15.33:0/2016627188 <== osd.0 v2:172.21.15.33:6802/52311839 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5440063140 con 0x7f542c0aee30 2024-04-02T13:37:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.164+0000 7f5446dec700 1 -- 172.21.15.33:0/2016627188 >> [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839] conn(0x7f542c0aee30 msgr2=0x7f542c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.164+0000 7f5446dec700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839] conn(0x7f542c0aee30 0x7f542c0b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.164+0000 7f5446dec700 1 -- 172.21.15.33:0/2016627188 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f542c0a3cd0 msgr2=0x7f542c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.164+0000 7f5446dec700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f542c0a3cd0 0x7f542c0a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f543800a400 tx=0x7f543800d040 comp rx=0 tx=0).stop 2024-04-02T13:37:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.164+0000 7f5446dec700 1 -- 172.21.15.33:0/2016627188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54401087d0 msgr2=0x7f544019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.164+0000 7f5446dec700 1 --2- 172.21.15.33:0/2016627188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54401087d0 0x7f544019d700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5428005910 tx=0x7f54280038b0 comp rx=0 tx=0).stop 2024-04-02T13:37:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.164+0000 7f5445389700 1 -- 172.21.15.33:0/2016627188 reap_dead start 2024-04-02T13:37:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.164+0000 7f5446dec700 1 -- 172.21.15.33:0/2016627188 shutdown_connections 2024-04-02T13:37:25.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.164+0000 7f5446dec700 1 -- 172.21.15.33:0/2016627188 >> 172.21.15.33:0/2016627188 conn(0x7f5440078350 msgr2=0x7f54401004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.164+0000 7f5446dec700 1 -- 172.21.15.33:0/2016627188 shutdown_connections 2024-04-02T13:37:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.164+0000 7f5446dec700 1 -- 172.21.15.33:0/2016627188 wait complete. 2024-04-02T13:37:25.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:25 smithi145 ceph-mon[35646]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 4 op/s; 72 B/s, 0 objects/s recovering 2024-04-02T13:37:25.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:25 smithi050 ceph-mon[35524]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 4 op/s; 72 B/s, 0 objects/s recovering 2024-04-02T13:37:25.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:25 smithi033 ceph-mon[32390]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 4 op/s; 72 B/s, 0 objects/s recovering 2024-04-02T13:37:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.430+0000 7fadd91f7700 1 -- 172.21.15.33:0/3472428363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd4107660 msgr2=0x7fadd4107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.430+0000 7fadd91f7700 1 --2- 172.21.15.33:0/3472428363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd4107660 0x7fadd4107a40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fadc40099c0 tx=0x7fadc4009cd0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.430+0000 7fadd91f7700 1 -- 172.21.15.33:0/3472428363 shutdown_connections 2024-04-02T13:37:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.430+0000 7fadd91f7700 1 --2- 172.21.15.33:0/3472428363 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadd406b6f0 0x7fadd410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.430+0000 7fadd91f7700 1 --2- 172.21.15.33:0/3472428363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd4107660 0x7fadd4107a40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.430+0000 7fadd91f7700 1 --2- 172.21.15.33:0/3472428363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd406ad30 0x7fadd406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.430+0000 7fadd91f7700 1 -- 172.21.15.33:0/3472428363 >> 172.21.15.33:0/3472428363 conn(0x7fadd4078350 msgr2=0x7fadd4078760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.430+0000 7fadd91f7700 1 -- 172.21.15.33:0/3472428363 shutdown_connections 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.430+0000 7fadd91f7700 1 -- 172.21.15.33:0/3472428363 wait complete. 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.431+0000 7fadd91f7700 1 Processor -- start 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.431+0000 7fadd91f7700 1 -- start start 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.431+0000 7fadd91f7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd406ad30 0x7fadd41018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.431+0000 7fadd91f7700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadd406b6f0 0x7fadd4101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.431+0000 7fadd91f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd4107660 0x7fadd4102320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.431+0000 7fadd91f7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fadd4112950 con 0x7fadd4107660 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.431+0000 7fadd91f7700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fadd41127d0 con 0x7fadd406ad30 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.431+0000 7fadd91f7700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fadd4112ad0 con 0x7fadd406b6f0 2024-04-02T13:37:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd4107660 0x7fadd4102320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd4107660 0x7fadd4102320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49308/0 (socket says 172.21.15.33:49308) 2024-04-02T13:37:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd359e700 1 -- 172.21.15.33:0/3628618082 learned_addr learned my addr 172.21.15.33:0/3628618082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd2d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd406ad30 0x7fadd41018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd259c700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadd406b6f0 0x7fadd4101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd359e700 1 -- 172.21.15.33:0/3628618082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadd406b6f0 msgr2=0x7fadd4101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd359e700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadd406b6f0 0x7fadd4101de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd359e700 1 -- 172.21.15.33:0/3628618082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd406ad30 msgr2=0x7fadd41018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd359e700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadd406ad30 0x7fadd41018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd359e700 1 -- 172.21.15.33:0/3628618082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd41a72f0 con 0x7fadd4107660 2024-04-02T13:37:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd259c700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadd406b6f0 0x7fadd4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd359e700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd4107660 0x7fadd4102320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00b9e0 tx=0x7fadcc00bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadbbfff700 1 -- 172.21.15.33:0/3628618082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc00d8e0 con 0x7fadd4107660 2024-04-02T13:37:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd91f7700 1 -- 172.21.15.33:0/3628618082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd41a75e0 con 0x7fadd4107660 2024-04-02T13:37:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadbbfff700 1 -- 172.21.15.33:0/3628618082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fadcc015420 con 0x7fadd4107660 2024-04-02T13:37:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadbbfff700 1 -- 172.21.15.33:0/3628618082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc014610 con 0x7fadd4107660 2024-04-02T13:37:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.432+0000 7fadd91f7700 1 -- 172.21.15.33:0/3628618082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd41a7b20 con 0x7fadd4107660 2024-04-02T13:37:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.433+0000 7fadd91f7700 1 -- 172.21.15.33:0/3628618082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fadd41001c0 con 0x7fadd4107660 2024-04-02T13:37:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.434+0000 7fadbbfff700 1 -- 172.21.15.33:0/3628618082 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fadcc0155d0 con 0x7fadd4107660 2024-04-02T13:37:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.435+0000 7fadbbfff700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fadbc0a3cd0 0x7fadbc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.435+0000 7fadd2d9d700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fadbc0a3cd0 0x7fadbc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.435+0000 7fadbbfff700 1 -- 172.21.15.33:0/3628618082 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fadcc0d0d70 con 0x7fadd4107660 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.435+0000 7fadbbfff700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123] conn(0x7fadbc0aee30 0x7fadbc0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.435+0000 7fadbbfff700 1 -- 172.21.15.33:0/3628618082 --> [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fadbc0b19a0 con 0x7fadbc0aee30 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.435+0000 7fadbbfff700 1 -- 172.21.15.33:0/3628618082 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fadcc0d11c0 con 0x7fadd4107660 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.435+0000 7fadd2d9d700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fadbc0a3cd0 0x7fadbc0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fadc4000c00 tx=0x7fadc4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.435+0000 7fadd259c700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123] conn(0x7fadbc0aee30 0x7fadbc0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.436+0000 7fadd259c700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123] conn(0x7fadbc0aee30 0x7fadbc0b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.437+0000 7fadbbfff700 1 -- 172.21.15.33:0/3628618082 <== osd.11 v2:172.21.15.145:6824/141823123 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fadbc0b19a0 con 0x7fadbc0aee30 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.454+0000 7fadd91f7700 1 -- 172.21.15.33:0/3628618082 --> [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fadd4103db0 con 0x7fadbc0aee30 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.455+0000 7fadbbfff700 1 -- 172.21.15.33:0/3628618082 <== osd.11 v2:172.21.15.145:6824/141823123 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fadd4103db0 con 0x7fadbc0aee30 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.455+0000 7fadd91f7700 1 -- 172.21.15.33:0/3628618082 >> [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123] conn(0x7fadbc0aee30 msgr2=0x7fadbc0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.455+0000 7fadd91f7700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123] conn(0x7fadbc0aee30 0x7fadbc0b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.455+0000 7fadd91f7700 1 -- 172.21.15.33:0/3628618082 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fadbc0a3cd0 msgr2=0x7fadbc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.455+0000 7fadd91f7700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fadbc0a3cd0 0x7fadbc0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fadc4000c00 tx=0x7fadc4005fb0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.456+0000 7fadd91f7700 1 -- 172.21.15.33:0/3628618082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd4107660 msgr2=0x7fadd4102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.456+0000 7fadd91f7700 1 --2- 172.21.15.33:0/3628618082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadd4107660 0x7fadd4102320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00b9e0 tx=0x7fadcc00bcf0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.456+0000 7fadd359e700 1 -- 172.21.15.33:0/3628618082 reap_dead start 2024-04-02T13:37:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.456+0000 7fadd91f7700 1 -- 172.21.15.33:0/3628618082 shutdown_connections 2024-04-02T13:37:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.456+0000 7fadd91f7700 1 -- 172.21.15.33:0/3628618082 >> 172.21.15.33:0/3628618082 conn(0x7fadd4078350 msgr2=0x7fadd40ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.456+0000 7fadd91f7700 1 -- 172.21.15.33:0/3628618082 shutdown_connections 2024-04-02T13:37:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:19.456+0000 7fadd91f7700 1 -- 172.21.15.33:0/3628618082 wait complete. 2024-04-02T13:37:26.710 INFO:teuthology.orchestra.run.smithi033.stdout:347892351001 2024-04-02T13:37:26.710 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.11 2024-04-02T13:37:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.903+0000 7f2553614700 1 -- 172.21.15.33:0/990116116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c100f10 msgr2=0x7f254c1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.903+0000 7f2553614700 1 --2- 172.21.15.33:0/990116116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c100f10 0x7f254c1012f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f25400099c0 tx=0x7f2540009cd0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.903+0000 7f2553614700 1 -- 172.21.15.33:0/990116116 shutdown_connections 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.903+0000 7f2553614700 1 --2- 172.21.15.33:0/990116116 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f254c105ff0 0x7f254c10dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.903+0000 7f2553614700 1 --2- 172.21.15.33:0/990116116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c100f10 0x7f254c1012f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.903+0000 7f2553614700 1 --2- 172.21.15.33:0/990116116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f254c1018c0 0x7f254c105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.903+0000 7f2553614700 1 -- 172.21.15.33:0/990116116 >> 172.21.15.33:0/990116116 conn(0x7f254c077af0 msgr2=0x7f254c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.903+0000 7f2553614700 1 -- 172.21.15.33:0/990116116 shutdown_connections 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.903+0000 7f2553614700 1 -- 172.21.15.33:0/990116116 wait complete. 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2553614700 1 Processor -- start 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2553614700 1 -- start start 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2553614700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f254c100f10 0x7f254c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2553614700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f254c1018c0 0x7f254c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2553614700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c105ff0 0x7f254c19d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2553614700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f254c111070 con 0x7f254c105ff0 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2553614700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f254c110ef0 con 0x7f254c100f10 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2553614700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f254c1111f0 con 0x7f254c1018c0 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2551bb1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c105ff0 0x7f254c19d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2551bb1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c105ff0 0x7f254c19d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43262/0 (socket says 172.21.15.33:43262) 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2551bb1700 1 -- 172.21.15.33:0/2330169171 learned_addr learned my addr 172.21.15.33:0/2330169171 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2550baf700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f254c1018c0 0x7f254c1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f25513b0700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f254c100f10 0x7f254c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2551bb1700 1 -- 172.21.15.33:0/2330169171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f254c1018c0 msgr2=0x7f254c1993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2551bb1700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f254c1018c0 0x7f254c1993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2551bb1700 1 -- 172.21.15.33:0/2330169171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f254c100f10 msgr2=0x7f254c198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2551bb1700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f254c100f10 0x7f254c198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2551bb1700 1 -- 172.21.15.33:0/2330169171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254c19dea0 con 0x7f254c105ff0 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.904+0000 7f2551bb1700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c105ff0 0x7f254c19d760 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f254400b970 tx=0x7f254400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.905+0000 7f253e7fc700 1 -- 172.21.15.33:0/2330169171 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544019070 con 0x7f254c105ff0 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.905+0000 7f253e7fc700 1 -- 172.21.15.33:0/2330169171 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2544014440 con 0x7f254c105ff0 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.905+0000 7f2553614700 1 -- 172.21.15.33:0/2330169171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254c19e190 con 0x7f254c105ff0 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.905+0000 7f2553614700 1 -- 172.21.15.33:0/2330169171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254c1a5a10 con 0x7f254c105ff0 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.905+0000 7f253e7fc700 1 -- 172.21.15.33:0/2330169171 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25440075d0 con 0x7f254c105ff0 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.905+0000 7f2553614700 1 -- 172.21.15.33:0/2330169171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2530001030 con 0x7f254c105ff0 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.907+0000 7f253e7fc700 1 -- 172.21.15.33:0/2330169171 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f254401c070 con 0x7f254c105ff0 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.907+0000 7f253e7fc700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f25380a3d50 0x7f25380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.908+0000 7f25513b0700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f25380a3d50 0x7f25380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.908+0000 7f253e7fc700 1 -- 172.21.15.33:0/2330169171 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f25440d0390 con 0x7f254c105ff0 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.908+0000 7f253e7fc700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485] conn(0x7f25380aeeb0 0x7f25380b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.908+0000 7f253e7fc700 1 -- 172.21.15.33:0/2330169171 --> [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2544024e50 con 0x7f25380aeeb0 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.908+0000 7f253e7fc700 1 -- 172.21.15.33:0/2330169171 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f25440d0740 con 0x7f254c105ff0 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.908+0000 7f2550baf700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485] conn(0x7f25380aeeb0 0x7f25380b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.908+0000 7f25513b0700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f25380a3d50 0x7f25380a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2540009fd0 tx=0x7f2540005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.908+0000 7f2550baf700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485] conn(0x7f25380aeeb0 0x7f25380b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.909+0000 7f253e7fc700 1 -- 172.21.15.33:0/2330169171 <== osd.9 v2:172.21.15.33:6826/1206256485 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2544024e50 con 0x7f25380aeeb0 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.925+0000 7f2553614700 1 -- 172.21.15.33:0/2330169171 --> [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2530002e60 con 0x7f25380aeeb0 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.925+0000 7f253e7fc700 1 -- 172.21.15.33:0/2330169171 <== osd.9 v2:172.21.15.33:6826/1206256485 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2530002e60 con 0x7f25380aeeb0 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.926+0000 7f2553614700 1 -- 172.21.15.33:0/2330169171 >> [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485] conn(0x7f25380aeeb0 msgr2=0x7f25380b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.926+0000 7f2553614700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485] conn(0x7f25380aeeb0 0x7f25380b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.926+0000 7f2553614700 1 -- 172.21.15.33:0/2330169171 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f25380a3d50 msgr2=0x7f25380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.926+0000 7f2553614700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f25380a3d50 0x7f25380a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2540009fd0 tx=0x7f2540005fd0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.926+0000 7f2553614700 1 -- 172.21.15.33:0/2330169171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c105ff0 msgr2=0x7f254c19d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.926+0000 7f2553614700 1 --2- 172.21.15.33:0/2330169171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f254c105ff0 0x7f254c19d760 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f254400b970 tx=0x7f254400bd30 comp rx=0 tx=0).stop 2024-04-02T13:37:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.926+0000 7f2551bb1700 1 -- 172.21.15.33:0/2330169171 reap_dead start 2024-04-02T13:37:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.926+0000 7f2553614700 1 -- 172.21.15.33:0/2330169171 shutdown_connections 2024-04-02T13:37:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.926+0000 7f2553614700 1 -- 172.21.15.33:0/2330169171 >> 172.21.15.33:0/2330169171 conn(0x7f254c077af0 msgr2=0x7f254c103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.926+0000 7f2553614700 1 -- 172.21.15.33:0/2330169171 shutdown_connections 2024-04-02T13:37:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:26.926+0000 7f2553614700 1 -- 172.21.15.33:0/2330169171 wait complete. 2024-04-02T13:37:27.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:27 smithi145 ceph-mon[35646]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 331 B/s wr, 6 op/s; 62 B/s, 0 objects/s recovering 2024-04-02T13:37:27.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:27 smithi050 ceph-mon[35524]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 331 B/s wr, 6 op/s; 62 B/s, 0 objects/s recovering 2024-04-02T13:37:27.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:27 smithi033 ceph-mon[32390]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 331 B/s wr, 6 op/s; 62 B/s, 0 objects/s recovering 2024-04-02T13:37:27.679 INFO:teuthology.orchestra.run.smithi033.stdout:292057776165 2024-04-02T13:37:27.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.9 2024-04-02T13:37:28.091 INFO:teuthology.orchestra.run.smithi033.stdout:111669149761 2024-04-02T13:37:28.091 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.3 2024-04-02T13:37:28.504 INFO:teuthology.orchestra.run.smithi033.stdout:34359738450 2024-04-02T13:37:28.504 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.0 2024-04-02T13:37:29.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.125+0000 7fba034cc700 1 -- 172.21.15.33:0/1851462286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc107520 msgr2=0x7fb9fc107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.125+0000 7fba034cc700 1 --2- 172.21.15.33:0/1851462286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc107520 0x7fb9fc107900 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009990 tx=0x7fb9f8009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.125+0000 7fba034cc700 1 -- 172.21.15.33:0/1851462286 shutdown_connections 2024-04-02T13:37:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.125+0000 7fba034cc700 1 --2- 172.21.15.33:0/1851462286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9fc102fe0 0x7fb9fc10f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.125+0000 7fba034cc700 1 --2- 172.21.15.33:0/1851462286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc107520 0x7fb9fc107900 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.125+0000 7fba034cc700 1 --2- 172.21.15.33:0/1851462286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9fc102640 0x7fb9fc102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.125+0000 7fba034cc700 1 -- 172.21.15.33:0/1851462286 >> 172.21.15.33:0/1851462286 conn(0x7fb9fc079720 msgr2=0x7fb9fc079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.125+0000 7fba034cc700 1 -- 172.21.15.33:0/1851462286 shutdown_connections 2024-04-02T13:37:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.125+0000 7fba034cc700 1 -- 172.21.15.33:0/1851462286 wait complete. 2024-04-02T13:37:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.125+0000 7fba034cc700 1 Processor -- start 2024-04-02T13:37:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.125+0000 7fba034cc700 1 -- start start 2024-04-02T13:37:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba034cc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9fc102640 0x7fb9fc19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba034cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc102fe0 0x7fb9fc19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba034cc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9fc107520 0x7fb9fc19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba034cc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc1128e0 con 0x7fb9fc102fe0 2024-04-02T13:37:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba034cc700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc112760 con 0x7fb9fc107520 2024-04-02T13:37:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba034cc700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc112a60 con 0x7fb9fc102640 2024-04-02T13:37:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba00a67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc102fe0 0x7fb9fc19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba00a67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc102fe0 0x7fb9fc19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43268/0 (socket says 172.21.15.33:43268) 2024-04-02T13:37:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba00a67700 1 -- 172.21.15.33:0/836181674 learned_addr learned my addr 172.21.15.33:0/836181674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba01268700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9fc102640 0x7fb9fc19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba01a69700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9fc107520 0x7fb9fc19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba00a67700 1 -- 172.21.15.33:0/836181674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9fc102640 msgr2=0x7fb9fc19af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba00a67700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9fc102640 0x7fb9fc19af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba00a67700 1 -- 172.21.15.33:0/836181674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9fc107520 msgr2=0x7fb9fc19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba00a67700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9fc107520 0x7fb9fc19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba00a67700 1 -- 172.21.15.33:0/836181674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9fc19ff40 con 0x7fb9fc102fe0 2024-04-02T13:37:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba01a69700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9fc107520 0x7fb9fc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba00a67700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc102fe0 0x7fb9fc19b470 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c9b0 tx=0x7fb9ec00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fb9f27fc700 1 -- 172.21.15.33:0/836181674 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec007b00 con 0x7fb9fc102fe0 2024-04-02T13:37:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fb9f27fc700 1 -- 172.21.15.33:0/836181674 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb9ec007ca0 con 0x7fb9fc102fe0 2024-04-02T13:37:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fba034cc700 1 -- 172.21.15.33:0/836181674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9fc1a0230 con 0x7fb9fc102fe0 2024-04-02T13:37:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.126+0000 7fb9f27fc700 1 -- 172.21.15.33:0/836181674 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec018780 con 0x7fb9fc102fe0 2024-04-02T13:37:29.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.127+0000 7fba034cc700 1 -- 172.21.15.33:0/836181674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9fc1a7ab0 con 0x7fb9fc102fe0 2024-04-02T13:37:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.127+0000 7fba034cc700 1 -- 172.21.15.33:0/836181674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9fc100f10 con 0x7fb9fc102fe0 2024-04-02T13:37:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.129+0000 7fb9f27fc700 1 -- 172.21.15.33:0/836181674 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb9ec00f450 con 0x7fb9fc102fe0 2024-04-02T13:37:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.129+0000 7fb9f27fc700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9e80a3cd0 0x7fb9e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.129+0000 7fba01268700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9e80a3cd0 0x7fb9e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.129+0000 7fb9f27fc700 1 -- 172.21.15.33:0/836181674 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb9ec0d0e10 con 0x7fb9fc102fe0 2024-04-02T13:37:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.130+0000 7fb9f27fc700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848] conn(0x7fb9e80aee30 0x7fb9e80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.130+0000 7fb9f27fc700 1 -- 172.21.15.33:0/836181674 --> [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9ec004500 con 0x7fb9e80aee30 2024-04-02T13:37:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.130+0000 7fb9f27fc700 1 -- 172.21.15.33:0/836181674 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb9ec0d1260 con 0x7fb9fc102fe0 2024-04-02T13:37:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.130+0000 7fba01268700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9e80a3cd0 0x7fb9e80a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8005b40 tx=0x7fb9f8005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.130+0000 7fba01a69700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848] conn(0x7fb9e80aee30 0x7fb9e80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.130+0000 7fba01a69700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848] conn(0x7fb9e80aee30 0x7fb9e80b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.131+0000 7fb9f27fc700 1 -- 172.21.15.33:0/836181674 <== osd.8 v2:172.21.15.145:6816/1364963848 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9ec004500 con 0x7fb9e80aee30 2024-04-02T13:37:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.148+0000 7fba034cc700 1 -- 172.21.15.33:0/836181674 --> [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb9fc103cd0 con 0x7fb9e80aee30 2024-04-02T13:37:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.148+0000 7fb9f27fc700 1 -- 172.21.15.33:0/836181674 <== osd.8 v2:172.21.15.145:6816/1364963848 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb9fc103cd0 con 0x7fb9e80aee30 2024-04-02T13:37:29.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.149+0000 7fba034cc700 1 -- 172.21.15.33:0/836181674 >> [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848] conn(0x7fb9e80aee30 msgr2=0x7fb9e80b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.149+0000 7fba034cc700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848] conn(0x7fb9e80aee30 0x7fb9e80b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.149+0000 7fba034cc700 1 -- 172.21.15.33:0/836181674 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9e80a3cd0 msgr2=0x7fb9e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.149+0000 7fba034cc700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9e80a3cd0 0x7fb9e80a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8005b40 tx=0x7fb9f8005a90 comp rx=0 tx=0).stop 2024-04-02T13:37:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.149+0000 7fba034cc700 1 -- 172.21.15.33:0/836181674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc102fe0 msgr2=0x7fb9fc19b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.149+0000 7fba034cc700 1 --2- 172.21.15.33:0/836181674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9fc102fe0 0x7fb9fc19b470 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c9b0 tx=0x7fb9ec00cd70 comp rx=0 tx=0).stop 2024-04-02T13:37:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.149+0000 7fba01a69700 1 -- 172.21.15.33:0/836181674 reap_dead start 2024-04-02T13:37:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.149+0000 7fba034cc700 1 -- 172.21.15.33:0/836181674 shutdown_connections 2024-04-02T13:37:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.149+0000 7fba034cc700 1 -- 172.21.15.33:0/836181674 >> 172.21.15.33:0/836181674 conn(0x7fb9fc079720 msgr2=0x7fb9fc1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.149+0000 7fba034cc700 1 -- 172.21.15.33:0/836181674 shutdown_connections 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.149+0000 7fba034cc700 1 -- 172.21.15.33:0/836181674 wait complete. 2024-04-02T13:37:29.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:29 smithi033 ceph-mon[32390]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 299 B/s wr, 6 op/s; 56 B/s, 0 objects/s recovering 2024-04-02T13:37:29.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.474+0000 7f8f09ffb700 1 -- 172.21.15.33:0/1248202127 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00bbe0 con 0x7f8f0c1075b0 2024-04-02T13:37:29.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.475+0000 7f8f115d2700 1 -- 172.21.15.33:0/1248202127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c1075b0 msgr2=0x7f8f0c06ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.475+0000 7f8f115d2700 1 --2- 172.21.15.33:0/1248202127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c1075b0 0x7f8f0c06ab30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009990 tx=0x7f8efc009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.475+0000 7f8f115d2700 1 -- 172.21.15.33:0/1248202127 shutdown_connections 2024-04-02T13:37:29.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.475+0000 7f8f115d2700 1 --2- 172.21.15.33:0/1248202127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c105020 0x7f8f0c1054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.475+0000 7f8f115d2700 1 --2- 172.21.15.33:0/1248202127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c1075b0 0x7f8f0c06ab30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.475+0000 7f8f115d2700 1 --2- 172.21.15.33:0/1248202127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f0c06b070 0x7f8f0c06b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.475+0000 7f8f115d2700 1 -- 172.21.15.33:0/1248202127 >> 172.21.15.33:0/1248202127 conn(0x7f8f0c077540 msgr2=0x7f8f0c077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.475+0000 7f8f115d2700 1 -- 172.21.15.33:0/1248202127 shutdown_connections 2024-04-02T13:37:29.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.475+0000 7f8f115d2700 1 -- 172.21.15.33:0/1248202127 wait complete. 2024-04-02T13:37:29.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:29 smithi145 ceph-mon[35646]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 299 B/s wr, 6 op/s; 56 B/s, 0 objects/s recovering 2024-04-02T13:37:29.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.476+0000 7f8f115d2700 1 Processor -- start 2024-04-02T13:37:29.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.476+0000 7f8f115d2700 1 -- start start 2024-04-02T13:37:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.477+0000 7f8f115d2700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c06b070 0x7f8f0c073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.477+0000 7f8f115d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c105020 0x7f8f0c0741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.477+0000 7f8f115d2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f0c1075b0 0x7f8f0c06ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.477+0000 7f8f115d2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c118f40 con 0x7f8f0c105020 2024-04-02T13:37:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.477+0000 7f8f115d2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c118dc0 con 0x7f8f0c1075b0 2024-04-02T13:37:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.477+0000 7f8f115d2700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c1190c0 con 0x7f8f0c06b070 2024-04-02T13:37:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.477+0000 7f8f0a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c105020 0x7f8f0c0741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.477+0000 7f8f0a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c105020 0x7f8f0c0741f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43284/0 (socket says 172.21.15.33:43284) 2024-04-02T13:37:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.477+0000 7f8f0a7fc700 1 -- 172.21.15.33:0/3775032034 learned_addr learned my addr 172.21.15.33:0/3775032034 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.477+0000 7f8f0affd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c06b070 0x7f8f0c073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.477+0000 7f8f0b7fe700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f0c1075b0 0x7f8f0c06ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.478+0000 7f8f0a7fc700 1 -- 172.21.15.33:0/3775032034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c06b070 msgr2=0x7f8f0c073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.478+0000 7f8f0a7fc700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f0c06b070 0x7f8f0c073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.478+0000 7f8f0a7fc700 1 -- 172.21.15.33:0/3775032034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f0c1075b0 msgr2=0x7f8f0c06ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.478+0000 7f8f0a7fc700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f0c1075b0 0x7f8f0c06ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.478+0000 7f8f0a7fc700 1 -- 172.21.15.33:0/3775032034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0c06d750 con 0x7f8f0c105020 2024-04-02T13:37:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.478+0000 7f8f0b7fe700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f0c1075b0 0x7f8f0c06ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:37:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.478+0000 7f8f0a7fc700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c105020 0x7f8f0c0741f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000d980 tx=0x7f8f0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.479+0000 7f8ef3fff700 1 -- 172.21.15.33:0/3775032034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00016070 con 0x7f8f0c105020 2024-04-02T13:37:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.479+0000 7f8ef3fff700 1 -- 172.21.15.33:0/3775032034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8f00004d00 con 0x7f8f0c105020 2024-04-02T13:37:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.479+0000 7f8ef3fff700 1 -- 172.21.15.33:0/3775032034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f000055d0 con 0x7f8f0c105020 2024-04-02T13:37:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.479+0000 7f8f115d2700 1 -- 172.21.15.33:0/3775032034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0c06da40 con 0x7f8f0c105020 2024-04-02T13:37:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.479+0000 7f8f115d2700 1 -- 172.21.15.33:0/3775032034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f0c071a10 con 0x7f8f0c105020 2024-04-02T13:37:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.481+0000 7f8f115d2700 1 -- 172.21.15.33:0/3775032034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f0c100f70 con 0x7f8f0c105020 2024-04-02T13:37:29.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.484+0000 7f8ef3fff700 1 -- 172.21.15.33:0/3775032034 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8f00004750 con 0x7f8f0c105020 2024-04-02T13:37:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.484+0000 7f8ef3fff700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8ef40a3e50 0x7f8ef40a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.485+0000 7f8ef3fff700 1 -- 172.21.15.33:0/3775032034 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8f000d0c60 con 0x7f8f0c105020 2024-04-02T13:37:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.485+0000 7f8ef3fff700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192] conn(0x7f8ef40aefb0 0x7f8ef40b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.485+0000 7f8ef3fff700 1 -- 172.21.15.33:0/3775032034 --> [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f00010e50 con 0x7f8ef40aefb0 2024-04-02T13:37:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.485+0000 7f8ef3fff700 1 -- 172.21.15.33:0/3775032034 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f8f000d1010 con 0x7f8f0c105020 2024-04-02T13:37:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.485+0000 7f8f0affd700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8ef40a3e50 0x7f8ef40a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.486+0000 7f8f0b7fe700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192] conn(0x7f8ef40aefb0 0x7f8ef40b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.486+0000 7f8f0affd700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8ef40a3e50 0x7f8ef40a6310 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009750 tx=0x7f8efc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.486+0000 7f8f0b7fe700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192] conn(0x7f8ef40aefb0 0x7f8ef40b1450 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.487+0000 7f8ef3fff700 1 -- 172.21.15.33:0/3775032034 <== osd.5 v2:172.21.15.145:6808/3244269192 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f00010e50 con 0x7f8ef40aefb0 2024-04-02T13:37:29.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:29 smithi050 ceph-mon[35524]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 299 B/s wr, 6 op/s; 56 B/s, 0 objects/s recovering 2024-04-02T13:37:29.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.514+0000 7f8f115d2700 1 -- 172.21.15.33:0/3775032034 --> [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8f0c101240 con 0x7f8ef40aefb0 2024-04-02T13:37:29.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.515+0000 7f8ef3fff700 1 -- 172.21.15.33:0/3775032034 <== osd.5 v2:172.21.15.145:6808/3244269192 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8f0c101240 con 0x7f8ef40aefb0 2024-04-02T13:37:29.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.515+0000 7f8f115d2700 1 -- 172.21.15.33:0/3775032034 >> [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192] conn(0x7f8ef40aefb0 msgr2=0x7f8ef40b1450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.515+0000 7f8f115d2700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192] conn(0x7f8ef40aefb0 0x7f8ef40b1450 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.515+0000 7f8f115d2700 1 -- 172.21.15.33:0/3775032034 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8ef40a3e50 msgr2=0x7f8ef40a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.515+0000 7f8f115d2700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8ef40a3e50 0x7f8ef40a6310 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009750 tx=0x7f8efc005fd0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.515+0000 7f8f115d2700 1 -- 172.21.15.33:0/3775032034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c105020 msgr2=0x7f8f0c0741f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.515+0000 7f8f115d2700 1 --2- 172.21.15.33:0/3775032034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f0c105020 0x7f8f0c0741f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000d980 tx=0x7f8f0000dc90 comp rx=0 tx=0).stop 2024-04-02T13:37:29.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.515+0000 7f8f0b7fe700 1 -- 172.21.15.33:0/3775032034 reap_dead start 2024-04-02T13:37:29.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.516+0000 7f8f115d2700 1 -- 172.21.15.33:0/3775032034 shutdown_connections 2024-04-02T13:37:29.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.516+0000 7f8f115d2700 1 -- 172.21.15.33:0/3775032034 >> 172.21.15.33:0/3775032034 conn(0x7f8f0c077540 msgr2=0x7f8f0c100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.516+0000 7f8f115d2700 1 -- 172.21.15.33:0/3775032034 shutdown_connections 2024-04-02T13:37:29.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:29.516+0000 7f8f115d2700 1 -- 172.21.15.33:0/3775032034 wait complete. 2024-04-02T13:37:30.344 INFO:teuthology.orchestra.run.smithi033.stdout:167503724601 2024-04-02T13:37:30.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.5 2024-04-02T13:37:30.860 INFO:teuthology.orchestra.run.smithi033.stdout:257698037802 2024-04-02T13:37:30.861 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.8 2024-04-02T13:37:30.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.808+0000 7fb6bfcad700 1 -- 172.21.15.33:0/1009721203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b806ac60 msgr2=0x7fb6b810f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:30.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.808+0000 7fb6bfcad700 1 --2- 172.21.15.33:0/1009721203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b806ac60 0x7fb6b810f450 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800b0a0 tx=0x7fb6a800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.808+0000 7fb6bfcad700 1 -- 172.21.15.33:0/1009721203 shutdown_connections 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.808+0000 7fb6bfcad700 1 --2- 172.21.15.33:0/1009721203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b806ac60 0x7fb6b810f450 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.808+0000 7fb6bfcad700 1 --2- 172.21.15.33:0/1009721203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6b8108650 0x7fb6b8108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.808+0000 7fb6bfcad700 1 --2- 172.21.15.33:0/1009721203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6b806a2a0 0x7fb6b806a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.808+0000 7fb6bfcad700 1 -- 172.21.15.33:0/1009721203 >> 172.21.15.33:0/1009721203 conn(0x7fb6b8078420 msgr2=0x7fb6b8078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.808+0000 7fb6bfcad700 1 -- 172.21.15.33:0/1009721203 shutdown_connections 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.808+0000 7fb6bfcad700 1 -- 172.21.15.33:0/1009721203 wait complete. 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6bfcad700 1 Processor -- start 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6bfcad700 1 -- start start 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6bfcad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b806a2a0 0x7fb6b819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6bfcad700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6b806ac60 0x7fb6b819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6bfcad700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6b8108650 0x7fb6b819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6bfcad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b81128b0 con 0x7fb6b806a2a0 2024-04-02T13:37:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6bfcad700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b8112730 con 0x7fb6b806ac60 2024-04-02T13:37:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6bfcad700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b8112a30 con 0x7fb6b8108650 2024-04-02T13:37:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6bda49700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b806a2a0 0x7fb6b819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6bda49700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b806a2a0 0x7fb6b819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43250/0 (socket says 172.21.15.33:43250) 2024-04-02T13:37:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6bda49700 1 -- 172.21.15.33:0/2500457438 learned_addr learned my addr 172.21.15.33:0/2500457438 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6be24a700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6b8108650 0x7fb6b819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6bd248700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6b806ac60 0x7fb6b819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.809+0000 7fb6bda49700 1 -- 172.21.15.33:0/2500457438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6b8108650 msgr2=0x7fb6b819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.810+0000 7fb6bda49700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6b8108650 0x7fb6b819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.810+0000 7fb6bda49700 1 -- 172.21.15.33:0/2500457438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6b806ac60 msgr2=0x7fb6b819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.810+0000 7fb6bda49700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6b806ac60 0x7fb6b819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.810+0000 7fb6bda49700 1 -- 172.21.15.33:0/2500457438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b819fee0 con 0x7fb6b806a2a0 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.810+0000 7fb6bd248700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6b806ac60 0x7fb6b819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.810+0000 7fb6bda49700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b806a2a0 0x7fb6b819aed0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000d980 tx=0x7fb6b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.810+0000 7fb6b6ffd700 1 -- 172.21.15.33:0/2500457438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0016070 con 0x7fb6b806a2a0 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.810+0000 7fb6b6ffd700 1 -- 172.21.15.33:0/2500457438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb6b0004d00 con 0x7fb6b806a2a0 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.810+0000 7fb6b6ffd700 1 -- 172.21.15.33:0/2500457438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0005550 con 0x7fb6b806a2a0 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.810+0000 7fb6bfcad700 1 -- 172.21.15.33:0/2500457438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b81a01d0 con 0x7fb6b806a2a0 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.810+0000 7fb6bfcad700 1 -- 172.21.15.33:0/2500457438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b81a7a50 con 0x7fb6b806a2a0 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.811+0000 7fb6bfcad700 1 -- 172.21.15.33:0/2500457438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb6b810ea20 con 0x7fb6b806a2a0 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.812+0000 7fb6b6ffd700 1 -- 172.21.15.33:0/2500457438 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb6b0004750 con 0x7fb6b806a2a0 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.813+0000 7fb6b6ffd700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb6a00a3cd0 0x7fb6a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.813+0000 7fb6bd248700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb6a00a3cd0 0x7fb6a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.813+0000 7fb6b6ffd700 1 -- 172.21.15.33:0/2500457438 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb6b00d0dc0 con 0x7fb6b806a2a0 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.813+0000 7fb6b6ffd700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736] conn(0x7fb6a00aee30 0x7fb6a00b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.813+0000 7fb6b6ffd700 1 -- 172.21.15.33:0/2500457438 --> [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6b0015ec0 con 0x7fb6a00aee30 2024-04-02T13:37:30.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.813+0000 7fb6b6ffd700 1 -- 172.21.15.33:0/2500457438 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb6b00d1170 con 0x7fb6b806a2a0 2024-04-02T13:37:30.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.813+0000 7fb6bd248700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb6a00a3cd0 0x7fb6a00a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8103550 tx=0x7fb6ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:30.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.813+0000 7fb6be24a700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736] conn(0x7fb6a00aee30 0x7fb6a00b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:30.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.814+0000 7fb6be24a700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736] conn(0x7fb6a00aee30 0x7fb6a00b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:30.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.815+0000 7fb6b6ffd700 1 -- 172.21.15.33:0/2500457438 <== osd.1 v2:172.21.15.50:6800/1987709736 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb6b0015ec0 con 0x7fb6a00aee30 2024-04-02T13:37:30.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.831+0000 7fb6bfcad700 1 -- 172.21.15.33:0/2500457438 --> [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb6b8103cf0 con 0x7fb6a00aee30 2024-04-02T13:37:30.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.832+0000 7fb6b6ffd700 1 -- 172.21.15.33:0/2500457438 <== osd.1 v2:172.21.15.50:6800/1987709736 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb6b8103cf0 con 0x7fb6a00aee30 2024-04-02T13:37:30.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.832+0000 7fb6bfcad700 1 -- 172.21.15.33:0/2500457438 >> [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736] conn(0x7fb6a00aee30 msgr2=0x7fb6a00b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:30.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.832+0000 7fb6bfcad700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736] conn(0x7fb6a00aee30 0x7fb6a00b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:30.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.832+0000 7fb6bfcad700 1 -- 172.21.15.33:0/2500457438 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb6a00a3cd0 msgr2=0x7fb6a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:30.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.832+0000 7fb6bfcad700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb6a00a3cd0 0x7fb6a00a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8103550 tx=0x7fb6ac009260 comp rx=0 tx=0).stop 2024-04-02T13:37:30.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.832+0000 7fb6bfcad700 1 -- 172.21.15.33:0/2500457438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b806a2a0 msgr2=0x7fb6b819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:30.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.832+0000 7fb6bfcad700 1 --2- 172.21.15.33:0/2500457438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6b806a2a0 0x7fb6b819aed0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000d980 tx=0x7fb6b000dc90 comp rx=0 tx=0).stop 2024-04-02T13:37:30.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.832+0000 7fb6be24a700 1 -- 172.21.15.33:0/2500457438 reap_dead start 2024-04-02T13:37:30.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.832+0000 7fb6bfcad700 1 -- 172.21.15.33:0/2500457438 shutdown_connections 2024-04-02T13:37:30.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.832+0000 7fb6bfcad700 1 -- 172.21.15.33:0/2500457438 >> 172.21.15.33:0/2500457438 conn(0x7fb6b8078420 msgr2=0x7fb6b8100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:30.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.832+0000 7fb6bfcad700 1 -- 172.21.15.33:0/2500457438 shutdown_connections 2024-04-02T13:37:30.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:25.832+0000 7fb6bfcad700 1 -- 172.21.15.33:0/2500457438 wait complete. 2024-04-02T13:37:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:31.143 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:31.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:31 smithi145 ceph-mon[35646]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s; 48 B/s, 0 objects/s recovering 2024-04-02T13:37:31.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:31 smithi050 ceph-mon[35524]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s; 48 B/s, 0 objects/s recovering 2024-04-02T13:37:31.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:31 smithi033 ceph-mon[32390]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s; 48 B/s, 0 objects/s recovering 2024-04-02T13:37:31.604 INFO:teuthology.orchestra.run.smithi033.stdout:55834574925 2024-04-02T13:37:31.604 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.1 2024-04-02T13:37:32.284 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:32.285 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:33.278 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:33 smithi033 ceph-mon[32390]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:37:33.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:33 smithi145 ceph-mon[35646]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:37:33.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:33 smithi050 ceph-mon[35524]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:37:35.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:35 smithi145 ceph-mon[35646]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:37:35.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:35 smithi050 ceph-mon[35524]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:37:35.502 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:35 smithi033 ceph-mon[32390]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:37:36.971 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:36.972 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:37.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:37 smithi145 ceph-mon[35646]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:37:37.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:37 smithi050 ceph-mon[35524]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:37:37.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:37 smithi033 ceph-mon[32390]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:37:39.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:39 smithi145 ceph-mon[35646]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:39.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:39 smithi050 ceph-mon[35524]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:39.495 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:39 smithi033 ceph-mon[32390]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.133+0000 7f3b78b3e700 1 -- 172.21.15.33:0/1478979842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b74107520 msgr2=0x7f3b74107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.133+0000 7f3b78b3e700 1 --2- 172.21.15.33:0/1478979842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b74107520 0x7f3b74107900 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009990 tx=0x7f3b64009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.133+0000 7f3b78b3e700 1 -- 172.21.15.33:0/1478979842 shutdown_connections 2024-04-02T13:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.133+0000 7f3b78b3e700 1 --2- 172.21.15.33:0/1478979842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b74102fe0 0x7f3b7410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.133+0000 7f3b78b3e700 1 --2- 172.21.15.33:0/1478979842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b74107520 0x7f3b74107900 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.133+0000 7f3b78b3e700 1 --2- 172.21.15.33:0/1478979842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b74102640 0x7f3b74102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.133+0000 7f3b78b3e700 1 -- 172.21.15.33:0/1478979842 >> 172.21.15.33:0/1478979842 conn(0x7f3b74079720 msgr2=0x7f3b74079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.133+0000 7f3b78b3e700 1 -- 172.21.15.33:0/1478979842 shutdown_connections 2024-04-02T13:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.133+0000 7f3b78b3e700 1 -- 172.21.15.33:0/1478979842 wait complete. 2024-04-02T13:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b78b3e700 1 Processor -- start 2024-04-02T13:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b78b3e700 1 -- start start 2024-04-02T13:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b78b3e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b74102640 0x7f3b74198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b78b3e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b74102fe0 0x7f3b741992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b78b3e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b74107520 0x7f3b7419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b78b3e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b741128e0 con 0x7f3b74102fe0 2024-04-02T13:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b78b3e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3b74112760 con 0x7f3b74102640 2024-04-02T13:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b78b3e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3b74112a60 con 0x7f3b74107520 2024-04-02T13:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b71d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b74102fe0 0x7f3b741992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b7259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b74102640 0x7f3b74198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b7259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b74102640 0x7f3b74198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:50716/0 (socket says 172.21.15.33:50716) 2024-04-02T13:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b71d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b74102fe0 0x7f3b741992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37336/0 (socket says 172.21.15.33:37336) 2024-04-02T13:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b7259c700 1 -- 172.21.15.33:0/979026688 learned_addr learned my addr 172.21.15.33:0/979026688 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.134+0000 7f3b72d9d700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b74107520 0x7f3b7419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.135+0000 7f3b71d9b700 1 -- 172.21.15.33:0/979026688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b74107520 msgr2=0x7f3b7419d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.135+0000 7f3b71d9b700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b74107520 0x7f3b7419d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.135+0000 7f3b71d9b700 1 -- 172.21.15.33:0/979026688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b74102640 msgr2=0x7f3b74198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.135+0000 7f3b71d9b700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b74102640 0x7f3b74198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.135+0000 7f3b71d9b700 1 -- 172.21.15.33:0/979026688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7419ddc0 con 0x7f3b74102fe0 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.135+0000 7f3b7259c700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b74102640 0x7f3b74198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.135+0000 7f3b71d9b700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b74102fe0 0x7f3b741992f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00c970 tx=0x7f3b5c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.135+0000 7f3b6b7fe700 1 -- 172.21.15.33:0/979026688 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c004a30 con 0x7f3b74102fe0 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.135+0000 7f3b78b3e700 1 -- 172.21.15.33:0/979026688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7419e0b0 con 0x7f3b74102fe0 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.135+0000 7f3b6b7fe700 1 -- 172.21.15.33:0/979026688 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3b5c0054f0 con 0x7f3b74102fe0 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.135+0000 7f3b6b7fe700 1 -- 172.21.15.33:0/979026688 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00ec80 con 0x7f3b74102fe0 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.135+0000 7f3b78b3e700 1 -- 172.21.15.33:0/979026688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b741a5930 con 0x7f3b74102fe0 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.136+0000 7f3b78b3e700 1 -- 172.21.15.33:0/979026688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b740692b0 con 0x7f3b74102fe0 2024-04-02T13:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.137+0000 7f3b6b7fe700 1 -- 172.21.15.33:0/979026688 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3b5c004fa0 con 0x7f3b74102fe0 2024-04-02T13:37:40.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.138+0000 7f3b6b7fe700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3b600a3cd0 0x7f3b600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:40.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.138+0000 7f3b7259c700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3b600a3cd0 0x7f3b600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:40.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.138+0000 7f3b6b7fe700 1 -- 172.21.15.33:0/979026688 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3b5c0d0680 con 0x7f3b74102fe0 2024-04-02T13:37:40.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.138+0000 7f3b7259c700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3b600a3cd0 0x7f3b600a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3b64006010 tx=0x7f3b64005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:40.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.142+0000 7f3b6b7fe700 1 -- 172.21.15.33:0/979026688 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3b5c014030 con 0x7f3b74102fe0 2024-04-02T13:37:40.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.360+0000 7f3b78b3e700 1 -- 172.21.15.33:0/979026688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3b7410ab50 con 0x7f3b74102fe0 2024-04-02T13:37:40.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.360+0000 7f3b6b7fe700 1 -- 172.21.15.33:0/979026688 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3b5c09cba0 con 0x7f3b74102fe0 2024-04-02T13:37:40.364 INFO:teuthology.orchestra.run.smithi033.stdout:201863462967 2024-04-02T13:37:40.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.362+0000 7f3b78b3e700 1 -- 172.21.15.33:0/979026688 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3b600a3cd0 msgr2=0x7f3b600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:40.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.362+0000 7f3b78b3e700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3b600a3cd0 0x7f3b600a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3b64006010 tx=0x7f3b64005960 comp rx=0 tx=0).stop 2024-04-02T13:37:40.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.362+0000 7f3b78b3e700 1 -- 172.21.15.33:0/979026688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b74102fe0 msgr2=0x7f3b741992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:40.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.362+0000 7f3b78b3e700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b74102fe0 0x7f3b741992f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00c970 tx=0x7f3b5c00cc80 comp rx=0 tx=0).stop 2024-04-02T13:37:40.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.362+0000 7f3b78b3e700 1 -- 172.21.15.33:0/979026688 shutdown_connections 2024-04-02T13:37:40.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.362+0000 7f3b78b3e700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3b600a3cd0 0x7f3b600a6190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.362+0000 7f3b78b3e700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3b74107520 0x7f3b7419d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.362+0000 7f3b78b3e700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b74102640 0x7f3b74198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.362+0000 7f3b78b3e700 1 --2- 172.21.15.33:0/979026688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b74102fe0 0x7f3b741992f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.362+0000 7f3b78b3e700 1 -- 172.21.15.33:0/979026688 >> 172.21.15.33:0/979026688 conn(0x7f3b74079720 msgr2=0x7f3b74100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:40.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.362+0000 7f3b78b3e700 1 -- 172.21.15.33:0/979026688 shutdown_connections 2024-04-02T13:37:40.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.362+0000 7f3b78b3e700 1 -- 172.21.15.33:0/979026688 wait complete. 2024-04-02T13:37:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.788+0000 7f35a638b700 1 -- 172.21.15.33:0/170259807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35a0101580 msgr2=0x7f35a0101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.788+0000 7f35a638b700 1 --2- 172.21.15.33:0/170259807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35a0101580 0x7f35a0101a00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f35880099c0 tx=0x7f3588009cd0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.788+0000 7f35a638b700 1 -- 172.21.15.33:0/170259807 shutdown_connections 2024-04-02T13:37:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.788+0000 7f35a638b700 1 --2- 172.21.15.33:0/170259807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35a0101f40 0x7f35a010e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.788+0000 7f35a638b700 1 --2- 172.21.15.33:0/170259807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35a0107630 0x7f35a0107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.788+0000 7f35a638b700 1 --2- 172.21.15.33:0/170259807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35a0101580 0x7f35a0101a00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.788+0000 7f35a638b700 1 -- 172.21.15.33:0/170259807 >> 172.21.15.33:0/170259807 conn(0x7f35a0078620 msgr2=0x7f35a0078a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.788+0000 7f35a638b700 1 -- 172.21.15.33:0/170259807 shutdown_connections 2024-04-02T13:37:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.788+0000 7f35a638b700 1 -- 172.21.15.33:0/170259807 wait complete. 2024-04-02T13:37:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.789+0000 7f35a638b700 1 Processor -- start 2024-04-02T13:37:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.789+0000 7f35a638b700 1 -- start start 2024-04-02T13:37:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.789+0000 7f35a638b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35a0101580 0x7f35a0198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.789+0000 7f35a638b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35a0101f40 0x7f35a0199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.789+0000 7f35a638b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35a0107630 0x7f35a019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.789+0000 7f35a638b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35a0111880 con 0x7f35a0107630 2024-04-02T13:37:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.789+0000 7f35a638b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f35a0111700 con 0x7f35a0101580 2024-04-02T13:37:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.789+0000 7f35a638b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f35a0111a00 con 0x7f35a0101f40 2024-04-02T13:37:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.789+0000 7f35a4928700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35a0107630 0x7f35a019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.789+0000 7f35a4928700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35a0107630 0x7f35a019d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37364/0 (socket says 172.21.15.33:37364) 2024-04-02T13:37:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.789+0000 7f35a4928700 1 -- 172.21.15.33:0/1481864098 learned_addr learned my addr 172.21.15.33:0/1481864098 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.789+0000 7f359ffff700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35a0101580 0x7f35a0198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.790+0000 7f359f7fe700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35a0101f40 0x7f35a0199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.790+0000 7f35a4928700 1 -- 172.21.15.33:0/1481864098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35a0101f40 msgr2=0x7f35a0199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.790+0000 7f35a4928700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35a0101f40 0x7f35a0199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.790+0000 7f35a4928700 1 -- 172.21.15.33:0/1481864098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35a0101580 msgr2=0x7f35a0198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.790+0000 7f35a4928700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35a0101580 0x7f35a0198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.790+0000 7f35a4928700 1 -- 172.21.15.33:0/1481864098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a019ddf0 con 0x7f35a0107630 2024-04-02T13:37:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.790+0000 7f359f7fe700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35a0101f40 0x7f35a0199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:40.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.790+0000 7f35a4928700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35a0107630 0x7f35a019d6b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f359800b970 tx=0x7f359800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:40.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.790+0000 7f359d7fa700 1 -- 172.21.15.33:0/1481864098 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598019070 con 0x7f35a0107630 2024-04-02T13:37:40.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.790+0000 7f35a638b700 1 -- 172.21.15.33:0/1481864098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a019e0e0 con 0x7f35a0107630 2024-04-02T13:37:40.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.790+0000 7f359d7fa700 1 -- 172.21.15.33:0/1481864098 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f359800ce00 con 0x7f35a0107630 2024-04-02T13:37:40.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.790+0000 7f359d7fa700 1 -- 172.21.15.33:0/1481864098 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598007370 con 0x7f35a0107630 2024-04-02T13:37:40.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.790+0000 7f35a638b700 1 -- 172.21.15.33:0/1481864098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a01a5960 con 0x7f35a0107630 2024-04-02T13:37:40.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.791+0000 7f35a638b700 1 -- 172.21.15.33:0/1481864098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a00692b0 con 0x7f35a0107630 2024-04-02T13:37:40.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.792+0000 7f359d7fa700 1 -- 172.21.15.33:0/1481864098 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f359801c070 con 0x7f35a0107630 2024-04-02T13:37:40.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.793+0000 7f359d7fa700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f358c0a3cd0 0x7f358c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:40.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.793+0000 7f359ffff700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f358c0a3cd0 0x7f358c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:40.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.793+0000 7f359d7fa700 1 -- 172.21.15.33:0/1481864098 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f35980d00a0 con 0x7f35a0107630 2024-04-02T13:37:40.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.793+0000 7f359ffff700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f358c0a3cd0 0x7f358c0a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3590005950 tx=0x7f359000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:40.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:40.797+0000 7f359d7fa700 1 -- 172.21.15.33:0/1481864098 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f359809c670 con 0x7f35a0107630 2024-04-02T13:37:41.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.008+0000 7f35a638b700 1 -- 172.21.15.33:0/1481864098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f35a01a5bf0 con 0x7f35a0107630 2024-04-02T13:37:41.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.009+0000 7f359d7fa700 1 -- 172.21.15.33:0/1481864098 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f359809bf70 con 0x7f35a0107630 2024-04-02T13:37:41.013 INFO:teuthology.orchestra.run.smithi033.stdout:223338299442 2024-04-02T13:37:41.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.011+0000 7f35a638b700 1 -- 172.21.15.33:0/1481864098 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f358c0a3cd0 msgr2=0x7f358c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.011+0000 7f35a638b700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f358c0a3cd0 0x7f358c0a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3590005950 tx=0x7f359000b410 comp rx=0 tx=0).stop 2024-04-02T13:37:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.011+0000 7f35a638b700 1 -- 172.21.15.33:0/1481864098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35a0107630 msgr2=0x7f35a019d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.011+0000 7f35a638b700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35a0107630 0x7f35a019d6b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f359800b970 tx=0x7f359800bc80 comp rx=0 tx=0).stop 2024-04-02T13:37:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.011+0000 7f35a638b700 1 -- 172.21.15.33:0/1481864098 shutdown_connections 2024-04-02T13:37:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.011+0000 7f35a638b700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f358c0a3cd0 0x7f358c0a6190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.011+0000 7f35a638b700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35a0107630 0x7f35a019d6b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.011+0000 7f35a638b700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f35a0101580 0x7f35a0198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.011+0000 7f35a638b700 1 --2- 172.21.15.33:0/1481864098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f35a0101f40 0x7f35a0199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.011+0000 7f35a638b700 1 -- 172.21.15.33:0/1481864098 >> 172.21.15.33:0/1481864098 conn(0x7f35a0078620 msgr2=0x7f35a01007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.011+0000 7f35a638b700 1 -- 172.21.15.33:0/1481864098 shutdown_connections 2024-04-02T13:37:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.011+0000 7f35a638b700 1 -- 172.21.15.33:0/1481864098 wait complete. 2024-04-02T13:37:41.254 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462963 got 201863462967 for osd.6 2024-04-02T13:37:41.254 DEBUG:teuthology.parallel:result is None 2024-04-02T13:37:41.338 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:41 smithi033 ceph-mon[32390]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-02T13:37:41.339 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:41 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/979026688' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T13:37:41.339 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:41 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1481864098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T13:37:41.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:41 smithi145 ceph-mon[35646]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-02T13:37:41.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:41 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/979026688' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T13:37:41.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:41 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1481864098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T13:37:41.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:41 smithi050 ceph-mon[35524]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-02T13:37:41.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:41 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/979026688' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T13:37:41.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:41 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1481864098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T13:37:41.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.937+0000 7f634218d700 1 -- 172.21.15.33:0/1720325280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c1014f0 msgr2=0x7f633c101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:41.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.937+0000 7f634218d700 1 --2- 172.21.15.33:0/1720325280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c1014f0 0x7f633c101970 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6330009990 tx=0x7f6330009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:41.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.937+0000 7f634218d700 1 -- 172.21.15.33:0/1720325280 shutdown_connections 2024-04-02T13:37:41.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.937+0000 7f634218d700 1 --2- 172.21.15.33:0/1720325280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f633c101eb0 0x7f633c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:41.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.937+0000 7f634218d700 1 --2- 172.21.15.33:0/1720325280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c107500 0x7f633c1078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:41.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.937+0000 7f634218d700 1 --2- 172.21.15.33:0/1720325280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c1014f0 0x7f633c101970 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:41.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.937+0000 7f634218d700 1 -- 172.21.15.33:0/1720325280 >> 172.21.15.33:0/1720325280 conn(0x7f633c078420 msgr2=0x7f633c078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:41.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.937+0000 7f634218d700 1 -- 172.21.15.33:0/1720325280 shutdown_connections 2024-04-02T13:37:41.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.937+0000 7f634218d700 1 -- 172.21.15.33:0/1720325280 wait complete. 2024-04-02T13:37:41.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.938+0000 7f634218d700 1 Processor -- start 2024-04-02T13:37:41.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.938+0000 7f634218d700 1 -- start start 2024-04-02T13:37:41.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.938+0000 7f634218d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c1014f0 0x7f633c19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:41.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.938+0000 7f634218d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c101eb0 0x7f633c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:41.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.938+0000 7f634218d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f633c107500 0x7f633c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.938+0000 7f634218d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f633c111770 con 0x7f633c1014f0 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.938+0000 7f634218d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1115f0 con 0x7f633c101eb0 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.938+0000 7f634218d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1118f0 con 0x7f633c107500 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.938+0000 7f633b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c1014f0 0x7f633c19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.938+0000 7f633b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c1014f0 0x7f633c19ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37380/0 (socket says 172.21.15.33:37380) 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.938+0000 7f633b7fe700 1 -- 172.21.15.33:0/4072028950 learned_addr learned my addr 172.21.15.33:0/4072028950 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.938+0000 7f633bfff700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f633c107500 0x7f633c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.938+0000 7f633affd700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c101eb0 0x7f633c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.939+0000 7f633b7fe700 1 -- 172.21.15.33:0/4072028950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f633c107500 msgr2=0x7f633c198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.939+0000 7f633b7fe700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f633c107500 0x7f633c198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.939+0000 7f633b7fe700 1 -- 172.21.15.33:0/4072028950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c101eb0 msgr2=0x7f633c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.939+0000 7f633b7fe700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c101eb0 0x7f633c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.939+0000 7f633b7fe700 1 -- 172.21.15.33:0/4072028950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633c1993f0 con 0x7f633c1014f0 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.939+0000 7f633b7fe700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c1014f0 0x7f633c19ea40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f632c00d950 tx=0x7f632c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:41.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.939+0000 7f6338ff9700 1 -- 172.21.15.33:0/4072028950 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c016070 con 0x7f633c1014f0 2024-04-02T13:37:41.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.939+0000 7f634218d700 1 -- 172.21.15.33:0/4072028950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633c1996e0 con 0x7f633c1014f0 2024-04-02T13:37:41.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.939+0000 7f6338ff9700 1 -- 172.21.15.33:0/4072028950 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f632c009e10 con 0x7f633c1014f0 2024-04-02T13:37:41.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.939+0000 7f6338ff9700 1 -- 172.21.15.33:0/4072028950 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c00f620 con 0x7f633c1014f0 2024-04-02T13:37:41.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.939+0000 7f634218d700 1 -- 172.21.15.33:0/4072028950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633c1a59c0 con 0x7f633c1014f0 2024-04-02T13:37:41.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.940+0000 7f634218d700 1 -- 172.21.15.33:0/4072028950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f633c102be0 con 0x7f633c1014f0 2024-04-02T13:37:41.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.941+0000 7f6338ff9700 1 -- 172.21.15.33:0/4072028950 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f632c019070 con 0x7f633c1014f0 2024-04-02T13:37:41.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.942+0000 7f6338ff9700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f63240a3cd0 0x7f63240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:41.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.942+0000 7f633affd700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f63240a3cd0 0x7f63240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:41.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.942+0000 7f6338ff9700 1 -- 172.21.15.33:0/4072028950 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f632c0d0540 con 0x7f633c1014f0 2024-04-02T13:37:41.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.942+0000 7f633affd700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f63240a3cd0 0x7f63240a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6330006010 tx=0x7f633000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:41.946+0000 7f6338ff9700 1 -- 172.21.15.33:0/4072028950 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f632c09ca00 con 0x7f633c1014f0 2024-04-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.145+0000 7f171e61e700 1 -- 172.21.15.33:0/785139967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718107500 msgr2=0x7f17181078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:42.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.145+0000 7f171e61e700 1 --2- 172.21.15.33:0/785139967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718107500 0x7f17181078e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1708009990 tx=0x7f1708009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.145+0000 7f171e61e700 1 -- 172.21.15.33:0/785139967 shutdown_connections 2024-04-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.145+0000 7f171e61e700 1 --2- 172.21.15.33:0/785139967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1718101eb0 0x7f171810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.145+0000 7f171e61e700 1 --2- 172.21.15.33:0/785139967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718107500 0x7f17181078e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.145+0000 7f171e61e700 1 --2- 172.21.15.33:0/785139967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17181014f0 0x7f1718101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.145+0000 7f171e61e700 1 -- 172.21.15.33:0/785139967 >> 172.21.15.33:0/785139967 conn(0x7f1718078420 msgr2=0x7f1718078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.145+0000 7f171e61e700 1 -- 172.21.15.33:0/785139967 shutdown_connections 2024-04-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.145+0000 7f171e61e700 1 -- 172.21.15.33:0/785139967 wait complete. 2024-04-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.146+0000 7f171e61e700 1 Processor -- start 2024-04-02T13:37:42.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.146+0000 7f171e61e700 1 -- start start 2024-04-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.146+0000 7f171e61e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17181014f0 0x7f171819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.146+0000 7f171e61e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718101eb0 0x7f171819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.146+0000 7f171e61e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1718107500 0x7f1718198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.146+0000 7f171e61e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1718111780 con 0x7f1718101eb0 2024-04-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.146+0000 7f171e61e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1718111600 con 0x7f17181014f0 2024-04-02T13:37:42.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.146+0000 7f171e61e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1718111900 con 0x7f1718107500 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.146+0000 7f17177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718101eb0 0x7f171819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.146+0000 7f17177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718101eb0 0x7f171819ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37404/0 (socket says 172.21.15.33:37404) 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.146+0000 7f17177fe700 1 -- 172.21.15.33:0/3440633872 learned_addr learned my addr 172.21.15.33:0/3440633872 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.147+0000 7f171cbbb700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1718107500 0x7f1718198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.147+0000 7f1717fff700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17181014f0 0x7f171819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.147+0000 7f17177fe700 1 -- 172.21.15.33:0/3440633872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1718107500 msgr2=0x7f1718198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.147+0000 7f17177fe700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1718107500 0x7f1718198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.147+0000 7f17177fe700 1 -- 172.21.15.33:0/3440633872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17181014f0 msgr2=0x7f171819e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.147+0000 7f17177fe700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17181014f0 0x7f171819e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.147+0000 7f17177fe700 1 -- 172.21.15.33:0/3440633872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1718199280 con 0x7f1718101eb0 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.147+0000 7f17177fe700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718101eb0 0x7f171819ee50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f170c00d950 tx=0x7f170c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.147+0000 7f17157fa700 1 -- 172.21.15.33:0/3440633872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c016070 con 0x7f1718101eb0 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.147+0000 7f171e61e700 1 -- 172.21.15.33:0/3440633872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1718199570 con 0x7f1718101eb0 2024-04-02T13:37:42.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.147+0000 7f17157fa700 1 -- 172.21.15.33:0/3440633872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f170c00fd60 con 0x7f1718101eb0 2024-04-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.147+0000 7f17157fa700 1 -- 172.21.15.33:0/3440633872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c00e3b0 con 0x7f1718101eb0 2024-04-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.147+0000 7f171e61e700 1 -- 172.21.15.33:0/3440633872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17181a57e0 con 0x7f1718101eb0 2024-04-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.148+0000 7f171e61e700 1 -- 172.21.15.33:0/3440633872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17180692b0 con 0x7f1718101eb0 2024-04-02T13:37:42.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.150+0000 7f17157fa700 1 -- 172.21.15.33:0/3440633872 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f170c019070 con 0x7f1718101eb0 2024-04-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.151+0000 7f17157fa700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f17000a3cd0 0x7f17000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.151+0000 7f1717fff700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f17000a3cd0 0x7f17000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.151+0000 7f17157fa700 1 -- 172.21.15.33:0/3440633872 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f170c0d0460 con 0x7f1718101eb0 2024-04-02T13:37:42.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.151+0000 7f1717fff700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f17000a3cd0 0x7f17000a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f17080098e0 tx=0x7f1708005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.155+0000 7f17157fa700 1 -- 172.21.15.33:0/3440633872 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f170c09c920 con 0x7f1718101eb0 2024-04-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.160+0000 7f634218d700 1 -- 172.21.15.33:0/4072028950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f633c199fd0 con 0x7f633c1014f0 2024-04-02T13:37:42.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.160+0000 7f6338ff9700 1 -- 172.21.15.33:0/4072028950 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f632c09c300 con 0x7f633c1014f0 2024-04-02T13:37:42.164 INFO:teuthology.orchestra.run.smithi033.stdout:313532612642 2024-04-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.162+0000 7f634218d700 1 -- 172.21.15.33:0/4072028950 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f63240a3cd0 msgr2=0x7f63240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.162+0000 7f634218d700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f63240a3cd0 0x7f63240a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6330006010 tx=0x7f633000b540 comp rx=0 tx=0).stop 2024-04-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.162+0000 7f634218d700 1 -- 172.21.15.33:0/4072028950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c1014f0 msgr2=0x7f633c19ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.162+0000 7f634218d700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c1014f0 0x7f633c19ea40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f632c00d950 tx=0x7f632c00dc60 comp rx=0 tx=0).stop 2024-04-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.162+0000 7f634218d700 1 -- 172.21.15.33:0/4072028950 shutdown_connections 2024-04-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.162+0000 7f634218d700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f63240a3cd0 0x7f63240a6190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.162+0000 7f634218d700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f633c107500 0x7f633c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.162+0000 7f634218d700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f633c1014f0 0x7f633c19ea40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.162+0000 7f634218d700 1 --2- 172.21.15.33:0/4072028950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c101eb0 0x7f633c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.162+0000 7f634218d700 1 -- 172.21.15.33:0/4072028950 >> 172.21.15.33:0/4072028950 conn(0x7f633c078420 msgr2=0x7f633c1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.162+0000 7f634218d700 1 -- 172.21.15.33:0/4072028950 shutdown_connections 2024-04-02T13:37:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.162+0000 7f634218d700 1 -- 172.21.15.33:0/4072028950 wait complete. 2024-04-02T13:37:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.374+0000 7f171e61e700 1 -- 172.21.15.33:0/3440633872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f17181a5ad0 con 0x7f1718101eb0 2024-04-02T13:37:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.374+0000 7f17157fa700 1 -- 172.21.15.33:0/3440633872 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f170c09c220 con 0x7f1718101eb0 2024-04-02T13:37:42.378 INFO:teuthology.orchestra.run.smithi033.stdout:146028888129 2024-04-02T13:37:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.376+0000 7f171e61e700 1 -- 172.21.15.33:0/3440633872 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f17000a3cd0 msgr2=0x7f17000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.376+0000 7f171e61e700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f17000a3cd0 0x7f17000a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f17080098e0 tx=0x7f1708005960 comp rx=0 tx=0).stop 2024-04-02T13:37:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.376+0000 7f171e61e700 1 -- 172.21.15.33:0/3440633872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718101eb0 msgr2=0x7f171819ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.376+0000 7f171e61e700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718101eb0 0x7f171819ee50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f170c00d950 tx=0x7f170c00dc60 comp rx=0 tx=0).stop 2024-04-02T13:37:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.376+0000 7f171e61e700 1 -- 172.21.15.33:0/3440633872 shutdown_connections 2024-04-02T13:37:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.376+0000 7f171e61e700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f17000a3cd0 0x7f17000a6190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.376+0000 7f171e61e700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1718107500 0x7f1718198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.376+0000 7f171e61e700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17181014f0 0x7f171819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.376+0000 7f171e61e700 1 --2- 172.21.15.33:0/3440633872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1718101eb0 0x7f171819ee50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.376+0000 7f171e61e700 1 -- 172.21.15.33:0/3440633872 >> 172.21.15.33:0/3440633872 conn(0x7f1718078420 msgr2=0x7f1718100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.377+0000 7f171e61e700 1 -- 172.21.15.33:0/3440633872 shutdown_connections 2024-04-02T13:37:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.377+0000 7f171e61e700 1 -- 172.21.15.33:0/3440633872 wait complete. 2024-04-02T13:37:42.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:42 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4072028950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T13:37:42.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:42 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4072028950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T13:37:42.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:42 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4072028950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T13:37:42.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.896+0000 7f1712e59700 1 -- 172.21.15.33:0/493207198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c101eb0 msgr2=0x7f170c10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:42.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.896+0000 7f1712e59700 1 --2- 172.21.15.33:0/493207198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c101eb0 0x7f170c10e340 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1704009990 tx=0x7f1704009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.896+0000 7f1712e59700 1 -- 172.21.15.33:0/493207198 shutdown_connections 2024-04-02T13:37:42.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.896+0000 7f1712e59700 1 --2- 172.21.15.33:0/493207198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c101eb0 0x7f170c10e340 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.896+0000 7f1712e59700 1 --2- 172.21.15.33:0/493207198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f170c107500 0x7f170c1078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.896+0000 7f1712e59700 1 --2- 172.21.15.33:0/493207198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f170c1014f0 0x7f170c101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.896+0000 7f1712e59700 1 -- 172.21.15.33:0/493207198 >> 172.21.15.33:0/493207198 conn(0x7f170c078420 msgr2=0x7f170c078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.896+0000 7f1712e59700 1 -- 172.21.15.33:0/493207198 shutdown_connections 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.896+0000 7f1712e59700 1 -- 172.21.15.33:0/493207198 wait complete. 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f1712e59700 1 Processor -- start 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f1712e59700 1 -- start start 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f1712e59700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f170c1014f0 0x7f170c199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f1712e59700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c101eb0 0x7f170c19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f1712e59700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f170c107500 0x7f170c1a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f1712e59700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f170c111780 con 0x7f170c101eb0 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f1712e59700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f170c111600 con 0x7f170c107500 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f1712e59700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f170c111900 con 0x7f170c1014f0 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f170bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c101eb0 0x7f170c19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f170bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c101eb0 0x7f170c19a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37428/0 (socket says 172.21.15.33:37428) 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f170bfff700 1 -- 172.21.15.33:0/4242725330 learned_addr learned my addr 172.21.15.33:0/4242725330 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:42.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f1710bf5700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f170c1014f0 0x7f170c199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:42.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f17113f6700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f170c107500 0x7f170c1a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:42.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f170bfff700 1 -- 172.21.15.33:0/4242725330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f170c1014f0 msgr2=0x7f170c199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:42.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f170bfff700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f170c1014f0 0x7f170c199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f170bfff700 1 -- 172.21.15.33:0/4242725330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f170c107500 msgr2=0x7f170c1a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:42.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.897+0000 7f170bfff700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f170c107500 0x7f170c1a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:42.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.898+0000 7f170bfff700 1 -- 172.21.15.33:0/4242725330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c1a50c0 con 0x7f170c101eb0 2024-04-02T13:37:42.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.898+0000 7f170bfff700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c101eb0 0x7f170c19a040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f16f400d980 tx=0x7f16f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:42.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.898+0000 7f1709ffb700 1 -- 172.21.15.33:0/4242725330 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4017070 con 0x7f170c101eb0 2024-04-02T13:37:42.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.898+0000 7f1709ffb700 1 -- 172.21.15.33:0/4242725330 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f16f4005020 con 0x7f170c101eb0 2024-04-02T13:37:42.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.898+0000 7f1712e59700 1 -- 172.21.15.33:0/4242725330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170c1a52f0 con 0x7f170c101eb0 2024-04-02T13:37:42.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.898+0000 7f1709ffb700 1 -- 172.21.15.33:0/4242725330 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f400fa60 con 0x7f170c101eb0 2024-04-02T13:37:42.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.898+0000 7f1712e59700 1 -- 172.21.15.33:0/4242725330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f170c1a5800 con 0x7f170c101eb0 2024-04-02T13:37:42.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.899+0000 7f1712e59700 1 -- 172.21.15.33:0/4242725330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f170c0692b0 con 0x7f170c101eb0 2024-04-02T13:37:42.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.900+0000 7f1709ffb700 1 -- 172.21.15.33:0/4242725330 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f16f40051d0 con 0x7f170c101eb0 2024-04-02T13:37:42.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.901+0000 7f1709ffb700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f16f80a3d50 0x7f16f80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:42.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.901+0000 7f1710bf5700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f16f80a3d50 0x7f16f80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:42.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.901+0000 7f1709ffb700 1 -- 172.21.15.33:0/4242725330 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f16f40d0640 con 0x7f170c101eb0 2024-04-02T13:37:42.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.901+0000 7f1710bf5700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f16f80a3d50 0x7f16f80a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009550 tx=0x7f16fc0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:42.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:42.905+0000 7f1709ffb700 1 -- 172.21.15.33:0/4242725330 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f16f4020020 con 0x7f170c101eb0 2024-04-02T13:37:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.114+0000 7f1712e59700 1 -- 172.21.15.33:0/4242725330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f170c02def0 con 0x7f170c101eb0 2024-04-02T13:37:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.114+0000 7f1709ffb700 1 -- 172.21.15.33:0/4242725330 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f16f409cb60 con 0x7f170c101eb0 2024-04-02T13:37:43.118 INFO:teuthology.orchestra.run.smithi033.stdout:111669149766 2024-04-02T13:37:43.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.116+0000 7f1712e59700 1 -- 172.21.15.33:0/4242725330 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f16f80a3d50 msgr2=0x7f16f80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:43.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.116+0000 7f1712e59700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f16f80a3d50 0x7f16f80a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009550 tx=0x7f16fc0091b0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.116+0000 7f1712e59700 1 -- 172.21.15.33:0/4242725330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c101eb0 msgr2=0x7f170c19a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.116+0000 7f1712e59700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c101eb0 0x7f170c19a040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f16f400d980 tx=0x7f16f400dc90 comp rx=0 tx=0).stop 2024-04-02T13:37:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.116+0000 7f1712e59700 1 -- 172.21.15.33:0/4242725330 shutdown_connections 2024-04-02T13:37:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.116+0000 7f1712e59700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f16f80a3d50 0x7f16f80a6210 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.116+0000 7f1712e59700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f170c107500 0x7f170c1a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.116+0000 7f1712e59700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f170c1014f0 0x7f170c199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.116+0000 7f1712e59700 1 --2- 172.21.15.33:0/4242725330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c101eb0 0x7f170c19a040 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.116+0000 7f1712e59700 1 -- 172.21.15.33:0/4242725330 >> 172.21.15.33:0/4242725330 conn(0x7f170c078420 msgr2=0x7f170c0fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.116+0000 7f1712e59700 1 -- 172.21.15.33:0/4242725330 shutdown_connections 2024-04-02T13:37:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.116+0000 7f1712e59700 1 -- 172.21.15.33:0/4242725330 wait complete. 2024-04-02T13:37:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.351+0000 7fbedeb30700 1 -- 172.21.15.33:0/873108229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed8108660 msgr2=0x7fbed8108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:43.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.351+0000 7fbedeb30700 1 --2- 172.21.15.33:0/873108229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed8108660 0x7fbed8108a40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009990 tx=0x7fbecc009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.351+0000 7fbedeb30700 1 -- 172.21.15.33:0/873108229 shutdown_connections 2024-04-02T13:37:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.351+0000 7fbedeb30700 1 --2- 172.21.15.33:0/873108229 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbed8102f90 0x7fbed810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.351+0000 7fbedeb30700 1 --2- 172.21.15.33:0/873108229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed8108660 0x7fbed8108a40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.351+0000 7fbedeb30700 1 --2- 172.21.15.33:0/873108229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbed81025d0 0x7fbed8102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.351+0000 7fbedeb30700 1 -- 172.21.15.33:0/873108229 >> 172.21.15.33:0/873108229 conn(0x7fbed80fe0a0 msgr2=0x7fbed81004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.351+0000 7fbedeb30700 1 -- 172.21.15.33:0/873108229 shutdown_connections 2024-04-02T13:37:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.351+0000 7fbedeb30700 1 -- 172.21.15.33:0/873108229 wait complete. 2024-04-02T13:37:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.351+0000 7fbedeb30700 1 Processor -- start 2024-04-02T13:37:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.351+0000 7fbedeb30700 1 -- start start 2024-04-02T13:37:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedeb30700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbed81025d0 0x7fbed8198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedeb30700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbed8102f90 0x7fbed8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedeb30700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed8108660 0x7fbed819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedeb30700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbed8112930 con 0x7fbed8108660 2024-04-02T13:37:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedeb30700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81127b0 con 0x7fbed81025d0 2024-04-02T13:37:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedeb30700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbed8112ab0 con 0x7fbed8102f90 2024-04-02T13:37:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedd0cd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed8108660 0x7fbed819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedd0cd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed8108660 0x7fbed819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37446/0 (socket says 172.21.15.33:37446) 2024-04-02T13:37:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedd0cd700 1 -- 172.21.15.33:0/897355875 learned_addr learned my addr 172.21.15.33:0/897355875 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:43.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedc8cc700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbed81025d0 0x7fbed8198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbed7fff700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbed8102f90 0x7fbed8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedd0cd700 1 -- 172.21.15.33:0/897355875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbed8102f90 msgr2=0x7fbed8199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedd0cd700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbed8102f90 0x7fbed8199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedd0cd700 1 -- 172.21.15.33:0/897355875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbed81025d0 msgr2=0x7fbed8198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedd0cd700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbed81025d0 0x7fbed8198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedd0cd700 1 -- 172.21.15.33:0/897355875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed819ddf0 con 0x7fbed8108660 2024-04-02T13:37:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbedd0cd700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed8108660 0x7fbed819d6b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbed000b970 tx=0x7fbed000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.352+0000 7fbed5ffb700 1 -- 172.21.15.33:0/897355875 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0019070 con 0x7fbed8108660 2024-04-02T13:37:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.353+0000 7fbedeb30700 1 -- 172.21.15.33:0/897355875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed819e0e0 con 0x7fbed8108660 2024-04-02T13:37:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.353+0000 7fbed5ffb700 1 -- 172.21.15.33:0/897355875 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbed000ce00 con 0x7fbed8108660 2024-04-02T13:37:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.353+0000 7fbed5ffb700 1 -- 172.21.15.33:0/897355875 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0007370 con 0x7fbed8108660 2024-04-02T13:37:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.353+0000 7fbedeb30700 1 -- 172.21.15.33:0/897355875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed81a5960 con 0x7fbed8108660 2024-04-02T13:37:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.354+0000 7fbedeb30700 1 -- 172.21.15.33:0/897355875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed80692b0 con 0x7fbed8108660 2024-04-02T13:37:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.355+0000 7fbed5ffb700 1 -- 172.21.15.33:0/897355875 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbed001c070 con 0x7fbed8108660 2024-04-02T13:37:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.355+0000 7fbed5ffb700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbec80a3cd0 0x7fbec80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.355+0000 7fbed5ffb700 1 -- 172.21.15.33:0/897355875 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbed00cfb70 con 0x7fbed8108660 2024-04-02T13:37:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.355+0000 7fbedc8cc700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbec80a3cd0 0x7fbec80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.356+0000 7fbedc8cc700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbec80a3cd0 0x7fbec80a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00b5c0 tx=0x7fbecc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:43.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.360+0000 7fbed5ffb700 1 -- 172.21.15.33:0/897355875 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbed009c1a0 con 0x7fbed8108660 2024-04-02T13:37:43.444 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:43 smithi033 ceph-mon[32390]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:43.444 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:43 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3440633872' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T13:37:43.444 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:43 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4242725330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T13:37:43.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:43 smithi145 ceph-mon[35646]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:43.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:43 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3440633872' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T13:37:43.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:43 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4242725330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T13:37:43.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:43 smithi050 ceph-mon[35524]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:43.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:43 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3440633872' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T13:37:43.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:43 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4242725330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T13:37:43.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.569+0000 7fbedeb30700 1 -- 172.21.15.33:0/897355875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbed819a140 con 0x7fbed8108660 2024-04-02T13:37:43.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.570+0000 7fbed5ffb700 1 -- 172.21.15.33:0/897355875 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbed009baa0 con 0x7fbed8108660 2024-04-02T13:37:43.573 INFO:teuthology.orchestra.run.smithi033.stdout:77309411404 2024-04-02T13:37:43.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.572+0000 7fbedeb30700 1 -- 172.21.15.33:0/897355875 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbec80a3cd0 msgr2=0x7fbec80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:43.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.572+0000 7fbedeb30700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbec80a3cd0 0x7fbec80a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00b5c0 tx=0x7fbecc005960 comp rx=0 tx=0).stop 2024-04-02T13:37:43.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.572+0000 7fbedeb30700 1 -- 172.21.15.33:0/897355875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed8108660 msgr2=0x7fbed819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:43.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.572+0000 7fbedeb30700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed8108660 0x7fbed819d6b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbed000b970 tx=0x7fbed000bc80 comp rx=0 tx=0).stop 2024-04-02T13:37:43.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.572+0000 7fbedeb30700 1 -- 172.21.15.33:0/897355875 shutdown_connections 2024-04-02T13:37:43.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.572+0000 7fbedeb30700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fbec80a3cd0 0x7fbec80a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.572+0000 7fbedeb30700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed8108660 0x7fbed819d6b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.572+0000 7fbedeb30700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbed81025d0 0x7fbed8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.572+0000 7fbedeb30700 1 --2- 172.21.15.33:0/897355875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbed8102f90 0x7fbed8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:43.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.572+0000 7fbedeb30700 1 -- 172.21.15.33:0/897355875 >> 172.21.15.33:0/897355875 conn(0x7fbed80fe0a0 msgr2=0x7fbed80ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:43.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.572+0000 7fbedeb30700 1 -- 172.21.15.33:0/897355875 shutdown_connections 2024-04-02T13:37:43.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:43.572+0000 7fbedeb30700 1 -- 172.21.15.33:0/897355875 wait complete. 2024-04-02T13:37:44.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:44 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/897355875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T13:37:44.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:44 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/897355875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T13:37:44.487 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612636 got 313532612642 for osd.10 2024-04-02T13:37:44.487 DEBUG:teuthology.parallel:result is None 2024-04-02T13:37:44.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:44 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/897355875' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T13:37:45.333 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:45 smithi033 ceph-mon[32390]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:37:45.371 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411398 got 77309411404 for osd.2 2024-04-02T13:37:45.372 DEBUG:teuthology.parallel:result is None 2024-04-02T13:37:45.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:45 smithi145 ceph-mon[35646]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:37:45.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:45 smithi050 ceph-mon[35524]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:37:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:46.672 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888124 got 146028888129 for osd.4 2024-04-02T13:37:46.673 DEBUG:teuthology.parallel:result is None 2024-04-02T13:37:47.477 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:47 smithi033 ceph-mon[32390]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:37:47.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:47 smithi145 ceph-mon[35646]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:37:47.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:47 smithi050 ceph-mon[35524]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:37:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.584+0000 7f83d8088700 1 -- 172.21.15.33:0/4126764043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d0101180 msgr2=0x7f83d0101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.584+0000 7f83d8088700 1 --2- 172.21.15.33:0/4126764043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d0101180 0x7f83d0101560 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0099c0 tx=0x7f83cc009cd0 comp rx=0 tx=0).stop 2024-04-02T13:37:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.584+0000 7f83d8088700 1 -- 172.21.15.33:0/4126764043 shutdown_connections 2024-04-02T13:37:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.584+0000 7f83d8088700 1 --2- 172.21.15.33:0/4126764043 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d01079a0 0x7f83d010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.584+0000 7f83d8088700 1 --2- 172.21.15.33:0/4126764043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d0101180 0x7f83d0101560 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.584+0000 7f83d8088700 1 --2- 172.21.15.33:0/4126764043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83d0101b30 0x7f83d0107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.584+0000 7f83d8088700 1 -- 172.21.15.33:0/4126764043 >> 172.21.15.33:0/4126764043 conn(0x7f83d0076820 msgr2=0x7f83d0076c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.584+0000 7f83d8088700 1 -- 172.21.15.33:0/4126764043 shutdown_connections 2024-04-02T13:37:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.584+0000 7f83d8088700 1 -- 172.21.15.33:0/4126764043 wait complete. 2024-04-02T13:37:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.584+0000 7f83d8088700 1 Processor -- start 2024-04-02T13:37:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.584+0000 7f83d8088700 1 -- start start 2024-04-02T13:37:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d8088700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d0101180 0x7f83d0198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d8088700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83d0101b30 0x7f83d0199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d8088700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d01079a0 0x7f83d019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d8088700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01129a0 con 0x7f83d01079a0 2024-04-02T13:37:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d8088700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0112820 con 0x7f83d0101b30 2024-04-02T13:37:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d8088700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0112b20 con 0x7f83d0101180 2024-04-02T13:37:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d6625700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d01079a0 0x7f83d019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d5e24700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d0101180 0x7f83d0198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d6625700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d01079a0 0x7f83d019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44504/0 (socket says 172.21.15.33:44504) 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d5e24700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d0101180 0x7f83d0198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:47684/0 (socket says 172.21.15.33:47684) 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d6625700 1 -- 172.21.15.33:0/572581614 learned_addr learned my addr 172.21.15.33:0/572581614 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d5623700 1 --2- 172.21.15.33:0/572581614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83d0101b30 0x7f83d0199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d6625700 1 -- 172.21.15.33:0/572581614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d0101180 msgr2=0x7f83d0198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d6625700 1 --2- 172.21.15.33:0/572581614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d0101180 0x7f83d0198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d6625700 1 -- 172.21.15.33:0/572581614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83d0101b30 msgr2=0x7f83d0199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d6625700 1 --2- 172.21.15.33:0/572581614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83d0101b30 0x7f83d0199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d6625700 1 -- 172.21.15.33:0/572581614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d019de00 con 0x7f83d01079a0 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d6625700 1 --2- 172.21.15.33:0/572581614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d01079a0 0x7f83d019d6c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f83c8007ee0 tx=0x7f83c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83c6ffd700 1 -- 172.21.15.33:0/572581614 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c800dbd0 con 0x7f83d01079a0 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83c6ffd700 1 -- 172.21.15.33:0/572581614 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f83c800dd70 con 0x7f83d01079a0 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83d8088700 1 -- 172.21.15.33:0/572581614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d019e0f0 con 0x7f83d01079a0 2024-04-02T13:37:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.585+0000 7f83c6ffd700 1 -- 172.21.15.33:0/572581614 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c80059b0 con 0x7f83d01079a0 2024-04-02T13:37:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.586+0000 7f83d8088700 1 -- 172.21.15.33:0/572581614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d01a5970 con 0x7f83d01079a0 2024-04-02T13:37:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.586+0000 7f83d8088700 1 -- 172.21.15.33:0/572581614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d00692b0 con 0x7f83d01079a0 2024-04-02T13:37:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.589+0000 7f83c6ffd700 1 -- 172.21.15.33:0/572581614 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f83c801a020 con 0x7f83d01079a0 2024-04-02T13:37:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.589+0000 7f83c6ffd700 1 --2- 172.21.15.33:0/572581614 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f83bc0a3d50 0x7f83bc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.589+0000 7f83c6ffd700 1 -- 172.21.15.33:0/572581614 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f83c80d0650 con 0x7f83d01079a0 2024-04-02T13:37:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.589+0000 7f83d5e24700 1 --2- 172.21.15.33:0/572581614 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f83bc0a3d50 0x7f83bc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.589+0000 7f83d5e24700 1 --2- 172.21.15.33:0/572581614 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f83bc0a3d50 0x7f83bc0a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009f50 tx=0x7f83cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.593+0000 7f83c6ffd700 1 -- 172.21.15.33:0/572581614 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f83c8010070 con 0x7f83d01079a0 2024-04-02T13:37:47.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.806+0000 7f83d8088700 1 -- 172.21.15.33:0/572581614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f83d002dd30 con 0x7f83d01079a0 2024-04-02T13:37:47.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.806+0000 7f83c6ffd700 1 -- 172.21.15.33:0/572581614 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f83c809cb70 con 0x7f83d01079a0 2024-04-02T13:37:47.810 INFO:teuthology.orchestra.run.smithi033.stdout:347892351006 2024-04-02T13:37:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.808+0000 7f83d8088700 1 -- 172.21.15.33:0/572581614 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f83bc0a3d50 msgr2=0x7f83bc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.808+0000 7f83d8088700 1 --2- 172.21.15.33:0/572581614 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f83bc0a3d50 0x7f83bc0a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009f50 tx=0x7f83cc005fb0 comp rx=0 tx=0).stop 2024-04-02T13:37:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.808+0000 7f83d8088700 1 -- 172.21.15.33:0/572581614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d01079a0 msgr2=0x7f83d019d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.808+0000 7f83d8088700 1 --2- 172.21.15.33:0/572581614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d01079a0 0x7f83d019d6c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f83c8007ee0 tx=0x7f83c800d3b0 comp rx=0 tx=0).stop 2024-04-02T13:37:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.808+0000 7f83d8088700 1 -- 172.21.15.33:0/572581614 shutdown_connections 2024-04-02T13:37:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.808+0000 7f83d8088700 1 --2- 172.21.15.33:0/572581614 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f83bc0a3d50 0x7f83bc0a6210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.808+0000 7f83d8088700 1 --2- 172.21.15.33:0/572581614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d01079a0 0x7f83d019d6c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.808+0000 7f83d8088700 1 --2- 172.21.15.33:0/572581614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83d0101180 0x7f83d0198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.808+0000 7f83d8088700 1 --2- 172.21.15.33:0/572581614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83d0101b30 0x7f83d0199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.808+0000 7f83d8088700 1 -- 172.21.15.33:0/572581614 >> 172.21.15.33:0/572581614 conn(0x7f83d0076820 msgr2=0x7f83d00fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:47.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.808+0000 7f83d8088700 1 -- 172.21.15.33:0/572581614 shutdown_connections 2024-04-02T13:37:47.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:47.808+0000 7f83d8088700 1 -- 172.21.15.33:0/572581614 wait complete. 2024-04-02T13:37:48.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:48 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/572581614' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T13:37:48.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:48 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/572581614' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T13:37:48.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:48 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/572581614' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T13:37:49.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:49 smithi145 ceph-mon[35646]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:49.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:49 smithi050 ceph-mon[35524]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:49.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:49 smithi033 ceph-mon[32390]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:49.877 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299436 got 223338299442 for osd.7 2024-04-02T13:37:49.877 DEBUG:teuthology.parallel:result is None 2024-04-02T13:37:50.351 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149761 got 111669149766 for osd.3 2024-04-02T13:37:50.351 DEBUG:teuthology.parallel:result is None 2024-04-02T13:37:50.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.544+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4138165459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5781086d0 msgr2=0x7fa578108ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:50.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.544+0000 7fa57dfc4700 1 --2- 172.21.15.33:0/4138165459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5781086d0 0x7fa578108ab0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa568009990 tx=0x7fa568009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:50.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.544+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4138165459 shutdown_connections 2024-04-02T13:37:50.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.544+0000 7fa57dfc4700 1 --2- 172.21.15.33:0/4138165459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa578103000 0x7fa57810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:50.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.544+0000 7fa57dfc4700 1 --2- 172.21.15.33:0/4138165459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5781086d0 0x7fa578108ab0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:50.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.544+0000 7fa57dfc4700 1 --2- 172.21.15.33:0/4138165459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa578102640 0x7fa578102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:50.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.544+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4138165459 >> 172.21.15.33:0/4138165459 conn(0x7fa578079720 msgr2=0x7fa578079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:50.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.544+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4138165459 shutdown_connections 2024-04-02T13:37:50.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.544+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4138165459 wait complete. 2024-04-02T13:37:50.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.545+0000 7fa57dfc4700 1 Processor -- start 2024-04-02T13:37:50.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.545+0000 7fa57dfc4700 1 -- start start 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa57dfc4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa578102640 0x7fa578198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa57dfc4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa578103000 0x7fa578199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa57dfc4700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5781086d0 0x7fa57819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa57dfc4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa578112a10 con 0x7fa578103000 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa57dfc4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa578112890 con 0x7fa578102640 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa57dfc4700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa578112b90 con 0x7fa5781086d0 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa576ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa578103000 0x7fa578199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa576ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa578103000 0x7fa578199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44514/0 (socket says 172.21.15.33:44514) 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa576ffd700 1 -- 172.21.15.33:0/4030373040 learned_addr learned my addr 172.21.15.33:0/4030373040 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa5777fe700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa578102640 0x7fa578198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa577fff700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5781086d0 0x7fa57819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa576ffd700 1 -- 172.21.15.33:0/4030373040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5781086d0 msgr2=0x7fa57819d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa576ffd700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5781086d0 0x7fa57819d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa576ffd700 1 -- 172.21.15.33:0/4030373040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa578102640 msgr2=0x7fa578198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa576ffd700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa578102640 0x7fa578198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa576ffd700 1 -- 172.21.15.33:0/4030373040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57819df50 con 0x7fa578103000 2024-04-02T13:37:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa5777fe700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa578102640 0x7fa578198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa577fff700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5781086d0 0x7fa57819d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa576ffd700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa578103000 0x7fa578199480 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00b530 tx=0x7fa56c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.546+0000 7fa574ff9700 1 -- 172.21.15.33:0/4030373040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c015020 con 0x7fa578103000 2024-04-02T13:37:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.547+0000 7fa574ff9700 1 -- 172.21.15.33:0/4030373040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa56c011d00 con 0x7fa578103000 2024-04-02T13:37:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.547+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4030373040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57819e240 con 0x7fa578103000 2024-04-02T13:37:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.547+0000 7fa574ff9700 1 -- 172.21.15.33:0/4030373040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c010e40 con 0x7fa578103000 2024-04-02T13:37:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.547+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4030373040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5781a5ac0 con 0x7fa578103000 2024-04-02T13:37:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.548+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4030373040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5780692b0 con 0x7fa578103000 2024-04-02T13:37:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.549+0000 7fa574ff9700 1 -- 172.21.15.33:0/4030373040 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa56c018020 con 0x7fa578103000 2024-04-02T13:37:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.549+0000 7fa574ff9700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa5600a3d50 0x7fa5600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.549+0000 7fa5777fe700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa5600a3d50 0x7fa5600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.549+0000 7fa574ff9700 1 -- 172.21.15.33:0/4030373040 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa56c0cff90 con 0x7fa578103000 2024-04-02T13:37:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.550+0000 7fa5777fe700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa5600a3d50 0x7fa5600a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa568006010 tx=0x7fa56800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:50.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.554+0000 7fa574ff9700 1 -- 172.21.15.33:0/4030373040 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa56c01f020 con 0x7fa578103000 2024-04-02T13:37:50.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.766+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4030373040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa57819a2a0 con 0x7fa578103000 2024-04-02T13:37:50.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.766+0000 7fa574ff9700 1 -- 172.21.15.33:0/4030373040 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa56c010380 con 0x7fa578103000 2024-04-02T13:37:50.769 INFO:teuthology.orchestra.run.smithi033.stdout:34359738455 2024-04-02T13:37:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.768+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4030373040 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa5600a3d50 msgr2=0x7fa5600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.768+0000 7fa57dfc4700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa5600a3d50 0x7fa5600a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa568006010 tx=0x7fa56800b540 comp rx=0 tx=0).stop 2024-04-02T13:37:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.768+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4030373040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa578103000 msgr2=0x7fa578199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.768+0000 7fa57dfc4700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa578103000 0x7fa578199480 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00b530 tx=0x7fa56c00b8f0 comp rx=0 tx=0).stop 2024-04-02T13:37:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.768+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4030373040 shutdown_connections 2024-04-02T13:37:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.768+0000 7fa57dfc4700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa5600a3d50 0x7fa5600a6210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.768+0000 7fa57dfc4700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5781086d0 0x7fa57819d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.768+0000 7fa57dfc4700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa578102640 0x7fa578198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.768+0000 7fa57dfc4700 1 --2- 172.21.15.33:0/4030373040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa578103000 0x7fa578199480 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:50.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.768+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4030373040 >> 172.21.15.33:0/4030373040 conn(0x7fa578079720 msgr2=0x7fa5781013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:50.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.769+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4030373040 shutdown_connections 2024-04-02T13:37:50.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:50.769+0000 7fa57dfc4700 1 -- 172.21.15.33:0/4030373040 wait complete. 2024-04-02T13:37:51.481 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:51 smithi145 ceph-mon[35646]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-02T13:37:51.481 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:51 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4030373040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T13:37:51.482 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:51.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:51 smithi033 ceph-mon[32390]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-02T13:37:51.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:51 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4030373040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T13:37:51.488 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738450 got 34359738455 for osd.0 2024-04-02T13:37:51.488 DEBUG:teuthology.parallel:result is None 2024-04-02T13:37:51.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:51 smithi050 ceph-mon[35524]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-02T13:37:51.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:51 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4030373040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T13:37:51.824 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351001 got 347892351006 for osd.11 2024-04-02T13:37:51.824 DEBUG:teuthology.parallel:result is None 2024-04-02T13:37:52.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.454+0000 7f542cf14700 1 -- 172.21.15.33:0/3111583348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5428101cf0 msgr2=0x7f5428102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:52.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.454+0000 7f542cf14700 1 --2- 172.21.15.33:0/3111583348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5428101cf0 0x7f5428102170 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f54100099c0 tx=0x7f5410009cd0 comp rx=0 tx=0).stop 2024-04-02T13:37:52.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.455+0000 7f542cf14700 1 -- 172.21.15.33:0/3111583348 shutdown_connections 2024-04-02T13:37:52.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.455+0000 7f542cf14700 1 --2- 172.21.15.33:0/3111583348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54281026b0 0x7f542810e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:52.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.455+0000 7f542cf14700 1 --2- 172.21.15.33:0/3111583348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5428107600 0x7f54281079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:52.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.455+0000 7f542cf14700 1 --2- 172.21.15.33:0/3111583348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5428101cf0 0x7f5428102170 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:52.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.455+0000 7f542cf14700 1 -- 172.21.15.33:0/3111583348 >> 172.21.15.33:0/3111583348 conn(0x7f5428077af0 msgr2=0x7f54280fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:52.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.455+0000 7f542cf14700 1 -- 172.21.15.33:0/3111583348 shutdown_connections 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.455+0000 7f542cf14700 1 -- 172.21.15.33:0/3111583348 wait complete. 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.455+0000 7f542cf14700 1 Processor -- start 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542cf14700 1 -- start start 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542cf14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5428101cf0 0x7f5428199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542cf14700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54281026b0 0x7f542819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542cf14700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5428107600 0x7f54281a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542cf14700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5428111830 con 0x7f5428101cf0 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542cf14700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f54281116b0 con 0x7f5428107600 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542cf14700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f54281119b0 con 0x7f54281026b0 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5428101cf0 0x7f5428199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5428101cf0 0x7f5428199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44544/0 (socket says 172.21.15.33:44544) 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542659c700 1 -- 172.21.15.33:0/4108142552 learned_addr learned my addr 172.21.15.33:0/4108142552 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f5425d9b700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54281026b0 0x7f542819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:52.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f5426d9d700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5428107600 0x7f54281a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542659c700 1 -- 172.21.15.33:0/4108142552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54281026b0 msgr2=0x7f542819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542659c700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54281026b0 0x7f542819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542659c700 1 -- 172.21.15.33:0/4108142552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5428107600 msgr2=0x7f54281a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542659c700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5428107600 0x7f54281a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.456+0000 7f542659c700 1 -- 172.21.15.33:0/4108142552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54281a50f0 con 0x7f5428101cf0 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.457+0000 7f542659c700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5428101cf0 0x7f5428199b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f541800d980 tx=0x7f541800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.457+0000 7f541f7fe700 1 -- 172.21.15.33:0/4108142552 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418017070 con 0x7f5428101cf0 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.457+0000 7f542cf14700 1 -- 172.21.15.33:0/4108142552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54281a5320 con 0x7f5428101cf0 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.457+0000 7f541f7fe700 1 -- 172.21.15.33:0/4108142552 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5418005020 con 0x7f5428101cf0 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.457+0000 7f542cf14700 1 -- 172.21.15.33:0/4108142552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54281a5830 con 0x7f5428101cf0 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.457+0000 7f541f7fe700 1 -- 172.21.15.33:0/4108142552 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418009b70 con 0x7f5428101cf0 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.458+0000 7f542cf14700 1 -- 172.21.15.33:0/4108142552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5428109a90 con 0x7f5428101cf0 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.459+0000 7f541f7fe700 1 -- 172.21.15.33:0/4108142552 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5418009d10 con 0x7f5428101cf0 2024-04-02T13:37:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.460+0000 7f541f7fe700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54140a3cd0 0x7f54140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.460+0000 7f5425d9b700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54140a3cd0 0x7f54140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.460+0000 7f541f7fe700 1 -- 172.21.15.33:0/4108142552 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f54180d0cf0 con 0x7f5428101cf0 2024-04-02T13:37:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.460+0000 7f5425d9b700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54140a3cd0 0x7f54140a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5428101750 tx=0x7f5410005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.467+0000 7f541f7fe700 1 -- 172.21.15.33:0/4108142552 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5418020020 con 0x7f5428101cf0 2024-04-02T13:37:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.679+0000 7f542cf14700 1 -- 172.21.15.33:0/4108142552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f54281011f0 con 0x7f5428101cf0 2024-04-02T13:37:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.679+0000 7f541f7fe700 1 -- 172.21.15.33:0/4108142552 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f541809d210 con 0x7f5428101cf0 2024-04-02T13:37:52.691 INFO:teuthology.orchestra.run.smithi033.stdout:257698037807 2024-04-02T13:37:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.681+0000 7f542cf14700 1 -- 172.21.15.33:0/4108142552 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54140a3cd0 msgr2=0x7f54140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.681+0000 7f542cf14700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54140a3cd0 0x7f54140a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5428101750 tx=0x7f5410005fb0 comp rx=0 tx=0).stop 2024-04-02T13:37:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.681+0000 7f542cf14700 1 -- 172.21.15.33:0/4108142552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5428101cf0 msgr2=0x7f5428199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.681+0000 7f542cf14700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5428101cf0 0x7f5428199b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f541800d980 tx=0x7f541800dc90 comp rx=0 tx=0).stop 2024-04-02T13:37:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.682+0000 7f542cf14700 1 -- 172.21.15.33:0/4108142552 shutdown_connections 2024-04-02T13:37:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.682+0000 7f542cf14700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54140a3cd0 0x7f54140a6190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.682+0000 7f542cf14700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5428107600 0x7f54281a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.682+0000 7f542cf14700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5428101cf0 0x7f5428199b30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.682+0000 7f542cf14700 1 --2- 172.21.15.33:0/4108142552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54281026b0 0x7f542819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.682+0000 7f542cf14700 1 -- 172.21.15.33:0/4108142552 >> 172.21.15.33:0/4108142552 conn(0x7f5428077af0 msgr2=0x7f54280ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.682+0000 7f542cf14700 1 -- 172.21.15.33:0/4108142552 shutdown_connections 2024-04-02T13:37:52.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:52.682+0000 7f542cf14700 1 -- 172.21.15.33:0/4108142552 wait complete. 2024-04-02T13:37:52.695 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:53.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:53 smithi050 ceph-mon[35524]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:53.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:53 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4108142552' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T13:37:53.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:53 smithi033 ceph-mon[32390]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:53.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:53 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4108142552' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T13:37:53.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:53 smithi145 ceph-mon[35646]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:53.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:53 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4108142552' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T13:37:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.031+0000 7f8997266700 1 -- 172.21.15.33:0/243323358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8990102640 msgr2=0x7f8990102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.031+0000 7f8997266700 1 --2- 172.21.15.33:0/243323358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8990102640 0x7f8990102ac0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8980009990 tx=0x7f8980009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.031+0000 7f8997266700 1 -- 172.21.15.33:0/243323358 shutdown_connections 2024-04-02T13:37:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.031+0000 7f8997266700 1 --2- 172.21.15.33:0/243323358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8990103000 0x7f899010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.031+0000 7f8997266700 1 --2- 172.21.15.33:0/243323358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89901086d0 0x7f8990108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.031+0000 7f8997266700 1 --2- 172.21.15.33:0/243323358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8990102640 0x7f8990102ac0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.031+0000 7f8997266700 1 -- 172.21.15.33:0/243323358 >> 172.21.15.33:0/243323358 conn(0x7f8990079720 msgr2=0x7f8990079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.031+0000 7f8997266700 1 -- 172.21.15.33:0/243323358 shutdown_connections 2024-04-02T13:37:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.031+0000 7f8997266700 1 -- 172.21.15.33:0/243323358 wait complete. 2024-04-02T13:37:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8997266700 1 Processor -- start 2024-04-02T13:37:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8997266700 1 -- start start 2024-04-02T13:37:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8997266700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8990102640 0x7f89900748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8997266700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8990103000 0x7f8990074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8997266700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89901086d0 0x7f8990075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8997266700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8990112950 con 0x7f8990102640 2024-04-02T13:37:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8997266700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f89901127d0 con 0x7f89901086d0 2024-04-02T13:37:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8997266700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8990112ad0 con 0x7f8990103000 2024-04-02T13:37:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8995002700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8990102640 0x7f89900748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8995002700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8990102640 0x7f89900748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51698/0 (socket says 172.21.15.33:51698) 2024-04-02T13:37:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8995002700 1 -- 172.21.15.33:0/3665207008 learned_addr learned my addr 172.21.15.33:0/3665207008 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8995803700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89901086d0 0x7f8990075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8994801700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8990103000 0x7f8990074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8995002700 1 -- 172.21.15.33:0/3665207008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8990103000 msgr2=0x7f8990074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8995002700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8990103000 0x7f8990074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8995002700 1 -- 172.21.15.33:0/3665207008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89901086d0 msgr2=0x7f8990075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:55.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8995002700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89901086d0 0x7f8990075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8995002700 1 -- 172.21.15.33:0/3665207008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89901a7390 con 0x7f8990102640 2024-04-02T13:37:55.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.032+0000 7f8994801700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8990103000 0x7f8990074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:55.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.033+0000 7f8995002700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8990102640 0x7f89900748f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f898c00d950 tx=0x7f898c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:55.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.033+0000 7f89867fc700 1 -- 172.21.15.33:0/3665207008 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c016070 con 0x7f8990102640 2024-04-02T13:37:55.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.033+0000 7f8997266700 1 -- 172.21.15.33:0/3665207008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89901a7680 con 0x7f8990102640 2024-04-02T13:37:55.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.033+0000 7f89867fc700 1 -- 172.21.15.33:0/3665207008 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f898c009e10 con 0x7f8990102640 2024-04-02T13:37:55.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.033+0000 7f89867fc700 1 -- 172.21.15.33:0/3665207008 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c00f620 con 0x7f8990102640 2024-04-02T13:37:55.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.033+0000 7f8997266700 1 -- 172.21.15.33:0/3665207008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89901a7bc0 con 0x7f8990102640 2024-04-02T13:37:55.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.034+0000 7f8997266700 1 -- 172.21.15.33:0/3665207008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89900692b0 con 0x7f8990102640 2024-04-02T13:37:55.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.036+0000 7f89867fc700 1 -- 172.21.15.33:0/3665207008 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f898c019070 con 0x7f8990102640 2024-04-02T13:37:55.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.036+0000 7f89867fc700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f897c0a3cd0 0x7f897c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:55.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.036+0000 7f89867fc700 1 -- 172.21.15.33:0/3665207008 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f898c0d0490 con 0x7f8990102640 2024-04-02T13:37:55.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.036+0000 7f8994801700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f897c0a3cd0 0x7f897c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:55.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.037+0000 7f8994801700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f897c0a3cd0 0x7f897c0a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8980006010 tx=0x7f898000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.040+0000 7f89867fc700 1 -- 172.21.15.33:0/3665207008 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f898c09c9b0 con 0x7f8990102640 2024-04-02T13:37:55.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.253+0000 7f8997266700 1 -- 172.21.15.33:0/3665207008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f899002dd30 con 0x7f8990102640 2024-04-02T13:37:55.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.253+0000 7f89867fc700 1 -- 172.21.15.33:0/3665207008 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f898c096e20 con 0x7f8990102640 2024-04-02T13:37:55.256 INFO:teuthology.orchestra.run.smithi033.stdout:292057776170 2024-04-02T13:37:55.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.255+0000 7f8997266700 1 -- 172.21.15.33:0/3665207008 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f897c0a3cd0 msgr2=0x7f897c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:55.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.255+0000 7f8997266700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f897c0a3cd0 0x7f897c0a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8980006010 tx=0x7f898000b540 comp rx=0 tx=0).stop 2024-04-02T13:37:55.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.255+0000 7f8997266700 1 -- 172.21.15.33:0/3665207008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8990102640 msgr2=0x7f89900748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:55.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.255+0000 7f8997266700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8990102640 0x7f89900748f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f898c00d950 tx=0x7f898c00dc60 comp rx=0 tx=0).stop 2024-04-02T13:37:55.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.255+0000 7f8997266700 1 -- 172.21.15.33:0/3665207008 shutdown_connections 2024-04-02T13:37:55.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.255+0000 7f8997266700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f897c0a3cd0 0x7f897c0a6190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.255+0000 7f8997266700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89901086d0 0x7f8990075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.255+0000 7f8997266700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8990102640 0x7f89900748f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.255+0000 7f8997266700 1 --2- 172.21.15.33:0/3665207008 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8990103000 0x7f8990074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.255+0000 7f8997266700 1 -- 172.21.15.33:0/3665207008 >> 172.21.15.33:0/3665207008 conn(0x7f8990079720 msgr2=0x7f8990101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:55.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.255+0000 7f8997266700 1 -- 172.21.15.33:0/3665207008 shutdown_connections 2024-04-02T13:37:55.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.256+0000 7f8997266700 1 -- 172.21.15.33:0/3665207008 wait complete. 2024-04-02T13:37:55.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:55 smithi050 ceph-mon[35524]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:37:55.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:55 smithi033 ceph-mon[32390]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:37:55.530 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:55 smithi145 ceph-mon[35646]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:37:55.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.515+0000 7fb1d64ab700 1 -- 172.21.15.33:0/359660122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1d0108650 msgr2=0x7fb1d0108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:55.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.515+0000 7fb1d64ab700 1 --2- 172.21.15.33:0/359660122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1d0108650 0x7fb1d0108a30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009990 tx=0x7fb1c0009ca0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.515+0000 7fb1d64ab700 1 -- 172.21.15.33:0/359660122 shutdown_connections 2024-04-02T13:37:55.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.515+0000 7fb1d64ab700 1 --2- 172.21.15.33:0/359660122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d006ac60 0x7fb1d010f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.515+0000 7fb1d64ab700 1 --2- 172.21.15.33:0/359660122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1d0108650 0x7fb1d0108a30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.515+0000 7fb1d64ab700 1 --2- 172.21.15.33:0/359660122 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1d006a2a0 0x7fb1d006a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.515+0000 7fb1d64ab700 1 -- 172.21.15.33:0/359660122 >> 172.21.15.33:0/359660122 conn(0x7fb1d0078420 msgr2=0x7fb1d0078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:55.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.515+0000 7fb1d64ab700 1 -- 172.21.15.33:0/359660122 shutdown_connections 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.515+0000 7fb1d64ab700 1 -- 172.21.15.33:0/359660122 wait complete. 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d64ab700 1 Processor -- start 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d64ab700 1 -- start start 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d64ab700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1d006a2a0 0x7fb1d0198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d64ab700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d006ac60 0x7fb1d01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d64ab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1d0108650 0x7fb1d019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d64ab700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d0112920 con 0x7fb1d0108650 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d64ab700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01127a0 con 0x7fb1d006ac60 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d64ab700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d0112aa0 con 0x7fb1d006a2a0 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d4a48700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1d0108650 0x7fb1d019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d4a48700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1d0108650 0x7fb1d019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51714/0 (socket says 172.21.15.33:51714) 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d4a48700 1 -- 172.21.15.33:0/3931791371 learned_addr learned my addr 172.21.15.33:0/3931791371 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1cffff700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1d006a2a0 0x7fb1d0198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1cf7fe700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d006ac60 0x7fb1d01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d4a48700 1 -- 172.21.15.33:0/3931791371 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1d006a2a0 msgr2=0x7fb1d0198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:55.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d4a48700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1d006a2a0 0x7fb1d0198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d4a48700 1 -- 172.21.15.33:0/3931791371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d006ac60 msgr2=0x7fb1d01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.516+0000 7fb1d4a48700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d006ac60 0x7fb1d01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.517+0000 7fb1d4a48700 1 -- 172.21.15.33:0/3931791371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d019dd80 con 0x7fb1d0108650 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.517+0000 7fb1d4a48700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1d0108650 0x7fb1d019d640 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8007ee0 tx=0x7fb1c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.517+0000 7fb1cd7fa700 1 -- 172.21.15.33:0/3931791371 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c800dbd0 con 0x7fb1d0108650 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.517+0000 7fb1cd7fa700 1 -- 172.21.15.33:0/3931791371 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb1c800dd70 con 0x7fb1d0108650 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.517+0000 7fb1d64ab700 1 -- 172.21.15.33:0/3931791371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d019e070 con 0x7fb1d0108650 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.517+0000 7fb1cd7fa700 1 -- 172.21.15.33:0/3931791371 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80133f0 con 0x7fb1d0108650 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.517+0000 7fb1d64ab700 1 -- 172.21.15.33:0/3931791371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d01a58f0 con 0x7fb1d0108650 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.518+0000 7fb1d64ab700 1 -- 172.21.15.33:0/3931791371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d00692b0 con 0x7fb1d0108650 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.520+0000 7fb1cd7fa700 1 -- 172.21.15.33:0/3931791371 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb1c801a020 con 0x7fb1d0108650 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.520+0000 7fb1cd7fa700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb1b80a3d50 0x7fb1b80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.520+0000 7fb1cd7fa700 1 -- 172.21.15.33:0/3931791371 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb1c80d0680 con 0x7fb1d0108650 2024-04-02T13:37:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.520+0000 7fb1cffff700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb1b80a3d50 0x7fb1b80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.521+0000 7fb1cffff700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb1b80a3d50 0x7fb1b80a6210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000b5c0 tx=0x7fb1c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.524+0000 7fb1cd7fa700 1 -- 172.21.15.33:0/3931791371 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb1c8010070 con 0x7fb1d0108650 2024-04-02T13:37:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.738+0000 7fb1d64ab700 1 -- 172.21.15.33:0/3931791371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb1d002dd30 con 0x7fb1d0108650 2024-04-02T13:37:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.739+0000 7fb1cd7fa700 1 -- 172.21.15.33:0/3931791371 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb1c809cba0 con 0x7fb1d0108650 2024-04-02T13:37:55.741 INFO:teuthology.orchestra.run.smithi033.stdout:55834574931 2024-04-02T13:37:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.741+0000 7fb1d64ab700 1 -- 172.21.15.33:0/3931791371 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb1b80a3d50 msgr2=0x7fb1b80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.741+0000 7fb1d64ab700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb1b80a3d50 0x7fb1b80a6210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000b5c0 tx=0x7fb1c0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.741+0000 7fb1d64ab700 1 -- 172.21.15.33:0/3931791371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1d0108650 msgr2=0x7fb1d019d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.741+0000 7fb1d64ab700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1d0108650 0x7fb1d019d640 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8007ee0 tx=0x7fb1c800d3b0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.741+0000 7fb1d64ab700 1 -- 172.21.15.33:0/3931791371 shutdown_connections 2024-04-02T13:37:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.741+0000 7fb1d64ab700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb1b80a3d50 0x7fb1b80a6210 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.741+0000 7fb1d64ab700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1d0108650 0x7fb1d019d640 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.741+0000 7fb1d64ab700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb1d006a2a0 0x7fb1d0198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.741+0000 7fb1d64ab700 1 --2- 172.21.15.33:0/3931791371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1d006ac60 0x7fb1d01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:55.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.741+0000 7fb1d64ab700 1 -- 172.21.15.33:0/3931791371 >> 172.21.15.33:0/3931791371 conn(0x7fb1d0078420 msgr2=0x7fb1d0100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:55.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.741+0000 7fb1d64ab700 1 -- 172.21.15.33:0/3931791371 shutdown_connections 2024-04-02T13:37:55.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:55.742+0000 7fb1d64ab700 1 -- 172.21.15.33:0/3931791371 wait complete. 2024-04-02T13:37:56.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:56 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3665207008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T13:37:56.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:56 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:56.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:56 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:56.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:56 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3931791371' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T13:37:56.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:56 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:37:56.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:56 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:37:56.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:56 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:56.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:56 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3665207008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T13:37:56.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:56.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:56.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:56 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3931791371' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T13:37:56.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:37:56.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:37:56.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:56 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:56.640 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574931 for osd.1 2024-04-02T13:37:56.641 DEBUG:teuthology.parallel:result is None 2024-04-02T13:37:56.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:56 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3665207008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T13:37:56.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:56 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:56.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:56 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:56.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:56 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3931791371' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T13:37:56.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:56 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:37:56.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:56 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:37:56.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:56 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' 2024-04-02T13:37:57.076 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037802 got 257698037807 for osd.8 2024-04-02T13:37:57.077 DEBUG:teuthology.parallel:result is None 2024-04-02T13:37:57.513 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776165 got 292057776170 for osd.9 2024-04-02T13:37:57.513 DEBUG:teuthology.parallel:result is None 2024-04-02T13:37:57.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:57 smithi033 ceph-mon[32390]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:37:57.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:57 smithi145 ceph-mon[35646]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:37:57.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:57 smithi050 ceph-mon[35524]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:37:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.504+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/324494874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e78107560 msgr2=0x7f9e78107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.504+0000 7f9e7e0cf700 1 --2- 172.21.15.33:0/324494874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e78107560 0x7f9e78107940 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0099c0 tx=0x7f9e6c009cd0 comp rx=0 tx=0).stop 2024-04-02T13:37:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.504+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/324494874 shutdown_connections 2024-04-02T13:37:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.504+0000 7f9e7e0cf700 1 --2- 172.21.15.33:0/324494874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e78101ee0 0x7f9e7810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.504+0000 7f9e7e0cf700 1 --2- 172.21.15.33:0/324494874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e78107560 0x7f9e78107940 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.504+0000 7f9e7e0cf700 1 --2- 172.21.15.33:0/324494874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e78101520 0x7f9e781019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.504+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/324494874 >> 172.21.15.33:0/324494874 conn(0x7f9e78077540 msgr2=0x7f9e78077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.504+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/324494874 shutdown_connections 2024-04-02T13:37:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.504+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/324494874 wait complete. 2024-04-02T13:37:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.505+0000 7f9e7e0cf700 1 Processor -- start 2024-04-02T13:37:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.505+0000 7f9e7e0cf700 1 -- start start 2024-04-02T13:37:58.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.505+0000 7f9e7e0cf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e78101520 0x7f9e7819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:58.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.505+0000 7f9e7e0cf700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e78101ee0 0x7f9e7819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.505+0000 7f9e7e0cf700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e78107560 0x7f9e78198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.505+0000 7f9e7e0cf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e781117b0 con 0x7f9e78101520 2024-04-02T13:37:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.505+0000 7f9e7e0cf700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e78111630 con 0x7f9e78101ee0 2024-04-02T13:37:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.505+0000 7f9e7e0cf700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9e78111930 con 0x7f9e78107560 2024-04-02T13:37:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.505+0000 7f9e777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e78101520 0x7f9e7819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.505+0000 7f9e777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e78101520 0x7f9e7819e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51720/0 (socket says 172.21.15.33:51720) 2024-04-02T13:37:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.505+0000 7f9e777fe700 1 -- 172.21.15.33:0/813437848 learned_addr learned my addr 172.21.15.33:0/813437848 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:37:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.505+0000 7f9e76ffd700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e78101ee0 0x7f9e7819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.505+0000 7f9e77fff700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e78107560 0x7f9e78198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.506+0000 7f9e777fe700 1 -- 172.21.15.33:0/813437848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e78107560 msgr2=0x7f9e78198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.506+0000 7f9e777fe700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e78107560 0x7f9e78198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:58.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.506+0000 7f9e777fe700 1 -- 172.21.15.33:0/813437848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e78101ee0 msgr2=0x7f9e7819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.506+0000 7f9e777fe700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e78101ee0 0x7f9e7819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.506+0000 7f9e777fe700 1 -- 172.21.15.33:0/813437848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e78199200 con 0x7f9e78101520 2024-04-02T13:37:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.506+0000 7f9e77fff700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e78107560 0x7f9e78198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.506+0000 7f9e777fe700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e78101520 0x7f9e7819e8e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009fd0 tx=0x7f9e6c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.506+0000 7f9e74ff9700 1 -- 172.21.15.33:0/813437848 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c027070 con 0x7f9e78101520 2024-04-02T13:37:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.506+0000 7f9e74ff9700 1 -- 172.21.15.33:0/813437848 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9e6c004130 con 0x7f9e78101520 2024-04-02T13:37:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.506+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/813437848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e78199490 con 0x7f9e78101520 2024-04-02T13:37:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.506+0000 7f9e74ff9700 1 -- 172.21.15.33:0/813437848 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c00f460 con 0x7f9e78101520 2024-04-02T13:37:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.506+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/813437848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e781a57b0 con 0x7f9e78101520 2024-04-02T13:37:58.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.507+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/813437848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e780692b0 con 0x7f9e78101520 2024-04-02T13:37:58.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.508+0000 7f9e74ff9700 1 -- 172.21.15.33:0/813437848 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9e6c02a070 con 0x7f9e78101520 2024-04-02T13:37:58.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.509+0000 7f9e74ff9700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9e600a3cd0 0x7f9e600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:58.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.509+0000 7f9e76ffd700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9e600a3cd0 0x7f9e600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:58.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.509+0000 7f9e74ff9700 1 -- 172.21.15.33:0/813437848 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9e6c0d17b0 con 0x7f9e78101520 2024-04-02T13:37:58.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.509+0000 7f9e76ffd700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9e600a3cd0 0x7f9e600a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9e68006fd0 tx=0x7f9e68008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:58.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.513+0000 7f9e74ff9700 1 -- 172.21.15.33:0/813437848 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9e6c013070 con 0x7f9e78101520 2024-04-02T13:37:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.721+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/813437848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9e781a5980 con 0x7f9e78101520 2024-04-02T13:37:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.721+0000 7f9e74ff9700 1 -- 172.21.15.33:0/813437848 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9e6c022470 con 0x7f9e78101520 2024-04-02T13:37:58.724 INFO:teuthology.orchestra.run.smithi033.stdout:167503724607 2024-04-02T13:37:58.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.724+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/813437848 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9e600a3cd0 msgr2=0x7f9e600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:58.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.724+0000 7f9e7e0cf700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9e600a3cd0 0x7f9e600a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9e68006fd0 tx=0x7f9e68008040 comp rx=0 tx=0).stop 2024-04-02T13:37:58.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.724+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/813437848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e78101520 msgr2=0x7f9e7819e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:58.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.724+0000 7f9e7e0cf700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e78101520 0x7f9e7819e8e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009fd0 tx=0x7f9e6c003730 comp rx=0 tx=0).stop 2024-04-02T13:37:58.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.724+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/813437848 shutdown_connections 2024-04-02T13:37:58.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.724+0000 7f9e7e0cf700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9e600a3cd0 0x7f9e600a6190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:58.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.724+0000 7f9e7e0cf700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e78107560 0x7f9e78198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:58.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.724+0000 7f9e7e0cf700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e78101520 0x7f9e7819e8e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:58.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.724+0000 7f9e7e0cf700 1 --2- 172.21.15.33:0/813437848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e78101ee0 0x7f9e7819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:58.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.724+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/813437848 >> 172.21.15.33:0/813437848 conn(0x7f9e78077540 msgr2=0x7f9e78100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:58.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.724+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/813437848 shutdown_connections 2024-04-02T13:37:58.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:37:58.724+0000 7f9e7e0cf700 1 -- 172.21.15.33:0/813437848 wait complete. 2024-04-02T13:37:59.471 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724601 got 167503724607 for osd.5 2024-04-02T13:37:59.472 DEBUG:teuthology.parallel:result is None 2024-04-02T13:37:59.472 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-02T13:37:59.472 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T13:37:59.720 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:37:59.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:59 smithi145 ceph-mon[35646]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:59.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:37:59 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/813437848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T13:37:59.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:59 smithi050 ceph-mon[35524]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:59.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:37:59 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/813437848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T13:37:59.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:59 smithi033 ceph-mon[32390]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:37:59.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:37:59 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/813437848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T13:38:01.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.412+0000 7f3024c9a700 1 -- 172.21.15.33:0/3796706987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302006acf0 msgr2=0x7f302006b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.412+0000 7f3024c9a700 1 --2- 172.21.15.33:0/3796706987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302006acf0 0x7f302006b170 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f30080099c0 tx=0x7f3008009cd0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.412+0000 7f3024c9a700 1 -- 172.21.15.33:0/3796706987 shutdown_connections 2024-04-02T13:38:01.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.412+0000 7f3024c9a700 1 --2- 172.21.15.33:0/3796706987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f302006b6b0 0x7f302010f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.412+0000 7f3024c9a700 1 --2- 172.21.15.33:0/3796706987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3020108650 0x7f3020108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.412+0000 7f3024c9a700 1 --2- 172.21.15.33:0/3796706987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302006acf0 0x7f302006b170 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.412+0000 7f3024c9a700 1 -- 172.21.15.33:0/3796706987 >> 172.21.15.33:0/3796706987 conn(0x7f3020077510 msgr2=0x7f3020077920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.412+0000 7f3024c9a700 1 -- 172.21.15.33:0/3796706987 shutdown_connections 2024-04-02T13:38:01.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.412+0000 7f3024c9a700 1 -- 172.21.15.33:0/3796706987 wait complete. 2024-04-02T13:38:01.416 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:01 smithi033 ceph-mon[32390]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-02T13:38:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.413+0000 7f3024c9a700 1 Processor -- start 2024-04-02T13:38:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.413+0000 7f3024c9a700 1 -- start start 2024-04-02T13:38:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.413+0000 7f3024c9a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f302006acf0 0x7f302019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.413+0000 7f3024c9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302006b6b0 0x7f302019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.413+0000 7f3024c9a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3020108650 0x7f3020198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.413+0000 7f3024c9a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3020112870 con 0x7f302006b6b0 2024-04-02T13:38:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.413+0000 7f3024c9a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f30201126f0 con 0x7f302006acf0 2024-04-02T13:38:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.413+0000 7f3024c9a700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f30201129f0 con 0x7f3020108650 2024-04-02T13:38:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.413+0000 7f301dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302006b6b0 0x7f302019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.413+0000 7f301dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302006b6b0 0x7f302019edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51730/0 (socket says 172.21.15.33:51730) 2024-04-02T13:38:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.413+0000 7f301dd9b700 1 -- 172.21.15.33:0/1891096195 learned_addr learned my addr 172.21.15.33:0/1891096195 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.413+0000 7f301e59c700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f302006acf0 0x7f302019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.413+0000 7f301ed9d700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3020108650 0x7f3020198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.414+0000 7f301dd9b700 1 -- 172.21.15.33:0/1891096195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3020108650 msgr2=0x7f3020198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.414+0000 7f301dd9b700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3020108650 0x7f3020198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.414+0000 7f301dd9b700 1 -- 172.21.15.33:0/1891096195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f302006acf0 msgr2=0x7f302019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.414+0000 7f301dd9b700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f302006acf0 0x7f302019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.414+0000 7f301dd9b700 1 -- 172.21.15.33:0/1891096195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30201991d0 con 0x7f302006b6b0 2024-04-02T13:38:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.414+0000 7f301e59c700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f302006acf0 0x7f302019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.414+0000 7f301dd9b700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302006b6b0 0x7f302019edf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f30080098e0 tx=0x7f30080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.414+0000 7f30177fe700 1 -- 172.21.15.33:0/1891096195 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008027070 con 0x7f302006b6b0 2024-04-02T13:38:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.414+0000 7f3024c9a700 1 -- 172.21.15.33:0/1891096195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3020199460 con 0x7f302006b6b0 2024-04-02T13:38:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.414+0000 7f30177fe700 1 -- 172.21.15.33:0/1891096195 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3008004060 con 0x7f302006b6b0 2024-04-02T13:38:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.414+0000 7f30177fe700 1 -- 172.21.15.33:0/1891096195 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008022dd0 con 0x7f302006b6b0 2024-04-02T13:38:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.414+0000 7f3024c9a700 1 -- 172.21.15.33:0/1891096195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30201a5780 con 0x7f302006b6b0 2024-04-02T13:38:01.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.416+0000 7f3024c9a700 1 -- 172.21.15.33:0/1891096195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3020103d20 con 0x7f302006b6b0 2024-04-02T13:38:01.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.416+0000 7f30177fe700 1 -- 172.21.15.33:0/1891096195 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f300802a070 con 0x7f302006b6b0 2024-04-02T13:38:01.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.417+0000 7f30177fe700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f300c0a3cd0 0x7f300c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.417+0000 7f301e59c700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f300c0a3cd0 0x7f300c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.417+0000 7f30177fe700 1 -- 172.21.15.33:0/1891096195 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f30080d16a0 con 0x7f302006b6b0 2024-04-02T13:38:01.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.418+0000 7f301e59c700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f300c0a3cd0 0x7f300c0a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3020199bb0 tx=0x7f3010009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.422+0000 7f30177fe700 1 -- 172.21.15.33:0/1891096195 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f300805ef20 con 0x7f302006b6b0 2024-04-02T13:38:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.626+0000 7f3024c9a700 1 -- 172.21.15.33:0/1891096195 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3020199fa0 con 0x7f300c0a3cd0 2024-04-02T13:38:01.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.635+0000 7f30177fe700 1 -- 172.21.15.33:0/1891096195 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340634 (secure 0 0 0) 0x7f3020199fa0 con 0x7f300c0a3cd0 2024-04-02T13:38:01.644 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:01.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.639+0000 7f3024c9a700 1 -- 172.21.15.33:0/1891096195 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f300c0a3cd0 msgr2=0x7f300c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.639+0000 7f3024c9a700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f300c0a3cd0 0x7f300c0a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3020199bb0 tx=0x7f3010009040 comp rx=0 tx=0).stop 2024-04-02T13:38:01.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.639+0000 7f3024c9a700 1 -- 172.21.15.33:0/1891096195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302006b6b0 msgr2=0x7f302019edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.639+0000 7f3024c9a700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302006b6b0 0x7f302019edf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f30080098e0 tx=0x7f30080047b0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.639+0000 7f3024c9a700 1 -- 172.21.15.33:0/1891096195 shutdown_connections 2024-04-02T13:38:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.639+0000 7f3024c9a700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f300c0a3cd0 0x7f300c0a6190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.639+0000 7f3024c9a700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3020108650 0x7f3020198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.639+0000 7f3024c9a700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f302006acf0 0x7f302019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.639+0000 7f3024c9a700 1 --2- 172.21.15.33:0/1891096195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302006b6b0 0x7f302019edf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.639+0000 7f3024c9a700 1 -- 172.21.15.33:0/1891096195 >> 172.21.15.33:0/1891096195 conn(0x7f3020077510 msgr2=0x7f3020100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.639+0000 7f3024c9a700 1 -- 172.21.15.33:0/1891096195 shutdown_connections 2024-04-02T13:38:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:01.639+0000 7f3024c9a700 1 -- 172.21.15.33:0/1891096195 wait complete. 2024-04-02T13:38:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-02T13:38:01.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:01 smithi145 ceph-mon[35646]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-02T13:38:01.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:01 smithi050 ceph-mon[35524]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-02T13:38:02.376 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-04-02T13:37:59.886889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658350,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":599,"num_read_kb":423,"num_write":345,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373968,"kb_used_data":12248,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567856,"statfs":{"total":1151940427776,"available":1151557484544,"internally_reserved":0,"allocated":12541952,"data_stored":7997372,"data_compressed":361470,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003392"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.623457+0000","last_change":"2024-04-02T13:36:21.492436+0000","last_active":"2024-04-02T13:37:17.623457+0000","last_peered":"2024-04-02T13:37:17.623457+0000","last_clean":"2024-04-02T13:37:17.623457+0000","last_became_active":"2024-04-02T13:36:21.491831+0000","last_became_peered":"2024-04-02T13:36:21.491831+0000","last_unstale":"2024-04-02T13:37:17.623457+0000","last_undegraded":"2024-04-02T13:37:17.623457+0000","last_fullsized":"2024-04-02T13:37:17.623457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:50:57.919680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627549+0000","last_change":"2024-04-02T13:36:21.498089+0000","last_active":"2024-04-02T13:37:17.627549+0000","last_peered":"2024-04-02T13:37:17.627549+0000","last_clean":"2024-04-02T13:37:17.627549+0000","last_became_active":"2024-04-02T13:36:21.497784+0000","last_became_peered":"2024-04-02T13:36:21.497784+0000","last_unstale":"2024-04-02T13:37:17.627549+0000","last_undegraded":"2024-04-02T13:37:17.627549+0000","last_fullsized":"2024-04-02T13:37:17.627549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:42.137396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626121+0000","last_change":"2024-04-02T13:36:21.492207+0000","last_active":"2024-04-02T13:37:17.626121+0000","last_peered":"2024-04-02T13:37:17.626121+0000","last_clean":"2024-04-02T13:37:17.626121+0000","last_became_active":"2024-04-02T13:36:21.490605+0000","last_became_peered":"2024-04-02T13:36:21.490605+0000","last_unstale":"2024-04-02T13:37:17.626121+0000","last_undegraded":"2024-04-02T13:37:17.626121+0000","last_fullsized":"2024-04-02T13:37:17.626121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:03.984226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.844290+0000","last_change":"2024-04-02T13:36:21.491225+0000","last_active":"2024-04-02T13:37:17.844290+0000","last_peered":"2024-04-02T13:37:17.844290+0000","last_clean":"2024-04-02T13:37:17.844290+0000","last_became_active":"2024-04-02T13:36:21.490886+0000","last_became_peered":"2024-04-02T13:36:21.490886+0000","last_unstale":"2024-04-02T13:37:17.844290+0000","last_undegraded":"2024-04-02T13:37:17.844290+0000","last_fullsized":"2024-04-02T13:37:17.844290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:55:53.316243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.619872+0000","last_change":"2024-04-02T13:36:21.495808+0000","last_active":"2024-04-02T13:37:17.619872+0000","last_peered":"2024-04-02T13:37:17.619872+0000","last_clean":"2024-04-02T13:37:17.619872+0000","last_became_active":"2024-04-02T13:36:21.495468+0000","last_became_peered":"2024-04-02T13:36:21.495468+0000","last_unstale":"2024-04-02T13:37:17.619872+0000","last_undegraded":"2024-04-02T13:37:17.619872+0000","last_fullsized":"2024-04-02T13:37:17.619872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:17.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620410+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:37:17.620410+0000","last_peered":"2024-04-02T13:37:17.620410+0000","last_clean":"2024-04-02T13:37:17.620410+0000","last_became_active":"2024-04-02T13:36:21.499200+0000","last_became_peered":"2024-04-02T13:36:21.499200+0000","last_unstale":"2024-04-02T13:37:17.620410+0000","last_undegraded":"2024-04-02T13:37:17.620410+0000","last_fullsized":"2024-04-02T13:37:17.620410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:54:52.113844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624474+0000","last_change":"2024-04-02T13:36:21.493429+0000","last_active":"2024-04-02T13:37:17.624474+0000","last_peered":"2024-04-02T13:37:17.624474+0000","last_clean":"2024-04-02T13:37:17.624474+0000","last_became_active":"2024-04-02T13:36:21.493097+0000","last_became_peered":"2024-04-02T13:36:21.493097+0000","last_unstale":"2024-04-02T13:37:17.624474+0000","last_undegraded":"2024-04-02T13:37:17.624474+0000","last_fullsized":"2024-04-02T13:37:17.624474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:47:03.330448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629635+0000","last_change":"2024-04-02T13:36:21.498261+0000","last_active":"2024-04-02T13:37:17.629635+0000","last_peered":"2024-04-02T13:37:17.629635+0000","last_clean":"2024-04-02T13:37:17.629635+0000","last_became_active":"2024-04-02T13:36:21.497951+0000","last_became_peered":"2024-04-02T13:36:21.497951+0000","last_unstale":"2024-04-02T13:37:17.629635+0000","last_undegraded":"2024-04-02T13:37:17.629635+0000","last_fullsized":"2024-04-02T13:37:17.629635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:37:57.070013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.871679+0000","last_change":"2024-04-02T13:36:21.490623+0000","last_active":"2024-04-02T13:37:17.871679+0000","last_peered":"2024-04-02T13:37:17.871679+0000","last_clean":"2024-04-02T13:37:17.871679+0000","last_became_active":"2024-04-02T13:36:21.490274+0000","last_became_peered":"2024-04-02T13:36:21.490274+0000","last_unstale":"2024-04-02T13:37:17.871679+0000","last_undegraded":"2024-04-02T13:37:17.871679+0000","last_fullsized":"2024-04-02T13:37:17.871679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:59:35.220430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.631180+0000","last_change":"2024-04-02T13:37:17.631180+0000","last_active":"2024-04-02T13:37:17.631180+0000","last_peered":"2024-04-02T13:37:17.631180+0000","last_clean":"2024-04-02T13:37:17.631180+0000","last_became_active":"2024-04-02T13:37:17.630778+0000","last_became_peered":"2024-04-02T13:37:17.630778+0000","last_unstale":"2024-04-02T13:37:17.631180+0000","last_undegraded":"2024-04-02T13:37:17.631180+0000","last_fullsized":"2024-04-02T13:37:17.631180+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:05:11.445074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625114+0000","last_change":"2024-04-02T13:36:21.492510+0000","last_active":"2024-04-02T13:37:17.625114+0000","last_peered":"2024-04-02T13:37:17.625114+0000","last_clean":"2024-04-02T13:37:17.625114+0000","last_became_active":"2024-04-02T13:36:21.492189+0000","last_became_peered":"2024-04-02T13:36:21.492189+0000","last_unstale":"2024-04-02T13:37:17.625114+0000","last_undegraded":"2024-04-02T13:37:17.625114+0000","last_fullsized":"2024-04-02T13:37:17.625114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:15.321621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.844172+0000","last_change":"2024-04-02T13:36:21.498015+0000","last_active":"2024-04-02T13:37:17.844172+0000","last_peered":"2024-04-02T13:37:17.844172+0000","last_clean":"2024-04-02T13:37:17.844172+0000","last_became_active":"2024-04-02T13:36:21.497677+0000","last_became_peered":"2024-04-02T13:36:21.497677+0000","last_unstale":"2024-04-02T13:37:17.844172+0000","last_undegraded":"2024-04-02T13:37:17.844172+0000","last_fullsized":"2024-04-02T13:37:17.844172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:43:19.035902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627856+0000","last_change":"2024-04-02T13:36:21.497047+0000","last_active":"2024-04-02T13:37:17.627856+0000","last_peered":"2024-04-02T13:37:17.627856+0000","last_clean":"2024-04-02T13:37:17.627856+0000","last_became_active":"2024-04-02T13:36:21.496722+0000","last_became_peered":"2024-04-02T13:36:21.496722+0000","last_unstale":"2024-04-02T13:37:17.627856+0000","last_undegraded":"2024-04-02T13:37:17.627856+0000","last_fullsized":"2024-04-02T13:37:17.627856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:03:14.400687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625718+0000","last_change":"2024-04-02T13:36:21.496513+0000","last_active":"2024-04-02T13:37:17.625718+0000","last_peered":"2024-04-02T13:37:17.625718+0000","last_clean":"2024-04-02T13:37:17.625718+0000","last_became_active":"2024-04-02T13:36:21.496209+0000","last_became_peered":"2024-04-02T13:36:21.496209+0000","last_unstale":"2024-04-02T13:37:17.625718+0000","last_undegraded":"2024-04-02T13:37:17.625718+0000","last_fullsized":"2024-04-02T13:37:17.625718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:50.684206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625789+0000","last_change":"2024-04-02T13:36:21.493418+0000","last_active":"2024-04-02T13:37:17.625789+0000","last_peered":"2024-04-02T13:37:17.625789+0000","last_clean":"2024-04-02T13:37:17.625789+0000","last_became_active":"2024-04-02T13:36:21.492785+0000","last_became_peered":"2024-04-02T13:36:21.492785+0000","last_unstale":"2024-04-02T13:37:17.625789+0000","last_undegraded":"2024-04-02T13:37:17.625789+0000","last_fullsized":"2024-04-02T13:37:17.625789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:11.973683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.870558+0000","last_change":"2024-04-02T13:36:21.490710+0000","last_active":"2024-04-02T13:37:17.870558+0000","last_peered":"2024-04-02T13:37:17.870558+0000","last_clean":"2024-04-02T13:37:17.870558+0000","last_became_active":"2024-04-02T13:36:21.489654+0000","last_became_peered":"2024-04-02T13:36:21.489654+0000","last_unstale":"2024-04-02T13:37:17.870558+0000","last_undegraded":"2024-04-02T13:37:17.870558+0000","last_fullsized":"2024-04-02T13:37:17.870558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:43:06.198540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.623136+0000","last_change":"2024-04-02T13:36:21.493306+0000","last_active":"2024-04-02T13:37:17.623136+0000","last_peered":"2024-04-02T13:37:17.623136+0000","last_clean":"2024-04-02T13:37:17.623136+0000","last_became_active":"2024-04-02T13:36:21.492949+0000","last_became_peered":"2024-04-02T13:36:21.492949+0000","last_unstale":"2024-04-02T13:37:17.623136+0000","last_undegraded":"2024-04-02T13:37:17.623136+0000","last_fullsized":"2024-04-02T13:37:17.623136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:34.264790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622206+0000","last_change":"2024-04-02T13:36:21.496631+0000","last_active":"2024-04-02T13:37:17.622206+0000","last_peered":"2024-04-02T13:37:17.622206+0000","last_clean":"2024-04-02T13:37:17.622206+0000","last_became_active":"2024-04-02T13:36:21.496245+0000","last_became_peered":"2024-04-02T13:36:21.496245+0000","last_unstale":"2024-04-02T13:37:17.622206+0000","last_undegraded":"2024-04-02T13:37:17.622206+0000","last_fullsized":"2024-04-02T13:37:17.622206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:14:51.126381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628822+0000","last_change":"2024-04-02T13:36:21.491640+0000","last_active":"2024-04-02T13:37:17.628822+0000","last_peered":"2024-04-02T13:37:17.628822+0000","last_clean":"2024-04-02T13:37:17.628822+0000","last_became_active":"2024-04-02T13:36:21.487799+0000","last_became_peered":"2024-04-02T13:36:21.487799+0000","last_unstale":"2024-04-02T13:37:17.628822+0000","last_undegraded":"2024-04-02T13:37:17.628822+0000","last_fullsized":"2024-04-02T13:37:17.628822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:16.893295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.630307+0000","last_change":"2024-04-02T13:36:25.499691+0000","last_active":"2024-04-02T13:37:17.630307+0000","last_peered":"2024-04-02T13:37:17.630307+0000","last_clean":"2024-04-02T13:37:17.630307+0000","last_became_active":"2024-04-02T13:36:25.499120+0000","last_became_peered":"2024-04-02T13:36:25.499120+0000","last_unstale":"2024-04-02T13:37:17.630307+0000","last_undegraded":"2024-04-02T13:37:17.630307+0000","last_fullsized":"2024-04-02T13:37:17.630307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:47:53.823456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.878343+0000","last_change":"2024-04-02T13:37:17.878343+0000","last_active":"2024-04-02T13:37:17.878343+0000","last_peered":"2024-04-02T13:37:17.878343+0000","last_clean":"2024-04-02T13:37:17.878343+0000","last_became_active":"2024-04-02T13:37:17.871984+0000","last_became_peered":"2024-04-02T13:37:17.871984+0000","last_unstale":"2024-04-02T13:37:17.878343+0000","last_undegraded":"2024-04-02T13:37:17.878343+0000","last_fullsized":"2024-04-02T13:37:17.878343+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:28.804070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628930+0000","last_change":"2024-04-02T13:36:21.499019+0000","last_active":"2024-04-02T13:37:17.628930+0000","last_peered":"2024-04-02T13:37:17.628930+0000","last_clean":"2024-04-02T13:37:17.628930+0000","last_became_active":"2024-04-02T13:36:21.498684+0000","last_became_peered":"2024-04-02T13:36:21.498684+0000","last_unstale":"2024-04-02T13:37:17.628930+0000","last_undegraded":"2024-04-02T13:37:17.628930+0000","last_fullsized":"2024-04-02T13:37:17.628930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:15.618382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.843388+0000","last_change":"2024-04-02T13:36:25.498544+0000","last_active":"2024-04-02T13:37:17.843388+0000","last_peered":"2024-04-02T13:37:17.843388+0000","last_clean":"2024-04-02T13:37:17.843388+0000","last_became_active":"2024-04-02T13:36:25.498076+0000","last_became_peered":"2024-04-02T13:36:25.498076+0000","last_unstale":"2024-04-02T13:37:17.843388+0000","last_undegraded":"2024-04-02T13:37:17.843388+0000","last_fullsized":"2024-04-02T13:37:17.843388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:04:47.311056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629614+0000","last_change":"2024-04-02T13:36:20.832649+0000","last_active":"2024-04-02T13:37:17.629614+0000","last_peered":"2024-04-02T13:37:17.629614+0000","last_clean":"2024-04-02T13:37:17.629614+0000","last_became_active":"2024-04-02T13:36:20.832184+0000","last_became_peered":"2024-04-02T13:36:20.832184+0000","last_unstale":"2024-04-02T13:37:17.629614+0000","last_undegraded":"2024-04-02T13:37:17.629614+0000","last_fullsized":"2024-04-02T13:37:17.629614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:37.639406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626479+0000","last_change":"2024-04-02T13:36:21.489988+0000","last_active":"2024-04-02T13:37:17.626479+0000","last_peered":"2024-04-02T13:37:17.626479+0000","last_clean":"2024-04-02T13:37:17.626479+0000","last_became_active":"2024-04-02T13:36:21.489636+0000","last_became_peered":"2024-04-02T13:36:21.489636+0000","last_unstale":"2024-04-02T13:37:17.626479+0000","last_undegraded":"2024-04-02T13:37:17.626479+0000","last_fullsized":"2024-04-02T13:37:17.626479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:52:36.670712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.843609+0000","last_change":"2024-04-02T13:36:25.499441+0000","last_active":"2024-04-02T13:37:17.843609+0000","last_peered":"2024-04-02T13:37:17.843609+0000","last_clean":"2024-04-02T13:37:17.843609+0000","last_became_active":"2024-04-02T13:36:25.498512+0000","last_became_peered":"2024-04-02T13:36:25.498512+0000","last_unstale":"2024-04-02T13:37:17.843609+0000","last_undegraded":"2024-04-02T13:37:17.843609+0000","last_fullsized":"2024-04-02T13:37:17.843609+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:48:59.291360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620931+0000","last_change":"2024-04-02T13:36:20.830008+0000","last_active":"2024-04-02T13:37:17.620931+0000","last_peered":"2024-04-02T13:37:17.620931+0000","last_clean":"2024-04-02T13:37:17.620931+0000","last_became_active":"2024-04-02T13:36:20.829048+0000","last_became_peered":"2024-04-02T13:36:20.829048+0000","last_unstale":"2024-04-02T13:37:17.620931+0000","last_undegraded":"2024-04-02T13:37:17.620931+0000","last_fullsized":"2024-04-02T13:37:17.620931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:08:34.405351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.871043+0000","last_change":"2024-04-02T13:36:21.497296+0000","last_active":"2024-04-02T13:37:17.871043+0000","last_peered":"2024-04-02T13:37:17.871043+0000","last_clean":"2024-04-02T13:37:17.871043+0000","last_became_active":"2024-04-02T13:36:21.496802+0000","last_became_peered":"2024-04-02T13:36:21.496802+0000","last_unstale":"2024-04-02T13:37:17.871043+0000","last_undegraded":"2024-04-02T13:37:17.871043+0000","last_fullsized":"2024-04-02T13:37:17.871043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:03.995029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.843396+0000","last_change":"2024-04-02T13:36:25.498661+0000","last_active":"2024-04-02T13:37:17.843396+0000","last_peered":"2024-04-02T13:37:17.843396+0000","last_clean":"2024-04-02T13:37:17.843396+0000","last_became_active":"2024-04-02T13:36:25.497866+0000","last_became_peered":"2024-04-02T13:36:25.497866+0000","last_unstale":"2024-04-02T13:37:17.843396+0000","last_undegraded":"2024-04-02T13:37:17.843396+0000","last_fullsized":"2024-04-02T13:37:17.843396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:47:05.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620388+0000","last_change":"2024-04-02T13:36:20.824586+0000","last_active":"2024-04-02T13:37:17.620388+0000","last_peered":"2024-04-02T13:37:17.620388+0000","last_clean":"2024-04-02T13:37:17.620388+0000","last_became_active":"2024-04-02T13:36:20.824254+0000","last_became_peered":"2024-04-02T13:36:20.824254+0000","last_unstale":"2024-04-02T13:37:17.620388+0000","last_undegraded":"2024-04-02T13:37:17.620388+0000","last_fullsized":"2024-04-02T13:37:17.620388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:08.774361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627957+0000","last_change":"2024-04-02T13:36:21.495125+0000","last_active":"2024-04-02T13:37:17.627957+0000","last_peered":"2024-04-02T13:37:17.627957+0000","last_clean":"2024-04-02T13:37:17.627957+0000","last_became_active":"2024-04-02T13:36:21.494724+0000","last_became_peered":"2024-04-02T13:36:21.494724+0000","last_unstale":"2024-04-02T13:37:17.627957+0000","last_undegraded":"2024-04-02T13:37:17.627957+0000","last_fullsized":"2024-04-02T13:37:17.627957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:27.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628666+0000","last_change":"2024-04-02T13:36:25.498708+0000","last_active":"2024-04-02T13:37:17.628666+0000","last_peered":"2024-04-02T13:37:17.628666+0000","last_clean":"2024-04-02T13:37:17.628666+0000","last_became_active":"2024-04-02T13:36:25.497834+0000","last_became_peered":"2024-04-02T13:36:25.497834+0000","last_unstale":"2024-04-02T13:37:17.628666+0000","last_undegraded":"2024-04-02T13:37:17.628666+0000","last_fullsized":"2024-04-02T13:37:17.628666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:14.524569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":109,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:58.596796+0000","last_change":"2024-04-02T13:36:20.824901+0000","last_active":"2024-04-02T13:37:58.596796+0000","last_peered":"2024-04-02T13:37:58.596796+0000","last_clean":"2024-04-02T13:37:58.596796+0000","last_became_active":"2024-04-02T13:36:20.823973+0000","last_became_peered":"2024-04-02T13:36:20.823973+0000","last_unstale":"2024-04-02T13:37:58.596796+0000","last_undegraded":"2024-04-02T13:37:58.596796+0000","last_fullsized":"2024-04-02T13:37:58.596796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:17.636527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620416+0000","last_change":"2024-04-02T13:36:21.493411+0000","last_active":"2024-04-02T13:37:17.620416+0000","last_peered":"2024-04-02T13:37:17.620416+0000","last_clean":"2024-04-02T13:37:17.620416+0000","last_became_active":"2024-04-02T13:36:21.493061+0000","last_became_peered":"2024-04-02T13:36:21.493061+0000","last_unstale":"2024-04-02T13:37:17.620416+0000","last_undegraded":"2024-04-02T13:37:17.620416+0000","last_fullsized":"2024-04-02T13:37:17.620416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:15.902656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628168+0000","last_change":"2024-04-02T13:36:25.494817+0000","last_active":"2024-04-02T13:37:17.628168+0000","last_peered":"2024-04-02T13:37:17.628168+0000","last_clean":"2024-04-02T13:37:17.628168+0000","last_became_active":"2024-04-02T13:36:25.494185+0000","last_became_peered":"2024-04-02T13:36:25.494185+0000","last_unstale":"2024-04-02T13:37:17.628168+0000","last_undegraded":"2024-04-02T13:37:17.628168+0000","last_fullsized":"2024-04-02T13:37:17.628168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:04:32.719985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620245+0000","last_change":"2024-04-02T13:36:20.821981+0000","last_active":"2024-04-02T13:37:17.620245+0000","last_peered":"2024-04-02T13:37:17.620245+0000","last_clean":"2024-04-02T13:37:17.620245+0000","last_became_active":"2024-04-02T13:36:20.821074+0000","last_became_peered":"2024-04-02T13:36:20.821074+0000","last_unstale":"2024-04-02T13:37:17.620245+0000","last_undegraded":"2024-04-02T13:37:17.620245+0000","last_fullsized":"2024-04-02T13:37:17.620245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:22.094586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626172+0000","last_change":"2024-04-02T13:36:21.490792+0000","last_active":"2024-04-02T13:37:17.626172+0000","last_peered":"2024-04-02T13:37:17.626172+0000","last_clean":"2024-04-02T13:37:17.626172+0000","last_became_active":"2024-04-02T13:36:21.490471+0000","last_became_peered":"2024-04-02T13:36:21.490471+0000","last_unstale":"2024-04-02T13:37:17.626172+0000","last_undegraded":"2024-04-02T13:37:17.626172+0000","last_fullsized":"2024-04-02T13:37:17.626172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:55.612316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622946+0000","last_change":"2024-04-02T13:36:25.503774+0000","last_active":"2024-04-02T13:37:17.622946+0000","last_peered":"2024-04-02T13:37:17.622946+0000","last_clean":"2024-04-02T13:37:17.622946+0000","last_became_active":"2024-04-02T13:36:25.503287+0000","last_became_peered":"2024-04-02T13:36:25.503287+0000","last_unstale":"2024-04-02T13:37:17.622946+0000","last_undegraded":"2024-04-02T13:37:17.622946+0000","last_fullsized":"2024-04-02T13:37:17.622946+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:57:29.939484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627378+0000","last_change":"2024-04-02T13:36:20.816760+0000","last_active":"2024-04-02T13:37:17.627378+0000","last_peered":"2024-04-02T13:37:17.627378+0000","last_clean":"2024-04-02T13:37:17.627378+0000","last_became_active":"2024-04-02T13:36:20.816291+0000","last_became_peered":"2024-04-02T13:36:20.816291+0000","last_unstale":"2024-04-02T13:37:17.627378+0000","last_undegraded":"2024-04-02T13:37:17.627378+0000","last_fullsized":"2024-04-02T13:37:17.627378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:25.543946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629524+0000","last_change":"2024-04-02T13:36:21.498541+0000","last_active":"2024-04-02T13:37:17.629524+0000","last_peered":"2024-04-02T13:37:17.629524+0000","last_clean":"2024-04-02T13:37:17.629524+0000","last_became_active":"2024-04-02T13:36:21.497498+0000","last_became_peered":"2024-04-02T13:36:21.497498+0000","last_unstale":"2024-04-02T13:37:17.629524+0000","last_undegraded":"2024-04-02T13:37:17.629524+0000","last_fullsized":"2024-04-02T13:37:17.629524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:08.315938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627339+0000","last_change":"2024-04-02T13:36:25.500245+0000","last_active":"2024-04-02T13:37:17.627339+0000","last_peered":"2024-04-02T13:37:17.627339+0000","last_clean":"2024-04-02T13:37:17.627339+0000","last_became_active":"2024-04-02T13:36:25.499844+0000","last_became_peered":"2024-04-02T13:36:25.499844+0000","last_unstale":"2024-04-02T13:37:17.627339+0000","last_undegraded":"2024-04-02T13:37:17.627339+0000","last_fullsized":"2024-04-02T13:37:17.627339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:17:01.105757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624925+0000","last_change":"2024-04-02T13:36:20.826851+0000","last_active":"2024-04-02T13:37:17.624925+0000","last_peered":"2024-04-02T13:37:17.624925+0000","last_clean":"2024-04-02T13:37:17.624925+0000","last_became_active":"2024-04-02T13:36:20.826227+0000","last_became_peered":"2024-04-02T13:36:20.826227+0000","last_unstale":"2024-04-02T13:37:17.624925+0000","last_undegraded":"2024-04-02T13:37:17.624925+0000","last_fullsized":"2024-04-02T13:37:17.624925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:45.841966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622901+0000","last_change":"2024-04-02T13:36:21.496520+0000","last_active":"2024-04-02T13:37:17.622901+0000","last_peered":"2024-04-02T13:37:17.622901+0000","last_clean":"2024-04-02T13:37:17.622901+0000","last_became_active":"2024-04-02T13:36:21.496082+0000","last_became_peered":"2024-04-02T13:36:21.496082+0000","last_unstale":"2024-04-02T13:37:17.622901+0000","last_undegraded":"2024-04-02T13:37:17.622901+0000","last_fullsized":"2024-04-02T13:37:17.622901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:12:06.517806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629033+0000","last_change":"2024-04-02T13:36:25.496064+0000","last_active":"2024-04-02T13:37:17.629033+0000","last_peered":"2024-04-02T13:37:17.629033+0000","last_clean":"2024-04-02T13:37:17.629033+0000","last_became_active":"2024-04-02T13:36:25.495464+0000","last_became_peered":"2024-04-02T13:36:25.495464+0000","last_unstale":"2024-04-02T13:37:17.629033+0000","last_undegraded":"2024-04-02T13:37:17.629033+0000","last_fullsized":"2024-04-02T13:37:17.629033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:06:15.206135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:58.596175+0000","last_change":"2024-04-02T13:36:20.821898+0000","last_active":"2024-04-02T13:37:58.596175+0000","last_peered":"2024-04-02T13:37:58.596175+0000","last_clean":"2024-04-02T13:37:58.596175+0000","last_became_active":"2024-04-02T13:36:20.821374+0000","last_became_peered":"2024-04-02T13:36:20.821374+0000","last_unstale":"2024-04-02T13:37:58.596175+0000","last_undegraded":"2024-04-02T13:37:58.596175+0000","last_fullsized":"2024-04-02T13:37:58.596175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:21.568976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.871572+0000","last_change":"2024-04-02T13:36:21.497343+0000","last_active":"2024-04-02T13:37:17.871572+0000","last_peered":"2024-04-02T13:37:17.871572+0000","last_clean":"2024-04-02T13:37:17.871572+0000","last_became_active":"2024-04-02T13:36:21.496733+0000","last_became_peered":"2024-04-02T13:36:21.496733+0000","last_unstale":"2024-04-02T13:37:17.871572+0000","last_undegraded":"2024-04-02T13:37:17.871572+0000","last_fullsized":"2024-04-02T13:37:17.871572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:26.068717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622387+0000","last_change":"2024-04-02T13:36:25.503011+0000","last_active":"2024-04-02T13:37:17.622387+0000","last_peered":"2024-04-02T13:37:17.622387+0000","last_clean":"2024-04-02T13:37:17.622387+0000","last_became_active":"2024-04-02T13:36:25.502571+0000","last_became_peered":"2024-04-02T13:36:25.502571+0000","last_unstale":"2024-04-02T13:37:17.622387+0000","last_undegraded":"2024-04-02T13:37:17.622387+0000","last_fullsized":"2024-04-02T13:37:17.622387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:48.672023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624779+0000","last_change":"2024-04-02T13:36:20.827705+0000","last_active":"2024-04-02T13:37:17.624779+0000","last_peered":"2024-04-02T13:37:17.624779+0000","last_clean":"2024-04-02T13:37:17.624779+0000","last_became_active":"2024-04-02T13:36:20.827285+0000","last_became_peered":"2024-04-02T13:36:20.827285+0000","last_unstale":"2024-04-02T13:37:17.624779+0000","last_undegraded":"2024-04-02T13:37:17.624779+0000","last_fullsized":"2024-04-02T13:37:17.624779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:12.175523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626756+0000","last_change":"2024-04-02T13:36:21.493365+0000","last_active":"2024-04-02T13:37:17.626756+0000","last_peered":"2024-04-02T13:37:17.626756+0000","last_clean":"2024-04-02T13:37:17.626756+0000","last_became_active":"2024-04-02T13:36:21.493011+0000","last_became_peered":"2024-04-02T13:36:21.493011+0000","last_unstale":"2024-04-02T13:37:17.626756+0000","last_undegraded":"2024-04-02T13:37:17.626756+0000","last_fullsized":"2024-04-02T13:37:17.626756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:53.165244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.843800+0000","last_change":"2024-04-02T13:36:25.499017+0000","last_active":"2024-04-02T13:37:17.843800+0000","last_peered":"2024-04-02T13:37:17.843800+0000","last_clean":"2024-04-02T13:37:17.843800+0000","last_became_active":"2024-04-02T13:36:25.498293+0000","last_became_peered":"2024-04-02T13:36:25.498293+0000","last_unstale":"2024-04-02T13:37:17.843800+0000","last_undegraded":"2024-04-02T13:37:17.843800+0000","last_fullsized":"2024-04-02T13:37:17.843800+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:55.910943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627781+0000","last_change":"2024-04-02T13:36:20.836345+0000","last_active":"2024-04-02T13:37:17.627781+0000","last_peered":"2024-04-02T13:37:17.627781+0000","last_clean":"2024-04-02T13:37:17.627781+0000","last_became_active":"2024-04-02T13:36:20.836037+0000","last_became_peered":"2024-04-02T13:36:20.836037+0000","last_unstale":"2024-04-02T13:37:17.627781+0000","last_undegraded":"2024-04-02T13:37:17.627781+0000","last_fullsized":"2024-04-02T13:37:17.627781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:47.070493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.871284+0000","last_change":"2024-04-02T13:36:21.493209+0000","last_active":"2024-04-02T13:37:17.871284+0000","last_peered":"2024-04-02T13:37:17.871284+0000","last_clean":"2024-04-02T13:37:17.871284+0000","last_became_active":"2024-04-02T13:36:21.492894+0000","last_became_peered":"2024-04-02T13:36:21.492894+0000","last_unstale":"2024-04-02T13:37:17.871284+0000","last_undegraded":"2024-04-02T13:37:17.871284+0000","last_fullsized":"2024-04-02T13:37:17.871284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:47.162354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627474+0000","last_change":"2024-04-02T13:37:17.626933+0000","last_active":"2024-04-02T13:37:17.627474+0000","last_peered":"2024-04-02T13:37:17.627474+0000","last_clean":"2024-04-02T13:37:17.627474+0000","last_became_active":"2024-04-02T13:37:17.626411+0000","last_became_peered":"2024-04-02T13:37:17.626411+0000","last_unstale":"2024-04-02T13:37:17.627474+0000","last_undegraded":"2024-04-02T13:37:17.627474+0000","last_fullsized":"2024-04-02T13:37:17.627474+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:40:56.903871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"90'33","reported_seq":216,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:59.597701+0000","last_change":"2024-04-02T13:36:20.823563+0000","last_active":"2024-04-02T13:37:59.597701+0000","last_peered":"2024-04-02T13:37:59.597701+0000","last_clean":"2024-04-02T13:37:59.597701+0000","last_became_active":"2024-04-02T13:36:20.823186+0000","last_became_peered":"2024-04-02T13:36:20.823186+0000","last_unstale":"2024-04-02T13:37:59.597701+0000","last_undegraded":"2024-04-02T13:37:59.597701+0000","last_fullsized":"2024-04-02T13:37:59.597701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:50.983357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63356,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":156,"num_read_kb":140,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620705+0000","last_change":"2024-04-02T13:36:21.500829+0000","last_active":"2024-04-02T13:37:17.620705+0000","last_peered":"2024-04-02T13:37:17.620705+0000","last_clean":"2024-04-02T13:37:17.620705+0000","last_became_active":"2024-04-02T13:36:21.500485+0000","last_became_peered":"2024-04-02T13:36:21.500485+0000","last_unstale":"2024-04-02T13:37:17.620705+0000","last_undegraded":"2024-04-02T13:37:17.620705+0000","last_fullsized":"2024-04-02T13:37:17.620705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:23:19.627565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628350+0000","last_change":"2024-04-02T13:36:25.506695+0000","last_active":"2024-04-02T13:37:17.628350+0000","last_peered":"2024-04-02T13:37:17.628350+0000","last_clean":"2024-04-02T13:37:17.628350+0000","last_became_active":"2024-04-02T13:36:25.506254+0000","last_became_peered":"2024-04-02T13:36:25.506254+0000","last_unstale":"2024-04-02T13:37:17.628350+0000","last_undegraded":"2024-04-02T13:37:17.628350+0000","last_fullsized":"2024-04-02T13:37:17.628350+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:57:00.315005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627963+0000","last_change":"2024-04-02T13:36:20.825827+0000","last_active":"2024-04-02T13:37:17.627963+0000","last_peered":"2024-04-02T13:37:17.627963+0000","last_clean":"2024-04-02T13:37:17.627963+0000","last_became_active":"2024-04-02T13:36:20.825530+0000","last_became_peered":"2024-04-02T13:36:20.825530+0000","last_unstale":"2024-04-02T13:37:17.627963+0000","last_undegraded":"2024-04-02T13:37:17.627963+0000","last_fullsized":"2024-04-02T13:37:17.627963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:27:02.962513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627435+0000","last_change":"2024-04-02T13:36:21.486343+0000","last_active":"2024-04-02T13:37:17.627435+0000","last_peered":"2024-04-02T13:37:17.627435+0000","last_clean":"2024-04-02T13:37:17.627435+0000","last_became_active":"2024-04-02T13:36:21.486021+0000","last_became_peered":"2024-04-02T13:36:21.486021+0000","last_unstale":"2024-04-02T13:37:17.627435+0000","last_undegraded":"2024-04-02T13:37:17.627435+0000","last_fullsized":"2024-04-02T13:37:17.627435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:53:42.430173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.631430+0000","last_change":"2024-04-02T13:37:17.631430+0000","last_active":"2024-04-02T13:37:17.631430+0000","last_peered":"2024-04-02T13:37:17.631430+0000","last_clean":"2024-04-02T13:37:17.631430+0000","last_became_active":"2024-04-02T13:37:17.630930+0000","last_became_peered":"2024-04-02T13:37:17.630930+0000","last_unstale":"2024-04-02T13:37:17.631430+0000","last_undegraded":"2024-04-02T13:37:17.631430+0000","last_fullsized":"2024-04-02T13:37:17.631430+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:10.514663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629023+0000","last_change":"2024-04-02T13:36:20.829303+0000","last_active":"2024-04-02T13:37:17.629023+0000","last_peered":"2024-04-02T13:37:17.629023+0000","last_clean":"2024-04-02T13:37:17.629023+0000","last_became_active":"2024-04-02T13:36:20.828952+0000","last_became_peered":"2024-04-02T13:36:20.828952+0000","last_unstale":"2024-04-02T13:37:17.629023+0000","last_undegraded":"2024-04-02T13:37:17.629023+0000","last_fullsized":"2024-04-02T13:37:17.629023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:14:04.385178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624883+0000","last_change":"2024-04-02T13:36:21.491767+0000","last_active":"2024-04-02T13:37:17.624883+0000","last_peered":"2024-04-02T13:37:17.624883+0000","last_clean":"2024-04-02T13:37:17.624883+0000","last_became_active":"2024-04-02T13:36:21.491369+0000","last_became_peered":"2024-04-02T13:36:21.491369+0000","last_unstale":"2024-04-02T13:37:17.624883+0000","last_undegraded":"2024-04-02T13:37:17.624883+0000","last_fullsized":"2024-04-02T13:37:17.624883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:14:35.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.623337+0000","last_change":"2024-04-02T13:36:25.503900+0000","last_active":"2024-04-02T13:37:17.623337+0000","last_peered":"2024-04-02T13:37:17.623337+0000","last_clean":"2024-04-02T13:37:17.623337+0000","last_became_active":"2024-04-02T13:36:25.502840+0000","last_became_peered":"2024-04-02T13:36:25.502840+0000","last_unstale":"2024-04-02T13:37:17.623337+0000","last_undegraded":"2024-04-02T13:37:17.623337+0000","last_fullsized":"2024-04-02T13:37:17.623337+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:31.106865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:54.620645+0000","last_change":"2024-04-02T13:36:20.823753+0000","last_active":"2024-04-02T13:37:54.620645+0000","last_peered":"2024-04-02T13:37:54.620645+0000","last_clean":"2024-04-02T13:37:54.620645+0000","last_became_active":"2024-04-02T13:36:20.823394+0000","last_became_peered":"2024-04-02T13:36:20.823394+0000","last_unstale":"2024-04-02T13:37:54.620645+0000","last_undegraded":"2024-04-02T13:37:54.620645+0000","last_fullsized":"2024-04-02T13:37:54.620645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:13.023125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625364+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:37:17.625364+0000","last_peered":"2024-04-02T13:37:17.625364+0000","last_clean":"2024-04-02T13:37:17.625364+0000","last_became_active":"2024-04-02T13:36:21.499197+0000","last_became_peered":"2024-04-02T13:36:21.499197+0000","last_unstale":"2024-04-02T13:37:17.625364+0000","last_undegraded":"2024-04-02T13:37:17.625364+0000","last_fullsized":"2024-04-02T13:37:17.625364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:26.435832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626882+0000","last_change":"2024-04-02T13:36:25.504643+0000","last_active":"2024-04-02T13:37:17.626882+0000","last_peered":"2024-04-02T13:37:17.626882+0000","last_clean":"2024-04-02T13:37:17.626882+0000","last_became_active":"2024-04-02T13:36:25.504249+0000","last_became_peered":"2024-04-02T13:36:25.504249+0000","last_unstale":"2024-04-02T13:37:17.626882+0000","last_undegraded":"2024-04-02T13:37:17.626882+0000","last_fullsized":"2024-04-02T13:37:17.626882+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:15.818958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.619628+0000","last_change":"2024-04-02T13:36:20.830097+0000","last_active":"2024-04-02T13:37:17.619628+0000","last_peered":"2024-04-02T13:37:17.619628+0000","last_clean":"2024-04-02T13:37:17.619628+0000","last_became_active":"2024-04-02T13:36:20.829779+0000","last_became_peered":"2024-04-02T13:36:20.829779+0000","last_unstale":"2024-04-02T13:37:17.619628+0000","last_undegraded":"2024-04-02T13:37:17.619628+0000","last_fullsized":"2024-04-02T13:37:17.619628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:59.477730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625384+0000","last_change":"2024-04-02T13:36:21.497480+0000","last_active":"2024-04-02T13:37:17.625384+0000","last_peered":"2024-04-02T13:37:17.625384+0000","last_clean":"2024-04-02T13:37:17.625384+0000","last_became_active":"2024-04-02T13:36:21.497106+0000","last_became_peered":"2024-04-02T13:36:21.497106+0000","last_unstale":"2024-04-02T13:37:17.625384+0000","last_undegraded":"2024-04-02T13:37:17.625384+0000","last_fullsized":"2024-04-02T13:37:17.625384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:30.394968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625245+0000","last_change":"2024-04-02T13:36:25.500040+0000","last_active":"2024-04-02T13:37:17.625245+0000","last_peered":"2024-04-02T13:37:17.625245+0000","last_clean":"2024-04-02T13:37:17.625245+0000","last_became_active":"2024-04-02T13:36:25.499576+0000","last_became_peered":"2024-04-02T13:36:25.499576+0000","last_unstale":"2024-04-02T13:37:17.625245+0000","last_undegraded":"2024-04-02T13:37:17.625245+0000","last_fullsized":"2024-04-02T13:37:17.625245+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:15:47.475234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620515+0000","last_change":"2024-04-02T13:36:20.823225+0000","last_active":"2024-04-02T13:37:17.620515+0000","last_peered":"2024-04-02T13:37:17.620515+0000","last_clean":"2024-04-02T13:37:17.620515+0000","last_became_active":"2024-04-02T13:36:20.822865+0000","last_became_peered":"2024-04-02T13:36:20.822865+0000","last_unstale":"2024-04-02T13:37:17.620515+0000","last_undegraded":"2024-04-02T13:37:17.620515+0000","last_fullsized":"2024-04-02T13:37:17.620515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:00.488190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622757+0000","last_change":"2024-04-02T13:36:21.497692+0000","last_active":"2024-04-02T13:37:17.622757+0000","last_peered":"2024-04-02T13:37:17.622757+0000","last_clean":"2024-04-02T13:37:17.622757+0000","last_became_active":"2024-04-02T13:36:21.497322+0000","last_became_peered":"2024-04-02T13:36:21.497322+0000","last_unstale":"2024-04-02T13:37:17.622757+0000","last_undegraded":"2024-04-02T13:37:17.622757+0000","last_fullsized":"2024-04-02T13:37:17.622757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:17.590235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.872416+0000","last_change":"2024-04-02T13:37:17.872416+0000","last_active":"2024-04-02T13:37:17.872416+0000","last_peered":"2024-04-02T13:37:17.872416+0000","last_clean":"2024-04-02T13:37:17.872416+0000","last_became_active":"2024-04-02T13:37:17.871932+0000","last_became_peered":"2024-04-02T13:37:17.871932+0000","last_unstale":"2024-04-02T13:37:17.872416+0000","last_undegraded":"2024-04-02T13:37:17.872416+0000","last_fullsized":"2024-04-02T13:37:17.872416+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:57:23.575737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622502+0000","last_change":"2024-04-02T13:36:20.835277+0000","last_active":"2024-04-02T13:37:17.622502+0000","last_peered":"2024-04-02T13:37:17.622502+0000","last_clean":"2024-04-02T13:37:17.622502+0000","last_became_active":"2024-04-02T13:36:20.834911+0000","last_became_peered":"2024-04-02T13:36:20.834911+0000","last_unstale":"2024-04-02T13:37:17.622502+0000","last_undegraded":"2024-04-02T13:37:17.622502+0000","last_fullsized":"2024-04-02T13:37:17.622502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:20.965544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620276+0000","last_change":"2024-04-02T13:36:21.495274+0000","last_active":"2024-04-02T13:37:17.620276+0000","last_peered":"2024-04-02T13:37:17.620276+0000","last_clean":"2024-04-02T13:37:17.620276+0000","last_became_active":"2024-04-02T13:36:21.494841+0000","last_became_peered":"2024-04-02T13:36:21.494841+0000","last_unstale":"2024-04-02T13:37:17.620276+0000","last_undegraded":"2024-04-02T13:37:17.620276+0000","last_fullsized":"2024-04-02T13:37:17.620276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:20.096507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627890+0000","last_change":"2024-04-02T13:36:25.499970+0000","last_active":"2024-04-02T13:37:17.627890+0000","last_peered":"2024-04-02T13:37:17.627890+0000","last_clean":"2024-04-02T13:37:17.627890+0000","last_became_active":"2024-04-02T13:36:25.499539+0000","last_became_peered":"2024-04-02T13:36:25.499539+0000","last_unstale":"2024-04-02T13:37:17.627890+0000","last_undegraded":"2024-04-02T13:37:17.627890+0000","last_fullsized":"2024-04-02T13:37:17.627890+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:43.136635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624544+0000","last_change":"2024-04-02T13:36:20.831150+0000","last_active":"2024-04-02T13:37:17.624544+0000","last_peered":"2024-04-02T13:37:17.624544+0000","last_clean":"2024-04-02T13:37:17.624544+0000","last_became_active":"2024-04-02T13:36:20.830838+0000","last_became_peered":"2024-04-02T13:36:20.830838+0000","last_unstale":"2024-04-02T13:37:17.624544+0000","last_undegraded":"2024-04-02T13:37:17.624544+0000","last_fullsized":"2024-04-02T13:37:17.624544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:23:09.566911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620004+0000","last_change":"2024-04-02T13:36:21.503128+0000","last_active":"2024-04-02T13:37:17.620004+0000","last_peered":"2024-04-02T13:37:17.620004+0000","last_clean":"2024-04-02T13:37:17.620004+0000","last_became_active":"2024-04-02T13:36:21.502695+0000","last_became_peered":"2024-04-02T13:36:21.502695+0000","last_unstale":"2024-04-02T13:37:17.620004+0000","last_undegraded":"2024-04-02T13:37:17.620004+0000","last_fullsized":"2024-04-02T13:37:17.620004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:38.261472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626017+0000","last_change":"2024-04-02T13:36:25.530911+0000","last_active":"2024-04-02T13:37:17.626017+0000","last_peered":"2024-04-02T13:37:17.626017+0000","last_clean":"2024-04-02T13:37:17.626017+0000","last_became_active":"2024-04-02T13:36:25.530428+0000","last_became_peered":"2024-04-02T13:36:25.530428+0000","last_unstale":"2024-04-02T13:37:17.626017+0000","last_undegraded":"2024-04-02T13:37:17.626017+0000","last_fullsized":"2024-04-02T13:37:17.626017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:49.185987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625224+0000","last_change":"2024-04-02T13:36:21.492251+0000","last_active":"2024-04-02T13:37:17.625224+0000","last_peered":"2024-04-02T13:37:17.625224+0000","last_clean":"2024-04-02T13:37:17.625224+0000","last_became_active":"2024-04-02T13:36:21.491929+0000","last_became_peered":"2024-04-02T13:36:21.491929+0000","last_unstale":"2024-04-02T13:37:17.625224+0000","last_undegraded":"2024-04-02T13:37:17.625224+0000","last_fullsized":"2024-04-02T13:37:17.625224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:59.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":100,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:55.621736+0000","last_change":"2024-04-02T13:36:20.821431+0000","last_active":"2024-04-02T13:37:55.621736+0000","last_peered":"2024-04-02T13:37:55.621736+0000","last_clean":"2024-04-02T13:37:55.621736+0000","last_became_active":"2024-04-02T13:36:20.821067+0000","last_became_peered":"2024-04-02T13:36:20.821067+0000","last_unstale":"2024-04-02T13:37:55.621736+0000","last_undegraded":"2024-04-02T13:37:55.621736+0000","last_fullsized":"2024-04-02T13:37:55.621736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:52.043907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624971+0000","last_change":"2024-04-02T13:36:25.496992+0000","last_active":"2024-04-02T13:37:17.624971+0000","last_peered":"2024-04-02T13:37:17.624971+0000","last_clean":"2024-04-02T13:37:17.624971+0000","last_became_active":"2024-04-02T13:36:25.496467+0000","last_became_peered":"2024-04-02T13:36:25.496467+0000","last_unstale":"2024-04-02T13:37:17.624971+0000","last_undegraded":"2024-04-02T13:37:17.624971+0000","last_fullsized":"2024-04-02T13:37:17.624971+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:13:00.747716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628776+0000","last_change":"2024-04-02T13:36:21.491399+0000","last_active":"2024-04-02T13:37:17.628776+0000","last_peered":"2024-04-02T13:37:17.628776+0000","last_clean":"2024-04-02T13:37:17.628776+0000","last_became_active":"2024-04-02T13:36:21.491038+0000","last_became_peered":"2024-04-02T13:36:21.491038+0000","last_unstale":"2024-04-02T13:37:17.628776+0000","last_undegraded":"2024-04-02T13:37:17.628776+0000","last_fullsized":"2024-04-02T13:37:17.628776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:25.603086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.843624+0000","last_change":"2024-04-02T13:36:20.822796+0000","last_active":"2024-04-02T13:37:17.843624+0000","last_peered":"2024-04-02T13:37:17.843624+0000","last_clean":"2024-04-02T13:37:17.843624+0000","last_became_active":"2024-04-02T13:36:20.822168+0000","last_became_peered":"2024-04-02T13:36:20.822168+0000","last_unstale":"2024-04-02T13:37:17.843624+0000","last_undegraded":"2024-04-02T13:37:17.843624+0000","last_fullsized":"2024-04-02T13:37:17.843624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:11.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625281+0000","last_change":"2024-04-02T13:35:08.678967+0000","last_active":"2024-04-02T13:37:17.625281+0000","last_peered":"2024-04-02T13:37:17.625281+0000","last_clean":"2024-04-02T13:37:17.625281+0000","last_became_active":"2024-04-02T13:35:08.678408+0000","last_became_peered":"2024-04-02T13:35:08.678408+0000","last_unstale":"2024-04-02T13:37:17.625281+0000","last_undegraded":"2024-04-02T13:37:17.625281+0000","last_fullsized":"2024-04-02T13:37:17.625281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_clean_scrub_stamp":"2024-04-02T13:31:42.453938+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:30:20.020809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626040+0000","last_change":"2024-04-02T13:36:25.532723+0000","last_active":"2024-04-02T13:37:17.626040+0000","last_peered":"2024-04-02T13:37:17.626040+0000","last_clean":"2024-04-02T13:37:17.626040+0000","last_became_active":"2024-04-02T13:36:25.532341+0000","last_became_peered":"2024-04-02T13:36:25.532341+0000","last_unstale":"2024-04-02T13:37:17.626040+0000","last_undegraded":"2024-04-02T13:37:17.626040+0000","last_fullsized":"2024-04-02T13:37:17.626040+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:53:43.190339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627018+0000","last_change":"2024-04-02T13:36:21.488983+0000","last_active":"2024-04-02T13:37:17.627018+0000","last_peered":"2024-04-02T13:37:17.627018+0000","last_clean":"2024-04-02T13:37:17.627018+0000","last_became_active":"2024-04-02T13:36:21.488580+0000","last_became_peered":"2024-04-02T13:36:21.488580+0000","last_unstale":"2024-04-02T13:37:17.627018+0000","last_undegraded":"2024-04-02T13:37:17.627018+0000","last_fullsized":"2024-04-02T13:37:17.627018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:39.382233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.634386+0000","last_change":"2024-04-02T13:37:17.633840+0000","last_active":"2024-04-02T13:37:17.634386+0000","last_peered":"2024-04-02T13:37:17.634386+0000","last_clean":"2024-04-02T13:37:17.634386+0000","last_became_active":"2024-04-02T13:37:17.626842+0000","last_became_peered":"2024-04-02T13:37:17.626842+0000","last_unstale":"2024-04-02T13:37:17.634386+0000","last_undegraded":"2024-04-02T13:37:17.634386+0000","last_fullsized":"2024-04-02T13:37:17.634386+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:55:17.437862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622442+0000","last_change":"2024-04-02T13:36:25.502814+0000","last_active":"2024-04-02T13:37:17.622442+0000","last_peered":"2024-04-02T13:37:17.622442+0000","last_clean":"2024-04-02T13:37:17.622442+0000","last_became_active":"2024-04-02T13:36:25.502288+0000","last_became_peered":"2024-04-02T13:36:25.502288+0000","last_unstale":"2024-04-02T13:37:17.622442+0000","last_undegraded":"2024-04-02T13:37:17.622442+0000","last_fullsized":"2024-04-02T13:37:17.622442+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:48.073286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620631+0000","last_change":"2024-04-02T13:36:20.830512+0000","last_active":"2024-04-02T13:37:17.620631+0000","last_peered":"2024-04-02T13:37:17.620631+0000","last_clean":"2024-04-02T13:37:17.620631+0000","last_became_active":"2024-04-02T13:36:20.830086+0000","last_became_peered":"2024-04-02T13:36:20.830086+0000","last_unstale":"2024-04-02T13:37:17.620631+0000","last_undegraded":"2024-04-02T13:37:17.620631+0000","last_fullsized":"2024-04-02T13:37:17.620631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:07.428363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628010+0000","last_change":"2024-04-02T13:36:21.478218+0000","last_active":"2024-04-02T13:37:17.628010+0000","last_peered":"2024-04-02T13:37:17.628010+0000","last_clean":"2024-04-02T13:37:17.628010+0000","last_became_active":"2024-04-02T13:36:21.477856+0000","last_became_peered":"2024-04-02T13:36:21.477856+0000","last_unstale":"2024-04-02T13:37:17.628010+0000","last_undegraded":"2024-04-02T13:37:17.628010+0000","last_fullsized":"2024-04-02T13:37:17.628010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:04.397606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629482+0000","last_change":"2024-04-02T13:36:25.530915+0000","last_active":"2024-04-02T13:37:17.629482+0000","last_peered":"2024-04-02T13:37:17.629482+0000","last_clean":"2024-04-02T13:37:17.629482+0000","last_became_active":"2024-04-02T13:36:25.530343+0000","last_became_peered":"2024-04-02T13:36:25.530343+0000","last_unstale":"2024-04-02T13:37:17.629482+0000","last_undegraded":"2024-04-02T13:37:17.629482+0000","last_fullsized":"2024-04-02T13:37:17.629482+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:32:41.181485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622371+0000","last_change":"2024-04-02T13:36:20.828527+0000","last_active":"2024-04-02T13:37:17.622371+0000","last_peered":"2024-04-02T13:37:17.622371+0000","last_clean":"2024-04-02T13:37:17.622371+0000","last_became_active":"2024-04-02T13:36:20.828136+0000","last_became_peered":"2024-04-02T13:36:20.828136+0000","last_unstale":"2024-04-02T13:37:17.622371+0000","last_undegraded":"2024-04-02T13:37:17.622371+0000","last_fullsized":"2024-04-02T13:37:17.622371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:57.709400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628777+0000","last_change":"2024-04-02T13:36:21.492035+0000","last_active":"2024-04-02T13:37:17.628777+0000","last_peered":"2024-04-02T13:37:17.628777+0000","last_clean":"2024-04-02T13:37:17.628777+0000","last_became_active":"2024-04-02T13:36:21.491682+0000","last_became_peered":"2024-04-02T13:36:21.491682+0000","last_unstale":"2024-04-02T13:37:17.628777+0000","last_undegraded":"2024-04-02T13:37:17.628777+0000","last_fullsized":"2024-04-02T13:37:17.628777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:09:40.600616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.630183+0000","last_change":"2024-04-02T13:36:25.493301+0000","last_active":"2024-04-02T13:37:17.630183+0000","last_peered":"2024-04-02T13:37:17.630183+0000","last_clean":"2024-04-02T13:37:17.630183+0000","last_became_active":"2024-04-02T13:36:25.492846+0000","last_became_peered":"2024-04-02T13:36:25.492846+0000","last_unstale":"2024-04-02T13:37:17.630183+0000","last_undegraded":"2024-04-02T13:37:17.630183+0000","last_fullsized":"2024-04-02T13:37:17.630183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:12:53.350261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.844269+0000","last_change":"2024-04-02T13:36:20.823559+0000","last_active":"2024-04-02T13:37:17.844269+0000","last_peered":"2024-04-02T13:37:17.844269+0000","last_clean":"2024-04-02T13:37:17.844269+0000","last_became_active":"2024-04-02T13:36:20.823178+0000","last_became_peered":"2024-04-02T13:36:20.823178+0000","last_unstale":"2024-04-02T13:37:17.844269+0000","last_undegraded":"2024-04-02T13:37:17.844269+0000","last_fullsized":"2024-04-02T13:37:17.844269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:42.628855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627447+0000","last_change":"2024-04-02T13:36:21.497258+0000","last_active":"2024-04-02T13:37:17.627447+0000","last_peered":"2024-04-02T13:37:17.627447+0000","last_clean":"2024-04-02T13:37:17.627447+0000","last_became_active":"2024-04-02T13:36:21.496943+0000","last_became_peered":"2024-04-02T13:36:21.496943+0000","last_unstale":"2024-04-02T13:37:17.627447+0000","last_undegraded":"2024-04-02T13:37:17.627447+0000","last_fullsized":"2024-04-02T13:37:17.627447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:51.989025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.843952+0000","last_change":"2024-04-02T13:36:25.531251+0000","last_active":"2024-04-02T13:37:17.843952+0000","last_peered":"2024-04-02T13:37:17.843952+0000","last_clean":"2024-04-02T13:37:17.843952+0000","last_became_active":"2024-04-02T13:36:25.530505+0000","last_became_peered":"2024-04-02T13:36:25.530505+0000","last_unstale":"2024-04-02T13:37:17.843952+0000","last_undegraded":"2024-04-02T13:37:17.843952+0000","last_fullsized":"2024-04-02T13:37:17.843952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:02:10.711138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.870569+0000","last_change":"2024-04-02T13:36:20.837865+0000","last_active":"2024-04-02T13:37:17.870569+0000","last_peered":"2024-04-02T13:37:17.870569+0000","last_clean":"2024-04-02T13:37:17.870569+0000","last_became_active":"2024-04-02T13:36:20.837567+0000","last_became_peered":"2024-04-02T13:36:20.837567+0000","last_unstale":"2024-04-02T13:37:17.870569+0000","last_undegraded":"2024-04-02T13:37:17.870569+0000","last_fullsized":"2024-04-02T13:37:17.870569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:11:59.365592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.619652+0000","last_change":"2024-04-02T13:36:21.500085+0000","last_active":"2024-04-02T13:37:17.619652+0000","last_peered":"2024-04-02T13:37:17.619652+0000","last_clean":"2024-04-02T13:37:17.619652+0000","last_became_active":"2024-04-02T13:36:21.499706+0000","last_became_peered":"2024-04-02T13:36:21.499706+0000","last_unstale":"2024-04-02T13:37:17.619652+0000","last_undegraded":"2024-04-02T13:37:17.619652+0000","last_fullsized":"2024-04-02T13:37:17.619652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.378226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.623475+0000","last_change":"2024-04-02T13:36:25.505797+0000","last_active":"2024-04-02T13:37:17.623475+0000","last_peered":"2024-04-02T13:37:17.623475+0000","last_clean":"2024-04-02T13:37:17.623475+0000","last_became_active":"2024-04-02T13:36:25.505409+0000","last_became_peered":"2024-04-02T13:36:25.505409+0000","last_unstale":"2024-04-02T13:37:17.623475+0000","last_undegraded":"2024-04-02T13:37:17.623475+0000","last_fullsized":"2024-04-02T13:37:17.623475+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:46:31.723248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629127+0000","last_change":"2024-04-02T13:36:20.821572+0000","last_active":"2024-04-02T13:37:17.629127+0000","last_peered":"2024-04-02T13:37:17.629127+0000","last_clean":"2024-04-02T13:37:17.629127+0000","last_became_active":"2024-04-02T13:36:20.821235+0000","last_became_peered":"2024-04-02T13:36:20.821235+0000","last_unstale":"2024-04-02T13:37:17.629127+0000","last_undegraded":"2024-04-02T13:37:17.629127+0000","last_fullsized":"2024-04-02T13:37:17.629127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:02.304145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.623246+0000","last_change":"2024-04-02T13:36:21.485255+0000","last_active":"2024-04-02T13:37:17.623246+0000","last_peered":"2024-04-02T13:37:17.623246+0000","last_clean":"2024-04-02T13:37:17.623246+0000","last_became_active":"2024-04-02T13:36:21.484870+0000","last_became_peered":"2024-04-02T13:36:21.484870+0000","last_unstale":"2024-04-02T13:37:17.623246+0000","last_undegraded":"2024-04-02T13:37:17.623246+0000","last_fullsized":"2024-04-02T13:37:17.623246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:03.851161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.844001+0000","last_change":"2024-04-02T13:36:25.494161+0000","last_active":"2024-04-02T13:37:17.844001+0000","last_peered":"2024-04-02T13:37:17.844001+0000","last_clean":"2024-04-02T13:37:17.844001+0000","last_became_active":"2024-04-02T13:36:25.493667+0000","last_became_peered":"2024-04-02T13:36:25.493667+0000","last_unstale":"2024-04-02T13:37:17.844001+0000","last_undegraded":"2024-04-02T13:37:17.844001+0000","last_fullsized":"2024-04-02T13:37:17.844001+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:35:45.703988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.636702+0000","last_change":"2024-04-02T13:37:17.636209+0000","last_active":"2024-04-02T13:37:17.636702+0000","last_peered":"2024-04-02T13:37:17.636702+0000","last_clean":"2024-04-02T13:37:17.636702+0000","last_became_active":"2024-04-02T13:37:17.631539+0000","last_became_peered":"2024-04-02T13:37:17.631539+0000","last_unstale":"2024-04-02T13:37:17.636702+0000","last_undegraded":"2024-04-02T13:37:17.636702+0000","last_fullsized":"2024-04-02T13:37:17.636702+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:05:52.619779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.630094+0000","last_change":"2024-04-02T13:36:21.498640+0000","last_active":"2024-04-02T13:37:17.630094+0000","last_peered":"2024-04-02T13:37:17.630094+0000","last_clean":"2024-04-02T13:37:17.630094+0000","last_became_active":"2024-04-02T13:36:21.497996+0000","last_became_peered":"2024-04-02T13:36:21.497996+0000","last_unstale":"2024-04-02T13:37:17.630094+0000","last_undegraded":"2024-04-02T13:37:17.630094+0000","last_fullsized":"2024-04-02T13:37:17.630094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:56:45.748945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.844057+0000","last_change":"2024-04-02T13:36:25.531119+0000","last_active":"2024-04-02T13:37:17.844057+0000","last_peered":"2024-04-02T13:37:17.844057+0000","last_clean":"2024-04-02T13:37:17.844057+0000","last_became_active":"2024-04-02T13:36:25.530307+0000","last_became_peered":"2024-04-02T13:36:25.530307+0000","last_unstale":"2024-04-02T13:37:17.844057+0000","last_undegraded":"2024-04-02T13:37:17.844057+0000","last_fullsized":"2024-04-02T13:37:17.844057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:58:50.543386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.619892+0000","last_change":"2024-04-02T13:36:20.829843+0000","last_active":"2024-04-02T13:37:17.619892+0000","last_peered":"2024-04-02T13:37:17.619892+0000","last_clean":"2024-04-02T13:37:17.619892+0000","last_became_active":"2024-04-02T13:36:20.829502+0000","last_became_peered":"2024-04-02T13:36:20.829502+0000","last_unstale":"2024-04-02T13:37:17.619892+0000","last_undegraded":"2024-04-02T13:37:17.619892+0000","last_fullsized":"2024-04-02T13:37:17.619892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:36:45.016174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624588+0000","last_change":"2024-04-02T13:36:21.491579+0000","last_active":"2024-04-02T13:37:17.624588+0000","last_peered":"2024-04-02T13:37:17.624588+0000","last_clean":"2024-04-02T13:37:17.624588+0000","last_became_active":"2024-04-02T13:36:21.490992+0000","last_became_peered":"2024-04-02T13:36:21.490992+0000","last_unstale":"2024-04-02T13:37:17.624588+0000","last_undegraded":"2024-04-02T13:37:17.624588+0000","last_fullsized":"2024-04-02T13:37:17.624588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:41.899766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.870929+0000","last_change":"2024-04-02T13:36:25.502658+0000","last_active":"2024-04-02T13:37:17.870929+0000","last_peered":"2024-04-02T13:37:17.870929+0000","last_clean":"2024-04-02T13:37:17.870929+0000","last_became_active":"2024-04-02T13:36:25.502219+0000","last_became_peered":"2024-04-02T13:36:25.502219+0000","last_unstale":"2024-04-02T13:37:17.870929+0000","last_undegraded":"2024-04-02T13:37:17.870929+0000","last_fullsized":"2024-04-02T13:37:17.870929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:34.189381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628406+0000","last_change":"2024-04-02T13:36:20.835176+0000","last_active":"2024-04-02T13:37:17.628406+0000","last_peered":"2024-04-02T13:37:17.628406+0000","last_clean":"2024-04-02T13:37:17.628406+0000","last_became_active":"2024-04-02T13:36:20.834871+0000","last_became_peered":"2024-04-02T13:36:20.834871+0000","last_unstale":"2024-04-02T13:37:17.628406+0000","last_undegraded":"2024-04-02T13:37:17.628406+0000","last_fullsized":"2024-04-02T13:37:17.628406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:17:14.148762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625234+0000","last_change":"2024-04-02T13:36:21.490336+0000","last_active":"2024-04-02T13:37:17.625234+0000","last_peered":"2024-04-02T13:37:17.625234+0000","last_clean":"2024-04-02T13:37:17.625234+0000","last_became_active":"2024-04-02T13:36:21.489869+0000","last_became_peered":"2024-04-02T13:36:21.489869+0000","last_unstale":"2024-04-02T13:37:17.625234+0000","last_undegraded":"2024-04-02T13:37:17.625234+0000","last_fullsized":"2024-04-02T13:37:17.625234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:42.243209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625720+0000","last_change":"2024-04-02T13:36:25.503124+0000","last_active":"2024-04-02T13:37:17.625720+0000","last_peered":"2024-04-02T13:37:17.625720+0000","last_clean":"2024-04-02T13:37:17.625720+0000","last_became_active":"2024-04-02T13:36:25.502737+0000","last_became_peered":"2024-04-02T13:36:25.502737+0000","last_unstale":"2024-04-02T13:37:17.625720+0000","last_undegraded":"2024-04-02T13:37:17.625720+0000","last_fullsized":"2024-04-02T13:37:17.625720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:02:56.830746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628083+0000","last_change":"2024-04-02T13:36:20.828597+0000","last_active":"2024-04-02T13:37:17.628083+0000","last_peered":"2024-04-02T13:37:17.628083+0000","last_clean":"2024-04-02T13:37:17.628083+0000","last_became_active":"2024-04-02T13:36:20.828138+0000","last_became_peered":"2024-04-02T13:36:20.828138+0000","last_unstale":"2024-04-02T13:37:17.628083+0000","last_undegraded":"2024-04-02T13:37:17.628083+0000","last_fullsized":"2024-04-02T13:37:17.628083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:05.171448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624652+0000","last_change":"2024-04-02T13:36:21.498498+0000","last_active":"2024-04-02T13:37:17.624652+0000","last_peered":"2024-04-02T13:37:17.624652+0000","last_clean":"2024-04-02T13:37:17.624652+0000","last_became_active":"2024-04-02T13:36:21.498119+0000","last_became_peered":"2024-04-02T13:36:21.498119+0000","last_unstale":"2024-04-02T13:37:17.624652+0000","last_undegraded":"2024-04-02T13:37:17.624652+0000","last_fullsized":"2024-04-02T13:37:17.624652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:01.528228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.872082+0000","last_change":"2024-04-02T13:37:17.872082+0000","last_active":"2024-04-02T13:37:17.872082+0000","last_peered":"2024-04-02T13:37:17.872082+0000","last_clean":"2024-04-02T13:37:17.872082+0000","last_became_active":"2024-04-02T13:37:17.871562+0000","last_became_peered":"2024-04-02T13:37:17.871562+0000","last_unstale":"2024-04-02T13:37:17.872082+0000","last_undegraded":"2024-04-02T13:37:17.872082+0000","last_fullsized":"2024-04-02T13:37:17.872082+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:32:39.849512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.848534+0000","last_change":"2024-04-02T13:37:17.848534+0000","last_active":"2024-04-02T13:37:17.848534+0000","last_peered":"2024-04-02T13:37:17.848534+0000","last_clean":"2024-04-02T13:37:17.848534+0000","last_became_active":"2024-04-02T13:37:17.844875+0000","last_became_peered":"2024-04-02T13:37:17.844875+0000","last_unstale":"2024-04-02T13:37:17.848534+0000","last_undegraded":"2024-04-02T13:37:17.848534+0000","last_fullsized":"2024-04-02T13:37:17.848534+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:29:05.037405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625157+0000","last_change":"2024-04-02T13:36:21.494069+0000","last_active":"2024-04-02T13:37:17.625157+0000","last_peered":"2024-04-02T13:37:17.625157+0000","last_clean":"2024-04-02T13:37:17.625157+0000","last_became_active":"2024-04-02T13:36:21.493751+0000","last_became_peered":"2024-04-02T13:36:21.493751+0000","last_unstale":"2024-04-02T13:37:17.625157+0000","last_undegraded":"2024-04-02T13:37:17.625157+0000","last_fullsized":"2024-04-02T13:37:17.625157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:24:39.648318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.623317+0000","last_change":"2024-04-02T13:36:21.494280+0000","last_active":"2024-04-02T13:37:17.623317+0000","last_peered":"2024-04-02T13:37:17.623317+0000","last_clean":"2024-04-02T13:37:17.623317+0000","last_became_active":"2024-04-02T13:36:21.491262+0000","last_became_peered":"2024-04-02T13:36:21.491262+0000","last_unstale":"2024-04-02T13:37:17.623317+0000","last_undegraded":"2024-04-02T13:37:17.623317+0000","last_fullsized":"2024-04-02T13:37:17.623317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:08:16.728646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.630422+0000","last_change":"2024-04-02T13:36:21.497103+0000","last_active":"2024-04-02T13:37:17.630422+0000","last_peered":"2024-04-02T13:37:17.630422+0000","last_clean":"2024-04-02T13:37:17.630422+0000","last_became_active":"2024-04-02T13:36:21.496784+0000","last_became_peered":"2024-04-02T13:36:21.496784+0000","last_unstale":"2024-04-02T13:37:17.630422+0000","last_undegraded":"2024-04-02T13:37:17.630422+0000","last_fullsized":"2024-04-02T13:37:17.630422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:38:21.903362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627624+0000","last_change":"2024-04-02T13:36:21.491853+0000","last_active":"2024-04-02T13:37:17.627624+0000","last_peered":"2024-04-02T13:37:17.627624+0000","last_clean":"2024-04-02T13:37:17.627624+0000","last_became_active":"2024-04-02T13:36:21.487986+0000","last_became_peered":"2024-04-02T13:36:21.487986+0000","last_unstale":"2024-04-02T13:37:17.627624+0000","last_undegraded":"2024-04-02T13:37:17.627624+0000","last_fullsized":"2024-04-02T13:37:17.627624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:37:47.098681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.631648+0000","last_change":"2024-04-02T13:37:17.631648+0000","last_active":"2024-04-02T13:37:17.631648+0000","last_peered":"2024-04-02T13:37:17.631648+0000","last_clean":"2024-04-02T13:37:17.631648+0000","last_became_active":"2024-04-02T13:37:17.631133+0000","last_became_peered":"2024-04-02T13:37:17.631133+0000","last_unstale":"2024-04-02T13:37:17.631648+0000","last_undegraded":"2024-04-02T13:37:17.631648+0000","last_fullsized":"2024-04-02T13:37:17.631648+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:10.791151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628154+0000","last_change":"2024-04-02T13:36:21.488723+0000","last_active":"2024-04-02T13:37:17.628154+0000","last_peered":"2024-04-02T13:37:17.628154+0000","last_clean":"2024-04-02T13:37:17.628154+0000","last_became_active":"2024-04-02T13:36:21.488391+0000","last_became_peered":"2024-04-02T13:36:21.488391+0000","last_unstale":"2024-04-02T13:37:17.628154+0000","last_undegraded":"2024-04-02T13:37:17.628154+0000","last_fullsized":"2024-04-02T13:37:17.628154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:23.768286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622972+0000","last_change":"2024-04-02T13:36:21.492676+0000","last_active":"2024-04-02T13:37:17.622972+0000","last_peered":"2024-04-02T13:37:17.622972+0000","last_clean":"2024-04-02T13:37:17.622972+0000","last_became_active":"2024-04-02T13:36:21.491663+0000","last_became_peered":"2024-04-02T13:36:21.491663+0000","last_unstale":"2024-04-02T13:37:17.622972+0000","last_undegraded":"2024-04-02T13:37:17.622972+0000","last_fullsized":"2024-04-02T13:37:17.622972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:00.131388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629336+0000","last_change":"2024-04-02T13:36:21.493380+0000","last_active":"2024-04-02T13:37:17.629336+0000","last_peered":"2024-04-02T13:37:17.629336+0000","last_clean":"2024-04-02T13:37:17.629336+0000","last_became_active":"2024-04-02T13:36:21.493073+0000","last_became_peered":"2024-04-02T13:36:21.493073+0000","last_unstale":"2024-04-02T13:37:17.629336+0000","last_undegraded":"2024-04-02T13:37:17.629336+0000","last_fullsized":"2024-04-02T13:37:17.629336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:59.412056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629236+0000","last_change":"2024-04-02T13:36:21.500881+0000","last_active":"2024-04-02T13:37:17.629236+0000","last_peered":"2024-04-02T13:37:17.629236+0000","last_clean":"2024-04-02T13:37:17.629236+0000","last_became_active":"2024-04-02T13:36:21.500516+0000","last_became_peered":"2024-04-02T13:36:21.500516+0000","last_unstale":"2024-04-02T13:37:17.629236+0000","last_undegraded":"2024-04-02T13:37:17.629236+0000","last_fullsized":"2024-04-02T13:37:17.629236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:33.842494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625093+0000","last_change":"2024-04-02T13:36:21.494558+0000","last_active":"2024-04-02T13:37:17.625093+0000","last_peered":"2024-04-02T13:37:17.625093+0000","last_clean":"2024-04-02T13:37:17.625093+0000","last_became_active":"2024-04-02T13:36:21.494180+0000","last_became_peered":"2024-04-02T13:36:21.494180+0000","last_unstale":"2024-04-02T13:37:17.625093+0000","last_undegraded":"2024-04-02T13:37:17.625093+0000","last_fullsized":"2024-04-02T13:37:17.625093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:07:54.363504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629262+0000","last_change":"2024-04-02T13:36:21.500406+0000","last_active":"2024-04-02T13:37:17.629262+0000","last_peered":"2024-04-02T13:37:17.629262+0000","last_clean":"2024-04-02T13:37:17.629262+0000","last_became_active":"2024-04-02T13:36:21.499927+0000","last_became_peered":"2024-04-02T13:36:21.499927+0000","last_unstale":"2024-04-02T13:37:17.629262+0000","last_undegraded":"2024-04-02T13:37:17.629262+0000","last_fullsized":"2024-04-02T13:37:17.629262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:23.702776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.871124+0000","last_change":"2024-04-02T13:36:21.486166+0000","last_active":"2024-04-02T13:37:17.871124+0000","last_peered":"2024-04-02T13:37:17.871124+0000","last_clean":"2024-04-02T13:37:17.871124+0000","last_became_active":"2024-04-02T13:36:21.485282+0000","last_became_peered":"2024-04-02T13:36:21.485282+0000","last_unstale":"2024-04-02T13:37:17.871124+0000","last_undegraded":"2024-04-02T13:37:17.871124+0000","last_fullsized":"2024-04-02T13:37:17.871124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:24:07.007675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627898+0000","last_change":"2024-04-02T13:36:21.484909+0000","last_active":"2024-04-02T13:37:17.627898+0000","last_peered":"2024-04-02T13:37:17.627898+0000","last_clean":"2024-04-02T13:37:17.627898+0000","last_became_active":"2024-04-02T13:36:21.484559+0000","last_became_peered":"2024-04-02T13:36:21.484559+0000","last_unstale":"2024-04-02T13:37:17.627898+0000","last_undegraded":"2024-04-02T13:37:17.627898+0000","last_fullsized":"2024-04-02T13:37:17.627898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:59:45.848407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622653+0000","last_change":"2024-04-02T13:36:21.493961+0000","last_active":"2024-04-02T13:37:17.622653+0000","last_peered":"2024-04-02T13:37:17.622653+0000","last_clean":"2024-04-02T13:37:17.622653+0000","last_became_active":"2024-04-02T13:36:21.493490+0000","last_became_peered":"2024-04-02T13:36:21.493490+0000","last_unstale":"2024-04-02T13:37:17.622653+0000","last_undegraded":"2024-04-02T13:37:17.622653+0000","last_fullsized":"2024-04-02T13:37:17.622653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:26:42.032550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":213880,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67748,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":463,"num_read_kb":304,"num_write":162,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":203244,"data_compressed":4818,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":503866,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612646,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":1154881,"data_compressed":58471,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":508198,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037808,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":503440,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":1100322,"data_compressed":56865,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":499278,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":563126,"data_compressed":22279,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":500666,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":564285,"data_compressed":22279,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":499472,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888132,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":504319,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":1095519,"data_compressed":56865,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34556,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34556,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63938,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65097,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65325,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:38:02.382 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T13:38:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:38:02.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:02 smithi145 ceph-mon[35646]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:02.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:02 smithi050 ceph-mon[35524]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:02.752 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:02 smithi033 ceph-mon[32390]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:03.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:03 smithi145 ceph-mon[35646]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:38:03.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:03 smithi050 ceph-mon[35524]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:38:03.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:03 smithi033 ceph-mon[32390]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:38:04.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.274+0000 7fb9a0443700 1 -- 172.21.15.33:0/3283936629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb998107500 msgr2=0x7fb9981078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:04.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.274+0000 7fb9a0443700 1 --2- 172.21.15.33:0/3283936629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb998107500 0x7fb9981078e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb99000b1a0 tx=0x7fb99000b4b0 comp rx=0 tx=0).stop 2024-04-02T13:38:04.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.274+0000 7fb9a0443700 1 -- 172.21.15.33:0/3283936629 shutdown_connections 2024-04-02T13:38:04.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.274+0000 7fb9a0443700 1 --2- 172.21.15.33:0/3283936629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb998101eb0 0x7fb99810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:04.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.274+0000 7fb9a0443700 1 --2- 172.21.15.33:0/3283936629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb998107500 0x7fb9981078e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:04.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.274+0000 7fb9a0443700 1 --2- 172.21.15.33:0/3283936629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9981014f0 0x7fb998101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:04.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.274+0000 7fb9a0443700 1 -- 172.21.15.33:0/3283936629 >> 172.21.15.33:0/3283936629 conn(0x7fb998078420 msgr2=0x7fb998078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.274+0000 7fb9a0443700 1 -- 172.21.15.33:0/3283936629 shutdown_connections 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.274+0000 7fb9a0443700 1 -- 172.21.15.33:0/3283936629 wait complete. 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb9a0443700 1 Processor -- start 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb9a0443700 1 -- start start 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb9a0443700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9981014f0 0x7fb99819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb9a0443700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb998101eb0 0x7fb99819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb9a0443700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb998107500 0x7fb99819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb9a0443700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb998110680 con 0x7fb998101eb0 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb9a0443700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb998110500 con 0x7fb9981014f0 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb9a0443700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb998110800 con 0x7fb998107500 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb99d9de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb998101eb0 0x7fb99819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb99d9de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb998101eb0 0x7fb99819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50232/0 (socket says 172.21.15.33:50232) 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb99d9de700 1 -- 172.21.15.33:0/2865557186 learned_addr learned my addr 172.21.15.33:0/2865557186 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb99d9de700 1 -- 172.21.15.33:0/2865557186 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb998107500 msgr2=0x7fb99819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb99d9de700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb998107500 0x7fb99819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb99d9de700 1 -- 172.21.15.33:0/2865557186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9981014f0 msgr2=0x7fb99819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb99e1df700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9981014f0 0x7fb99819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb99d9de700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9981014f0 0x7fb99819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.275+0000 7fb99d9de700 1 -- 172.21.15.33:0/2865557186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99819fe70 con 0x7fb998101eb0 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.276+0000 7fb99e1df700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9981014f0 0x7fb99819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.276+0000 7fb99d9de700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb998101eb0 0x7fb99819b3a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00d980 tx=0x7fb98c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.276+0000 7fb9977fe700 1 -- 172.21.15.33:0/2865557186 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c017070 con 0x7fb998101eb0 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.276+0000 7fb9977fe700 1 -- 172.21.15.33:0/2865557186 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb98c004d00 con 0x7fb998101eb0 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.276+0000 7fb9a0443700 1 -- 172.21.15.33:0/2865557186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9981a0160 con 0x7fb998101eb0 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.276+0000 7fb9977fe700 1 -- 172.21.15.33:0/2865557186 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c005520 con 0x7fb998101eb0 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.276+0000 7fb9a0443700 1 -- 172.21.15.33:0/2865557186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9981a79e0 con 0x7fb998101eb0 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.277+0000 7fb9a0443700 1 -- 172.21.15.33:0/2865557186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9980692b0 con 0x7fb998101eb0 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.278+0000 7fb9977fe700 1 -- 172.21.15.33:0/2865557186 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb98c004750 con 0x7fb998101eb0 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.278+0000 7fb9977fe700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9800a3d50 0x7fb9800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:04.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.279+0000 7fb99e1df700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9800a3d50 0x7fb9800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:04.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.279+0000 7fb9977fe700 1 -- 172.21.15.33:0/2865557186 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb98c0d0810 con 0x7fb998101eb0 2024-04-02T13:38:04.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.279+0000 7fb99e1df700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9800a3d50 0x7fb9800a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb9900062a0 tx=0x7fb9900061f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:04.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.283+0000 7fb9977fe700 1 -- 172.21.15.33:0/2865557186 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb98c020020 con 0x7fb998101eb0 2024-04-02T13:38:04.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.485+0000 7fb9a0443700 1 -- 172.21.15.33:0/2865557186 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb99802dda0 con 0x7fb9800a3d50 2024-04-02T13:38:04.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.494+0000 7fb9977fe700 1 -- 172.21.15.33:0/2865557186 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340634 (secure 0 0 0) 0x7fb99802dda0 con 0x7fb9800a3d50 2024-04-02T13:38:04.501 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.499+0000 7fb9a0443700 1 -- 172.21.15.33:0/2865557186 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9800a3d50 msgr2=0x7fb9800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.499+0000 7fb9a0443700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9800a3d50 0x7fb9800a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb9900062a0 tx=0x7fb9900061f0 comp rx=0 tx=0).stop 2024-04-02T13:38:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.499+0000 7fb9a0443700 1 -- 172.21.15.33:0/2865557186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb998101eb0 msgr2=0x7fb99819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.499+0000 7fb9a0443700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb998101eb0 0x7fb99819b3a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00d980 tx=0x7fb98c00dd40 comp rx=0 tx=0).stop 2024-04-02T13:38:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.499+0000 7fb9a0443700 1 -- 172.21.15.33:0/2865557186 shutdown_connections 2024-04-02T13:38:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.499+0000 7fb9a0443700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fb9800a3d50 0x7fb9800a6210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.499+0000 7fb9a0443700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb998107500 0x7fb99819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.499+0000 7fb9a0443700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9981014f0 0x7fb99819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.499+0000 7fb9a0443700 1 --2- 172.21.15.33:0/2865557186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb998101eb0 0x7fb99819b3a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.499+0000 7fb9a0443700 1 -- 172.21.15.33:0/2865557186 >> 172.21.15.33:0/2865557186 conn(0x7fb998078420 msgr2=0x7fb998100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.499+0000 7fb9a0443700 1 -- 172.21.15.33:0/2865557186 shutdown_connections 2024-04-02T13:38:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:04.499+0000 7fb9a0443700 1 -- 172.21.15.33:0/2865557186 wait complete. 2024-04-02T13:38:04.504 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-02T13:38:05.279 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-04-02T13:38:03.888015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658350,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":609,"num_read_kb":428,"num_write":345,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373968,"kb_used_data":12248,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567856,"statfs":{"total":1151940427776,"available":1151557484544,"internally_reserved":0,"allocated":12541952,"data_stored":7997372,"data_compressed":361470,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003478"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.623457+0000","last_change":"2024-04-02T13:36:21.492436+0000","last_active":"2024-04-02T13:37:17.623457+0000","last_peered":"2024-04-02T13:37:17.623457+0000","last_clean":"2024-04-02T13:37:17.623457+0000","last_became_active":"2024-04-02T13:36:21.491831+0000","last_became_peered":"2024-04-02T13:36:21.491831+0000","last_unstale":"2024-04-02T13:37:17.623457+0000","last_undegraded":"2024-04-02T13:37:17.623457+0000","last_fullsized":"2024-04-02T13:37:17.623457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:50:57.919680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627549+0000","last_change":"2024-04-02T13:36:21.498089+0000","last_active":"2024-04-02T13:37:17.627549+0000","last_peered":"2024-04-02T13:37:17.627549+0000","last_clean":"2024-04-02T13:37:17.627549+0000","last_became_active":"2024-04-02T13:36:21.497784+0000","last_became_peered":"2024-04-02T13:36:21.497784+0000","last_unstale":"2024-04-02T13:37:17.627549+0000","last_undegraded":"2024-04-02T13:37:17.627549+0000","last_fullsized":"2024-04-02T13:37:17.627549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:42.137396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626121+0000","last_change":"2024-04-02T13:36:21.492207+0000","last_active":"2024-04-02T13:37:17.626121+0000","last_peered":"2024-04-02T13:37:17.626121+0000","last_clean":"2024-04-02T13:37:17.626121+0000","last_became_active":"2024-04-02T13:36:21.490605+0000","last_became_peered":"2024-04-02T13:36:21.490605+0000","last_unstale":"2024-04-02T13:37:17.626121+0000","last_undegraded":"2024-04-02T13:37:17.626121+0000","last_fullsized":"2024-04-02T13:37:17.626121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:03.984226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.844290+0000","last_change":"2024-04-02T13:36:21.491225+0000","last_active":"2024-04-02T13:37:17.844290+0000","last_peered":"2024-04-02T13:37:17.844290+0000","last_clean":"2024-04-02T13:37:17.844290+0000","last_became_active":"2024-04-02T13:36:21.490886+0000","last_became_peered":"2024-04-02T13:36:21.490886+0000","last_unstale":"2024-04-02T13:37:17.844290+0000","last_undegraded":"2024-04-02T13:37:17.844290+0000","last_fullsized":"2024-04-02T13:37:17.844290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:55:53.316243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.619872+0000","last_change":"2024-04-02T13:36:21.495808+0000","last_active":"2024-04-02T13:37:17.619872+0000","last_peered":"2024-04-02T13:37:17.619872+0000","last_clean":"2024-04-02T13:37:17.619872+0000","last_became_active":"2024-04-02T13:36:21.495468+0000","last_became_peered":"2024-04-02T13:36:21.495468+0000","last_unstale":"2024-04-02T13:37:17.619872+0000","last_undegraded":"2024-04-02T13:37:17.619872+0000","last_fullsized":"2024-04-02T13:37:17.619872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:17.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620410+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:37:17.620410+0000","last_peered":"2024-04-02T13:37:17.620410+0000","last_clean":"2024-04-02T13:37:17.620410+0000","last_became_active":"2024-04-02T13:36:21.499200+0000","last_became_peered":"2024-04-02T13:36:21.499200+0000","last_unstale":"2024-04-02T13:37:17.620410+0000","last_undegraded":"2024-04-02T13:37:17.620410+0000","last_fullsized":"2024-04-02T13:37:17.620410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:54:52.113844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624474+0000","last_change":"2024-04-02T13:36:21.493429+0000","last_active":"2024-04-02T13:37:17.624474+0000","last_peered":"2024-04-02T13:37:17.624474+0000","last_clean":"2024-04-02T13:37:17.624474+0000","last_became_active":"2024-04-02T13:36:21.493097+0000","last_became_peered":"2024-04-02T13:36:21.493097+0000","last_unstale":"2024-04-02T13:37:17.624474+0000","last_undegraded":"2024-04-02T13:37:17.624474+0000","last_fullsized":"2024-04-02T13:37:17.624474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:47:03.330448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629635+0000","last_change":"2024-04-02T13:36:21.498261+0000","last_active":"2024-04-02T13:37:17.629635+0000","last_peered":"2024-04-02T13:37:17.629635+0000","last_clean":"2024-04-02T13:37:17.629635+0000","last_became_active":"2024-04-02T13:36:21.497951+0000","last_became_peered":"2024-04-02T13:36:21.497951+0000","last_unstale":"2024-04-02T13:37:17.629635+0000","last_undegraded":"2024-04-02T13:37:17.629635+0000","last_fullsized":"2024-04-02T13:37:17.629635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:37:57.070013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.871679+0000","last_change":"2024-04-02T13:36:21.490623+0000","last_active":"2024-04-02T13:37:17.871679+0000","last_peered":"2024-04-02T13:37:17.871679+0000","last_clean":"2024-04-02T13:37:17.871679+0000","last_became_active":"2024-04-02T13:36:21.490274+0000","last_became_peered":"2024-04-02T13:36:21.490274+0000","last_unstale":"2024-04-02T13:37:17.871679+0000","last_undegraded":"2024-04-02T13:37:17.871679+0000","last_fullsized":"2024-04-02T13:37:17.871679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:59:35.220430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.631180+0000","last_change":"2024-04-02T13:37:17.631180+0000","last_active":"2024-04-02T13:37:17.631180+0000","last_peered":"2024-04-02T13:37:17.631180+0000","last_clean":"2024-04-02T13:37:17.631180+0000","last_became_active":"2024-04-02T13:37:17.630778+0000","last_became_peered":"2024-04-02T13:37:17.630778+0000","last_unstale":"2024-04-02T13:37:17.631180+0000","last_undegraded":"2024-04-02T13:37:17.631180+0000","last_fullsized":"2024-04-02T13:37:17.631180+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:05:11.445074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625114+0000","last_change":"2024-04-02T13:36:21.492510+0000","last_active":"2024-04-02T13:37:17.625114+0000","last_peered":"2024-04-02T13:37:17.625114+0000","last_clean":"2024-04-02T13:37:17.625114+0000","last_became_active":"2024-04-02T13:36:21.492189+0000","last_became_peered":"2024-04-02T13:36:21.492189+0000","last_unstale":"2024-04-02T13:37:17.625114+0000","last_undegraded":"2024-04-02T13:37:17.625114+0000","last_fullsized":"2024-04-02T13:37:17.625114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:15.321621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.844172+0000","last_change":"2024-04-02T13:36:21.498015+0000","last_active":"2024-04-02T13:37:17.844172+0000","last_peered":"2024-04-02T13:37:17.844172+0000","last_clean":"2024-04-02T13:37:17.844172+0000","last_became_active":"2024-04-02T13:36:21.497677+0000","last_became_peered":"2024-04-02T13:36:21.497677+0000","last_unstale":"2024-04-02T13:37:17.844172+0000","last_undegraded":"2024-04-02T13:37:17.844172+0000","last_fullsized":"2024-04-02T13:37:17.844172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:43:19.035902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627856+0000","last_change":"2024-04-02T13:36:21.497047+0000","last_active":"2024-04-02T13:37:17.627856+0000","last_peered":"2024-04-02T13:37:17.627856+0000","last_clean":"2024-04-02T13:37:17.627856+0000","last_became_active":"2024-04-02T13:36:21.496722+0000","last_became_peered":"2024-04-02T13:36:21.496722+0000","last_unstale":"2024-04-02T13:37:17.627856+0000","last_undegraded":"2024-04-02T13:37:17.627856+0000","last_fullsized":"2024-04-02T13:37:17.627856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:03:14.400687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625718+0000","last_change":"2024-04-02T13:36:21.496513+0000","last_active":"2024-04-02T13:37:17.625718+0000","last_peered":"2024-04-02T13:37:17.625718+0000","last_clean":"2024-04-02T13:37:17.625718+0000","last_became_active":"2024-04-02T13:36:21.496209+0000","last_became_peered":"2024-04-02T13:36:21.496209+0000","last_unstale":"2024-04-02T13:37:17.625718+0000","last_undegraded":"2024-04-02T13:37:17.625718+0000","last_fullsized":"2024-04-02T13:37:17.625718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:50.684206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625789+0000","last_change":"2024-04-02T13:36:21.493418+0000","last_active":"2024-04-02T13:37:17.625789+0000","last_peered":"2024-04-02T13:37:17.625789+0000","last_clean":"2024-04-02T13:37:17.625789+0000","last_became_active":"2024-04-02T13:36:21.492785+0000","last_became_peered":"2024-04-02T13:36:21.492785+0000","last_unstale":"2024-04-02T13:37:17.625789+0000","last_undegraded":"2024-04-02T13:37:17.625789+0000","last_fullsized":"2024-04-02T13:37:17.625789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:11.973683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.870558+0000","last_change":"2024-04-02T13:36:21.490710+0000","last_active":"2024-04-02T13:37:17.870558+0000","last_peered":"2024-04-02T13:37:17.870558+0000","last_clean":"2024-04-02T13:37:17.870558+0000","last_became_active":"2024-04-02T13:36:21.489654+0000","last_became_peered":"2024-04-02T13:36:21.489654+0000","last_unstale":"2024-04-02T13:37:17.870558+0000","last_undegraded":"2024-04-02T13:37:17.870558+0000","last_fullsized":"2024-04-02T13:37:17.870558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:43:06.198540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.623136+0000","last_change":"2024-04-02T13:36:21.493306+0000","last_active":"2024-04-02T13:37:17.623136+0000","last_peered":"2024-04-02T13:37:17.623136+0000","last_clean":"2024-04-02T13:37:17.623136+0000","last_became_active":"2024-04-02T13:36:21.492949+0000","last_became_peered":"2024-04-02T13:36:21.492949+0000","last_unstale":"2024-04-02T13:37:17.623136+0000","last_undegraded":"2024-04-02T13:37:17.623136+0000","last_fullsized":"2024-04-02T13:37:17.623136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:34.264790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622206+0000","last_change":"2024-04-02T13:36:21.496631+0000","last_active":"2024-04-02T13:37:17.622206+0000","last_peered":"2024-04-02T13:37:17.622206+0000","last_clean":"2024-04-02T13:37:17.622206+0000","last_became_active":"2024-04-02T13:36:21.496245+0000","last_became_peered":"2024-04-02T13:36:21.496245+0000","last_unstale":"2024-04-02T13:37:17.622206+0000","last_undegraded":"2024-04-02T13:37:17.622206+0000","last_fullsized":"2024-04-02T13:37:17.622206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:14:51.126381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628822+0000","last_change":"2024-04-02T13:36:21.491640+0000","last_active":"2024-04-02T13:37:17.628822+0000","last_peered":"2024-04-02T13:37:17.628822+0000","last_clean":"2024-04-02T13:37:17.628822+0000","last_became_active":"2024-04-02T13:36:21.487799+0000","last_became_peered":"2024-04-02T13:36:21.487799+0000","last_unstale":"2024-04-02T13:37:17.628822+0000","last_undegraded":"2024-04-02T13:37:17.628822+0000","last_fullsized":"2024-04-02T13:37:17.628822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:16.893295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.630307+0000","last_change":"2024-04-02T13:36:25.499691+0000","last_active":"2024-04-02T13:37:17.630307+0000","last_peered":"2024-04-02T13:37:17.630307+0000","last_clean":"2024-04-02T13:37:17.630307+0000","last_became_active":"2024-04-02T13:36:25.499120+0000","last_became_peered":"2024-04-02T13:36:25.499120+0000","last_unstale":"2024-04-02T13:37:17.630307+0000","last_undegraded":"2024-04-02T13:37:17.630307+0000","last_fullsized":"2024-04-02T13:37:17.630307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:47:53.823456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.878343+0000","last_change":"2024-04-02T13:37:17.878343+0000","last_active":"2024-04-02T13:37:17.878343+0000","last_peered":"2024-04-02T13:37:17.878343+0000","last_clean":"2024-04-02T13:37:17.878343+0000","last_became_active":"2024-04-02T13:37:17.871984+0000","last_became_peered":"2024-04-02T13:37:17.871984+0000","last_unstale":"2024-04-02T13:37:17.878343+0000","last_undegraded":"2024-04-02T13:37:17.878343+0000","last_fullsized":"2024-04-02T13:37:17.878343+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:28.804070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628930+0000","last_change":"2024-04-02T13:36:21.499019+0000","last_active":"2024-04-02T13:37:17.628930+0000","last_peered":"2024-04-02T13:37:17.628930+0000","last_clean":"2024-04-02T13:37:17.628930+0000","last_became_active":"2024-04-02T13:36:21.498684+0000","last_became_peered":"2024-04-02T13:36:21.498684+0000","last_unstale":"2024-04-02T13:37:17.628930+0000","last_undegraded":"2024-04-02T13:37:17.628930+0000","last_fullsized":"2024-04-02T13:37:17.628930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:15.618382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.843388+0000","last_change":"2024-04-02T13:36:25.498544+0000","last_active":"2024-04-02T13:37:17.843388+0000","last_peered":"2024-04-02T13:37:17.843388+0000","last_clean":"2024-04-02T13:37:17.843388+0000","last_became_active":"2024-04-02T13:36:25.498076+0000","last_became_peered":"2024-04-02T13:36:25.498076+0000","last_unstale":"2024-04-02T13:37:17.843388+0000","last_undegraded":"2024-04-02T13:37:17.843388+0000","last_fullsized":"2024-04-02T13:37:17.843388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:04:47.311056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629614+0000","last_change":"2024-04-02T13:36:20.832649+0000","last_active":"2024-04-02T13:37:17.629614+0000","last_peered":"2024-04-02T13:37:17.629614+0000","last_clean":"2024-04-02T13:37:17.629614+0000","last_became_active":"2024-04-02T13:36:20.832184+0000","last_became_peered":"2024-04-02T13:36:20.832184+0000","last_unstale":"2024-04-02T13:37:17.629614+0000","last_undegraded":"2024-04-02T13:37:17.629614+0000","last_fullsized":"2024-04-02T13:37:17.629614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:37.639406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626479+0000","last_change":"2024-04-02T13:36:21.489988+0000","last_active":"2024-04-02T13:37:17.626479+0000","last_peered":"2024-04-02T13:37:17.626479+0000","last_clean":"2024-04-02T13:37:17.626479+0000","last_became_active":"2024-04-02T13:36:21.489636+0000","last_became_peered":"2024-04-02T13:36:21.489636+0000","last_unstale":"2024-04-02T13:37:17.626479+0000","last_undegraded":"2024-04-02T13:37:17.626479+0000","last_fullsized":"2024-04-02T13:37:17.626479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:52:36.670712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.843609+0000","last_change":"2024-04-02T13:36:25.499441+0000","last_active":"2024-04-02T13:37:17.843609+0000","last_peered":"2024-04-02T13:37:17.843609+0000","last_clean":"2024-04-02T13:37:17.843609+0000","last_became_active":"2024-04-02T13:36:25.498512+0000","last_became_peered":"2024-04-02T13:36:25.498512+0000","last_unstale":"2024-04-02T13:37:17.843609+0000","last_undegraded":"2024-04-02T13:37:17.843609+0000","last_fullsized":"2024-04-02T13:37:17.843609+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:48:59.291360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620931+0000","last_change":"2024-04-02T13:36:20.830008+0000","last_active":"2024-04-02T13:37:17.620931+0000","last_peered":"2024-04-02T13:37:17.620931+0000","last_clean":"2024-04-02T13:37:17.620931+0000","last_became_active":"2024-04-02T13:36:20.829048+0000","last_became_peered":"2024-04-02T13:36:20.829048+0000","last_unstale":"2024-04-02T13:37:17.620931+0000","last_undegraded":"2024-04-02T13:37:17.620931+0000","last_fullsized":"2024-04-02T13:37:17.620931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:08:34.405351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.871043+0000","last_change":"2024-04-02T13:36:21.497296+0000","last_active":"2024-04-02T13:37:17.871043+0000","last_peered":"2024-04-02T13:37:17.871043+0000","last_clean":"2024-04-02T13:37:17.871043+0000","last_became_active":"2024-04-02T13:36:21.496802+0000","last_became_peered":"2024-04-02T13:36:21.496802+0000","last_unstale":"2024-04-02T13:37:17.871043+0000","last_undegraded":"2024-04-02T13:37:17.871043+0000","last_fullsized":"2024-04-02T13:37:17.871043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:03.995029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.843396+0000","last_change":"2024-04-02T13:36:25.498661+0000","last_active":"2024-04-02T13:37:17.843396+0000","last_peered":"2024-04-02T13:37:17.843396+0000","last_clean":"2024-04-02T13:37:17.843396+0000","last_became_active":"2024-04-02T13:36:25.497866+0000","last_became_peered":"2024-04-02T13:36:25.497866+0000","last_unstale":"2024-04-02T13:37:17.843396+0000","last_undegraded":"2024-04-02T13:37:17.843396+0000","last_fullsized":"2024-04-02T13:37:17.843396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:47:05.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620388+0000","last_change":"2024-04-02T13:36:20.824586+0000","last_active":"2024-04-02T13:37:17.620388+0000","last_peered":"2024-04-02T13:37:17.620388+0000","last_clean":"2024-04-02T13:37:17.620388+0000","last_became_active":"2024-04-02T13:36:20.824254+0000","last_became_peered":"2024-04-02T13:36:20.824254+0000","last_unstale":"2024-04-02T13:37:17.620388+0000","last_undegraded":"2024-04-02T13:37:17.620388+0000","last_fullsized":"2024-04-02T13:37:17.620388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:08.774361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627957+0000","last_change":"2024-04-02T13:36:21.495125+0000","last_active":"2024-04-02T13:37:17.627957+0000","last_peered":"2024-04-02T13:37:17.627957+0000","last_clean":"2024-04-02T13:37:17.627957+0000","last_became_active":"2024-04-02T13:36:21.494724+0000","last_became_peered":"2024-04-02T13:36:21.494724+0000","last_unstale":"2024-04-02T13:37:17.627957+0000","last_undegraded":"2024-04-02T13:37:17.627957+0000","last_fullsized":"2024-04-02T13:37:17.627957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:27.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628666+0000","last_change":"2024-04-02T13:36:25.498708+0000","last_active":"2024-04-02T13:37:17.628666+0000","last_peered":"2024-04-02T13:37:17.628666+0000","last_clean":"2024-04-02T13:37:17.628666+0000","last_became_active":"2024-04-02T13:36:25.497834+0000","last_became_peered":"2024-04-02T13:36:25.497834+0000","last_unstale":"2024-04-02T13:37:17.628666+0000","last_undegraded":"2024-04-02T13:37:17.628666+0000","last_fullsized":"2024-04-02T13:37:17.628666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:14.524569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":109,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:58.596796+0000","last_change":"2024-04-02T13:36:20.824901+0000","last_active":"2024-04-02T13:37:58.596796+0000","last_peered":"2024-04-02T13:37:58.596796+0000","last_clean":"2024-04-02T13:37:58.596796+0000","last_became_active":"2024-04-02T13:36:20.823973+0000","last_became_peered":"2024-04-02T13:36:20.823973+0000","last_unstale":"2024-04-02T13:37:58.596796+0000","last_undegraded":"2024-04-02T13:37:58.596796+0000","last_fullsized":"2024-04-02T13:37:58.596796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:17.636527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620416+0000","last_change":"2024-04-02T13:36:21.493411+0000","last_active":"2024-04-02T13:37:17.620416+0000","last_peered":"2024-04-02T13:37:17.620416+0000","last_clean":"2024-04-02T13:37:17.620416+0000","last_became_active":"2024-04-02T13:36:21.493061+0000","last_became_peered":"2024-04-02T13:36:21.493061+0000","last_unstale":"2024-04-02T13:37:17.620416+0000","last_undegraded":"2024-04-02T13:37:17.620416+0000","last_fullsized":"2024-04-02T13:37:17.620416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:15.902656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628168+0000","last_change":"2024-04-02T13:36:25.494817+0000","last_active":"2024-04-02T13:37:17.628168+0000","last_peered":"2024-04-02T13:37:17.628168+0000","last_clean":"2024-04-02T13:37:17.628168+0000","last_became_active":"2024-04-02T13:36:25.494185+0000","last_became_peered":"2024-04-02T13:36:25.494185+0000","last_unstale":"2024-04-02T13:37:17.628168+0000","last_undegraded":"2024-04-02T13:37:17.628168+0000","last_fullsized":"2024-04-02T13:37:17.628168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:04:32.719985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620245+0000","last_change":"2024-04-02T13:36:20.821981+0000","last_active":"2024-04-02T13:37:17.620245+0000","last_peered":"2024-04-02T13:37:17.620245+0000","last_clean":"2024-04-02T13:37:17.620245+0000","last_became_active":"2024-04-02T13:36:20.821074+0000","last_became_peered":"2024-04-02T13:36:20.821074+0000","last_unstale":"2024-04-02T13:37:17.620245+0000","last_undegraded":"2024-04-02T13:37:17.620245+0000","last_fullsized":"2024-04-02T13:37:17.620245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:22.094586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626172+0000","last_change":"2024-04-02T13:36:21.490792+0000","last_active":"2024-04-02T13:37:17.626172+0000","last_peered":"2024-04-02T13:37:17.626172+0000","last_clean":"2024-04-02T13:37:17.626172+0000","last_became_active":"2024-04-02T13:36:21.490471+0000","last_became_peered":"2024-04-02T13:36:21.490471+0000","last_unstale":"2024-04-02T13:37:17.626172+0000","last_undegraded":"2024-04-02T13:37:17.626172+0000","last_fullsized":"2024-04-02T13:37:17.626172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:55.612316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622946+0000","last_change":"2024-04-02T13:36:25.503774+0000","last_active":"2024-04-02T13:37:17.622946+0000","last_peered":"2024-04-02T13:37:17.622946+0000","last_clean":"2024-04-02T13:37:17.622946+0000","last_became_active":"2024-04-02T13:36:25.503287+0000","last_became_peered":"2024-04-02T13:36:25.503287+0000","last_unstale":"2024-04-02T13:37:17.622946+0000","last_undegraded":"2024-04-02T13:37:17.622946+0000","last_fullsized":"2024-04-02T13:37:17.622946+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:57:29.939484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627378+0000","last_change":"2024-04-02T13:36:20.816760+0000","last_active":"2024-04-02T13:37:17.627378+0000","last_peered":"2024-04-02T13:37:17.627378+0000","last_clean":"2024-04-02T13:37:17.627378+0000","last_became_active":"2024-04-02T13:36:20.816291+0000","last_became_peered":"2024-04-02T13:36:20.816291+0000","last_unstale":"2024-04-02T13:37:17.627378+0000","last_undegraded":"2024-04-02T13:37:17.627378+0000","last_fullsized":"2024-04-02T13:37:17.627378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:25.543946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629524+0000","last_change":"2024-04-02T13:36:21.498541+0000","last_active":"2024-04-02T13:37:17.629524+0000","last_peered":"2024-04-02T13:37:17.629524+0000","last_clean":"2024-04-02T13:37:17.629524+0000","last_became_active":"2024-04-02T13:36:21.497498+0000","last_became_peered":"2024-04-02T13:36:21.497498+0000","last_unstale":"2024-04-02T13:37:17.629524+0000","last_undegraded":"2024-04-02T13:37:17.629524+0000","last_fullsized":"2024-04-02T13:37:17.629524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:08.315938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627339+0000","last_change":"2024-04-02T13:36:25.500245+0000","last_active":"2024-04-02T13:37:17.627339+0000","last_peered":"2024-04-02T13:37:17.627339+0000","last_clean":"2024-04-02T13:37:17.627339+0000","last_became_active":"2024-04-02T13:36:25.499844+0000","last_became_peered":"2024-04-02T13:36:25.499844+0000","last_unstale":"2024-04-02T13:37:17.627339+0000","last_undegraded":"2024-04-02T13:37:17.627339+0000","last_fullsized":"2024-04-02T13:37:17.627339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:17:01.105757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624925+0000","last_change":"2024-04-02T13:36:20.826851+0000","last_active":"2024-04-02T13:37:17.624925+0000","last_peered":"2024-04-02T13:37:17.624925+0000","last_clean":"2024-04-02T13:37:17.624925+0000","last_became_active":"2024-04-02T13:36:20.826227+0000","last_became_peered":"2024-04-02T13:36:20.826227+0000","last_unstale":"2024-04-02T13:37:17.624925+0000","last_undegraded":"2024-04-02T13:37:17.624925+0000","last_fullsized":"2024-04-02T13:37:17.624925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:45.841966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622901+0000","last_change":"2024-04-02T13:36:21.496520+0000","last_active":"2024-04-02T13:37:17.622901+0000","last_peered":"2024-04-02T13:37:17.622901+0000","last_clean":"2024-04-02T13:37:17.622901+0000","last_became_active":"2024-04-02T13:36:21.496082+0000","last_became_peered":"2024-04-02T13:36:21.496082+0000","last_unstale":"2024-04-02T13:37:17.622901+0000","last_undegraded":"2024-04-02T13:37:17.622901+0000","last_fullsized":"2024-04-02T13:37:17.622901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:12:06.517806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629033+0000","last_change":"2024-04-02T13:36:25.496064+0000","last_active":"2024-04-02T13:37:17.629033+0000","last_peered":"2024-04-02T13:37:17.629033+0000","last_clean":"2024-04-02T13:37:17.629033+0000","last_became_active":"2024-04-02T13:36:25.495464+0000","last_became_peered":"2024-04-02T13:36:25.495464+0000","last_unstale":"2024-04-02T13:37:17.629033+0000","last_undegraded":"2024-04-02T13:37:17.629033+0000","last_fullsized":"2024-04-02T13:37:17.629033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:06:15.206135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:58.596175+0000","last_change":"2024-04-02T13:36:20.821898+0000","last_active":"2024-04-02T13:37:58.596175+0000","last_peered":"2024-04-02T13:37:58.596175+0000","last_clean":"2024-04-02T13:37:58.596175+0000","last_became_active":"2024-04-02T13:36:20.821374+0000","last_became_peered":"2024-04-02T13:36:20.821374+0000","last_unstale":"2024-04-02T13:37:58.596175+0000","last_undegraded":"2024-04-02T13:37:58.596175+0000","last_fullsized":"2024-04-02T13:37:58.596175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:21.568976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.871572+0000","last_change":"2024-04-02T13:36:21.497343+0000","last_active":"2024-04-02T13:37:17.871572+0000","last_peered":"2024-04-02T13:37:17.871572+0000","last_clean":"2024-04-02T13:37:17.871572+0000","last_became_active":"2024-04-02T13:36:21.496733+0000","last_became_peered":"2024-04-02T13:36:21.496733+0000","last_unstale":"2024-04-02T13:37:17.871572+0000","last_undegraded":"2024-04-02T13:37:17.871572+0000","last_fullsized":"2024-04-02T13:37:17.871572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:26.068717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622387+0000","last_change":"2024-04-02T13:36:25.503011+0000","last_active":"2024-04-02T13:37:17.622387+0000","last_peered":"2024-04-02T13:37:17.622387+0000","last_clean":"2024-04-02T13:37:17.622387+0000","last_became_active":"2024-04-02T13:36:25.502571+0000","last_became_peered":"2024-04-02T13:36:25.502571+0000","last_unstale":"2024-04-02T13:37:17.622387+0000","last_undegraded":"2024-04-02T13:37:17.622387+0000","last_fullsized":"2024-04-02T13:37:17.622387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:48.672023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624779+0000","last_change":"2024-04-02T13:36:20.827705+0000","last_active":"2024-04-02T13:37:17.624779+0000","last_peered":"2024-04-02T13:37:17.624779+0000","last_clean":"2024-04-02T13:37:17.624779+0000","last_became_active":"2024-04-02T13:36:20.827285+0000","last_became_peered":"2024-04-02T13:36:20.827285+0000","last_unstale":"2024-04-02T13:37:17.624779+0000","last_undegraded":"2024-04-02T13:37:17.624779+0000","last_fullsized":"2024-04-02T13:37:17.624779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:12.175523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626756+0000","last_change":"2024-04-02T13:36:21.493365+0000","last_active":"2024-04-02T13:37:17.626756+0000","last_peered":"2024-04-02T13:37:17.626756+0000","last_clean":"2024-04-02T13:37:17.626756+0000","last_became_active":"2024-04-02T13:36:21.493011+0000","last_became_peered":"2024-04-02T13:36:21.493011+0000","last_unstale":"2024-04-02T13:37:17.626756+0000","last_undegraded":"2024-04-02T13:37:17.626756+0000","last_fullsized":"2024-04-02T13:37:17.626756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:53.165244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.843800+0000","last_change":"2024-04-02T13:36:25.499017+0000","last_active":"2024-04-02T13:37:17.843800+0000","last_peered":"2024-04-02T13:37:17.843800+0000","last_clean":"2024-04-02T13:37:17.843800+0000","last_became_active":"2024-04-02T13:36:25.498293+0000","last_became_peered":"2024-04-02T13:36:25.498293+0000","last_unstale":"2024-04-02T13:37:17.843800+0000","last_undegraded":"2024-04-02T13:37:17.843800+0000","last_fullsized":"2024-04-02T13:37:17.843800+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:55.910943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627781+0000","last_change":"2024-04-02T13:36:20.836345+0000","last_active":"2024-04-02T13:37:17.627781+0000","last_peered":"2024-04-02T13:37:17.627781+0000","last_clean":"2024-04-02T13:37:17.627781+0000","last_became_active":"2024-04-02T13:36:20.836037+0000","last_became_peered":"2024-04-02T13:36:20.836037+0000","last_unstale":"2024-04-02T13:37:17.627781+0000","last_undegraded":"2024-04-02T13:37:17.627781+0000","last_fullsized":"2024-04-02T13:37:17.627781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:47.070493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.871284+0000","last_change":"2024-04-02T13:36:21.493209+0000","last_active":"2024-04-02T13:37:17.871284+0000","last_peered":"2024-04-02T13:37:17.871284+0000","last_clean":"2024-04-02T13:37:17.871284+0000","last_became_active":"2024-04-02T13:36:21.492894+0000","last_became_peered":"2024-04-02T13:36:21.492894+0000","last_unstale":"2024-04-02T13:37:17.871284+0000","last_undegraded":"2024-04-02T13:37:17.871284+0000","last_fullsized":"2024-04-02T13:37:17.871284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:47.162354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627474+0000","last_change":"2024-04-02T13:37:17.626933+0000","last_active":"2024-04-02T13:37:17.627474+0000","last_peered":"2024-04-02T13:37:17.627474+0000","last_clean":"2024-04-02T13:37:17.627474+0000","last_became_active":"2024-04-02T13:37:17.626411+0000","last_became_peered":"2024-04-02T13:37:17.626411+0000","last_unstale":"2024-04-02T13:37:17.627474+0000","last_undegraded":"2024-04-02T13:37:17.627474+0000","last_fullsized":"2024-04-02T13:37:17.627474+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:40:56.903871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"90'33","reported_seq":216,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:59.597701+0000","last_change":"2024-04-02T13:36:20.823563+0000","last_active":"2024-04-02T13:37:59.597701+0000","last_peered":"2024-04-02T13:37:59.597701+0000","last_clean":"2024-04-02T13:37:59.597701+0000","last_became_active":"2024-04-02T13:36:20.823186+0000","last_became_peered":"2024-04-02T13:36:20.823186+0000","last_unstale":"2024-04-02T13:37:59.597701+0000","last_undegraded":"2024-04-02T13:37:59.597701+0000","last_fullsized":"2024-04-02T13:37:59.597701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:50.983357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63356,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":156,"num_read_kb":140,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620705+0000","last_change":"2024-04-02T13:36:21.500829+0000","last_active":"2024-04-02T13:37:17.620705+0000","last_peered":"2024-04-02T13:37:17.620705+0000","last_clean":"2024-04-02T13:37:17.620705+0000","last_became_active":"2024-04-02T13:36:21.500485+0000","last_became_peered":"2024-04-02T13:36:21.500485+0000","last_unstale":"2024-04-02T13:37:17.620705+0000","last_undegraded":"2024-04-02T13:37:17.620705+0000","last_fullsized":"2024-04-02T13:37:17.620705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:23:19.627565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628350+0000","last_change":"2024-04-02T13:36:25.506695+0000","last_active":"2024-04-02T13:37:17.628350+0000","last_peered":"2024-04-02T13:37:17.628350+0000","last_clean":"2024-04-02T13:37:17.628350+0000","last_became_active":"2024-04-02T13:36:25.506254+0000","last_became_peered":"2024-04-02T13:36:25.506254+0000","last_unstale":"2024-04-02T13:37:17.628350+0000","last_undegraded":"2024-04-02T13:37:17.628350+0000","last_fullsized":"2024-04-02T13:37:17.628350+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:57:00.315005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627963+0000","last_change":"2024-04-02T13:36:20.825827+0000","last_active":"2024-04-02T13:37:17.627963+0000","last_peered":"2024-04-02T13:37:17.627963+0000","last_clean":"2024-04-02T13:37:17.627963+0000","last_became_active":"2024-04-02T13:36:20.825530+0000","last_became_peered":"2024-04-02T13:36:20.825530+0000","last_unstale":"2024-04-02T13:37:17.627963+0000","last_undegraded":"2024-04-02T13:37:17.627963+0000","last_fullsized":"2024-04-02T13:37:17.627963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:27:02.962513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627435+0000","last_change":"2024-04-02T13:36:21.486343+0000","last_active":"2024-04-02T13:37:17.627435+0000","last_peered":"2024-04-02T13:37:17.627435+0000","last_clean":"2024-04-02T13:37:17.627435+0000","last_became_active":"2024-04-02T13:36:21.486021+0000","last_became_peered":"2024-04-02T13:36:21.486021+0000","last_unstale":"2024-04-02T13:37:17.627435+0000","last_undegraded":"2024-04-02T13:37:17.627435+0000","last_fullsized":"2024-04-02T13:37:17.627435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:53:42.430173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.631430+0000","last_change":"2024-04-02T13:37:17.631430+0000","last_active":"2024-04-02T13:37:17.631430+0000","last_peered":"2024-04-02T13:37:17.631430+0000","last_clean":"2024-04-02T13:37:17.631430+0000","last_became_active":"2024-04-02T13:37:17.630930+0000","last_became_peered":"2024-04-02T13:37:17.630930+0000","last_unstale":"2024-04-02T13:37:17.631430+0000","last_undegraded":"2024-04-02T13:37:17.631430+0000","last_fullsized":"2024-04-02T13:37:17.631430+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:10.514663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629023+0000","last_change":"2024-04-02T13:36:20.829303+0000","last_active":"2024-04-02T13:37:17.629023+0000","last_peered":"2024-04-02T13:37:17.629023+0000","last_clean":"2024-04-02T13:37:17.629023+0000","last_became_active":"2024-04-02T13:36:20.828952+0000","last_became_peered":"2024-04-02T13:36:20.828952+0000","last_unstale":"2024-04-02T13:37:17.629023+0000","last_undegraded":"2024-04-02T13:37:17.629023+0000","last_fullsized":"2024-04-02T13:37:17.629023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:14:04.385178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624883+0000","last_change":"2024-04-02T13:36:21.491767+0000","last_active":"2024-04-02T13:37:17.624883+0000","last_peered":"2024-04-02T13:37:17.624883+0000","last_clean":"2024-04-02T13:37:17.624883+0000","last_became_active":"2024-04-02T13:36:21.491369+0000","last_became_peered":"2024-04-02T13:36:21.491369+0000","last_unstale":"2024-04-02T13:37:17.624883+0000","last_undegraded":"2024-04-02T13:37:17.624883+0000","last_fullsized":"2024-04-02T13:37:17.624883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:14:35.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.623337+0000","last_change":"2024-04-02T13:36:25.503900+0000","last_active":"2024-04-02T13:37:17.623337+0000","last_peered":"2024-04-02T13:37:17.623337+0000","last_clean":"2024-04-02T13:37:17.623337+0000","last_became_active":"2024-04-02T13:36:25.502840+0000","last_became_peered":"2024-04-02T13:36:25.502840+0000","last_unstale":"2024-04-02T13:37:17.623337+0000","last_undegraded":"2024-04-02T13:37:17.623337+0000","last_fullsized":"2024-04-02T13:37:17.623337+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:31.106865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:59.629299+0000","last_change":"2024-04-02T13:36:20.823753+0000","last_active":"2024-04-02T13:37:59.629299+0000","last_peered":"2024-04-02T13:37:59.629299+0000","last_clean":"2024-04-02T13:37:59.629299+0000","last_became_active":"2024-04-02T13:36:20.823394+0000","last_became_peered":"2024-04-02T13:36:20.823394+0000","last_unstale":"2024-04-02T13:37:59.629299+0000","last_undegraded":"2024-04-02T13:37:59.629299+0000","last_fullsized":"2024-04-02T13:37:59.629299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:13.023125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625364+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:37:17.625364+0000","last_peered":"2024-04-02T13:37:17.625364+0000","last_clean":"2024-04-02T13:37:17.625364+0000","last_became_active":"2024-04-02T13:36:21.499197+0000","last_became_peered":"2024-04-02T13:36:21.499197+0000","last_unstale":"2024-04-02T13:37:17.625364+0000","last_undegraded":"2024-04-02T13:37:17.625364+0000","last_fullsized":"2024-04-02T13:37:17.625364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:26.435832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626882+0000","last_change":"2024-04-02T13:36:25.504643+0000","last_active":"2024-04-02T13:37:17.626882+0000","last_peered":"2024-04-02T13:37:17.626882+0000","last_clean":"2024-04-02T13:37:17.626882+0000","last_became_active":"2024-04-02T13:36:25.504249+0000","last_became_peered":"2024-04-02T13:36:25.504249+0000","last_unstale":"2024-04-02T13:37:17.626882+0000","last_undegraded":"2024-04-02T13:37:17.626882+0000","last_fullsized":"2024-04-02T13:37:17.626882+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:15.818958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.619628+0000","last_change":"2024-04-02T13:36:20.830097+0000","last_active":"2024-04-02T13:37:17.619628+0000","last_peered":"2024-04-02T13:37:17.619628+0000","last_clean":"2024-04-02T13:37:17.619628+0000","last_became_active":"2024-04-02T13:36:20.829779+0000","last_became_peered":"2024-04-02T13:36:20.829779+0000","last_unstale":"2024-04-02T13:37:17.619628+0000","last_undegraded":"2024-04-02T13:37:17.619628+0000","last_fullsized":"2024-04-02T13:37:17.619628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:59.477730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625384+0000","last_change":"2024-04-02T13:36:21.497480+0000","last_active":"2024-04-02T13:37:17.625384+0000","last_peered":"2024-04-02T13:37:17.625384+0000","last_clean":"2024-04-02T13:37:17.625384+0000","last_became_active":"2024-04-02T13:36:21.497106+0000","last_became_peered":"2024-04-02T13:36:21.497106+0000","last_unstale":"2024-04-02T13:37:17.625384+0000","last_undegraded":"2024-04-02T13:37:17.625384+0000","last_fullsized":"2024-04-02T13:37:17.625384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:30.394968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625245+0000","last_change":"2024-04-02T13:36:25.500040+0000","last_active":"2024-04-02T13:37:17.625245+0000","last_peered":"2024-04-02T13:37:17.625245+0000","last_clean":"2024-04-02T13:37:17.625245+0000","last_became_active":"2024-04-02T13:36:25.499576+0000","last_became_peered":"2024-04-02T13:36:25.499576+0000","last_unstale":"2024-04-02T13:37:17.625245+0000","last_undegraded":"2024-04-02T13:37:17.625245+0000","last_fullsized":"2024-04-02T13:37:17.625245+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:15:47.475234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620515+0000","last_change":"2024-04-02T13:36:20.823225+0000","last_active":"2024-04-02T13:37:17.620515+0000","last_peered":"2024-04-02T13:37:17.620515+0000","last_clean":"2024-04-02T13:37:17.620515+0000","last_became_active":"2024-04-02T13:36:20.822865+0000","last_became_peered":"2024-04-02T13:36:20.822865+0000","last_unstale":"2024-04-02T13:37:17.620515+0000","last_undegraded":"2024-04-02T13:37:17.620515+0000","last_fullsized":"2024-04-02T13:37:17.620515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:00.488190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622757+0000","last_change":"2024-04-02T13:36:21.497692+0000","last_active":"2024-04-02T13:37:17.622757+0000","last_peered":"2024-04-02T13:37:17.622757+0000","last_clean":"2024-04-02T13:37:17.622757+0000","last_became_active":"2024-04-02T13:36:21.497322+0000","last_became_peered":"2024-04-02T13:36:21.497322+0000","last_unstale":"2024-04-02T13:37:17.622757+0000","last_undegraded":"2024-04-02T13:37:17.622757+0000","last_fullsized":"2024-04-02T13:37:17.622757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:17.590235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.872416+0000","last_change":"2024-04-02T13:37:17.872416+0000","last_active":"2024-04-02T13:37:17.872416+0000","last_peered":"2024-04-02T13:37:17.872416+0000","last_clean":"2024-04-02T13:37:17.872416+0000","last_became_active":"2024-04-02T13:37:17.871932+0000","last_became_peered":"2024-04-02T13:37:17.871932+0000","last_unstale":"2024-04-02T13:37:17.872416+0000","last_undegraded":"2024-04-02T13:37:17.872416+0000","last_fullsized":"2024-04-02T13:37:17.872416+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:57:23.575737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622502+0000","last_change":"2024-04-02T13:36:20.835277+0000","last_active":"2024-04-02T13:37:17.622502+0000","last_peered":"2024-04-02T13:37:17.622502+0000","last_clean":"2024-04-02T13:37:17.622502+0000","last_became_active":"2024-04-02T13:36:20.834911+0000","last_became_peered":"2024-04-02T13:36:20.834911+0000","last_unstale":"2024-04-02T13:37:17.622502+0000","last_undegraded":"2024-04-02T13:37:17.622502+0000","last_fullsized":"2024-04-02T13:37:17.622502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:20.965544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620276+0000","last_change":"2024-04-02T13:36:21.495274+0000","last_active":"2024-04-02T13:37:17.620276+0000","last_peered":"2024-04-02T13:37:17.620276+0000","last_clean":"2024-04-02T13:37:17.620276+0000","last_became_active":"2024-04-02T13:36:21.494841+0000","last_became_peered":"2024-04-02T13:36:21.494841+0000","last_unstale":"2024-04-02T13:37:17.620276+0000","last_undegraded":"2024-04-02T13:37:17.620276+0000","last_fullsized":"2024-04-02T13:37:17.620276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:20.096507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627890+0000","last_change":"2024-04-02T13:36:25.499970+0000","last_active":"2024-04-02T13:37:17.627890+0000","last_peered":"2024-04-02T13:37:17.627890+0000","last_clean":"2024-04-02T13:37:17.627890+0000","last_became_active":"2024-04-02T13:36:25.499539+0000","last_became_peered":"2024-04-02T13:36:25.499539+0000","last_unstale":"2024-04-02T13:37:17.627890+0000","last_undegraded":"2024-04-02T13:37:17.627890+0000","last_fullsized":"2024-04-02T13:37:17.627890+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:43.136635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624544+0000","last_change":"2024-04-02T13:36:20.831150+0000","last_active":"2024-04-02T13:37:17.624544+0000","last_peered":"2024-04-02T13:37:17.624544+0000","last_clean":"2024-04-02T13:37:17.624544+0000","last_became_active":"2024-04-02T13:36:20.830838+0000","last_became_peered":"2024-04-02T13:36:20.830838+0000","last_unstale":"2024-04-02T13:37:17.624544+0000","last_undegraded":"2024-04-02T13:37:17.624544+0000","last_fullsized":"2024-04-02T13:37:17.624544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:23:09.566911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620004+0000","last_change":"2024-04-02T13:36:21.503128+0000","last_active":"2024-04-02T13:37:17.620004+0000","last_peered":"2024-04-02T13:37:17.620004+0000","last_clean":"2024-04-02T13:37:17.620004+0000","last_became_active":"2024-04-02T13:36:21.502695+0000","last_became_peered":"2024-04-02T13:36:21.502695+0000","last_unstale":"2024-04-02T13:37:17.620004+0000","last_undegraded":"2024-04-02T13:37:17.620004+0000","last_fullsized":"2024-04-02T13:37:17.620004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:38.261472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626017+0000","last_change":"2024-04-02T13:36:25.530911+0000","last_active":"2024-04-02T13:37:17.626017+0000","last_peered":"2024-04-02T13:37:17.626017+0000","last_clean":"2024-04-02T13:37:17.626017+0000","last_became_active":"2024-04-02T13:36:25.530428+0000","last_became_peered":"2024-04-02T13:36:25.530428+0000","last_unstale":"2024-04-02T13:37:17.626017+0000","last_undegraded":"2024-04-02T13:37:17.626017+0000","last_fullsized":"2024-04-02T13:37:17.626017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:49.185987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625224+0000","last_change":"2024-04-02T13:36:21.492251+0000","last_active":"2024-04-02T13:37:17.625224+0000","last_peered":"2024-04-02T13:37:17.625224+0000","last_clean":"2024-04-02T13:37:17.625224+0000","last_became_active":"2024-04-02T13:36:21.491929+0000","last_became_peered":"2024-04-02T13:36:21.491929+0000","last_unstale":"2024-04-02T13:37:17.625224+0000","last_undegraded":"2024-04-02T13:37:17.625224+0000","last_fullsized":"2024-04-02T13:37:17.625224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:59.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:38:00.630538+0000","last_change":"2024-04-02T13:36:20.821431+0000","last_active":"2024-04-02T13:38:00.630538+0000","last_peered":"2024-04-02T13:38:00.630538+0000","last_clean":"2024-04-02T13:38:00.630538+0000","last_became_active":"2024-04-02T13:36:20.821067+0000","last_became_peered":"2024-04-02T13:36:20.821067+0000","last_unstale":"2024-04-02T13:38:00.630538+0000","last_undegraded":"2024-04-02T13:38:00.630538+0000","last_fullsized":"2024-04-02T13:38:00.630538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:52.043907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624971+0000","last_change":"2024-04-02T13:36:25.496992+0000","last_active":"2024-04-02T13:37:17.624971+0000","last_peered":"2024-04-02T13:37:17.624971+0000","last_clean":"2024-04-02T13:37:17.624971+0000","last_became_active":"2024-04-02T13:36:25.496467+0000","last_became_peered":"2024-04-02T13:36:25.496467+0000","last_unstale":"2024-04-02T13:37:17.624971+0000","last_undegraded":"2024-04-02T13:37:17.624971+0000","last_fullsized":"2024-04-02T13:37:17.624971+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:13:00.747716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628776+0000","last_change":"2024-04-02T13:36:21.491399+0000","last_active":"2024-04-02T13:37:17.628776+0000","last_peered":"2024-04-02T13:37:17.628776+0000","last_clean":"2024-04-02T13:37:17.628776+0000","last_became_active":"2024-04-02T13:36:21.491038+0000","last_became_peered":"2024-04-02T13:36:21.491038+0000","last_unstale":"2024-04-02T13:37:17.628776+0000","last_undegraded":"2024-04-02T13:37:17.628776+0000","last_fullsized":"2024-04-02T13:37:17.628776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:25.603086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.843624+0000","last_change":"2024-04-02T13:36:20.822796+0000","last_active":"2024-04-02T13:37:17.843624+0000","last_peered":"2024-04-02T13:37:17.843624+0000","last_clean":"2024-04-02T13:37:17.843624+0000","last_became_active":"2024-04-02T13:36:20.822168+0000","last_became_peered":"2024-04-02T13:36:20.822168+0000","last_unstale":"2024-04-02T13:37:17.843624+0000","last_undegraded":"2024-04-02T13:37:17.843624+0000","last_fullsized":"2024-04-02T13:37:17.843624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:11.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625281+0000","last_change":"2024-04-02T13:35:08.678967+0000","last_active":"2024-04-02T13:37:17.625281+0000","last_peered":"2024-04-02T13:37:17.625281+0000","last_clean":"2024-04-02T13:37:17.625281+0000","last_became_active":"2024-04-02T13:35:08.678408+0000","last_became_peered":"2024-04-02T13:35:08.678408+0000","last_unstale":"2024-04-02T13:37:17.625281+0000","last_undegraded":"2024-04-02T13:37:17.625281+0000","last_fullsized":"2024-04-02T13:37:17.625281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_clean_scrub_stamp":"2024-04-02T13:31:42.453938+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:30:20.020809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.626040+0000","last_change":"2024-04-02T13:36:25.532723+0000","last_active":"2024-04-02T13:37:17.626040+0000","last_peered":"2024-04-02T13:37:17.626040+0000","last_clean":"2024-04-02T13:37:17.626040+0000","last_became_active":"2024-04-02T13:36:25.532341+0000","last_became_peered":"2024-04-02T13:36:25.532341+0000","last_unstale":"2024-04-02T13:37:17.626040+0000","last_undegraded":"2024-04-02T13:37:17.626040+0000","last_fullsized":"2024-04-02T13:37:17.626040+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:53:43.190339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627018+0000","last_change":"2024-04-02T13:36:21.488983+0000","last_active":"2024-04-02T13:37:17.627018+0000","last_peered":"2024-04-02T13:37:17.627018+0000","last_clean":"2024-04-02T13:37:17.627018+0000","last_became_active":"2024-04-02T13:36:21.488580+0000","last_became_peered":"2024-04-02T13:36:21.488580+0000","last_unstale":"2024-04-02T13:37:17.627018+0000","last_undegraded":"2024-04-02T13:37:17.627018+0000","last_fullsized":"2024-04-02T13:37:17.627018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:39.382233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.634386+0000","last_change":"2024-04-02T13:37:17.633840+0000","last_active":"2024-04-02T13:37:17.634386+0000","last_peered":"2024-04-02T13:37:17.634386+0000","last_clean":"2024-04-02T13:37:17.634386+0000","last_became_active":"2024-04-02T13:37:17.626842+0000","last_became_peered":"2024-04-02T13:37:17.626842+0000","last_unstale":"2024-04-02T13:37:17.634386+0000","last_undegraded":"2024-04-02T13:37:17.634386+0000","last_fullsized":"2024-04-02T13:37:17.634386+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:55:17.437862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622442+0000","last_change":"2024-04-02T13:36:25.502814+0000","last_active":"2024-04-02T13:37:17.622442+0000","last_peered":"2024-04-02T13:37:17.622442+0000","last_clean":"2024-04-02T13:37:17.622442+0000","last_became_active":"2024-04-02T13:36:25.502288+0000","last_became_peered":"2024-04-02T13:36:25.502288+0000","last_unstale":"2024-04-02T13:37:17.622442+0000","last_undegraded":"2024-04-02T13:37:17.622442+0000","last_fullsized":"2024-04-02T13:37:17.622442+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:48.073286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.620631+0000","last_change":"2024-04-02T13:36:20.830512+0000","last_active":"2024-04-02T13:37:17.620631+0000","last_peered":"2024-04-02T13:37:17.620631+0000","last_clean":"2024-04-02T13:37:17.620631+0000","last_became_active":"2024-04-02T13:36:20.830086+0000","last_became_peered":"2024-04-02T13:36:20.830086+0000","last_unstale":"2024-04-02T13:37:17.620631+0000","last_undegraded":"2024-04-02T13:37:17.620631+0000","last_fullsized":"2024-04-02T13:37:17.620631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:07.428363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628010+0000","last_change":"2024-04-02T13:36:21.478218+0000","last_active":"2024-04-02T13:37:17.628010+0000","last_peered":"2024-04-02T13:37:17.628010+0000","last_clean":"2024-04-02T13:37:17.628010+0000","last_became_active":"2024-04-02T13:36:21.477856+0000","last_became_peered":"2024-04-02T13:36:21.477856+0000","last_unstale":"2024-04-02T13:37:17.628010+0000","last_undegraded":"2024-04-02T13:37:17.628010+0000","last_fullsized":"2024-04-02T13:37:17.628010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:04.397606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629482+0000","last_change":"2024-04-02T13:36:25.530915+0000","last_active":"2024-04-02T13:37:17.629482+0000","last_peered":"2024-04-02T13:37:17.629482+0000","last_clean":"2024-04-02T13:37:17.629482+0000","last_became_active":"2024-04-02T13:36:25.530343+0000","last_became_peered":"2024-04-02T13:36:25.530343+0000","last_unstale":"2024-04-02T13:37:17.629482+0000","last_undegraded":"2024-04-02T13:37:17.629482+0000","last_fullsized":"2024-04-02T13:37:17.629482+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:32:41.181485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622371+0000","last_change":"2024-04-02T13:36:20.828527+0000","last_active":"2024-04-02T13:37:17.622371+0000","last_peered":"2024-04-02T13:37:17.622371+0000","last_clean":"2024-04-02T13:37:17.622371+0000","last_became_active":"2024-04-02T13:36:20.828136+0000","last_became_peered":"2024-04-02T13:36:20.828136+0000","last_unstale":"2024-04-02T13:37:17.622371+0000","last_undegraded":"2024-04-02T13:37:17.622371+0000","last_fullsized":"2024-04-02T13:37:17.622371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:57.709400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628777+0000","last_change":"2024-04-02T13:36:21.492035+0000","last_active":"2024-04-02T13:37:17.628777+0000","last_peered":"2024-04-02T13:37:17.628777+0000","last_clean":"2024-04-02T13:37:17.628777+0000","last_became_active":"2024-04-02T13:36:21.491682+0000","last_became_peered":"2024-04-02T13:36:21.491682+0000","last_unstale":"2024-04-02T13:37:17.628777+0000","last_undegraded":"2024-04-02T13:37:17.628777+0000","last_fullsized":"2024-04-02T13:37:17.628777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:09:40.600616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.630183+0000","last_change":"2024-04-02T13:36:25.493301+0000","last_active":"2024-04-02T13:37:17.630183+0000","last_peered":"2024-04-02T13:37:17.630183+0000","last_clean":"2024-04-02T13:37:17.630183+0000","last_became_active":"2024-04-02T13:36:25.492846+0000","last_became_peered":"2024-04-02T13:36:25.492846+0000","last_unstale":"2024-04-02T13:37:17.630183+0000","last_undegraded":"2024-04-02T13:37:17.630183+0000","last_fullsized":"2024-04-02T13:37:17.630183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:12:53.350261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.844269+0000","last_change":"2024-04-02T13:36:20.823559+0000","last_active":"2024-04-02T13:37:17.844269+0000","last_peered":"2024-04-02T13:37:17.844269+0000","last_clean":"2024-04-02T13:37:17.844269+0000","last_became_active":"2024-04-02T13:36:20.823178+0000","last_became_peered":"2024-04-02T13:36:20.823178+0000","last_unstale":"2024-04-02T13:37:17.844269+0000","last_undegraded":"2024-04-02T13:37:17.844269+0000","last_fullsized":"2024-04-02T13:37:17.844269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:42.628855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627447+0000","last_change":"2024-04-02T13:36:21.497258+0000","last_active":"2024-04-02T13:37:17.627447+0000","last_peered":"2024-04-02T13:37:17.627447+0000","last_clean":"2024-04-02T13:37:17.627447+0000","last_became_active":"2024-04-02T13:36:21.496943+0000","last_became_peered":"2024-04-02T13:36:21.496943+0000","last_unstale":"2024-04-02T13:37:17.627447+0000","last_undegraded":"2024-04-02T13:37:17.627447+0000","last_fullsized":"2024-04-02T13:37:17.627447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:51.989025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.843952+0000","last_change":"2024-04-02T13:36:25.531251+0000","last_active":"2024-04-02T13:37:17.843952+0000","last_peered":"2024-04-02T13:37:17.843952+0000","last_clean":"2024-04-02T13:37:17.843952+0000","last_became_active":"2024-04-02T13:36:25.530505+0000","last_became_peered":"2024-04-02T13:36:25.530505+0000","last_unstale":"2024-04-02T13:37:17.843952+0000","last_undegraded":"2024-04-02T13:37:17.843952+0000","last_fullsized":"2024-04-02T13:37:17.843952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:02:10.711138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.870569+0000","last_change":"2024-04-02T13:36:20.837865+0000","last_active":"2024-04-02T13:37:17.870569+0000","last_peered":"2024-04-02T13:37:17.870569+0000","last_clean":"2024-04-02T13:37:17.870569+0000","last_became_active":"2024-04-02T13:36:20.837567+0000","last_became_peered":"2024-04-02T13:36:20.837567+0000","last_unstale":"2024-04-02T13:37:17.870569+0000","last_undegraded":"2024-04-02T13:37:17.870569+0000","last_fullsized":"2024-04-02T13:37:17.870569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:11:59.365592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.619652+0000","last_change":"2024-04-02T13:36:21.500085+0000","last_active":"2024-04-02T13:37:17.619652+0000","last_peered":"2024-04-02T13:37:17.619652+0000","last_clean":"2024-04-02T13:37:17.619652+0000","last_became_active":"2024-04-02T13:36:21.499706+0000","last_became_peered":"2024-04-02T13:36:21.499706+0000","last_unstale":"2024-04-02T13:37:17.619652+0000","last_undegraded":"2024-04-02T13:37:17.619652+0000","last_fullsized":"2024-04-02T13:37:17.619652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.378226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.623475+0000","last_change":"2024-04-02T13:36:25.505797+0000","last_active":"2024-04-02T13:37:17.623475+0000","last_peered":"2024-04-02T13:37:17.623475+0000","last_clean":"2024-04-02T13:37:17.623475+0000","last_became_active":"2024-04-02T13:36:25.505409+0000","last_became_peered":"2024-04-02T13:36:25.505409+0000","last_unstale":"2024-04-02T13:37:17.623475+0000","last_undegraded":"2024-04-02T13:37:17.623475+0000","last_fullsized":"2024-04-02T13:37:17.623475+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:46:31.723248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629127+0000","last_change":"2024-04-02T13:36:20.821572+0000","last_active":"2024-04-02T13:37:17.629127+0000","last_peered":"2024-04-02T13:37:17.629127+0000","last_clean":"2024-04-02T13:37:17.629127+0000","last_became_active":"2024-04-02T13:36:20.821235+0000","last_became_peered":"2024-04-02T13:36:20.821235+0000","last_unstale":"2024-04-02T13:37:17.629127+0000","last_undegraded":"2024-04-02T13:37:17.629127+0000","last_fullsized":"2024-04-02T13:37:17.629127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:02.304145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.623246+0000","last_change":"2024-04-02T13:36:21.485255+0000","last_active":"2024-04-02T13:37:17.623246+0000","last_peered":"2024-04-02T13:37:17.623246+0000","last_clean":"2024-04-02T13:37:17.623246+0000","last_became_active":"2024-04-02T13:36:21.484870+0000","last_became_peered":"2024-04-02T13:36:21.484870+0000","last_unstale":"2024-04-02T13:37:17.623246+0000","last_undegraded":"2024-04-02T13:37:17.623246+0000","last_fullsized":"2024-04-02T13:37:17.623246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:03.851161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.844001+0000","last_change":"2024-04-02T13:36:25.494161+0000","last_active":"2024-04-02T13:37:17.844001+0000","last_peered":"2024-04-02T13:37:17.844001+0000","last_clean":"2024-04-02T13:37:17.844001+0000","last_became_active":"2024-04-02T13:36:25.493667+0000","last_became_peered":"2024-04-02T13:36:25.493667+0000","last_unstale":"2024-04-02T13:37:17.844001+0000","last_undegraded":"2024-04-02T13:37:17.844001+0000","last_fullsized":"2024-04-02T13:37:17.844001+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:35:45.703988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.636702+0000","last_change":"2024-04-02T13:37:17.636209+0000","last_active":"2024-04-02T13:37:17.636702+0000","last_peered":"2024-04-02T13:37:17.636702+0000","last_clean":"2024-04-02T13:37:17.636702+0000","last_became_active":"2024-04-02T13:37:17.631539+0000","last_became_peered":"2024-04-02T13:37:17.631539+0000","last_unstale":"2024-04-02T13:37:17.636702+0000","last_undegraded":"2024-04-02T13:37:17.636702+0000","last_fullsized":"2024-04-02T13:37:17.636702+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:05:52.619779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.630094+0000","last_change":"2024-04-02T13:36:21.498640+0000","last_active":"2024-04-02T13:37:17.630094+0000","last_peered":"2024-04-02T13:37:17.630094+0000","last_clean":"2024-04-02T13:37:17.630094+0000","last_became_active":"2024-04-02T13:36:21.497996+0000","last_became_peered":"2024-04-02T13:36:21.497996+0000","last_unstale":"2024-04-02T13:37:17.630094+0000","last_undegraded":"2024-04-02T13:37:17.630094+0000","last_fullsized":"2024-04-02T13:37:17.630094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:56:45.748945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.844057+0000","last_change":"2024-04-02T13:36:25.531119+0000","last_active":"2024-04-02T13:37:17.844057+0000","last_peered":"2024-04-02T13:37:17.844057+0000","last_clean":"2024-04-02T13:37:17.844057+0000","last_became_active":"2024-04-02T13:36:25.530307+0000","last_became_peered":"2024-04-02T13:36:25.530307+0000","last_unstale":"2024-04-02T13:37:17.844057+0000","last_undegraded":"2024-04-02T13:37:17.844057+0000","last_fullsized":"2024-04-02T13:37:17.844057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:58:50.543386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.619892+0000","last_change":"2024-04-02T13:36:20.829843+0000","last_active":"2024-04-02T13:37:17.619892+0000","last_peered":"2024-04-02T13:37:17.619892+0000","last_clean":"2024-04-02T13:37:17.619892+0000","last_became_active":"2024-04-02T13:36:20.829502+0000","last_became_peered":"2024-04-02T13:36:20.829502+0000","last_unstale":"2024-04-02T13:37:17.619892+0000","last_undegraded":"2024-04-02T13:37:17.619892+0000","last_fullsized":"2024-04-02T13:37:17.619892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:36:45.016174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624588+0000","last_change":"2024-04-02T13:36:21.491579+0000","last_active":"2024-04-02T13:37:17.624588+0000","last_peered":"2024-04-02T13:37:17.624588+0000","last_clean":"2024-04-02T13:37:17.624588+0000","last_became_active":"2024-04-02T13:36:21.490992+0000","last_became_peered":"2024-04-02T13:36:21.490992+0000","last_unstale":"2024-04-02T13:37:17.624588+0000","last_undegraded":"2024-04-02T13:37:17.624588+0000","last_fullsized":"2024-04-02T13:37:17.624588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:41.899766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.870929+0000","last_change":"2024-04-02T13:36:25.502658+0000","last_active":"2024-04-02T13:37:17.870929+0000","last_peered":"2024-04-02T13:37:17.870929+0000","last_clean":"2024-04-02T13:37:17.870929+0000","last_became_active":"2024-04-02T13:36:25.502219+0000","last_became_peered":"2024-04-02T13:36:25.502219+0000","last_unstale":"2024-04-02T13:37:17.870929+0000","last_undegraded":"2024-04-02T13:37:17.870929+0000","last_fullsized":"2024-04-02T13:37:17.870929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:34.189381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628406+0000","last_change":"2024-04-02T13:36:20.835176+0000","last_active":"2024-04-02T13:37:17.628406+0000","last_peered":"2024-04-02T13:37:17.628406+0000","last_clean":"2024-04-02T13:37:17.628406+0000","last_became_active":"2024-04-02T13:36:20.834871+0000","last_became_peered":"2024-04-02T13:36:20.834871+0000","last_unstale":"2024-04-02T13:37:17.628406+0000","last_undegraded":"2024-04-02T13:37:17.628406+0000","last_fullsized":"2024-04-02T13:37:17.628406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:17:14.148762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625234+0000","last_change":"2024-04-02T13:36:21.490336+0000","last_active":"2024-04-02T13:37:17.625234+0000","last_peered":"2024-04-02T13:37:17.625234+0000","last_clean":"2024-04-02T13:37:17.625234+0000","last_became_active":"2024-04-02T13:36:21.489869+0000","last_became_peered":"2024-04-02T13:36:21.489869+0000","last_unstale":"2024-04-02T13:37:17.625234+0000","last_undegraded":"2024-04-02T13:37:17.625234+0000","last_fullsized":"2024-04-02T13:37:17.625234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:42.243209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625720+0000","last_change":"2024-04-02T13:36:25.503124+0000","last_active":"2024-04-02T13:37:17.625720+0000","last_peered":"2024-04-02T13:37:17.625720+0000","last_clean":"2024-04-02T13:37:17.625720+0000","last_became_active":"2024-04-02T13:36:25.502737+0000","last_became_peered":"2024-04-02T13:36:25.502737+0000","last_unstale":"2024-04-02T13:37:17.625720+0000","last_undegraded":"2024-04-02T13:37:17.625720+0000","last_fullsized":"2024-04-02T13:37:17.625720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:02:56.830746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628083+0000","last_change":"2024-04-02T13:36:20.828597+0000","last_active":"2024-04-02T13:37:17.628083+0000","last_peered":"2024-04-02T13:37:17.628083+0000","last_clean":"2024-04-02T13:37:17.628083+0000","last_became_active":"2024-04-02T13:36:20.828138+0000","last_became_peered":"2024-04-02T13:36:20.828138+0000","last_unstale":"2024-04-02T13:37:17.628083+0000","last_undegraded":"2024-04-02T13:37:17.628083+0000","last_fullsized":"2024-04-02T13:37:17.628083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:05.171448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.624652+0000","last_change":"2024-04-02T13:36:21.498498+0000","last_active":"2024-04-02T13:37:17.624652+0000","last_peered":"2024-04-02T13:37:17.624652+0000","last_clean":"2024-04-02T13:37:17.624652+0000","last_became_active":"2024-04-02T13:36:21.498119+0000","last_became_peered":"2024-04-02T13:36:21.498119+0000","last_unstale":"2024-04-02T13:37:17.624652+0000","last_undegraded":"2024-04-02T13:37:17.624652+0000","last_fullsized":"2024-04-02T13:37:17.624652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:01.528228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.872082+0000","last_change":"2024-04-02T13:37:17.872082+0000","last_active":"2024-04-02T13:37:17.872082+0000","last_peered":"2024-04-02T13:37:17.872082+0000","last_clean":"2024-04-02T13:37:17.872082+0000","last_became_active":"2024-04-02T13:37:17.871562+0000","last_became_peered":"2024-04-02T13:37:17.871562+0000","last_unstale":"2024-04-02T13:37:17.872082+0000","last_undegraded":"2024-04-02T13:37:17.872082+0000","last_fullsized":"2024-04-02T13:37:17.872082+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:32:39.849512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.848534+0000","last_change":"2024-04-02T13:37:17.848534+0000","last_active":"2024-04-02T13:37:17.848534+0000","last_peered":"2024-04-02T13:37:17.848534+0000","last_clean":"2024-04-02T13:37:17.848534+0000","last_became_active":"2024-04-02T13:37:17.844875+0000","last_became_peered":"2024-04-02T13:37:17.844875+0000","last_unstale":"2024-04-02T13:37:17.848534+0000","last_undegraded":"2024-04-02T13:37:17.848534+0000","last_fullsized":"2024-04-02T13:37:17.848534+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:29:05.037405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625157+0000","last_change":"2024-04-02T13:36:21.494069+0000","last_active":"2024-04-02T13:37:17.625157+0000","last_peered":"2024-04-02T13:37:17.625157+0000","last_clean":"2024-04-02T13:37:17.625157+0000","last_became_active":"2024-04-02T13:36:21.493751+0000","last_became_peered":"2024-04-02T13:36:21.493751+0000","last_unstale":"2024-04-02T13:37:17.625157+0000","last_undegraded":"2024-04-02T13:37:17.625157+0000","last_fullsized":"2024-04-02T13:37:17.625157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:24:39.648318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.623317+0000","last_change":"2024-04-02T13:36:21.494280+0000","last_active":"2024-04-02T13:37:17.623317+0000","last_peered":"2024-04-02T13:37:17.623317+0000","last_clean":"2024-04-02T13:37:17.623317+0000","last_became_active":"2024-04-02T13:36:21.491262+0000","last_became_peered":"2024-04-02T13:36:21.491262+0000","last_unstale":"2024-04-02T13:37:17.623317+0000","last_undegraded":"2024-04-02T13:37:17.623317+0000","last_fullsized":"2024-04-02T13:37:17.623317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:08:16.728646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.630422+0000","last_change":"2024-04-02T13:36:21.497103+0000","last_active":"2024-04-02T13:37:17.630422+0000","last_peered":"2024-04-02T13:37:17.630422+0000","last_clean":"2024-04-02T13:37:17.630422+0000","last_became_active":"2024-04-02T13:36:21.496784+0000","last_became_peered":"2024-04-02T13:36:21.496784+0000","last_unstale":"2024-04-02T13:37:17.630422+0000","last_undegraded":"2024-04-02T13:37:17.630422+0000","last_fullsized":"2024-04-02T13:37:17.630422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:38:21.903362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627624+0000","last_change":"2024-04-02T13:36:21.491853+0000","last_active":"2024-04-02T13:37:17.627624+0000","last_peered":"2024-04-02T13:37:17.627624+0000","last_clean":"2024-04-02T13:37:17.627624+0000","last_became_active":"2024-04-02T13:36:21.487986+0000","last_became_peered":"2024-04-02T13:36:21.487986+0000","last_unstale":"2024-04-02T13:37:17.627624+0000","last_undegraded":"2024-04-02T13:37:17.627624+0000","last_fullsized":"2024-04-02T13:37:17.627624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:37:47.098681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.631648+0000","last_change":"2024-04-02T13:37:17.631648+0000","last_active":"2024-04-02T13:37:17.631648+0000","last_peered":"2024-04-02T13:37:17.631648+0000","last_clean":"2024-04-02T13:37:17.631648+0000","last_became_active":"2024-04-02T13:37:17.631133+0000","last_became_peered":"2024-04-02T13:37:17.631133+0000","last_unstale":"2024-04-02T13:37:17.631648+0000","last_undegraded":"2024-04-02T13:37:17.631648+0000","last_fullsized":"2024-04-02T13:37:17.631648+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:10.791151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.628154+0000","last_change":"2024-04-02T13:36:21.488723+0000","last_active":"2024-04-02T13:37:17.628154+0000","last_peered":"2024-04-02T13:37:17.628154+0000","last_clean":"2024-04-02T13:37:17.628154+0000","last_became_active":"2024-04-02T13:36:21.488391+0000","last_became_peered":"2024-04-02T13:36:21.488391+0000","last_unstale":"2024-04-02T13:37:17.628154+0000","last_undegraded":"2024-04-02T13:37:17.628154+0000","last_fullsized":"2024-04-02T13:37:17.628154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:23.768286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622972+0000","last_change":"2024-04-02T13:36:21.492676+0000","last_active":"2024-04-02T13:37:17.622972+0000","last_peered":"2024-04-02T13:37:17.622972+0000","last_clean":"2024-04-02T13:37:17.622972+0000","last_became_active":"2024-04-02T13:36:21.491663+0000","last_became_peered":"2024-04-02T13:36:21.491663+0000","last_unstale":"2024-04-02T13:37:17.622972+0000","last_undegraded":"2024-04-02T13:37:17.622972+0000","last_fullsized":"2024-04-02T13:37:17.622972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:00.131388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629336+0000","last_change":"2024-04-02T13:36:21.493380+0000","last_active":"2024-04-02T13:37:17.629336+0000","last_peered":"2024-04-02T13:37:17.629336+0000","last_clean":"2024-04-02T13:37:17.629336+0000","last_became_active":"2024-04-02T13:36:21.493073+0000","last_became_peered":"2024-04-02T13:36:21.493073+0000","last_unstale":"2024-04-02T13:37:17.629336+0000","last_undegraded":"2024-04-02T13:37:17.629336+0000","last_fullsized":"2024-04-02T13:37:17.629336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:59.412056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629236+0000","last_change":"2024-04-02T13:36:21.500881+0000","last_active":"2024-04-02T13:37:17.629236+0000","last_peered":"2024-04-02T13:37:17.629236+0000","last_clean":"2024-04-02T13:37:17.629236+0000","last_became_active":"2024-04-02T13:36:21.500516+0000","last_became_peered":"2024-04-02T13:36:21.500516+0000","last_unstale":"2024-04-02T13:37:17.629236+0000","last_undegraded":"2024-04-02T13:37:17.629236+0000","last_fullsized":"2024-04-02T13:37:17.629236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:33.842494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.625093+0000","last_change":"2024-04-02T13:36:21.494558+0000","last_active":"2024-04-02T13:37:17.625093+0000","last_peered":"2024-04-02T13:37:17.625093+0000","last_clean":"2024-04-02T13:37:17.625093+0000","last_became_active":"2024-04-02T13:36:21.494180+0000","last_became_peered":"2024-04-02T13:36:21.494180+0000","last_unstale":"2024-04-02T13:37:17.625093+0000","last_undegraded":"2024-04-02T13:37:17.625093+0000","last_fullsized":"2024-04-02T13:37:17.625093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:07:54.363504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.629262+0000","last_change":"2024-04-02T13:36:21.500406+0000","last_active":"2024-04-02T13:37:17.629262+0000","last_peered":"2024-04-02T13:37:17.629262+0000","last_clean":"2024-04-02T13:37:17.629262+0000","last_became_active":"2024-04-02T13:36:21.499927+0000","last_became_peered":"2024-04-02T13:36:21.499927+0000","last_unstale":"2024-04-02T13:37:17.629262+0000","last_undegraded":"2024-04-02T13:37:17.629262+0000","last_fullsized":"2024-04-02T13:37:17.629262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:23.702776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.871124+0000","last_change":"2024-04-02T13:36:21.486166+0000","last_active":"2024-04-02T13:37:17.871124+0000","last_peered":"2024-04-02T13:37:17.871124+0000","last_clean":"2024-04-02T13:37:17.871124+0000","last_became_active":"2024-04-02T13:36:21.485282+0000","last_became_peered":"2024-04-02T13:36:21.485282+0000","last_unstale":"2024-04-02T13:37:17.871124+0000","last_undegraded":"2024-04-02T13:37:17.871124+0000","last_fullsized":"2024-04-02T13:37:17.871124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:24:07.007675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.627898+0000","last_change":"2024-04-02T13:36:21.484909+0000","last_active":"2024-04-02T13:37:17.627898+0000","last_peered":"2024-04-02T13:37:17.627898+0000","last_clean":"2024-04-02T13:37:17.627898+0000","last_became_active":"2024-04-02T13:36:21.484559+0000","last_became_peered":"2024-04-02T13:36:21.484559+0000","last_unstale":"2024-04-02T13:37:17.627898+0000","last_undegraded":"2024-04-02T13:37:17.627898+0000","last_fullsized":"2024-04-02T13:37:17.627898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:59:45.848407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:37:17.622653+0000","last_change":"2024-04-02T13:36:21.493961+0000","last_active":"2024-04-02T13:37:17.622653+0000","last_peered":"2024-04-02T13:37:17.622653+0000","last_clean":"2024-04-02T13:37:17.622653+0000","last_became_active":"2024-04-02T13:36:21.493490+0000","last_became_peered":"2024-04-02T13:36:21.493490+0000","last_unstale":"2024-04-02T13:37:17.622653+0000","last_undegraded":"2024-04-02T13:37:17.622653+0000","last_fullsized":"2024-04-02T13:37:17.622653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:26:42.032550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":213880,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67748,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":473,"num_read_kb":309,"num_write":162,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":203244,"data_compressed":4818,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":503866,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612646,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":1154881,"data_compressed":58471,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":508198,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037809,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":503440,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":1100322,"data_compressed":56865,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462972,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":499278,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":563126,"data_compressed":22279,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":500666,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":564285,"data_compressed":22279,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":499472,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":504319,"data_compressed":20673,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":1095519,"data_compressed":56865,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34556,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34556,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63938,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65097,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65325,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:38:05.284 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-02T13:38:05.284 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-02T13:38:05.284 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-02T13:38:05.284 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph health --format=json 2024-04-02T13:38:05.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:05 smithi033 ceph-mon[32390]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:38:05.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:05 smithi033 ceph-mon[32390]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:05.537 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:38:05.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:05 smithi145 ceph-mon[35646]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:38:05.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:05 smithi145 ceph-mon[35646]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:05.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:05 smithi050 ceph-mon[35524]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:38:05.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:05 smithi050 ceph-mon[35524]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:07.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.276+0000 7f5df33f9700 1 -- 172.21.15.33:0/1134763371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dec06acf0 msgr2=0x7f5dec06b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:07.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.276+0000 7f5df33f9700 1 --2- 172.21.15.33:0/1134763371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dec06acf0 0x7f5dec06b170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0099c0 tx=0x7f5ddc009cd0 comp rx=0 tx=0).stop 2024-04-02T13:38:07.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.276+0000 7f5df33f9700 1 -- 172.21.15.33:0/1134763371 shutdown_connections 2024-04-02T13:38:07.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.276+0000 7f5df33f9700 1 --2- 172.21.15.33:0/1134763371 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5dec06b6b0 0x7f5dec10f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:07.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.276+0000 7f5df33f9700 1 --2- 172.21.15.33:0/1134763371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5dec108650 0x7f5dec108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:07.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.276+0000 7f5df33f9700 1 --2- 172.21.15.33:0/1134763371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dec06acf0 0x7f5dec06b170 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:07.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.276+0000 7f5df33f9700 1 -- 172.21.15.33:0/1134763371 >> 172.21.15.33:0/1134763371 conn(0x7f5dec077510 msgr2=0x7f5dec077920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:07.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.276+0000 7f5df33f9700 1 -- 172.21.15.33:0/1134763371 shutdown_connections 2024-04-02T13:38:07.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.276+0000 7f5df33f9700 1 -- 172.21.15.33:0/1134763371 wait complete. 2024-04-02T13:38:07.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.276+0000 7f5df33f9700 1 Processor -- start 2024-04-02T13:38:07.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df33f9700 1 -- start start 2024-04-02T13:38:07.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df33f9700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5dec06acf0 0x7f5dec19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:07.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df33f9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dec06b6b0 0x7f5dec19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df33f9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5dec108650 0x7f5dec19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df33f9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec112890 con 0x7f5dec06b6b0 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df33f9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec112710 con 0x7f5dec108650 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df33f9700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec112a10 con 0x7f5dec06acf0 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df0994700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dec06b6b0 0x7f5dec19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df0994700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dec06b6b0 0x7f5dec19b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50256/0 (socket says 172.21.15.33:50256) 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df0994700 1 -- 172.21.15.33:0/484251544 learned_addr learned my addr 172.21.15.33:0/484251544 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df1195700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5dec06acf0 0x7f5dec19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df1996700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5dec108650 0x7f5dec19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df0994700 1 -- 172.21.15.33:0/484251544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5dec06acf0 msgr2=0x7f5dec19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df0994700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5dec06acf0 0x7f5dec19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df0994700 1 -- 172.21.15.33:0/484251544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5dec108650 msgr2=0x7f5dec19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df0994700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5dec108650 0x7f5dec19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df0994700 1 -- 172.21.15.33:0/484251544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec19fec0 con 0x7f5dec06b6b0 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df1996700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5dec108650 0x7f5dec19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.277+0000 7f5df0994700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dec06b6b0 0x7f5dec19b3f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc006010 tx=0x7f5ddc003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.278+0000 7f5de27fc700 1 -- 172.21.15.33:0/484251544 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc009e60 con 0x7f5dec06b6b0 2024-04-02T13:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.278+0000 7f5de27fc700 1 -- 172.21.15.33:0/484251544 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5ddc005590 con 0x7f5dec06b6b0 2024-04-02T13:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.278+0000 7f5df33f9700 1 -- 172.21.15.33:0/484251544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec1a0150 con 0x7f5dec06b6b0 2024-04-02T13:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.278+0000 7f5de27fc700 1 -- 172.21.15.33:0/484251544 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc0173f0 con 0x7f5dec06b6b0 2024-04-02T13:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.278+0000 7f5df33f9700 1 -- 172.21.15.33:0/484251544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec1a7a30 con 0x7f5dec06b6b0 2024-04-02T13:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.279+0000 7f5df33f9700 1 -- 172.21.15.33:0/484251544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec103d20 con 0x7f5dec06b6b0 2024-04-02T13:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.280+0000 7f5de27fc700 1 -- 172.21.15.33:0/484251544 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ddc00f680 con 0x7f5dec06b6b0 2024-04-02T13:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.280+0000 7f5de27fc700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5dd80a3cd0 0x7f5dd80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.281+0000 7f5de27fc700 1 -- 172.21.15.33:0/484251544 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5ddc0d16c0 con 0x7f5dec06b6b0 2024-04-02T13:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.281+0000 7f5df1195700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5dd80a3cd0 0x7f5dd80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.281+0000 7f5df1195700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5dd80a3cd0 0x7f5dd80a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009680 tx=0x7f5de8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.284+0000 7f5de27fc700 1 -- 172.21.15.33:0/484251544 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5ddc013070 con 0x7f5dec06b6b0 2024-04-02T13:38:07.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.536+0000 7f5df33f9700 1 -- 172.21.15.33:0/484251544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5dec0692b0 con 0x7f5dec06b6b0 2024-04-02T13:38:07.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.536+0000 7f5de27fc700 1 -- 172.21.15.33:0/484251544 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5dec0692b0 con 0x7f5dec06b6b0 2024-04-02T13:38:07.539 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:07.539 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-02T13:38:07.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.538+0000 7f5df33f9700 1 -- 172.21.15.33:0/484251544 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5dd80a3cd0 msgr2=0x7f5dd80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:07.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.538+0000 7f5df33f9700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5dd80a3cd0 0x7f5dd80a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009680 tx=0x7f5de8009260 comp rx=0 tx=0).stop 2024-04-02T13:38:07.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.538+0000 7f5df33f9700 1 -- 172.21.15.33:0/484251544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dec06b6b0 msgr2=0x7f5dec19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:07.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.538+0000 7f5df33f9700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dec06b6b0 0x7f5dec19b3f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc006010 tx=0x7f5ddc003af0 comp rx=0 tx=0).stop 2024-04-02T13:38:07.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.538+0000 7f5df33f9700 1 -- 172.21.15.33:0/484251544 shutdown_connections 2024-04-02T13:38:07.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.538+0000 7f5df33f9700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5dd80a3cd0 0x7f5dd80a6190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:07.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.538+0000 7f5df33f9700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5dec108650 0x7f5dec19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:07.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.538+0000 7f5df33f9700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5dec06acf0 0x7f5dec19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:07.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.538+0000 7f5df33f9700 1 --2- 172.21.15.33:0/484251544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dec06b6b0 0x7f5dec19b3f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:07.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.538+0000 7f5df33f9700 1 -- 172.21.15.33:0/484251544 >> 172.21.15.33:0/484251544 conn(0x7f5dec077510 msgr2=0x7f5dec10d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:07.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.539+0000 7f5df33f9700 1 -- 172.21.15.33:0/484251544 shutdown_connections 2024-04-02T13:38:07.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:07.539+0000 7f5df33f9700 1 -- 172.21.15.33:0/484251544 wait complete. 2024-04-02T13:38:07.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:07 smithi145 ceph-mon[35646]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:38:07.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:07 smithi050 ceph-mon[35524]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:38:07.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:07 smithi033 ceph-mon[32390]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:38:08.301 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-02T13:38:08.301 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:38:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.411+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4037709387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63f01078e0 msgr2=0x7f63f0109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.411+0000 7f63f7ea5640 1 --2- 172.21.15.33:0/4037709387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63f01078e0 0x7f63f0109cd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009ca0 tx=0x7f63e802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.411+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4037709387 shutdown_connections 2024-04-02T13:38:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.411+0000 7f63f7ea5640 1 --2- 172.21.15.33:0/4037709387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63f01078e0 0x7f63f0109cd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.411+0000 7f63f7ea5640 1 --2- 172.21.15.33:0/4037709387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63f0104fb0 0x7f63f01073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.411+0000 7f63f7ea5640 1 --2- 172.21.15.33:0/4037709387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63f0102680 0x7f63f0104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.411+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4037709387 >> 172.21.15.33:0/4037709387 conn(0x7f63f00fc030 msgr2=0x7f63f00fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.411+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4037709387 shutdown_connections 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.411+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4037709387 wait complete. 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f7ea5640 1 Processor -- start 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f7ea5640 1 -- start start 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f7ea5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63f0102680 0x7f63f0197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f7ea5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63f0104fb0 0x7f63f0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f7ea5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63f01078e0 0x7f63f019f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f7ea5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f63f010d790 con 0x7f63f0102680 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f7ea5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f63f010d610 con 0x7f63f0104fb0 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f7ea5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f63f010d910 con 0x7f63f01078e0 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f5c41640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63f0102680 0x7f63f0197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f5c41640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63f0102680 0x7f63f0197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50272/0 (socket says 172.21.15.33:50272) 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f5c41640 1 -- 172.21.15.33:0/4208993388 learned_addr learned my addr 172.21.15.33:0/4208993388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f6442640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63f01078e0 0x7f63f019f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f5440640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63f0104fb0 0x7f63f0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f5c41640 1 -- 172.21.15.33:0/4208993388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63f01078e0 msgr2=0x7f63f019f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f5c41640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63f01078e0 0x7f63f019f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f5c41640 1 -- 172.21.15.33:0/4208993388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63f0104fb0 msgr2=0x7f63f0198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f5c41640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63f0104fb0 0x7f63f0198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f5c41640 1 -- 172.21.15.33:0/4208993388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f019f960 con 0x7f63f0102680 2024-04-02T13:38:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.412+0000 7f63f5440640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63f0104fb0 0x7f63f0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.413+0000 7f63f5c41640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63f0102680 0x7f63f0197d40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00c920 tx=0x7f63ec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.413+0000 7f63e6ffd640 1 -- 172.21.15.33:0/4208993388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec007c70 con 0x7f63f0102680 2024-04-02T13:38:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.413+0000 7f63e6ffd640 1 -- 172.21.15.33:0/4208993388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f63ec007e10 con 0x7f63f0102680 2024-04-02T13:38:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.413+0000 7f63e6ffd640 1 -- 172.21.15.33:0/4208993388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec018850 con 0x7f63f0102680 2024-04-02T13:38:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.413+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4208993388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f019fc50 con 0x7f63f0102680 2024-04-02T13:38:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.413+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4208993388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f01a0150 con 0x7f63f0102680 2024-04-02T13:38:08.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.414+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4208993388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f00fdb20 con 0x7f63f0102680 2024-04-02T13:38:08.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.415+0000 7f63e6ffd640 1 -- 172.21.15.33:0/4208993388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f63ec018a10 con 0x7f63f0102680 2024-04-02T13:38:08.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.416+0000 7f63e6ffd640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f63dc0a3c40 0x7f63dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:08.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.416+0000 7f63f5440640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f63dc0a3c40 0x7f63dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:08.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.416+0000 7f63e6ffd640 1 -- 172.21.15.33:0/4208993388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f63ec0d0d80 con 0x7f63f0102680 2024-04-02T13:38:08.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.416+0000 7f63f5440640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f63dc0a3c40 0x7f63dc0a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f63f0199250 tx=0x7f63e0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:08.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.419+0000 7f63e6ffd640 1 -- 172.21.15.33:0/4208993388 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f63ec014030 con 0x7f63f0102680 2024-04-02T13:38:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.606+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4208993388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63f002d370 con 0x7f63f0102680 2024-04-02T13:38:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.608+0000 7f63e6ffd640 1 -- 172.21.15.33:0/4208993388 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24527 (secure 0 0 0) 0x7f63ec09d2d0 con 0x7f63f0102680 2024-04-02T13:38:08.611 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:08.611 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":92,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:37:17.615217+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:38:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.610+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4208993388 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f63dc0a3c40 msgr2=0x7f63dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.610+0000 7f63f7ea5640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f63dc0a3c40 0x7f63dc0a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f63f0199250 tx=0x7f63e0005e50 comp rx=0 tx=0).stop 2024-04-02T13:38:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.610+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4208993388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63f0102680 msgr2=0x7f63f0197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.610+0000 7f63f7ea5640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63f0102680 0x7f63f0197d40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00c920 tx=0x7f63ec00cde0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.610+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4208993388 shutdown_connections 2024-04-02T13:38:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.610+0000 7f63f7ea5640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f63dc0a3c40 0x7f63dc0a6100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.610+0000 7f63f7ea5640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63f01078e0 0x7f63f019f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.610+0000 7f63f7ea5640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63f0104fb0 0x7f63f0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.610+0000 7f63f7ea5640 1 --2- 172.21.15.33:0/4208993388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63f0102680 0x7f63f0197d40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.610+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4208993388 >> 172.21.15.33:0/4208993388 conn(0x7f63f00fc030 msgr2=0x7f63f0102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.610+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4208993388 shutdown_connections 2024-04-02T13:38:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.610+0000 7f63f7ea5640 1 -- 172.21.15.33:0/4208993388 wait complete. 2024-04-02T13:38:08.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:08 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/484251544' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T13:38:08.624 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:36:19.501121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:36:20.161569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:36:23.482134+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:38:08.624 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:38:08.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:08 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/484251544' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T13:38:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.727+0000 7f7d3fd74640 1 -- 172.21.15.33:0/3692224056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d38102d00 msgr2=0x7f7d38103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.727+0000 7f7d3fd74640 1 --2- 172.21.15.33:0/3692224056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d38102d00 0x7f7d38103180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009900 tx=0x7f7d2802bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.727+0000 7f7d3fd74640 1 -- 172.21.15.33:0/3692224056 shutdown_connections 2024-04-02T13:38:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.727+0000 7f7d3fd74640 1 --2- 172.21.15.33:0/3692224056 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d381036c0 0x7f7d38109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.727+0000 7f7d3fd74640 1 --2- 172.21.15.33:0/3692224056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d38102d00 0x7f7d38103180 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.727+0000 7f7d3fd74640 1 --2- 172.21.15.33:0/3692224056 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d38101b10 0x7f7d38101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.727+0000 7f7d3fd74640 1 -- 172.21.15.33:0/3692224056 >> 172.21.15.33:0/3692224056 conn(0x7f7d380fd0a0 msgr2=0x7f7d380ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3fd74640 1 -- 172.21.15.33:0/3692224056 shutdown_connections 2024-04-02T13:38:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3fd74640 1 -- 172.21.15.33:0/3692224056 wait complete. 2024-04-02T13:38:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3fd74640 1 Processor -- start 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3fd74640 1 -- start start 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3fd74640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d38101b10 0x7f7d38197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3fd74640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38102d00 0x7f7d38198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3fd74640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d381036c0 0x7f7d3819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3fd74640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3810d770 con 0x7f7d38101b10 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3fd74640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3810d5f0 con 0x7f7d38102d00 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3fd74640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3810d8f0 con 0x7f7d381036c0 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3db10640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d38101b10 0x7f7d38197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3db10640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d38101b10 0x7f7d38197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50290/0 (socket says 172.21.15.33:50290) 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3db10640 1 -- 172.21.15.33:0/393669060 learned_addr learned my addr 172.21.15.33:0/393669060 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.728+0000 7f7d3d30f640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38102d00 0x7f7d38198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.729+0000 7f7d3e311640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d381036c0 0x7f7d3819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.729+0000 7f7d3db10640 1 -- 172.21.15.33:0/393669060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d381036c0 msgr2=0x7f7d3819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.729+0000 7f7d3db10640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d381036c0 0x7f7d3819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.729+0000 7f7d3db10640 1 -- 172.21.15.33:0/393669060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38102d00 msgr2=0x7f7d38198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:08.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.729+0000 7f7d3db10640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38102d00 0x7f7d38198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.729+0000 7f7d3db10640 1 -- 172.21.15.33:0/393669060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3819fa60 con 0x7f7d38101b10 2024-04-02T13:38:08.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.729+0000 7f7d3d30f640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38102d00 0x7f7d38198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:08.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.729+0000 7f7d3db10640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d38101b10 0x7f7d38197e40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000d950 tx=0x7f7d3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:08.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.729+0000 7f7d2effd640 1 -- 172.21.15.33:0/393669060 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30017070 con 0x7f7d38101b10 2024-04-02T13:38:08.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.729+0000 7f7d2effd640 1 -- 172.21.15.33:0/393669060 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7d30009c30 con 0x7f7d38101b10 2024-04-02T13:38:08.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.729+0000 7f7d3fd74640 1 -- 172.21.15.33:0/393669060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3819fd50 con 0x7f7d38101b10 2024-04-02T13:38:08.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.729+0000 7f7d2effd640 1 -- 172.21.15.33:0/393669060 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000f620 con 0x7f7d38101b10 2024-04-02T13:38:08.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.729+0000 7f7d3fd74640 1 -- 172.21.15.33:0/393669060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d381a0190 con 0x7f7d38101b10 2024-04-02T13:38:08.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.730+0000 7f7d3fd74640 1 -- 172.21.15.33:0/393669060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d38108eb0 con 0x7f7d38101b10 2024-04-02T13:38:08.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.731+0000 7f7d2effd640 1 -- 172.21.15.33:0/393669060 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7d3001a070 con 0x7f7d38101b10 2024-04-02T13:38:08.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.732+0000 7f7d2effd640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7d240a3c40 0x7f7d240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:08.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.732+0000 7f7d3d30f640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7d240a3c40 0x7f7d240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:08.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.732+0000 7f7d2effd640 1 -- 172.21.15.33:0/393669060 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7d300d04a0 con 0x7f7d38101b10 2024-04-02T13:38:08.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.732+0000 7f7d3d30f640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7d240a3c40 0x7f7d240a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d38199350 tx=0x7f7d28000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:08.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.737+0000 7f7d2effd640 1 -- 172.21.15.33:0/393669060 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7d30009de0 con 0x7f7d38101b10 2024-04-02T13:38:08.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:08 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/484251544' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T13:38:08.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.923+0000 7f7d3fd74640 1 -- 172.21.15.33:0/393669060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7d38062800 con 0x7f7d38101b10 2024-04-02T13:38:08.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.923+0000 7f7d2effd640 1 -- 172.21.15.33:0/393669060 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f7d3009c960 con 0x7f7d38101b10 2024-04-02T13:38:08.926 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:38:08.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.925+0000 7f7d3fd74640 1 -- 172.21.15.33:0/393669060 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7d240a3c40 msgr2=0x7f7d240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:08.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.925+0000 7f7d3fd74640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7d240a3c40 0x7f7d240a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d38199350 tx=0x7f7d28000bc0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.925+0000 7f7d3fd74640 1 -- 172.21.15.33:0/393669060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d38101b10 msgr2=0x7f7d38197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:08.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.925+0000 7f7d3fd74640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d38101b10 0x7f7d38197e40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000d950 tx=0x7f7d3000de10 comp rx=0 tx=0).stop 2024-04-02T13:38:08.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.925+0000 7f7d3fd74640 1 -- 172.21.15.33:0/393669060 shutdown_connections 2024-04-02T13:38:08.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.925+0000 7f7d3fd74640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7d240a3c40 0x7f7d240a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.925+0000 7f7d3fd74640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d381036c0 0x7f7d3819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.925+0000 7f7d3fd74640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d38102d00 0x7f7d38198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.925+0000 7f7d3fd74640 1 --2- 172.21.15.33:0/393669060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d38101b10 0x7f7d38197e40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:08.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.925+0000 7f7d3fd74640 1 -- 172.21.15.33:0/393669060 >> 172.21.15.33:0/393669060 conn(0x7f7d380fd0a0 msgr2=0x7f7d38106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:08.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.925+0000 7f7d3fd74640 1 -- 172.21.15.33:0/393669060 shutdown_connections 2024-04-02T13:38:08.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:08.925+0000 7f7d3fd74640 1 -- 172.21.15.33:0/393669060 wait complete. 2024-04-02T13:38:08.939 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:38:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.043+0000 7f8565932640 1 -- 172.21.15.33:0/408461029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8560102d00 msgr2=0x7f8560103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.043+0000 7f8565932640 1 --2- 172.21.15.33:0/408461029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8560102d00 0x7f8560103180 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8548009900 tx=0x7f854802bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.043+0000 7f8565932640 1 -- 172.21.15.33:0/408461029 shutdown_connections 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.043+0000 7f8565932640 1 --2- 172.21.15.33:0/408461029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85601036c0 0x7f8560109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.043+0000 7f8565932640 1 --2- 172.21.15.33:0/408461029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8560102d00 0x7f8560103180 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.043+0000 7f8565932640 1 --2- 172.21.15.33:0/408461029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8560101b10 0x7f8560101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.043+0000 7f8565932640 1 -- 172.21.15.33:0/408461029 >> 172.21.15.33:0/408461029 conn(0x7f85600fd060 msgr2=0x7f85600ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.043+0000 7f8565932640 1 -- 172.21.15.33:0/408461029 shutdown_connections 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.043+0000 7f8565932640 1 -- 172.21.15.33:0/408461029 wait complete. 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f8565932640 1 Processor -- start 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f8565932640 1 -- start start 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f8565932640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8560101b10 0x7f8560197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f8565932640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8560102d00 0x7f8560198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f8565932640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85601036c0 0x7f856019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f8565932640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f856010d770 con 0x7f85601036c0 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f8565932640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f856010d5f0 con 0x7f8560102d00 2024-04-02T13:38:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f8565932640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f856010d8f0 con 0x7f8560101b10 2024-04-02T13:38:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f855f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85601036c0 0x7f856019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f855f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85601036c0 0x7f856019f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50304/0 (socket says 172.21.15.33:50304) 2024-04-02T13:38:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f855f7fe640 1 -- 172.21.15.33:0/1334042434 learned_addr learned my addr 172.21.15.33:0/1334042434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f855effd640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8560101b10 0x7f8560197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f855f7fe640 1 -- 172.21.15.33:0/1334042434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8560101b10 msgr2=0x7f8560197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f855f7fe640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8560101b10 0x7f8560197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f855f7fe640 1 -- 172.21.15.33:0/1334042434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8560102d00 msgr2=0x7f8560198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f8557fff640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8560102d00 0x7f8560198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f855f7fe640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8560102d00 0x7f8560198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f855f7fe640 1 -- 172.21.15.33:0/1334042434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856019fa40 con 0x7f85601036c0 2024-04-02T13:38:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.044+0000 7f855effd640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8560101b10 0x7f8560197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.045+0000 7f855f7fe640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85601036c0 0x7f856019f3d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f855800eb10 tx=0x7f855800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.045+0000 7f8564930640 1 -- 172.21.15.33:0/1334042434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558004b20 con 0x7f85601036c0 2024-04-02T13:38:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.045+0000 7f8564930640 1 -- 172.21.15.33:0/1334042434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8558004cc0 con 0x7f85601036c0 2024-04-02T13:38:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.045+0000 7f8565932640 1 -- 172.21.15.33:0/1334042434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856019fd30 con 0x7f85601036c0 2024-04-02T13:38:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.045+0000 7f8564930640 1 -- 172.21.15.33:0/1334042434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558010700 con 0x7f85601036c0 2024-04-02T13:38:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.045+0000 7f8565932640 1 -- 172.21.15.33:0/1334042434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85601a0220 con 0x7f85601036c0 2024-04-02T13:38:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.046+0000 7f8565932640 1 -- 172.21.15.33:0/1334042434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8560108ea0 con 0x7f85601036c0 2024-04-02T13:38:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.047+0000 7f8564930640 1 -- 172.21.15.33:0/1334042434 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8558004e70 con 0x7f85601036c0 2024-04-02T13:38:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.047+0000 7f8564930640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f854c0a3cc0 0x7f854c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.047+0000 7f8564930640 1 -- 172.21.15.33:0/1334042434 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f85580d0fa0 con 0x7f85601036c0 2024-04-02T13:38:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.048+0000 7f855effd640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f854c0a3cc0 0x7f854c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.048+0000 7f855effd640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f854c0a3cc0 0x7f854c0a6180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8560102b60 tx=0x7f8550005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:09.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.052+0000 7f8564930640 1 -- 172.21.15.33:0/1334042434 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f855809d460 con 0x7f85601036c0 2024-04-02T13:38:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.237+0000 7f8565932640 1 -- 172.21.15.33:0/1334042434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f85600627a0 con 0x7f85601036c0 2024-04-02T13:38:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.237+0000 7f8564930640 1 -- 172.21.15.33:0/1334042434 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f855809cd60 con 0x7f85601036c0 2024-04-02T13:38:09.240 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.239+0000 7f8565932640 1 -- 172.21.15.33:0/1334042434 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f854c0a3cc0 msgr2=0x7f854c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.239+0000 7f8565932640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f854c0a3cc0 0x7f854c0a6180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8560102b60 tx=0x7f8550005f90 comp rx=0 tx=0).stop 2024-04-02T13:38:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.239+0000 7f8565932640 1 -- 172.21.15.33:0/1334042434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85601036c0 msgr2=0x7f856019f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.239+0000 7f8565932640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85601036c0 0x7f856019f3d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f855800eb10 tx=0x7f855800c2d0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.239+0000 7f8565932640 1 -- 172.21.15.33:0/1334042434 shutdown_connections 2024-04-02T13:38:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.239+0000 7f8565932640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f854c0a3cc0 0x7f854c0a6180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.239+0000 7f8565932640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85601036c0 0x7f856019f3d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.239+0000 7f8565932640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8560102d00 0x7f8560198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.239+0000 7f8565932640 1 --2- 172.21.15.33:0/1334042434 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8560101b10 0x7f8560197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.239+0000 7f8565932640 1 -- 172.21.15.33:0/1334042434 >> 172.21.15.33:0/1334042434 conn(0x7f85600fd060 msgr2=0x7f8560106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:09.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.239+0000 7f8565932640 1 -- 172.21.15.33:0/1334042434 shutdown_connections 2024-04-02T13:38:09.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.239+0000 7f8565932640 1 -- 172.21.15.33:0/1334042434 wait complete. 2024-04-02T13:38:09.253 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:38:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.356+0000 7fa453ce4640 1 -- 172.21.15.33:0/3635447283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c107840 msgr2=0x7fa44c109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 --2- 172.21.15.33:0/3635447283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c107840 0x7fa44c109c30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa444009ca0 tx=0x7fa44402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 -- 172.21.15.33:0/3635447283 shutdown_connections 2024-04-02T13:38:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 --2- 172.21.15.33:0/3635447283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c107840 0x7fa44c109c30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 --2- 172.21.15.33:0/3635447283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa44c104f10 0x7fa44c107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 --2- 172.21.15.33:0/3635447283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa44c1025e0 0x7fa44c1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 -- 172.21.15.33:0/3635447283 >> 172.21.15.33:0/3635447283 conn(0x7fa44c0fbf90 msgr2=0x7fa44c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 -- 172.21.15.33:0/3635447283 shutdown_connections 2024-04-02T13:38:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 -- 172.21.15.33:0/3635447283 wait complete. 2024-04-02T13:38:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 Processor -- start 2024-04-02T13:38:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 -- start start 2024-04-02T13:38:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa44c1025e0 0x7fa44c101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c104f10 0x7fa44c100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa44c107840 0x7fa44c100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c10d760 con 0x7fa44c104f10 2024-04-02T13:38:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c10d5e0 con 0x7fa44c1025e0 2024-04-02T13:38:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa453ce4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c10d8e0 con 0x7fa44c107840 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.357+0000 7fa45127f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c104f10 0x7fa44c100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa45127f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c104f10 0x7fa44c100590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50322/0 (socket says 172.21.15.33:50322) 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa45127f640 1 -- 172.21.15.33:0/703208935 learned_addr learned my addr 172.21.15.33:0/703208935 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa451a80640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa44c1025e0 0x7fa44c101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa452281640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa44c107840 0x7fa44c100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa45127f640 1 -- 172.21.15.33:0/703208935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa44c107840 msgr2=0x7fa44c100ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa45127f640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa44c107840 0x7fa44c100ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa45127f640 1 -- 172.21.15.33:0/703208935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa44c1025e0 msgr2=0x7fa44c101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa45127f640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa44c1025e0 0x7fa44c101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa45127f640 1 -- 172.21.15.33:0/703208935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c1012f0 con 0x7fa44c104f10 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa45127f640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c104f10 0x7fa44c100590 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00c970 tx=0x7fa43c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa442ffd640 1 -- 172.21.15.33:0/703208935 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c014070 con 0x7fa44c104f10 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa442ffd640 1 -- 172.21.15.33:0/703208935 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa43c004440 con 0x7fa44c104f10 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa453ce4640 1 -- 172.21.15.33:0/703208935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa44c1a1ed0 con 0x7fa44c104f10 2024-04-02T13:38:09.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa442ffd640 1 -- 172.21.15.33:0/703208935 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c00f450 con 0x7fa44c104f10 2024-04-02T13:38:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.358+0000 7fa453ce4640 1 -- 172.21.15.33:0/703208935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa44c1a2390 con 0x7fa44c104f10 2024-04-02T13:38:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.359+0000 7fa453ce4640 1 -- 172.21.15.33:0/703208935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44c0fda80 con 0x7fa44c104f10 2024-04-02T13:38:09.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.360+0000 7fa442ffd640 1 -- 172.21.15.33:0/703208935 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa43c017070 con 0x7fa44c104f10 2024-04-02T13:38:09.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.361+0000 7fa442ffd640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa4380a3c40 0x7fa4380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.361+0000 7fa451a80640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa4380a3c40 0x7fa4380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.361+0000 7fa442ffd640 1 -- 172.21.15.33:0/703208935 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa43c0d0740 con 0x7fa44c104f10 2024-04-02T13:38:09.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.362+0000 7fa451a80640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa4380a3c40 0x7fa4380a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa448005d00 tx=0x7fa44800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.366+0000 7fa442ffd640 1 -- 172.21.15.33:0/703208935 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa43c01e020 con 0x7fa44c104f10 2024-04-02T13:38:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:09 smithi033 ceph-mon[32390]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:38:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:09 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4208993388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:09 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/393669060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:09.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:09 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1334042434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.555+0000 7fa453ce4640 1 -- 172.21.15.33:0/703208935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa44c02d5c0 con 0x7fa44c104f10 2024-04-02T13:38:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.555+0000 7fa442ffd640 1 -- 172.21.15.33:0/703208935 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fa43c09cc60 con 0x7fa44c104f10 2024-04-02T13:38:09.559 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-02T13:38:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.557+0000 7fa453ce4640 1 -- 172.21.15.33:0/703208935 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa4380a3c40 msgr2=0x7fa4380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.557+0000 7fa453ce4640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa4380a3c40 0x7fa4380a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa448005d00 tx=0x7fa44800b410 comp rx=0 tx=0).stop 2024-04-02T13:38:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.557+0000 7fa453ce4640 1 -- 172.21.15.33:0/703208935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c104f10 msgr2=0x7fa44c100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.557+0000 7fa453ce4640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c104f10 0x7fa44c100590 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00c970 tx=0x7fa43c00ce30 comp rx=0 tx=0).stop 2024-04-02T13:38:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.557+0000 7fa453ce4640 1 -- 172.21.15.33:0/703208935 shutdown_connections 2024-04-02T13:38:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.557+0000 7fa453ce4640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa4380a3c40 0x7fa4380a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.557+0000 7fa453ce4640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa44c107840 0x7fa44c100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.557+0000 7fa453ce4640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa44c104f10 0x7fa44c100590 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.557+0000 7fa453ce4640 1 --2- 172.21.15.33:0/703208935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa44c1025e0 0x7fa44c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.557+0000 7fa453ce4640 1 -- 172.21.15.33:0/703208935 >> 172.21.15.33:0/703208935 conn(0x7fa44c0fbf90 msgr2=0x7fa44c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.558+0000 7fa453ce4640 1 -- 172.21.15.33:0/703208935 shutdown_connections 2024-04-02T13:38:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.558+0000 7fa453ce4640 1 -- 172.21.15.33:0/703208935 wait complete. 2024-04-02T13:38:09.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:38:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.676+0000 7f0f1c455640 1 -- 172.21.15.33:0/64282115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f14100a00 msgr2=0x7f0f14100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.676+0000 7f0f1c455640 1 --2- 172.21.15.33:0/64282115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f14100a00 0x7f0f14100e20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009ca0 tx=0x7f0f1002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.676+0000 7f0f1c455640 1 -- 172.21.15.33:0/64282115 shutdown_connections 2024-04-02T13:38:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.676+0000 7f0f1c455640 1 --2- 172.21.15.33:0/64282115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f141025b0 0x7f0f14108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.676+0000 7f0f1c455640 1 --2- 172.21.15.33:0/64282115 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f14101bf0 0x7f0f14102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.676+0000 7f0f1c455640 1 --2- 172.21.15.33:0/64282115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f14100a00 0x7f0f14100e20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.676+0000 7f0f1c455640 1 -- 172.21.15.33:0/64282115 >> 172.21.15.33:0/64282115 conn(0x7f0f140fbf90 msgr2=0x7f0f140fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.676+0000 7f0f1c455640 1 -- 172.21.15.33:0/64282115 shutdown_connections 2024-04-02T13:38:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.676+0000 7f0f1c455640 1 -- 172.21.15.33:0/64282115 wait complete. 2024-04-02T13:38:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1c455640 1 Processor -- start 2024-04-02T13:38:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1c455640 1 -- start start 2024-04-02T13:38:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1c455640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f14100a00 0x7f0f14197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1c455640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f14101bf0 0x7f0f14198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1c455640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f141025b0 0x7f0f1419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1c455640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1410c660 con 0x7f0f14100a00 2024-04-02T13:38:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1c455640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1410c4e0 con 0x7f0f141025b0 2024-04-02T13:38:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1c455640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1410c7e0 con 0x7f0f14101bf0 2024-04-02T13:38:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1a1f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f14100a00 0x7f0f14197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1a1f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f14100a00 0x7f0f14197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50342/0 (socket says 172.21.15.33:50342) 2024-04-02T13:38:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1a1f1640 1 -- 172.21.15.33:0/316459136 learned_addr learned my addr 172.21.15.33:0/316459136 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1a9f2640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f141025b0 0x7f0f1419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f199f0640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f14101bf0 0x7f0f14198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1a1f1640 1 -- 172.21.15.33:0/316459136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f14101bf0 msgr2=0x7f0f14198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1a1f1640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f14101bf0 0x7f0f14198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1a1f1640 1 -- 172.21.15.33:0/316459136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f141025b0 msgr2=0x7f0f1419f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1a1f1640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f141025b0 0x7f0f1419f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1a1f1640 1 -- 172.21.15.33:0/316459136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1419fb10 con 0x7f0f14100a00 2024-04-02T13:38:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.677+0000 7f0f1a1f1640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f14100a00 0x7f0f14197ef0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0f1002c630 tx=0x7f0f10032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.678+0000 7f0f0b7fe640 1 -- 172.21.15.33:0/316459136 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10039050 con 0x7f0f14100a00 2024-04-02T13:38:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.678+0000 7f0f0b7fe640 1 -- 172.21.15.33:0/316459136 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0f1003a040 con 0x7f0f14100a00 2024-04-02T13:38:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.678+0000 7f0f1c455640 1 -- 172.21.15.33:0/316459136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1419fda0 con 0x7f0f14100a00 2024-04-02T13:38:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.678+0000 7f0f0b7fe640 1 -- 172.21.15.33:0/316459136 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10040450 con 0x7f0f14100a00 2024-04-02T13:38:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.678+0000 7f0f1c455640 1 -- 172.21.15.33:0/316459136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f141a01d0 con 0x7f0f14100a00 2024-04-02T13:38:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.679+0000 7f0f1c455640 1 -- 172.21.15.33:0/316459136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f141a0400 con 0x7f0f14100a00 2024-04-02T13:38:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.680+0000 7f0f0b7fe640 1 -- 172.21.15.33:0/316459136 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0f10004040 con 0x7f0f14100a00 2024-04-02T13:38:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.680+0000 7f0f0b7fe640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0f000a3c40 0x7f0f000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.681+0000 7f0f199f0640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0f000a3c40 0x7f0f000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.681+0000 7f0f0b7fe640 1 -- 172.21.15.33:0/316459136 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0f100f3550 con 0x7f0f14100a00 2024-04-02T13:38:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.681+0000 7f0f199f0640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0f000a3c40 0x7f0f000a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005fd0 tx=0x7f0f04005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:09.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.685+0000 7f0f0b7fe640 1 -- 172.21.15.33:0/316459136 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0f100bfb20 con 0x7f0f14100a00 2024-04-02T13:38:09.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:09 smithi145 ceph-mon[35646]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:38:09.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:09 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4208993388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:09.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:09 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/393669060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:09.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:09 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1334042434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:09.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:09 smithi050 ceph-mon[35524]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:38:09.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:09 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4208993388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:09.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:09 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/393669060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:09.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:09 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1334042434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:09.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.873+0000 7f0f1c455640 1 -- 172.21.15.33:0/316459136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0f141a0760 con 0x7f0f14100a00 2024-04-02T13:38:09.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.873+0000 7f0f0b7fe640 1 -- 172.21.15.33:0/316459136 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f0f100098b0 con 0x7f0f14100a00 2024-04-02T13:38:09.876 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.875+0000 7f0f1c455640 1 -- 172.21.15.33:0/316459136 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0f000a3c40 msgr2=0x7f0f000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.875+0000 7f0f1c455640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0f000a3c40 0x7f0f000a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005fd0 tx=0x7f0f04005850 comp rx=0 tx=0).stop 2024-04-02T13:38:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.875+0000 7f0f1c455640 1 -- 172.21.15.33:0/316459136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f14100a00 msgr2=0x7f0f14197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.875+0000 7f0f1c455640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f14100a00 0x7f0f14197ef0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0f1002c630 tx=0x7f0f10032980 comp rx=0 tx=0).stop 2024-04-02T13:38:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.875+0000 7f0f1c455640 1 -- 172.21.15.33:0/316459136 shutdown_connections 2024-04-02T13:38:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.875+0000 7f0f1c455640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0f000a3c40 0x7f0f000a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.875+0000 7f0f1c455640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f141025b0 0x7f0f1419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.875+0000 7f0f1c455640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f14101bf0 0x7f0f14198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.875+0000 7f0f1c455640 1 --2- 172.21.15.33:0/316459136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f14100a00 0x7f0f14197ef0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.875+0000 7f0f1c455640 1 -- 172.21.15.33:0/316459136 >> 172.21.15.33:0/316459136 conn(0x7f0f140fbf90 msgr2=0x7f0f14068d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.875+0000 7f0f1c455640 1 -- 172.21.15.33:0/316459136 shutdown_connections 2024-04-02T13:38:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.875+0000 7f0f1c455640 1 -- 172.21.15.33:0/316459136 wait complete. 2024-04-02T13:38:09.889 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:38:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.989+0000 7f41ea900640 1 -- 172.21.15.33:0/3986807557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4100a40 msgr2=0x7f41e4100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.989+0000 7f41ea900640 1 --2- 172.21.15.33:0/3986807557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4100a40 0x7f41e4100e60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009900 tx=0x7f41d402bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.989+0000 7f41ea900640 1 -- 172.21.15.33:0/3986807557 shutdown_connections 2024-04-02T13:38:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.989+0000 7f41ea900640 1 --2- 172.21.15.33:0/3986807557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41e41025f0 0x7f41e4108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.989+0000 7f41ea900640 1 --2- 172.21.15.33:0/3986807557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41e4101c30 0x7f41e41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.989+0000 7f41ea900640 1 --2- 172.21.15.33:0/3986807557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4100a40 0x7f41e4100e60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.989+0000 7f41ea900640 1 -- 172.21.15.33:0/3986807557 >> 172.21.15.33:0/3986807557 conn(0x7f41e40fbfb0 msgr2=0x7f41e40fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.989+0000 7f41ea900640 1 -- 172.21.15.33:0/3986807557 shutdown_connections 2024-04-02T13:38:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.989+0000 7f41ea900640 1 -- 172.21.15.33:0/3986807557 wait complete. 2024-04-02T13:38:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.990+0000 7f41ea900640 1 Processor -- start 2024-04-02T13:38:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.990+0000 7f41ea900640 1 -- start start 2024-04-02T13:38:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.990+0000 7f41ea900640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4100a40 0x7f41e406ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.990+0000 7f41ea900640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41e4101c30 0x7f41e419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.990+0000 7f41ea900640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41e41025f0 0x7f41e419f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.990+0000 7f41ea900640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f41e410c6a0 con 0x7f41e4100a40 2024-04-02T13:38:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.990+0000 7f41ea900640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f41e410c520 con 0x7f41e41025f0 2024-04-02T13:38:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.990+0000 7f41ea900640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f41e410c820 con 0x7f41e4101c30 2024-04-02T13:38:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.990+0000 7f41e3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4100a40 0x7f41e406ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.990+0000 7f41e3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4100a40 0x7f41e406ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50362/0 (socket says 172.21.15.33:50362) 2024-04-02T13:38:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.990+0000 7f41e3fff640 1 -- 172.21.15.33:0/2320409934 learned_addr learned my addr 172.21.15.33:0/2320409934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41e8e9d640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41e41025f0 0x7f41e419f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41e37fe640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41e4101c30 0x7f41e419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41e3fff640 1 -- 172.21.15.33:0/2320409934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41e4101c30 msgr2=0x7f41e419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41e3fff640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41e4101c30 0x7f41e419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41e3fff640 1 -- 172.21.15.33:0/2320409934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41e41025f0 msgr2=0x7f41e419f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41e3fff640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41e41025f0 0x7f41e419f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41e3fff640 1 -- 172.21.15.33:0/2320409934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e419fa70 con 0x7f41e4100a40 2024-04-02T13:38:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41e37fe640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41e4101c30 0x7f41e419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41e8e9d640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41e41025f0 0x7f41e419f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41e3fff640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4100a40 0x7f41e406ad20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f41d4005ab0 tx=0x7f41d4004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41e17fa640 1 -- 172.21.15.33:0/2320409934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4043020 con 0x7f41e4100a40 2024-04-02T13:38:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41ea900640 1 -- 172.21.15.33:0/2320409934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e419fd00 con 0x7f41e4100a40 2024-04-02T13:38:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41e17fa640 1 -- 172.21.15.33:0/2320409934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f41d40054f0 con 0x7f41e4100a40 2024-04-02T13:38:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41e17fa640 1 -- 172.21.15.33:0/2320409934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4003d10 con 0x7f41e4100a40 2024-04-02T13:38:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.991+0000 7f41ea900640 1 -- 172.21.15.33:0/2320409934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e41a01f0 con 0x7f41e4100a40 2024-04-02T13:38:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.992+0000 7f41ea900640 1 -- 172.21.15.33:0/2320409934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e4107d60 con 0x7f41e4100a40 2024-04-02T13:38:09.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.993+0000 7f41e17fa640 1 -- 172.21.15.33:0/2320409934 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f41d4046020 con 0x7f41e4100a40 2024-04-02T13:38:09.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.994+0000 7f41e17fa640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f41cc0a3c40 0x7f41cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:09.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.994+0000 7f41e37fe640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f41cc0a3c40 0x7f41cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:09.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.994+0000 7f41e17fa640 1 -- 172.21.15.33:0/2320409934 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f41d40f2d30 con 0x7f41e4100a40 2024-04-02T13:38:09.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.994+0000 7f41e37fe640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f41cc0a3c40 0x7f41cc0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41e406aa90 tx=0x7f41d8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:10.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:09.997+0000 7f41e17fa640 1 -- 172.21.15.33:0/2320409934 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f41d404d070 con 0x7f41e4100a40 2024-04-02T13:38:10.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.226+0000 7f41ea900640 1 -- 172.21.15.33:0/2320409934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41e402d5c0 con 0x7f41e4100a40 2024-04-02T13:38:10.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.227+0000 7f41e17fa640 1 -- 172.21.15.33:0/2320409934 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11873 (secure 0 0 0) 0x7f41d40dde20 con 0x7f41e4100a40 2024-04-02T13:38:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:10.230 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:36:47.489590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:38:10.230 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-02T13:38:10.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.229+0000 7f41ea900640 1 -- 172.21.15.33:0/2320409934 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f41cc0a3c40 msgr2=0x7f41cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.229+0000 7f41ea900640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f41cc0a3c40 0x7f41cc0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41e406aa90 tx=0x7f41d8005e50 comp rx=0 tx=0).stop 2024-04-02T13:38:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.229+0000 7f41ea900640 1 -- 172.21.15.33:0/2320409934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4100a40 msgr2=0x7f41e406ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.229+0000 7f41ea900640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4100a40 0x7f41e406ad20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f41d4005ab0 tx=0x7f41d4004e90 comp rx=0 tx=0).stop 2024-04-02T13:38:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.229+0000 7f41ea900640 1 -- 172.21.15.33:0/2320409934 shutdown_connections 2024-04-02T13:38:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.230+0000 7f41ea900640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f41cc0a3c40 0x7f41cc0a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.230+0000 7f41ea900640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41e41025f0 0x7f41e419f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.230+0000 7f41ea900640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41e4101c30 0x7f41e419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.230+0000 7f41ea900640 1 --2- 172.21.15.33:0/2320409934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e4100a40 0x7f41e406ad20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.230+0000 7f41ea900640 1 -- 172.21.15.33:0/2320409934 >> 172.21.15.33:0/2320409934 conn(0x7f41e40fbfb0 msgr2=0x7f41e4104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.230+0000 7f41ea900640 1 -- 172.21.15.33:0/2320409934 shutdown_connections 2024-04-02T13:38:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.230+0000 7f41ea900640 1 -- 172.21.15.33:0/2320409934 wait complete. 2024-04-02T13:38:10.244 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.346+0000 7f70c4c24640 1 -- 172.21.15.33:0/111775084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70c01025d0 msgr2=0x7f70c0108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.346+0000 7f70c4c24640 1 --2- 172.21.15.33:0/111775084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70c01025d0 0x7f70c0108e60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009ca0 tx=0x7f70b802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.346+0000 7f70c4c24640 1 -- 172.21.15.33:0/111775084 shutdown_connections 2024-04-02T13:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.346+0000 7f70c4c24640 1 --2- 172.21.15.33:0/111775084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70c01025d0 0x7f70c0108e60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.346+0000 7f70c4c24640 1 --2- 172.21.15.33:0/111775084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70c0101c10 0x7f70c0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.346+0000 7f70c4c24640 1 --2- 172.21.15.33:0/111775084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70c0100a20 0x7f70c0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.346+0000 7f70c4c24640 1 -- 172.21.15.33:0/111775084 >> 172.21.15.33:0/111775084 conn(0x7f70c00fbf90 msgr2=0x7f70c00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.346+0000 7f70c4c24640 1 -- 172.21.15.33:0/111775084 shutdown_connections 2024-04-02T13:38:10.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.346+0000 7f70c4c24640 1 -- 172.21.15.33:0/111775084 wait complete. 2024-04-02T13:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70c4c24640 1 Processor -- start 2024-04-02T13:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70c4c24640 1 -- start start 2024-04-02T13:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70c4c24640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70c0100a20 0x7f70c010b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70c4c24640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70c0101c10 0x7f70c01099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70c4c24640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70c01025d0 0x7f70c0109f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70c4c24640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f70c010d790 con 0x7f70c0100a20 2024-04-02T13:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70c4c24640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f70c010d610 con 0x7f70c0101c10 2024-04-02T13:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70c4c24640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f70c010d910 con 0x7f70c01025d0 2024-04-02T13:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70be59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70c0100a20 0x7f70c010b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70be59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70c0100a20 0x7f70c010b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50384/0 (socket says 172.21.15.33:50384) 2024-04-02T13:38:10.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70be59c640 1 -- 172.21.15.33:0/1772887287 learned_addr learned my addr 172.21.15.33:0/1772887287 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70bed9d640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70c01025d0 0x7f70c0109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70be59c640 1 -- 172.21.15.33:0/1772887287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70c01025d0 msgr2=0x7f70c0109f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70be59c640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70c01025d0 0x7f70c0109f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70be59c640 1 -- 172.21.15.33:0/1772887287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70c0101c10 msgr2=0x7f70c01099e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70be59c640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70c0101c10 0x7f70c01099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70be59c640 1 -- 172.21.15.33:0/1772887287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c010a7b0 con 0x7f70c0100a20 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70bed9d640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70c01025d0 0x7f70c0109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.347+0000 7f70be59c640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70c0100a20 0x7f70c010b3a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f70b000c970 tx=0x7f70b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.348+0000 7f70b77fe640 1 -- 172.21.15.33:0/1772887287 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0004220 con 0x7f70c0100a20 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.348+0000 7f70b77fe640 1 -- 172.21.15.33:0/1772887287 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f70b00043c0 con 0x7f70c0100a20 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.348+0000 7f70c4c24640 1 -- 172.21.15.33:0/1772887287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c01a1ed0 con 0x7f70c0100a20 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.348+0000 7f70b77fe640 1 -- 172.21.15.33:0/1772887287 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b000f450 con 0x7f70c0100a20 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.348+0000 7f70c4c24640 1 -- 172.21.15.33:0/1772887287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c01a2330 con 0x7f70c0100a20 2024-04-02T13:38:10.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.349+0000 7f70c4c24640 1 -- 172.21.15.33:0/1772887287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c0107d40 con 0x7f70c0100a20 2024-04-02T13:38:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.350+0000 7f70b77fe640 1 -- 172.21.15.33:0/1772887287 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f70b00079b0 con 0x7f70c0100a20 2024-04-02T13:38:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.350+0000 7f70b77fe640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f70ac0a3c40 0x7f70ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.351+0000 7f70bdd9b640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f70ac0a3c40 0x7f70ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:10.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.351+0000 7f70b77fe640 1 -- 172.21.15.33:0/1772887287 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f70b00d0740 con 0x7f70c0100a20 2024-04-02T13:38:10.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.351+0000 7f70bdd9b640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f70ac0a3c40 0x7f70ac0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f70a8005fd0 tx=0x7f70a8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:10.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.355+0000 7f70b77fe640 1 -- 172.21.15.33:0/1772887287 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f70b0014030 con 0x7f70c0100a20 2024-04-02T13:38:10.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:10 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/703208935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:10.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:10 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/316459136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:10.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:10 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2320409934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:10.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.548+0000 7f70c4c24640 1 -- 172.21.15.33:0/1772887287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70c002d5d0 con 0x7f70c0100a20 2024-04-02T13:38:10.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.549+0000 7f70b77fe640 1 -- 172.21.15.33:0/1772887287 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24527 (secure 0 0 0) 0x7f70b009cc00 con 0x7f70c0100a20 2024-04-02T13:38:10.552 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:10.553 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":92,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:37:17.615217+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:38:10.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.551+0000 7f70c4c24640 1 -- 172.21.15.33:0/1772887287 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f70ac0a3c40 msgr2=0x7f70ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.551+0000 7f70c4c24640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f70ac0a3c40 0x7f70ac0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f70a8005fd0 tx=0x7f70a8005850 comp rx=0 tx=0).stop 2024-04-02T13:38:10.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.551+0000 7f70c4c24640 1 -- 172.21.15.33:0/1772887287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70c0100a20 msgr2=0x7f70c010b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.551+0000 7f70c4c24640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70c0100a20 0x7f70c010b3a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f70b000c970 tx=0x7f70b000ce30 comp rx=0 tx=0).stop 2024-04-02T13:38:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.551+0000 7f70c4c24640 1 -- 172.21.15.33:0/1772887287 shutdown_connections 2024-04-02T13:38:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.551+0000 7f70c4c24640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f70ac0a3c40 0x7f70ac0a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.551+0000 7f70c4c24640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70c01025d0 0x7f70c0109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.551+0000 7f70c4c24640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70c0101c10 0x7f70c01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.551+0000 7f70c4c24640 1 --2- 172.21.15.33:0/1772887287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70c0100a20 0x7f70c010b3a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.551+0000 7f70c4c24640 1 -- 172.21.15.33:0/1772887287 >> 172.21.15.33:0/1772887287 conn(0x7f70c00fbf90 msgr2=0x7f70c0104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.551+0000 7f70c4c24640 1 -- 172.21.15.33:0/1772887287 shutdown_connections 2024-04-02T13:38:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.551+0000 7f70c4c24640 1 -- 172.21.15.33:0/1772887287 wait complete. 2024-04-02T13:38:10.565 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:36:19.501121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:36:20.161569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:36:23.482134+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:38:10.566 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:38:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.670+0000 7fab8fceb640 1 -- 172.21.15.33:0/4188336963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab88107800 msgr2=0x7fab88109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.670+0000 7fab8fceb640 1 --2- 172.21.15.33:0/4188336963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab88107800 0x7fab88109c50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fab80009ca0 tx=0x7fab8002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.670+0000 7fab8fceb640 1 -- 172.21.15.33:0/4188336963 shutdown_connections 2024-04-02T13:38:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.670+0000 7fab8fceb640 1 --2- 172.21.15.33:0/4188336963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab88107800 0x7fab88109c50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.670+0000 7fab8fceb640 1 --2- 172.21.15.33:0/4188336963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab88104ed0 0x7fab881072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.670+0000 7fab8fceb640 1 --2- 172.21.15.33:0/4188336963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab881025a0 0x7fab88104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.670+0000 7fab8fceb640 1 -- 172.21.15.33:0/4188336963 >> 172.21.15.33:0/4188336963 conn(0x7fab880fbfb0 msgr2=0x7fab880fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.670+0000 7fab8fceb640 1 -- 172.21.15.33:0/4188336963 shutdown_connections 2024-04-02T13:38:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.670+0000 7fab8fceb640 1 -- 172.21.15.33:0/4188336963 wait complete. 2024-04-02T13:38:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8fceb640 1 Processor -- start 2024-04-02T13:38:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8fceb640 1 -- start start 2024-04-02T13:38:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8fceb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab881025a0 0x7fab88197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:10.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8fceb640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab88104ed0 0x7fab88198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8fceb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab88107800 0x7fab8819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8fceb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab8810d800 con 0x7fab881025a0 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8fceb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fab8810d680 con 0x7fab88107800 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8fceb640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fab8810d980 con 0x7fab88104ed0 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8da87640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab881025a0 0x7fab88197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8da87640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab881025a0 0x7fab88197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50408/0 (socket says 172.21.15.33:50408) 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8da87640 1 -- 172.21.15.33:0/1884999554 learned_addr learned my addr 172.21.15.33:0/1884999554 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8d286640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab88104ed0 0x7fab88198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8e288640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab88107800 0x7fab8819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8da87640 1 -- 172.21.15.33:0/1884999554 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab88104ed0 msgr2=0x7fab88198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8da87640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab88104ed0 0x7fab88198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8da87640 1 -- 172.21.15.33:0/1884999554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab88107800 msgr2=0x7fab8819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8da87640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab88107800 0x7fab8819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8da87640 1 -- 172.21.15.33:0/1884999554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8819f9f0 con 0x7fab881025a0 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8e288640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab88107800 0x7fab8819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8da87640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab881025a0 0x7fab88197e40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fab8400c920 tx=0x7fab8400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab7effd640 1 -- 172.21.15.33:0/1884999554 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84007c70 con 0x7fab881025a0 2024-04-02T13:38:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab7effd640 1 -- 172.21.15.33:0/1884999554 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fab84007e10 con 0x7fab881025a0 2024-04-02T13:38:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab7effd640 1 -- 172.21.15.33:0/1884999554 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84018850 con 0x7fab881025a0 2024-04-02T13:38:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.671+0000 7fab8fceb640 1 -- 172.21.15.33:0/1884999554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8819fce0 con 0x7fab881025a0 2024-04-02T13:38:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.672+0000 7fab8fceb640 1 -- 172.21.15.33:0/1884999554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab881a0200 con 0x7fab881025a0 2024-04-02T13:38:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.672+0000 7fab8fceb640 1 -- 172.21.15.33:0/1884999554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab880fdaa0 con 0x7fab881025a0 2024-04-02T13:38:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.674+0000 7fab7effd640 1 -- 172.21.15.33:0/1884999554 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fab84018a10 con 0x7fab881025a0 2024-04-02T13:38:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.674+0000 7fab7effd640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fab740a3c40 0x7fab740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.674+0000 7fab8d286640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fab740a3c40 0x7fab740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.674+0000 7fab7effd640 1 -- 172.21.15.33:0/1884999554 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fab840d0da0 con 0x7fab881025a0 2024-04-02T13:38:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.675+0000 7fab8d286640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fab740a3c40 0x7fab740a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fab881992e0 tx=0x7fab78008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:10.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.678+0000 7fab7effd640 1 -- 172.21.15.33:0/1884999554 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fab84014030 con 0x7fab881025a0 2024-04-02T13:38:10.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:10 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/703208935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:10.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:10 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/316459136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:10.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:10 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2320409934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:10.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:10 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/703208935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:10.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:10 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/316459136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:10.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:10 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2320409934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.862+0000 7fab8fceb640 1 -- 172.21.15.33:0/1884999554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fab8802d370 con 0x7fab881025a0 2024-04-02T13:38:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.862+0000 7fab7effd640 1 -- 172.21.15.33:0/1884999554 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fab8400de20 con 0x7fab881025a0 2024-04-02T13:38:10.865 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:38:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.864+0000 7fab8fceb640 1 -- 172.21.15.33:0/1884999554 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fab740a3c40 msgr2=0x7fab740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.864+0000 7fab8fceb640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fab740a3c40 0x7fab740a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fab881992e0 tx=0x7fab78008040 comp rx=0 tx=0).stop 2024-04-02T13:38:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.864+0000 7fab8fceb640 1 -- 172.21.15.33:0/1884999554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab881025a0 msgr2=0x7fab88197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.864+0000 7fab8fceb640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab881025a0 0x7fab88197e40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fab8400c920 tx=0x7fab8400cde0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.864+0000 7fab8fceb640 1 -- 172.21.15.33:0/1884999554 shutdown_connections 2024-04-02T13:38:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.864+0000 7fab8fceb640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fab740a3c40 0x7fab740a6100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.864+0000 7fab8fceb640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab88107800 0x7fab8819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.864+0000 7fab8fceb640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab88104ed0 0x7fab88198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.864+0000 7fab8fceb640 1 --2- 172.21.15.33:0/1884999554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab881025a0 0x7fab88197e40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.864+0000 7fab8fceb640 1 -- 172.21.15.33:0/1884999554 >> 172.21.15.33:0/1884999554 conn(0x7fab880fbfb0 msgr2=0x7fab88102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.864+0000 7fab8fceb640 1 -- 172.21.15.33:0/1884999554 shutdown_connections 2024-04-02T13:38:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.864+0000 7fab8fceb640 1 -- 172.21.15.33:0/1884999554 wait complete. 2024-04-02T13:38:10.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:38:10.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.981+0000 7fa4786f2640 1 -- 172.21.15.33:0/2069752466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa470102600 msgr2=0x7fa4701049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.981+0000 7fa4786f2640 1 --2- 172.21.15.33:0/2069752466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa470102600 0x7fa4701049f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa468009900 tx=0x7fa46802bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 -- 172.21.15.33:0/2069752466 shutdown_connections 2024-04-02T13:38:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 --2- 172.21.15.33:0/2069752466 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa470107860 0x7fa470109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 --2- 172.21.15.33:0/2069752466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa470104f30 0x7fa470107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 --2- 172.21.15.33:0/2069752466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa470102600 0x7fa4701049f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 -- 172.21.15.33:0/2069752466 >> 172.21.15.33:0/2069752466 conn(0x7fa4700fbfb0 msgr2=0x7fa4700fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 -- 172.21.15.33:0/2069752466 shutdown_connections 2024-04-02T13:38:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 -- 172.21.15.33:0/2069752466 wait complete. 2024-04-02T13:38:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 Processor -- start 2024-04-02T13:38:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 -- start start 2024-04-02T13:38:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa470102600 0x7fa470197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa470104f30 0x7fa470198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa470107860 0x7fa47019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa47010d780 con 0x7fa470107860 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa47010d600 con 0x7fa470102600 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa4786f2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa47010d900 con 0x7fa470104f30 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.982+0000 7fa476c8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa470107860 0x7fa47019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa476c8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa470107860 0x7fa47019f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50416/0 (socket says 172.21.15.33:50416) 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa476c8f640 1 -- 172.21.15.33:0/2963186713 learned_addr learned my addr 172.21.15.33:0/2963186713 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa47648e640 1 --2- 172.21.15.33:0/2963186713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa470102600 0x7fa470197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa476c8f640 1 -- 172.21.15.33:0/2963186713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa470104f30 msgr2=0x7fa470198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa476c8f640 1 --2- 172.21.15.33:0/2963186713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa470104f30 0x7fa470198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa476c8f640 1 -- 172.21.15.33:0/2963186713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa470102600 msgr2=0x7fa470197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa476c8f640 1 --2- 172.21.15.33:0/2963186713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa470102600 0x7fa470197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa476c8f640 1 -- 172.21.15.33:0/2963186713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47019fa20 con 0x7fa470107860 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa476c8f640 1 --2- 172.21.15.33:0/2963186713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa470107860 0x7fa47019f3b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa46000cce0 tx=0x7fa46000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa46f7fe640 1 -- 172.21.15.33:0/2963186713 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46000f450 con 0x7fa470107860 2024-04-02T13:38:10.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa46f7fe640 1 -- 172.21.15.33:0/2963186713 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa46000fac0 con 0x7fa470107860 2024-04-02T13:38:10.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa4786f2640 1 -- 172.21.15.33:0/2963186713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47019fd10 con 0x7fa470107860 2024-04-02T13:38:10.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa46f7fe640 1 -- 172.21.15.33:0/2963186713 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460018be0 con 0x7fa470107860 2024-04-02T13:38:10.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.983+0000 7fa4786f2640 1 -- 172.21.15.33:0/2963186713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4701a0210 con 0x7fa470107860 2024-04-02T13:38:10.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.984+0000 7fa4786f2640 1 -- 172.21.15.33:0/2963186713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4700fdaa0 con 0x7fa470107860 2024-04-02T13:38:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.985+0000 7fa46f7fe640 1 -- 172.21.15.33:0/2963186713 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa46000fc70 con 0x7fa470107860 2024-04-02T13:38:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.986+0000 7fa46f7fe640 1 --2- 172.21.15.33:0/2963186713 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa4580a3c40 0x7fa4580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.986+0000 7fa47648e640 1 --2- 172.21.15.33:0/2963186713 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa4580a3c40 0x7fa4580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.986+0000 7fa46f7fe640 1 -- 172.21.15.33:0/2963186713 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa4600d0fc0 con 0x7fa470107860 2024-04-02T13:38:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.986+0000 7fa47648e640 1 --2- 172.21.15.33:0/2963186713 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa4580a3c40 0x7fa4580a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa468006010 tx=0x7fa468000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:10.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:10.991+0000 7fa46f7fe640 1 -- 172.21.15.33:0/2963186713 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa460014030 con 0x7fa470107860 2024-04-02T13:38:11.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.176+0000 7fa4786f2640 1 -- 172.21.15.33:0/2963186713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa47002d520 con 0x7fa470107860 2024-04-02T13:38:11.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.176+0000 7fa46f7fe640 1 -- 172.21.15.33:0/2963186713 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fa46009d4e0 con 0x7fa470107860 2024-04-02T13:38:11.179 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:11.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.178+0000 7fa4786f2640 1 -- 172.21.15.33:0/2963186713 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa4580a3c40 msgr2=0x7fa4580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.178+0000 7fa4786f2640 1 --2- 172.21.15.33:0/2963186713 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa4580a3c40 0x7fa4580a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa468006010 tx=0x7fa468000bc0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.178+0000 7fa4786f2640 1 -- 172.21.15.33:0/2963186713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa470107860 msgr2=0x7fa47019f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.178+0000 7fa4786f2640 1 --2- 172.21.15.33:0/2963186713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa470107860 0x7fa47019f3b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa46000cce0 tx=0x7fa46000a620 comp rx=0 tx=0).stop 2024-04-02T13:38:11.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.178+0000 7fa4786f2640 1 -- 172.21.15.33:0/2963186713 shutdown_connections 2024-04-02T13:38:11.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.178+0000 7fa4786f2640 1 --2- 172.21.15.33:0/2963186713 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fa4580a3c40 0x7fa4580a6100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.178+0000 7fa4786f2640 1 --2- 172.21.15.33:0/2963186713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa470107860 0x7fa47019f3b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.178+0000 7fa4786f2640 1 --2- 172.21.15.33:0/2963186713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa470104f30 0x7fa470198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.178+0000 7fa4786f2640 1 --2- 172.21.15.33:0/2963186713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa470102600 0x7fa470197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.178+0000 7fa4786f2640 1 -- 172.21.15.33:0/2963186713 >> 172.21.15.33:0/2963186713 conn(0x7fa4700fbfb0 msgr2=0x7fa470102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:11.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.178+0000 7fa4786f2640 1 -- 172.21.15.33:0/2963186713 shutdown_connections 2024-04-02T13:38:11.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.178+0000 7fa4786f2640 1 -- 172.21.15.33:0/2963186713 wait complete. 2024-04-02T13:38:11.192 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:38:11.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.296+0000 7f174eb50640 1 -- 172.21.15.33:0/4284256510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1748101c00 msgr2=0x7f1748102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.296+0000 7f174eb50640 1 --2- 172.21.15.33:0/4284256510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1748101c00 0x7f1748102080 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1734009900 tx=0x7f173402bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:11.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.296+0000 7f174eb50640 1 -- 172.21.15.33:0/4284256510 shutdown_connections 2024-04-02T13:38:11.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.296+0000 7f174eb50640 1 --2- 172.21.15.33:0/4284256510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17481025c0 0x7f1748108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.296+0000 7f174eb50640 1 --2- 172.21.15.33:0/4284256510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1748101c00 0x7f1748102080 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.296+0000 7f174eb50640 1 --2- 172.21.15.33:0/4284256510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1748100a10 0x7f1748100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.296+0000 7f174eb50640 1 -- 172.21.15.33:0/4284256510 >> 172.21.15.33:0/4284256510 conn(0x7f17480fbfa0 msgr2=0x7f17480fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:11.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.296+0000 7f174eb50640 1 -- 172.21.15.33:0/4284256510 shutdown_connections 2024-04-02T13:38:11.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.296+0000 7f174eb50640 1 -- 172.21.15.33:0/4284256510 wait complete. 2024-04-02T13:38:11.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f174eb50640 1 Processor -- start 2024-04-02T13:38:11.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f174eb50640 1 -- start start 2024-04-02T13:38:11.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f174eb50640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1748100a10 0x7f174806aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:11.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f174eb50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1748101c00 0x7f174819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:11.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f174eb50640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17481025c0 0x7f174819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:11.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f174eb50640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f174810c770 con 0x7f1748101c00 2024-04-02T13:38:11.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f174eb50640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f174810c5f0 con 0x7f1748100a10 2024-04-02T13:38:11.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f174eb50640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f174810c8f0 con 0x7f17481025c0 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f1747fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1748101c00 0x7f174819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f1747fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1748101c00 0x7f174819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50434/0 (socket says 172.21.15.33:50434) 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f1747fff640 1 -- 172.21.15.33:0/2728675974 learned_addr learned my addr 172.21.15.33:0/2728675974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f174c8ec640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1748100a10 0x7f174806aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f174d0ed640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17481025c0 0x7f174819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f1747fff640 1 -- 172.21.15.33:0/2728675974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17481025c0 msgr2=0x7f174819f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f1747fff640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17481025c0 0x7f174819f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f1747fff640 1 -- 172.21.15.33:0/2728675974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1748100a10 msgr2=0x7f174806aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f1747fff640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1748100a10 0x7f174806aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f1747fff640 1 -- 172.21.15.33:0/2728675974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174819fc20 con 0x7f1748101c00 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f174d0ed640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17481025c0 0x7f174819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.297+0000 7f1747fff640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1748101c00 0x7f174819ccf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1734009230 tx=0x7f1734004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.298+0000 7f1745ffb640 1 -- 172.21.15.33:0/2728675974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734043020 con 0x7f1748101c00 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.298+0000 7f1745ffb640 1 -- 172.21.15.33:0/2728675974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f17340054f0 con 0x7f1748101c00 2024-04-02T13:38:11.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.298+0000 7f174eb50640 1 -- 172.21.15.33:0/2728675974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174819feb0 con 0x7f1748101c00 2024-04-02T13:38:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.298+0000 7f1745ffb640 1 -- 172.21.15.33:0/2728675974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734003d40 con 0x7f1748101c00 2024-04-02T13:38:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.298+0000 7f174eb50640 1 -- 172.21.15.33:0/2728675974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17481a03f0 con 0x7f1748101c00 2024-04-02T13:38:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.299+0000 7f174eb50640 1 -- 172.21.15.33:0/2728675974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1748107df0 con 0x7f1748101c00 2024-04-02T13:38:11.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.300+0000 7f1745ffb640 1 -- 172.21.15.33:0/2728675974 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1734046020 con 0x7f1748101c00 2024-04-02T13:38:11.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.300+0000 7f1745ffb640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f17380a3c40 0x7f17380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:11.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.300+0000 7f1745ffb640 1 -- 172.21.15.33:0/2728675974 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f17340f25c0 con 0x7f1748101c00 2024-04-02T13:38:11.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.300+0000 7f174c8ec640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f17380a3c40 0x7f17380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:11.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.301+0000 7f174c8ec640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f17380a3c40 0x7f17380a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1748101a60 tx=0x7f173c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:11.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.305+0000 7f1745ffb640 1 -- 172.21.15.33:0/2728675974 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f173404d070 con 0x7f1748101c00 2024-04-02T13:38:11.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.495+0000 7f174eb50640 1 -- 172.21.15.33:0/2728675974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f174802d5c0 con 0x7f1748101c00 2024-04-02T13:38:11.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.496+0000 7f1745ffb640 1 -- 172.21.15.33:0/2728675974 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f17340bebf0 con 0x7f1748101c00 2024-04-02T13:38:11.498 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-02T13:38:11.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.498+0000 7f174eb50640 1 -- 172.21.15.33:0/2728675974 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f17380a3c40 msgr2=0x7f17380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.498+0000 7f174eb50640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f17380a3c40 0x7f17380a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1748101a60 tx=0x7f173c009110 comp rx=0 tx=0).stop 2024-04-02T13:38:11.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.498+0000 7f174eb50640 1 -- 172.21.15.33:0/2728675974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1748101c00 msgr2=0x7f174819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.498+0000 7f174eb50640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1748101c00 0x7f174819ccf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1734009230 tx=0x7f1734004f10 comp rx=0 tx=0).stop 2024-04-02T13:38:11.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.498+0000 7f174eb50640 1 -- 172.21.15.33:0/2728675974 shutdown_connections 2024-04-02T13:38:11.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.498+0000 7f174eb50640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f17380a3c40 0x7f17380a6100 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.498+0000 7f174eb50640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17481025c0 0x7f174819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.498+0000 7f174eb50640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1748101c00 0x7f174819ccf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.498+0000 7f174eb50640 1 --2- 172.21.15.33:0/2728675974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1748100a10 0x7f174806aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.498+0000 7f174eb50640 1 -- 172.21.15.33:0/2728675974 >> 172.21.15.33:0/2728675974 conn(0x7f17480fbfa0 msgr2=0x7f1748104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:11.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.498+0000 7f174eb50640 1 -- 172.21.15.33:0/2728675974 shutdown_connections 2024-04-02T13:38:11.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.498+0000 7f174eb50640 1 -- 172.21.15.33:0/2728675974 wait complete. 2024-04-02T13:38:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:11 smithi033 ceph-mon[32390]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-02T13:38:11.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:11 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1772887287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:11.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:11 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1884999554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:11.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:11 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2963186713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:11.512 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:38:11.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.615+0000 7fd6be214640 1 -- 172.21.15.33:0/502563000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6b8100a00 msgr2=0x7fd6b8100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.615+0000 7fd6be214640 1 --2- 172.21.15.33:0/502563000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6b8100a00 0x7fd6b8100e20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009900 tx=0x7fd6a802bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:11.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.615+0000 7fd6be214640 1 -- 172.21.15.33:0/502563000 shutdown_connections 2024-04-02T13:38:11.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.615+0000 7fd6be214640 1 --2- 172.21.15.33:0/502563000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b81025b0 0x7fd6b8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.615+0000 7fd6be214640 1 --2- 172.21.15.33:0/502563000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6b8101bf0 0x7fd6b8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.615+0000 7fd6be214640 1 --2- 172.21.15.33:0/502563000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6b8100a00 0x7fd6b8100e20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.615+0000 7fd6be214640 1 -- 172.21.15.33:0/502563000 >> 172.21.15.33:0/502563000 conn(0x7fd6b80fbf90 msgr2=0x7fd6b80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.615+0000 7fd6be214640 1 -- 172.21.15.33:0/502563000 shutdown_connections 2024-04-02T13:38:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.615+0000 7fd6be214640 1 -- 172.21.15.33:0/502563000 wait complete. 2024-04-02T13:38:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.616+0000 7fd6be214640 1 Processor -- start 2024-04-02T13:38:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.616+0000 7fd6be214640 1 -- start start 2024-04-02T13:38:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.616+0000 7fd6be214640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b8100a00 0x7fd6b8197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.616+0000 7fd6be214640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6b8101bf0 0x7fd6b8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.616+0000 7fd6be214640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6b81025b0 0x7fd6b819f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.616+0000 7fd6be214640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b810c650 con 0x7fd6b8101bf0 2024-04-02T13:38:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.616+0000 7fd6be214640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b810c4d0 con 0x7fd6b8100a00 2024-04-02T13:38:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.616+0000 7fd6be214640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b810c7d0 con 0x7fd6b81025b0 2024-04-02T13:38:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.616+0000 7fd6b6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6b8101bf0 0x7fd6b8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.616+0000 7fd6b6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6b8101bf0 0x7fd6b8198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50448/0 (socket says 172.21.15.33:50448) 2024-04-02T13:38:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.616+0000 7fd6b6ffd640 1 -- 172.21.15.33:0/2634197811 learned_addr learned my addr 172.21.15.33:0/2634197811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.617+0000 7fd6b77fe640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b8100a00 0x7fd6b8197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.617+0000 7fd6b7fff640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6b81025b0 0x7fd6b819f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.617+0000 7fd6b6ffd640 1 -- 172.21.15.33:0/2634197811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6b81025b0 msgr2=0x7fd6b819f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.617+0000 7fd6b6ffd640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6b81025b0 0x7fd6b819f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.617+0000 7fd6b6ffd640 1 -- 172.21.15.33:0/2634197811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b8100a00 msgr2=0x7fd6b8197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.617+0000 7fd6b6ffd640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b8100a00 0x7fd6b8197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.617+0000 7fd6b6ffd640 1 -- 172.21.15.33:0/2634197811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b819fb00 con 0x7fd6b8101bf0 2024-04-02T13:38:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.617+0000 7fd6b6ffd640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6b8101bf0 0x7fd6b8198420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00d950 tx=0x7fd6ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.617+0000 7fd6b4ff9640 1 -- 172.21.15.33:0/2634197811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac017070 con 0x7fd6b8101bf0 2024-04-02T13:38:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.617+0000 7fd6b4ff9640 1 -- 172.21.15.33:0/2634197811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd6ac009c30 con 0x7fd6b8101bf0 2024-04-02T13:38:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.617+0000 7fd6be214640 1 -- 172.21.15.33:0/2634197811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b819fdf0 con 0x7fd6b8101bf0 2024-04-02T13:38:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.617+0000 7fd6b4ff9640 1 -- 172.21.15.33:0/2634197811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac00f6b0 con 0x7fd6b8101bf0 2024-04-02T13:38:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.617+0000 7fd6be214640 1 -- 172.21.15.33:0/2634197811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b81a0300 con 0x7fd6b8101bf0 2024-04-02T13:38:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.618+0000 7fd6be214640 1 -- 172.21.15.33:0/2634197811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b81a0530 con 0x7fd6b8101bf0 2024-04-02T13:38:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.619+0000 7fd6b4ff9640 1 -- 172.21.15.33:0/2634197811 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd6ac010a50 con 0x7fd6b8101bf0 2024-04-02T13:38:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.620+0000 7fd6b4ff9640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd6a00a3cc0 0x7fd6a00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.620+0000 7fd6b77fe640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd6a00a3cc0 0x7fd6a00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.620+0000 7fd6b4ff9640 1 -- 172.21.15.33:0/2634197811 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd6ac0d03b0 con 0x7fd6b8101bf0 2024-04-02T13:38:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.621+0000 7fd6b77fe640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd6a00a3cc0 0x7fd6a00a6180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8000c00 tx=0x7fd6a8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.624+0000 7fd6b4ff9640 1 -- 172.21.15.33:0/2634197811 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd6ac009de0 con 0x7fd6b8101bf0 2024-04-02T13:38:11.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:11 smithi145 ceph-mon[35646]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-02T13:38:11.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:11 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1772887287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:11.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:11 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1884999554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:11.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:11 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2963186713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:11.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:11 smithi050 ceph-mon[35524]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-02T13:38:11.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:11 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1772887287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:11.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:11 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1884999554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:11.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:11 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2963186713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.810+0000 7fd6be214640 1 -- 172.21.15.33:0/2634197811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd6b81a0700 con 0x7fd6b8101bf0 2024-04-02T13:38:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.811+0000 7fd6b4ff9640 1 -- 172.21.15.33:0/2634197811 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fd6ac09c2a0 con 0x7fd6b8101bf0 2024-04-02T13:38:11.813 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.813+0000 7fd6be214640 1 -- 172.21.15.33:0/2634197811 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd6a00a3cc0 msgr2=0x7fd6a00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.813+0000 7fd6be214640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd6a00a3cc0 0x7fd6a00a6180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8000c00 tx=0x7fd6a8005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.813+0000 7fd6be214640 1 -- 172.21.15.33:0/2634197811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6b8101bf0 msgr2=0x7fd6b8198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.813+0000 7fd6be214640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6b8101bf0 0x7fd6b8198420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00d950 tx=0x7fd6ac00de10 comp rx=0 tx=0).stop 2024-04-02T13:38:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.813+0000 7fd6be214640 1 -- 172.21.15.33:0/2634197811 shutdown_connections 2024-04-02T13:38:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.813+0000 7fd6be214640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd6a00a3cc0 0x7fd6a00a6180 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.813+0000 7fd6be214640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6b81025b0 0x7fd6b819f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.813+0000 7fd6be214640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6b8101bf0 0x7fd6b8198420 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.813+0000 7fd6be214640 1 --2- 172.21.15.33:0/2634197811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6b8100a00 0x7fd6b8197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.813+0000 7fd6be214640 1 -- 172.21.15.33:0/2634197811 >> 172.21.15.33:0/2634197811 conn(0x7fd6b80fbf90 msgr2=0x7fd6b8068d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.813+0000 7fd6be214640 1 -- 172.21.15.33:0/2634197811 shutdown_connections 2024-04-02T13:38:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.813+0000 7fd6be214640 1 -- 172.21.15.33:0/2634197811 wait complete. 2024-04-02T13:38:11.827 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:38:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.927+0000 7f6943575640 1 -- 172.21.15.33:0/2662024449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f693c100a20 msgr2=0x7f693c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.927+0000 7f6943575640 1 --2- 172.21.15.33:0/2662024449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f693c100a20 0x7f693c100e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6938009900 tx=0x7f693802bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.927+0000 7f6943575640 1 -- 172.21.15.33:0/2662024449 shutdown_connections 2024-04-02T13:38:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.927+0000 7f6943575640 1 --2- 172.21.15.33:0/2662024449 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f693c1025d0 0x7f693c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.927+0000 7f6943575640 1 --2- 172.21.15.33:0/2662024449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693c101c10 0x7f693c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.927+0000 7f6943575640 1 --2- 172.21.15.33:0/2662024449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f693c100a20 0x7f693c100e40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.927+0000 7f6943575640 1 -- 172.21.15.33:0/2662024449 >> 172.21.15.33:0/2662024449 conn(0x7f693c0fbf90 msgr2=0x7f693c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.928+0000 7f6943575640 1 -- 172.21.15.33:0/2662024449 shutdown_connections 2024-04-02T13:38:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.928+0000 7f6943575640 1 -- 172.21.15.33:0/2662024449 wait complete. 2024-04-02T13:38:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.928+0000 7f6943575640 1 Processor -- start 2024-04-02T13:38:11.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.928+0000 7f6943575640 1 -- start start 2024-04-02T13:38:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.928+0000 7f6943575640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693c100a20 0x7f693c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.928+0000 7f6943575640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f693c101c10 0x7f693c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.928+0000 7f6943575640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f693c1025d0 0x7f693c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.928+0000 7f6943575640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f693c10d790 con 0x7f693c1025d0 2024-04-02T13:38:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.928+0000 7f6943575640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f693c10d610 con 0x7f693c100a20 2024-04-02T13:38:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.928+0000 7f6943575640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f693c10d910 con 0x7f693c101c10 2024-04-02T13:38:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.928+0000 7f6941b12640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f693c1025d0 0x7f693c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.928+0000 7f6941b12640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f693c1025d0 0x7f693c19f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50452/0 (socket says 172.21.15.33:50452) 2024-04-02T13:38:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.928+0000 7f6941b12640 1 -- 172.21.15.33:0/356573025 learned_addr learned my addr 172.21.15.33:0/356573025 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f6941311640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693c100a20 0x7f693c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f6940b10640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f693c101c10 0x7f693c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f6941b12640 1 -- 172.21.15.33:0/356573025 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f693c101c10 msgr2=0x7f693c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f6941b12640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f693c101c10 0x7f693c198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f6941b12640 1 -- 172.21.15.33:0/356573025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693c100a20 msgr2=0x7f693c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:11.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f6941b12640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693c100a20 0x7f693c197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:11.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f6941b12640 1 -- 172.21.15.33:0/356573025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c19f9f0 con 0x7f693c1025d0 2024-04-02T13:38:11.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f6941311640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693c100a20 0x7f693c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:11.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f6941b12640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f693c1025d0 0x7f693c19f380 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f693400eaa0 tx=0x7f693400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:11.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f69327fc640 1 -- 172.21.15.33:0/356573025 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693400cdb0 con 0x7f693c1025d0 2024-04-02T13:38:11.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f69327fc640 1 -- 172.21.15.33:0/356573025 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6934005020 con 0x7f693c1025d0 2024-04-02T13:38:11.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f6943575640 1 -- 172.21.15.33:0/356573025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c19fce0 con 0x7f693c1025d0 2024-04-02T13:38:11.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f69327fc640 1 -- 172.21.15.33:0/356573025 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69340116b0 con 0x7f693c1025d0 2024-04-02T13:38:11.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.929+0000 7f6943575640 1 -- 172.21.15.33:0/356573025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c1a0220 con 0x7f693c1025d0 2024-04-02T13:38:11.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.930+0000 7f6943575640 1 -- 172.21.15.33:0/356573025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c107d40 con 0x7f693c1025d0 2024-04-02T13:38:11.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.931+0000 7f69327fc640 1 -- 172.21.15.33:0/356573025 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f69340051d0 con 0x7f693c1025d0 2024-04-02T13:38:11.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.932+0000 7f69327fc640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f69280a3cc0 0x7f69280a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:11.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.932+0000 7f69327fc640 1 -- 172.21.15.33:0/356573025 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f69340d0f20 con 0x7f693c1025d0 2024-04-02T13:38:11.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.932+0000 7f6941311640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f69280a3cc0 0x7f69280a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:11.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.932+0000 7f6941311640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f69280a3cc0 0x7f69280a6180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6938006010 tx=0x7f6938000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:11.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:11.935+0000 7f69327fc640 1 -- 172.21.15.33:0/356573025 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6934016030 con 0x7f693c1025d0 2024-04-02T13:38:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.161+0000 7f6943575640 1 -- 172.21.15.33:0/356573025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f693c02d370 con 0x7f693c1025d0 2024-04-02T13:38:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.162+0000 7f69327fc640 1 -- 172.21.15.33:0/356573025 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11873 (secure 0 0 0) 0x7f693409d4c0 con 0x7f693c1025d0 2024-04-02T13:38:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:12.165 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:36:47.489590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:38:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-02T13:38:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.164+0000 7f6943575640 1 -- 172.21.15.33:0/356573025 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f69280a3cc0 msgr2=0x7f69280a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.164+0000 7f6943575640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f69280a3cc0 0x7f69280a6180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6938006010 tx=0x7f6938000bc0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.164+0000 7f6943575640 1 -- 172.21.15.33:0/356573025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f693c1025d0 msgr2=0x7f693c19f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.164+0000 7f6943575640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f693c1025d0 0x7f693c19f380 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f693400eaa0 tx=0x7f693400ef60 comp rx=0 tx=0).stop 2024-04-02T13:38:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.164+0000 7f6943575640 1 -- 172.21.15.33:0/356573025 shutdown_connections 2024-04-02T13:38:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.164+0000 7f6943575640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f69280a3cc0 0x7f69280a6180 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.164+0000 7f6943575640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f693c1025d0 0x7f693c19f380 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.164+0000 7f6943575640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f693c101c10 0x7f693c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.164+0000 7f6943575640 1 --2- 172.21.15.33:0/356573025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f693c100a20 0x7f693c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.164+0000 7f6943575640 1 -- 172.21.15.33:0/356573025 >> 172.21.15.33:0/356573025 conn(0x7f693c0fbf90 msgr2=0x7f693c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.164+0000 7f6943575640 1 -- 172.21.15.33:0/356573025 shutdown_connections 2024-04-02T13:38:12.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.164+0000 7f6943575640 1 -- 172.21.15.33:0/356573025 wait complete. 2024-04-02T13:38:12.178 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:38:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.279+0000 7f66ede0d640 1 -- 172.21.15.33:0/3866971328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e8102e80 msgr2=0x7f66e8103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.279+0000 7f66ede0d640 1 --2- 172.21.15.33:0/3866971328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e8102e80 0x7f66e8103300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009930 tx=0x7f66d802bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.279+0000 7f66ede0d640 1 -- 172.21.15.33:0/3866971328 shutdown_connections 2024-04-02T13:38:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.279+0000 7f66ede0d640 1 --2- 172.21.15.33:0/3866971328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66e8103840 0x7f66e810a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.279+0000 7f66ede0d640 1 --2- 172.21.15.33:0/3866971328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e8102e80 0x7f66e8103300 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.279+0000 7f66ede0d640 1 --2- 172.21.15.33:0/3866971328 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66e80ffe10 0x7f66e8100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.279+0000 7f66ede0d640 1 -- 172.21.15.33:0/3866971328 >> 172.21.15.33:0/3866971328 conn(0x7f66e80fb6e0 msgr2=0x7f66e80fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.279+0000 7f66ede0d640 1 -- 172.21.15.33:0/3866971328 shutdown_connections 2024-04-02T13:38:12.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.279+0000 7f66ede0d640 1 -- 172.21.15.33:0/3866971328 wait complete. 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.279+0000 7f66ede0d640 1 Processor -- start 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.279+0000 7f66ede0d640 1 -- start start 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66ede0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e80ffe10 0x7f66e8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66ede0d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66e8102e80 0x7f66e81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66ede0d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66e8103840 0x7f66e819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66ede0d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f66e810d7a0 con 0x7f66e80ffe10 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66ede0d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f66e810d620 con 0x7f66e8102e80 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66ede0d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f66e810d920 con 0x7f66e8103840 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e80ffe10 0x7f66e8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e80ffe10 0x7f66e8197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50482/0 (socket says 172.21.15.33:50482) 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e77fe640 1 -- 172.21.15.33:0/4081801563 learned_addr learned my addr 172.21.15.33:0/4081801563 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e7fff640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66e8103840 0x7f66e819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e6ffd640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66e8102e80 0x7f66e81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e77fe640 1 -- 172.21.15.33:0/4081801563 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66e8103840 msgr2=0x7f66e819f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e77fe640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66e8103840 0x7f66e819f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e77fe640 1 -- 172.21.15.33:0/4081801563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66e8102e80 msgr2=0x7f66e81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e77fe640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66e8102e80 0x7f66e81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e77fe640 1 -- 172.21.15.33:0/4081801563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e819fac0 con 0x7f66e80ffe10 2024-04-02T13:38:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e77fe640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e80ffe10 0x7f66e8197ea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c970 tx=0x7f66d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e4ff9640 1 -- 172.21.15.33:0/4081801563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0004220 con 0x7f66e80ffe10 2024-04-02T13:38:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e4ff9640 1 -- 172.21.15.33:0/4081801563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f66d00043c0 con 0x7f66e80ffe10 2024-04-02T13:38:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66ede0d640 1 -- 172.21.15.33:0/4081801563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e819fdb0 con 0x7f66e80ffe10 2024-04-02T13:38:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66e4ff9640 1 -- 172.21.15.33:0/4081801563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0004a00 con 0x7f66e80ffe10 2024-04-02T13:38:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.280+0000 7f66ede0d640 1 -- 172.21.15.33:0/4081801563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e81a0220 con 0x7f66e80ffe10 2024-04-02T13:38:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.281+0000 7f66ede0d640 1 -- 172.21.15.33:0/4081801563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e8109020 con 0x7f66e80ffe10 2024-04-02T13:38:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.283+0000 7f66e4ff9640 1 -- 172.21.15.33:0/4081801563 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f66d0004ba0 con 0x7f66e80ffe10 2024-04-02T13:38:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.283+0000 7f66e4ff9640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f66d40a3cc0 0x7f66d40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.283+0000 7f66e6ffd640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f66d40a3cc0 0x7f66d40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.283+0000 7f66e4ff9640 1 -- 172.21.15.33:0/4081801563 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f66d00d0670 con 0x7f66e80ffe10 2024-04-02T13:38:12.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.284+0000 7f66e6ffd640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f66d40a3cc0 0x7f66d40a6180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f66d802c710 tx=0x7f66d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.288+0000 7f66e4ff9640 1 -- 172.21.15.33:0/4081801563 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f66d0014030 con 0x7f66e80ffe10 2024-04-02T13:38:12.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.477+0000 7f66ede0d640 1 -- 172.21.15.33:0/4081801563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66e802d600 con 0x7f66e80ffe10 2024-04-02T13:38:12.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.479+0000 7f66e4ff9640 1 -- 172.21.15.33:0/4081801563 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24527 (secure 0 0 0) 0x7f66d009cb30 con 0x7f66e80ffe10 2024-04-02T13:38:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:12.482 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":92,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:37:17.615217+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:38:12.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.480+0000 7f66ede0d640 1 -- 172.21.15.33:0/4081801563 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f66d40a3cc0 msgr2=0x7f66d40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.480+0000 7f66ede0d640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f66d40a3cc0 0x7f66d40a6180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f66d802c710 tx=0x7f66d8005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.481+0000 7f66ede0d640 1 -- 172.21.15.33:0/4081801563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e80ffe10 msgr2=0x7f66e8197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.481+0000 7f66ede0d640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e80ffe10 0x7f66e8197ea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c970 tx=0x7f66d000ce30 comp rx=0 tx=0).stop 2024-04-02T13:38:12.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.481+0000 7f66ede0d640 1 -- 172.21.15.33:0/4081801563 shutdown_connections 2024-04-02T13:38:12.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.481+0000 7f66ede0d640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f66d40a3cc0 0x7f66d40a6180 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.481+0000 7f66ede0d640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66e8103840 0x7f66e819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.481+0000 7f66ede0d640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66e8102e80 0x7f66e81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.481+0000 7f66ede0d640 1 --2- 172.21.15.33:0/4081801563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66e80ffe10 0x7f66e8197ea0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.481+0000 7f66ede0d640 1 -- 172.21.15.33:0/4081801563 >> 172.21.15.33:0/4081801563 conn(0x7f66e80fb6e0 msgr2=0x7f66e81061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:12.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.481+0000 7f66ede0d640 1 -- 172.21.15.33:0/4081801563 shutdown_connections 2024-04-02T13:38:12.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.481+0000 7f66ede0d640 1 -- 172.21.15.33:0/4081801563 wait complete. 2024-04-02T13:38:12.493 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:12 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2728675974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:12.494 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:12 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2634197811' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:12.494 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:12 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/356573025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:12.495 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:38:12.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.595+0000 7fafe8704640 1 -- 172.21.15.33:0/3285474403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe0101c10 msgr2=0x7fafe0102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.595+0000 7fafe8704640 1 --2- 172.21.15.33:0/3285474403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe0101c10 0x7fafe0102090 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009900 tx=0x7fafd002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:12.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 -- 172.21.15.33:0/3285474403 shutdown_connections 2024-04-02T13:38:12.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 --2- 172.21.15.33:0/3285474403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafe01025d0 0x7fafe0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 --2- 172.21.15.33:0/3285474403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe0101c10 0x7fafe0102090 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 --2- 172.21.15.33:0/3285474403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafe0100a20 0x7fafe0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 -- 172.21.15.33:0/3285474403 >> 172.21.15.33:0/3285474403 conn(0x7fafe00fbf90 msgr2=0x7fafe00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:12.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 -- 172.21.15.33:0/3285474403 shutdown_connections 2024-04-02T13:38:12.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 -- 172.21.15.33:0/3285474403 wait complete. 2024-04-02T13:38:12.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 Processor -- start 2024-04-02T13:38:12.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 -- start start 2024-04-02T13:38:12.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafe0100a20 0x7fafe00697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe0101c10 0x7fafe019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafe01025d0 0x7fafe019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fafe010c680 con 0x7fafe0101c10 2024-04-02T13:38:12.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fafe010c500 con 0x7fafe0100a20 2024-04-02T13:38:12.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe8704640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fafe010c800 con 0x7fafe01025d0 2024-04-02T13:38:12.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe5c9f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe0101c10 0x7fafe019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe5c9f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe0101c10 0x7fafe019cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50502/0 (socket says 172.21.15.33:50502) 2024-04-02T13:38:12.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe5c9f640 1 -- 172.21.15.33:0/3383432376 learned_addr learned my addr 172.21.15.33:0/3383432376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.596+0000 7fafe64a0640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafe0100a20 0x7fafe00697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.597+0000 7fafe6ca1640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafe01025d0 0x7fafe019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.597+0000 7fafe5c9f640 1 -- 172.21.15.33:0/3383432376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafe01025d0 msgr2=0x7fafe019f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.597+0000 7fafe5c9f640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafe01025d0 0x7fafe019f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.597+0000 7fafe5c9f640 1 -- 172.21.15.33:0/3383432376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafe0100a20 msgr2=0x7fafe00697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.597+0000 7fafe5c9f640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafe0100a20 0x7fafe00697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.597+0000 7fafe5c9f640 1 -- 172.21.15.33:0/3383432376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe019fa30 con 0x7fafe0101c10 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.597+0000 7fafe6ca1640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafe01025d0 0x7fafe019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.597+0000 7fafe5c9f640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe0101c10 0x7fafe019cb30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009230 tx=0x7fafd0004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.597+0000 7fafd77fe640 1 -- 172.21.15.33:0/3383432376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0043020 con 0x7fafe0101c10 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.597+0000 7fafd77fe640 1 -- 172.21.15.33:0/3383432376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fafd0004410 con 0x7fafe0101c10 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.597+0000 7fafe8704640 1 -- 172.21.15.33:0/3383432376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe019fcc0 con 0x7fafe0101c10 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.597+0000 7fafd77fe640 1 -- 172.21.15.33:0/3383432376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0003e50 con 0x7fafe0101c10 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.597+0000 7fafe8704640 1 -- 172.21.15.33:0/3383432376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe01a01c0 con 0x7fafe0101c10 2024-04-02T13:38:12.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.598+0000 7fafe8704640 1 -- 172.21.15.33:0/3383432376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe0107ce0 con 0x7fafe0101c10 2024-04-02T13:38:12.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.599+0000 7fafd77fe640 1 -- 172.21.15.33:0/3383432376 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fafd0046020 con 0x7fafe0101c10 2024-04-02T13:38:12.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.600+0000 7fafd77fe640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fafcc0a3de0 0x7fafcc0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.600+0000 7fafe64a0640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fafcc0a3de0 0x7fafcc0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.600+0000 7fafd77fe640 1 -- 172.21.15.33:0/3383432376 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fafd00f2ce0 con 0x7fafe0101c10 2024-04-02T13:38:12.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.600+0000 7fafe64a0640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fafcc0a3de0 0x7fafcc0a62a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fafe0101a70 tx=0x7fafdc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:12.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.603+0000 7fafd77fe640 1 -- 172.21.15.33:0/3383432376 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fafd004d070 con 0x7fafe0101c10 2024-04-02T13:38:12.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:12 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2728675974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:12.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:12 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2634197811' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:12.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:12 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/356573025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:12.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:12 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2728675974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:12.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:12 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2634197811' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:12.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:12 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/356573025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:12.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.832+0000 7fafe8704640 1 -- 172.21.15.33:0/3383432376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafe002d560 con 0x7fafe0101c10 2024-04-02T13:38:12.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.832+0000 7fafd77fe640 1 -- 172.21.15.33:0/3383432376 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11873 (secure 0 0 0) 0x7fafd00bf310 con 0x7fafe0101c10 2024-04-02T13:38:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:36:47.489590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:38:12.836 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-02T13:38:12.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.834+0000 7fafe8704640 1 -- 172.21.15.33:0/3383432376 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fafcc0a3de0 msgr2=0x7fafcc0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.834+0000 7fafe8704640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fafcc0a3de0 0x7fafcc0a62a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fafe0101a70 tx=0x7fafdc005f90 comp rx=0 tx=0).stop 2024-04-02T13:38:12.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.834+0000 7fafe8704640 1 -- 172.21.15.33:0/3383432376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe0101c10 msgr2=0x7fafe019cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.834+0000 7fafe8704640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe0101c10 0x7fafe019cb30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009230 tx=0x7fafd0004f10 comp rx=0 tx=0).stop 2024-04-02T13:38:12.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.835+0000 7fafe8704640 1 -- 172.21.15.33:0/3383432376 shutdown_connections 2024-04-02T13:38:12.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.835+0000 7fafe8704640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fafcc0a3de0 0x7fafcc0a62a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.835+0000 7fafe8704640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafe01025d0 0x7fafe019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.835+0000 7fafe8704640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe0101c10 0x7fafe019cb30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.835+0000 7fafe8704640 1 --2- 172.21.15.33:0/3383432376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fafe0100a20 0x7fafe00697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.835+0000 7fafe8704640 1 -- 172.21.15.33:0/3383432376 >> 172.21.15.33:0/3383432376 conn(0x7fafe00fbf90 msgr2=0x7fafe0104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:12.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.835+0000 7fafe8704640 1 -- 172.21.15.33:0/3383432376 shutdown_connections 2024-04-02T13:38:12.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.835+0000 7fafe8704640 1 -- 172.21.15.33:0/3383432376 wait complete. 2024-04-02T13:38:12.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-02T13:36:21.184650+0000', 'modified': '2024-04-02T13:36:47.489590+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24551, 'mds_1': 24512, 'mds_2': 24473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6835/4110737670', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6834', 'nonce': 4110737670}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 4110737670}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.50:6835/1580073716', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6834', 'nonce': 1580073716}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 1580073716}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6837/4234446304', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6836', 'nonce': 4234446304}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 4234446304}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6839/3484016588', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6838', 'nonce': 3484016588}, {'type': 'v1', 'addr': '172.21.15.145:6839', 'nonce': 3484016588}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.50:6837/3788578797', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6836', 'nonce': 3788578797}, {'type': 'v1', 'addr': '172.21.15.50:6837', 'nonce': 3788578797}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24551': {'gid': 24551, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6839/932333028', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6838', 'nonce': 932333028}, {'type': 'v1', 'addr': '172.21.15.50:6839', 'nonce': 932333028}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24551, 'qdb_cluster': [24473, 24512, 24551]} 2024-04-02T13:38:12.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-02T13:38:12.849 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-02T13:38:12.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.951+0000 7f9a56882640 1 -- 172.21.15.33:0/3549268798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a50100cf0 msgr2=0x7f9a50101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.951+0000 7f9a56882640 1 --2- 172.21.15.33:0/3549268798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a50100cf0 0x7f9a50101170 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009900 tx=0x7f9a3802bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:12.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.951+0000 7f9a56882640 1 -- 172.21.15.33:0/3549268798 shutdown_connections 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.951+0000 7f9a56882640 1 --2- 172.21.15.33:0/3549268798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a501016b0 0x7f9a50108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.951+0000 7f9a56882640 1 --2- 172.21.15.33:0/3549268798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a50100cf0 0x7f9a50101170 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.951+0000 7f9a56882640 1 --2- 172.21.15.33:0/3549268798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a50101fc0 0x7f9a501023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.951+0000 7f9a56882640 1 -- 172.21.15.33:0/3549268798 >> 172.21.15.33:0/3549268798 conn(0x7f9a500fb700 msgr2=0x7f9a500fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.951+0000 7f9a56882640 1 -- 172.21.15.33:0/3549268798 shutdown_connections 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.951+0000 7f9a56882640 1 -- 172.21.15.33:0/3549268798 wait complete. 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a56882640 1 Processor -- start 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a56882640 1 -- start start 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a56882640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a50100cf0 0x7f9a50100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a56882640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a501016b0 0x7f9a5019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a56882640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a50101fc0 0x7f9a5019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a56882640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5010c6a0 con 0x7f9a50100cf0 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a56882640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5010c520 con 0x7f9a501016b0 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a56882640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5010c820 con 0x7f9a50101fc0 2024-04-02T13:38:12.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a50100cf0 0x7f9a50100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a50100cf0 0x7f9a50100750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50522/0 (socket says 172.21.15.33:50522) 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a4ffff640 1 -- 172.21.15.33:0/2867939667 learned_addr learned my addr 172.21.15.33:0/2867939667 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a4f7fe640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a501016b0 0x7f9a5019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a54e1f640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a50101fc0 0x7f9a5019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a4ffff640 1 -- 172.21.15.33:0/2867939667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a50101fc0 msgr2=0x7f9a5019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a4ffff640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a50101fc0 0x7f9a5019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a4ffff640 1 -- 172.21.15.33:0/2867939667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a501016b0 msgr2=0x7f9a5019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a4ffff640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a501016b0 0x7f9a5019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.952+0000 7f9a4ffff640 1 -- 172.21.15.33:0/2867939667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5019f9c0 con 0x7f9a50100cf0 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.953+0000 7f9a4f7fe640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a501016b0 0x7f9a5019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.953+0000 7f9a54e1f640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a50101fc0 0x7f9a5019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.953+0000 7f9a4ffff640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a50100cf0 0x7f9a50100750 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c970 tx=0x7f9a4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.953+0000 7f9a4d7fa640 1 -- 172.21.15.33:0/2867939667 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40007c10 con 0x7f9a50100cf0 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.953+0000 7f9a4d7fa640 1 -- 172.21.15.33:0/2867939667 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9a40007db0 con 0x7f9a50100cf0 2024-04-02T13:38:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.953+0000 7f9a56882640 1 -- 172.21.15.33:0/2867939667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5019fc50 con 0x7f9a50100cf0 2024-04-02T13:38:12.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.953+0000 7f9a4d7fa640 1 -- 172.21.15.33:0/2867939667 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000f8e0 con 0x7f9a50100cf0 2024-04-02T13:38:12.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.953+0000 7f9a56882640 1 -- 172.21.15.33:0/2867939667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a501a01a0 con 0x7f9a50100cf0 2024-04-02T13:38:12.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.955+0000 7f9a4d7fa640 1 -- 172.21.15.33:0/2867939667 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9a4000fa80 con 0x7f9a50100cf0 2024-04-02T13:38:12.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.955+0000 7f9a5945f740 1 Processor -- start 2024-04-02T13:38:12.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.955+0000 7f9a5945f740 1 -- start start 2024-04-02T13:38:12.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a5945f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be7ef0 0x5653c5be8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a5945f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be88e0 0x5653c5bed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a5945f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 0x5653c5befee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a5945f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653c5b48060 con 0x5653c5beda50 2024-04-02T13:38:12.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a5945f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5653c5af0650 con 0x5653c5be88e0 2024-04-02T13:38:12.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a5945f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5653c5b44010 con 0x5653c5be7ef0 2024-04-02T13:38:12.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a4d7fa640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9a3c0a3c40 0x7f9a3c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 0x5653c5befee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 0x5653c5befee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50524/0 (socket says 172.21.15.33:50524) 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a4f7fe640 1 -- 172.21.15.33:0/1343880922 learned_addr learned my addr 172.21.15.33:0/1343880922 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a4ffff640 1 --2- 172.21.15.33:0/1343880922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be88e0 0x5653c5bed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a54e1f640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9a3c0a3c40 0x7f9a3c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a54e1f640 1 --2- 172.21.15.33:0/1343880922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be7ef0 0x5653c5be8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a4d7fa640 1 -- 172.21.15.33:0/2867939667 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9a400d0ce0 con 0x7f9a50100cf0 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a4f7fe640 1 -- 172.21.15.33:0/1343880922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be7ef0 msgr2=0x5653c5be8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a4f7fe640 1 --2- 172.21.15.33:0/1343880922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be7ef0 0x5653c5be8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a4f7fe640 1 -- 172.21.15.33:0/1343880922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be88e0 msgr2=0x5653c5bed3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a4f7fe640 1 --2- 172.21.15.33:0/1343880922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be88e0 0x5653c5bed3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a4f7fe640 1 -- 172.21.15.33:0/1343880922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c585c2d0 con 0x5653c5beda50 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a4f7fe640 1 --2- 172.21.15.33:0/1343880922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 0x5653c5befee0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9a38036040 tx=0x7f9a38005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260a5aca4b76d74 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:12.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a54e1f640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9a3c0a3c40 0x7f9a3c0a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5653c5be9140 tx=0x7f9a4800a420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a467fc640 1 -- 172.21.15.33:0/1343880922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38042070 con 0x5653c5beda50 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.956+0000 7f9a467fc640 1 -- 172.21.15.33:0/1343880922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9a3802cc20 con 0x5653c5beda50 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.957+0000 7f9a467fc640 1 -- 172.21.15.33:0/1343880922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a380048e0 con 0x5653c5beda50 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.957+0000 7f9a5945f740 1 -- 172.21.15.33:0/1343880922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 msgr2=0x5653c5befee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.957+0000 7f9a5945f740 1 --2- 172.21.15.33:0/1343880922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 0x5653c5befee0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9a38036040 tx=0x7f9a38005dd0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.957+0000 7f9a5945f740 1 -- 172.21.15.33:0/1343880922 shutdown_connections 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.957+0000 7f9a5945f740 1 --2- 172.21.15.33:0/1343880922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 0x5653c5befee0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.957+0000 7f9a5945f740 1 --2- 172.21.15.33:0/1343880922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be88e0 0x5653c5bed3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.957+0000 7f9a5945f740 1 --2- 172.21.15.33:0/1343880922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be7ef0 0x5653c5be8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.957+0000 7f9a5945f740 1 -- 172.21.15.33:0/1343880922 >> 172.21.15.33:0/1343880922 conn(0x5653c5bdf050 msgr2=0x5653c5bdf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.957+0000 7f9a5945f740 1 -- 172.21.15.33:0/1343880922 shutdown_connections 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.957+0000 7f9a5945f740 1 -- 172.21.15.33:0/1343880922 wait complete. 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.957+0000 7f9a5945f740 1 -- start start 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a5945f740 1 Processor -- start 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a5945f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be7ef0 0x5653c5bf2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a5945f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be88e0 0x5653c5bf2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a5945f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 0x5653c5bf98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a5945f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653c5bf09a0 con 0x5653c5beda50 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a5945f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5653c5b03a10 con 0x5653c5be7ef0 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a5945f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5653c5bf0b20 con 0x5653c5be88e0 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a5945f740 10 client.0 _ll_register_callbacks cb 0x5653c5b400b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a54e1f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 0x5653c5bf98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a54e1f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 0x5653c5bf98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50540/0 (socket says 172.21.15.33:50540) 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a54e1f640 1 -- 172.21.15.33:0/3276923438 learned_addr learned my addr 172.21.15.33:0/3276923438 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a4f7fe640 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be88e0 0x5653c5bf2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a4ffff640 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be7ef0 0x5653c5bf2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a54e1f640 1 -- 172.21.15.33:0/3276923438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be88e0 msgr2=0x5653c5bf2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a54e1f640 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be88e0 0x5653c5bf2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a54e1f640 1 -- 172.21.15.33:0/3276923438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be7ef0 msgr2=0x5653c5bf2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a54e1f640 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be7ef0 0x5653c5bf2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a54e1f640 1 -- 172.21.15.33:0/3276923438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c59266b0 con 0x5653c5beda50 2024-04-02T13:38:12.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.958+0000 7f9a4ffff640 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be7ef0 0x5653c5bf2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a54e1f640 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 0x5653c5bf98b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800d9d0 tx=0x7f9a48004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca0af7b60404166 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a2f7fe640 1 -- 172.21.15.33:0/3276923438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4802b060 con 0x5653c5beda50 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a2f7fe640 1 -- 172.21.15.33:0/3276923438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9a4801cb90 con 0x5653c5beda50 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a5945f740 1 -- 172.21.15.33:0/3276923438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a3802cc20 con 0x5653c5beda50 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a2f7fe640 1 -- 172.21.15.33:0/3276923438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9a48032050 con 0x5653c5beda50 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a5945f740 10 client.15219 fetch_fsmap learned FSMap version 24 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a5945f740 1 -- 172.21.15.33:0/3276923438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5653c5bfcd40 con 0x5653c5beda50 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a2f7fe640 1 -- 172.21.15.33:0/3276923438 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9a3802cc20 con 0x5653c5beda50 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a2f7fe640 1 -- 172.21.15.33:0/3276923438 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9a480103d0 con 0x5653c5beda50 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a5945f740 10 client.15219 fetch_fsmap finished waiting for FSMap version 24 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a5945f740 10 client.15219 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a5945f740 20 client.15219 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:12.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a5945f740 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5653c5bff430 0x5653c5c008a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a5945f740 4 client.15219 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a5945f740 1 -- 172.21.15.33:0/3276923438 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653c5aefcf0 con 0x5653c5bff430 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.959+0000 7f9a4ffff640 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5653c5bff430 0x5653c5c008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.960+0000 7f9a4ffff640 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5653c5bff430 0x5653c5c008a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc86775e6df73f19 server_cookie=dd414db45fa7471e in_seq=0 out_seq=0 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.960+0000 7f9a2f7fe640 10 client.15219 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a2f7fe640 1 -- 172.21.15.33:0/3276923438 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653c5aefcf0 con 0x5653c5bff430 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a2f7fe640 10 client.15219 handle_command_reply: tid=0 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a5945f740 1 client.15219 shutdown 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a5945f740 1 -- 172.21.15.33:0/3276923438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 msgr2=0x5653c5bf98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a5945f740 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 0x5653c5bf98b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800d9d0 tx=0x7f9a48004ab0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a5945f740 1 -- 172.21.15.33:0/3276923438 shutdown_connections 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a5945f740 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5653c5bff430 0x5653c5c008a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a5945f740 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5beda50 0x5653c5bf98b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a5945f740 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be88e0 0x5653c5bf2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a2f7fe640 0 client.15219 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:38:12.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a5945f740 1 --2- 172.21.15.33:0/3276923438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be7ef0 0x5653c5bf2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a5945f740 1 -- 172.21.15.33:0/3276923438 >> 172.21.15.33:0/3276923438 conn(0x5653c5bdf050 msgr2=0x5653c5bde190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:12.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a5945f740 1 -- 172.21.15.33:0/3276923438 shutdown_connections 2024-04-02T13:38:12.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a5945f740 1 -- 172.21.15.33:0/3276923438 wait complete. 2024-04-02T13:38:12.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.961+0000 7f9a5945f740 20 client.15219 trim_cache size 0 max 16384 2024-04-02T13:38:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.979+0000 7f9a5945f740 1 Processor -- start 2024-04-02T13:38:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.979+0000 7f9a5945f740 1 -- start start 2024-04-02T13:38:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.979+0000 7f9a5945f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be7ef0 0x5653c5be9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.979+0000 7f9a5945f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 0x5653c5bea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.979+0000 7f9a5945f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5beda50 0x5653c5bf97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.979+0000 7f9a5945f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653c5bfebe0 con 0x5653c5be88e0 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.979+0000 7f9a5945f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5653c5bff2b0 con 0x5653c5be7ef0 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.979+0000 7f9a5945f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5653c5bff980 con 0x5653c5beda50 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.979+0000 7f9a4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 0x5653c5bea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.979+0000 7f9a4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 0x5653c5bea320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50548/0 (socket says 172.21.15.33:50548) 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.979+0000 7f9a4f7fe640 1 -- 172.21.15.33:0/4057044865 learned_addr learned my addr 172.21.15.33:0/4057044865 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.980+0000 7f9a4ffff640 1 --2- 172.21.15.33:0/4057044865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be7ef0 0x5653c5be9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.980+0000 7f9a4f7fe640 1 -- 172.21.15.33:0/4057044865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5beda50 msgr2=0x5653c5bf97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.980+0000 7f9a4f7fe640 1 --2- 172.21.15.33:0/4057044865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5beda50 0x5653c5bf97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.980+0000 7f9a4f7fe640 1 -- 172.21.15.33:0/4057044865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be7ef0 msgr2=0x5653c5be9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.980+0000 7f9a4f7fe640 1 --2- 172.21.15.33:0/4057044865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be7ef0 0x5653c5be9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.980+0000 7f9a4f7fe640 1 -- 172.21.15.33:0/4057044865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c5be8cc0 con 0x5653c5be88e0 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.980+0000 7f9a4f7fe640 1 --2- 172.21.15.33:0/4057044865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 0x5653c5bea320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x5653c5bdffc0 tx=0x7f9a38005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d515a7072cbe89bb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.980+0000 7f9a45ffb640 1 -- 172.21.15.33:0/4057044865 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38042040 con 0x5653c5be88e0 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.980+0000 7f9a45ffb640 1 -- 172.21.15.33:0/4057044865 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9a3804b070 con 0x5653c5be88e0 2024-04-02T13:38:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.980+0000 7f9a45ffb640 1 -- 172.21.15.33:0/4057044865 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38046440 con 0x5653c5be88e0 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 -- 172.21.15.33:0/4057044865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 msgr2=0x5653c5bea320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 --2- 172.21.15.33:0/4057044865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 0x5653c5bea320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x5653c5bdffc0 tx=0x7f9a38005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 -- 172.21.15.33:0/4057044865 shutdown_connections 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 --2- 172.21.15.33:0/4057044865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5beda50 0x5653c5bf97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 --2- 172.21.15.33:0/4057044865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 0x5653c5bea320 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 --2- 172.21.15.33:0/4057044865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5be7ef0 0x5653c5be9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 -- 172.21.15.33:0/4057044865 >> 172.21.15.33:0/4057044865 conn(0x5653c5bdf050 msgr2=0x5653c5bdf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 -- 172.21.15.33:0/4057044865 shutdown_connections 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 -- 172.21.15.33:0/4057044865 wait complete. 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 -- start start 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 Processor -- start 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be7ef0 0x5653c5bf77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 0x5653c5c613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5beda50 0x5653c5c63990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653c5c00720 con 0x5653c5be88e0 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5653c5c00050 con 0x5653c5beda50 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.981+0000 7f9a5945f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5653c5c00df0 con 0x5653c5be7ef0 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a5945f740 10 client.0 _ll_register_callbacks cb 0x7f9a4800aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 0x5653c5c613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 0x5653c5c613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50560/0 (socket says 172.21.15.33:50560) 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a4f7fe640 1 -- 172.21.15.33:0/2386314149 learned_addr learned my addr 172.21.15.33:0/2386314149 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a54e1f640 1 --2- 172.21.15.33:0/2386314149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5beda50 0x5653c5c63990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a4f7fe640 1 -- 172.21.15.33:0/2386314149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be7ef0 msgr2=0x5653c5bf77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a4f7fe640 1 --2- 172.21.15.33:0/2386314149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be7ef0 0x5653c5bf77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a4f7fe640 1 -- 172.21.15.33:0/2386314149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5beda50 msgr2=0x5653c5c63990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a4f7fe640 1 --2- 172.21.15.33:0/2386314149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5beda50 0x5653c5c63990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a4f7fe640 1 -- 172.21.15.33:0/2386314149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c5c64210 con 0x5653c5be88e0 2024-04-02T13:38:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a4f7fe640 1 --2- 172.21.15.33:0/2386314149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 0x5653c5c613c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9a38000c00 tx=0x7f9a38004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01e4c81c9ec19f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a2f7fe640 1 -- 172.21.15.33:0/2386314149 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38042040 con 0x5653c5be88e0 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a5945f740 1 -- 172.21.15.33:0/2386314149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a3804b070 con 0x5653c5be88e0 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a2f7fe640 1 -- 172.21.15.33:0/2386314149 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9a380054d0 con 0x5653c5be88e0 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a2f7fe640 1 -- 172.21.15.33:0/2386314149 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9a38060050 con 0x5653c5be88e0 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a5945f740 10 client.15225 fetch_fsmap learned FSMap version 24 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.982+0000 7f9a5945f740 1 -- 172.21.15.33:0/2386314149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5653c5c654b0 con 0x5653c5be88e0 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.983+0000 7f9a2f7fe640 1 -- 172.21.15.33:0/2386314149 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9a3804b070 con 0x5653c5be88e0 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.983+0000 7f9a2f7fe640 1 -- 172.21.15.33:0/2386314149 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9a380464b0 con 0x5653c5be88e0 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.983+0000 7f9a5945f740 10 client.15225 fetch_fsmap finished waiting for FSMap version 24 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.983+0000 7f9a5945f740 10 client.15225 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.983+0000 7f9a5945f740 20 client.15225 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.983+0000 7f9a5945f740 1 --2- 172.21.15.33:0/2386314149 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5653c5c69a80 0x5653c5c68790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.983+0000 7f9a5945f740 4 client.15225 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.983+0000 7f9a5945f740 1 -- 172.21.15.33:0/2386314149 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "status"}) -- 0x5653c5b555a0 con 0x5653c5c69a80 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.983+0000 7f9a4ffff640 1 --2- 172.21.15.33:0/2386314149 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5653c5c69a80 0x5653c5c68790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a2f7fe640 10 client.15225 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a4ffff640 1 --2- 172.21.15.33:0/2386314149 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5653c5c69a80 0x5653c5c68790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8dce34bf82c62c8 server_cookie=335c7cacac890758 in_seq=0 out_seq=0 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a2f7fe640 1 -- 172.21.15.33:0/2386314149 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5653c5b555a0 con 0x5653c5c69a80 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a2f7fe640 10 client.15225 handle_command_reply: tid=0 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a5945f740 1 client.15225 shutdown 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a5945f740 1 -- 172.21.15.33:0/2386314149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 msgr2=0x5653c5c613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a5945f740 1 --2- 172.21.15.33:0/2386314149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 0x5653c5c613c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9a38000c00 tx=0x7f9a38004f50 comp rx=0 tx=0).stop 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a5945f740 1 -- 172.21.15.33:0/2386314149 shutdown_connections 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a5945f740 1 --2- 172.21.15.33:0/2386314149 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5653c5c69a80 0x5653c5c68790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a5945f740 1 --2- 172.21.15.33:0/2386314149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653c5beda50 0x5653c5c63990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a2f7fe640 0 client.15225 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:38:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a5945f740 1 --2- 172.21.15.33:0/2386314149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653c5be88e0 0x5653c5c613c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a5945f740 1 --2- 172.21.15.33:0/2386314149 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5653c5be7ef0 0x5653c5bf77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a5945f740 1 -- 172.21.15.33:0/2386314149 >> 172.21.15.33:0/2386314149 conn(0x5653c5bdf050 msgr2=0x5653c5bfe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a5945f740 1 -- 172.21.15.33:0/2386314149 shutdown_connections 2024-04-02T13:38:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a5945f740 1 -- 172.21.15.33:0/2386314149 wait complete. 2024-04-02T13:38:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.984+0000 7f9a5945f740 20 client.15225 trim_cache size 0 max 16384 2024-04-02T13:38:12.993 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:38:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "b5fd982e-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:38:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-04-02T13:38:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24473, 2024-04-02T13:38:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-02T13:38:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-02T13:38:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-02T13:38:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 86.499524480999995, 2024-04-02T13:38:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-04-02T13:38:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 92, 2024-04-02T13:38:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:38:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 137.295063129 2024-04-02T13:38:12.994 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:38:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.985+0000 7f9a56882640 1 -- 172.21.15.33:0/2867939667 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9a3c0a3c40 msgr2=0x7f9a3c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.985+0000 7f9a56882640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9a3c0a3c40 0x7f9a3c0a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5653c5be9140 tx=0x7f9a4800a420 comp rx=0 tx=0).stop 2024-04-02T13:38:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.985+0000 7f9a56882640 1 -- 172.21.15.33:0/2867939667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a50100cf0 msgr2=0x7f9a50100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.985+0000 7f9a56882640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a50100cf0 0x7f9a50100750 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c970 tx=0x7f9a4000ce30 comp rx=0 tx=0).stop 2024-04-02T13:38:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.985+0000 7f9a56882640 1 -- 172.21.15.33:0/2867939667 shutdown_connections 2024-04-02T13:38:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.985+0000 7f9a56882640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9a3c0a3c40 0x7f9a3c0a6100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.985+0000 7f9a56882640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a50101fc0 0x7f9a5019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.985+0000 7f9a56882640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a501016b0 0x7f9a5019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.985+0000 7f9a56882640 1 --2- 172.21.15.33:0/2867939667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a50100cf0 0x7f9a50100750 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.985+0000 7f9a56882640 1 -- 172.21.15.33:0/2867939667 >> 172.21.15.33:0/2867939667 conn(0x7f9a500fb700 msgr2=0x7f9a50104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.985+0000 7f9a56882640 1 -- 172.21.15.33:0/2867939667 shutdown_connections 2024-04-02T13:38:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:12.985+0000 7f9a56882640 1 -- 172.21.15.33:0/2867939667 wait complete. 2024-04-02T13:38:12.998 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-02T13:38:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.099+0000 7f5b6efcd640 1 -- 172.21.15.33:0/1824737424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b681025c0 msgr2=0x7f5b68108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.099+0000 7f5b6efcd640 1 --2- 172.21.15.33:0/1824737424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b681025c0 0x7f5b68108e50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009ca0 tx=0x7f5b6002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.099+0000 7f5b6efcd640 1 -- 172.21.15.33:0/1824737424 shutdown_connections 2024-04-02T13:38:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.099+0000 7f5b6efcd640 1 --2- 172.21.15.33:0/1824737424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b681025c0 0x7f5b68108e50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.099+0000 7f5b6efcd640 1 --2- 172.21.15.33:0/1824737424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b68101c00 0x7f5b68102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.099+0000 7f5b6efcd640 1 --2- 172.21.15.33:0/1824737424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b68100a10 0x7f5b68100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.099+0000 7f5b6efcd640 1 -- 172.21.15.33:0/1824737424 >> 172.21.15.33:0/1824737424 conn(0x7f5b680fbfa0 msgr2=0x7f5b680fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.099+0000 7f5b6efcd640 1 -- 172.21.15.33:0/1824737424 shutdown_connections 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.099+0000 7f5b6efcd640 1 -- 172.21.15.33:0/1824737424 wait complete. 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6efcd640 1 Processor -- start 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6efcd640 1 -- start start 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6efcd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b68100a10 0x7f5b6810b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6efcd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b68101c00 0x7f5b681099d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6efcd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b681025c0 0x7f5b68109f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6efcd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6810d780 con 0x7f5b68100a10 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6efcd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6810d600 con 0x7f5b681025c0 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6efcd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6810d900 con 0x7f5b68101c00 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6cd69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b68100a10 0x7f5b6810b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6cd69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b68100a10 0x7f5b6810b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50568/0 (socket says 172.21.15.33:50568) 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6cd69640 1 -- 172.21.15.33:0/1993588027 learned_addr learned my addr 172.21.15.33:0/1993588027 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b67fff640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b68101c00 0x7f5b681099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6d56a640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b681025c0 0x7f5b68109f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6cd69640 1 -- 172.21.15.33:0/1993588027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b68101c00 msgr2=0x7f5b681099d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b68101c00 0x7f5b681099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6cd69640 1 -- 172.21.15.33:0/1993588027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b681025c0 msgr2=0x7f5b68109f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b681025c0 0x7f5b68109f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6cd69640 1 -- 172.21.15.33:0/1993588027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6810a450 con 0x7f5b68100a10 2024-04-02T13:38:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.100+0000 7f5b6d56a640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b681025c0 0x7f5b68109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.101+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b68100a10 0x7f5b6810b2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c970 tx=0x7f5b5c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.101+0000 7f5b65ffb640 1 -- 172.21.15.33:0/1993588027 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c004220 con 0x7f5b68100a10 2024-04-02T13:38:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.101+0000 7f5b65ffb640 1 -- 172.21.15.33:0/1993588027 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5b5c0043c0 con 0x7f5b68100a10 2024-04-02T13:38:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.101+0000 7f5b6efcd640 1 -- 172.21.15.33:0/1993588027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6810a740 con 0x7f5b68100a10 2024-04-02T13:38:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.101+0000 7f5b65ffb640 1 -- 172.21.15.33:0/1993588027 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c004a00 con 0x7f5b68100a10 2024-04-02T13:38:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.101+0000 7f5b6efcd640 1 -- 172.21.15.33:0/1993588027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b681a68c0 con 0x7f5b68100a10 2024-04-02T13:38:13.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.103+0000 7f5b65ffb640 1 -- 172.21.15.33:0/1993588027 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5b5c004ba0 con 0x7f5b68100a10 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b71bbc740 1 Processor -- start 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b71bbc740 1 -- start start 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b65ffb640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5b580a3cc0 0x7f5b580a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b6d56a640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5b580a3cc0 0x7f5b580a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b71bbc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 0x562365337f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b71bbc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623653384d0 0x56236533cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b71bbc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 0x56236533fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b71bbc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56236529c060 con 0x562365337ae0 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b71bbc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562365244650 con 0x56236533d640 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b65ffb640 1 -- 172.21.15.33:0/1993588027 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5b5c0d0670 con 0x7f5b68100a10 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b71bbc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562365298010 con 0x5623653384d0 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b6cd69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 0x562365337f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b6cd69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 0x562365337f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50576/0 (socket says 172.21.15.33:50576) 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b6cd69640 1 -- 172.21.15.33:0/3494794899 learned_addr learned my addr 172.21.15.33:0/3494794899 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:13.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b67fff640 1 --2- 172.21.15.33:0/3494794899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623653384d0 0x56236533cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b6d56a640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5b580a3cc0 0x7f5b580a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5b60032460 tx=0x7f5b60009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b6cd69640 1 -- 172.21.15.33:0/3494794899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623653384d0 msgr2=0x56236533cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b6d56a640 1 --2- 172.21.15.33:0/3494794899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 0x56236533fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/3494794899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623653384d0 0x56236533cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b6cd69640 1 -- 172.21.15.33:0/3494794899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 msgr2=0x56236533fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/3494794899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 0x56236533fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b6cd69640 1 -- 172.21.15.33:0/3494794899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562364fb02d0 con 0x562365337ae0 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.104+0000 7f5b67fff640 1 --2- 172.21.15.33:0/3494794899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623653384d0 0x56236533cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.105+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/3494794899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 0x562365337f00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0a4b70 tx=0x7f5b5c09c0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efea03bc19921092 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.105+0000 7f5b52ffd640 1 -- 172.21.15.33:0/3494794899 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c01d070 con 0x562365337ae0 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.105+0000 7f5b52ffd640 1 -- 172.21.15.33:0/3494794899 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5b5c09a730 con 0x562365337ae0 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.105+0000 7f5b52ffd640 1 -- 172.21.15.33:0/3494794899 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c098180 con 0x562365337ae0 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.105+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3494794899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 msgr2=0x562365337f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.105+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3494794899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 0x562365337f00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0a4b70 tx=0x7f5b5c09c0a0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.105+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3494794899 shutdown_connections 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.105+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3494794899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 0x56236533fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.105+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3494794899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623653384d0 0x56236533cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.105+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3494794899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 0x562365337f00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.105+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3494794899 >> 172.21.15.33:0/3494794899 conn(0x562365333050 msgr2=0x562365333450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.105+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3494794899 shutdown_connections 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.105+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3494794899 wait complete. 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b71bbc740 1 -- start start 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b71bbc740 1 Processor -- start 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b71bbc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562365337ae0 0x5623653420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b71bbc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623653384d0 0x5623653425e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b71bbc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56236533d640 0x562365349620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b71bbc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5623653405f0 con 0x56236533d640 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b71bbc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562365257a10 con 0x5623653384d0 2024-04-02T13:38:13.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b71bbc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562365340770 con 0x562365337ae0 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b71bbc740 10 client.0 _ll_register_callbacks cb 0x5623652940b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b6d56a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56236533d640 0x562365349620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b6d56a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56236533d640 0x562365349620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50584/0 (socket says 172.21.15.33:50584) 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b6d56a640 1 -- 172.21.15.33:0/3429938850 learned_addr learned my addr 172.21.15.33:0/3429938850 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b67fff640 1 --2- 172.21.15.33:0/3429938850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623653384d0 0x5623653425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.106+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/3429938850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562365337ae0 0x5623653420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b6d56a640 1 -- 172.21.15.33:0/3429938850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562365337ae0 msgr2=0x5623653420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b6d56a640 1 --2- 172.21.15.33:0/3429938850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562365337ae0 0x5623653420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b6d56a640 1 -- 172.21.15.33:0/3429938850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623653384d0 msgr2=0x5623653425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b6d56a640 1 --2- 172.21.15.33:0/3429938850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623653384d0 0x5623653425e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b6d56a640 1 -- 172.21.15.33:0/3429938850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56236507a6b0 con 0x56236533d640 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b6d56a640 1 --2- 172.21.15.33:0/3429938850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56236533d640 0x562365349620 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b60036000 tx=0x7f5b6002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36dc5ce09f65cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b47fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b47fff640 1 -- 172.21.15.33:0/3429938850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60039040 con 0x56236533d640 2024-04-02T13:38:13.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b47fff640 1 -- 172.21.15.33:0/3429938850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5b60003d00 con 0x56236533d640 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3429938850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b5c09a730 con 0x56236533d640 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b47fff640 1 -- 172.21.15.33:0/3429938850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5b60058050 con 0x56236533d640 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b71bbc740 10 client.15237 fetch_fsmap learned FSMap version 24 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3429938850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56236534c9a0 con 0x56236533d640 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b47fff640 1 -- 172.21.15.33:0/3429938850 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5b5c09a730 con 0x56236533d640 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.107+0000 7f5b47fff640 1 -- 172.21.15.33:0/3429938850 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5b60040820 con 0x56236533d640 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.108+0000 7f5b71bbc740 10 client.15237 fetch_fsmap finished waiting for FSMap version 24 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.108+0000 7f5b71bbc740 10 client.15237 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.108+0000 7f5b71bbc740 20 client.15237 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.108+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3429938850 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56236534f120 0x562365350590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.108+0000 7f5b71bbc740 4 client.15237 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.108+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3429938850 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562365243cf0 con 0x56236534f120 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.108+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/3429938850 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56236534f120 0x562365350590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.108+0000 7f5b47fff640 10 client.15237 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.108+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/3429938850 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56236534f120 0x562365350590 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b603c4ae8ac8ee2 server_cookie=b62266354041b3fb in_seq=0 out_seq=0 2024-04-02T13:38:13.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.109+0000 7f5b47fff640 1 -- 172.21.15.33:0/3429938850 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562365243cf0 con 0x56236534f120 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.109+0000 7f5b47fff640 10 client.15237 handle_command_reply: tid=0 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.109+0000 7f5b71bbc740 1 client.15237 shutdown 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.109+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3429938850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56236533d640 msgr2=0x562365349620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.109+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3429938850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56236533d640 0x562365349620 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b60036000 tx=0x7f5b6002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.109+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3429938850 shutdown_connections 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.109+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3429938850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56236533d640 0x562365349620 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.109+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3429938850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623653384d0 0x5623653425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.109+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3429938850 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56236534f120 0x562365350590 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.109+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3429938850 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562365337ae0 0x5623653420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.109+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3429938850 >> 172.21.15.33:0/3429938850 conn(0x562365333050 msgr2=0x562365332190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.109+0000 7f5b47fff640 0 client.15237 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.110+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3429938850 shutdown_connections 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.110+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3429938850 wait complete. 2024-04-02T13:38:13.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.110+0000 7f5b71bbc740 20 client.15237 trim_cache size 0 max 16384 2024-04-02T13:38:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b71bbc740 1 Processor -- start 2024-04-02T13:38:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b71bbc740 1 -- start start 2024-04-02T13:38:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b71bbc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 0x562365333430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b71bbc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623653384d0 0x5623653495e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b71bbc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 0x562365331900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b71bbc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56236534e8d0 con 0x562365337ae0 2024-04-02T13:38:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b71bbc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56236534efa0 con 0x56236533d640 2024-04-02T13:38:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b71bbc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56236534f670 con 0x5623653384d0 2024-04-02T13:38:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b6cd69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 0x562365333430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b6cd69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 0x562365333430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50590/0 (socket says 172.21.15.33:50590) 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b6cd69640 1 -- 172.21.15.33:0/3953711646 learned_addr learned my addr 172.21.15.33:0/3953711646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b6cd69640 1 -- 172.21.15.33:0/3953711646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623653384d0 msgr2=0x5623653495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/3953711646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623653384d0 0x5623653495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b6d56a640 1 --2- 172.21.15.33:0/3953711646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 0x562365331900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b6cd69640 1 -- 172.21.15.33:0/3953711646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 msgr2=0x562365331900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/3953711646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 0x562365331900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b6cd69640 1 -- 172.21.15.33:0/3953711646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562365343220 con 0x562365337ae0 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b6d56a640 1 --2- 172.21.15.33:0/3953711646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 0x562365331900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/3953711646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 0x562365333430 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c098890 tx=0x7f5b5c0051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d500e8f709803b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.128+0000 7f5b527fc640 1 -- 172.21.15.33:0/3953711646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c01d040 con 0x562365337ae0 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.129+0000 7f5b527fc640 1 -- 172.21.15.33:0/3953711646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5b5c0d0380 con 0x562365337ae0 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.129+0000 7f5b527fc640 1 -- 172.21.15.33:0/3953711646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c098470 con 0x562365337ae0 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.129+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3953711646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 msgr2=0x562365333430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.129+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3953711646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 0x562365333430 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c098890 tx=0x7f5b5c0051b0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.129+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3953711646 shutdown_connections 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.129+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3953711646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 0x562365331900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.129+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3953711646 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5623653384d0 0x5623653495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.129+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/3953711646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562365337ae0 0x562365333430 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.129+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3953711646 >> 172.21.15.33:0/3953711646 conn(0x562365333050 msgr2=0x562365338f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.129+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3953711646 shutdown_connections 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.129+0000 7f5b71bbc740 1 -- 172.21.15.33:0/3953711646 wait complete. 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.129+0000 7f5b71bbc740 1 -- start start 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.129+0000 7f5b71bbc740 1 Processor -- start 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b71bbc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562365337ae0 0x56236533bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b71bbc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623653384d0 0x56236533c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b71bbc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 0x562365347cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b71bbc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562365350410 con 0x5623653384d0 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b71bbc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56236534fd40 con 0x56236533d640 2024-04-02T13:38:13.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b71bbc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562365350ae0 con 0x562365337ae0 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b71bbc740 10 client.0 _ll_register_callbacks cb 0x7f5b60037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623653384d0 0x56236533c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623653384d0 0x56236533c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50596/0 (socket says 172.21.15.33:50596) 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b67fff640 1 -- 172.21.15.33:0/1879626276 learned_addr learned my addr 172.21.15.33:0/1879626276 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562365337ae0 0x56236533bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b6d56a640 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 0x562365347cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b67fff640 1 -- 172.21.15.33:0/1879626276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562365337ae0 msgr2=0x56236533bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b67fff640 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562365337ae0 0x56236533bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b67fff640 1 -- 172.21.15.33:0/1879626276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 msgr2=0x562365347cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b67fff640 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 0x562365347cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b67fff640 1 -- 172.21.15.33:0/1879626276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623653482b0 con 0x5623653384d0 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562365337ae0 0x56236533bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b67fff640 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623653384d0 0x56236533c170 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x562365338f40 tx=0x7f5b5400f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e8322b30ce927a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.130+0000 7f5b47fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:13.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b47fff640 1 -- 172.21.15.33:0/1879626276 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5401d060 con 0x5623653384d0 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b47fff640 1 -- 172.21.15.33:0/1879626276 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5b5400e620 con 0x5623653384d0 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b71bbc740 1 -- 172.21.15.33:0/1879626276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b5c0d0380 con 0x5623653384d0 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b47fff640 1 -- 172.21.15.33:0/1879626276 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5b54024050 con 0x5623653384d0 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b71bbc740 10 client.15243 fetch_fsmap learned FSMap version 24 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b71bbc740 1 -- 172.21.15.33:0/1879626276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562365349600 con 0x5623653384d0 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b47fff640 1 -- 172.21.15.33:0/1879626276 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5b5c0d0380 con 0x5623653384d0 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b47fff640 1 -- 172.21.15.33:0/1879626276 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5b5400da50 con 0x5623653384d0 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b71bbc740 10 client.15243 fetch_fsmap finished waiting for FSMap version 24 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b71bbc740 10 client.15243 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b71bbc740 20 client.15243 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x5623653b97e0 0x5623653b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b71bbc740 4 client.15243 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b71bbc740 1 -- 172.21.15.33:0/1879626276 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "status"}) -- 0x5623652a95a0 con 0x5623653b97e0 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.131+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x5623653b97e0 0x5623653b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b6cd69640 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x5623653b97e0 0x5623653b84f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1bb32ff0b38f464 server_cookie=ac90f0858cfc954a in_seq=0 out_seq=0 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b47fff640 10 client.15243 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b47fff640 1 -- 172.21.15.33:0/1879626276 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5623652a95a0 con 0x5623653b97e0 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b47fff640 10 client.15243 handle_command_reply: tid=0 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b71bbc740 1 client.15243 shutdown 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b71bbc740 1 -- 172.21.15.33:0/1879626276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623653384d0 msgr2=0x56236533c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623653384d0 0x56236533c170 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x562365338f40 tx=0x7f5b5400f7d0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b71bbc740 1 -- 172.21.15.33:0/1879626276 shutdown_connections 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56236533d640 0x562365347cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5623653384d0 0x56236533c170 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x5623653b97e0 0x5623653b84f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b71bbc740 1 --2- 172.21.15.33:0/1879626276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562365337ae0 0x56236533bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b47fff640 0 client.15243 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b71bbc740 1 -- 172.21.15.33:0/1879626276 >> 172.21.15.33:0/1879626276 conn(0x562365333050 msgr2=0x562365333430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b71bbc740 1 -- 172.21.15.33:0/1879626276 shutdown_connections 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b71bbc740 1 -- 172.21.15.33:0/1879626276 wait complete. 2024-04-02T13:38:13.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.132+0000 7f5b71bbc740 20 client.15243 trim_cache size 0 max 16384 2024-04-02T13:38:13.142 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:38:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "b5fd982e-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:38:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-04-02T13:38:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24512, 2024-04-02T13:38:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-02T13:38:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-02T13:38:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-02T13:38:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 87.596130325000004, 2024-04-02T13:38:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-04-02T13:38:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 92, 2024-04-02T13:38:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:38:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 128.91930273599999 2024-04-02T13:38:13.143 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:38:13.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.133+0000 7f5b6efcd640 1 -- 172.21.15.33:0/1993588027 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5b580a3cc0 msgr2=0x7f5b580a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.133+0000 7f5b6efcd640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5b580a3cc0 0x7f5b580a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5b60032460 tx=0x7f5b60009940 comp rx=0 tx=0).stop 2024-04-02T13:38:13.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.133+0000 7f5b6efcd640 1 -- 172.21.15.33:0/1993588027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b68100a10 msgr2=0x7f5b6810b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.133+0000 7f5b6efcd640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b68100a10 0x7f5b6810b2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c970 tx=0x7f5b5c00ce30 comp rx=0 tx=0).stop 2024-04-02T13:38:13.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.133+0000 7f5b6efcd640 1 -- 172.21.15.33:0/1993588027 shutdown_connections 2024-04-02T13:38:13.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.133+0000 7f5b6efcd640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5b580a3cc0 0x7f5b580a6180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.133+0000 7f5b6efcd640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b681025c0 0x7f5b68109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.133+0000 7f5b6efcd640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b68101c00 0x7f5b681099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.133+0000 7f5b6efcd640 1 --2- 172.21.15.33:0/1993588027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b68100a10 0x7f5b6810b2f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.133+0000 7f5b6efcd640 1 -- 172.21.15.33:0/1993588027 >> 172.21.15.33:0/1993588027 conn(0x7f5b680fbfa0 msgr2=0x7f5b68104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:13.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.133+0000 7f5b6efcd640 1 -- 172.21.15.33:0/1993588027 shutdown_connections 2024-04-02T13:38:13.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.133+0000 7f5b6efcd640 1 -- 172.21.15.33:0/1993588027 wait complete. 2024-04-02T13:38:13.147 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T13:38:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.248+0000 7f6e5c7af640 1 -- 172.21.15.33:0/1117892506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e54103100 msgr2=0x7f6e54103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.248+0000 7f6e5c7af640 1 --2- 172.21.15.33:0/1117892506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e54103100 0x7f6e54103520 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009900 tx=0x7f6e5002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.248+0000 7f6e5c7af640 1 -- 172.21.15.33:0/1117892506 shutdown_connections 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.248+0000 7f6e5c7af640 1 --2- 172.21.15.33:0/1117892506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e54100530 0x7f6e5410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.248+0000 7f6e5c7af640 1 --2- 172.21.15.33:0/1117892506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e540ffb70 0x7f6e540ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.248+0000 7f6e5c7af640 1 --2- 172.21.15.33:0/1117892506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e54103100 0x7f6e54103520 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.248+0000 7f6e5c7af640 1 -- 172.21.15.33:0/1117892506 >> 172.21.15.33:0/1117892506 conn(0x7f6e540fb730 msgr2=0x7f6e540fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.248+0000 7f6e5c7af640 1 -- 172.21.15.33:0/1117892506 shutdown_connections 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.248+0000 7f6e5c7af640 1 -- 172.21.15.33:0/1117892506 wait complete. 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.248+0000 7f6e5c7af640 1 Processor -- start 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.248+0000 7f6e5c7af640 1 -- start start 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5c7af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e540ffb70 0x7f6e54102800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5c7af640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e54100530 0x7f6e54100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5c7af640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e54103100 0x7f6e54101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5c7af640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5410d7b0 con 0x7f6e540ffb70 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5c7af640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5410d630 con 0x7f6e54100530 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5c7af640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5410d930 con 0x7f6e54103100 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5a54b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e540ffb70 0x7f6e54102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5a54b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e540ffb70 0x7f6e54102800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50622/0 (socket says 172.21.15.33:50622) 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5a54b640 1 -- 172.21.15.33:0/3954512289 learned_addr learned my addr 172.21.15.33:0/3954512289 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5ad4c640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e54103100 0x7f6e54101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e59d4a640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e54100530 0x7f6e54100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5a54b640 1 -- 172.21.15.33:0/3954512289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e54103100 msgr2=0x7f6e54101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5a54b640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e54103100 0x7f6e54101380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5a54b640 1 -- 172.21.15.33:0/3954512289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e54100530 msgr2=0x7f6e54100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5a54b640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e54100530 0x7f6e54100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5a54b640 1 -- 172.21.15.33:0/3954512289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e54101c10 con 0x7f6e540ffb70 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5a54b640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e540ffb70 0x7f6e54102800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6e50000c00 tx=0x7f6e50004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e4b7fe640 1 -- 172.21.15.33:0/3954512289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50043020 con 0x7f6e540ffb70 2024-04-02T13:38:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e4b7fe640 1 -- 172.21.15.33:0/3954512289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6e50004440 con 0x7f6e540ffb70 2024-04-02T13:38:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5c7af640 1 -- 172.21.15.33:0/3954512289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e541a1ea0 con 0x7f6e540ffb70 2024-04-02T13:38:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e4b7fe640 1 -- 172.21.15.33:0/3954512289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50003d70 con 0x7f6e540ffb70 2024-04-02T13:38:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.249+0000 7f6e5c7af640 1 -- 172.21.15.33:0/3954512289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e541a22b0 con 0x7f6e540ffb70 2024-04-02T13:38:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.251+0000 7f6e4b7fe640 1 -- 172.21.15.33:0/3954512289 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6e50046020 con 0x7f6e540ffb70 2024-04-02T13:38:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e5f398740 1 Processor -- start 2024-04-02T13:38:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e5f398740 1 -- start start 2024-04-02T13:38:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e5f398740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 0x55d3f9803f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e5f398740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f98044d0 0x55d3f9808fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e5f398740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9809640 0x55d3f980bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e5f398740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d3f9768060 con 0x55d3f9803ae0 2024-04-02T13:38:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e5f398740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d3f9710650 con 0x55d3f98044d0 2024-04-02T13:38:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e5f398740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d3f9764010 con 0x55d3f9809640 2024-04-02T13:38:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e4b7fe640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6e400a3c40 0x7f6e400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e5ad4c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 0x55d3f9803f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e5ad4c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 0x55d3f9803f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50626/0 (socket says 172.21.15.33:50626) 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e5ad4c640 1 -- 172.21.15.33:0/361560020 learned_addr learned my addr 172.21.15.33:0/361560020 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e5a54b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f98044d0 0x55d3f9808fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e5ad4c640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6e400a3c40 0x7f6e400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e59d4a640 1 --2- 172.21.15.33:0/361560020 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9809640 0x55d3f980bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.252+0000 7f6e4b7fe640 1 -- 172.21.15.33:0/3954512289 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6e500f25b0 con 0x7f6e540ffb70 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.253+0000 7f6e5ad4c640 1 -- 172.21.15.33:0/361560020 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9809640 msgr2=0x55d3f980bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.253+0000 7f6e5ad4c640 1 --2- 172.21.15.33:0/361560020 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9809640 0x55d3f980bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.253+0000 7f6e5ad4c640 1 -- 172.21.15.33:0/361560020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f98044d0 msgr2=0x55d3f9808fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.253+0000 7f6e5ad4c640 1 --2- 172.21.15.33:0/361560020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f98044d0 0x55d3f9808fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.253+0000 7f6e5ad4c640 1 -- 172.21.15.33:0/361560020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f947c2d0 con 0x55d3f9803ae0 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.253+0000 7f6e5a54b640 1 --2- 172.21.15.33:0/361560020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f98044d0 0x55d3f9808fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.253+0000 7f6e5ad4c640 1 --2- 172.21.15.33:0/361560020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 0x55d3f9803f00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c013e60 tx=0x7f6e4c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566a1c6347347e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.253+0000 7f6e5ad4c640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6e400a3c40 0x7f6e400a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c010ca0 tx=0x7f6e4c015000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:13.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.253+0000 7f6e48ff9640 1 -- 172.21.15.33:0/361560020 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00ae10 con 0x55d3f9803ae0 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.253+0000 7f6e48ff9640 1 -- 172.21.15.33:0/361560020 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6e4c007be0 con 0x55d3f9803ae0 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.253+0000 7f6e48ff9640 1 -- 172.21.15.33:0/361560020 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c01f600 con 0x55d3f9803ae0 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 -- 172.21.15.33:0/361560020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 msgr2=0x55d3f9803f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 --2- 172.21.15.33:0/361560020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 0x55d3f9803f00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c013e60 tx=0x7f6e4c0107a0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 -- 172.21.15.33:0/361560020 shutdown_connections 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 --2- 172.21.15.33:0/361560020 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9809640 0x55d3f980bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 --2- 172.21.15.33:0/361560020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f98044d0 0x55d3f9808fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 --2- 172.21.15.33:0/361560020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 0x55d3f9803f00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 -- 172.21.15.33:0/361560020 >> 172.21.15.33:0/361560020 conn(0x55d3f97ff050 msgr2=0x55d3f97ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 -- 172.21.15.33:0/361560020 shutdown_connections 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 -- 172.21.15.33:0/361560020 wait complete. 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 -- start start 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 Processor -- start 2024-04-02T13:38:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9803ae0 0x55d3f980df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f98044d0 0x55d3f980e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f9809640 0x55d3f9815510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d3f980c590 con 0x55d3f98044d0 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d3f9723a10 con 0x55d3f9809640 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.254+0000 7f6e5f398740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d3f980c710 con 0x55d3f9803ae0 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e5f398740 10 client.0 _ll_register_callbacks cb 0x55d3f97600b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e59d4a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f98044d0 0x55d3f980e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e59d4a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f98044d0 0x55d3f980e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50634/0 (socket says 172.21.15.33:50634) 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e59d4a640 1 -- 172.21.15.33:0/3397476910 learned_addr learned my addr 172.21.15.33:0/3397476910 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e5ad4c640 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f9809640 0x55d3f9815510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e5a54b640 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9803ae0 0x55d3f980df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e59d4a640 1 -- 172.21.15.33:0/3397476910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9803ae0 msgr2=0x55d3f980df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e59d4a640 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9803ae0 0x55d3f980df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e59d4a640 1 -- 172.21.15.33:0/3397476910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f9809640 msgr2=0x55d3f9815510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e59d4a640 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f9809640 0x55d3f9815510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e59d4a640 1 -- 172.21.15.33:0/3397476910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f95466b0 con 0x55d3f98044d0 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e5a54b640 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9803ae0 0x55d3f980df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e59d4a640 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f98044d0 0x55d3f980e4d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400cd50 tx=0x7f6e440039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3a3203046cef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e3dffb640 1 -- 172.21.15.33:0/3397476910 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44010840 con 0x55d3f98044d0 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e3dffb640 1 -- 172.21.15.33:0/3397476910 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6e440109e0 con 0x55d3f98044d0 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e5f398740 1 -- 172.21.15.33:0/3397476910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e4c007be0 con 0x55d3f98044d0 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e3dffb640 1 -- 172.21.15.33:0/3397476910 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6e44010be0 con 0x55d3f98044d0 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e5f398740 10 client.15255 fetch_fsmap learned FSMap version 24 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e5f398740 1 -- 172.21.15.33:0/3397476910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d3f98188c0 con 0x55d3f98044d0 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.255+0000 7f6e3dffb640 1 -- 172.21.15.33:0/3397476910 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6e4c007be0 con 0x55d3f98044d0 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.256+0000 7f6e3dffb640 1 -- 172.21.15.33:0/3397476910 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6e44018020 con 0x55d3f98044d0 2024-04-02T13:38:13.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.256+0000 7f6e5f398740 10 client.15255 fetch_fsmap finished waiting for FSMap version 24 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.256+0000 7f6e5f398740 10 client.15255 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.256+0000 7f6e5f398740 20 client.15255 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.256+0000 7f6e5f398740 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d3f981af90 0x55d3f981c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.256+0000 7f6e5f398740 4 client.15255 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.256+0000 7f6e5f398740 1 -- 172.21.15.33:0/3397476910 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3f970fcf0 con 0x55d3f981af90 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.256+0000 7f6e5a54b640 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d3f981af90 0x55d3f981c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.256+0000 7f6e5a54b640 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d3f981af90 0x55d3f981c400 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92c382d9f87bc31a server_cookie=8f9f1d38eff4fc57 in_seq=0 out_seq=0 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.256+0000 7f6e3dffb640 10 client.15255 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e3dffb640 1 -- 172.21.15.33:0/3397476910 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d3f970fcf0 con 0x55d3f981af90 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e3dffb640 10 client.15255 handle_command_reply: tid=0 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e5f398740 1 client.15255 shutdown 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e5f398740 1 -- 172.21.15.33:0/3397476910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f98044d0 msgr2=0x55d3f980e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e5f398740 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f98044d0 0x55d3f980e4d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400cd50 tx=0x7f6e440039c0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e5f398740 1 -- 172.21.15.33:0/3397476910 shutdown_connections 2024-04-02T13:38:13.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e5f398740 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d3f981af90 0x55d3f981c400 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e5f398740 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f9809640 0x55d3f9815510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e5f398740 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f98044d0 0x55d3f980e4d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e3dffb640 0 client.15255 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e5f398740 1 --2- 172.21.15.33:0/3397476910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9803ae0 0x55d3f980df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e5f398740 1 -- 172.21.15.33:0/3397476910 >> 172.21.15.33:0/3397476910 conn(0x55d3f97ff050 msgr2=0x55d3f97fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e5f398740 1 -- 172.21.15.33:0/3397476910 shutdown_connections 2024-04-02T13:38:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.257+0000 7f6e5f398740 1 -- 172.21.15.33:0/3397476910 wait complete. 2024-04-02T13:38:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.258+0000 7f6e5f398740 20 client.15255 trim_cache size 0 max 16384 2024-04-02T13:38:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e5f398740 1 Processor -- start 2024-04-02T13:38:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e5f398740 1 -- start start 2024-04-02T13:38:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e5f398740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9803ae0 0x55d3f9815f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e5f398740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f98044d0 0x55d3f98154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e5f398740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9809640 0x55d3f97ff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e5f398740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d3f981a740 con 0x55d3f9809640 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e5f398740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d3f981ae10 con 0x55d3f98044d0 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e5f398740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d3f981b4e0 con 0x55d3f9803ae0 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e59d4a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f98044d0 0x55d3f98154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e59d4a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f98044d0 0x55d3f98154d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:43942/0 (socket says 172.21.15.33:43942) 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e59d4a640 1 -- 172.21.15.33:0/4169336614 learned_addr learned my addr 172.21.15.33:0/4169336614 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e5a54b640 1 --2- 172.21.15.33:0/4169336614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9803ae0 0x55d3f9815f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e59d4a640 1 -- 172.21.15.33:0/4169336614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9803ae0 msgr2=0x55d3f9815f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e59d4a640 1 --2- 172.21.15.33:0/4169336614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9803ae0 0x55d3f9815f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e59d4a640 1 -- 172.21.15.33:0/4169336614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9809640 msgr2=0x55d3f97ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e59d4a640 1 --2- 172.21.15.33:0/4169336614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9809640 0x55d3f97ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e59d4a640 1 -- 172.21.15.33:0/4169336614 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f9808ca0 con 0x55d3f98044d0 2024-04-02T13:38:13.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.276+0000 7f6e5a54b640 1 --2- 172.21.15.33:0/4169336614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9803ae0 0x55d3f9815f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.277+0000 7f6e59d4a640 1 --2- 172.21.15.33:0/4169336614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f98044d0 0x55d3f98154d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400e3e0 tx=0x7f6e4400c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54ce3acd0bb543d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.277+0000 7f6e3ffff640 1 -- 172.21.15.33:0/4169336614 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4402b070 con 0x55d3f98044d0 2024-04-02T13:38:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.277+0000 7f6e3ffff640 1 -- 172.21.15.33:0/4169336614 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6e44005540 con 0x55d3f98044d0 2024-04-02T13:38:13.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.277+0000 7f6e3ffff640 1 -- 172.21.15.33:0/4169336614 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4400ec20 con 0x55d3f98044d0 2024-04-02T13:38:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.277+0000 7f6e5f398740 1 -- 172.21.15.33:0/4169336614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f98044d0 msgr2=0x55d3f98154d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.277+0000 7f6e5f398740 1 --2- 172.21.15.33:0/4169336614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f98044d0 0x55d3f98154d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400e3e0 tx=0x7f6e4400c9f0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.277+0000 7f6e5f398740 1 -- 172.21.15.33:0/4169336614 shutdown_connections 2024-04-02T13:38:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.277+0000 7f6e5f398740 1 --2- 172.21.15.33:0/4169336614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9809640 0x55d3f97ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.277+0000 7f6e5f398740 1 --2- 172.21.15.33:0/4169336614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f98044d0 0x55d3f98154d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.277+0000 7f6e5f398740 1 --2- 172.21.15.33:0/4169336614 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f9803ae0 0x55d3f9815f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.277+0000 7f6e5f398740 1 -- 172.21.15.33:0/4169336614 >> 172.21.15.33:0/4169336614 conn(0x55d3f97ff050 msgr2=0x55d3f9804f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.277+0000 7f6e5f398740 1 -- 172.21.15.33:0/4169336614 shutdown_connections 2024-04-02T13:38:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.277+0000 7f6e5f398740 1 -- 172.21.15.33:0/4169336614 wait complete. 2024-04-02T13:38:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5f398740 1 -- start start 2024-04-02T13:38:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5f398740 1 Processor -- start 2024-04-02T13:38:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5f398740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 0x55d3f9811330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5f398740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f98044d0 0x55d3f9811870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5f398740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f9809640 0x55d3f9814890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5f398740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d3f981c280 con 0x55d3f9803ae0 2024-04-02T13:38:13.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5f398740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d3f981bbb0 con 0x55d3f9809640 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5f398740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d3f981c950 con 0x55d3f98044d0 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5f398740 10 client.0 _ll_register_callbacks cb 0x7f6e44004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5a54b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 0x55d3f9811330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5a54b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 0x55d3f9811330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50664/0 (socket says 172.21.15.33:50664) 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5a54b640 1 -- 172.21.15.33:0/535497260 learned_addr learned my addr 172.21.15.33:0/535497260 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5ad4c640 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f9809640 0x55d3f9814890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e59d4a640 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f98044d0 0x55d3f9811870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5a54b640 1 -- 172.21.15.33:0/535497260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f98044d0 msgr2=0x55d3f9811870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5a54b640 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f98044d0 0x55d3f9811870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5a54b640 1 -- 172.21.15.33:0/535497260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f9809640 msgr2=0x55d3f9814890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5a54b640 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f9809640 0x55d3f9814890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5a54b640 1 -- 172.21.15.33:0/535497260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f9814e80 con 0x55d3f9803ae0 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5ad4c640 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f9809640 0x55d3f9814890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e59d4a640 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f98044d0 0x55d3f9811870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e5a54b640 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 0x55d3f9811330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6e500efd40 tx=0x7f6e50088fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5b1a6cbcdccdca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:13.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.278+0000 7f6e3dffb640 1 -- 172.21.15.33:0/535497260 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50089040 con 0x55d3f9803ae0 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e3dffb640 1 -- 172.21.15.33:0/535497260 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6e500035f0 con 0x55d3f9803ae0 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e5f398740 1 -- 172.21.15.33:0/535497260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e44005540 con 0x55d3f9803ae0 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e3dffb640 1 -- 172.21.15.33:0/535497260 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6e50004c40 con 0x55d3f9803ae0 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e5f398740 10 client.15258 fetch_fsmap learned FSMap version 24 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e5f398740 1 -- 172.21.15.33:0/535497260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d3f9881630 con 0x55d3f9803ae0 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e3dffb640 1 -- 172.21.15.33:0/535497260 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6e44005540 con 0x55d3f9803ae0 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e3dffb640 1 -- 172.21.15.33:0/535497260 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6e500bab50 con 0x55d3f9803ae0 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e5f398740 10 client.15258 fetch_fsmap finished waiting for FSMap version 24 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e5f398740 10 client.15258 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e5f398740 20 client.15258 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e5f398740 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d3f9886350 0x55d3f9885060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e5f398740 4 client.15258 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e5f398740 1 -- 172.21.15.33:0/535497260 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "status"}) -- 0x55d3f97755a0 con 0x55d3f9886350 2024-04-02T13:38:13.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.279+0000 7f6e59d4a640 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d3f9886350 0x55d3f9885060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e3dffb640 10 client.15258 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e59d4a640 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d3f9886350 0x55d3f9885060 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24de13834c2e55fb server_cookie=68495ba0af19511c in_seq=0 out_seq=0 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e3dffb640 1 -- 172.21.15.33:0/535497260 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55d3f97755a0 con 0x55d3f9886350 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e3dffb640 10 client.15258 handle_command_reply: tid=0 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e5f398740 1 client.15258 shutdown 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e5f398740 1 -- 172.21.15.33:0/535497260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 msgr2=0x55d3f9811330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e5f398740 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 0x55d3f9811330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6e500efd40 tx=0x7f6e50088fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e5f398740 1 -- 172.21.15.33:0/535497260 shutdown_connections 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e5f398740 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d3f9886350 0x55d3f9885060 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e5f398740 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3f9809640 0x55d3f9814890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e3dffb640 0 client.15258 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e5f398740 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3f98044d0 0x55d3f9811870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e5f398740 1 --2- 172.21.15.33:0/535497260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3f9803ae0 0x55d3f9811330 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e5f398740 1 -- 172.21.15.33:0/535497260 >> 172.21.15.33:0/535497260 conn(0x55d3f97ff050 msgr2=0x55d3f9804d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:13.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e5f398740 1 -- 172.21.15.33:0/535497260 shutdown_connections 2024-04-02T13:38:13.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e5f398740 1 -- 172.21.15.33:0/535497260 wait complete. 2024-04-02T13:38:13.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.280+0000 7f6e5f398740 20 client.15258 trim_cache size 0 max 16384 2024-04-02T13:38:13.290 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:38:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "b5fd982e-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:38:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-04-02T13:38:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24551, 2024-04-02T13:38:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-02T13:38:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-02T13:38:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-02T13:38:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 111.441036513, 2024-04-02T13:38:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-04-02T13:38:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 92, 2024-04-02T13:38:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:38:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 117.619611475 2024-04-02T13:38:13.291 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:38:13.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.281+0000 7f6e5c7af640 1 -- 172.21.15.33:0/3954512289 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6e400a3c40 msgr2=0x7f6e400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.281+0000 7f6e5c7af640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6e400a3c40 0x7f6e400a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c010ca0 tx=0x7f6e4c015000 comp rx=0 tx=0).stop 2024-04-02T13:38:13.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.281+0000 7f6e5c7af640 1 -- 172.21.15.33:0/3954512289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e540ffb70 msgr2=0x7f6e54102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:13.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.281+0000 7f6e5c7af640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e540ffb70 0x7f6e54102800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6e50000c00 tx=0x7f6e50004040 comp rx=0 tx=0).stop 2024-04-02T13:38:13.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.281+0000 7f6e5c7af640 1 -- 172.21.15.33:0/3954512289 shutdown_connections 2024-04-02T13:38:13.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.281+0000 7f6e5c7af640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6e400a3c40 0x7f6e400a6100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.281+0000 7f6e5c7af640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e54103100 0x7f6e54101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.281+0000 7f6e5c7af640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e54100530 0x7f6e54100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.281+0000 7f6e5c7af640 1 --2- 172.21.15.33:0/3954512289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e540ffb70 0x7f6e54102800 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:13.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.281+0000 7f6e5c7af640 1 -- 172.21.15.33:0/3954512289 >> 172.21.15.33:0/3954512289 conn(0x7f6e540fb730 msgr2=0x7f6e54106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:13.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.281+0000 7f6e5c7af640 1 -- 172.21.15.33:0/3954512289 shutdown_connections 2024-04-02T13:38:13.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:13.281+0000 7f6e5c7af640 1 -- 172.21.15.33:0/3954512289 wait complete. 2024-04-02T13:38:13.294 INFO:tasks.cephadm:Setup complete, yielding 2024-04-02T13:38:13.295 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T13:38:13.304 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-02T13:38:13.305 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch status' 2024-04-02T13:38:13.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:13 smithi033 ceph-mon[32390]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:38:13.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:13 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4081801563' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:13.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:13 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3383432376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:13.548 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:38:13.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:13 smithi145 ceph-mon[35646]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:38:13.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:13 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4081801563' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:13.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:13 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3383432376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:13.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:13 smithi050 ceph-mon[35524]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:38:13.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:13 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4081801563' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:13.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:13 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3383432376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:15.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.370+0000 7f34908b2700 1 -- 172.21.15.33:0/1236523890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f348c102620 msgr2=0x7f348c102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:15.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.370+0000 7f34908b2700 1 --2- 172.21.15.33:0/1236523890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f348c102620 0x7f348c102a40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f347c0099c0 tx=0x7f347c009cd0 comp rx=0 tx=0).stop 2024-04-02T13:38:15.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.370+0000 7f34908b2700 1 -- 172.21.15.33:0/1236523890 shutdown_connections 2024-04-02T13:38:15.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.370+0000 7f34908b2700 1 --2- 172.21.15.33:0/1236523890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f348c1041c0 0x7f348c10aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:15.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.370+0000 7f34908b2700 1 --2- 172.21.15.33:0/1236523890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f348c102620 0x7f348c102a40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:15.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.370+0000 7f34908b2700 1 --2- 172.21.15.33:0/1236523890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f348c103820 0x7f348c103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:15.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.370+0000 7f34908b2700 1 -- 172.21.15.33:0/1236523890 >> 172.21.15.33:0/1236523890 conn(0x7f348c0fdc00 msgr2=0x7f348c100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:15.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.370+0000 7f34908b2700 1 -- 172.21.15.33:0/1236523890 shutdown_connections 2024-04-02T13:38:15.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.370+0000 7f34908b2700 1 -- 172.21.15.33:0/1236523890 wait complete. 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.371+0000 7f34908b2700 1 Processor -- start 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.371+0000 7f34908b2700 1 -- start start 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.371+0000 7f34908b2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f348c102620 0x7f348c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.371+0000 7f34908b2700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f348c103820 0x7f348c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.371+0000 7f34908b2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f348c1041c0 0x7f348c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.371+0000 7f34908b2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f348c10e180 con 0x7f348c102620 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.371+0000 7f34908b2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f348c10e000 con 0x7f348c1041c0 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.371+0000 7f34908b2700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f348c10e300 con 0x7f348c103820 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.371+0000 7f348a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f348c102620 0x7f348c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f348a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f348c102620 0x7f348c198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39600/0 (socket says 172.21.15.33:39600) 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f348a59c700 1 -- 172.21.15.33:0/4281324949 learned_addr learned my addr 172.21.15.33:0/4281324949 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f3489d9b700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f348c103820 0x7f348c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f348ad9d700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f348c1041c0 0x7f348c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f348a59c700 1 -- 172.21.15.33:0/4281324949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f348c103820 msgr2=0x7f348c198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f348a59c700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f348c103820 0x7f348c198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f348a59c700 1 -- 172.21.15.33:0/4281324949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f348c1041c0 msgr2=0x7f348c19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f348a59c700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f348c1041c0 0x7f348c19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f348a59c700 1 -- 172.21.15.33:0/4281324949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348c1a04e0 con 0x7f348c102620 2024-04-02T13:38:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f348a59c700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f348c102620 0x7f348c198880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f347c005b40 tx=0x7f347c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f34837fe700 1 -- 172.21.15.33:0/4281324949 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c027070 con 0x7f348c102620 2024-04-02T13:38:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f34837fe700 1 -- 172.21.15.33:0/4281324949 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f347c00be70 con 0x7f348c102620 2024-04-02T13:38:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f34908b2700 1 -- 172.21.15.33:0/4281324949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348c1a0770 con 0x7f348c102620 2024-04-02T13:38:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f34837fe700 1 -- 172.21.15.33:0/4281324949 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c022c10 con 0x7f348c102620 2024-04-02T13:38:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.372+0000 7f34908b2700 1 -- 172.21.15.33:0/4281324949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f348c1a0c50 con 0x7f348c102620 2024-04-02T13:38:15.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.374+0000 7f34908b2700 1 -- 172.21.15.33:0/4281324949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f348c0692b0 con 0x7f348c102620 2024-04-02T13:38:15.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.375+0000 7f34837fe700 1 -- 172.21.15.33:0/4281324949 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f347c02a070 con 0x7f348c102620 2024-04-02T13:38:15.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.375+0000 7f34837fe700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f34780a3cd0 0x7f34780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:15.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.375+0000 7f3489d9b700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f34780a3cd0 0x7f34780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:15.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.376+0000 7f34837fe700 1 -- 172.21.15.33:0/4281324949 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f347c0d1620 con 0x7f348c102620 2024-04-02T13:38:15.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.376+0000 7f3489d9b700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f34780a3cd0 0x7f34780a6190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3474005950 tx=0x7f347400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:15.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.380+0000 7f34837fe700 1 -- 172.21.15.33:0/4281324949 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f347c013070 con 0x7f348c102620 2024-04-02T13:38:15.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.605+0000 7f34908b2700 1 -- 172.21.15.33:0/4281324949 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f348c062a30 con 0x7f34780a3cd0 2024-04-02T13:38:15.609 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:15 smithi033 ceph-mon[32390]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:38:15.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.607+0000 7f34837fe700 1 -- 172.21.15.33:0/4281324949 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f348c062a30 con 0x7f34780a3cd0 2024-04-02T13:38:15.610 INFO:teuthology.orchestra.run.smithi033.stdout:Backend: cephadm 2024-04-02T13:38:15.610 INFO:teuthology.orchestra.run.smithi033.stdout:Available: Yes 2024-04-02T13:38:15.610 INFO:teuthology.orchestra.run.smithi033.stdout:Paused: No 2024-04-02T13:38:15.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.609+0000 7f34908b2700 1 -- 172.21.15.33:0/4281324949 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f34780a3cd0 msgr2=0x7f34780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:15.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.609+0000 7f34908b2700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f34780a3cd0 0x7f34780a6190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3474005950 tx=0x7f347400b410 comp rx=0 tx=0).stop 2024-04-02T13:38:15.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.609+0000 7f34908b2700 1 -- 172.21.15.33:0/4281324949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f348c102620 msgr2=0x7f348c198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:15.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.609+0000 7f34908b2700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f348c102620 0x7f348c198880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f347c005b40 tx=0x7f347c009510 comp rx=0 tx=0).stop 2024-04-02T13:38:15.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.609+0000 7f34908b2700 1 -- 172.21.15.33:0/4281324949 shutdown_connections 2024-04-02T13:38:15.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.609+0000 7f34908b2700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f34780a3cd0 0x7f34780a6190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:15.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.609+0000 7f34908b2700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f348c1041c0 0x7f348c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:15.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.609+0000 7f34908b2700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f348c102620 0x7f348c198880 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:15.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.609+0000 7f34908b2700 1 --2- 172.21.15.33:0/4281324949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f348c103820 0x7f348c198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:15.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.609+0000 7f34908b2700 1 -- 172.21.15.33:0/4281324949 >> 172.21.15.33:0/4281324949 conn(0x7f348c0fdc00 msgr2=0x7f348c106c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:15.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.609+0000 7f34908b2700 1 -- 172.21.15.33:0/4281324949 shutdown_connections 2024-04-02T13:38:15.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:15.609+0000 7f34908b2700 1 -- 172.21.15.33:0/4281324949 wait complete. 2024-04-02T13:38:15.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:15 smithi145 ceph-mon[35646]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:38:15.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:15 smithi050 ceph-mon[35524]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:38:16.382 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ps' 2024-04-02T13:38:16.633 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:38:16.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:16 smithi145 ceph-mon[35646]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:16.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:16 smithi050 ceph-mon[35524]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:16.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:16 smithi033 ceph-mon[32390]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:17.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:17 smithi145 ceph-mon[35646]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:38:17.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:17 smithi145 ceph-mon[35646]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-02T13:38:17.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-04-02T13:38:17.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:17 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-02T13:38:17.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:17 smithi050 ceph-mon[35524]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:38:17.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:17 smithi050 ceph-mon[35524]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-02T13:38:17.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-04-02T13:38:17.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:17 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-02T13:38:17.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:17 smithi033 ceph-mon[32390]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:38:17.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:17 smithi033 ceph-mon[32390]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-02T13:38:17.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-04-02T13:38:17.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:17 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-02T13:38:18.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.464+0000 7f9d94f7d700 1 -- 172.21.15.33:0/1555808280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d90102650 msgr2=0x7f9d90102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:18.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.464+0000 7f9d94f7d700 1 --2- 172.21.15.33:0/1555808280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d90102650 0x7f9d90102a70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000b0d0 tx=0x7f9d8000b3e0 comp rx=0 tx=0).stop 2024-04-02T13:38:18.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.464+0000 7f9d94f7d700 1 -- 172.21.15.33:0/1555808280 shutdown_connections 2024-04-02T13:38:18.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.464+0000 7f9d94f7d700 1 --2- 172.21.15.33:0/1555808280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d901041f0 0x7f9d9010aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:18.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.464+0000 7f9d94f7d700 1 --2- 172.21.15.33:0/1555808280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d90102650 0x7f9d90102a70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:18.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.464+0000 7f9d94f7d700 1 --2- 172.21.15.33:0/1555808280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d90103850 0x7f9d90103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:18.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.464+0000 7f9d94f7d700 1 -- 172.21.15.33:0/1555808280 >> 172.21.15.33:0/1555808280 conn(0x7f9d900fdbf0 msgr2=0x7f9d90100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:18.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.464+0000 7f9d94f7d700 1 -- 172.21.15.33:0/1555808280 shutdown_connections 2024-04-02T13:38:18.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.464+0000 7f9d94f7d700 1 -- 172.21.15.33:0/1555808280 wait complete. 2024-04-02T13:38:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d94f7d700 1 Processor -- start 2024-04-02T13:38:18.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d94f7d700 1 -- start start 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d94f7d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d90102650 0x7f9d90076390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d94f7d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d90103850 0x7f9d900749e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d94f7d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d901041f0 0x7f9d90074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d94f7d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9010e280 con 0x7f9d901041f0 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d94f7d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9010e100 con 0x7f9d90103850 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d94f7d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9010e400 con 0x7f9d90102650 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d8ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d901041f0 0x7f9d90074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d8ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d901041f0 0x7f9d90074f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39614/0 (socket says 172.21.15.33:39614) 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d8ed9d700 1 -- 172.21.15.33:0/797665475 learned_addr learned my addr 172.21.15.33:0/797665475 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d8dd9b700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d90103850 0x7f9d900749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d8e59c700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d90102650 0x7f9d90076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d8ed9d700 1 -- 172.21.15.33:0/797665475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d90102650 msgr2=0x7f9d90076390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d8ed9d700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d90102650 0x7f9d90076390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:18.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d8ed9d700 1 -- 172.21.15.33:0/797665475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d90103850 msgr2=0x7f9d900749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:18.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d8ed9d700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d90103850 0x7f9d900749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:18.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.465+0000 7f9d8ed9d700 1 -- 172.21.15.33:0/797665475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d900757d0 con 0x7f9d901041f0 2024-04-02T13:38:18.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.466+0000 7f9d8ed9d700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d901041f0 0x7f9d90074f20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800d980 tx=0x7f9d8800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:18.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.466+0000 7f9d877fe700 1 -- 172.21.15.33:0/797665475 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88016070 con 0x7f9d901041f0 2024-04-02T13:38:18.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.466+0000 7f9d877fe700 1 -- 172.21.15.33:0/797665475 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9d88004d00 con 0x7f9d901041f0 2024-04-02T13:38:18.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.466+0000 7f9d94f7d700 1 -- 172.21.15.33:0/797665475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d901a2a50 con 0x7f9d901041f0 2024-04-02T13:38:18.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.466+0000 7f9d877fe700 1 -- 172.21.15.33:0/797665475 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88005620 con 0x7f9d901041f0 2024-04-02T13:38:18.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.466+0000 7f9d94f7d700 1 -- 172.21.15.33:0/797665475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d901a2f90 con 0x7f9d901041f0 2024-04-02T13:38:18.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.467+0000 7f9d94f7d700 1 -- 172.21.15.33:0/797665475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d900692b0 con 0x7f9d901041f0 2024-04-02T13:38:18.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.468+0000 7f9d877fe700 1 -- 172.21.15.33:0/797665475 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9d88004750 con 0x7f9d901041f0 2024-04-02T13:38:18.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.469+0000 7f9d877fe700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9d7c0a3cc0 0x7f9d7c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:18.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.469+0000 7f9d8e59c700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9d7c0a3cc0 0x7f9d7c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:18.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.469+0000 7f9d877fe700 1 -- 172.21.15.33:0/797665475 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f9d880d0950 con 0x7f9d901041f0 2024-04-02T13:38:18.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.469+0000 7f9d8e59c700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9d7c0a3cc0 0x7f9d7c0a6180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000d010 tx=0x7f9d8000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:18.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.472+0000 7f9d877fe700 1 -- 172.21.15.33:0/797665475 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9d8801f020 con 0x7f9d901041f0 2024-04-02T13:38:18.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.696+0000 7f9d94f7d700 1 -- 172.21.15.33:0/797665475 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f9d90062a30 con 0x7f9d7c0a3cc0 2024-04-02T13:38:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.710+0000 7f9d877fe700 1 -- 172.21.15.33:0/797665475 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f9d90062a30 con 0x7f9d7c0a3cc0 2024-04-02T13:38:18.712 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mds.a smithi033 running (2m) 23s ago 2m 16.3M - 19.0.0-2609-g191d963a f7301e070545 213ccd759d4c 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mds.b smithi050 running (2m) 56s ago 2m 25.0M - 19.0.0-2609-g191d963a f7301e070545 a6141b466874 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mds.c smithi145 running (2m) 56s ago 2m 18.4M - 19.0.0-2609-g191d963a f7301e070545 21a7f37a992b 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mds.d smithi033 running (2m) 23s ago 2m 14.0M - 19.0.0-2609-g191d963a f7301e070545 34b9deba6824 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mds.e smithi050 running (2m) 56s ago 2m 20.0M - 19.0.0-2609-g191d963a f7301e070545 985dba5ca7f6 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mds.f smithi145 running (2m) 56s ago 2m 15.9M - 19.0.0-2609-g191d963a f7301e070545 feb4f52be8ac 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mds.g smithi033 running (2m) 23s ago 2m 16.4M - 19.0.0-2609-g191d963a f7301e070545 ed7fe4339c38 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mds.h smithi050 running (2m) 56s ago 2m 18.8M - 19.0.0-2609-g191d963a f7301e070545 398fcfd4da0b 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mds.i smithi145 running (2m) 56s ago 2m 18.6M - 19.0.0-2609-g191d963a f7301e070545 df105ad19076 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mds.j smithi033 running (2m) 23s ago 2m 13.7M - 19.0.0-2609-g191d963a f7301e070545 e135d8098509 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mds.k smithi050 running (2m) 56s ago 2m 13.0M - 19.0.0-2609-g191d963a f7301e070545 a9da2e12226d 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mds.l smithi145 running (2m) 56s ago 2m 15.4M - 19.0.0-2609-g191d963a f7301e070545 cb3d3318fbc9 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.x smithi033 *:9283,8765 running (9m) 23s ago 9m 515M - 19.0.0-2609-g191d963a f7301e070545 0bdee59f1aa5 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.y smithi050 *:8443,8765 running (7m) 56s ago 7m 453M - 19.0.0-2609-g191d963a f7301e070545 aea4ca159235 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.z smithi145 *:8443,8765 running (7m) 56s ago 7m 455M - 19.0.0-2609-g191d963a f7301e070545 fb0412f4a589 2024-04-02T13:38:18.713 INFO:teuthology.orchestra.run.smithi033.stdout:mon.a smithi033 running (9m) 23s ago 9m 51.0M 2048M 19.0.0-2609-g191d963a f7301e070545 35cc49612e7c 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:mon.b smithi050 running (8m) 56s ago 8m 39.7M 2048M 19.0.0-2609-g191d963a f7301e070545 d9039c32ac9c 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:mon.c smithi145 running (8m) 56s ago 8m 39.9M 2048M 19.0.0-2609-g191d963a f7301e070545 6fc32637c96f 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (7m) 23s ago 7m 48.1M 4273M 19.0.0-2609-g191d963a f7301e070545 5fd88653491d 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi050 running (7m) 56s ago 7m 60.2M 4273M 19.0.0-2609-g191d963a f7301e070545 5607cb37233d 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi145 running (6m) 56s ago 6m 46.7M 4272M 19.0.0-2609-g191d963a f7301e070545 3872fc45c5b2 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi033 running (6m) 23s ago 6m 44.1M 4273M 19.0.0-2609-g191d963a f7301e070545 b33b1bfbe7d6 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi050 running (5m) 56s ago 5m 42.6M 4273M 19.0.0-2609-g191d963a f7301e070545 5f7dcf531e34 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi145 running (5m) 56s ago 5m 47.3M 4272M 19.0.0-2609-g191d963a f7301e070545 74cfd1be6218 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 smithi033 running (5m) 23s ago 5m 43.2M 4273M 19.0.0-2609-g191d963a f7301e070545 37c10c1a396d 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 smithi050 running (4m) 56s ago 4m 54.8M 4273M 19.0.0-2609-g191d963a f7301e070545 e36070aae524 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 smithi145 running (4m) 56s ago 4m 55.4M 4272M 19.0.0-2609-g191d963a f7301e070545 d0eb6b2b3da2 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 smithi033 running (3m) 23s ago 3m 58.3M 4273M 19.0.0-2609-g191d963a f7301e070545 ca187e333dd6 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 smithi050 running (3m) 56s ago 3m 54.0M 4273M 19.0.0-2609-g191d963a f7301e070545 4b100a3a758c 2024-04-02T13:38:18.714 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 smithi145 running (2m) 56s ago 2m 53.5M 4272M 19.0.0-2609-g191d963a f7301e070545 2a8ec003a4bf 2024-04-02T13:38:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.712+0000 7f9d94f7d700 1 -- 172.21.15.33:0/797665475 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9d7c0a3cc0 msgr2=0x7f9d7c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.712+0000 7f9d94f7d700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9d7c0a3cc0 0x7f9d7c0a6180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000d010 tx=0x7f9d8000b040 comp rx=0 tx=0).stop 2024-04-02T13:38:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.712+0000 7f9d94f7d700 1 -- 172.21.15.33:0/797665475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d901041f0 msgr2=0x7f9d90074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.712+0000 7f9d94f7d700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d901041f0 0x7f9d90074f20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800d980 tx=0x7f9d8800dc90 comp rx=0 tx=0).stop 2024-04-02T13:38:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.712+0000 7f9d94f7d700 1 -- 172.21.15.33:0/797665475 shutdown_connections 2024-04-02T13:38:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.712+0000 7f9d94f7d700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f9d7c0a3cc0 0x7f9d7c0a6180 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.712+0000 7f9d94f7d700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d901041f0 0x7f9d90074f20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.712+0000 7f9d94f7d700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9d90102650 0x7f9d90076390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.712+0000 7f9d94f7d700 1 --2- 172.21.15.33:0/797665475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d90103850 0x7f9d900749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.712+0000 7f9d94f7d700 1 -- 172.21.15.33:0/797665475 >> 172.21.15.33:0/797665475 conn(0x7f9d900fdbf0 msgr2=0x7f9d90106bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.712+0000 7f9d94f7d700 1 -- 172.21.15.33:0/797665475 shutdown_connections 2024-04-02T13:38:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:18.712+0000 7f9d94f7d700 1 -- 172.21.15.33:0/797665475 wait complete. 2024-04-02T13:38:18.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-04-02T13:38:18.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:18 smithi145 ceph-mon[35646]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-02T13:38:18.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:18 smithi145 ceph-mon[35646]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T13:38:18.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-04-02T13:38:18.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:18 smithi050 ceph-mon[35524]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-02T13:38:18.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:18 smithi050 ceph-mon[35524]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T13:38:19.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-04-02T13:38:19.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:18 smithi033 ceph-mon[32390]: from='mgr.14152 172.21.15.33:0/3726562746' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-02T13:38:19.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:18 smithi033 ceph-mon[32390]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T13:38:19.564 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ls' 2024-04-02T13:38:19.812 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:38:19.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:19 smithi033 ceph-mon[32390]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-02T13:38:19.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:19 smithi033 ceph-mon[32390]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T13:38:19.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:19 smithi033 ceph-mon[32390]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:19.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:19 smithi145 ceph-mon[35646]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-02T13:38:19.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:19 smithi145 ceph-mon[35646]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T13:38:19.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:19 smithi145 ceph-mon[35646]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:19.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:19 smithi050 ceph-mon[35524]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-02T13:38:19.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:19 smithi050 ceph-mon[35524]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T13:38:19.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:19 smithi050 ceph-mon[35524]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.484+0000 7f2604fb1700 1 -- 172.21.15.33:0/482086459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26001059e0 msgr2=0x7f2600107dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.484+0000 7f2604fb1700 1 --2- 172.21.15.33:0/482086459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26001059e0 0x7f2600107dd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009990 tx=0x7f25e8009ca0 comp rx=0 tx=0).stop 2024-04-02T13:38:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.484+0000 7f2604fb1700 1 -- 172.21.15.33:0/482086459 shutdown_connections 2024-04-02T13:38:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.484+0000 7f2604fb1700 1 --2- 172.21.15.33:0/482086459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2600108310 0x7f260010a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.484+0000 7f2604fb1700 1 --2- 172.21.15.33:0/482086459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26001030b0 0x7f26001054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.484+0000 7f2604fb1700 1 --2- 172.21.15.33:0/482086459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26001059e0 0x7f2600107dd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.484+0000 7f2604fb1700 1 -- 172.21.15.33:0/482086459 >> 172.21.15.33:0/482086459 conn(0x7f26000fcac0 msgr2=0x7f26000fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.484+0000 7f2604fb1700 1 -- 172.21.15.33:0/482086459 shutdown_connections 2024-04-02T13:38:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.484+0000 7f2604fb1700 1 -- 172.21.15.33:0/482086459 wait complete. 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f2604fb1700 1 Processor -- start 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f2604fb1700 1 -- start start 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f2604fb1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26001030b0 0x7f260019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f2604fb1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26001059e0 0x7f260019b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f2604fb1700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2600108310 0x7f26001a2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f2604fb1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f260010e200 con 0x7f26001030b0 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f2604fb1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f260010e080 con 0x7f26001059e0 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f2604fb1700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f260010e380 con 0x7f2600108310 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f25fe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26001030b0 0x7f260019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f25fe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26001030b0 0x7f260019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39626/0 (socket says 172.21.15.33:39626) 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f25fe59c700 1 -- 172.21.15.33:0/3580600487 learned_addr learned my addr 172.21.15.33:0/3580600487 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f25fdd9b700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26001059e0 0x7f260019b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f25fed9d700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2600108310 0x7f26001a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.485+0000 7f25fe59c700 1 -- 172.21.15.33:0/3580600487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2600108310 msgr2=0x7f26001a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.486+0000 7f25fe59c700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2600108310 0x7f26001a2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:21.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.486+0000 7f25fe59c700 1 -- 172.21.15.33:0/3580600487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26001059e0 msgr2=0x7f260019b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:21.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.486+0000 7f25fe59c700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26001059e0 0x7f260019b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:21.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.486+0000 7f25fe59c700 1 -- 172.21.15.33:0/3580600487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26001a2720 con 0x7f26001030b0 2024-04-02T13:38:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.486+0000 7f25fe59c700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26001030b0 0x7f260019aac0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f25f0007f40 tx=0x7f25f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.486+0000 7f25f77fe700 1 -- 172.21.15.33:0/3580600487 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f000dd50 con 0x7f26001030b0 2024-04-02T13:38:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.486+0000 7f2604fb1700 1 -- 172.21.15.33:0/3580600487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26001a2a10 con 0x7f26001030b0 2024-04-02T13:38:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.486+0000 7f25f77fe700 1 -- 172.21.15.33:0/3580600487 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f25f0004bb0 con 0x7f26001030b0 2024-04-02T13:38:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.486+0000 7f25f77fe700 1 -- 172.21.15.33:0/3580600487 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f00133f0 con 0x7f26001030b0 2024-04-02T13:38:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.486+0000 7f2604fb1700 1 -- 172.21.15.33:0/3580600487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26001a2f00 con 0x7f26001030b0 2024-04-02T13:38:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.487+0000 7f2604fb1700 1 -- 172.21.15.33:0/3580600487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26000692b0 con 0x7f26001030b0 2024-04-02T13:38:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.488+0000 7f25f77fe700 1 -- 172.21.15.33:0/3580600487 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f25f001a020 con 0x7f26001030b0 2024-04-02T13:38:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.489+0000 7f25f77fe700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f25ec0a3c40 0x7f25ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.489+0000 7f25fdd9b700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f25ec0a3c40 0x7f25ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.489+0000 7f25f77fe700 1 -- 172.21.15.33:0/3580600487 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f25f00d00c0 con 0x7f26001030b0 2024-04-02T13:38:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.489+0000 7f25fdd9b700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f25ec0a3c40 0x7f25ec0a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f25e800b5c0 tx=0x7f25e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:21.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.494+0000 7f25f77fe700 1 -- 172.21.15.33:0/3580600487 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f25f0010070 con 0x7f26001030b0 2024-04-02T13:38:21.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.714+0000 7f2604fb1700 1 -- 172.21.15.33:0/3580600487 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f2600062af0 con 0x7f25ec0a3c40 2024-04-02T13:38:21.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.720+0000 7f25f77fe700 1 -- 172.21.15.33:0/3580600487 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f2600062af0 con 0x7f25ec0a3c40 2024-04-02T13:38:21.723 INFO:teuthology.orchestra.run.smithi033.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-02T13:38:21.723 INFO:teuthology.orchestra.run.smithi033.stdout:mds.all 12/12 59s ago 2m smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-04-02T13:38:21.723 INFO:teuthology.orchestra.run.smithi033.stdout:mgr 3/3 59s ago 8m smithi033=x;smithi050=y;smithi145=z;count:3 2024-04-02T13:38:21.723 INFO:teuthology.orchestra.run.smithi033.stdout:mon 3/3 59s ago 8m smithi033:172.21.15.33=a;smithi050:172.21.15.50=b;smithi145:172.21.15.145=c;count:3 2024-04-02T13:38:21.723 INFO:teuthology.orchestra.run.smithi033.stdout:osd 12 59s ago - 2024-04-02T13:38:21.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.722+0000 7f2604fb1700 1 -- 172.21.15.33:0/3580600487 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f25ec0a3c40 msgr2=0x7f25ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:21.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.722+0000 7f2604fb1700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f25ec0a3c40 0x7f25ec0a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f25e800b5c0 tx=0x7f25e8005fb0 comp rx=0 tx=0).stop 2024-04-02T13:38:21.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.722+0000 7f2604fb1700 1 -- 172.21.15.33:0/3580600487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26001030b0 msgr2=0x7f260019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:21.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.722+0000 7f2604fb1700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26001030b0 0x7f260019aac0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f25f0007f40 tx=0x7f25f000d3b0 comp rx=0 tx=0).stop 2024-04-02T13:38:21.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.722+0000 7f2604fb1700 1 -- 172.21.15.33:0/3580600487 shutdown_connections 2024-04-02T13:38:21.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.722+0000 7f2604fb1700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f25ec0a3c40 0x7f25ec0a6100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:21.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.722+0000 7f2604fb1700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2600108310 0x7f26001a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:21.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.722+0000 7f2604fb1700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26001030b0 0x7f260019aac0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:21.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.722+0000 7f2604fb1700 1 --2- 172.21.15.33:0/3580600487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26001059e0 0x7f260019b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:21.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.722+0000 7f2604fb1700 1 -- 172.21.15.33:0/3580600487 >> 172.21.15.33:0/3580600487 conn(0x7f26000fcac0 msgr2=0x7f2600102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:21.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.722+0000 7f2604fb1700 1 -- 172.21.15.33:0/3580600487 shutdown_connections 2024-04-02T13:38:21.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:21.722+0000 7f2604fb1700 1 -- 172.21.15.33:0/3580600487 wait complete. 2024-04-02T13:38:21.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:21 smithi033 ceph-mon[32390]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-02T13:38:21.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:21 smithi145 ceph-mon[35646]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-02T13:38:21.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:21 smithi050 ceph-mon[35524]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-02T13:38:22.492 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch host ls' 2024-04-02T13:38:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:38:22.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:22 smithi033 ceph-mon[32390]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:22.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:22 smithi145 ceph-mon[35646]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:22.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:22 smithi050 ceph-mon[35524]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:23.871 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:23 smithi033 ceph-mon[32390]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s 2024-04-02T13:38:23.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:23 smithi145 ceph-mon[35646]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s 2024-04-02T13:38:23.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:23 smithi050 ceph-mon[35524]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s 2024-04-02T13:38:24.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.428+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1537838770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd8108300 msgr2=0x7f8bd810a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:24.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.428+0000 7f8bdfb15700 1 --2- 172.21.15.33:0/1537838770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd8108300 0x7f8bd810a770 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009990 tx=0x7f8bd0009ca0 comp rx=0 tx=0).stop 2024-04-02T13:38:24.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.428+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1537838770 shutdown_connections 2024-04-02T13:38:24.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.428+0000 7f8bdfb15700 1 --2- 172.21.15.33:0/1537838770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd8108300 0x7f8bd810a770 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:24.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.428+0000 7f8bdfb15700 1 --2- 172.21.15.33:0/1537838770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd81030a0 0x7f8bd8105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:24.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.428+0000 7f8bdfb15700 1 --2- 172.21.15.33:0/1537838770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8bd81059d0 0x7f8bd8107dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:24.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.428+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1537838770 >> 172.21.15.33:0/1537838770 conn(0x7f8bd80fcab0 msgr2=0x7f8bd80feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:24.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.428+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1537838770 shutdown_connections 2024-04-02T13:38:24.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.428+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1537838770 wait complete. 2024-04-02T13:38:24.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.429+0000 7f8bdfb15700 1 Processor -- start 2024-04-02T13:38:24.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.429+0000 7f8bdfb15700 1 -- start start 2024-04-02T13:38:24.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.429+0000 7f8bdfb15700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd81030a0 0x7f8bd81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:24.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.429+0000 7f8bdfb15700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd81059d0 0x7f8bd8198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:24.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.429+0000 7f8bdfb15700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8bd8108300 0x7f8bd819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.429+0000 7f8bdfb15700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd810e230 con 0x7f8bd81030a0 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.429+0000 7f8bdfb15700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd810e0b0 con 0x7f8bd81059d0 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.429+0000 7f8bdfb15700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd810e3b0 con 0x7f8bd8108300 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bdd8b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd81030a0 0x7f8bd81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bdd8b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd81030a0 0x7f8bd81987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44224/0 (socket says 172.21.15.33:44224) 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bdd8b1700 1 -- 172.21.15.33:0/1575845794 learned_addr learned my addr 172.21.15.33:0/1575845794 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bdd0b0700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd81059d0 0x7f8bd8198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bde0b2700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8bd8108300 0x7f8bd819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bdd8b1700 1 -- 172.21.15.33:0/1575845794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8bd8108300 msgr2=0x7f8bd819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bdd8b1700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8bd8108300 0x7f8bd819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bdd8b1700 1 -- 172.21.15.33:0/1575845794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd81059d0 msgr2=0x7f8bd8198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bdd8b1700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd81059d0 0x7f8bd8198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bdd8b1700 1 -- 172.21.15.33:0/1575845794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd81a0450 con 0x7f8bd81030a0 2024-04-02T13:38:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bdd8b1700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd81030a0 0x7f8bd81987f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8007ee0 tx=0x7f8bc800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:24.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bd6ffd700 1 -- 172.21.15.33:0/1575845794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc801a070 con 0x7f8bd81030a0 2024-04-02T13:38:24.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bd6ffd700 1 -- 172.21.15.33:0/1575845794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8bc800ddd0 con 0x7f8bd81030a0 2024-04-02T13:38:24.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1575845794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd81a0740 con 0x7f8bd81030a0 2024-04-02T13:38:24.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bd6ffd700 1 -- 172.21.15.33:0/1575845794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc80147c0 con 0x7f8bd81030a0 2024-04-02T13:38:24.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.430+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1575845794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd81a0d50 con 0x7f8bd81030a0 2024-04-02T13:38:24.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.431+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1575845794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd80692b0 con 0x7f8bd81030a0 2024-04-02T13:38:24.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.433+0000 7f8bd6ffd700 1 -- 172.21.15.33:0/1575845794 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8bc801d070 con 0x7f8bd81030a0 2024-04-02T13:38:24.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.433+0000 7f8bd6ffd700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8bc00a3cc0 0x7f8bc00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:24.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.434+0000 7f8bdd0b0700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8bc00a3cc0 0x7f8bc00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:24.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.434+0000 7f8bd6ffd700 1 -- 172.21.15.33:0/1575845794 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8bc80d0100 con 0x7f8bd81030a0 2024-04-02T13:38:24.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.434+0000 7f8bdd0b0700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8bc00a3cc0 0x7f8bc00a6180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009a10 tx=0x7f8bcc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.438+0000 7f8bd6ffd700 1 -- 172.21.15.33:0/1575845794 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8bc8010070 con 0x7f8bd81030a0 2024-04-02T13:38:24.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.655+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1575845794 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f8bd8062af0 con 0x7f8bc00a3cc0 2024-04-02T13:38:24.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.656+0000 7f8bd6ffd700 1 -- 172.21.15.33:0/1575845794 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f8bd8062af0 con 0x7f8bc00a3cc0 2024-04-02T13:38:24.659 INFO:teuthology.orchestra.run.smithi033.stdout:HOST ADDR LABELS STATUS 2024-04-02T13:38:24.659 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 172.21.15.33 2024-04-02T13:38:24.659 INFO:teuthology.orchestra.run.smithi033.stdout:smithi050 172.21.15.50 2024-04-02T13:38:24.659 INFO:teuthology.orchestra.run.smithi033.stdout:smithi145 172.21.15.145 2024-04-02T13:38:24.659 INFO:teuthology.orchestra.run.smithi033.stdout:3 hosts in cluster 2024-04-02T13:38:24.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.658+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1575845794 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8bc00a3cc0 msgr2=0x7f8bc00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:24.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.658+0000 7f8bdfb15700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8bc00a3cc0 0x7f8bc00a6180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009a10 tx=0x7f8bcc009190 comp rx=0 tx=0).stop 2024-04-02T13:38:24.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.658+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1575845794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd81030a0 msgr2=0x7f8bd81987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:24.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.658+0000 7f8bdfb15700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd81030a0 0x7f8bd81987f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8007ee0 tx=0x7f8bc800d3b0 comp rx=0 tx=0).stop 2024-04-02T13:38:24.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.658+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1575845794 shutdown_connections 2024-04-02T13:38:24.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.658+0000 7f8bdfb15700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f8bc00a3cc0 0x7f8bc00a6180 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:24.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.658+0000 7f8bdfb15700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8bd8108300 0x7f8bd819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:24.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.658+0000 7f8bdfb15700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd81030a0 0x7f8bd81987f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:24.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.658+0000 7f8bdfb15700 1 --2- 172.21.15.33:0/1575845794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bd81059d0 0x7f8bd8198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:24.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.658+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1575845794 >> 172.21.15.33:0/1575845794 conn(0x7f8bd80fcab0 msgr2=0x7f8bd81039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:24.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.659+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1575845794 shutdown_connections 2024-04-02T13:38:24.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:24.659+0000 7f8bdfb15700 1 -- 172.21.15.33:0/1575845794 wait complete. 2024-04-02T13:38:25.332 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch device ls' 2024-04-02T13:38:25.582 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:38:25.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:25 smithi033 ceph-mon[32390]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 895 B/s wr, 5 op/s 2024-04-02T13:38:25.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:25 smithi033 ceph-mon[32390]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:25.781 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:25 smithi145 ceph-mon[35646]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 895 B/s wr, 5 op/s 2024-04-02T13:38:25.781 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:25 smithi145 ceph-mon[35646]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:25.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:25 smithi050 ceph-mon[35524]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 895 B/s wr, 5 op/s 2024-04-02T13:38:25.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:25 smithi050 ceph-mon[35524]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.400+0000 7f494f614700 1 -- 172.21.15.33:0/3297455337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4948102620 msgr2=0x7f4948102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.400+0000 7f494f614700 1 --2- 172.21.15.33:0/3297455337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4948102620 0x7f4948102a40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f49440099c0 tx=0x7f4944009cd0 comp rx=0 tx=0).stop 2024-04-02T13:38:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.400+0000 7f494f614700 1 -- 172.21.15.33:0/3297455337 shutdown_connections 2024-04-02T13:38:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.400+0000 7f494f614700 1 --2- 172.21.15.33:0/3297455337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49481041c0 0x7f494810aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.400+0000 7f494f614700 1 --2- 172.21.15.33:0/3297455337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4948102620 0x7f4948102a40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.400+0000 7f494f614700 1 --2- 172.21.15.33:0/3297455337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948103820 0x7f4948103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.400+0000 7f494f614700 1 -- 172.21.15.33:0/3297455337 >> 172.21.15.33:0/3297455337 conn(0x7f49480fdc00 msgr2=0x7f4948100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.400+0000 7f494f614700 1 -- 172.21.15.33:0/3297455337 shutdown_connections 2024-04-02T13:38:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.400+0000 7f494f614700 1 -- 172.21.15.33:0/3297455337 wait complete. 2024-04-02T13:38:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.400+0000 7f494f614700 1 Processor -- start 2024-04-02T13:38:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.400+0000 7f494f614700 1 -- start start 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494f614700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4948102620 0x7f4948198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494f614700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948103820 0x7f4948198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494f614700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49481041c0 0x7f494819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494f614700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f494810e180 con 0x7f49481041c0 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494f614700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f494810e000 con 0x7f4948102620 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494f614700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f494810e300 con 0x7f4948103820 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494dbb1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49481041c0 0x7f494819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494dbb1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49481041c0 0x7f494819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44238/0 (socket says 172.21.15.33:44238) 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494dbb1700 1 -- 172.21.15.33:0/3825878816 learned_addr learned my addr 172.21.15.33:0/3825878816 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494d3b0700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4948102620 0x7f4948198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494cbaf700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948103820 0x7f4948198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494dbb1700 1 -- 172.21.15.33:0/3825878816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948103820 msgr2=0x7f4948198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494dbb1700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948103820 0x7f4948198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494dbb1700 1 -- 172.21.15.33:0/3825878816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4948102620 msgr2=0x7f4948198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494dbb1700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4948102620 0x7f4948198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494dbb1700 1 -- 172.21.15.33:0/3825878816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49481a03a0 con 0x7f49481041c0 2024-04-02T13:38:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494cbaf700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948103820 0x7f4948198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494d3b0700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4948102620 0x7f4948198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.401+0000 7f494dbb1700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49481041c0 0x7f494819fd00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4940007da0 tx=0x7f494000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.402+0000 7f493e7fc700 1 -- 172.21.15.33:0/3825878816 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940016020 con 0x7f49481041c0 2024-04-02T13:38:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.402+0000 7f493e7fc700 1 -- 172.21.15.33:0/3825878816 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f494000ad00 con 0x7f49481041c0 2024-04-02T13:38:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.402+0000 7f494f614700 1 -- 172.21.15.33:0/3825878816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49481a0690 con 0x7f49481041c0 2024-04-02T13:38:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.402+0000 7f493e7fc700 1 -- 172.21.15.33:0/3825878816 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49400115e0 con 0x7f49481041c0 2024-04-02T13:38:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.402+0000 7f494f614700 1 -- 172.21.15.33:0/3825878816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49481a0bd0 con 0x7f49481041c0 2024-04-02T13:38:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.403+0000 7f494f614700 1 -- 172.21.15.33:0/3825878816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49480692b0 con 0x7f49481041c0 2024-04-02T13:38:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.404+0000 7f493e7fc700 1 -- 172.21.15.33:0/3825878816 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4940019020 con 0x7f49481041c0 2024-04-02T13:38:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.405+0000 7f493e7fc700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f49340a3cc0 0x7f49340a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.405+0000 7f494d3b0700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f49340a3cc0 0x7f49340a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.405+0000 7f493e7fc700 1 -- 172.21.15.33:0/3825878816 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f49400d0b90 con 0x7f49481041c0 2024-04-02T13:38:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.405+0000 7f494d3b0700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f49340a3cc0 0x7f49340a6180 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4944003900 tx=0x7f4944005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.409+0000 7f493e7fc700 1 -- 172.21.15.33:0/3825878816 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4940020020 con 0x7f49481041c0 2024-04-02T13:38:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.628+0000 7f494f614700 1 -- 172.21.15.33:0/3825878816 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f4948062a30 con 0x7f49340a3cc0 2024-04-02T13:38:27.631 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:27 smithi033 ceph-mon[32390]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 731 B/s wr, 7 op/s 2024-04-02T13:38:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.629+0000 7f493e7fc700 1 -- 172.21.15.33:0/3825878816 <== mgr.14152 v2:172.21.15.33:6800/2562306694 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f4948062a30 con 0x7f49340a3cc0 2024-04-02T13:38:27.632 INFO:teuthology.orchestra.run.smithi033.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-02T13:38:27.632 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-02T13:38:27.632 INFO:teuthology.orchestra.run.smithi033.stdout:smithi050 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001F480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-02T13:38:27.632 INFO:teuthology.orchestra.run.smithi033.stdout:smithi145 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500N9400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-02T13:38:27.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.632+0000 7f494f614700 1 -- 172.21.15.33:0/3825878816 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f49340a3cc0 msgr2=0x7f49340a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:27.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.632+0000 7f494f614700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f49340a3cc0 0x7f49340a6180 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4944003900 tx=0x7f4944005fb0 comp rx=0 tx=0).stop 2024-04-02T13:38:27.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.632+0000 7f494f614700 1 -- 172.21.15.33:0/3825878816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49481041c0 msgr2=0x7f494819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:27.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.632+0000 7f494f614700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49481041c0 0x7f494819fd00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4940007da0 tx=0x7f494000a340 comp rx=0 tx=0).stop 2024-04-02T13:38:27.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.632+0000 7f494f614700 1 -- 172.21.15.33:0/3825878816 shutdown_connections 2024-04-02T13:38:27.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.632+0000 7f494f614700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f49340a3cc0 0x7f49340a6180 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:27.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.632+0000 7f494f614700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49481041c0 0x7f494819fd00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:27.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.632+0000 7f494f614700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4948102620 0x7f4948198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:27.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.632+0000 7f494f614700 1 --2- 172.21.15.33:0/3825878816 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4948103820 0x7f4948198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:27.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.632+0000 7f494f614700 1 -- 172.21.15.33:0/3825878816 >> 172.21.15.33:0/3825878816 conn(0x7f49480fdc00 msgr2=0x7f4948106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:27.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.632+0000 7f494f614700 1 -- 172.21.15.33:0/3825878816 shutdown_connections 2024-04-02T13:38:27.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:27.632+0000 7f494f614700 1 -- 172.21.15.33:0/3825878816 wait complete. 2024-04-02T13:38:27.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:27 smithi145 ceph-mon[35646]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 731 B/s wr, 7 op/s 2024-04-02T13:38:27.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:27 smithi050 ceph-mon[35524]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 731 B/s wr, 7 op/s 2024-04-02T13:38:28.340 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T13:38:28.349 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-02T13:38:28.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-02T13:38:28.599 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:38:28.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:28 smithi033 ceph-mon[32390]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:28.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:28 smithi145 ceph-mon[35646]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:28.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:28 smithi050 ceph-mon[35524]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:29.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:29 smithi033 ceph-mon[32390]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 7 op/s 2024-04-02T13:38:29.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:29 smithi050 ceph-mon[35524]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 7 op/s 2024-04-02T13:38:29.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:29 smithi145 ceph-mon[35646]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 7 op/s 2024-04-02T13:38:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.320+0000 7f289aaab700 1 -- 172.21.15.33:0/310825809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28941041e0 msgr2=0x7f289410aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.320+0000 7f289aaab700 1 --2- 172.21.15.33:0/310825809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28941041e0 0x7f289410aa90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f288c00b0d0 tx=0x7f288c00b3e0 comp rx=0 tx=0).stop 2024-04-02T13:38:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.320+0000 7f289aaab700 1 -- 172.21.15.33:0/310825809 shutdown_connections 2024-04-02T13:38:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.320+0000 7f289aaab700 1 --2- 172.21.15.33:0/310825809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28941041e0 0x7f289410aa90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.320+0000 7f289aaab700 1 --2- 172.21.15.33:0/310825809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2894102640 0x7f2894102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.320+0000 7f289aaab700 1 --2- 172.21.15.33:0/310825809 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2894103840 0x7f2894103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.320+0000 7f289aaab700 1 -- 172.21.15.33:0/310825809 >> 172.21.15.33:0/310825809 conn(0x7f28940fdba0 msgr2=0x7f2894100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.320+0000 7f289aaab700 1 -- 172.21.15.33:0/310825809 shutdown_connections 2024-04-02T13:38:30.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.320+0000 7f289aaab700 1 -- 172.21.15.33:0/310825809 wait complete. 2024-04-02T13:38:30.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.320+0000 7f289aaab700 1 Processor -- start 2024-04-02T13:38:30.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f289aaab700 1 -- start start 2024-04-02T13:38:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f289aaab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2894102640 0x7f2894076390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f289aaab700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2894103840 0x7f28940749e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f289aaab700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28941041e0 0x7f2894074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f289aaab700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f289410e270 con 0x7f2894102640 2024-04-02T13:38:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f289aaab700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f289410e0f0 con 0x7f28941041e0 2024-04-02T13:38:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f289aaab700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f289410e3f0 con 0x7f2894103840 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f2898847700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2894102640 0x7f2894076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f2898847700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2894102640 0x7f2894076390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44258/0 (socket says 172.21.15.33:44258) 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f2898847700 1 -- 172.21.15.33:0/2099170318 learned_addr learned my addr 172.21.15.33:0/2099170318 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f2893fff700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2894103840 0x7f28940749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f2899048700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28941041e0 0x7f2894074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f2898847700 1 -- 172.21.15.33:0/2099170318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2894103840 msgr2=0x7f28940749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f2898847700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2894103840 0x7f28940749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f2898847700 1 -- 172.21.15.33:0/2099170318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28941041e0 msgr2=0x7f2894074f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f2898847700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28941041e0 0x7f2894074f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f2898847700 1 -- 172.21.15.33:0/2099170318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28940757d0 con 0x7f2894102640 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f2899048700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28941041e0 0x7f2894074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f2898847700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2894102640 0x7f2894076390 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f288800e970 tx=0x7f288800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.321+0000 7f2891ffb700 1 -- 172.21.15.33:0/2099170318 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288800cbd0 con 0x7f2894102640 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.322+0000 7f2891ffb700 1 -- 172.21.15.33:0/2099170318 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f28880044f0 con 0x7f2894102640 2024-04-02T13:38:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.322+0000 7f289aaab700 1 -- 172.21.15.33:0/2099170318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28941a2980 con 0x7f2894102640 2024-04-02T13:38:30.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.322+0000 7f2891ffb700 1 -- 172.21.15.33:0/2099170318 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888010430 con 0x7f2894102640 2024-04-02T13:38:30.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.322+0000 7f289aaab700 1 -- 172.21.15.33:0/2099170318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28941a2f40 con 0x7f2894102640 2024-04-02T13:38:30.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.323+0000 7f289aaab700 1 -- 172.21.15.33:0/2099170318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28940692b0 con 0x7f2894102640 2024-04-02T13:38:30.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.324+0000 7f2891ffb700 1 -- 172.21.15.33:0/2099170318 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2888003e80 con 0x7f2894102640 2024-04-02T13:38:30.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.324+0000 7f2891ffb700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f28840a3cc0 0x7f28840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:30.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.324+0000 7f2893fff700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f28840a3cc0 0x7f28840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:30.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.324+0000 7f2891ffb700 1 -- 172.21.15.33:0/2099170318 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f28880d09d0 con 0x7f2894102640 2024-04-02T13:38:30.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.325+0000 7f2893fff700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f28840a3cc0 0x7f28840a6180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2880009a10 tx=0x7f2880009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:30.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.329+0000 7f2891ffb700 1 -- 172.21.15.33:0/2099170318 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f288809d020 con 0x7f2894102640 2024-04-02T13:38:30.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.584+0000 7f289aaab700 1 -- 172.21.15.33:0/2099170318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2894063200 con 0x7f2894102640 2024-04-02T13:38:30.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.584+0000 7f2891ffb700 1 -- 172.21.15.33:0/2099170318 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3052 (secure 0 0 0) 0x7f288809c920 con 0x7f2894102640 2024-04-02T13:38:30.587 INFO:teuthology.orchestra.run.smithi033.stdout:e24 2024-04-02T13:38:30.587 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T13:38:30.587 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T13:38:30.587 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-02T13:38:30.587 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:30.587 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-02T13:38:30.587 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-02T13:38:30.587 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 24 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-02T13:36:21.184650+0000 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-02T13:36:47.489590+0000 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 300 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T13:38:30.588 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 3 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1,2 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=24551,1=24512,2=24473} 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3,4] 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 0 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 24551 members: 24473,24512,24551 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b{0:24551} state up:active seq 3 addr [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:38:30.589 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.i{0:24484} state up:standby-replay seq 1 addr [v2:172.21.15.145:6838/3484016588,v1:172.21.15.145:6839/3484016588] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.h{1:24512} state up:active seq 12 addr [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c{1:24475} state up:standby-replay seq 1 addr [v2:172.21.15.145:6836/4234446304,v1:172.21.15.145:6837/4234446304] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.e{2:24473} state up:active seq 14 addr [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.f{2:24457} state up:standby-replay seq 1 addr [v2:172.21.15.145:6834/4110737670,v1:172.21.15.145:6835/4110737670] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.j{-1:14532} state up:standby seq 1 addr [v2:172.21.15.33:6834/1918898100,v1:172.21.15.33:6835/1918898100] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d{-1:14544} state up:standby seq 1 addr [v2:172.21.15.33:6836/1083419876,v1:172.21.15.33:6837/1083419876] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.g{-1:14613} state up:standby seq 1 addr [v2:172.21.15.33:6838/3175531919,v1:172.21.15.33:6839/3175531919] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a{-1:14625} state up:standby seq 1 addr [v2:172.21.15.33:6840/2274071500,v1:172.21.15.33:6841/2274071500] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.l{-1:24412} state up:standby seq 1 addr [v2:172.21.15.145:6832/3754674798,v1:172.21.15.145:6833/3754674798] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:38:30.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.k{-1:24434} state up:standby seq 1 addr [v2:172.21.15.50:6832/1584719197,v1:172.21.15.50:6833/1584719197] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:38:30.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.587+0000 7f289aaab700 1 -- 172.21.15.33:0/2099170318 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f28840a3cc0 msgr2=0x7f28840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:30.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.587+0000 7f289aaab700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f28840a3cc0 0x7f28840a6180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2880009a10 tx=0x7f2880009190 comp rx=0 tx=0).stop 2024-04-02T13:38:30.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.587+0000 7f289aaab700 1 -- 172.21.15.33:0/2099170318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2894102640 msgr2=0x7f2894076390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:30.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.587+0000 7f289aaab700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2894102640 0x7f2894076390 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f288800e970 tx=0x7f288800ec80 comp rx=0 tx=0).stop 2024-04-02T13:38:30.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.587+0000 7f289aaab700 1 -- 172.21.15.33:0/2099170318 shutdown_connections 2024-04-02T13:38:30.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.587+0000 7f289aaab700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f28840a3cc0 0x7f28840a6180 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:30.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.587+0000 7f289aaab700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28941041e0 0x7f2894074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:30.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.587+0000 7f289aaab700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2894102640 0x7f2894076390 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:30.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.587+0000 7f289aaab700 1 --2- 172.21.15.33:0/2099170318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2894103840 0x7f28940749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:30.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.587+0000 7f289aaab700 1 -- 172.21.15.33:0/2099170318 >> 172.21.15.33:0/2099170318 conn(0x7f28940fdba0 msgr2=0x7f2894106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:30.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.587+0000 7f289aaab700 1 -- 172.21.15.33:0/2099170318 shutdown_connections 2024-04-02T13:38:30.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:30.587+0000 7f289aaab700 1 -- 172.21.15.33:0/2099170318 wait complete. 2024-04-02T13:38:30.592 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-02T13:38:31.303 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph osd dump' 2024-04-02T13:38:31.550 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:38:31.793 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:31 smithi033 ceph-mon[32390]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 539 B/s wr, 7 op/s 2024-04-02T13:38:31.794 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:31 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2099170318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T13:38:31.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:31 smithi145 ceph-mon[35646]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 539 B/s wr, 7 op/s 2024-04-02T13:38:31.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:31 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2099170318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T13:38:31.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:31 smithi050 ceph-mon[35524]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 539 B/s wr, 7 op/s 2024-04-02T13:38:31.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:31 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2099170318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T13:38:33.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.212+0000 7f7d483c4700 1 -- 172.21.15.33:0/803391442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d40102620 msgr2=0x7f7d40102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.212+0000 7f7d483c4700 1 --2- 172.21.15.33:0/803391442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d40102620 0x7f7d40102a40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0099c0 tx=0x7f7d3c009cd0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.213+0000 7f7d483c4700 1 -- 172.21.15.33:0/803391442 shutdown_connections 2024-04-02T13:38:33.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.213+0000 7f7d483c4700 1 --2- 172.21.15.33:0/803391442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d401041c0 0x7f7d4010aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.213+0000 7f7d483c4700 1 --2- 172.21.15.33:0/803391442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d40102620 0x7f7d40102a40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.213+0000 7f7d483c4700 1 --2- 172.21.15.33:0/803391442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d40103820 0x7f7d40103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.213+0000 7f7d483c4700 1 -- 172.21.15.33:0/803391442 >> 172.21.15.33:0/803391442 conn(0x7f7d400fdc00 msgr2=0x7f7d40100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.213+0000 7f7d483c4700 1 -- 172.21.15.33:0/803391442 shutdown_connections 2024-04-02T13:38:33.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.213+0000 7f7d483c4700 1 -- 172.21.15.33:0/803391442 wait complete. 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.213+0000 7f7d483c4700 1 Processor -- start 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d483c4700 1 -- start start 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d483c4700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d40102620 0x7f7d40198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d483c4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d40103820 0x7f7d40198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d483c4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d401041c0 0x7f7d4019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d483c4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4010e1a0 con 0x7f7d401041c0 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d483c4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4010e020 con 0x7f7d40103820 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d483c4700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4010e320 con 0x7f7d40102620 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d46961700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d401041c0 0x7f7d4019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d46961700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d401041c0 0x7f7d4019fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44280/0 (socket says 172.21.15.33:44280) 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d46160700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d40102620 0x7f7d40198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d46961700 1 -- 172.21.15.33:0/2344848340 learned_addr learned my addr 172.21.15.33:0/2344848340 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d4595f700 1 --2- 172.21.15.33:0/2344848340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d40103820 0x7f7d40198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d46961700 1 -- 172.21.15.33:0/2344848340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d40102620 msgr2=0x7f7d40198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d46961700 1 --2- 172.21.15.33:0/2344848340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d40102620 0x7f7d40198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d46961700 1 -- 172.21.15.33:0/2344848340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d40103820 msgr2=0x7f7d40198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d46961700 1 --2- 172.21.15.33:0/2344848340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d40103820 0x7f7d40198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.214+0000 7f7d46961700 1 -- 172.21.15.33:0/2344848340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d401a03e0 con 0x7f7d401041c0 2024-04-02T13:38:33.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.215+0000 7f7d46961700 1 --2- 172.21.15.33:0/2344848340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d401041c0 0x7f7d4019fd40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7d38007f10 tx=0x7f7d3800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.215+0000 7f7d377fe700 1 -- 172.21.15.33:0/2344848340 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38004bb0 con 0x7f7d401041c0 2024-04-02T13:38:33.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.215+0000 7f7d377fe700 1 -- 172.21.15.33:0/2344848340 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7d38004d50 con 0x7f7d401041c0 2024-04-02T13:38:33.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.215+0000 7f7d483c4700 1 -- 172.21.15.33:0/2344848340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d401a06d0 con 0x7f7d401041c0 2024-04-02T13:38:33.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.215+0000 7f7d377fe700 1 -- 172.21.15.33:0/2344848340 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38005a00 con 0x7f7d401041c0 2024-04-02T13:38:33.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.215+0000 7f7d483c4700 1 -- 172.21.15.33:0/2344848340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d401a0c60 con 0x7f7d401041c0 2024-04-02T13:38:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.216+0000 7f7d483c4700 1 -- 172.21.15.33:0/2344848340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d400692b0 con 0x7f7d401041c0 2024-04-02T13:38:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.217+0000 7f7d377fe700 1 -- 172.21.15.33:0/2344848340 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7d3801a020 con 0x7f7d401041c0 2024-04-02T13:38:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.218+0000 7f7d377fe700 1 --2- 172.21.15.33:0/2344848340 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7d2c0a3cc0 0x7f7d2c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.218+0000 7f7d46160700 1 --2- 172.21.15.33:0/2344848340 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7d2c0a3cc0 0x7f7d2c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.218+0000 7f7d377fe700 1 -- 172.21.15.33:0/2344848340 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7d380cfe90 con 0x7f7d401041c0 2024-04-02T13:38:33.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.218+0000 7f7d46160700 1 --2- 172.21.15.33:0/2344848340 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7d2c0a3cc0 0x7f7d2c0a6180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c003900 tx=0x7f7d3c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.222+0000 7f7d377fe700 1 -- 172.21.15.33:0/2344848340 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7d38010070 con 0x7f7d401041c0 2024-04-02T13:38:33.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.433+0000 7f7d483c4700 1 -- 172.21.15.33:0/2344848340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f7d40063200 con 0x7f7d401041c0 2024-04-02T13:38:33.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.434+0000 7f7d377fe700 1 -- 172.21.15.33:0/2344848340 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5630 (secure 0 0 0) 0x7f7d3809c480 con 0x7f7d401041c0 2024-04-02T13:38:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 94 2024-04-02T13:38:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:38:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-02T13:28:21.504900+0000 2024-04-02T13:38:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-02T13:38:18.498977+0000 2024-04-02T13:38:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-02T13:38:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:crush_version 27 2024-04-02T13:38:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:full_ratio 0.95 2024-04-02T13:38:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:backfillfull_ratio 0.9 2024-04-02T13:38:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:nearfull_ratio 0.85 2024-04-02T13:38:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:require_min_compat_client luminous 2024-04-02T13:38:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:min_compat_client luminous 2024-04-02T13:38:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:require_osd_release squid 2024-04-02T13:38:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:stretch_mode_enabled false 2024-04-02T13:38:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-02T13:38:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-02T13:38:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-02T13:38:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-02T13:38:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:max_osd 12 2024-04-02T13:38:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6802/52311839,v1:172.21.15.33:6803/52311839] [v2:172.21.15.33:6804/52311839,v1:172.21.15.33:6805/52311839] exists,up 78695977-281e-41e3-9c9e-9a3c1fca4744 2024-04-02T13:38:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6800/1987709736,v1:172.21.15.50:6801/1987709736] [v2:172.21.15.50:6802/1987709736,v1:172.21.15.50:6803/1987709736] exists,up a6e22709-5642-4aae-a9b3-f7b27c4bed96 2024-04-02T13:38:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6800/1898887265,v1:172.21.15.145:6801/1898887265] [v2:172.21.15.145:6802/1898887265,v1:172.21.15.145:6803/1898887265] exists,up 5bdfdc32-0526-4f03-bd3f-50b251e80e53 2024-04-02T13:38:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6810/1162388141,v1:172.21.15.33:6811/1162388141] [v2:172.21.15.33:6812/1162388141,v1:172.21.15.33:6813/1162388141] exists,up 78b02725-b00b-4bdc-927f-b5d72ad92753 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 up in weight 1 up_from 34 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6808/367423754,v1:172.21.15.50:6809/367423754] [v2:172.21.15.50:6810/367423754,v1:172.21.15.50:6811/367423754] exists,up b02546aa-52cc-43d0-93c8-ba2ad32ceb0d 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6808/3244269192,v1:172.21.15.145:6809/3244269192] [v2:172.21.15.145:6810/3244269192,v1:172.21.15.145:6811/3244269192] exists,up 39e973fe-56fa-49d5-b155-17569a72a422 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 up in weight 1 up_from 47 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6818/529207637,v1:172.21.15.33:6819/529207637] [v2:172.21.15.33:6820/529207637,v1:172.21.15.33:6821/529207637] exists,up 523d9581-9067-4a63-8177-403ca4352df7 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 up in weight 1 up_from 52 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6816/2798485676,v1:172.21.15.50:6817/2798485676] [v2:172.21.15.50:6818/2798485676,v1:172.21.15.50:6819/2798485676] exists,up 53f96776-fc11-4b58-b6c7-dce5e873948e 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 up in weight 1 up_from 60 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6816/1364963848,v1:172.21.15.145:6817/1364963848] [v2:172.21.15.145:6818/1364963848,v1:172.21.15.145:6819/1364963848] exists,up 1957ffe4-4131-4157-b7b1-de97b339f24c 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 up in weight 1 up_from 68 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6826/1206256485,v1:172.21.15.33:6827/1206256485] [v2:172.21.15.33:6828/1206256485,v1:172.21.15.33:6829/1206256485] exists,up 32d3ed72-1341-4dd0-928e-e0e331751178 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 up in weight 1 up_from 73 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6824/1632195410,v1:172.21.15.50:6825/1632195410] [v2:172.21.15.50:6826/1632195410,v1:172.21.15.50:6827/1632195410] exists,up 0252727c-a17e-4542-b7c3-c409c8a6d0bd 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6824/141823123,v1:172.21.15.145:6825/141823123] [v2:172.21.15.145:6826/141823123,v1:172.21.15.145:6827/141823123] exists,up 7b33c049-e9a7-4f62-a514-4cc24a0c4267 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.0 [7,11] 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.a [2,11] 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.e [2,4] 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.1f [8,1] 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.1b [5,9] 2024-04-02T13:38:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.23 [4,8] 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.36 [4,9] 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.38 [3,9] 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.2 [9,1] 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.8 [5,8] 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.12 [9,10] 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.14 [9,8] 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/1117299953 expires 2024-04-03T13:29:15.778774+0000 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/2272183007 expires 2024-04-03T13:29:15.778774+0000 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/2666001000 expires 2024-04-03T13:29:15.778774+0000 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/2666001000 expires 2024-04-03T13:29:15.778774+0000 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/2869265288 expires 2024-04-03T13:28:48.425104+0000 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/3041616812 expires 2024-04-03T13:28:48.425104+0000 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/2869265288 expires 2024-04-03T13:28:48.425104+0000 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/3449207963 expires 2024-04-03T13:28:48.425104+0000 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/917705077 expires 2024-04-03T13:28:48.425104+0000 2024-04-02T13:38:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/1792579346 expires 2024-04-03T13:29:15.778774+0000 2024-04-02T13:38:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.436+0000 7f7d483c4700 1 -- 172.21.15.33:0/2344848340 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7d2c0a3cc0 msgr2=0x7f7d2c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.436+0000 7f7d483c4700 1 --2- 172.21.15.33:0/2344848340 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7d2c0a3cc0 0x7f7d2c0a6180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c003900 tx=0x7f7d3c005fb0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.436+0000 7f7d483c4700 1 -- 172.21.15.33:0/2344848340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d401041c0 msgr2=0x7f7d4019fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.436+0000 7f7d483c4700 1 --2- 172.21.15.33:0/2344848340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d401041c0 0x7f7d4019fd40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7d38007f10 tx=0x7f7d3800d3b0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.436+0000 7f7d483c4700 1 -- 172.21.15.33:0/2344848340 shutdown_connections 2024-04-02T13:38:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.436+0000 7f7d483c4700 1 --2- 172.21.15.33:0/2344848340 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7d2c0a3cc0 0x7f7d2c0a6180 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.436+0000 7f7d483c4700 1 --2- 172.21.15.33:0/2344848340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d401041c0 0x7f7d4019fd40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.436+0000 7f7d483c4700 1 --2- 172.21.15.33:0/2344848340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d40102620 0x7f7d40198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.436+0000 7f7d483c4700 1 --2- 172.21.15.33:0/2344848340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d40103820 0x7f7d40198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.436+0000 7f7d483c4700 1 -- 172.21.15.33:0/2344848340 >> 172.21.15.33:0/2344848340 conn(0x7f7d400fdc00 msgr2=0x7f7d40106b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.436+0000 7f7d483c4700 1 -- 172.21.15.33:0/2344848340 shutdown_connections 2024-04-02T13:38:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:33.437+0000 7f7d483c4700 1 -- 172.21.15.33:0/2344848340 wait complete. 2024-04-02T13:38:33.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:33 smithi033 ceph-mon[32390]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-02T13:38:33.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:33 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2344848340' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T13:38:33.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:33 smithi145 ceph-mon[35646]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-02T13:38:33.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:33 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2344848340' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T13:38:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:33 smithi050 ceph-mon[35524]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-02T13:38:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:33 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2344848340' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T13:38:34.196 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-02T13:38:34.208 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:38:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.316+0000 7f6afc911640 1 -- 172.21.15.33:0/2956310757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af81025d0 msgr2=0x7f6af8108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.316+0000 7f6afc911640 1 --2- 172.21.15.33:0/2956310757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af81025d0 0x7f6af8108e60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009ca0 tx=0x7f6af002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.316+0000 7f6afc911640 1 -- 172.21.15.33:0/2956310757 shutdown_connections 2024-04-02T13:38:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.316+0000 7f6afc911640 1 --2- 172.21.15.33:0/2956310757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af81025d0 0x7f6af8108e60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.316+0000 7f6afc911640 1 --2- 172.21.15.33:0/2956310757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af8101c10 0x7f6af8102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.316+0000 7f6afc911640 1 --2- 172.21.15.33:0/2956310757 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6af8100a20 0x7f6af8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.316+0000 7f6afc911640 1 -- 172.21.15.33:0/2956310757 >> 172.21.15.33:0/2956310757 conn(0x7f6af80fbf90 msgr2=0x7f6af80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.316+0000 7f6afc911640 1 -- 172.21.15.33:0/2956310757 shutdown_connections 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.316+0000 7f6afc911640 1 -- 172.21.15.33:0/2956310757 wait complete. 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.317+0000 7f6afc911640 1 Processor -- start 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.317+0000 7f6afc911640 1 -- start start 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.317+0000 7f6afc911640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6af8100a20 0x7f6af80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.317+0000 7f6afc911640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af8101c10 0x7f6af819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.317+0000 7f6afc911640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af81025d0 0x7f6af819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.317+0000 7f6afc911640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6af810c680 con 0x7f6af81025d0 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.317+0000 7f6afc911640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6af810c500 con 0x7f6af8101c10 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.317+0000 7f6afc911640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6af810c800 con 0x7f6af8100a20 2024-04-02T13:38:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.317+0000 7f6af6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af81025d0 0x7f6af819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.317+0000 7f6af6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af81025d0 0x7f6af819f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48304/0 (socket says 172.21.15.33:48304) 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.317+0000 7f6af6d9d640 1 -- 172.21.15.33:0/3163572139 learned_addr learned my addr 172.21.15.33:0/3163572139 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6aeffff640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af8101c10 0x7f6af819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6af659c640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6af8100a20 0x7f6af80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6af6d9d640 1 -- 172.21.15.33:0/3163572139 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6af8100a20 msgr2=0x7f6af80697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6af6d9d640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6af8100a20 0x7f6af80697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6af6d9d640 1 -- 172.21.15.33:0/3163572139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af8101c10 msgr2=0x7f6af819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6af6d9d640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af8101c10 0x7f6af819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6af6d9d640 1 -- 172.21.15.33:0/3163572139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af819fa50 con 0x7f6af81025d0 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6aeffff640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af8101c10 0x7f6af819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6af6d9d640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af81025d0 0x7f6af819f140 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6af002c630 tx=0x7f6af0032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6aef7fe640 1 -- 172.21.15.33:0/3163572139 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0039050 con 0x7f6af81025d0 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6aef7fe640 1 -- 172.21.15.33:0/3163572139 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6af003a040 con 0x7f6af81025d0 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6afc911640 1 -- 172.21.15.33:0/3163572139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af819fc80 con 0x7f6af81025d0 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6aef7fe640 1 -- 172.21.15.33:0/3163572139 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0004850 con 0x7f6af81025d0 2024-04-02T13:38:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.318+0000 7f6afc911640 1 -- 172.21.15.33:0/3163572139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af81a0170 con 0x7f6af81025d0 2024-04-02T13:38:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.319+0000 7f6afc911640 1 -- 172.21.15.33:0/3163572139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af8107d40 con 0x7f6af81025d0 2024-04-02T13:38:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.320+0000 7f6aef7fe640 1 -- 172.21.15.33:0/3163572139 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6af0004040 con 0x7f6af81025d0 2024-04-02T13:38:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.321+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6ae40a3d50 0x7f6ae40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.321+0000 7f6aef7fe640 1 -- 172.21.15.33:0/3163572139 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6af00f3340 con 0x7f6af81025d0 2024-04-02T13:38:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.321+0000 7f6af659c640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6ae40a3d50 0x7f6ae40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.321+0000 7f6af659c640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6ae40a3d50 0x7f6ae40a6210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6af8101a70 tx=0x7f6ae8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.325+0000 7f6aef7fe640 1 -- 172.21.15.33:0/3163572139 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6af0004330 con 0x7f6af81025d0 2024-04-02T13:38:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.512+0000 7f6afc911640 1 -- 172.21.15.33:0/3163572139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6af80627a0 con 0x7f6af81025d0 2024-04-02T13:38:34.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.513+0000 7f6aef7fe640 1 -- 172.21.15.33:0/3163572139 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24621 (secure 0 0 0) 0x7f6af00bf990 con 0x7f6af81025d0 2024-04-02T13:38:34.516 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:34.516 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:38:18.498977+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.515+0000 7f6afc911640 1 -- 172.21.15.33:0/3163572139 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6ae40a3d50 msgr2=0x7f6ae40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.515+0000 7f6afc911640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6ae40a3d50 0x7f6ae40a6210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6af8101a70 tx=0x7f6ae8005f90 comp rx=0 tx=0).stop 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.515+0000 7f6afc911640 1 -- 172.21.15.33:0/3163572139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af81025d0 msgr2=0x7f6af819f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.515+0000 7f6afc911640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af81025d0 0x7f6af819f140 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6af002c630 tx=0x7f6af0032980 comp rx=0 tx=0).stop 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.515+0000 7f6afc911640 1 -- 172.21.15.33:0/3163572139 shutdown_connections 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.515+0000 7f6afc911640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6ae40a3d50 0x7f6ae40a6210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.515+0000 7f6afc911640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af81025d0 0x7f6af819f140 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.515+0000 7f6afc911640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af8101c10 0x7f6af819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.515+0000 7f6afc911640 1 --2- 172.21.15.33:0/3163572139 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6af8100a20 0x7f6af80697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.515+0000 7f6afc911640 1 -- 172.21.15.33:0/3163572139 >> 172.21.15.33:0/3163572139 conn(0x7f6af80fbf90 msgr2=0x7f6af8104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.515+0000 7f6afc911640 1 -- 172.21.15.33:0/3163572139 shutdown_connections 2024-04-02T13:38:34.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.515+0000 7f6afc911640 1 -- 172.21.15.33:0/3163572139 wait complete. 2024-04-02T13:38:34.529 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:36:19.501121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:36:20.161569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:36:23.482134+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:38:34.530 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:38:34.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.635+0000 7f3f20893640 1 -- 172.21.15.33:0/4233086977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1c100a00 msgr2=0x7f3f1c100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.635+0000 7f3f20893640 1 --2- 172.21.15.33:0/4233086977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1c100a00 0x7f3f1c100e20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009900 tx=0x7f3f1402bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.635+0000 7f3f20893640 1 -- 172.21.15.33:0/4233086977 shutdown_connections 2024-04-02T13:38:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.635+0000 7f3f20893640 1 --2- 172.21.15.33:0/4233086977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f1c1025b0 0x7f3f1c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.635+0000 7f3f20893640 1 --2- 172.21.15.33:0/4233086977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f1c101bf0 0x7f3f1c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.635+0000 7f3f20893640 1 --2- 172.21.15.33:0/4233086977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1c100a00 0x7f3f1c100e20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.635+0000 7f3f20893640 1 -- 172.21.15.33:0/4233086977 >> 172.21.15.33:0/4233086977 conn(0x7f3f1c0fbfb0 msgr2=0x7f3f1c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.635+0000 7f3f20893640 1 -- 172.21.15.33:0/4233086977 shutdown_connections 2024-04-02T13:38:34.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.635+0000 7f3f20893640 1 -- 172.21.15.33:0/4233086977 wait complete. 2024-04-02T13:38:34.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.635+0000 7f3f20893640 1 Processor -- start 2024-04-02T13:38:34.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.635+0000 7f3f20893640 1 -- start start 2024-04-02T13:38:34.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f20893640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f1c100a00 0x7f3f1c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f20893640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1c101bf0 0x7f3f1c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f20893640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f1c1025b0 0x7f3f1c19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f20893640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c10d750 con 0x7f3f1c101bf0 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f20893640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c10d5d0 con 0x7f3f1c1025b0 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f20893640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c10d8d0 con 0x7f3f1c100a00 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f19d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1c101bf0 0x7f3f1c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f19d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1c101bf0 0x7f3f1c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48326/0 (socket says 172.21.15.33:48326) 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f19d9b640 1 -- 172.21.15.33:0/3868915949 learned_addr learned my addr 172.21.15.33:0/3868915949 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f1a59c640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f1c100a00 0x7f3f1c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f1ad9d640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f1c1025b0 0x7f3f1c19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f19d9b640 1 -- 172.21.15.33:0/3868915949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f1c100a00 msgr2=0x7f3f1c197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f19d9b640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f1c100a00 0x7f3f1c197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f19d9b640 1 -- 172.21.15.33:0/3868915949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f1c1025b0 msgr2=0x7f3f1c19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f19d9b640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f1c1025b0 0x7f3f1c19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f19d9b640 1 -- 172.21.15.33:0/3868915949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1c19fa80 con 0x7f3f1c101bf0 2024-04-02T13:38:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f1ad9d640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f1c1025b0 0x7f3f1c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f19d9b640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1c101bf0 0x7f3f1c1983a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400d950 tx=0x7f3f0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f137fe640 1 -- 172.21.15.33:0/3868915949 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04017070 con 0x7f3f1c101bf0 2024-04-02T13:38:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f137fe640 1 -- 172.21.15.33:0/3868915949 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3f04009c30 con 0x7f3f1c101bf0 2024-04-02T13:38:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f20893640 1 -- 172.21.15.33:0/3868915949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1c19fd70 con 0x7f3f1c101bf0 2024-04-02T13:38:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f137fe640 1 -- 172.21.15.33:0/3868915949 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0400f6b0 con 0x7f3f1c101bf0 2024-04-02T13:38:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.636+0000 7f3f20893640 1 -- 172.21.15.33:0/3868915949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f1c1a0280 con 0x7f3f1c101bf0 2024-04-02T13:38:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.638+0000 7f3f20893640 1 -- 172.21.15.33:0/3868915949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f1c107e10 con 0x7f3f1c101bf0 2024-04-02T13:38:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.638+0000 7f3f137fe640 1 -- 172.21.15.33:0/3868915949 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3f04010a50 con 0x7f3f1c101bf0 2024-04-02T13:38:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.639+0000 7f3f137fe640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3f080a3c30 0x7f3f080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.639+0000 7f3f1a59c640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3f080a3c30 0x7f3f080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.639+0000 7f3f137fe640 1 -- 172.21.15.33:0/3868915949 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3f040d02e0 con 0x7f3f1c101bf0 2024-04-02T13:38:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.640+0000 7f3f1a59c640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3f080a3c30 0x7f3f080a60f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3f14000c00 tx=0x7f3f14005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.644+0000 7f3f137fe640 1 -- 172.21.15.33:0/3868915949 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3f04009de0 con 0x7f3f1c101bf0 2024-04-02T13:38:34.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.829+0000 7f3f20893640 1 -- 172.21.15.33:0/3868915949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f1c02d370 con 0x7f3f1c101bf0 2024-04-02T13:38:34.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.829+0000 7f3f137fe640 1 -- 172.21.15.33:0/3868915949 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f3f0409c930 con 0x7f3f1c101bf0 2024-04-02T13:38:34.832 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:38:34.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.831+0000 7f3f20893640 1 -- 172.21.15.33:0/3868915949 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3f080a3c30 msgr2=0x7f3f080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.831+0000 7f3f20893640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3f080a3c30 0x7f3f080a60f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3f14000c00 tx=0x7f3f14005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.831+0000 7f3f20893640 1 -- 172.21.15.33:0/3868915949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1c101bf0 msgr2=0x7f3f1c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.831+0000 7f3f20893640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1c101bf0 0x7f3f1c1983a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400d950 tx=0x7f3f0400de10 comp rx=0 tx=0).stop 2024-04-02T13:38:34.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.831+0000 7f3f20893640 1 -- 172.21.15.33:0/3868915949 shutdown_connections 2024-04-02T13:38:34.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.831+0000 7f3f20893640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f3f080a3c30 0x7f3f080a60f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.831+0000 7f3f20893640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f1c1025b0 0x7f3f1c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.831+0000 7f3f20893640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1c101bf0 0x7f3f1c1983a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.831+0000 7f3f20893640 1 --2- 172.21.15.33:0/3868915949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3f1c100a00 0x7f3f1c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.831+0000 7f3f20893640 1 -- 172.21.15.33:0/3868915949 >> 172.21.15.33:0/3868915949 conn(0x7f3f1c0fbfb0 msgr2=0x7f3f1c104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.831+0000 7f3f20893640 1 -- 172.21.15.33:0/3868915949 shutdown_connections 2024-04-02T13:38:34.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.831+0000 7f3f20893640 1 -- 172.21.15.33:0/3868915949 wait complete. 2024-04-02T13:38:34.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:34 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3163572139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:34.845 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:38:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.949+0000 7fdb9ec47640 1 -- 172.21.15.33:0/633689023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb98101c00 msgr2=0x7fdb98102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.949+0000 7fdb9ec47640 1 --2- 172.21.15.33:0/633689023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb98101c00 0x7fdb98102080 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009900 tx=0x7fdb8402bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.949+0000 7fdb9ec47640 1 -- 172.21.15.33:0/633689023 shutdown_connections 2024-04-02T13:38:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.949+0000 7fdb9ec47640 1 --2- 172.21.15.33:0/633689023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb981025c0 0x7fdb98108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.949+0000 7fdb9ec47640 1 --2- 172.21.15.33:0/633689023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb98101c00 0x7fdb98102080 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.949+0000 7fdb9ec47640 1 --2- 172.21.15.33:0/633689023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb98100a10 0x7fdb98100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.949+0000 7fdb9ec47640 1 -- 172.21.15.33:0/633689023 >> 172.21.15.33:0/633689023 conn(0x7fdb980fbfa0 msgr2=0x7fdb980fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.949+0000 7fdb9ec47640 1 -- 172.21.15.33:0/633689023 shutdown_connections 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.949+0000 7fdb9ec47640 1 -- 172.21.15.33:0/633689023 wait complete. 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.949+0000 7fdb9ec47640 1 Processor -- start 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.949+0000 7fdb9ec47640 1 -- start start 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb9ec47640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb98100a10 0x7fdb98197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb9ec47640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb98101c00 0x7fdb981983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb9ec47640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb981025c0 0x7fdb9819f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb9ec47640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9810c670 con 0x7fdb98101c00 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb9ec47640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9810c4f0 con 0x7fdb98100a10 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb9ec47640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9810c7f0 con 0x7fdb981025c0 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb97fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb98101c00 0x7fdb981983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb97fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb98101c00 0x7fdb981983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48344/0 (socket says 172.21.15.33:48344) 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb97fff640 1 -- 172.21.15.33:0/3376921548 learned_addr learned my addr 172.21.15.33:0/3376921548 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb97fff640 1 -- 172.21.15.33:0/3376921548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb981025c0 msgr2=0x7fdb9819f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb9d1e4640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb981025c0 0x7fdb9819f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb97fff640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb981025c0 0x7fdb9819f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb97fff640 1 -- 172.21.15.33:0/3376921548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb98100a10 msgr2=0x7fdb98197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb9c9e3640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb98100a10 0x7fdb98197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb97fff640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb98100a10 0x7fdb98197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb97fff640 1 -- 172.21.15.33:0/3376921548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9819fa80 con 0x7fdb98101c00 2024-04-02T13:38:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb9c9e3640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb98100a10 0x7fdb98197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb97fff640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb98101c00 0x7fdb981983a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009230 tx=0x7fdb84004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb95ffb640 1 -- 172.21.15.33:0/3376921548 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84043020 con 0x7fdb98101c00 2024-04-02T13:38:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb9ec47640 1 -- 172.21.15.33:0/3376921548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9819fd10 con 0x7fdb98101c00 2024-04-02T13:38:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb95ffb640 1 -- 172.21.15.33:0/3376921548 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdb84004350 con 0x7fdb98101c00 2024-04-02T13:38:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.950+0000 7fdb95ffb640 1 -- 172.21.15.33:0/3376921548 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84003ca0 con 0x7fdb98101c00 2024-04-02T13:38:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.951+0000 7fdb9ec47640 1 -- 172.21.15.33:0/3376921548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb981a0200 con 0x7fdb98101c00 2024-04-02T13:38:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.952+0000 7fdb9ec47640 1 -- 172.21.15.33:0/3376921548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb981a0430 con 0x7fdb98101c00 2024-04-02T13:38:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.953+0000 7fdb95ffb640 1 -- 172.21.15.33:0/3376921548 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdb84046020 con 0x7fdb98101c00 2024-04-02T13:38:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.954+0000 7fdb95ffb640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fdb880a3bb0 0x7fdb880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.954+0000 7fdb9c9e3640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fdb880a3bb0 0x7fdb880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.954+0000 7fdb95ffb640 1 -- 172.21.15.33:0/3376921548 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdb840f2cf0 con 0x7fdb98101c00 2024-04-02T13:38:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.954+0000 7fdb9c9e3640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fdb880a3bb0 0x7fdb880a6070 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb98101a60 tx=0x7fdb8c005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:34.957+0000 7fdb95ffb640 1 -- 172.21.15.33:0/3376921548 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdb8404d070 con 0x7fdb98101c00 2024-04-02T13:38:34.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:34 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3163572139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:34 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3163572139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.143+0000 7fdb9ec47640 1 -- 172.21.15.33:0/3376921548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdb981a0600 con 0x7fdb98101c00 2024-04-02T13:38:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.144+0000 7fdb95ffb640 1 -- 172.21.15.33:0/3376921548 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fdb840bf340 con 0x7fdb98101c00 2024-04-02T13:38:35.147 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.146+0000 7fdb9ec47640 1 -- 172.21.15.33:0/3376921548 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fdb880a3bb0 msgr2=0x7fdb880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.146+0000 7fdb9ec47640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fdb880a3bb0 0x7fdb880a6070 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb98101a60 tx=0x7fdb8c005e70 comp rx=0 tx=0).stop 2024-04-02T13:38:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.146+0000 7fdb9ec47640 1 -- 172.21.15.33:0/3376921548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb98101c00 msgr2=0x7fdb981983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.146+0000 7fdb9ec47640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb98101c00 0x7fdb981983a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009230 tx=0x7fdb84004ea0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.146+0000 7fdb9ec47640 1 -- 172.21.15.33:0/3376921548 shutdown_connections 2024-04-02T13:38:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.146+0000 7fdb9ec47640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fdb880a3bb0 0x7fdb880a6070 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.146+0000 7fdb9ec47640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdb981025c0 0x7fdb9819f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.146+0000 7fdb9ec47640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb98101c00 0x7fdb981983a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.146+0000 7fdb9ec47640 1 --2- 172.21.15.33:0/3376921548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb98100a10 0x7fdb98197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.146+0000 7fdb9ec47640 1 -- 172.21.15.33:0/3376921548 >> 172.21.15.33:0/3376921548 conn(0x7fdb980fbfa0 msgr2=0x7fdb980fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.146+0000 7fdb9ec47640 1 -- 172.21.15.33:0/3376921548 shutdown_connections 2024-04-02T13:38:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.146+0000 7fdb9ec47640 1 -- 172.21.15.33:0/3376921548 wait complete. 2024-04-02T13:38:35.160 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:38:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.263+0000 7f0fda2b1640 1 -- 172.21.15.33:0/652291347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4101c00 msgr2=0x7f0fd4102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.263+0000 7f0fda2b1640 1 --2- 172.21.15.33:0/652291347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4101c00 0x7f0fd4102080 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009900 tx=0x7f0fc402bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.263+0000 7f0fda2b1640 1 -- 172.21.15.33:0/652291347 shutdown_connections 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.263+0000 7f0fda2b1640 1 --2- 172.21.15.33:0/652291347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0fd41025c0 0x7f0fd4108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.263+0000 7f0fda2b1640 1 --2- 172.21.15.33:0/652291347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4101c00 0x7f0fd4102080 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.263+0000 7f0fda2b1640 1 --2- 172.21.15.33:0/652291347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fd4100a10 0x7f0fd4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.263+0000 7f0fda2b1640 1 -- 172.21.15.33:0/652291347 >> 172.21.15.33:0/652291347 conn(0x7f0fd40fbfa0 msgr2=0x7f0fd40fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.263+0000 7f0fda2b1640 1 -- 172.21.15.33:0/652291347 shutdown_connections 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.263+0000 7f0fda2b1640 1 -- 172.21.15.33:0/652291347 wait complete. 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.263+0000 7f0fda2b1640 1 Processor -- start 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.263+0000 7f0fda2b1640 1 -- start start 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fda2b1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fd4100a10 0x7f0fd4197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fda2b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4101c00 0x7f0fd4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fda2b1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0fd41025c0 0x7f0fd419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fda2b1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd410c670 con 0x7f0fd4101c00 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fda2b1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd410c4f0 con 0x7f0fd4100a10 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fda2b1640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd410c7f0 con 0x7f0fd41025c0 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4101c00 0x7f0fd4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4101c00 0x7f0fd4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48360/0 (socket says 172.21.15.33:48360) 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd37fe640 1 -- 172.21.15.33:0/2321188003 learned_addr learned my addr 172.21.15.33:0/2321188003 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd3fff640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fd4100a10 0x7f0fd4197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd884e640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0fd41025c0 0x7f0fd419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd37fe640 1 -- 172.21.15.33:0/2321188003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0fd41025c0 msgr2=0x7f0fd419f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd37fe640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0fd41025c0 0x7f0fd419f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd37fe640 1 -- 172.21.15.33:0/2321188003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fd4100a10 msgr2=0x7f0fd4197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd37fe640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fd4100a10 0x7f0fd4197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd37fe640 1 -- 172.21.15.33:0/2321188003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd419f9e0 con 0x7f0fd4101c00 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd3fff640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fd4100a10 0x7f0fd4197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd37fe640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4101c00 0x7f0fd4198300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009230 tx=0x7f0fc4004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd17fa640 1 -- 172.21.15.33:0/2321188003 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4043020 con 0x7f0fd4101c00 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd17fa640 1 -- 172.21.15.33:0/2321188003 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0fc4004350 con 0x7f0fd4101c00 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fda2b1640 1 -- 172.21.15.33:0/2321188003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd419fc70 con 0x7f0fd4101c00 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fd17fa640 1 -- 172.21.15.33:0/2321188003 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4003ca0 con 0x7f0fd4101c00 2024-04-02T13:38:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.264+0000 7f0fda2b1640 1 -- 172.21.15.33:0/2321188003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd41a01c0 con 0x7f0fd4101c00 2024-04-02T13:38:35.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.265+0000 7f0fda2b1640 1 -- 172.21.15.33:0/2321188003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd41a0450 con 0x7f0fd4101c00 2024-04-02T13:38:35.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.266+0000 7f0fd17fa640 1 -- 172.21.15.33:0/2321188003 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0fc4046020 con 0x7f0fd4101c00 2024-04-02T13:38:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.267+0000 7f0fd17fa640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0fc00a3bb0 0x7f0fc00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.267+0000 7f0fd3fff640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0fc00a3bb0 0x7f0fc00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.267+0000 7f0fd17fa640 1 -- 172.21.15.33:0/2321188003 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0fc40f2d00 con 0x7f0fd4101c00 2024-04-02T13:38:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.267+0000 7f0fd3fff640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0fc00a3bb0 0x7f0fc00a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc005d00 tx=0x7f0fbc005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:35.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.271+0000 7f0fd17fa640 1 -- 172.21.15.33:0/2321188003 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0fc404d070 con 0x7f0fd4101c00 2024-04-02T13:38:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.479+0000 7f0fda2b1640 1 -- 172.21.15.33:0/2321188003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0fd41a0620 con 0x7f0fd4101c00 2024-04-02T13:38:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.479+0000 7f0fd17fa640 1 -- 172.21.15.33:0/2321188003 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f0fc40bf350 con 0x7f0fd4101c00 2024-04-02T13:38:35.482 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-02T13:38:35.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.482+0000 7f0fda2b1640 1 -- 172.21.15.33:0/2321188003 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0fc00a3bb0 msgr2=0x7f0fc00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.482+0000 7f0fda2b1640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0fc00a3bb0 0x7f0fc00a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc005d00 tx=0x7f0fbc005c90 comp rx=0 tx=0).stop 2024-04-02T13:38:35.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.482+0000 7f0fda2b1640 1 -- 172.21.15.33:0/2321188003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4101c00 msgr2=0x7f0fd4198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.482+0000 7f0fda2b1640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4101c00 0x7f0fd4198300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009230 tx=0x7f0fc4004ea0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.482+0000 7f0fda2b1640 1 -- 172.21.15.33:0/2321188003 shutdown_connections 2024-04-02T13:38:35.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.482+0000 7f0fda2b1640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0fc00a3bb0 0x7f0fc00a6070 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.482+0000 7f0fda2b1640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0fd41025c0 0x7f0fd419f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.482+0000 7f0fda2b1640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4101c00 0x7f0fd4198300 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.482+0000 7f0fda2b1640 1 --2- 172.21.15.33:0/2321188003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fd4100a10 0x7f0fd4197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.482+0000 7f0fda2b1640 1 -- 172.21.15.33:0/2321188003 >> 172.21.15.33:0/2321188003 conn(0x7f0fd40fbfa0 msgr2=0x7f0fd406a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:35.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.482+0000 7f0fda2b1640 1 -- 172.21.15.33:0/2321188003 shutdown_connections 2024-04-02T13:38:35.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.482+0000 7f0fda2b1640 1 -- 172.21.15.33:0/2321188003 wait complete. 2024-04-02T13:38:35.496 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:38:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.599+0000 7f2b03752640 1 -- 172.21.15.33:0/937816760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc101bf0 msgr2=0x7f2afc102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.599+0000 7f2b03752640 1 --2- 172.21.15.33:0/937816760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc101bf0 0x7f2afc102070 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009900 tx=0x7f2af002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.599+0000 7f2b03752640 1 -- 172.21.15.33:0/937816760 shutdown_connections 2024-04-02T13:38:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.599+0000 7f2b03752640 1 --2- 172.21.15.33:0/937816760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2afc1025b0 0x7f2afc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.599+0000 7f2b03752640 1 --2- 172.21.15.33:0/937816760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc101bf0 0x7f2afc102070 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.599+0000 7f2b03752640 1 --2- 172.21.15.33:0/937816760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2afc100a00 0x7f2afc100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.599+0000 7f2b03752640 1 -- 172.21.15.33:0/937816760 >> 172.21.15.33:0/937816760 conn(0x7f2afc0fbf70 msgr2=0x7f2afc0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b03752640 1 -- 172.21.15.33:0/937816760 shutdown_connections 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b03752640 1 -- 172.21.15.33:0/937816760 wait complete. 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b03752640 1 Processor -- start 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b03752640 1 -- start start 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b03752640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2afc100a00 0x7f2afc104b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b03752640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2afc101bf0 0x7f2afc1031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b03752640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc1025b0 0x7f2afc103700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b03752640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc10c6e0 con 0x7f2afc1025b0 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b03752640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc10c560 con 0x7f2afc101bf0 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b03752640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc10c860 con 0x7f2afc100a00 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b01cef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc1025b0 0x7f2afc103700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b01cef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc1025b0 0x7f2afc103700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48384/0 (socket says 172.21.15.33:48384) 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b01cef640 1 -- 172.21.15.33:0/2838276619 learned_addr learned my addr 172.21.15.33:0/2838276619 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b00ced640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2afc101bf0 0x7f2afc1031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.600+0000 7f2b014ee640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2afc100a00 0x7f2afc104b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.601+0000 7f2b01cef640 1 -- 172.21.15.33:0/2838276619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2afc100a00 msgr2=0x7f2afc104b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.601+0000 7f2b01cef640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2afc100a00 0x7f2afc104b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.601+0000 7f2b01cef640 1 -- 172.21.15.33:0/2838276619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2afc101bf0 msgr2=0x7f2afc1031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.601+0000 7f2b01cef640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2afc101bf0 0x7f2afc1031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.601+0000 7f2b01cef640 1 -- 172.21.15.33:0/2838276619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc103f90 con 0x7f2afc1025b0 2024-04-02T13:38:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.601+0000 7f2b01cef640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc1025b0 0x7f2afc103700 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2af400cad0 tx=0x7f2af400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.601+0000 7f2aee7fc640 1 -- 172.21.15.33:0/2838276619 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af400ad60 con 0x7f2afc1025b0 2024-04-02T13:38:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.601+0000 7f2b03752640 1 -- 172.21.15.33:0/2838276619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc1a1ff0 con 0x7f2afc1025b0 2024-04-02T13:38:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.601+0000 7f2aee7fc640 1 -- 172.21.15.33:0/2838276619 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2af4005020 con 0x7f2afc1025b0 2024-04-02T13:38:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.601+0000 7f2aee7fc640 1 -- 172.21.15.33:0/2838276619 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af400f6b0 con 0x7f2afc1025b0 2024-04-02T13:38:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.601+0000 7f2b03752640 1 -- 172.21.15.33:0/2838276619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc1a24d0 con 0x7f2afc1025b0 2024-04-02T13:38:35.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.602+0000 7f2b03752640 1 -- 172.21.15.33:0/2838276619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2afc1a27c0 con 0x7f2afc1025b0 2024-04-02T13:38:35.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.603+0000 7f2aee7fc640 1 -- 172.21.15.33:0/2838276619 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2af40051d0 con 0x7f2afc1025b0 2024-04-02T13:38:35.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.604+0000 7f2aee7fc640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2ae80a3c30 0x7f2ae80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:35.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.604+0000 7f2b014ee640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2ae80a3c30 0x7f2ae80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:35.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.604+0000 7f2aee7fc640 1 -- 172.21.15.33:0/2838276619 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2af40d0a10 con 0x7f2afc1025b0 2024-04-02T13:38:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.604+0000 7f2b014ee640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2ae80a3c30 0x7f2ae80a60f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2afc101a50 tx=0x7f2af8009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.607+0000 7f2aee7fc640 1 -- 172.21.15.33:0/2838276619 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2af4014030 con 0x7f2afc1025b0 2024-04-02T13:38:35.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:35 smithi033 ceph-mon[32390]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:38:35.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:35 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3868915949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:35.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:35 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3376921548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:35.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:35 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2321188003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:35.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.794+0000 7f2b03752640 1 -- 172.21.15.33:0/2838276619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2afc1a2b50 con 0x7f2afc1025b0 2024-04-02T13:38:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.794+0000 7f2aee7fc640 1 -- 172.21.15.33:0/2838276619 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f2af409d0e0 con 0x7f2afc1025b0 2024-04-02T13:38:35.797 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.796+0000 7f2b03752640 1 -- 172.21.15.33:0/2838276619 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2ae80a3c30 msgr2=0x7f2ae80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.796+0000 7f2b03752640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2ae80a3c30 0x7f2ae80a60f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2afc101a50 tx=0x7f2af8009110 comp rx=0 tx=0).stop 2024-04-02T13:38:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.796+0000 7f2b03752640 1 -- 172.21.15.33:0/2838276619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc1025b0 msgr2=0x7f2afc103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.796+0000 7f2b03752640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc1025b0 0x7f2afc103700 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2af400cad0 tx=0x7f2af400ced0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.796+0000 7f2b03752640 1 -- 172.21.15.33:0/2838276619 shutdown_connections 2024-04-02T13:38:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.796+0000 7f2b03752640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f2ae80a3c30 0x7f2ae80a60f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.796+0000 7f2b03752640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc1025b0 0x7f2afc103700 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.796+0000 7f2b03752640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2afc101bf0 0x7f2afc1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.796+0000 7f2b03752640 1 --2- 172.21.15.33:0/2838276619 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2afc100a00 0x7f2afc104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.796+0000 7f2b03752640 1 -- 172.21.15.33:0/2838276619 >> 172.21.15.33:0/2838276619 conn(0x7f2afc0fbf70 msgr2=0x7f2afc06a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.796+0000 7f2b03752640 1 -- 172.21.15.33:0/2838276619 shutdown_connections 2024-04-02T13:38:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.796+0000 7f2b03752640 1 -- 172.21.15.33:0/2838276619 wait complete. 2024-04-02T13:38:35.810 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:38:35.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.910+0000 7f83675e9640 1 -- 172.21.15.33:0/1948692672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100a40 msgr2=0x7f8360100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.910+0000 7f83675e9640 1 --2- 172.21.15.33:0/1948692672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100a40 0x7f8360100e60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f835c009ca0 tx=0x7f835c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.910+0000 7f83675e9640 1 -- 172.21.15.33:0/1948692672 shutdown_connections 2024-04-02T13:38:35.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.910+0000 7f83675e9640 1 --2- 172.21.15.33:0/1948692672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83601025f0 0x7f8360108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.910+0000 7f83675e9640 1 --2- 172.21.15.33:0/1948692672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8360101c30 0x7f83601020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.910+0000 7f83675e9640 1 --2- 172.21.15.33:0/1948692672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100a40 0x7f8360100e60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.910+0000 7f83675e9640 1 -- 172.21.15.33:0/1948692672 >> 172.21.15.33:0/1948692672 conn(0x7f83600fbfb0 msgr2=0x7f83600fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:35.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.910+0000 7f83675e9640 1 -- 172.21.15.33:0/1948692672 shutdown_connections 2024-04-02T13:38:35.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.910+0000 7f83675e9640 1 -- 172.21.15.33:0/1948692672 wait complete. 2024-04-02T13:38:35.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f83675e9640 1 Processor -- start 2024-04-02T13:38:35.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f83675e9640 1 -- start start 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f83675e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100a40 0x7f83601049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f83675e9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8360101c30 0x7f8360103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f83675e9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83601025f0 0x7f8360103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f83675e9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f836010c700 con 0x7f8360100a40 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f83675e9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f836010c580 con 0x7f8360101c30 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f83675e9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f836010c880 con 0x7f83601025f0 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f8365385640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100a40 0x7f83601049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f8365385640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100a40 0x7f83601049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48400/0 (socket says 172.21.15.33:48400) 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f8365385640 1 -- 172.21.15.33:0/3714302722 learned_addr learned my addr 172.21.15.33:0/3714302722 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f8364b84640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8360101c30 0x7f8360103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f8365385640 1 -- 172.21.15.33:0/3714302722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83601025f0 msgr2=0x7f8360103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f8365385640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83601025f0 0x7f8360103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f8365385640 1 -- 172.21.15.33:0/3714302722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8360101c30 msgr2=0x7f8360103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f8365385640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8360101c30 0x7f8360103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:35.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f8365385640 1 -- 172.21.15.33:0/3714302722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8360103de0 con 0x7f8360100a40 2024-04-02T13:38:35.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f8364b84640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8360101c30 0x7f8360103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:35.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.911+0000 7f8365385640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100a40 0x7f83601049d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f835c02c630 tx=0x7f835c032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:35.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.912+0000 7f83567fc640 1 -- 172.21.15.33:0/3714302722 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c039050 con 0x7f8360100a40 2024-04-02T13:38:35.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.912+0000 7f83567fc640 1 -- 172.21.15.33:0/3714302722 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f835c045070 con 0x7f8360100a40 2024-04-02T13:38:35.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.912+0000 7f83675e9640 1 -- 172.21.15.33:0/3714302722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83601a1f50 con 0x7f8360100a40 2024-04-02T13:38:35.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.912+0000 7f83567fc640 1 -- 172.21.15.33:0/3714302722 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c040450 con 0x7f8360100a40 2024-04-02T13:38:35.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.912+0000 7f83675e9640 1 -- 172.21.15.33:0/3714302722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83601a2440 con 0x7f8360100a40 2024-04-02T13:38:35.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.913+0000 7f83675e9640 1 -- 172.21.15.33:0/3714302722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83601a26a0 con 0x7f8360100a40 2024-04-02T13:38:35.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.914+0000 7f83567fc640 1 -- 172.21.15.33:0/3714302722 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f835c0040d0 con 0x7f8360100a40 2024-04-02T13:38:35.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.914+0000 7f83567fc640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f834c0a3c30 0x7f834c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:35.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.914+0000 7f83567fc640 1 -- 172.21.15.33:0/3714302722 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f835c0f41f0 con 0x7f8360100a40 2024-04-02T13:38:35.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.914+0000 7f8364b84640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f834c0a3c30 0x7f834c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:35.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.915+0000 7f8364b84640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f834c0a3c30 0x7f834c0a60f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8360104740 tx=0x7f835000a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:35.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:35.918+0000 7f83567fc640 1 -- 172.21.15.33:0/3714302722 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f835c0c0900 con 0x7f8360100a40 2024-04-02T13:38:35.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:35 smithi145 ceph-mon[35646]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:38:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:35 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3868915949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:35 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3376921548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:35.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:35 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2321188003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:35 smithi050 ceph-mon[35524]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:38:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:35 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3868915949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:35 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3376921548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:35 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2321188003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.148+0000 7f83675e9640 1 -- 172.21.15.33:0/3714302722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83601a2a30 con 0x7f8360100a40 2024-04-02T13:38:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.149+0000 7f83567fc640 1 -- 172.21.15.33:0/3714302722 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11873 (secure 0 0 0) 0x7f835c0c0200 con 0x7f8360100a40 2024-04-02T13:38:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:36.152 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:36:47.489590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:38:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-02T13:38:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.151+0000 7f83675e9640 1 -- 172.21.15.33:0/3714302722 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f834c0a3c30 msgr2=0x7f834c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.151+0000 7f83675e9640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f834c0a3c30 0x7f834c0a60f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8360104740 tx=0x7f835000a5c0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.151+0000 7f83675e9640 1 -- 172.21.15.33:0/3714302722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100a40 msgr2=0x7f83601049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.151+0000 7f83675e9640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100a40 0x7f83601049d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f835c02c630 tx=0x7f835c032980 comp rx=0 tx=0).stop 2024-04-02T13:38:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.151+0000 7f83675e9640 1 -- 172.21.15.33:0/3714302722 shutdown_connections 2024-04-02T13:38:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.151+0000 7f83675e9640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f834c0a3c30 0x7f834c0a60f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.151+0000 7f83675e9640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83601025f0 0x7f8360103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.151+0000 7f83675e9640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8360101c30 0x7f8360103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.151+0000 7f83675e9640 1 --2- 172.21.15.33:0/3714302722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100a40 0x7f83601049d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.151+0000 7f83675e9640 1 -- 172.21.15.33:0/3714302722 >> 172.21.15.33:0/3714302722 conn(0x7f83600fbfb0 msgr2=0x7f83600fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.151+0000 7f83675e9640 1 -- 172.21.15.33:0/3714302722 shutdown_connections 2024-04-02T13:38:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.151+0000 7f83675e9640 1 -- 172.21.15.33:0/3714302722 wait complete. 2024-04-02T13:38:36.165 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:38:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.266+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1367525458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8f41025d0 msgr2=0x7fc8f4108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.266+0000 7fc8fbf0e640 1 --2- 172.21.15.33:0/1367525458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8f41025d0 0x7fc8f4108e60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009ca0 tx=0x7fc8ec02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.266+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1367525458 shutdown_connections 2024-04-02T13:38:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.266+0000 7fc8fbf0e640 1 --2- 172.21.15.33:0/1367525458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8f41025d0 0x7fc8f4108e60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.266+0000 7fc8fbf0e640 1 --2- 172.21.15.33:0/1367525458 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8f4101c10 0x7fc8f4102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.266+0000 7fc8fbf0e640 1 --2- 172.21.15.33:0/1367525458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8f4100a20 0x7fc8f4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.266+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1367525458 >> 172.21.15.33:0/1367525458 conn(0x7fc8f40fbf90 msgr2=0x7fc8f40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.266+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1367525458 shutdown_connections 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.266+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1367525458 wait complete. 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fbf0e640 1 Processor -- start 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fbf0e640 1 -- start start 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fbf0e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8f4100a20 0x7fc8f40697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fbf0e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8f4101c10 0x7fc8f419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fbf0e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8f41025d0 0x7fc8f419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fbf0e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f410c680 con 0x7fc8f41025d0 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fbf0e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f410c500 con 0x7fc8f4100a20 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fbf0e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f410c800 con 0x7fc8f4101c10 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fa4ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8f41025d0 0x7fc8f419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fa4ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8f41025d0 0x7fc8f419f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48414/0 (socket says 172.21.15.33:48414) 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fa4ab640 1 -- 172.21.15.33:0/1698208027 learned_addr learned my addr 172.21.15.33:0/1698208027 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8f94a9640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8f4101c10 0x7fc8f419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8f9caa640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8f4100a20 0x7fc8f40697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fa4ab640 1 -- 172.21.15.33:0/1698208027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8f4101c10 msgr2=0x7fc8f419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fa4ab640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8f4101c10 0x7fc8f419cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fa4ab640 1 -- 172.21.15.33:0/1698208027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8f4100a20 msgr2=0x7fc8f40697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fa4ab640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8f4100a20 0x7fc8f40697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fa4ab640 1 -- 172.21.15.33:0/1698208027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f419fa20 con 0x7fc8f41025d0 2024-04-02T13:38:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8f9caa640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8f4100a20 0x7fc8f40697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8fa4ab640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8f41025d0 0x7fc8f419f160 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec02c630 tx=0x7fc8ec032900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.267+0000 7fc8eaffd640 1 -- 172.21.15.33:0/1698208027 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec039050 con 0x7fc8f41025d0 2024-04-02T13:38:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.268+0000 7fc8eaffd640 1 -- 172.21.15.33:0/1698208027 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc8ec03a040 con 0x7fc8f41025d0 2024-04-02T13:38:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.268+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1698208027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f419fcb0 con 0x7fc8f41025d0 2024-04-02T13:38:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.268+0000 7fc8eaffd640 1 -- 172.21.15.33:0/1698208027 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec040450 con 0x7fc8f41025d0 2024-04-02T13:38:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.268+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1698208027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f41a01a0 con 0x7fc8f41025d0 2024-04-02T13:38:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.268+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1698208027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f4107d40 con 0x7fc8f41025d0 2024-04-02T13:38:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.270+0000 7fc8eaffd640 1 -- 172.21.15.33:0/1698208027 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc8ec003c00 con 0x7fc8f41025d0 2024-04-02T13:38:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.270+0000 7fc8eaffd640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc8e00a3bb0 0x7fc8e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.270+0000 7fc8f9caa640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc8e00a3bb0 0x7fc8e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.271+0000 7fc8eaffd640 1 -- 172.21.15.33:0/1698208027 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc8ec0f3e80 con 0x7fc8f41025d0 2024-04-02T13:38:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.271+0000 7fc8f9caa640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc8e00a3bb0 0x7fc8e00a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0005d00 tx=0x7fc8f0005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.274+0000 7fc8eaffd640 1 -- 172.21.15.33:0/1698208027 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc8ec0c04d0 con 0x7fc8f41025d0 2024-04-02T13:38:36.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.460+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1698208027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8f402d570 con 0x7fc8f41025d0 2024-04-02T13:38:36.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.462+0000 7fc8eaffd640 1 -- 172.21.15.33:0/1698208027 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24621 (secure 0 0 0) 0x7fc8ec0bfdd0 con 0x7fc8f41025d0 2024-04-02T13:38:36.466 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:38:18.498977+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:38:36.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.464+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1698208027 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc8e00a3bb0 msgr2=0x7fc8e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.464+0000 7fc8fbf0e640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc8e00a3bb0 0x7fc8e00a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0005d00 tx=0x7fc8f0005ef0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.464+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1698208027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8f41025d0 msgr2=0x7fc8f419f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.464+0000 7fc8fbf0e640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8f41025d0 0x7fc8f419f160 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec02c630 tx=0x7fc8ec032900 comp rx=0 tx=0).stop 2024-04-02T13:38:36.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.464+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1698208027 shutdown_connections 2024-04-02T13:38:36.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.464+0000 7fc8fbf0e640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc8e00a3bb0 0x7fc8e00a6070 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.464+0000 7fc8fbf0e640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8f41025d0 0x7fc8f419f160 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.464+0000 7fc8fbf0e640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8f4101c10 0x7fc8f419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.464+0000 7fc8fbf0e640 1 --2- 172.21.15.33:0/1698208027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8f4100a20 0x7fc8f40697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.464+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1698208027 >> 172.21.15.33:0/1698208027 conn(0x7fc8f40fbf90 msgr2=0x7fc8f4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:36.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.464+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1698208027 shutdown_connections 2024-04-02T13:38:36.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.464+0000 7fc8fbf0e640 1 -- 172.21.15.33:0/1698208027 wait complete. 2024-04-02T13:38:36.479 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:36:19.501121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:36:20.161569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:36:23.482134+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:38:36.480 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:38:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.582+0000 7f58b8a93640 1 -- 172.21.15.33:0/1054089969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4100ce0 msgr2=0x7f58b4101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.582+0000 7f58b8a93640 1 --2- 172.21.15.33:0/1054089969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4100ce0 0x7f58b4101160 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009900 tx=0x7f58a802bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.583+0000 7f58b8a93640 1 -- 172.21.15.33:0/1054089969 shutdown_connections 2024-04-02T13:38:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.583+0000 7f58b8a93640 1 --2- 172.21.15.33:0/1054089969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58b41016a0 0x7f58b4108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.583+0000 7f58b8a93640 1 --2- 172.21.15.33:0/1054089969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4100ce0 0x7f58b4101160 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.583+0000 7f58b8a93640 1 --2- 172.21.15.33:0/1054089969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58b4101fb0 0x7f58b41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.583+0000 7f58b8a93640 1 -- 172.21.15.33:0/1054089969 >> 172.21.15.33:0/1054089969 conn(0x7f58b40fb6f0 msgr2=0x7f58b40fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.583+0000 7f58b8a93640 1 -- 172.21.15.33:0/1054089969 shutdown_connections 2024-04-02T13:38:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.583+0000 7f58b8a93640 1 -- 172.21.15.33:0/1054089969 wait complete. 2024-04-02T13:38:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.583+0000 7f58b8a93640 1 Processor -- start 2024-04-02T13:38:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.583+0000 7f58b8a93640 1 -- start start 2024-04-02T13:38:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b8a93640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58b4100ce0 0x7f58b4197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b8a93640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58b41016a0 0x7f58b4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b8a93640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4101fb0 0x7f58b419f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b8a93640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f58b410d790 con 0x7f58b4101fb0 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b8a93640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f58b410d610 con 0x7f58b41016a0 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b8a93640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f58b410d910 con 0x7f58b4100ce0 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4101fb0 0x7f58b419f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4101fb0 0x7f58b419f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48434/0 (socket says 172.21.15.33:48434) 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b2d9d640 1 -- 172.21.15.33:0/2298501317 learned_addr learned my addr 172.21.15.33:0/2298501317 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b1d9b640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58b41016a0 0x7f58b4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b259c640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58b4100ce0 0x7f58b4197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b2d9d640 1 -- 172.21.15.33:0/2298501317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58b4100ce0 msgr2=0x7f58b4197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b2d9d640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58b4100ce0 0x7f58b4197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b2d9d640 1 -- 172.21.15.33:0/2298501317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58b41016a0 msgr2=0x7f58b4198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b2d9d640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58b41016a0 0x7f58b4198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b2d9d640 1 -- 172.21.15.33:0/2298501317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b419fa30 con 0x7f58b4101fb0 2024-04-02T13:38:36.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b259c640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58b4100ce0 0x7f58b4197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:36.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b2d9d640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4101fb0 0x7f58b419f3c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00eaa0 tx=0x7f58ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:36.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58a37fe640 1 -- 172.21.15.33:0/2298501317 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00cdb0 con 0x7f58b4101fb0 2024-04-02T13:38:36.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b8a93640 1 -- 172.21.15.33:0/2298501317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b419fd20 con 0x7f58b4101fb0 2024-04-02T13:38:36.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58a37fe640 1 -- 172.21.15.33:0/2298501317 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f58ac005020 con 0x7f58b4101fb0 2024-04-02T13:38:36.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58a37fe640 1 -- 172.21.15.33:0/2298501317 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0116d0 con 0x7f58b4101fb0 2024-04-02T13:38:36.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.584+0000 7f58b8a93640 1 -- 172.21.15.33:0/2298501317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b41a01e0 con 0x7f58b4101fb0 2024-04-02T13:38:36.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.585+0000 7f58b8a93640 1 -- 172.21.15.33:0/2298501317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b4107ef0 con 0x7f58b4101fb0 2024-04-02T13:38:36.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.586+0000 7f58a37fe640 1 -- 172.21.15.33:0/2298501317 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f58ac0051d0 con 0x7f58b4101fb0 2024-04-02T13:38:36.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.587+0000 7f58a37fe640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f589c0a3bb0 0x7f589c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:36.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.587+0000 7f58b259c640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f589c0a3bb0 0x7f589c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:36.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.587+0000 7f58a37fe640 1 -- 172.21.15.33:0/2298501317 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f58ac0d09a0 con 0x7f58b4101fb0 2024-04-02T13:38:36.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.587+0000 7f58b259c640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f589c0a3bb0 0x7f589c0a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f58a40058c0 tx=0x7f58a4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:36.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.592+0000 7f58a37fe640 1 -- 172.21.15.33:0/2298501317 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f58ac016030 con 0x7f58b4101fb0 2024-04-02T13:38:36.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:36 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2838276619' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:36.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:36 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3714302722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:36.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:36 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1698208027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:36.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.777+0000 7f58b8a93640 1 -- 172.21.15.33:0/2298501317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f58b40627a0 con 0x7f58b4101fb0 2024-04-02T13:38:36.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.778+0000 7f58a37fe640 1 -- 172.21.15.33:0/2298501317 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f58ac09cff0 con 0x7f58b4101fb0 2024-04-02T13:38:36.781 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:38:36.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.779+0000 7f58b8a93640 1 -- 172.21.15.33:0/2298501317 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f589c0a3bb0 msgr2=0x7f589c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.779+0000 7f58b8a93640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f589c0a3bb0 0x7f589c0a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f58a40058c0 tx=0x7f58a4005850 comp rx=0 tx=0).stop 2024-04-02T13:38:36.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.779+0000 7f58b8a93640 1 -- 172.21.15.33:0/2298501317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4101fb0 msgr2=0x7f58b419f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.779+0000 7f58b8a93640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4101fb0 0x7f58b419f3c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00eaa0 tx=0x7f58ac00ef60 comp rx=0 tx=0).stop 2024-04-02T13:38:36.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.779+0000 7f58b8a93640 1 -- 172.21.15.33:0/2298501317 shutdown_connections 2024-04-02T13:38:36.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.779+0000 7f58b8a93640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f589c0a3bb0 0x7f589c0a6070 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.779+0000 7f58b8a93640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4101fb0 0x7f58b419f3c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.779+0000 7f58b8a93640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58b41016a0 0x7f58b4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.779+0000 7f58b8a93640 1 --2- 172.21.15.33:0/2298501317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f58b4100ce0 0x7f58b4197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.780+0000 7f58b8a93640 1 -- 172.21.15.33:0/2298501317 >> 172.21.15.33:0/2298501317 conn(0x7f58b40fb6f0 msgr2=0x7f58b41050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:36.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.780+0000 7f58b8a93640 1 -- 172.21.15.33:0/2298501317 shutdown_connections 2024-04-02T13:38:36.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.780+0000 7f58b8a93640 1 -- 172.21.15.33:0/2298501317 wait complete. 2024-04-02T13:38:36.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:38:36.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.899+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/3307640611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac103760 msgr2=0x7f6cac109ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.899+0000 7f6cb1c7b640 1 --2- 172.21.15.33:0/3307640611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac103760 0x7f6cac109ff0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009cd0 tx=0x7f6ca402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.899+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/3307640611 shutdown_connections 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.899+0000 7f6cb1c7b640 1 --2- 172.21.15.33:0/3307640611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac103760 0x7f6cac109ff0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.899+0000 7f6cb1c7b640 1 --2- 172.21.15.33:0/3307640611 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cac102da0 0x7f6cac103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.899+0000 7f6cb1c7b640 1 --2- 172.21.15.33:0/3307640611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cac0ffdd0 0x7f6cac1001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.899+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/3307640611 >> 172.21.15.33:0/3307640611 conn(0x7f6cac0fb720 msgr2=0x7f6cac0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.899+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/3307640611 shutdown_connections 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.899+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/3307640611 wait complete. 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.899+0000 7f6cb1c7b640 1 Processor -- start 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cb1c7b640 1 -- start start 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cb1c7b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cac0ffdd0 0x7f6cac102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cb1c7b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cac102da0 0x7f6cac100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cb1c7b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac103760 0x7f6cac101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:36.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cb1c7b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac10d7f0 con 0x7f6cac103760 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cb1c7b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac10d670 con 0x7f6cac0ffdd0 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cb1c7b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac10d970 con 0x7f6cac102da0 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac103760 0x7f6cac101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac103760 0x7f6cac101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48456/0 (socket says 172.21.15.33:48456) 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cabfff640 1 -- 172.21.15.33:0/576326678 learned_addr learned my addr 172.21.15.33:0/576326678 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cab7fe640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cac0ffdd0 0x7f6cac102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6caaffd640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cac102da0 0x7f6cac100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cabfff640 1 -- 172.21.15.33:0/576326678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cac102da0 msgr2=0x7f6cac100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cabfff640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cac102da0 0x7f6cac100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cabfff640 1 -- 172.21.15.33:0/576326678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cac0ffdd0 msgr2=0x7f6cac102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cabfff640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cac0ffdd0 0x7f6cac102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cabfff640 1 -- 172.21.15.33:0/576326678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cac101b20 con 0x7f6cac103760 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6cabfff640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac103760 0x7f6cac101290 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6ca402c630 tx=0x7f6ca4032900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:36.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.900+0000 7f6ca8ff9640 1 -- 172.21.15.33:0/576326678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4039050 con 0x7f6cac103760 2024-04-02T13:38:36.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.901+0000 7f6ca8ff9640 1 -- 172.21.15.33:0/576326678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6ca403a040 con 0x7f6cac103760 2024-04-02T13:38:36.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.901+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/576326678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cac1a1ef0 con 0x7f6cac103760 2024-04-02T13:38:36.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.901+0000 7f6ca8ff9640 1 -- 172.21.15.33:0/576326678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4040450 con 0x7f6cac103760 2024-04-02T13:38:36.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.901+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/576326678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cac1a2430 con 0x7f6cac103760 2024-04-02T13:38:36.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.902+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/576326678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cac108ed0 con 0x7f6cac103760 2024-04-02T13:38:36.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.903+0000 7f6ca8ff9640 1 -- 172.21.15.33:0/576326678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6ca4003c00 con 0x7f6cac103760 2024-04-02T13:38:36.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.903+0000 7f6ca8ff9640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6c940a3c30 0x7f6c940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:36.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.903+0000 7f6cab7fe640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6c940a3c30 0x7f6c940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:36.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.903+0000 7f6ca8ff9640 1 -- 172.21.15.33:0/576326678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6ca40f3dd0 con 0x7f6cac103760 2024-04-02T13:38:36.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.904+0000 7f6cab7fe640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6c940a3c30 0x7f6c940a60f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6cac069f70 tx=0x7f6c9c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:36.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:36.907+0000 7f6ca8ff9640 1 -- 172.21.15.33:0/576326678 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6ca40c0540 con 0x7f6cac103760 2024-04-02T13:38:36.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:36 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2838276619' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:36.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:36 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3714302722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:36.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:36 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1698208027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:36 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2838276619' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:36 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3714302722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:36 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1698208027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:37.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.094+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/576326678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6cac02d5c0 con 0x7f6cac103760 2024-04-02T13:38:37.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.094+0000 7f6ca8ff9640 1 -- 172.21.15.33:0/576326678 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f6ca40bfe40 con 0x7f6cac103760 2024-04-02T13:38:37.098 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:37.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.096+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/576326678 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6c940a3c30 msgr2=0x7f6c940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.096+0000 7f6cb1c7b640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6c940a3c30 0x7f6c940a60f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6cac069f70 tx=0x7f6c9c00b390 comp rx=0 tx=0).stop 2024-04-02T13:38:37.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.096+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/576326678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac103760 msgr2=0x7f6cac101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.096+0000 7f6cb1c7b640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac103760 0x7f6cac101290 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6ca402c630 tx=0x7f6ca4032900 comp rx=0 tx=0).stop 2024-04-02T13:38:37.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.096+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/576326678 shutdown_connections 2024-04-02T13:38:37.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.096+0000 7f6cb1c7b640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f6c940a3c30 0x7f6c940a60f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.096+0000 7f6cb1c7b640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cac103760 0x7f6cac101290 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.096+0000 7f6cb1c7b640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cac102da0 0x7f6cac100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.096+0000 7f6cb1c7b640 1 --2- 172.21.15.33:0/576326678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cac0ffdd0 0x7f6cac102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.096+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/576326678 >> 172.21.15.33:0/576326678 conn(0x7f6cac0fb720 msgr2=0x7f6cac1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:37.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.097+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/576326678 shutdown_connections 2024-04-02T13:38:37.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.097+0000 7f6cb1c7b640 1 -- 172.21.15.33:0/576326678 wait complete. 2024-04-02T13:38:37.111 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:38:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.213+0000 7fcb9afa8640 1 -- 172.21.15.33:0/2861587508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb94101bf0 msgr2=0x7fcb94102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.213+0000 7fcb9afa8640 1 --2- 172.21.15.33:0/2861587508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb94101bf0 0x7fcb94102070 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009900 tx=0x7fcb8002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.213+0000 7fcb9afa8640 1 -- 172.21.15.33:0/2861587508 shutdown_connections 2024-04-02T13:38:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.213+0000 7fcb9afa8640 1 --2- 172.21.15.33:0/2861587508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb941025b0 0x7fcb94108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.213+0000 7fcb9afa8640 1 --2- 172.21.15.33:0/2861587508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb94101bf0 0x7fcb94102070 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.213+0000 7fcb9afa8640 1 --2- 172.21.15.33:0/2861587508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb94100a00 0x7fcb94100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb9afa8640 1 -- 172.21.15.33:0/2861587508 >> 172.21.15.33:0/2861587508 conn(0x7fcb940fbf90 msgr2=0x7fcb940fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb9afa8640 1 -- 172.21.15.33:0/2861587508 shutdown_connections 2024-04-02T13:38:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb9afa8640 1 -- 172.21.15.33:0/2861587508 wait complete. 2024-04-02T13:38:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb9afa8640 1 Processor -- start 2024-04-02T13:38:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb9afa8640 1 -- start start 2024-04-02T13:38:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb9afa8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb94100a00 0x7fcb94069830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb9afa8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb94101bf0 0x7fcb9419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb9afa8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb941025b0 0x7fcb9419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb9afa8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9410c660 con 0x7fcb94101bf0 2024-04-02T13:38:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb9afa8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9410c4e0 con 0x7fcb941025b0 2024-04-02T13:38:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb9afa8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9410c7e0 con 0x7fcb94100a00 2024-04-02T13:38:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb93fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb94101bf0 0x7fcb9419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb93fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb94101bf0 0x7fcb9419cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48484/0 (socket says 172.21.15.33:48484) 2024-04-02T13:38:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb93fff640 1 -- 172.21.15.33:0/988726349 learned_addr learned my addr 172.21.15.33:0/988726349 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.214+0000 7fcb98d44640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb94100a00 0x7fcb94069830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.215+0000 7fcb99545640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb941025b0 0x7fcb9419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.215+0000 7fcb93fff640 1 -- 172.21.15.33:0/988726349 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb94100a00 msgr2=0x7fcb94069830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.215+0000 7fcb93fff640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb94100a00 0x7fcb94069830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.215+0000 7fcb93fff640 1 -- 172.21.15.33:0/988726349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb941025b0 msgr2=0x7fcb9419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.215+0000 7fcb93fff640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb941025b0 0x7fcb9419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.215+0000 7fcb93fff640 1 -- 172.21.15.33:0/988726349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9419fa50 con 0x7fcb94101bf0 2024-04-02T13:38:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.215+0000 7fcb93fff640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb94101bf0 0x7fcb9419cb20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009230 tx=0x7fcb80004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.215+0000 7fcb91ffb640 1 -- 172.21.15.33:0/988726349 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb80043020 con 0x7fcb94101bf0 2024-04-02T13:38:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.215+0000 7fcb9afa8640 1 -- 172.21.15.33:0/988726349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9419fce0 con 0x7fcb94101bf0 2024-04-02T13:38:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.215+0000 7fcb91ffb640 1 -- 172.21.15.33:0/988726349 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcb80005460 con 0x7fcb94101bf0 2024-04-02T13:38:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.215+0000 7fcb91ffb640 1 -- 172.21.15.33:0/988726349 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb80003c60 con 0x7fcb94101bf0 2024-04-02T13:38:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.215+0000 7fcb9afa8640 1 -- 172.21.15.33:0/988726349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb941a01d0 con 0x7fcb94101bf0 2024-04-02T13:38:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.216+0000 7fcb9afa8640 1 -- 172.21.15.33:0/988726349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb94107e40 con 0x7fcb94101bf0 2024-04-02T13:38:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.217+0000 7fcb91ffb640 1 -- 172.21.15.33:0/988726349 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcb80046020 con 0x7fcb94101bf0 2024-04-02T13:38:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.218+0000 7fcb91ffb640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fcb840a3bb0 0x7fcb840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.218+0000 7fcb91ffb640 1 -- 172.21.15.33:0/988726349 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fcb800f24a0 con 0x7fcb94101bf0 2024-04-02T13:38:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.218+0000 7fcb98d44640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fcb840a3bb0 0x7fcb840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.218+0000 7fcb98d44640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fcb840a3bb0 0x7fcb840a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcb94101a50 tx=0x7fcb88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.222+0000 7fcb91ffb640 1 -- 172.21.15.33:0/988726349 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcb8004d070 con 0x7fcb94101bf0 2024-04-02T13:38:37.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.408+0000 7fcb9afa8640 1 -- 172.21.15.33:0/988726349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb940627a0 con 0x7fcb94101bf0 2024-04-02T13:38:37.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.409+0000 7fcb91ffb640 1 -- 172.21.15.33:0/988726349 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fcb800beaf0 con 0x7fcb94101bf0 2024-04-02T13:38:37.413 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-02T13:38:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.411+0000 7fcb9afa8640 1 -- 172.21.15.33:0/988726349 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fcb840a3bb0 msgr2=0x7fcb840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.411+0000 7fcb9afa8640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fcb840a3bb0 0x7fcb840a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcb94101a50 tx=0x7fcb88005f90 comp rx=0 tx=0).stop 2024-04-02T13:38:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.411+0000 7fcb9afa8640 1 -- 172.21.15.33:0/988726349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb94101bf0 msgr2=0x7fcb9419cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.411+0000 7fcb9afa8640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb94101bf0 0x7fcb9419cb20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009230 tx=0x7fcb80004e20 comp rx=0 tx=0).stop 2024-04-02T13:38:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.411+0000 7fcb9afa8640 1 -- 172.21.15.33:0/988726349 shutdown_connections 2024-04-02T13:38:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.411+0000 7fcb9afa8640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fcb840a3bb0 0x7fcb840a6070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.411+0000 7fcb9afa8640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb941025b0 0x7fcb9419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.411+0000 7fcb9afa8640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb94101bf0 0x7fcb9419cb20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.411+0000 7fcb9afa8640 1 --2- 172.21.15.33:0/988726349 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb94100a00 0x7fcb94069830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.411+0000 7fcb9afa8640 1 -- 172.21.15.33:0/988726349 >> 172.21.15.33:0/988726349 conn(0x7fcb940fbf90 msgr2=0x7fcb94105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.411+0000 7fcb9afa8640 1 -- 172.21.15.33:0/988726349 shutdown_connections 2024-04-02T13:38:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.411+0000 7fcb9afa8640 1 -- 172.21.15.33:0/988726349 wait complete. 2024-04-02T13:38:37.426 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:38:37.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.531+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/3794672835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5edc1036c0 msgr2=0x7f5edc109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.531+0000 7f5ee3eeb640 1 --2- 172.21.15.33:0/3794672835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5edc1036c0 0x7f5edc109f50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009ca0 tx=0x7f5ed402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.531+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/3794672835 shutdown_connections 2024-04-02T13:38:37.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.531+0000 7f5ee3eeb640 1 --2- 172.21.15.33:0/3794672835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5edc1036c0 0x7f5edc109f50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.531+0000 7f5ee3eeb640 1 --2- 172.21.15.33:0/3794672835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5edc102d00 0x7f5edc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.531+0000 7f5ee3eeb640 1 --2- 172.21.15.33:0/3794672835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5edc101b10 0x7f5edc101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.531+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/3794672835 >> 172.21.15.33:0/3794672835 conn(0x7f5edc0fd0a0 msgr2=0x7f5edc0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:37.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.531+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/3794672835 shutdown_connections 2024-04-02T13:38:37.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.531+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/3794672835 wait complete. 2024-04-02T13:38:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee3eeb640 1 Processor -- start 2024-04-02T13:38:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee3eeb640 1 -- start start 2024-04-02T13:38:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee3eeb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5edc101b10 0x7f5edc19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee3eeb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5edc102d00 0x7f5edc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee3eeb640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5edc1036c0 0x7f5edc1a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee3eeb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc10d770 con 0x7f5edc101b10 2024-04-02T13:38:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee3eeb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc10d5f0 con 0x7f5edc102d00 2024-04-02T13:38:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee3eeb640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc10d8f0 con 0x7f5edc1036c0 2024-04-02T13:38:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee1c87640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5edc101b10 0x7f5edc19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee1c87640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5edc101b10 0x7f5edc19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48500/0 (socket says 172.21.15.33:48500) 2024-04-02T13:38:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee1c87640 1 -- 172.21.15.33:0/1420943643 learned_addr learned my addr 172.21.15.33:0/1420943643 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee1486640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5edc102d00 0x7f5edc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:37.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee2488640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5edc1036c0 0x7f5edc1a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee1c87640 1 -- 172.21.15.33:0/1420943643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5edc1036c0 msgr2=0x7f5edc1a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee1c87640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5edc1036c0 0x7f5edc1a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee1c87640 1 -- 172.21.15.33:0/1420943643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5edc102d00 msgr2=0x7f5edc19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee1c87640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5edc102d00 0x7f5edc19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee1c87640 1 -- 172.21.15.33:0/1420943643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc1a1c40 con 0x7f5edc101b10 2024-04-02T13:38:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee1486640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5edc102d00 0x7f5edc19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee1c87640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5edc101b10 0x7f5edc19a020 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800c970 tx=0x7f5ed800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ed2ffd640 1 -- 172.21.15.33:0/1420943643 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8004220 con 0x7f5edc101b10 2024-04-02T13:38:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/1420943643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5edc1a1f30 con 0x7f5edc101b10 2024-04-02T13:38:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.532+0000 7f5ed2ffd640 1 -- 172.21.15.33:0/1420943643 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5ed80043c0 con 0x7f5edc101b10 2024-04-02T13:38:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.533+0000 7f5ed2ffd640 1 -- 172.21.15.33:0/1420943643 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8004a00 con 0x7f5edc101b10 2024-04-02T13:38:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.533+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/1420943643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5edc1a2390 con 0x7f5edc101b10 2024-04-02T13:38:37.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.533+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/1420943643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5edc1a2620 con 0x7f5edc101b10 2024-04-02T13:38:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.535+0000 7f5ed2ffd640 1 -- 172.21.15.33:0/1420943643 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5ed8004ba0 con 0x7f5edc101b10 2024-04-02T13:38:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.535+0000 7f5ed2ffd640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5ec80a3bb0 0x7f5ec80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.535+0000 7f5ee1486640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5ec80a3bb0 0x7f5ec80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.535+0000 7f5ed2ffd640 1 -- 172.21.15.33:0/1420943643 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5ed80d0920 con 0x7f5edc101b10 2024-04-02T13:38:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.536+0000 7f5ee1486640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5ec80a3bb0 0x7f5ec80a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5edc19b530 tx=0x7f5ecc00b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:37.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.539+0000 7f5ed2ffd640 1 -- 172.21.15.33:0/1420943643 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5ed8014030 con 0x7f5edc101b10 2024-04-02T13:38:37.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.729+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/1420943643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5edc1a27f0 con 0x7f5edc101b10 2024-04-02T13:38:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.730+0000 7f5ed2ffd640 1 -- 172.21.15.33:0/1420943643 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f5ed809cf70 con 0x7f5edc101b10 2024-04-02T13:38:37.734 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.732+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/1420943643 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5ec80a3bb0 msgr2=0x7f5ec80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.732+0000 7f5ee3eeb640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5ec80a3bb0 0x7f5ec80a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5edc19b530 tx=0x7f5ecc00b5f0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.732+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/1420943643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5edc101b10 msgr2=0x7f5edc19a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.732+0000 7f5ee3eeb640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5edc101b10 0x7f5edc19a020 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800c970 tx=0x7f5ed800ce30 comp rx=0 tx=0).stop 2024-04-02T13:38:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.732+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/1420943643 shutdown_connections 2024-04-02T13:38:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.732+0000 7f5ee3eeb640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f5ec80a3bb0 0x7f5ec80a6070 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.732+0000 7f5ee3eeb640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5edc1036c0 0x7f5edc1a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.732+0000 7f5ee3eeb640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5edc102d00 0x7f5edc19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.732+0000 7f5ee3eeb640 1 --2- 172.21.15.33:0/1420943643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5edc101b10 0x7f5edc19a020 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.732+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/1420943643 >> 172.21.15.33:0/1420943643 conn(0x7f5edc0fd0a0 msgr2=0x7f5edc073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.732+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/1420943643 shutdown_connections 2024-04-02T13:38:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.732+0000 7f5ee3eeb640 1 -- 172.21.15.33:0/1420943643 wait complete. 2024-04-02T13:38:37.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:37 smithi033 ceph-mon[32390]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:38:37.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:37 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2298501317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:37.747 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:37 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/576326678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:37.747 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:37 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/988726349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:37.749 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:38:37.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.848+0000 7fce49d1b640 1 -- 172.21.15.33:0/4121360786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44107890 msgr2=0x7fce44109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.848+0000 7fce49d1b640 1 --2- 172.21.15.33:0/4121360786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44107890 0x7fce44109c80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009900 tx=0x7fce3c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.848+0000 7fce49d1b640 1 -- 172.21.15.33:0/4121360786 shutdown_connections 2024-04-02T13:38:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.848+0000 7fce49d1b640 1 --2- 172.21.15.33:0/4121360786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44107890 0x7fce44109c80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.848+0000 7fce49d1b640 1 --2- 172.21.15.33:0/4121360786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce44104f60 0x7fce44107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.848+0000 7fce49d1b640 1 --2- 172.21.15.33:0/4121360786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce44102630 0x7fce44104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.848+0000 7fce49d1b640 1 -- 172.21.15.33:0/4121360786 >> 172.21.15.33:0/4121360786 conn(0x7fce440fbf80 msgr2=0x7fce440fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:37.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.849+0000 7fce49d1b640 1 -- 172.21.15.33:0/4121360786 shutdown_connections 2024-04-02T13:38:37.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.849+0000 7fce49d1b640 1 -- 172.21.15.33:0/4121360786 wait complete. 2024-04-02T13:38:37.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.849+0000 7fce49d1b640 1 Processor -- start 2024-04-02T13:38:37.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.849+0000 7fce49d1b640 1 -- start start 2024-04-02T13:38:37.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.849+0000 7fce49d1b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44102630 0x7fce44101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:37.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.849+0000 7fce49d1b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce44104f60 0x7fce441005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:37.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.849+0000 7fce49d1b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce44107890 0x7fce44100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:37.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.849+0000 7fce49d1b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce4410d7f0 con 0x7fce44102630 2024-04-02T13:38:37.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.849+0000 7fce49d1b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fce4410d670 con 0x7fce44104f60 2024-04-02T13:38:37.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.849+0000 7fce49d1b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fce4410d970 con 0x7fce44107890 2024-04-02T13:38:37.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce437fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44102630 0x7fce44101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:37.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce437fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44102630 0x7fce44101fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48518/0 (socket says 172.21.15.33:48518) 2024-04-02T13:38:37.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce437fe640 1 -- 172.21.15.33:0/2836158947 learned_addr learned my addr 172.21.15.33:0/2836158947 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:37.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce42ffd640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce44104f60 0x7fce441005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:37.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce43fff640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce44107890 0x7fce44100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:37.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce437fe640 1 -- 172.21.15.33:0/2836158947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce44107890 msgr2=0x7fce44100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce437fe640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce44107890 0x7fce44100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce437fe640 1 -- 172.21.15.33:0/2836158947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce44104f60 msgr2=0x7fce441005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:37.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce437fe640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce44104f60 0x7fce441005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:37.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce437fe640 1 -- 172.21.15.33:0/2836158947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce441013b0 con 0x7fce44102630 2024-04-02T13:38:37.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce43fff640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce44107890 0x7fce44100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:37.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce437fe640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44102630 0x7fce44101fa0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fce3400db80 tx=0x7fce3400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:37.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce40ff9640 1 -- 172.21.15.33:0/2836158947 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34017070 con 0x7fce44102630 2024-04-02T13:38:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce49d1b640 1 -- 172.21.15.33:0/2836158947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce441a1fb0 con 0x7fce44102630 2024-04-02T13:38:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce40ff9640 1 -- 172.21.15.33:0/2836158947 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fce34003e40 con 0x7fce44102630 2024-04-02T13:38:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce40ff9640 1 -- 172.21.15.33:0/2836158947 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce340106c0 con 0x7fce44102630 2024-04-02T13:38:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.850+0000 7fce49d1b640 1 -- 172.21.15.33:0/2836158947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce441a2410 con 0x7fce44102630 2024-04-02T13:38:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.851+0000 7fce49d1b640 1 -- 172.21.15.33:0/2836158947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce440fda70 con 0x7fce44102630 2024-04-02T13:38:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.852+0000 7fce40ff9640 1 -- 172.21.15.33:0/2836158947 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fce34003970 con 0x7fce44102630 2024-04-02T13:38:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.853+0000 7fce40ff9640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fce2c0a3bb0 0x7fce2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:37.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.853+0000 7fce42ffd640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fce2c0a3bb0 0x7fce2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.853+0000 7fce40ff9640 1 -- 172.21.15.33:0/2836158947 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fce340d0800 con 0x7fce44102630 2024-04-02T13:38:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.853+0000 7fce42ffd640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fce2c0a3bb0 0x7fce2c0a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fce44101d10 tx=0x7fce38009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:37.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:37.857+0000 7fce40ff9640 1 -- 172.21.15.33:0/2836158947 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fce34020070 con 0x7fce44102630 2024-04-02T13:38:37.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:37 smithi145 ceph-mon[35646]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:38:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:37 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2298501317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:37 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/576326678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:37.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:37 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/988726349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:37.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:37 smithi050 ceph-mon[35524]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-02T13:38:37.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:37 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2298501317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:37.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:37 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/576326678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:37.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:37 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/988726349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:38.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.085+0000 7fce49d1b640 1 -- 172.21.15.33:0/2836158947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce4402d370 con 0x7fce44102630 2024-04-02T13:38:38.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.086+0000 7fce40ff9640 1 -- 172.21.15.33:0/2836158947 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11873 (secure 0 0 0) 0x7fce34003c60 con 0x7fce44102630 2024-04-02T13:38:38.089 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:38.089 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:36:47.489590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:38:38.090 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-02T13:38:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.087+0000 7fce49d1b640 1 -- 172.21.15.33:0/2836158947 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fce2c0a3bb0 msgr2=0x7fce2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.087+0000 7fce49d1b640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fce2c0a3bb0 0x7fce2c0a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fce44101d10 tx=0x7fce38009110 comp rx=0 tx=0).stop 2024-04-02T13:38:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.087+0000 7fce49d1b640 1 -- 172.21.15.33:0/2836158947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44102630 msgr2=0x7fce44101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.087+0000 7fce49d1b640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44102630 0x7fce44101fa0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fce3400db80 tx=0x7fce3400df80 comp rx=0 tx=0).stop 2024-04-02T13:38:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.087+0000 7fce49d1b640 1 -- 172.21.15.33:0/2836158947 shutdown_connections 2024-04-02T13:38:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.087+0000 7fce49d1b640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fce2c0a3bb0 0x7fce2c0a6070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.087+0000 7fce49d1b640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce44107890 0x7fce44100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.087+0000 7fce49d1b640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce44104f60 0x7fce441005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.087+0000 7fce49d1b640 1 --2- 172.21.15.33:0/2836158947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce44102630 0x7fce44101fa0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.088+0000 7fce49d1b640 1 -- 172.21.15.33:0/2836158947 >> 172.21.15.33:0/2836158947 conn(0x7fce440fbf80 msgr2=0x7fce44102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.088+0000 7fce49d1b640 1 -- 172.21.15.33:0/2836158947 shutdown_connections 2024-04-02T13:38:38.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.088+0000 7fce49d1b640 1 -- 172.21.15.33:0/2836158947 wait complete. 2024-04-02T13:38:38.103 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:38:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2094693706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c4100a20 msgr2=0x7fd1c4100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 --2- 172.21.15.33:0/2094693706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c4100a20 0x7fd1c4100e40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009900 tx=0x7fd1b402bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2094693706 shutdown_connections 2024-04-02T13:38:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 --2- 172.21.15.33:0/2094693706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1c41025d0 0x7fd1c4108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 --2- 172.21.15.33:0/2094693706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1c4101c10 0x7fd1c4102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 --2- 172.21.15.33:0/2094693706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c4100a20 0x7fd1c4100e40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2094693706 >> 172.21.15.33:0/2094693706 conn(0x7fd1c40fbfb0 msgr2=0x7fd1c40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2094693706 shutdown_connections 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2094693706 wait complete. 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 Processor -- start 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 -- start start 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1c4100a20 0x7fd1c4196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1c4101c10 0x7fd1c4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.203+0000 7fd1c95e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c41025d0 0x7fd1c419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c95e2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c410c680 con 0x7fd1c41025d0 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c95e2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c410c500 con 0x7fd1c4100a20 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c95e2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c410c800 con 0x7fd1c4101c10 2024-04-02T13:38:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c41025d0 0x7fd1c419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c2ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1c4100a20 0x7fd1c4196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c41025d0 0x7fd1c419e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48536/0 (socket says 172.21.15.33:48536) 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c37fe640 1 -- 172.21.15.33:0/2337334686 learned_addr learned my addr 172.21.15.33:0/2337334686 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1bbfff640 1 --2- 172.21.15.33:0/2337334686 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1c4101c10 0x7fd1c4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c37fe640 1 -- 172.21.15.33:0/2337334686 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1c4101c10 msgr2=0x7fd1c4197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c37fe640 1 --2- 172.21.15.33:0/2337334686 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1c4101c10 0x7fd1c4197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c37fe640 1 -- 172.21.15.33:0/2337334686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1c4100a20 msgr2=0x7fd1c4196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c37fe640 1 --2- 172.21.15.33:0/2337334686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1c4100a20 0x7fd1c4196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c37fe640 1 -- 172.21.15.33:0/2337334686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c419e970 con 0x7fd1c41025d0 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c37fe640 1 --2- 172.21.15.33:0/2337334686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c41025d0 0x7fd1c419e300 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00eaa0 tx=0x7fd1bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c0ff9640 1 -- 172.21.15.33:0/2337334686 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00cdb0 con 0x7fd1c41025d0 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c0ff9640 1 -- 172.21.15.33:0/2337334686 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd1bc005020 con 0x7fd1c41025d0 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c0ff9640 1 -- 172.21.15.33:0/2337334686 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0116b0 con 0x7fd1c41025d0 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2337334686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c419ec60 con 0x7fd1c41025d0 2024-04-02T13:38:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.204+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2337334686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c419f160 con 0x7fd1c41025d0 2024-04-02T13:38:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.205+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2337334686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c4107dc0 con 0x7fd1c41025d0 2024-04-02T13:38:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.206+0000 7fd1c0ff9640 1 -- 172.21.15.33:0/2337334686 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd1bc0051d0 con 0x7fd1c41025d0 2024-04-02T13:38:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.207+0000 7fd1c0ff9640 1 --2- 172.21.15.33:0/2337334686 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd1b00a3bb0 0x7fd1b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.207+0000 7fd1c0ff9640 1 -- 172.21.15.33:0/2337334686 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd1bc0d0e20 con 0x7fd1c41025d0 2024-04-02T13:38:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.207+0000 7fd1c2ffd640 1 --2- 172.21.15.33:0/2337334686 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd1b00a3bb0 0x7fd1b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.207+0000 7fd1c2ffd640 1 --2- 172.21.15.33:0/2337334686 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd1b00a3bb0 0x7fd1b00a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4000c00 tx=0x7fd1b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.211+0000 7fd1c0ff9640 1 -- 172.21.15.33:0/2337334686 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd1bc016030 con 0x7fd1c41025d0 2024-04-02T13:38:38.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.397+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2337334686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1c402d5c0 con 0x7fd1c41025d0 2024-04-02T13:38:38.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.399+0000 7fd1c0ff9640 1 -- 172.21.15.33:0/2337334686 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24621 (secure 0 0 0) 0x7fd1bc09d410 con 0x7fd1c41025d0 2024-04-02T13:38:38.403 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:38.403 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":94,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:38:18.498977+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:38:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.401+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2337334686 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd1b00a3bb0 msgr2=0x7fd1b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.401+0000 7fd1c95e2640 1 --2- 172.21.15.33:0/2337334686 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd1b00a3bb0 0x7fd1b00a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4000c00 tx=0x7fd1b4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.401+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2337334686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c41025d0 msgr2=0x7fd1c419e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.401+0000 7fd1c95e2640 1 --2- 172.21.15.33:0/2337334686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c41025d0 0x7fd1c419e300 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00eaa0 tx=0x7fd1bc00ef60 comp rx=0 tx=0).stop 2024-04-02T13:38:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.401+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2337334686 shutdown_connections 2024-04-02T13:38:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.401+0000 7fd1c95e2640 1 --2- 172.21.15.33:0/2337334686 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd1b00a3bb0 0x7fd1b00a6070 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.401+0000 7fd1c95e2640 1 --2- 172.21.15.33:0/2337334686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1c41025d0 0x7fd1c419e300 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.401+0000 7fd1c95e2640 1 --2- 172.21.15.33:0/2337334686 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1c4101c10 0x7fd1c4197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.401+0000 7fd1c95e2640 1 --2- 172.21.15.33:0/2337334686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1c4100a20 0x7fd1c4196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.401+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2337334686 >> 172.21.15.33:0/2337334686 conn(0x7fd1c40fbfb0 msgr2=0x7fd1c4104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.401+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2337334686 shutdown_connections 2024-04-02T13:38:38.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.401+0000 7fd1c95e2640 1 -- 172.21.15.33:0/2337334686 wait complete. 2024-04-02T13:38:38.417 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-02T13:36:21.184650+0000', 'modified': '2024-04-02T13:36:47.489590+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24551, 'mds_1': 24512, 'mds_2': 24473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6835/4110737670', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6834', 'nonce': 4110737670}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 4110737670}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.50:6835/1580073716', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6834', 'nonce': 1580073716}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 1580073716}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6837/4234446304', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6836', 'nonce': 4234446304}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 4234446304}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6839/3484016588', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6838', 'nonce': 3484016588}, {'type': 'v1', 'addr': '172.21.15.145:6839', 'nonce': 3484016588}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.50:6837/3788578797', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6836', 'nonce': 3788578797}, {'type': 'v1', 'addr': '172.21.15.50:6837', 'nonce': 3788578797}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24551': {'gid': 24551, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.50:6839/932333028', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6838', 'nonce': 932333028}, {'type': 'v1', 'addr': '172.21.15.50:6839', 'nonce': 932333028}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24551, 'qdb_cluster': [24473, 24512, 24551]} 2024-04-02T13:38:38.417 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-02T13:38:38.417 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-02T13:38:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.517+0000 7fabaec77640 1 -- 172.21.15.33:0/2474652934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba8102d20 msgr2=0x7faba81031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.517+0000 7fabaec77640 1 --2- 172.21.15.33:0/2474652934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba8102d20 0x7faba81031a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fab98009ca0 tx=0x7fab9802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.517+0000 7fabaec77640 1 -- 172.21.15.33:0/2474652934 shutdown_connections 2024-04-02T13:38:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.517+0000 7fabaec77640 1 --2- 172.21.15.33:0/2474652934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faba81036e0 0x7faba8109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.517+0000 7fabaec77640 1 --2- 172.21.15.33:0/2474652934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba8102d20 0x7faba81031a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.517+0000 7fabaec77640 1 --2- 172.21.15.33:0/2474652934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faba8101b30 0x7faba8101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.517+0000 7fabaec77640 1 -- 172.21.15.33:0/2474652934 >> 172.21.15.33:0/2474652934 conn(0x7faba80fd0a0 msgr2=0x7faba80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.517+0000 7fabaec77640 1 -- 172.21.15.33:0/2474652934 shutdown_connections 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.517+0000 7fabaec77640 1 -- 172.21.15.33:0/2474652934 wait complete. 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabaec77640 1 Processor -- start 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabaec77640 1 -- start start 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabaec77640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faba8101b30 0x7faba8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabaec77640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faba8102d20 0x7faba8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabaec77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba81036e0 0x7faba819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabaec77640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faba810d790 con 0x7faba81036e0 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabaec77640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faba810d610 con 0x7faba8101b30 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabaec77640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faba810d910 con 0x7faba8102d20 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabad214640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba81036e0 0x7faba819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabad214640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba81036e0 0x7faba819f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48556/0 (socket says 172.21.15.33:48556) 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabad214640 1 -- 172.21.15.33:0/2109521221 learned_addr learned my addr 172.21.15.33:0/2109521221 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabaca13640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faba8101b30 0x7faba8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7faba7fff640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faba8102d20 0x7faba8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabad214640 1 -- 172.21.15.33:0/2109521221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faba8102d20 msgr2=0x7faba8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabad214640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faba8102d20 0x7faba8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabad214640 1 -- 172.21.15.33:0/2109521221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faba8101b30 msgr2=0x7faba8197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabad214640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faba8101b30 0x7faba8197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7fabad214640 1 -- 172.21.15.33:0/2109521221 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba819fa60 con 0x7faba81036e0 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.518+0000 7faba7fff640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faba8102d20 0x7faba8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.519+0000 7fabad214640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba81036e0 0x7faba819f3f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fab9000d750 tx=0x7fab9000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.519+0000 7faba5ffb640 1 -- 172.21.15.33:0/2109521221 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90017070 con 0x7faba81036e0 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.519+0000 7faba5ffb640 1 -- 172.21.15.33:0/2109521221 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fab900054f0 con 0x7faba81036e0 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.519+0000 7fabaec77640 1 -- 172.21.15.33:0/2109521221 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba819fd50 con 0x7faba81036e0 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.519+0000 7faba5ffb640 1 -- 172.21.15.33:0/2109521221 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90004cf0 con 0x7faba81036e0 2024-04-02T13:38:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.519+0000 7fabaec77640 1 -- 172.21.15.33:0/2109521221 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba81a01c0 con 0x7faba81036e0 2024-04-02T13:38:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.521+0000 7faba5ffb640 1 -- 172.21.15.33:0/2109521221 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fab90004e90 con 0x7faba81036e0 2024-04-02T13:38:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.521+0000 7fabb185b740 1 Processor -- start 2024-04-02T13:38:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabb185b740 1 -- start start 2024-04-02T13:38:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7faba5ffb640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fab940a3bb0 0x7fab940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabb185b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 0x55fc48cb7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabb185b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cb84d0 0x55fc48cbcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabb185b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cbd640 0x55fc48cbfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabb185b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fc48c1c060 con 0x55fc48cbd640 2024-04-02T13:38:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabb185b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fc48bc4650 con 0x55fc48cb84d0 2024-04-02T13:38:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabb185b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fc48c18010 con 0x55fc48cb7ae0 2024-04-02T13:38:38.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7faba7fff640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fab940a3bb0 0x7fab940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7faba5ffb640 1 -- 172.21.15.33:0/2109521221 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fab900d01c0 con 0x7faba81036e0 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabaca13640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 0x55fc48cb7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabad214640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cbd640 0x55fc48cbfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabaca13640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 0x55fc48cb7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:33576/0 (socket says 172.21.15.33:33576) 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabaca13640 1 -- 172.21.15.33:0/3959095419 learned_addr learned my addr 172.21.15.33:0/3959095419 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7faba7fff640 1 --2- 172.21.15.33:0/3959095419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cb84d0 0x55fc48cbcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabad214640 1 -- 172.21.15.33:0/3959095419 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 msgr2=0x55fc48cb7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabad214640 1 --2- 172.21.15.33:0/3959095419 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 0x55fc48cb7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabad214640 1 -- 172.21.15.33:0/3959095419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cb84d0 msgr2=0x55fc48cbcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabad214640 1 --2- 172.21.15.33:0/3959095419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cb84d0 0x55fc48cbcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabad214640 1 -- 172.21.15.33:0/3959095419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc489302d0 con 0x55fc48cbd640 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7faba7fff640 1 --2- 172.21.15.33:0/3959095419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cb84d0 0x55fc48cbcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7faba7fff640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fab940a3bb0 0x7fab940a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fab98006010 tx=0x7fab98005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.522+0000 7fabad214640 1 --2- 172.21.15.33:0/3959095419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cbd640 0x55fc48cbfad0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fab90009610 tx=0x7fab9009bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee003b7ceac12c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.523+0000 7fab9effd640 1 -- 172.21.15.33:0/3959095419 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab900a0460 con 0x55fc48cbd640 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.523+0000 7fab9effd640 1 -- 172.21.15.33:0/3959095419 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fab900a4d20 con 0x55fc48cbd640 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.523+0000 7fab9effd640 1 -- 172.21.15.33:0/3959095419 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab900a0080 con 0x55fc48cbd640 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.523+0000 7fabb185b740 1 -- 172.21.15.33:0/3959095419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cbd640 msgr2=0x55fc48cbfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.523+0000 7fabb185b740 1 --2- 172.21.15.33:0/3959095419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cbd640 0x55fc48cbfad0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fab90009610 tx=0x7fab9009bf90 comp rx=0 tx=0).stop 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.523+0000 7fabb185b740 1 -- 172.21.15.33:0/3959095419 shutdown_connections 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.523+0000 7fabb185b740 1 --2- 172.21.15.33:0/3959095419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cbd640 0x55fc48cbfad0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.523+0000 7fabb185b740 1 --2- 172.21.15.33:0/3959095419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cb84d0 0x55fc48cbcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.523+0000 7fabb185b740 1 --2- 172.21.15.33:0/3959095419 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 0x55fc48cb7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.523+0000 7fabb185b740 1 -- 172.21.15.33:0/3959095419 >> 172.21.15.33:0/3959095419 conn(0x55fc48cb3050 msgr2=0x55fc48cb3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.523+0000 7fabb185b740 1 -- 172.21.15.33:0/3959095419 shutdown_connections 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.523+0000 7fabb185b740 1 -- 172.21.15.33:0/3959095419 wait complete. 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7fabb185b740 1 -- start start 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7fabb185b740 1 Processor -- start 2024-04-02T13:38:38.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7fabb185b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 0x55fc48cc2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7fabb185b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 0x55fc48cc2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7fabb185b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cbd640 0x55fc48cc9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7fabb185b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fc48cc05f0 con 0x55fc48cb84d0 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7fabb185b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fc48bd7a10 con 0x55fc48cbd640 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7fabb185b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fc48cc0770 con 0x55fc48cb7ae0 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7fabb185b740 10 client.0 _ll_register_callbacks cb 0x55fc48c140b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7faba7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 0x55fc48cc2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7faba7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 0x55fc48cc2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48574/0 (socket says 172.21.15.33:48574) 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7faba7fff640 1 -- 172.21.15.33:0/730778468 learned_addr learned my addr 172.21.15.33:0/730778468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7fabaca13640 1 --2- 172.21.15.33:0/730778468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 0x55fc48cc2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7fabad214640 1 --2- 172.21.15.33:0/730778468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cbd640 0x55fc48cc9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7faba7fff640 1 -- 172.21.15.33:0/730778468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 msgr2=0x55fc48cc2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7faba7fff640 1 --2- 172.21.15.33:0/730778468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 0x55fc48cc2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7faba7fff640 1 -- 172.21.15.33:0/730778468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cbd640 msgr2=0x55fc48cc9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7faba7fff640 1 --2- 172.21.15.33:0/730778468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cbd640 0x55fc48cc9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.524+0000 7faba7fff640 1 -- 172.21.15.33:0/730778468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc489fa6b0 con 0x55fc48cb84d0 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7faba7fff640 1 --2- 172.21.15.33:0/730778468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 0x55fc48cc2650 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fab98005fe0 tx=0x7fab9802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d68100640495bca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fab87fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fab87fff640 1 -- 172.21.15.33:0/730778468 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98051060 con 0x55fc48cb84d0 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fab87fff640 1 -- 172.21.15.33:0/730778468 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fab98004450 con 0x55fc48cb84d0 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fabb185b740 1 -- 172.21.15.33:0/730778468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab900a4d20 con 0x55fc48cb84d0 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fab87fff640 1 -- 172.21.15.33:0/730778468 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fab98058050 con 0x55fc48cb84d0 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fabb185b740 10 client.15390 fetch_fsmap learned FSMap version 24 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fabb185b740 1 -- 172.21.15.33:0/730778468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fc48ccc9c0 con 0x55fc48cb84d0 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fab87fff640 1 -- 172.21.15.33:0/730778468 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fab900a4d20 con 0x55fc48cb84d0 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fab87fff640 1 -- 172.21.15.33:0/730778468 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fab9803f040 con 0x55fc48cb84d0 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fabb185b740 10 client.15390 fetch_fsmap finished waiting for FSMap version 24 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fabb185b740 10 client.15390 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-02T13:38:38.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fabb185b740 20 client.15390 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fabb185b740 1 --2- 172.21.15.33:0/730778468 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55fc48ccefe0 0x55fc48cd0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fabb185b740 4 client.15390 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.525+0000 7fabb185b740 1 -- 172.21.15.33:0/730778468 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc48bc3cf0 con 0x55fc48ccefe0 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.526+0000 7fabaca13640 1 --2- 172.21.15.33:0/730778468 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55fc48ccefe0 0x55fc48cd0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.526+0000 7fab87fff640 10 client.15390 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.526+0000 7fabaca13640 1 --2- 172.21.15.33:0/730778468 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55fc48ccefe0 0x55fc48cd0450 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbfff7faf3e6814c server_cookie=6a2b4ba4583ac033 in_seq=0 out_seq=0 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fab87fff640 1 -- 172.21.15.33:0/730778468 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc48bc3cf0 con 0x55fc48ccefe0 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fab87fff640 10 client.15390 handle_command_reply: tid=0 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fabb185b740 1 client.15390 shutdown 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fabb185b740 1 -- 172.21.15.33:0/730778468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 msgr2=0x55fc48cc2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fabb185b740 1 --2- 172.21.15.33:0/730778468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 0x55fc48cc2650 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fab98005fe0 tx=0x7fab9802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fabb185b740 1 -- 172.21.15.33:0/730778468 shutdown_connections 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fabb185b740 1 --2- 172.21.15.33:0/730778468 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55fc48ccefe0 0x55fc48cd0450 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fabb185b740 1 --2- 172.21.15.33:0/730778468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cbd640 0x55fc48cc9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fabb185b740 1 --2- 172.21.15.33:0/730778468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 0x55fc48cc2650 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fab87fff640 0 client.15390 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:38:38.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fabb185b740 1 --2- 172.21.15.33:0/730778468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 0x55fc48cc2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fabb185b740 1 -- 172.21.15.33:0/730778468 >> 172.21.15.33:0/730778468 conn(0x55fc48cb3050 msgr2=0x55fc48cb2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fabb185b740 1 -- 172.21.15.33:0/730778468 shutdown_connections 2024-04-02T13:38:38.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fabb185b740 1 -- 172.21.15.33:0/730778468 wait complete. 2024-04-02T13:38:38.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.527+0000 7fabb185b740 20 client.15390 trim_cache size 0 max 16384 2024-04-02T13:38:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.545+0000 7fabb185b740 1 Processor -- start 2024-04-02T13:38:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.545+0000 7fabb185b740 1 -- start start 2024-04-02T13:38:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.545+0000 7fabb185b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 0x55fc48cc0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.545+0000 7fabb185b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 0x55fc48cc95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.545+0000 7fabb185b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cbd640 0x55fc48cc1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.545+0000 7fabb185b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fc48cce790 con 0x55fc48cb84d0 2024-04-02T13:38:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.545+0000 7fabb185b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fc48ccee60 con 0x55fc48cbd640 2024-04-02T13:38:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.545+0000 7fabb185b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fc48ccf530 con 0x55fc48cb7ae0 2024-04-02T13:38:38.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.546+0000 7faba7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 0x55fc48cc95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.546+0000 7faba7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 0x55fc48cc95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48588/0 (socket says 172.21.15.33:48588) 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.546+0000 7faba7fff640 1 -- 172.21.15.33:0/547870613 learned_addr learned my addr 172.21.15.33:0/547870613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.546+0000 7faba7fff640 1 -- 172.21.15.33:0/547870613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 msgr2=0x55fc48cc0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.546+0000 7faba7fff640 1 --2- 172.21.15.33:0/547870613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 0x55fc48cc0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.546+0000 7faba7fff640 1 -- 172.21.15.33:0/547870613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cbd640 msgr2=0x55fc48cc1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.546+0000 7faba7fff640 1 --2- 172.21.15.33:0/547870613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cbd640 0x55fc48cc1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.546+0000 7faba7fff640 1 -- 172.21.15.33:0/547870613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc48cb88b0 con 0x55fc48cb84d0 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.546+0000 7faba7fff640 1 --2- 172.21.15.33:0/547870613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 0x55fc48cc95e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fab98009ca0 tx=0x7fab98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d3c32a2016386a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.546+0000 7fab9e7fc640 1 -- 172.21.15.33:0/547870613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98061070 con 0x55fc48cb84d0 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.546+0000 7fab9e7fc640 1 -- 172.21.15.33:0/547870613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fab98004720 con 0x55fc48cb84d0 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.546+0000 7fab9e7fc640 1 -- 172.21.15.33:0/547870613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98051070 con 0x55fc48cb84d0 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.547+0000 7fabb185b740 1 -- 172.21.15.33:0/547870613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 msgr2=0x55fc48cc95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.547+0000 7fabb185b740 1 --2- 172.21.15.33:0/547870613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 0x55fc48cc95e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fab98009ca0 tx=0x7fab98004a40 comp rx=0 tx=0).stop 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.547+0000 7fabb185b740 1 -- 172.21.15.33:0/547870613 shutdown_connections 2024-04-02T13:38:38.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.547+0000 7fabb185b740 1 --2- 172.21.15.33:0/547870613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cbd640 0x55fc48cc1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.547+0000 7fabb185b740 1 --2- 172.21.15.33:0/547870613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cb84d0 0x55fc48cc95e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.547+0000 7fabb185b740 1 --2- 172.21.15.33:0/547870613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb7ae0 0x55fc48cc0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.547+0000 7fabb185b740 1 -- 172.21.15.33:0/547870613 >> 172.21.15.33:0/547870613 conn(0x55fc48cb3050 msgr2=0x55fc48cb8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.547+0000 7fabb185b740 1 -- 172.21.15.33:0/547870613 shutdown_connections 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.547+0000 7fabb185b740 1 -- 172.21.15.33:0/547870613 wait complete. 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.547+0000 7fabb185b740 1 -- start start 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.547+0000 7fabb185b740 1 Processor -- start 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabb185b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cb7ae0 0x55fc48cc50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabb185b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb84d0 0x55fc48cc55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabb185b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cbd640 0x55fc48cc8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabb185b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fc48cd02d0 con 0x55fc48cbd640 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabb185b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fc48ccfc00 con 0x55fc48cb7ae0 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabb185b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fc48cd09a0 con 0x55fc48cb84d0 2024-04-02T13:38:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabb185b740 10 client.0 _ll_register_callbacks cb 0x7fab98037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabad214640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cbd640 0x55fc48cc8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabad214640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cbd640 0x55fc48cc8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48594/0 (socket says 172.21.15.33:48594) 2024-04-02T13:38:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabad214640 1 -- 172.21.15.33:0/647359041 learned_addr learned my addr 172.21.15.33:0/647359041 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7faba7fff640 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb84d0 0x55fc48cc55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabaca13640 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cb7ae0 0x55fc48cc50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabad214640 1 -- 172.21.15.33:0/647359041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb84d0 msgr2=0x55fc48cc55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabad214640 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb84d0 0x55fc48cc55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabad214640 1 -- 172.21.15.33:0/647359041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cb7ae0 msgr2=0x55fc48cc50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabad214640 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cb7ae0 0x55fc48cc50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabad214640 1 -- 172.21.15.33:0/647359041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc48cc8c00 con 0x55fc48cbd640 2024-04-02T13:38:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabaca13640 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cb7ae0 0x55fc48cc50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fabad214640 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cbd640 0x55fc48cc8610 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fab9009bb00 tx=0x7fab9009c200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa71f1fc93c6e483 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.548+0000 7fab87fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.550+0000 7fab87fff640 1 -- 172.21.15.33:0/647359041 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab900a0460 con 0x55fc48cbd640 2024-04-02T13:38:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.550+0000 7fab87fff640 1 -- 172.21.15.33:0/647359041 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fab90009c60 con 0x55fc48cbd640 2024-04-02T13:38:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.550+0000 7fabb185b740 1 -- 172.21.15.33:0/647359041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab98004720 con 0x55fc48cbd640 2024-04-02T13:38:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.550+0000 7fab87fff640 1 -- 172.21.15.33:0/647359041 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fab900a4d20 con 0x55fc48cbd640 2024-04-02T13:38:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.551+0000 7fabb185b740 10 client.15396 fetch_fsmap learned FSMap version 24 2024-04-02T13:38:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.551+0000 7fabb185b740 1 -- 172.21.15.33:0/647359041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fc48d35330 con 0x55fc48cbd640 2024-04-02T13:38:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.551+0000 7fab87fff640 1 -- 172.21.15.33:0/647359041 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fab98004720 con 0x55fc48cbd640 2024-04-02T13:38:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.551+0000 7fab87fff640 1 -- 172.21.15.33:0/647359041 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fab900a0090 con 0x55fc48cbd640 2024-04-02T13:38:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.551+0000 7fabb185b740 10 client.15396 fetch_fsmap finished waiting for FSMap version 24 2024-04-02T13:38:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.551+0000 7fabb185b740 10 client.15396 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-02T13:38:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.551+0000 7fabb185b740 20 client.15396 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.551+0000 7fabb185b740 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55fc48d39a80 0x55fc48d36590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.551+0000 7fabb185b740 4 client.15396 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.551+0000 7fabb185b740 1 -- 172.21.15.33:0/647359041 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "status"}) -- 0x55fc48c295a0 con 0x55fc48d39a80 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.551+0000 7fabaca13640 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55fc48d39a80 0x55fc48d36590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fab87fff640 10 client.15396 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fabaca13640 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55fc48d39a80 0x55fc48d36590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6f3282d4fdcb519 server_cookie=c1c01a99cf279419 in_seq=0 out_seq=0 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fab87fff640 1 -- 172.21.15.33:0/647359041 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fc48c295a0 con 0x55fc48d39a80 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fab87fff640 10 client.15396 handle_command_reply: tid=0 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fabb185b740 1 client.15396 shutdown 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fabb185b740 1 -- 172.21.15.33:0/647359041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cbd640 msgr2=0x55fc48cc8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fabb185b740 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cbd640 0x55fc48cc8610 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fab9009bb00 tx=0x7fab9009c200 comp rx=0 tx=0).stop 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fabb185b740 1 -- 172.21.15.33:0/647359041 shutdown_connections 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fabb185b740 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55fc48d39a80 0x55fc48d36590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fabb185b740 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc48cbd640 0x55fc48cc8610 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fabb185b740 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc48cb84d0 0x55fc48cc55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fab87fff640 0 client.15396 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:38:38.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fabb185b740 1 --2- 172.21.15.33:0/647359041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc48cb7ae0 0x55fc48cc50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fabb185b740 1 -- 172.21.15.33:0/647359041 >> 172.21.15.33:0/647359041 conn(0x55fc48cb3050 msgr2=0x55fc48cc0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fabb185b740 1 -- 172.21.15.33:0/647359041 shutdown_connections 2024-04-02T13:38:38.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fabb185b740 1 -- 172.21.15.33:0/647359041 wait complete. 2024-04-02T13:38:38.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.552+0000 7fabb185b740 20 client.15396 trim_cache size 0 max 16384 2024-04-02T13:38:38.560 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:38:38.560 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "b5fd982e-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:38:38.560 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-04-02T13:38:38.560 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24473, 2024-04-02T13:38:38.560 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-02T13:38:38.560 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-02T13:38:38.561 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-02T13:38:38.561 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 112.068575541, 2024-04-02T13:38:38.561 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-04-02T13:38:38.561 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 94, 2024-04-02T13:38:38.561 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:38:38.561 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 162.86411368099999 2024-04-02T13:38:38.561 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:38:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.553+0000 7fabaec77640 1 -- 172.21.15.33:0/2109521221 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fab940a3bb0 msgr2=0x7fab940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.553+0000 7fabaec77640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fab940a3bb0 0x7fab940a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fab98006010 tx=0x7fab98005f70 comp rx=0 tx=0).stop 2024-04-02T13:38:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.553+0000 7fabaec77640 1 -- 172.21.15.33:0/2109521221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba81036e0 msgr2=0x7faba819f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.553+0000 7fabaec77640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba81036e0 0x7faba819f3f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fab9000d750 tx=0x7fab9000dc10 comp rx=0 tx=0).stop 2024-04-02T13:38:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.553+0000 7fabaec77640 1 -- 172.21.15.33:0/2109521221 shutdown_connections 2024-04-02T13:38:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.553+0000 7fabaec77640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fab940a3bb0 0x7fab940a6070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.553+0000 7fabaec77640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faba81036e0 0x7faba819f3f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.553+0000 7fabaec77640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faba8102d20 0x7faba8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.553+0000 7fabaec77640 1 --2- 172.21.15.33:0/2109521221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faba8101b30 0x7faba8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.553+0000 7fabaec77640 1 -- 172.21.15.33:0/2109521221 >> 172.21.15.33:0/2109521221 conn(0x7faba80fd0a0 msgr2=0x7faba8106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.553+0000 7fabaec77640 1 -- 172.21.15.33:0/2109521221 shutdown_connections 2024-04-02T13:38:38.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.553+0000 7fabaec77640 1 -- 172.21.15.33:0/2109521221 wait complete. 2024-04-02T13:38:38.568 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-02T13:38:38.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.669+0000 7fd6a62d6640 1 -- 172.21.15.33:0/1897870982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6a0104ed0 msgr2=0x7fd6a01072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.669+0000 7fd6a62d6640 1 --2- 172.21.15.33:0/1897870982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6a0104ed0 0x7fd6a01072c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd690009900 tx=0x7fd69002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 -- 172.21.15.33:0/1897870982 shutdown_connections 2024-04-02T13:38:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 --2- 172.21.15.33:0/1897870982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0107800 0x7fd6a0109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 --2- 172.21.15.33:0/1897870982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6a0104ed0 0x7fd6a01072c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 --2- 172.21.15.33:0/1897870982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6a01025a0 0x7fd6a0104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 -- 172.21.15.33:0/1897870982 >> 172.21.15.33:0/1897870982 conn(0x7fd6a00fbfb0 msgr2=0x7fd6a00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 -- 172.21.15.33:0/1897870982 shutdown_connections 2024-04-02T13:38:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 -- 172.21.15.33:0/1897870982 wait complete. 2024-04-02T13:38:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 Processor -- start 2024-04-02T13:38:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 -- start start 2024-04-02T13:38:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6a01025a0 0x7fd6a0197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6a0104ed0 0x7fd6a01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0107800 0x7fd6a019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a010d780 con 0x7fd6a01025a0 2024-04-02T13:38:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a010d600 con 0x7fd6a0107800 2024-04-02T13:38:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.670+0000 7fd6a62d6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a010d900 con 0x7fd6a0104ed0 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6a01025a0 0x7fd6a0197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6a01025a0 0x7fd6a0197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48614/0 (socket says 172.21.15.33:48614) 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd69ffff640 1 -- 172.21.15.33:0/4250570793 learned_addr learned my addr 172.21.15.33:0/4250570793 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd69f7fe640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6a0104ed0 0x7fd6a01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd6a4873640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0107800 0x7fd6a019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd69ffff640 1 -- 172.21.15.33:0/4250570793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6a0104ed0 msgr2=0x7fd6a01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd69ffff640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6a0104ed0 0x7fd6a01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd69ffff640 1 -- 172.21.15.33:0/4250570793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0107800 msgr2=0x7fd6a019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd69ffff640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0107800 0x7fd6a019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd69ffff640 1 -- 172.21.15.33:0/4250570793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a019fab0 con 0x7fd6a01025a0 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd6a4873640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0107800 0x7fd6a019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd69ffff640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6a01025a0 0x7fd6a0197e90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd68800d770 tx=0x7fd68800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd69d7fa640 1 -- 172.21.15.33:0/4250570793 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688017070 con 0x7fd6a01025a0 2024-04-02T13:38:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd69d7fa640 1 -- 172.21.15.33:0/4250570793 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd6880054f0 con 0x7fd6a01025a0 2024-04-02T13:38:38.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd69d7fa640 1 -- 172.21.15.33:0/4250570793 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688004d60 con 0x7fd6a01025a0 2024-04-02T13:38:38.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd6a62d6640 1 -- 172.21.15.33:0/4250570793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a019fda0 con 0x7fd6a01025a0 2024-04-02T13:38:38.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.671+0000 7fd6a62d6640 1 -- 172.21.15.33:0/4250570793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a01a01b0 con 0x7fd6a01025a0 2024-04-02T13:38:38.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.673+0000 7fd69d7fa640 1 -- 172.21.15.33:0/4250570793 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd688004f00 con 0x7fd6a01025a0 2024-04-02T13:38:38.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd6a8f99740 1 Processor -- start 2024-04-02T13:38:38.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd6a8f99740 1 -- start start 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd6a8f99740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5b7ae0 0x55a90e5b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd6a8f99740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5bcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd6a8f99740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5bd640 0x55a90e5bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd6a8f99740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a90e51c060 con 0x55a90e5b84d0 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd6a8f99740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a90e4c4650 con 0x55a90e5bd640 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd6a8f99740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a90e518010 con 0x55a90e5b7ae0 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5bcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5bcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48622/0 (socket says 172.21.15.33:48622) 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd69ffff640 1 -- 172.21.15.33:0/3422885631 learned_addr learned my addr 172.21.15.33:0/3422885631 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd69d7fa640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd68c0a3bb0 0x7fd68c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd6a4873640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd68c0a3bb0 0x7fd68c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd69f7fe640 1 --2- 172.21.15.33:0/3422885631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5bd640 0x55a90e5bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd6a4873640 1 --2- 172.21.15.33:0/3422885631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5b7ae0 0x55a90e5b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd69d7fa640 1 -- 172.21.15.33:0/4250570793 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd6880d0290 con 0x7fd6a01025a0 2024-04-02T13:38:38.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd69ffff640 1 -- 172.21.15.33:0/3422885631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5b7ae0 msgr2=0x55a90e5b7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd69ffff640 1 --2- 172.21.15.33:0/3422885631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5b7ae0 0x55a90e5b7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd69ffff640 1 -- 172.21.15.33:0/3422885631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5bd640 msgr2=0x55a90e5bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd69ffff640 1 --2- 172.21.15.33:0/3422885631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5bd640 0x55a90e5bfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd69ffff640 1 -- 172.21.15.33:0/3422885631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90e2302d0 con 0x55a90e5b84d0 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.674+0000 7fd69f7fe640 1 --2- 172.21.15.33:0/3422885631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5bd640 0x55a90e5bfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.675+0000 7fd69ffff640 1 --2- 172.21.15.33:0/3422885631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5bcfd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd688098b50 tx=0x7fd688009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0e600e3750f730 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.675+0000 7fd6a4873640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd68c0a3bb0 0x7fd68c0a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55a90e5b8d30 tx=0x7fd698009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.675+0000 7fd6967fc640 1 -- 172.21.15.33:0/3422885631 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6880cde60 con 0x55a90e5b84d0 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.675+0000 7fd6967fc640 1 -- 172.21.15.33:0/3422885631 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd688009cc0 con 0x55a90e5b84d0 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.675+0000 7fd6967fc640 1 -- 172.21.15.33:0/3422885631 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688020070 con 0x55a90e5b84d0 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.675+0000 7fd6a8f99740 1 -- 172.21.15.33:0/3422885631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 msgr2=0x55a90e5bcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.675+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/3422885631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5bcfd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd688098b50 tx=0x7fd688009c30 comp rx=0 tx=0).stop 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.675+0000 7fd6a8f99740 1 -- 172.21.15.33:0/3422885631 shutdown_connections 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.675+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/3422885631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5bd640 0x55a90e5bfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.675+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/3422885631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5bcfd0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.675+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/3422885631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5b7ae0 0x55a90e5b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.675+0000 7fd6a8f99740 1 -- 172.21.15.33:0/3422885631 >> 172.21.15.33:0/3422885631 conn(0x55a90e5b3050 msgr2=0x55a90e5b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd6a8f99740 1 -- 172.21.15.33:0/3422885631 shutdown_connections 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd6a8f99740 1 -- 172.21.15.33:0/3422885631 wait complete. 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd6a8f99740 1 -- start start 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd6a8f99740 1 Processor -- start 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd6a8f99740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b7ae0 0x55a90e5c2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd6a8f99740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd6a8f99740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5bd640 0x55a90e5c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd6a8f99740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a90e5c0590 con 0x55a90e5b84d0 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd6a8f99740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a90e4d7a10 con 0x55a90e5b7ae0 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd6a8f99740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a90e5c0710 con 0x55a90e5bd640 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd6a8f99740 10 client.0 _ll_register_callbacks cb 0x55a90e5140b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd69f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd69f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5c2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48632/0 (socket says 172.21.15.33:48632) 2024-04-02T13:38:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd69f7fe640 1 -- 172.21.15.33:0/2203497162 learned_addr learned my addr 172.21.15.33:0/2203497162 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.676+0000 7fd69ffff640 1 --2- 172.21.15.33:0/2203497162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b7ae0 0x55a90e5c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd6a4873640 1 --2- 172.21.15.33:0/2203497162 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5bd640 0x55a90e5c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd69f7fe640 1 -- 172.21.15.33:0/2203497162 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5bd640 msgr2=0x55a90e5c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd69f7fe640 1 --2- 172.21.15.33:0/2203497162 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5bd640 0x55a90e5c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd69f7fe640 1 -- 172.21.15.33:0/2203497162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b7ae0 msgr2=0x55a90e5c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd69f7fe640 1 --2- 172.21.15.33:0/2203497162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b7ae0 0x55a90e5c2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd69f7fe640 1 -- 172.21.15.33:0/2203497162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a90e2fa6b0 con 0x55a90e5b84d0 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd69f7fe640 1 --2- 172.21.15.33:0/2203497162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5c2540 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd69002c880 tx=0x7fd690004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca698e93067bcf9b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd67f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd67f7fe640 1 -- 172.21.15.33:0/2203497162 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd690005d50 con 0x55a90e5b84d0 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd67f7fe640 1 -- 172.21.15.33:0/2203497162 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd690005030 con 0x55a90e5b84d0 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd6a8f99740 1 -- 172.21.15.33:0/2203497162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a90e4c3cf0 con 0x55a90e5b84d0 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd67f7fe640 1 -- 172.21.15.33:0/2203497162 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd69004f050 con 0x55a90e5b84d0 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd6a8f99740 10 client.15408 fetch_fsmap learned FSMap version 24 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd6a8f99740 1 -- 172.21.15.33:0/2203497162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a90e5cc960 con 0x55a90e5b84d0 2024-04-02T13:38:38.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.677+0000 7fd67f7fe640 1 -- 172.21.15.33:0/2203497162 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55a90e4c3cf0 con 0x55a90e5b84d0 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.678+0000 7fd67f7fe640 1 -- 172.21.15.33:0/2203497162 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd6900035f0 con 0x55a90e5b84d0 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.678+0000 7fd6a8f99740 10 client.15408 fetch_fsmap finished waiting for FSMap version 24 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.678+0000 7fd6a8f99740 10 client.15408 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.678+0000 7fd6a8f99740 20 client.15408 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.678+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/2203497162 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55a90e5cf030 0x55a90e5d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.678+0000 7fd6a8f99740 4 client.15408 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.678+0000 7fd6a8f99740 1 -- 172.21.15.33:0/2203497162 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a90e5295a0 con 0x55a90e5cf030 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.678+0000 7fd69ffff640 1 --2- 172.21.15.33:0/2203497162 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55a90e5cf030 0x55a90e5d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.678+0000 7fd69ffff640 1 --2- 172.21.15.33:0/2203497162 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55a90e5cf030 0x55a90e5d04a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d1108dca675b02f9 server_cookie=23a8cc4caf56db7d in_seq=0 out_seq=0 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.678+0000 7fd67f7fe640 10 client.15408 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.679+0000 7fd67f7fe640 1 -- 172.21.15.33:0/2203497162 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a90e5295a0 con 0x55a90e5cf030 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.679+0000 7fd67f7fe640 10 client.15408 handle_command_reply: tid=0 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.679+0000 7fd6a8f99740 1 client.15408 shutdown 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.679+0000 7fd6a8f99740 1 -- 172.21.15.33:0/2203497162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 msgr2=0x55a90e5c2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.679+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/2203497162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5c2540 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd69002c880 tx=0x7fd690004270 comp rx=0 tx=0).stop 2024-04-02T13:38:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.679+0000 7fd6a8f99740 1 -- 172.21.15.33:0/2203497162 shutdown_connections 2024-04-02T13:38:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.679+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/2203497162 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55a90e5cf030 0x55a90e5d04a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.679+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/2203497162 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5bd640 0x55a90e5c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.679+0000 7fd67f7fe640 0 client.15408 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:38:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.679+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/2203497162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5c2540 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.679+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/2203497162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b7ae0 0x55a90e5c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.679+0000 7fd6a8f99740 1 -- 172.21.15.33:0/2203497162 >> 172.21.15.33:0/2203497162 conn(0x55a90e5b3050 msgr2=0x55a90e5b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.680+0000 7fd6a8f99740 1 -- 172.21.15.33:0/2203497162 shutdown_connections 2024-04-02T13:38:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.680+0000 7fd6a8f99740 1 -- 172.21.15.33:0/2203497162 wait complete. 2024-04-02T13:38:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.680+0000 7fd6a8f99740 20 client.15408 trim_cache size 0 max 16384 2024-04-02T13:38:38.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd6a8f99740 1 Processor -- start 2024-04-02T13:38:38.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd6a8f99740 1 -- start start 2024-04-02T13:38:38.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd6a8f99740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b7ae0 0x55a90e5b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd6a8f99740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd6a8f99740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5bd640 0x55a90e5c2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd6a8f99740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a90e5ce7e0 con 0x55a90e5b84d0 2024-04-02T13:38:38.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd6a8f99740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a90e5ceeb0 con 0x55a90e5b7ae0 2024-04-02T13:38:38.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd6a8f99740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a90e5cf580 con 0x55a90e5bd640 2024-04-02T13:38:38.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd69f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd69f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48646/0 (socket says 172.21.15.33:48646) 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd69f7fe640 1 -- 172.21.15.33:0/1679408150 learned_addr learned my addr 172.21.15.33:0/1679408150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd6a4873640 1 --2- 172.21.15.33:0/1679408150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5bd640 0x55a90e5c2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd69ffff640 1 --2- 172.21.15.33:0/1679408150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b7ae0 0x55a90e5b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd69f7fe640 1 -- 172.21.15.33:0/1679408150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5bd640 msgr2=0x55a90e5c2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd69f7fe640 1 --2- 172.21.15.33:0/1679408150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5bd640 0x55a90e5c2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd69f7fe640 1 -- 172.21.15.33:0/1679408150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b7ae0 msgr2=0x55a90e5b8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd69f7fe640 1 --2- 172.21.15.33:0/1679408150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b7ae0 0x55a90e5b8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd69f7fe640 1 -- 172.21.15.33:0/1679408150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90e5c2ea0 con 0x55a90e5b84d0 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.698+0000 7fd69ffff640 1 --2- 172.21.15.33:0/1679408150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b7ae0 0x55a90e5b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.699+0000 7fd69f7fe640 1 --2- 172.21.15.33:0/1679408150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5c1c10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd690003bd0 tx=0x7fd69002ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c6c4d94a31a678 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.699+0000 7fd695ffb640 1 -- 172.21.15.33:0/1679408150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd690058070 con 0x55a90e5b84d0 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.699+0000 7fd695ffb640 1 -- 172.21.15.33:0/1679408150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd690004b90 con 0x55a90e5b84d0 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.699+0000 7fd695ffb640 1 -- 172.21.15.33:0/1679408150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd690004f10 con 0x55a90e5b84d0 2024-04-02T13:38:38.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.699+0000 7fd6a8f99740 1 -- 172.21.15.33:0/1679408150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 msgr2=0x55a90e5c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.699+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/1679408150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5c1c10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd690003bd0 tx=0x7fd69002ccc0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.699+0000 7fd6a8f99740 1 -- 172.21.15.33:0/1679408150 shutdown_connections 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.699+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/1679408150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5bd640 0x55a90e5c2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.699+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/1679408150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5b84d0 0x55a90e5c1c10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.699+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/1679408150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b7ae0 0x55a90e5b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.699+0000 7fd6a8f99740 1 -- 172.21.15.33:0/1679408150 >> 172.21.15.33:0/1679408150 conn(0x55a90e5b3050 msgr2=0x55a90e5b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a8f99740 1 -- 172.21.15.33:0/1679408150 shutdown_connections 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a8f99740 1 -- 172.21.15.33:0/1679408150 wait complete. 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a8f99740 1 -- start start 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a8f99740 1 Processor -- start 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a8f99740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5b7ae0 0x55a90e631230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a8f99740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b84d0 0x55a90e631770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a8f99740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5bd640 0x55a90e638200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a8f99740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a90e5d0320 con 0x55a90e5bd640 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a8f99740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a90e5cfc50 con 0x55a90e5b84d0 2024-04-02T13:38:38.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a8f99740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a90e5d09f0 con 0x55a90e5b7ae0 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a8f99740 10 client.0 _ll_register_callbacks cb 0x7fd69002c690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a4873640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5bd640 0x55a90e638200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a4873640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5bd640 0x55a90e638200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48648/0 (socket says 172.21.15.33:48648) 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.700+0000 7fd6a4873640 1 -- 172.21.15.33:0/23404839 learned_addr learned my addr 172.21.15.33:0/23404839 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd69f7fe640 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b84d0 0x55a90e631770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd6a4873640 1 -- 172.21.15.33:0/23404839 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5b7ae0 msgr2=0x55a90e631230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd69ffff640 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5b7ae0 0x55a90e631230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd6a4873640 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5b7ae0 0x55a90e631230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd6a4873640 1 -- 172.21.15.33:0/23404839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b84d0 msgr2=0x55a90e631770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd6a4873640 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b84d0 0x55a90e631770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd6a4873640 1 -- 172.21.15.33:0/23404839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a90e638810 con 0x55a90e5bd640 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd69f7fe640 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b84d0 0x55a90e631770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd69ffff640 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5b7ae0 0x55a90e631230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:38.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd6a4873640 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5bd640 0x55a90e638200 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd698010af0 tx=0x7fd698004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69097616345c134 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd67f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd67f7fe640 1 -- 172.21.15.33:0/23404839 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698004b20 con 0x55a90e5bd640 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd67f7fe640 1 -- 172.21.15.33:0/23404839 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd69801a070 con 0x55a90e5bd640 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd6a8f99740 1 -- 172.21.15.33:0/23404839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd690004b90 con 0x55a90e5bd640 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd67f7fe640 1 -- 172.21.15.33:0/23404839 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd698034050 con 0x55a90e5bd640 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd6a8f99740 10 client.15414 fetch_fsmap learned FSMap version 24 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd6a8f99740 1 -- 172.21.15.33:0/23404839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a90e639b80 con 0x55a90e5bd640 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd67f7fe640 1 -- 172.21.15.33:0/23404839 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd690004b90 con 0x55a90e5bd640 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd67f7fe640 1 -- 172.21.15.33:0/23404839 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd698028070 con 0x55a90e5bd640 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.701+0000 7fd6a8f99740 10 client.15414 fetch_fsmap finished waiting for FSMap version 24 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.702+0000 7fd6a8f99740 10 client.15414 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.702+0000 7fd6a8f99740 20 client.15414 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.702+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55a90e63e890 0x55a90e63d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.702+0000 7fd6a8f99740 4 client.15414 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.702+0000 7fd6a8f99740 1 -- 172.21.15.33:0/23404839 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "status"}) -- 0x55a90e5238f0 con 0x55a90e63e890 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.702+0000 7fd69ffff640 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55a90e63e890 0x55a90e63d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.702+0000 7fd67f7fe640 10 client.15414 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.702+0000 7fd69ffff640 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55a90e63e890 0x55a90e63d5a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e39c8934e5794248 server_cookie=49982c1fa49231b1 in_seq=0 out_seq=0 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd67f7fe640 1 -- 172.21.15.33:0/23404839 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a90e5238f0 con 0x55a90e63e890 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd67f7fe640 10 client.15414 handle_command_reply: tid=0 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a8f99740 1 client.15414 shutdown 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a8f99740 1 -- 172.21.15.33:0/23404839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5bd640 msgr2=0x55a90e638200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5bd640 0x55a90e638200 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd698010af0 tx=0x7fd698004a40 comp rx=0 tx=0).stop 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a8f99740 1 -- 172.21.15.33:0/23404839 shutdown_connections 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a90e5bd640 0x55a90e638200 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a90e5b84d0 0x55a90e631770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55a90e63e890 0x55a90e63d5a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a8f99740 1 --2- 172.21.15.33:0/23404839 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a90e5b7ae0 0x55a90e631230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd67f7fe640 0 client.15414 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:38:38.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a8f99740 1 -- 172.21.15.33:0/23404839 >> 172.21.15.33:0/23404839 conn(0x55a90e5b3050 msgr2=0x55a90e5b8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a8f99740 1 -- 172.21.15.33:0/23404839 shutdown_connections 2024-04-02T13:38:38.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a8f99740 1 -- 172.21.15.33:0/23404839 wait complete. 2024-04-02T13:38:38.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a8f99740 20 client.15414 trim_cache size 0 max 16384 2024-04-02T13:38:38.713 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:38:38.713 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "b5fd982e-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:38:38.713 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-04-02T13:38:38.713 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24512, 2024-04-02T13:38:38.713 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-02T13:38:38.713 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-02T13:38:38.714 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-02T13:38:38.714 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 113.167671949, 2024-04-02T13:38:38.714 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-04-02T13:38:38.714 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 94, 2024-04-02T13:38:38.714 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:38:38.714 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 154.49084501600001 2024-04-02T13:38:38.714 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:38:38.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a62d6640 1 -- 172.21.15.33:0/4250570793 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd68c0a3bb0 msgr2=0x7fd68c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a62d6640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd68c0a3bb0 0x7fd68c0a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55a90e5b8d30 tx=0x7fd698009590 comp rx=0 tx=0).stop 2024-04-02T13:38:38.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a62d6640 1 -- 172.21.15.33:0/4250570793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6a01025a0 msgr2=0x7fd6a0197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a62d6640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6a01025a0 0x7fd6a0197e90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd68800d770 tx=0x7fd68800dc30 comp rx=0 tx=0).stop 2024-04-02T13:38:38.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a62d6640 1 -- 172.21.15.33:0/4250570793 shutdown_connections 2024-04-02T13:38:38.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a62d6640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fd68c0a3bb0 0x7fd68c0a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a62d6640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd6a0107800 0x7fd6a019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a62d6640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6a0104ed0 0x7fd6a01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.703+0000 7fd6a62d6640 1 --2- 172.21.15.33:0/4250570793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6a01025a0 0x7fd6a0197e90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.704+0000 7fd6a62d6640 1 -- 172.21.15.33:0/4250570793 >> 172.21.15.33:0/4250570793 conn(0x7fd6a00fbfb0 msgr2=0x7fd6a0103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.704+0000 7fd6a62d6640 1 -- 172.21.15.33:0/4250570793 shutdown_connections 2024-04-02T13:38:38.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.704+0000 7fd6a62d6640 1 -- 172.21.15.33:0/4250570793 wait complete. 2024-04-02T13:38:38.717 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:38 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1420943643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:38.717 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:38 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2836158947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:38.717 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:38 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2337334686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:38.718 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T13:38:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.818+0000 7fc4b0f9d640 1 -- 172.21.15.33:0/1522623515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac1013a0 msgr2=0x7fc4ac101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.818+0000 7fc4b0f9d640 1 --2- 172.21.15.33:0/1522623515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac1013a0 0x7fc4ac101820 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009900 tx=0x7fc4a002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.818+0000 7fc4b0f9d640 1 -- 172.21.15.33:0/1522623515 shutdown_connections 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.818+0000 7fc4b0f9d640 1 --2- 172.21.15.33:0/1522623515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ac101d60 0x7fc4ac1085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.818+0000 7fc4b0f9d640 1 --2- 172.21.15.33:0/1522623515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac1013a0 0x7fc4ac101820 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.818+0000 7fc4b0f9d640 1 --2- 172.21.15.33:0/1522623515 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ac1001b0 0x7fc4ac1005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.818+0000 7fc4b0f9d640 1 -- 172.21.15.33:0/1522623515 >> 172.21.15.33:0/1522623515 conn(0x7fc4ac0fb700 msgr2=0x7fc4ac0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.818+0000 7fc4b0f9d640 1 -- 172.21.15.33:0/1522623515 shutdown_connections 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.818+0000 7fc4b0f9d640 1 -- 172.21.15.33:0/1522623515 wait complete. 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.819+0000 7fc4b0f9d640 1 Processor -- start 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.819+0000 7fc4b0f9d640 1 -- start start 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.819+0000 7fc4b0f9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ac1001b0 0x7fc4ac109650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.819+0000 7fc4b0f9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ac1013a0 0x7fc4ac19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.819+0000 7fc4b0f9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac101d60 0x7fc4ac19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.819+0000 7fc4b0f9d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac10c6a0 con 0x7fc4ac101d60 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.819+0000 7fc4b0f9d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac10c520 con 0x7fc4ac1001b0 2024-04-02T13:38:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.819+0000 7fc4b0f9d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac10c820 con 0x7fc4ac1013a0 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.819+0000 7fc4aad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac101d60 0x7fc4ac19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.819+0000 7fc4aad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac101d60 0x7fc4ac19f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48666/0 (socket says 172.21.15.33:48666) 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.819+0000 7fc4aad9d640 1 -- 172.21.15.33:0/1634649549 learned_addr learned my addr 172.21.15.33:0/1634649549 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.819+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ac1001b0 0x7fc4ac109650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.819+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ac1013a0 0x7fc4ac19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.820+0000 7fc4aad9d640 1 -- 172.21.15.33:0/1634649549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ac1013a0 msgr2=0x7fc4ac19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.820+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ac1013a0 0x7fc4ac19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.820+0000 7fc4aad9d640 1 -- 172.21.15.33:0/1634649549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ac1001b0 msgr2=0x7fc4ac109650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.820+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ac1001b0 0x7fc4ac109650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.820+0000 7fc4aad9d640 1 -- 172.21.15.33:0/1634649549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ac19fab0 con 0x7fc4ac101d60 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.820+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ac1001b0 0x7fc4ac109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.820+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ac1013a0 0x7fc4ac19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.820+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac101d60 0x7fc4ac19f1f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400e970 tx=0x7fc4a400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.820+0000 7fc49b7fe640 1 -- 172.21.15.33:0/1634649549 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a400ccc0 con 0x7fc4ac101d60 2024-04-02T13:38:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.820+0000 7fc49b7fe640 1 -- 172.21.15.33:0/1634649549 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc4a400ce60 con 0x7fc4ac101d60 2024-04-02T13:38:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.820+0000 7fc4b0f9d640 1 -- 172.21.15.33:0/1634649549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ac19fda0 con 0x7fc4ac101d60 2024-04-02T13:38:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.820+0000 7fc49b7fe640 1 -- 172.21.15.33:0/1634649549 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4010430 con 0x7fc4ac101d60 2024-04-02T13:38:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.820+0000 7fc4b0f9d640 1 -- 172.21.15.33:0/1634649549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ac1a0210 con 0x7fc4ac101d60 2024-04-02T13:38:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.822+0000 7fc49b7fe640 1 -- 172.21.15.33:0/1634649549 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc4a4005000 con 0x7fc4ac101d60 2024-04-02T13:38:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.822+0000 7fc4b3b87740 1 Processor -- start 2024-04-02T13:38:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4b3b87740 1 -- start start 2024-04-02T13:38:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4b3b87740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471eae0 0x556a4471ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4b3b87740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 0x556a44723fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4b3b87740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a44724640 0x556a44726ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4b3b87740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a44683060 con 0x556a4471f4d0 2024-04-02T13:38:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4b3b87740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556a4462b650 con 0x556a44724640 2024-04-02T13:38:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4b3b87740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556a4467f010 con 0x556a4471eae0 2024-04-02T13:38:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 0x556a44723fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 0x556a44723fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48676/0 (socket says 172.21.15.33:48676) 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4aa59c640 1 -- 172.21.15.33:0/540340967 learned_addr learned my addr 172.21.15.33:0/540340967 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc49b7fe640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc4940a3bb0 0x7fc4940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/540340967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471eae0 0x556a4471ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc4940a3bb0 0x7fc4940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/540340967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a44724640 0x556a44726ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4aa59c640 1 -- 172.21.15.33:0/540340967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471eae0 msgr2=0x556a4471ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/540340967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471eae0 0x556a4471ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4aa59c640 1 -- 172.21.15.33:0/540340967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a44724640 msgr2=0x556a44726ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/540340967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a44724640 0x556a44726ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc49b7fe640 1 -- 172.21.15.33:0/1634649549 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc4a40d0cb0 con 0x7fc4ac101d60 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4aa59c640 1 -- 172.21.15.33:0/540340967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a443972d0 con 0x556a4471f4d0 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.823+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/540340967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 0x556a44723fd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00aef0 tx=0x7fc49c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51b17b276f89c2e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.824+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc4940a3bb0 0x7fc4940a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40995d0 tx=0x7fc4a406c530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.824+0000 7fc498ff9640 1 -- 172.21.15.33:0/540340967 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c011850 con 0x556a4471f4d0 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.824+0000 7fc498ff9640 1 -- 172.21.15.33:0/540340967 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc49c01a480 con 0x556a4471f4d0 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.824+0000 7fc498ff9640 1 -- 172.21.15.33:0/540340967 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c010630 con 0x556a4471f4d0 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.824+0000 7fc4b3b87740 1 -- 172.21.15.33:0/540340967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 msgr2=0x556a44723fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.824+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/540340967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 0x556a44723fd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00aef0 tx=0x7fc49c00f730 comp rx=0 tx=0).stop 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.824+0000 7fc4b3b87740 1 -- 172.21.15.33:0/540340967 shutdown_connections 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.824+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/540340967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a44724640 0x556a44726ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.824+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/540340967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 0x556a44723fd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.824+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/540340967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471eae0 0x556a4471ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.824+0000 7fc4b3b87740 1 -- 172.21.15.33:0/540340967 >> 172.21.15.33:0/540340967 conn(0x556a4471a050 msgr2=0x556a4471a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.824+0000 7fc4b3b87740 1 -- 172.21.15.33:0/540340967 shutdown_connections 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.824+0000 7fc4b3b87740 1 -- 172.21.15.33:0/540340967 wait complete. 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4b3b87740 1 -- start start 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4b3b87740 1 Processor -- start 2024-04-02T13:38:38.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4b3b87740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a4471eae0 0x556a44728ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4b3b87740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 0x556a44729530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4b3b87740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a44724640 0x556a44730570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4b3b87740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a447275f0 con 0x556a4471f4d0 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4b3b87740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556a4463ea10 con 0x556a4471eae0 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4b3b87740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556a44727770 con 0x556a44724640 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4b3b87740 10 client.0 _ll_register_callbacks cb 0x556a4467b0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 0x556a44729530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 0x556a44729530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48678/0 (socket says 172.21.15.33:48678) 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4aa59c640 1 -- 172.21.15.33:0/4196183420 learned_addr learned my addr 172.21.15.33:0/4196183420 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a4471eae0 0x556a44728ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a44724640 0x556a44730570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4aa59c640 1 -- 172.21.15.33:0/4196183420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a44724640 msgr2=0x556a44730570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a44724640 0x556a44730570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4aa59c640 1 -- 172.21.15.33:0/4196183420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a4471eae0 msgr2=0x556a44728ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a4471eae0 0x556a44728ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4aa59c640 1 -- 172.21.15.33:0/4196183420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a444616b0 con 0x556a4471f4d0 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.825+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a4471eae0 0x556a44728ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 0x556a44729530 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc49c005c40 tx=0x7fc49c01a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e53210dcdf31f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc491ffb640 1 -- 172.21.15.33:0/4196183420 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c021070 con 0x556a4471f4d0 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc491ffb640 1 -- 172.21.15.33:0/4196183420 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc49c003970 con 0x556a4471f4d0 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc4b3b87740 1 -- 172.21.15.33:0/4196183420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc49c01a480 con 0x556a4471f4d0 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc491ffb640 1 -- 172.21.15.33:0/4196183420 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc49c02f050 con 0x556a4471f4d0 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc4b3b87740 10 client.15426 fetch_fsmap learned FSMap version 24 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc4b3b87740 1 -- 172.21.15.33:0/4196183420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556a44733910 con 0x556a4471f4d0 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc491ffb640 1 -- 172.21.15.33:0/4196183420 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc49c01a480 con 0x556a4471f4d0 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc491ffb640 1 -- 172.21.15.33:0/4196183420 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc49c0046c0 con 0x556a4471f4d0 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc4b3b87740 10 client.15426 fetch_fsmap finished waiting for FSMap version 24 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc4b3b87740 10 client.15426 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc4b3b87740 20 client.15426 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556a44735bb0 0x556a44737020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc4b3b87740 4 client.15426 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.826+0000 7fc4b3b87740 1 -- 172.21.15.33:0/4196183420 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a4462acf0 con 0x556a44735bb0 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.827+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556a44735bb0 0x556a44737020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.827+0000 7fc491ffb640 10 client.15426 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.827+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556a44735bb0 0x556a44737020 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea8f93d5465f929 server_cookie=28fb6e908242949b in_seq=0 out_seq=0 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc491ffb640 1 -- 172.21.15.33:0/4196183420 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a4462acf0 con 0x556a44735bb0 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc491ffb640 10 client.15426 handle_command_reply: tid=0 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc4b3b87740 1 client.15426 shutdown 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc4b3b87740 1 -- 172.21.15.33:0/4196183420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 msgr2=0x556a44729530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 0x556a44729530 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc49c005c40 tx=0x7fc49c01a8e0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc4b3b87740 1 -- 172.21.15.33:0/4196183420 shutdown_connections 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556a44735bb0 0x556a44737020 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a44724640 0x556a44730570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471f4d0 0x556a44729530 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc491ffb640 0 client.15426 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/4196183420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a4471eae0 0x556a44728ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc4b3b87740 1 -- 172.21.15.33:0/4196183420 >> 172.21.15.33:0/4196183420 conn(0x556a4471a050 msgr2=0x556a44719190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc4b3b87740 1 -- 172.21.15.33:0/4196183420 shutdown_connections 2024-04-02T13:38:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc4b3b87740 1 -- 172.21.15.33:0/4196183420 wait complete. 2024-04-02T13:38:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.828+0000 7fc4b3b87740 20 client.15426 trim_cache size 0 max 16384 2024-04-02T13:38:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.846+0000 7fc4b3b87740 1 Processor -- start 2024-04-02T13:38:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.846+0000 7fc4b3b87740 1 -- start start 2024-04-02T13:38:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.846+0000 7fc4b3b87740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471eae0 0x556a44723fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.846+0000 7fc4b3b87740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471f4d0 0x556a44730530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.846+0000 7fc4b3b87740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a44724640 0x556a447209d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.846+0000 7fc4b3b87740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a44735360 con 0x556a4471eae0 2024-04-02T13:38:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.846+0000 7fc4b3b87740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556a44735a30 con 0x556a44724640 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.846+0000 7fc4b3b87740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556a44736100 con 0x556a4471f4d0 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.846+0000 7fc4a9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471eae0 0x556a44723fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.846+0000 7fc4a9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471eae0 0x556a44723fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48694/0 (socket says 172.21.15.33:48694) 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.846+0000 7fc4a9d9b640 1 -- 172.21.15.33:0/466174119 learned_addr learned my addr 172.21.15.33:0/466174119 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.846+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/466174119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471f4d0 0x556a44730530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.846+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/466174119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a44724640 0x556a447209d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.847+0000 7fc4a9d9b640 1 -- 172.21.15.33:0/466174119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471f4d0 msgr2=0x556a44730530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.847+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/466174119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471f4d0 0x556a44730530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.847+0000 7fc4a9d9b640 1 -- 172.21.15.33:0/466174119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a44724640 msgr2=0x556a447209d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.847+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/466174119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a44724640 0x556a447209d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.847+0000 7fc4a9d9b640 1 -- 172.21.15.33:0/466174119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a4471f8b0 con 0x556a4471eae0 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.847+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/466174119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471eae0 0x556a44723fc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006010 tx=0x7fc4a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6953ff306e193e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.847+0000 7fc493fff640 1 -- 172.21.15.33:0/466174119 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0050070 con 0x556a4471eae0 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.847+0000 7fc493fff640 1 -- 172.21.15.33:0/466174119 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc4a004add0 con 0x556a4471eae0 2024-04-02T13:38:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.847+0000 7fc493fff640 1 -- 172.21.15.33:0/466174119 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a004a4c0 con 0x556a4471eae0 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.847+0000 7fc4b3b87740 1 -- 172.21.15.33:0/466174119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471eae0 msgr2=0x556a44723fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.847+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/466174119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471eae0 0x556a44723fc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006010 tx=0x7fc4a0000bc0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.847+0000 7fc4b3b87740 1 -- 172.21.15.33:0/466174119 shutdown_connections 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/466174119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a44724640 0x556a447209d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/466174119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471f4d0 0x556a44730530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/466174119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a4471eae0 0x556a44723fc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 -- 172.21.15.33:0/466174119 >> 172.21.15.33:0/466174119 conn(0x556a4471a050 msgr2=0x556a4471a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 -- 172.21.15.33:0/466174119 shutdown_connections 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 -- 172.21.15.33:0/466174119 wait complete. 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 -- start start 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 Processor -- start 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471eae0 0x556a4472be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a4471f4d0 0x556a4472c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a44724640 0x556a4472f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a44736ea0 con 0x556a44724640 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556a447367d0 con 0x556a4471f4d0 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556a44737570 con 0x556a4471eae0 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4b3b87740 10 client.0 _ll_register_callbacks cb 0x7fc49c00ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4a9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a44724640 0x556a4472f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4a9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a44724640 0x556a4472f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48698/0 (socket says 172.21.15.33:48698) 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4a9d9b640 1 -- 172.21.15.33:0/1353720951 learned_addr learned my addr 172.21.15.33:0/1353720951 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/1353720951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a4471f4d0 0x556a4472c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/1353720951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471eae0 0x556a4472be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4a9d9b640 1 -- 172.21.15.33:0/1353720951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471eae0 msgr2=0x556a4472be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/1353720951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471eae0 0x556a4472be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4a9d9b640 1 -- 172.21.15.33:0/1353720951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a4471f4d0 msgr2=0x556a4472c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/1353720951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a4471f4d0 0x556a4472c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.848+0000 7fc4a9d9b640 1 -- 172.21.15.33:0/1353720951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a4472f8f0 con 0x556a44724640 2024-04-02T13:38:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc4a9d9b640 1 --2- 172.21.15.33:0/1353720951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a44724640 0x556a4472f300 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009230 tx=0x7fc4a0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d63963f74ac4ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc491ffb640 1 -- 172.21.15.33:0/1353720951 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a005c060 con 0x556a44724640 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc491ffb640 1 -- 172.21.15.33:0/1353720951 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fc4a00052c0 con 0x556a44724640 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc4b3b87740 1 -- 172.21.15.33:0/1353720951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a004add0 con 0x556a44724640 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc491ffb640 1 -- 172.21.15.33:0/1353720951 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc4a0063050 con 0x556a44724640 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc4b3b87740 10 client.15432 fetch_fsmap learned FSMap version 24 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc4b3b87740 1 -- 172.21.15.33:0/1353720951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556a4479bc20 con 0x556a44724640 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc491ffb640 1 -- 172.21.15.33:0/1353720951 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc4a004add0 con 0x556a44724640 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc491ffb640 1 -- 172.21.15.33:0/1353720951 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc4a0056070 con 0x556a44724640 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc4b3b87740 10 client.15432 fetch_fsmap finished waiting for FSMap version 24 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc4b3b87740 10 client.15432 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc4b3b87740 20 client.15432 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/1353720951 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556a447a0220 0x556a4479cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc4b3b87740 4 client.15432 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:38:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.849+0000 7fc4b3b87740 1 -- 172.21.15.33:0/1353720951 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "status"}) -- 0x556a446905a0 con 0x556a447a0220 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/1353720951 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556a447a0220 0x556a4479cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc491ffb640 10 client.15432 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/1353720951 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556a447a0220 0x556a4479cd30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5620b0d79ff2c3bc server_cookie=64645d0f60a3428e in_seq=0 out_seq=0 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc491ffb640 1 -- 172.21.15.33:0/1353720951 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556a446905a0 con 0x556a447a0220 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc491ffb640 10 client.15432 handle_command_reply: tid=0 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4b3b87740 1 client.15432 shutdown 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4b3b87740 1 -- 172.21.15.33:0/1353720951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a44724640 msgr2=0x556a4472f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/1353720951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a44724640 0x556a4472f300 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009230 tx=0x7fc4a0005850 comp rx=0 tx=0).stop 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4b3b87740 1 -- 172.21.15.33:0/1353720951 shutdown_connections 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/1353720951 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556a447a0220 0x556a4479cd30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/1353720951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a44724640 0x556a4472f300 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/1353720951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556a4471f4d0 0x556a4472c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc491ffb640 0 client.15432 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4b3b87740 1 --2- 172.21.15.33:0/1353720951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556a4471eae0 0x556a4472be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4b3b87740 1 -- 172.21.15.33:0/1353720951 >> 172.21.15.33:0/1353720951 conn(0x556a4471a050 msgr2=0x556a44723e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4b3b87740 1 -- 172.21.15.33:0/1353720951 shutdown_connections 2024-04-02T13:38:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4b3b87740 1 -- 172.21.15.33:0/1353720951 wait complete. 2024-04-02T13:38:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.850+0000 7fc4b3b87740 20 client.15432 trim_cache size 0 max 16384 2024-04-02T13:38:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:38:38.861 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "b5fd982e-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:38:38.861 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-04-02T13:38:38.861 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24551, 2024-04-02T13:38:38.861 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-02T13:38:38.861 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-02T13:38:38.862 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-02T13:38:38.862 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 137.01228483599999, 2024-04-02T13:38:38.862 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-04-02T13:38:38.862 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 94, 2024-04-02T13:38:38.862 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:38:38.862 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 143.19085997400001 2024-04-02T13:38:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:38:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.851+0000 7fc4b0f9d640 1 -- 172.21.15.33:0/1634649549 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc4940a3bb0 msgr2=0x7fc4940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.851+0000 7fc4b0f9d640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc4940a3bb0 0x7fc4940a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40995d0 tx=0x7fc4a406c530 comp rx=0 tx=0).stop 2024-04-02T13:38:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.851+0000 7fc4b0f9d640 1 -- 172.21.15.33:0/1634649549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac101d60 msgr2=0x7fc4ac19f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.851+0000 7fc4b0f9d640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac101d60 0x7fc4ac19f1f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400e970 tx=0x7fc4a400ed70 comp rx=0 tx=0).stop 2024-04-02T13:38:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.851+0000 7fc4b0f9d640 1 -- 172.21.15.33:0/1634649549 shutdown_connections 2024-04-02T13:38:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.851+0000 7fc4b0f9d640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fc4940a3bb0 0x7fc4940a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.851+0000 7fc4b0f9d640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac101d60 0x7fc4ac19f1f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.851+0000 7fc4b0f9d640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ac1013a0 0x7fc4ac19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.851+0000 7fc4b0f9d640 1 --2- 172.21.15.33:0/1634649549 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ac1001b0 0x7fc4ac109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.851+0000 7fc4b0f9d640 1 -- 172.21.15.33:0/1634649549 >> 172.21.15.33:0/1634649549 conn(0x7fc4ac0fb700 msgr2=0x7fc4ac1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.851+0000 7fc4b0f9d640 1 -- 172.21.15.33:0/1634649549 shutdown_connections 2024-04-02T13:38:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.851+0000 7fc4b0f9d640 1 -- 172.21.15.33:0/1634649549 wait complete. 2024-04-02T13:38:38.865 INFO:teuthology.run_tasks:Running task sequential... 2024-04-02T13:38:38.876 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-02T13:38:38.876 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-02T13:38:38.876 INFO:teuthology.task.print:Enabling mgr modules 2024-04-02T13:38:38.876 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-02T13:38:38.877 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T13:38:38.877 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-02T13:38:38.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-02T13:38:38.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:38 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1420943643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:38 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2836158947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:38 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2337334686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.976+0000 7f0f31622640 1 -- 172.21.15.33:0/331483782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2c102d40 msgr2=0x7f0f2c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.976+0000 7f0f31622640 1 --2- 172.21.15.33:0/331483782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2c102d40 0x7f0f2c1031c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009900 tx=0x7f0f2002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.976+0000 7f0f31622640 1 -- 172.21.15.33:0/331483782 shutdown_connections 2024-04-02T13:38:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.976+0000 7f0f31622640 1 --2- 172.21.15.33:0/331483782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f2c103700 0x7f0f2c109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.976+0000 7f0f31622640 1 --2- 172.21.15.33:0/331483782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2c102d40 0x7f0f2c1031c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.976+0000 7f0f31622640 1 --2- 172.21.15.33:0/331483782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c101b50 0x7f0f2c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.976+0000 7f0f31622640 1 -- 172.21.15.33:0/331483782 >> 172.21.15.33:0/331483782 conn(0x7f0f2c0fd0c0 msgr2=0x7f0f2c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.976+0000 7f0f31622640 1 -- 172.21.15.33:0/331483782 shutdown_connections 2024-04-02T13:38:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.976+0000 7f0f31622640 1 -- 172.21.15.33:0/331483782 wait complete. 2024-04-02T13:38:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.977+0000 7f0f31622640 1 Processor -- start 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.977+0000 7f0f31622640 1 -- start start 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.977+0000 7f0f31622640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2c101b50 0x7f0f2c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.977+0000 7f0f31622640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f2c102d40 0x7f0f2c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.977+0000 7f0f31622640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c103700 0x7f0f2c19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.977+0000 7f0f31622640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c10d810 con 0x7f0f2c101b50 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.977+0000 7f0f31622640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c10d690 con 0x7f0f2c103700 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.977+0000 7f0f31622640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c10d990 con 0x7f0f2c102d40 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.977+0000 7f0f2affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2c101b50 0x7f0f2c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.977+0000 7f0f2affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2c101b50 0x7f0f2c197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48716/0 (socket says 172.21.15.33:48716) 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.977+0000 7f0f2affd640 1 -- 172.21.15.33:0/2935408774 learned_addr learned my addr 172.21.15.33:0/2935408774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.977+0000 7f0f2a7fc640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f2c102d40 0x7f0f2c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f2b7fe640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c103700 0x7f0f2c19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f2affd640 1 -- 172.21.15.33:0/2935408774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f2c102d40 msgr2=0x7f0f2c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f2affd640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f2c102d40 0x7f0f2c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f2affd640 1 -- 172.21.15.33:0/2935408774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c103700 msgr2=0x7f0f2c19f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:38.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f2affd640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c103700 0x7f0f2c19f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:38.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f2affd640 1 -- 172.21.15.33:0/2935408774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c19fa50 con 0x7f0f2c101b50 2024-04-02T13:38:38.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f2a7fc640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f2c102d40 0x7f0f2c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:38.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f2b7fe640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c103700 0x7f0f2c19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:38.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f2affd640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2c101b50 0x7f0f2c197e30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400d770 tx=0x7f0f2400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f13fff640 1 -- 172.21.15.33:0/2935408774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24017070 con 0x7f0f2c101b50 2024-04-02T13:38:38.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f13fff640 1 -- 172.21.15.33:0/2935408774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0f24009c30 con 0x7f0f2c101b50 2024-04-02T13:38:38.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f31622640 1 -- 172.21.15.33:0/2935408774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2c19fd40 con 0x7f0f2c101b50 2024-04-02T13:38:38.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f13fff640 1 -- 172.21.15.33:0/2935408774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24003e10 con 0x7f0f2c101b50 2024-04-02T13:38:38.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.978+0000 7f0f31622640 1 -- 172.21.15.33:0/2935408774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2c1a0250 con 0x7f0f2c101b50 2024-04-02T13:38:38.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.979+0000 7f0f31622640 1 -- 172.21.15.33:0/2935408774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2c108e10 con 0x7f0f2c101b50 2024-04-02T13:38:38.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.980+0000 7f0f13fff640 1 -- 172.21.15.33:0/2935408774 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0f24004020 con 0x7f0f2c101b50 2024-04-02T13:38:38.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.981+0000 7f0f13fff640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0f140a3c50 0x7f0f140a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:38.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.981+0000 7f0f2a7fc640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0f140a3c50 0x7f0f140a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:38.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.981+0000 7f0f13fff640 1 -- 172.21.15.33:0/2935408774 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f0f240d05f0 con 0x7f0f2c101b50 2024-04-02T13:38:38.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.981+0000 7f0f2a7fc640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0f140a3c50 0x7f0f140a6110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c199340 tx=0x7f0f20000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:38.984+0000 7f0f13fff640 1 -- 172.21.15.33:0/2935408774 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0f2409cbe0 con 0x7f0f2c101b50 2024-04-02T13:38:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:38 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1420943643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:38 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2836158947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:38 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2337334686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:39.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.222+0000 7f0f31622640 1 -- 172.21.15.33:0/2935408774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f0f2c062840 con 0x7f0f2c101b50 2024-04-02T13:38:39.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.557+0000 7f0f13fff640 1 -- 172.21.15.33:0/2935408774 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f0f2409c4e0 con 0x7f0f2c101b50 2024-04-02T13:38:39.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.560+0000 7f0f31622640 1 -- 172.21.15.33:0/2935408774 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0f140a3c50 msgr2=0x7f0f140a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:39.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.560+0000 7f0f31622640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0f140a3c50 0x7f0f140a6110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c199340 tx=0x7f0f20000bc0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.560+0000 7f0f31622640 1 -- 172.21.15.33:0/2935408774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2c101b50 msgr2=0x7f0f2c197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:39.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.560+0000 7f0f31622640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2c101b50 0x7f0f2c197e30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400d770 tx=0x7f0f2400dc30 comp rx=0 tx=0).stop 2024-04-02T13:38:39.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.560+0000 7f0f31622640 1 -- 172.21.15.33:0/2935408774 shutdown_connections 2024-04-02T13:38:39.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.560+0000 7f0f31622640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f0f140a3c50 0x7f0f140a6110 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.560+0000 7f0f31622640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f2c103700 0x7f0f2c19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.560+0000 7f0f31622640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f2c102d40 0x7f0f2c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.560+0000 7f0f31622640 1 --2- 172.21.15.33:0/2935408774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2c101b50 0x7f0f2c197e30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.560+0000 7f0f31622640 1 -- 172.21.15.33:0/2935408774 >> 172.21.15.33:0/2935408774 conn(0x7f0f2c0fd0c0 msgr2=0x7f0f2c105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:39.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.560+0000 7f0f31622640 1 -- 172.21.15.33:0/2935408774 shutdown_connections 2024-04-02T13:38:39.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.560+0000 7f0f31622640 1 -- 172.21.15.33:0/2935408774 wait complete. 2024-04-02T13:38:39.582 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-02T13:38:39.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.691+0000 7f7e3d10e640 1 -- 172.21.15.33:0/3426690162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e38101b60 msgr2=0x7f7e38101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:39.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.691+0000 7f7e3d10e640 1 --2- 172.21.15.33:0/3426690162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e38101b60 0x7f7e38101f80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009900 tx=0x7f7e3002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:39.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.691+0000 7f7e3d10e640 1 -- 172.21.15.33:0/3426690162 shutdown_connections 2024-04-02T13:38:39.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.691+0000 7f7e3d10e640 1 --2- 172.21.15.33:0/3426690162 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e38103710 0x7f7e38109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.691+0000 7f7e3d10e640 1 --2- 172.21.15.33:0/3426690162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e38102d50 0x7f7e381031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.691+0000 7f7e3d10e640 1 --2- 172.21.15.33:0/3426690162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e38101b60 0x7f7e38101f80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.691+0000 7f7e3d10e640 1 -- 172.21.15.33:0/3426690162 >> 172.21.15.33:0/3426690162 conn(0x7f7e380fd0d0 msgr2=0x7f7e380ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:39.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.691+0000 7f7e3d10e640 1 -- 172.21.15.33:0/3426690162 shutdown_connections 2024-04-02T13:38:39.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.691+0000 7f7e3d10e640 1 -- 172.21.15.33:0/3426690162 wait complete. 2024-04-02T13:38:39.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.691+0000 7f7e3d10e640 1 Processor -- start 2024-04-02T13:38:39.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e3d10e640 1 -- start start 2024-04-02T13:38:39.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e3d10e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e38101b60 0x7f7e38197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:39.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e3d10e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e38102d50 0x7f7e381982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e3d10e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e38103710 0x7f7e3819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e3d10e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3810d7c0 con 0x7f7e38103710 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e3d10e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3810d640 con 0x7f7e38101b60 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e3d10e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3810d940 con 0x7f7e38102d50 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e3759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e38103710 0x7f7e3819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e3759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e38103710 0x7f7e3819f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48762/0 (socket says 172.21.15.33:48762) 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e36d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e38101b60 0x7f7e38197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e3759e640 1 -- 172.21.15.33:0/181870495 learned_addr learned my addr 172.21.15.33:0/181870495 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e3659c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e38102d50 0x7f7e381982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e36d9d640 1 -- 172.21.15.33:0/181870495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e38102d50 msgr2=0x7f7e381982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e36d9d640 1 --2- 172.21.15.33:0/181870495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e38102d50 0x7f7e381982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e36d9d640 1 -- 172.21.15.33:0/181870495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e38103710 msgr2=0x7f7e3819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e36d9d640 1 --2- 172.21.15.33:0/181870495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e38103710 0x7f7e3819f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e36d9d640 1 -- 172.21.15.33:0/181870495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3819f9b0 con 0x7f7e38101b60 2024-04-02T13:38:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.692+0000 7f7e3759e640 1 --2- 172.21.15.33:0/181870495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e38103710 0x7f7e3819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.693+0000 7f7e36d9d640 1 --2- 172.21.15.33:0/181870495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e38101b60 0x7f7e38197da0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7e3002c170 tx=0x7f7e30005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.693+0000 7f7e2ffff640 1 -- 172.21.15.33:0/181870495 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30043020 con 0x7f7e38101b60 2024-04-02T13:38:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.693+0000 7f7e3d10e640 1 -- 172.21.15.33:0/181870495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3819fc40 con 0x7f7e38101b60 2024-04-02T13:38:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.693+0000 7f7e2ffff640 1 -- 172.21.15.33:0/181870495 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7e300054f0 con 0x7f7e38101b60 2024-04-02T13:38:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.693+0000 7f7e2ffff640 1 -- 172.21.15.33:0/181870495 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30003c30 con 0x7f7e38101b60 2024-04-02T13:38:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.693+0000 7f7e3d10e640 1 -- 172.21.15.33:0/181870495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e381a01b0 con 0x7f7e38101b60 2024-04-02T13:38:39.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.694+0000 7f7e3d10e640 1 -- 172.21.15.33:0/181870495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e38108e20 con 0x7f7e38101b60 2024-04-02T13:38:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.695+0000 7f7e2ffff640 1 -- 172.21.15.33:0/181870495 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f7e30046020 con 0x7f7e38101b60 2024-04-02T13:38:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.695+0000 7f7e2ffff640 1 --2- 172.21.15.33:0/181870495 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7e240a3ca0 0x7f7e240a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.695+0000 7f7e3659c640 1 -- 172.21.15.33:0/181870495 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7e240a3ca0 msgr2=0x7f7e240a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2562306694 2024-04-02T13:38:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.695+0000 7f7e2ffff640 1 -- 172.21.15.33:0/181870495 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7e300f2560 con 0x7f7e38101b60 2024-04-02T13:38:39.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.695+0000 7f7e3659c640 1 --2- 172.21.15.33:0/181870495 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7e240a3ca0 0x7f7e240a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:38:39.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.699+0000 7f7e2ffff640 1 -- 172.21.15.33:0/181870495 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7e3004d070 con 0x7f7e38101b60 2024-04-02T13:38:39.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.886+0000 7f7e3d10e640 1 -- 172.21.15.33:0/181870495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f7e3802d530 con 0x7f7e38101b60 2024-04-02T13:38:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.894+0000 7f7e2ffff640 1 -- 172.21.15.33:0/181870495 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f7e3802d530 con 0x7f7e38101b60 2024-04-02T13:38:39.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3659c640 1 -- 172.21.15.33:0/181870495 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7e240a3ca0 msgr2=0x7f7e240a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2562306694 2024-04-02T13:38:39.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3659c640 1 --2- 172.21.15.33:0/181870495 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7e240a3ca0 0x7f7e240a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:38:39.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3d10e640 1 -- 172.21.15.33:0/181870495 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7e240a3ca0 msgr2=0x7f7e240a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:38:39.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3d10e640 1 --2- 172.21.15.33:0/181870495 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7e240a3ca0 0x7f7e240a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3d10e640 1 -- 172.21.15.33:0/181870495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e38101b60 msgr2=0x7f7e38197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:39.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3d10e640 1 --2- 172.21.15.33:0/181870495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e38101b60 0x7f7e38197da0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7e3002c170 tx=0x7f7e30005dc0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3d10e640 1 -- 172.21.15.33:0/181870495 shutdown_connections 2024-04-02T13:38:39.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3d10e640 1 --2- 172.21.15.33:0/181870495 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7e240a3ca0 0x7f7e240a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3d10e640 1 --2- 172.21.15.33:0/181870495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e38103710 0x7f7e3819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3d10e640 1 --2- 172.21.15.33:0/181870495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e38102d50 0x7f7e381982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3d10e640 1 --2- 172.21.15.33:0/181870495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e38101b60 0x7f7e38197da0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:39.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3d10e640 1 -- 172.21.15.33:0/181870495 >> 172.21.15.33:0/181870495 conn(0x7f7e380fd0d0 msgr2=0x7f7e38105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:39.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3d10e640 1 -- 172.21.15.33:0/181870495 shutdown_connections 2024-04-02T13:38:39.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:39.896+0000 7f7e3d10e640 1 -- 172.21.15.33:0/181870495 wait complete. 2024-04-02T13:38:39.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:39 smithi033 ceph-mon[32390]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:38:39.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:39 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2935408774' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T13:38:39.910 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:39 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: ignoring --setuser ceph since I am not root 2024-04-02T13:38:39.911 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:39 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: ignoring --setgroup ceph since I am not root 2024-04-02T13:38:39.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-02T13:38:39.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:39 smithi145 ceph-mon[35646]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:38:39.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:39 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2935408774' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T13:38:39.979 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:39 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: ignoring --setuser ceph since I am not root 2024-04-02T13:38:39.979 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:39 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: ignoring --setgroup ceph since I am not root 2024-04-02T13:38:39.979 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:39 smithi145 ceph-mgr[37512]: -- 172.21.15.145:0/1031443652 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644c62205a0 con 0x5644c61f5000 2024-04-02T13:38:39.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:39 smithi050 ceph-mon[35524]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-02T13:38:39.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:39 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2935408774' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T13:38:39.994 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:39 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: ignoring --setuser ceph since I am not root 2024-04-02T13:38:39.994 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:39 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: ignoring --setgroup ceph since I am not root 2024-04-02T13:38:39.994 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:39 smithi050 ceph-mgr[36907]: -- 172.21.15.50:0/2455019161 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a8d20ee5a0 con 0x55a8d20c2800 2024-04-02T13:38:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.012+0000 7fea3cd96640 1 -- 172.21.15.33:0/3308359572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38100a20 msgr2=0x7fea38100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.012+0000 7fea3cd96640 1 --2- 172.21.15.33:0/3308359572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38100a20 0x7fea38100e40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fea28009900 tx=0x7fea2802bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.012+0000 7fea3cd96640 1 -- 172.21.15.33:0/3308359572 shutdown_connections 2024-04-02T13:38:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.012+0000 7fea3cd96640 1 --2- 172.21.15.33:0/3308359572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea381025d0 0x7fea38108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.012+0000 7fea3cd96640 1 --2- 172.21.15.33:0/3308359572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea38101c10 0x7fea38102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.012+0000 7fea3cd96640 1 --2- 172.21.15.33:0/3308359572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38100a20 0x7fea38100e40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.012+0000 7fea3cd96640 1 -- 172.21.15.33:0/3308359572 >> 172.21.15.33:0/3308359572 conn(0x7fea380fbf90 msgr2=0x7fea380fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.012+0000 7fea3cd96640 1 -- 172.21.15.33:0/3308359572 shutdown_connections 2024-04-02T13:38:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.012+0000 7fea3cd96640 1 -- 172.21.15.33:0/3308359572 wait complete. 2024-04-02T13:38:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.013+0000 7fea3cd96640 1 Processor -- start 2024-04-02T13:38:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.013+0000 7fea3cd96640 1 -- start start 2024-04-02T13:38:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.013+0000 7fea3cd96640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea38100a20 0x7fea38069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.013+0000 7fea3cd96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38101c10 0x7fea3819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.013+0000 7fea3cd96640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea381025d0 0x7fea3819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.013+0000 7fea3cd96640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea3810c680 con 0x7fea38101c10 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.013+0000 7fea3cd96640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fea3810c500 con 0x7fea38100a20 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.013+0000 7fea3cd96640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fea3810c800 con 0x7fea381025d0 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.013+0000 7fea2ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38101c10 0x7fea3819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.013+0000 7fea2ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38101c10 0x7fea3819cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48780/0 (socket says 172.21.15.33:48780) 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.013+0000 7fea2ffff640 1 -- 172.21.15.33:0/677876714 learned_addr learned my addr 172.21.15.33:0/677876714 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.013+0000 7fea36d9d640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea381025d0 0x7fea3819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.013+0000 7fea3659c640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea38100a20 0x7fea38069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.014+0000 7fea2ffff640 1 -- 172.21.15.33:0/677876714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea381025d0 msgr2=0x7fea3819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.014+0000 7fea2ffff640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea381025d0 0x7fea3819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.014+0000 7fea2ffff640 1 -- 172.21.15.33:0/677876714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea38100a20 msgr2=0x7fea38069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.014+0000 7fea2ffff640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea38100a20 0x7fea38069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:40.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.014+0000 7fea2ffff640 1 -- 172.21.15.33:0/677876714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3819f9e0 con 0x7fea38101c10 2024-04-02T13:38:40.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.014+0000 7fea2ffff640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38101c10 0x7fea3819cab0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fea2000e9f0 tx=0x7fea2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:40.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.014+0000 7fea2f7fe640 1 -- 172.21.15.33:0/677876714 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2000cd80 con 0x7fea38101c10 2024-04-02T13:38:40.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.014+0000 7fea3cd96640 1 -- 172.21.15.33:0/677876714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3819fcd0 con 0x7fea38101c10 2024-04-02T13:38:40.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.014+0000 7fea2f7fe640 1 -- 172.21.15.33:0/677876714 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fea200054f0 con 0x7fea38101c10 2024-04-02T13:38:40.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.014+0000 7fea2f7fe640 1 -- 172.21.15.33:0/677876714 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20010690 con 0x7fea38101c10 2024-04-02T13:38:40.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.014+0000 7fea3cd96640 1 -- 172.21.15.33:0/677876714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea381a01d0 con 0x7fea38101c10 2024-04-02T13:38:40.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.015+0000 7fea3cd96640 1 -- 172.21.15.33:0/677876714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea38107d40 con 0x7fea38101c10 2024-04-02T13:38:40.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.016+0000 7fea2f7fe640 1 -- 172.21.15.33:0/677876714 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fea20004f20 con 0x7fea38101c10 2024-04-02T13:38:40.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.017+0000 7fea2f7fe640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fea240a3ca0 0x7fea240a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:40.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.017+0000 7fea3659c640 1 -- 172.21.15.33:0/677876714 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fea240a3ca0 msgr2=0x7fea240a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2562306694 2024-04-02T13:38:40.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.017+0000 7fea2f7fe640 1 -- 172.21.15.33:0/677876714 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fea200d0c30 con 0x7fea38101c10 2024-04-02T13:38:40.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.017+0000 7fea3659c640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fea240a3ca0 0x7fea240a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:38:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.021+0000 7fea2f7fe640 1 -- 172.21.15.33:0/677876714 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fea2009d220 con 0x7fea38101c10 2024-04-02T13:38:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.218+0000 7fea3659c640 1 -- 172.21.15.33:0/677876714 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fea240a3ca0 msgr2=0x7fea240a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2562306694 2024-04-02T13:38:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.218+0000 7fea3659c640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fea240a3ca0 0x7fea240a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:38:40.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.222+0000 7fea3cd96640 1 -- 172.21.15.33:0/677876714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fea38062770 con 0x7fea38101c10 2024-04-02T13:38:40.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.228+0000 7fea2f7fe640 1 -- 172.21.15.33:0/677876714 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7fea2009cb20 con 0x7fea38101c10 2024-04-02T13:38:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.230+0000 7fea3cd96640 1 -- 172.21.15.33:0/677876714 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fea240a3ca0 msgr2=0x7fea240a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:38:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.230+0000 7fea3cd96640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fea240a3ca0 0x7fea240a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.230+0000 7fea3cd96640 1 -- 172.21.15.33:0/677876714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38101c10 msgr2=0x7fea3819cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.230+0000 7fea3cd96640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38101c10 0x7fea3819cab0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fea2000e9f0 tx=0x7fea2000edf0 comp rx=0 tx=0).stop 2024-04-02T13:38:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.230+0000 7fea3cd96640 1 -- 172.21.15.33:0/677876714 shutdown_connections 2024-04-02T13:38:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.230+0000 7fea3cd96640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7fea240a3ca0 0x7fea240a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.230+0000 7fea3cd96640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea381025d0 0x7fea3819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.230+0000 7fea3cd96640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38101c10 0x7fea3819cab0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.230+0000 7fea3cd96640 1 --2- 172.21.15.33:0/677876714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea38100a20 0x7fea38069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.230+0000 7fea3cd96640 1 -- 172.21.15.33:0/677876714 >> 172.21.15.33:0/677876714 conn(0x7fea380fbf90 msgr2=0x7fea38104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.230+0000 7fea3cd96640 1 -- 172.21.15.33:0/677876714 shutdown_connections 2024-04-02T13:38:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:40.230+0000 7fea3cd96640 1 -- 172.21.15.33:0/677876714 wait complete. 2024-04-02T13:38:40.245 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-02T13:38:40.256 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-02T13:38:40.256 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-02T13:38:40.256 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-04-02T13:38:40.256 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T13:38:40.256 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T13:38:40.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:40.256 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-02T13:38:40.271 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-02T13:38:40.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:40.272 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:38:40.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:40.343 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-04-02T13:38:40.437 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:40 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:40.172+0000 7f557f795200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:38:40.437 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:40 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:40.272+0000 7f557f795200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:38:40.443 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-02T13:38:40.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-04-02T13:38:40.478 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:40.139+0000 7fe531d5f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:38:40.478 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:40.235+0000 7fe531d5f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:38:40.494 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:40 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:40.152+0000 7f2bf1727200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:38:40.494 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:40 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:40.249+0000 7f2bf1727200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:38:40.697 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:38:40.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:40 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2935408774' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-02T13:38:40.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:40 smithi033 ceph-mon[32390]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-02T13:38:40.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:40 smithi033 ceph-mon[32390]: from='client.? ' entity='client.admin' 2024-04-02T13:38:40.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:40 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/677876714' entity='client.admin' 2024-04-02T13:38:40.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:40 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2935408774' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-02T13:38:40.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:40 smithi145 ceph-mon[35646]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-02T13:38:40.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:40 smithi145 ceph-mon[35646]: from='client.? ' entity='client.admin' 2024-04-02T13:38:40.888 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:40 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/677876714' entity='client.admin' 2024-04-02T13:38:40.916 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:40 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2935408774' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-02T13:38:40.917 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:40 smithi050 ceph-mon[35524]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-02T13:38:40.917 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:40 smithi050 ceph-mon[35524]: from='client.? ' entity='client.admin' 2024-04-02T13:38:40.917 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:40 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/677876714' entity='client.admin' 2024-04-02T13:38:41.077 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:40 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:40.975+0000 7f557f795200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:38:41.077 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:41 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:41.073+0000 7f557f795200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:38:41.221 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:40.885+0000 7fe531d5f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:38:41.221 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:40.978+0000 7fe531d5f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:38:41.221 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:41.122+0000 7fe531d5f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:38:41.244 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:40 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:40.914+0000 7f2bf1727200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:38:41.244 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:41 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:41.010+0000 7f2bf1727200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:38:41.244 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:41 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:41.153+0000 7f2bf1727200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:38:41.412 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:41 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:41.222+0000 7f557f795200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:38:41.412 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:41 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:41.322+0000 7f557f795200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:38:41.412 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:41 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:41.408+0000 7f557f795200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:38:41.478 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:41.218+0000 7fe531d5f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:38:41.478 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:41.302+0000 7fe531d5f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:38:41.613 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:41 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:41.251+0000 7f2bf1727200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:38:41.613 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:41 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:41.337+0000 7f2bf1727200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:38:41.869 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:41.574+0000 7fe531d5f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:38:41.870 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:41.685+0000 7fe531d5f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:38:41.870 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:41.771+0000 7fe531d5f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:38:41.911 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:41 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:41.610+0000 7f2bf1727200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:38:41.911 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:41 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:41.723+0000 7f2bf1727200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:38:41.912 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:41 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:41.811+0000 7f2bf1727200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:38:41.912 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:41 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:41.909+0000 7f2bf1727200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:38:41.998 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:41 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:41.686+0000 7f557f795200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:38:41.998 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:41 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:41.800+0000 7f557f795200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:38:41.998 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:41 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:41.891+0000 7f557f795200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:38:42.228 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:41.867+0000 7fe531d5f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:38:42.228 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:42 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:42.045+0000 7fe531d5f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:38:42.244 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:42 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:42.088+0000 7f2bf1727200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:38:42.253 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:41 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:41.994+0000 7f557f795200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:38:42.254 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:42 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:42.177+0000 7f557f795200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:38:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.439+0000 7f1fc631c700 1 -- 172.21.15.33:0/137870795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc006ac60 msgr2=0x7f1fc010f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.439+0000 7f1fc631c700 1 --2- 172.21.15.33:0/137870795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc006ac60 0x7f1fc010f450 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800b0a0 tx=0x7f1fb800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:38:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.440+0000 7f1fc631c700 1 -- 172.21.15.33:0/137870795 shutdown_connections 2024-04-02T13:38:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.440+0000 7f1fc631c700 1 --2- 172.21.15.33:0/137870795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc006ac60 0x7f1fc010f450 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.440+0000 7f1fc631c700 1 --2- 172.21.15.33:0/137870795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fc0108650 0x7f1fc0108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.440+0000 7f1fc631c700 1 --2- 172.21.15.33:0/137870795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fc006a2a0 0x7f1fc006a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.440+0000 7f1fc631c700 1 -- 172.21.15.33:0/137870795 >> 172.21.15.33:0/137870795 conn(0x7f1fc0078420 msgr2=0x7f1fc0078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.440+0000 7f1fc631c700 1 -- 172.21.15.33:0/137870795 shutdown_connections 2024-04-02T13:38:42.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.440+0000 7f1fc631c700 1 -- 172.21.15.33:0/137870795 wait complete. 2024-04-02T13:38:42.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.440+0000 7f1fc631c700 1 Processor -- start 2024-04-02T13:38:42.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.440+0000 7f1fc631c700 1 -- start start 2024-04-02T13:38:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fc631c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fc006a2a0 0x7f1fc019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fc631c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc006ac60 0x7f1fc019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fc631c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fc0108650 0x7f1fc0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fc631c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc01128b0 con 0x7f1fc006ac60 2024-04-02T13:38:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fc631c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0112730 con 0x7f1fc006a2a0 2024-04-02T13:38:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fc631c700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0112a30 con 0x7f1fc0108650 2024-04-02T13:38:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fbf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc006ac60 0x7f1fc019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fbf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc006ac60 0x7f1fc019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48802/0 (socket says 172.21.15.33:48802) 2024-04-02T13:38:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fbf7fe700 1 -- 172.21.15.33:0/1387264630 learned_addr learned my addr 172.21.15.33:0/1387264630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:42.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fbffff700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fc006a2a0 0x7f1fc019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fc48b9700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fc0108650 0x7f1fc0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fbf7fe700 1 -- 172.21.15.33:0/1387264630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fc0108650 msgr2=0x7f1fc0198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fbf7fe700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fc0108650 0x7f1fc0198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fbf7fe700 1 -- 172.21.15.33:0/1387264630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fc006a2a0 msgr2=0x7f1fc019ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fbf7fe700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fc006a2a0 0x7f1fc019ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fbf7fe700 1 -- 172.21.15.33:0/1387264630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc01993f0 con 0x7f1fc006ac60 2024-04-02T13:38:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.441+0000 7f1fbf7fe700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc006ac60 0x7f1fc019ef80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400d950 tx=0x7f1fb400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.442+0000 7f1fbd7fa700 1 -- 172.21.15.33:0/1387264630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4016070 con 0x7f1fc006ac60 2024-04-02T13:38:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.442+0000 7f1fbd7fa700 1 -- 172.21.15.33:0/1387264630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1fb400fd60 con 0x7f1fc006ac60 2024-04-02T13:38:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.442+0000 7f1fbd7fa700 1 -- 172.21.15.33:0/1387264630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400e3b0 con 0x7f1fc006ac60 2024-04-02T13:38:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.442+0000 7f1fc631c700 1 -- 172.21.15.33:0/1387264630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc01996e0 con 0x7f1fc006ac60 2024-04-02T13:38:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.442+0000 7f1fc631c700 1 -- 172.21.15.33:0/1387264630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc01a59c0 con 0x7f1fc006ac60 2024-04-02T13:38:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.443+0000 7f1fa6ffd700 1 -- 172.21.15.33:0/1387264630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc0103cf0 con 0x7f1fc006ac60 2024-04-02T13:38:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.445+0000 7f1fbd7fa700 1 -- 172.21.15.33:0/1387264630 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f1fb4019070 con 0x7f1fc006ac60 2024-04-02T13:38:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.445+0000 7f1fbd7fa700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1fa80a3c90 0x7f1fa80a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.445+0000 7f1fbffff700 1 -- 172.21.15.33:0/1387264630 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1fa80a3c90 msgr2=0x7f1fa80a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2562306694 2024-04-02T13:38:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.445+0000 7f1fbd7fa700 1 -- 172.21.15.33:0/1387264630 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1fb40d0300 con 0x7f1fc006ac60 2024-04-02T13:38:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.446+0000 7f1fbffff700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1fa80a3c90 0x7f1fa80a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:38:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.448+0000 7f1fbd7fa700 1 -- 172.21.15.33:0/1387264630 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1fb409c950 con 0x7f1fc006ac60 2024-04-02T13:38:42.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.647+0000 7f1fbffff700 1 -- 172.21.15.33:0/1387264630 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1fa80a3c90 msgr2=0x7f1fa80a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2562306694 2024-04-02T13:38:42.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.647+0000 7f1fbffff700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1fa80a3c90 0x7f1fa80a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:38:42.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.741+0000 7f1fa6ffd700 1 -- 172.21.15.33:0/1387264630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1fc0199fd0 con 0x7f1fc006ac60 2024-04-02T13:38:42.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.741+0000 7f1fbd7fa700 1 -- 172.21.15.33:0/1387264630 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f1fb409c250 con 0x7f1fc006ac60 2024-04-02T13:38:42.745 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-02T13:38:42.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.743+0000 7f1fa6ffd700 1 -- 172.21.15.33:0/1387264630 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1fa80a3c90 msgr2=0x7f1fa80a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:38:42.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.743+0000 7f1fa6ffd700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1fa80a3c90 0x7f1fa80a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.743+0000 7f1fa6ffd700 1 -- 172.21.15.33:0/1387264630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc006ac60 msgr2=0x7f1fc019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.743+0000 7f1fa6ffd700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc006ac60 0x7f1fc019ef80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400d950 tx=0x7f1fb400dc60 comp rx=0 tx=0).stop 2024-04-02T13:38:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.744+0000 7f1fa6ffd700 1 -- 172.21.15.33:0/1387264630 shutdown_connections 2024-04-02T13:38:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.744+0000 7f1fa6ffd700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f1fa80a3c90 0x7f1fa80a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.744+0000 7f1fa6ffd700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fc0108650 0x7f1fc0198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.744+0000 7f1fa6ffd700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1fc006a2a0 0x7f1fc019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.744+0000 7f1fa6ffd700 1 --2- 172.21.15.33:0/1387264630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc006ac60 0x7f1fc019ef80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.744+0000 7f1fa6ffd700 1 -- 172.21.15.33:0/1387264630 >> 172.21.15.33:0/1387264630 conn(0x7f1fc0078420 msgr2=0x7f1fc01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.744+0000 7f1fa6ffd700 1 -- 172.21.15.33:0/1387264630 shutdown_connections 2024-04-02T13:38:42.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:42.744+0000 7f1fa6ffd700 1 -- 172.21.15.33:0/1387264630 wait complete. 2024-04-02T13:38:43.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:42 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1387264630' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:38:43.087 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:42 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1387264630' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:38:43.159 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:42 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1387264630' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:38:43.160 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:43.024+0000 7fe531d5f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:38:43.340 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:43 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:43.198+0000 7f557f795200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:38:43.340 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:43 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:43.336+0000 7f557f795200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:38:43.446 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T13:38:43.446 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T13:38:43.446 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-04-02T13:38:43.446 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-02T13:38:43.446 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T13:38:43.446 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T13:38:43.446 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T13:38:43.447 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T13:38:43.447 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-02T13:38:43.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:43.447 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-04-02T13:38:43.463 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T13:38:43.463 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T13:38:43.463 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T13:38:43.463 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:38:43.463 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-04-02T13:38:43.463 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:38:43.463 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:38:43.463 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:38:43.464 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-04-02T13:38:43.464 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:38:43.464 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:38:43.464 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-04-02T13:38:43.464 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T13:38:43.464 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:38:43.464 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-04-02T13:38:43.464 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-02T13:38:43.464 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42095sec preferred_lft 42095sec 2024-04-02T13:38:43.464 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-04-02T13:38:43.464 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:38:43.464 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:38:43.465 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:38:43.465 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-04-02T13:38:43.465 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T13:38:43.465 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:38:43.465 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-02T13:38:43.465 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-04-02T13:38:43.465 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-04-02T13:38:43.465 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-04-02T13:38:43.465 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T13:38:43.465 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-02T13:38:43.478 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:43.156+0000 7fe531d5f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:38:43.494 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:43 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:43.084+0000 7f2bf1727200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:38:43.494 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:43 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:43.219+0000 7f2bf1727200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:38:43.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:38:43.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:38:43.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:43.624 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T13:38:43.651 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-04-02T13:38:43.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:43.653 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-04-02T13:38:43.709 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-04-02T13:38:43.709 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T13:38:43.709 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-02T13:38:43.709 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-02T13:38:43.709 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T13:38:43.710 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:38:43.710 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-02T13:38:43.710 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-02T13:38:43.710 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-02T13:38:43.710 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-02T13:38:43.710 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-02T13:38:43.788 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:43.494+0000 7fe531d5f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:38:43.788 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:43.584+0000 7fe531d5f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:38:43.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:38:43.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:38:43.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:43.848 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-02T13:38:43.857 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:43 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:43.559+0000 7f2bf1727200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:38:43.858 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:43 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:43.652+0000 7f2bf1727200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:38:43.858 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:43 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:43.855+0000 7f2bf1727200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:38:43.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:43.864 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-04-02T13:38:43.919 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:38:43.919 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-02T13:38:43.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T13:38:43.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-02T13:38:43.919 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-02T13:38:44.003 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:43 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:43.689+0000 7f557f795200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:38:44.003 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:43 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:43.786+0000 7f557f795200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:38:44.003 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:44 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:43.999+0000 7f557f795200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:38:44.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:38:44.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:38:44.043 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-02T13:38:44.043 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:38:44.043 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-02T13:38:44.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-02T13:38:44.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T13:38:44.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-02T13:38:44.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-02T13:38:44.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-02T13:38:44.043 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-02T13:38:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:38:44.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:38:44.212 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:38:44.212 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-02T13:38:44.213 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-04-02T13:38:44.213 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T13:38:44.213 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-02T13:38:44.228 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:43.785+0000 7fe531d5f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:38:44.228 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:43.974+0000 7fe531d5f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:38:44.244 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:44 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:44.045+0000 7f2bf1727200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:38:44.253 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:44 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:44.203+0000 7f557f795200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:38:44.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:38:44.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:38:44.297 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-04-02T13:38:44.297 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T13:38:44.297 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-02T13:38:44.352 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-02T13:38:44.352 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T13:38:44.352 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-02T13:38:44.407 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-02T13:38:44.479 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T13:38:44.530 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:44 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:44.264+0000 7fe531d5f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:38:44.534 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-02T13:38:44.534 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-02T13:38:44.534 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-02T13:38:44.534 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-02T13:38:44.534 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-02T13:38:44.534 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-02T13:38:44.534 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-02T13:38:44.534 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-02T13:38:44.534 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-02T13:38:44.534 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-02T13:38:44.534 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-02T13:38:44.535 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-04-02T13:38:44.536 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/0ebeb3da0dd34f8d94007fc151b6d2588192fff9ab353aa3f4acfb0cf084df55/merged 2024-04-02T13:38:44.536 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/dfe8c37ae7135c0a7f78a5cf8699396447f2dad62742b90e685d6e7739afaa7b/merged 2024-04-02T13:38:44.536 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-04-02T13:38:44.536 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:38:44.536 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/6670f8ed89d9e5fff5c9aa1fae6de93f2f45287d5c03ec0031f40bc22622da28/merged 2024-04-02T13:38:44.536 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/a917a17da40ebb0aa0673f46fa16bbc2efca845d0b3e85aaf3134a25576b1412/merged 2024-04-02T13:38:44.536 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/ad73ca7d4949a816f571ffc36b08e4c009dc9dab17c6a4e83b390814ebb235e3/merged 2024-04-02T13:38:44.537 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/fca44f73029ac4cc1fb49198880e18a9e1bda9a2b2fae3463330f27a42d2e2ac/merged 2024-04-02T13:38:44.537 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/9847228aaa43b9471a6980db4b10398714757c25c8061d6017166911e5ff8d02/merged 2024-04-02T13:38:44.537 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/3ed9f8d0604af5def8f2cf080564de09304b6b24f4e8095b605ccf6e91ad89c2/merged 2024-04-02T13:38:44.537 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/ff71119cf0c70afd63a8e6548b8092b9ff819d7decc1df28d7e9d5dc532a5757/merged 2024-04-02T13:38:44.537 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/5150cff3add9666036b2b9b04dd935d8b0e27d025b4d2ad90a004d6451c7eaab/merged 2024-04-02T13:38:44.537 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-02T13:38:44.537 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T13:38:44.537 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T13:38:44.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:44.537 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-02T13:38:44.590 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-02T13:38:44.590 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/bd756cb6-7314-4c9e-8eb1-a4a1e6d486b4 --client_fs=cephfs) 2024-04-02T13:38:44.606 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:44 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:44.340+0000 7f2bf1727200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:38:44.633 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-02T13:38:44.655 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:44 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:44.516+0000 7f557f795200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:38:44.662 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T13:38:44.717 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-02T13:38:44.718 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/0ebeb3da0dd34f8d94007fc151b6d2588192fff9ab353aa3f4acfb0cf084df55/merged 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/dfe8c37ae7135c0a7f78a5cf8699396447f2dad62742b90e685d6e7739afaa7b/merged 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-04-02T13:38:44.719 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:38:44.720 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/6670f8ed89d9e5fff5c9aa1fae6de93f2f45287d5c03ec0031f40bc22622da28/merged 2024-04-02T13:38:44.720 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/a917a17da40ebb0aa0673f46fa16bbc2efca845d0b3e85aaf3134a25576b1412/merged 2024-04-02T13:38:44.720 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/ad73ca7d4949a816f571ffc36b08e4c009dc9dab17c6a4e83b390814ebb235e3/merged 2024-04-02T13:38:44.720 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/fca44f73029ac4cc1fb49198880e18a9e1bda9a2b2fae3463330f27a42d2e2ac/merged 2024-04-02T13:38:44.720 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/9847228aaa43b9471a6980db4b10398714757c25c8061d6017166911e5ff8d02/merged 2024-04-02T13:38:44.720 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/3ed9f8d0604af5def8f2cf080564de09304b6b24f4e8095b605ccf6e91ad89c2/merged 2024-04-02T13:38:44.720 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/ff71119cf0c70afd63a8e6548b8092b9ff819d7decc1df28d7e9d5dc532a5757/merged 2024-04-02T13:38:44.720 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/5150cff3add9666036b2b9b04dd935d8b0e27d025b4d2ad90a004d6451c7eaab/merged 2024-04-02T13:38:44.720 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-02T13:38:44.720 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T13:38:44.720 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T13:38:44.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:44.721 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-02T13:38:44.723 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:2024-04-02T13:38:44.720+0000 7f75bec6d480 -1 init, newargv = 0x55739e001550 newargc=15 2024-04-02T13:38:44.723 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[84163]: starting ceph client 2024-04-02T13:38:44.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[84163]: starting fuse 2024-04-02T13:38:44.739 INFO:teuthology.orchestra.run.smithi033.stdout:119 2024-04-02T13:38:44.739 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-04-02T13:38:44.739 DEBUG:teuthology.orchestra.run.smithi033:> sudo stdin-killer -- python3 -c ' 2024-04-02T13:38:44.739 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-04-02T13:38:44.739 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-04-02T13:38:44.739 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-04-02T13:38:44.739 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-04-02T13:38:44.739 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-04-02T13:38:44.740 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-02T13:38:44.741 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-02T13:38:44.741 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.0")) 2024-04-02T13:38:44.741 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-04-02T13:38:44.856 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:44 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:44.603+0000 7f2bf1727200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:38:44.862 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.0.84163.asok 2024-04-02T13:38:44.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:38:44.874 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.84163.asok 2024-04-02T13:38:44.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:44.874 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.84163.asok status 2024-04-02T13:38:44.938 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:44 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:44.795+0000 7f557f795200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:38:44.978 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:44 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:44.528+0000 7fe531d5f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:38:44.978 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:44 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:44.771+0000 7fe531d5f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:38:45.013 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:38:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-04-02T13:38:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "191d963ac9de7322da8aff20b567b3fa458e2ef6", 2024-04-02T13:38:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T13:38:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "0", 2024-04-02T13:38:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-04-02T13:38:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-02T13:38:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "84163", 2024-04-02T13:38:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/volumes/qa/sv_0/bd756cb6-7314-4c9e-8eb1-a4a1e6d486b4" 2024-04-02T13:38:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:38:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-04-02T13:38:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-04-02T13:38:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 25057, 2024-04-02T13:38:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-04-02T13:38:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-04-02T13:38:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-04-02T13:38:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 25057 2024-04-02T13:38:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:38:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-02T13:38:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-02T13:38:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-02T13:38:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 503048036 2024-04-02T13:38:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:38:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:38:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-02T13:38:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-02T13:38:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-02T13:38:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 503048036 2024-04-02T13:38:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:38:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.25057 192.168.0.1:0/503048036", 2024-04-02T13:38:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/503048036", 2024-04-02T13:38:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 5, 2024-04-02T13:38:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 24, 2024-04-02T13:38:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 94, 2024-04-02T13:38:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-04-02T13:38:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-04-02T13:38:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-04-02T13:38:45.016 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:38:45.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:45.022 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-04-02T13:38:45.022 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-02T13:38:45.077 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-04-02T13:38:45.077 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-02T13:38:45.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:45.078 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-02T13:38:45.146 INFO:teuthology.run_tasks:Running task exec... 2024-04-02T13:38:45.156 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T13:38:45.156 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-02T13:38:45.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-02T13:38:45.212 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:45 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:45.053+0000 7f557f795200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:38:45.244 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:44 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:44.854+0000 7f2bf1727200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:38:45.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.290+0000 7f5463ca5640 1 -- 172.21.15.33:0/2168440400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f545c1025a0 msgr2=0x7f545c104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.290+0000 7f5463ca5640 1 --2- 172.21.15.33:0/2168440400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f545c1025a0 0x7f545c104990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f544c009900 tx=0x7f544c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.290+0000 7f5463ca5640 1 -- 172.21.15.33:0/2168440400 shutdown_connections 2024-04-02T13:38:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.290+0000 7f5463ca5640 1 --2- 172.21.15.33:0/2168440400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c107800 0x7f545c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.290+0000 7f5463ca5640 1 --2- 172.21.15.33:0/2168440400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f545c104ed0 0x7f545c1072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.290+0000 7f5463ca5640 1 --2- 172.21.15.33:0/2168440400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f545c1025a0 0x7f545c104990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.290+0000 7f5463ca5640 1 -- 172.21.15.33:0/2168440400 >> 172.21.15.33:0/2168440400 conn(0x7f545c0fbf90 msgr2=0x7f545c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.290+0000 7f5463ca5640 1 -- 172.21.15.33:0/2168440400 shutdown_connections 2024-04-02T13:38:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.290+0000 7f5463ca5640 1 -- 172.21.15.33:0/2168440400 wait complete. 2024-04-02T13:38:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.290+0000 7f5463ca5640 1 Processor -- start 2024-04-02T13:38:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.290+0000 7f5463ca5640 1 -- start start 2024-04-02T13:38:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5463ca5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f545c1025a0 0x7f545c101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5463ca5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f545c104ed0 0x7f545c100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5463ca5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c107800 0x7f545c100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5463ca5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f545c10d7a0 con 0x7f545c104ed0 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5463ca5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f545c10d620 con 0x7f545c107800 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5463ca5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f545c10d920 con 0x7f545c1025a0 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5461240640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f545c104ed0 0x7f545c100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5461240640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f545c104ed0 0x7f545c100550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58994/0 (socket says 172.21.15.33:58994) 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5461240640 1 -- 172.21.15.33:0/3321456075 learned_addr learned my addr 172.21.15.33:0/3321456075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5461a41640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f545c1025a0 0x7f545c101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5462242640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c107800 0x7f545c100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5461240640 1 -- 172.21.15.33:0/3321456075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f545c1025a0 msgr2=0x7f545c101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5461240640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f545c1025a0 0x7f545c101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5461240640 1 -- 172.21.15.33:0/3321456075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c107800 msgr2=0x7f545c100a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5461240640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c107800 0x7f545c100a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5461240640 1 -- 172.21.15.33:0/3321456075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c101320 con 0x7f545c104ed0 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5461240640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f545c104ed0 0x7f545c100550 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f545800e9f0 tx=0x7f545800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5452ffd640 1 -- 172.21.15.33:0/3321456075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545800ccc0 con 0x7f545c104ed0 2024-04-02T13:38:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5463ca5640 1 -- 172.21.15.33:0/3321456075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545c1a1f00 con 0x7f545c104ed0 2024-04-02T13:38:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5452ffd640 1 -- 172.21.15.33:0/3321456075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f54580054f0 con 0x7f545c104ed0 2024-04-02T13:38:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5452ffd640 1 -- 172.21.15.33:0/3321456075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458010690 con 0x7f545c104ed0 2024-04-02T13:38:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.291+0000 7f5463ca5640 1 -- 172.21.15.33:0/3321456075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545c1a2360 con 0x7f545c104ed0 2024-04-02T13:38:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.292+0000 7f5463ca5640 1 -- 172.21.15.33:0/3321456075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f545c0fda80 con 0x7f545c104ed0 2024-04-02T13:38:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.294+0000 7f5452ffd640 1 -- 172.21.15.33:0/3321456075 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f5458004e80 con 0x7f545c104ed0 2024-04-02T13:38:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.295+0000 7f5452ffd640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54480a3ca0 0x7f54480a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.295+0000 7f5461a41640 1 -- 172.21.15.33:0/3321456075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54480a3ca0 msgr2=0x7f54480a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2562306694 2024-04-02T13:38:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.295+0000 7f5452ffd640 1 -- 172.21.15.33:0/3321456075 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f54580d0ca0 con 0x7f545c104ed0 2024-04-02T13:38:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.295+0000 7f5461a41640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54480a3ca0 0x7f54480a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:38:45.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.298+0000 7f5452ffd640 1 -- 172.21.15.33:0/3321456075 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5458053f20 con 0x7f545c104ed0 2024-04-02T13:38:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.496+0000 7f5461a41640 1 -- 172.21.15.33:0/3321456075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54480a3ca0 msgr2=0x7f54480a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2562306694 2024-04-02T13:38:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.496+0000 7f5461a41640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54480a3ca0 0x7f54480a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:38:45.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.529+0000 7f5463ca5640 1 -- 172.21.15.33:0/3321456075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f545c02d570 con 0x7f545c104ed0 2024-04-02T13:38:45.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.581+0000 7f5452ffd640 1 -- 172.21.15.33:0/3321456075 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f545809d2f0 con 0x7f545c104ed0 2024-04-02T13:38:45.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.583+0000 7f5463ca5640 1 -- 172.21.15.33:0/3321456075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54480a3ca0 msgr2=0x7f54480a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:38:45.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.583+0000 7f5463ca5640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54480a3ca0 0x7f54480a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.583+0000 7f5463ca5640 1 -- 172.21.15.33:0/3321456075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f545c104ed0 msgr2=0x7f545c100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:45.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.583+0000 7f5463ca5640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f545c104ed0 0x7f545c100550 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f545800e9f0 tx=0x7f545800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.583+0000 7f5463ca5640 1 -- 172.21.15.33:0/3321456075 shutdown_connections 2024-04-02T13:38:45.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.583+0000 7f5463ca5640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f54480a3ca0 0x7f54480a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.583+0000 7f5463ca5640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f545c107800 0x7f545c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.583+0000 7f5463ca5640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f545c104ed0 0x7f545c100550 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.583+0000 7f5463ca5640 1 --2- 172.21.15.33:0/3321456075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f545c1025a0 0x7f545c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.583+0000 7f5463ca5640 1 -- 172.21.15.33:0/3321456075 >> 172.21.15.33:0/3321456075 conn(0x7f545c0fbf90 msgr2=0x7f545c102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:45.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.583+0000 7f5463ca5640 1 -- 172.21.15.33:0/3321456075 shutdown_connections 2024-04-02T13:38:45.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.583+0000 7f5463ca5640 1 -- 172.21.15.33:0/3321456075 wait complete. 2024-04-02T13:38:45.598 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-02T13:38:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.742+0000 7f7be2ac7640 1 -- 172.21.15.33:0/3396351401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bdc102600 msgr2=0x7f7bdc1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.742+0000 7f7be2ac7640 1 --2- 172.21.15.33:0/3396351401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bdc102600 0x7f7bdc1049f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009900 tx=0x7f7bd002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.742+0000 7f7be2ac7640 1 -- 172.21.15.33:0/3396351401 shutdown_connections 2024-04-02T13:38:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.742+0000 7f7be2ac7640 1 --2- 172.21.15.33:0/3396351401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bdc107860 0x7f7bdc109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.742+0000 7f7be2ac7640 1 --2- 172.21.15.33:0/3396351401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bdc104f30 0x7f7bdc107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.742+0000 7f7be2ac7640 1 --2- 172.21.15.33:0/3396351401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bdc102600 0x7f7bdc1049f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.742+0000 7f7be2ac7640 1 -- 172.21.15.33:0/3396351401 >> 172.21.15.33:0/3396351401 conn(0x7f7bdc0fbf90 msgr2=0x7f7bdc0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.742+0000 7f7be2ac7640 1 -- 172.21.15.33:0/3396351401 shutdown_connections 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.742+0000 7f7be2ac7640 1 -- 172.21.15.33:0/3396351401 wait complete. 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7be2ac7640 1 Processor -- start 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7be2ac7640 1 -- start start 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7be2ac7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bdc102600 0x7f7bdc101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7be2ac7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bdc104f30 0x7f7bdc1005b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7be2ac7640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bdc107860 0x7f7bdc100af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7be2ac7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc10d780 con 0x7f7bdc104f30 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7be2ac7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc10d600 con 0x7f7bdc102600 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7be2ac7640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc10d900 con 0x7f7bdc107860 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bdc104f30 0x7f7bdc1005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bdc104f30 0x7f7bdc1005b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59004/0 (socket says 172.21.15.33:59004) 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7bdbfff640 1 -- 172.21.15.33:0/801744845 learned_addr learned my addr 172.21.15.33:0/801744845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7be0863640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bdc102600 0x7f7bdc101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7be1064640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bdc107860 0x7f7bdc100af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7bdbfff640 1 -- 172.21.15.33:0/801744845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bdc107860 msgr2=0x7f7bdc100af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7bdbfff640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bdc107860 0x7f7bdc100af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7bdbfff640 1 -- 172.21.15.33:0/801744845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bdc102600 msgr2=0x7f7bdc101f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7bdbfff640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bdc102600 0x7f7bdc101f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.743+0000 7f7bdbfff640 1 -- 172.21.15.33:0/801744845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc101380 con 0x7f7bdc104f30 2024-04-02T13:38:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.744+0000 7f7bdbfff640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bdc104f30 0x7f7bdc1005b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8005fd0 tx=0x7f7bc800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.744+0000 7f7bd9ffb640 1 -- 172.21.15.33:0/801744845 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8017070 con 0x7f7bdc104f30 2024-04-02T13:38:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.744+0000 7f7bd9ffb640 1 -- 172.21.15.33:0/801744845 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7bc8009c30 con 0x7f7bdc104f30 2024-04-02T13:38:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.744+0000 7f7bd9ffb640 1 -- 172.21.15.33:0/801744845 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8003e20 con 0x7f7bdc104f30 2024-04-02T13:38:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.744+0000 7f7be2ac7640 1 -- 172.21.15.33:0/801744845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bdc1a1f20 con 0x7f7bdc104f30 2024-04-02T13:38:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.744+0000 7f7be2ac7640 1 -- 172.21.15.33:0/801744845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bdc1a2390 con 0x7f7bdc104f30 2024-04-02T13:38:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.745+0000 7f7be2ac7640 1 -- 172.21.15.33:0/801744845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bdc0fda80 con 0x7f7bdc104f30 2024-04-02T13:38:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.746+0000 7f7bd9ffb640 1 -- 172.21.15.33:0/801744845 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f7bc8004020 con 0x7f7bdc104f30 2024-04-02T13:38:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.747+0000 7f7bd9ffb640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bcc0a3d20 0x7f7bcc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:45.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.747+0000 7f7be0863640 1 -- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bcc0a3d20 msgr2=0x7f7bcc0a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2562306694 2024-04-02T13:38:45.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.747+0000 7f7bd9ffb640 1 -- 172.21.15.33:0/801744845 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7bc80d0360 con 0x7f7bdc104f30 2024-04-02T13:38:45.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.747+0000 7f7be0863640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bcc0a3d20 0x7f7bcc0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:38:45.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.751+0000 7f7bd9ffb640 1 -- 172.21.15.33:0/801744845 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7bc8020030 con 0x7f7bdc104f30 2024-04-02T13:38:45.932 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:45 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3321456075' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-02T13:38:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.948+0000 7f7be0863640 1 -- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bcc0a3d20 msgr2=0x7f7bcc0a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2562306694 2024-04-02T13:38:45.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.948+0000 7f7be0863640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bcc0a3d20 0x7f7bcc0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:38:45.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:45.967+0000 7f7be2ac7640 1 -- 172.21.15.33:0/801744845 --> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f7bdc0620a0 con 0x7f7bcc0a3d20 2024-04-02T13:38:45.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:45 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3321456075' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-02T13:38:45.979 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:45 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:45.593+0000 7fe531d5f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:38:45.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:45 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3321456075' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-02T13:38:45.994 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:45 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:45.693+0000 7f2bf1727200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:38:46.253 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:45 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:45.928+0000 7f557f795200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:38:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:46.352+0000 7f7be0863640 1 -- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bcc0a3d20 msgr2=0x7f7bcc0a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2562306694 2024-04-02T13:38:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:46.352+0000 7f7be0863640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bcc0a3d20 0x7f7bcc0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T13:38:46.478 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:46 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:46.172+0000 7fe531d5f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:38:46.478 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:46 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:46.271+0000 7fe531d5f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:38:46.592 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:46 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:46.280+0000 7f2bf1727200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:38:46.592 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:46 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:46.381+0000 7f2bf1727200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:38:46.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:46.602+0000 7f7bd9ffb640 1 -- 172.21.15.33:0/801744845 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f7bc8004310 con 0x7f7bdc104f30 2024-04-02T13:38:46.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:46 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3321456075' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-02T13:38:46.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:46 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:38:46.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:46 smithi033 ceph-mon[32390]: Standby manager daemon z restarted 2024-04-02T13:38:46.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:46 smithi033 ceph-mon[32390]: Standby manager daemon z started 2024-04-02T13:38:46.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:46 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.145:0/515130669' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:38:46.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:46 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.145:0/515130669' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:38:46.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:46 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.145:0/515130669' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:38:46.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:46 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.145:0/515130669' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:38:46.868 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:46 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:46.525+0000 7f557f795200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:38:46.869 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:46 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:46.632+0000 7f557f795200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:38:46.869 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:46 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:46.864+0000 7f557f795200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:38:46.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:46.877+0000 7f7bd9ffb640 1 -- 172.21.15.33:0/801744845 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f7bc80cdaf0 con 0x7f7bdc104f30 2024-04-02T13:38:46.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:46.878+0000 7f7bd9ffb640 1 -- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bcc0a3d20 msgr2=0x7f7bcc0a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:38:46.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:46.878+0000 7f7bd9ffb640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/2562306694,v1:172.21.15.33:6801/2562306694] conn(0x7f7bcc0a3d20 0x7f7bcc0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:46.978 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:38:46 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37508]: 2024-04-02T13:38:46.491+0000 7fe531d5f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:38:46.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:46 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3321456075' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-02T13:38:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:46 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:38:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:46 smithi145 ceph-mon[35646]: Standby manager daemon z restarted 2024-04-02T13:38:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:46 smithi145 ceph-mon[35646]: Standby manager daemon z started 2024-04-02T13:38:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:46 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.145:0/515130669' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:38:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:46 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.145:0/515130669' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:38:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:46 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.145:0/515130669' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:38:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:46 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.145:0/515130669' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:38:46.994 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:38:46 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y[36903]: 2024-04-02T13:38:46.609+0000 7f2bf1727200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:38:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:46 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3321456075' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-02T13:38:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:46 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:38:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:46 smithi050 ceph-mon[35524]: Standby manager daemon z restarted 2024-04-02T13:38:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:46 smithi050 ceph-mon[35524]: Standby manager daemon z started 2024-04-02T13:38:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:46 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.145:0/515130669' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:38:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:46 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.145:0/515130669' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:38:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:46 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.145:0/515130669' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:38:46.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:46 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.145:0/515130669' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:38:47.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.125+0000 7f54df0e4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:47.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.125+0000 7f54df0e4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:47.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.125+0000 7f54df0e4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:47.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.125+0000 7f54df0e4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:47.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.125+0000 7f54df0e4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:47.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.125+0000 7f54e18e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:47.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.125+0000 7f54e18e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:47.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.125+0000 7f54e18e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:47.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.125+0000 7f54e18e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:47.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.125+0000 7f54e18e9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:47.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.883+0000 7f7bd9ffb640 1 -- 172.21.15.33:0/801744845 <== mon.0 v2:172.21.15.33:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f7bc804d260 con 0x7f7bdc104f30 2024-04-02T13:38:47.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.884+0000 7f7bd9ffb640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7bcc035890 0x7f7bcc035c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:47.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.884+0000 7f7bd9ffb640 1 -- 172.21.15.33:0/801744845 --> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f7bc80056e0 con 0x7f7bcc035890 2024-04-02T13:38:47.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.885+0000 7f7be0863640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7bcc035890 0x7f7bcc035c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.890+0000 7f7be0863640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7bcc035890 0x7f7bcc035c70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0005ab0 tx=0x7f7bd0005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:47.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-02T13:38:47.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: Standby manager daemon y restarted 2024-04-02T13:38:47.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: Standby manager daemon y started 2024-04-02T13:38:47.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.50:0/2308805382' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:38:47.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.50:0/2308805382' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:38:47.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.50:0/2308805382' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:38:47.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.? 172.21.15.50:0/2308805382' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:38:47.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: Active manager daemon x restarted 2024-04-02T13:38:47.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: Activating manager daemon x 2024-04-02T13:38:47.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T13:38:47.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: mgrmap e21: x(active, starting, since 0.0122392s), standbys: z, y 2024-04-02T13:38:47.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:38:47.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:38:47.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:38:47.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:38:47.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:38:47.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:38:47.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:38:47.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:38:47.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: Manager daemon x is now available 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:38:47.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:38:47.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:38:47.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.912+0000 7f7bd9ffb640 1 -- 172.21.15.33:0/801744845 <== mgr.15456 v2:172.21.15.33:6800/619073725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f7bc80056e0 con 0x7f7bcc035890 2024-04-02T13:38:47.916 INFO:teuthology.orchestra.run.smithi033.stdout:{} 2024-04-02T13:38:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.914+0000 7f7be2ac7640 1 -- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7bcc035890 msgr2=0x7f7bcc035c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.914+0000 7f7be2ac7640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7bcc035890 0x7f7bcc035c70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0005ab0 tx=0x7f7bd0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.914+0000 7f7be2ac7640 1 -- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bdc104f30 msgr2=0x7f7bdc1005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.914+0000 7f7be2ac7640 1 --2- 172.21.15.33:0/801744845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bdc104f30 0x7f7bdc1005b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8005fd0 tx=0x7f7bc800dde0 comp rx=0 tx=0).stop 2024-04-02T13:38:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.914+0000 7f7be1064640 1 -- 172.21.15.33:0/801744845 reap_dead start 2024-04-02T13:38:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.915+0000 7f7be2ac7640 1 -- 172.21.15.33:0/801744845 shutdown_connections 2024-04-02T13:38:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.915+0000 7f7be2ac7640 1 -- 172.21.15.33:0/801744845 >> 172.21.15.33:0/801744845 conn(0x7f7bdc0fbf90 msgr2=0x7f7bdc1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.915+0000 7f7be2ac7640 1 -- 172.21.15.33:0/801744845 shutdown_connections 2024-04-02T13:38:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:47.915+0000 7f7be2ac7640 1 -- 172.21.15.33:0/801744845 wait complete. 2024-04-02T13:38:47.931 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-02T13:38:47.944 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-02T13:38:47.955 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:38:47.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-02T13:38:47.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: Standby manager daemon y restarted 2024-04-02T13:38:47.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: Standby manager daemon y started 2024-04-02T13:38:47.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.50:0/2308805382' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:38:47.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.50:0/2308805382' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:38:47.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.50:0/2308805382' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:38:47.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.? 172.21.15.50:0/2308805382' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:38:47.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: Active manager daemon x restarted 2024-04-02T13:38:47.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: Activating manager daemon x 2024-04-02T13:38:47.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T13:38:47.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: mgrmap e21: x(active, starting, since 0.0122392s), standbys: z, y 2024-04-02T13:38:47.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:38:47.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:38:47.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:38:47.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:38:47.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:38:47.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:38:47.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:38:47.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:38:47.983 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:38:47.983 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:38:47.983 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:38:47.983 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:38:47.983 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:38:47.983 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:38:47.984 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:38:47.984 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:38:47.984 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:38:47.984 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:38:47.984 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:38:47.984 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:38:47.984 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:38:47.984 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:38:47.984 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:38:47.984 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:38:47.984 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:38:47.984 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:38:47.984 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:38:47.985 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:38:47.985 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:38:47.985 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:38:47.985 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:38:47.985 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:38:47.985 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:38:47.985 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: Manager daemon x is now available 2024-04-02T13:38:47.985 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:38:47.985 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:38:47.985 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:47 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:38:47.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-02T13:38:47.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: Standby manager daemon y restarted 2024-04-02T13:38:47.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: Standby manager daemon y started 2024-04-02T13:38:47.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.50:0/2308805382' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.50:0/2308805382' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.50:0/2308805382' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.? 172.21.15.50:0/2308805382' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: Active manager daemon x restarted 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: Activating manager daemon x 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: mgrmap e21: x(active, starting, since 0.0122392s), standbys: z, y 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:38:47.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:38:47.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: Manager daemon x is now available 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:38:47.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:47 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:38:48.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.057+0000 7f68bfb82640 1 -- 172.21.15.33:0/501840230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8101c10 msgr2=0x7f68b8102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.057+0000 7f68bfb82640 1 --2- 172.21.15.33:0/501840230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8101c10 0x7f68b8102090 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009900 tx=0x7f68a802bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:48.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.057+0000 7f68bfb82640 1 -- 172.21.15.33:0/501840230 shutdown_connections 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.057+0000 7f68bfb82640 1 --2- 172.21.15.33:0/501840230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68b81025d0 0x7f68b8108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.057+0000 7f68bfb82640 1 --2- 172.21.15.33:0/501840230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8101c10 0x7f68b8102090 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.057+0000 7f68bfb82640 1 --2- 172.21.15.33:0/501840230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68b8100a20 0x7f68b8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.057+0000 7f68bfb82640 1 -- 172.21.15.33:0/501840230 >> 172.21.15.33:0/501840230 conn(0x7f68b80fbf90 msgr2=0x7f68b80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.057+0000 7f68bfb82640 1 -- 172.21.15.33:0/501840230 shutdown_connections 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.057+0000 7f68bfb82640 1 -- 172.21.15.33:0/501840230 wait complete. 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bfb82640 1 Processor -- start 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bfb82640 1 -- start start 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bfb82640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68b8100a20 0x7f68b8196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bfb82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8101c10 0x7f68b8197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bfb82640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68b81025d0 0x7f68b819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bfb82640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68b810c680 con 0x7f68b8101c10 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bfb82640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f68b810c500 con 0x7f68b81025d0 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bfb82640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f68b810c800 con 0x7f68b8100a20 2024-04-02T13:38:48.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bd11d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8101c10 0x7f68b8197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bd11d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8101c10 0x7f68b8197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59150/0 (socket says 172.21.15.33:59150) 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bd11d640 1 -- 172.21.15.33:0/2460903838 learned_addr learned my addr 172.21.15.33:0/2460903838 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bd91e640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68b8100a20 0x7f68b8196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68be11f640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68b81025d0 0x7f68b819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bd11d640 1 -- 172.21.15.33:0/2460903838 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68b8100a20 msgr2=0x7f68b8196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bd11d640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68b8100a20 0x7f68b8196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bd11d640 1 -- 172.21.15.33:0/2460903838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68b81025d0 msgr2=0x7f68b819e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bd11d640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68b81025d0 0x7f68b819e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.058+0000 7f68bd11d640 1 -- 172.21.15.33:0/2460903838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b819e960 con 0x7f68b8101c10 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.059+0000 7f68bd11d640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8101c10 0x7f68b8197290 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009230 tx=0x7f68a8004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.059+0000 7f68aeffd640 1 -- 172.21.15.33:0/2460903838 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8004f90 con 0x7f68b8101c10 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.059+0000 7f68aeffd640 1 -- 172.21.15.33:0/2460903838 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f68a8005600 con 0x7f68b8101c10 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.059+0000 7f68aeffd640 1 -- 172.21.15.33:0/2460903838 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a80476e0 con 0x7f68b8101c10 2024-04-02T13:38:48.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.059+0000 7f68bfb82640 1 -- 172.21.15.33:0/2460903838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b8068c60 con 0x7f68b8101c10 2024-04-02T13:38:48.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.059+0000 7f68bfb82640 1 -- 172.21.15.33:0/2460903838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b80691b0 con 0x7f68b8101c10 2024-04-02T13:38:48.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.060+0000 7f68bfb82640 1 -- 172.21.15.33:0/2460903838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b80688f0 con 0x7f68b8101c10 2024-04-02T13:38:48.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.061+0000 7f68aeffd640 1 -- 172.21.15.33:0/2460903838 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f68a8003840 con 0x7f68b8101c10 2024-04-02T13:38:48.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.061+0000 7f68aeffd640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f68a40a39b0 0x7f68a40a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:48.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.062+0000 7f68bd91e640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f68a40a39b0 0x7f68a40a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:48.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.062+0000 7f68aeffd640 1 -- 172.21.15.33:0/2460903838 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f68a80f4a50 con 0x7f68b8101c10 2024-04-02T13:38:48.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.062+0000 7f68bd91e640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f68a40a39b0 0x7f68a40a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f68b8198220 tx=0x7f68b400a2d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:48.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.067+0000 7f68aeffd640 1 -- 172.21.15.33:0/2460903838 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f68a80c1290 con 0x7f68b8101c10 2024-04-02T13:38:48.243 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.898+0000 7f54dc0de700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:48.243 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.898+0000 7f54dc0de700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:48.243 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.898+0000 7f54dc0de700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:48.243 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.898+0000 7f54dc0de700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:48.243 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:38:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:38:47.898+0000 7f54dc0de700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:38:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.259+0000 7f68bfb82640 1 -- 172.21.15.33:0/2460903838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68b8107800 con 0x7f68b8101c10 2024-04-02T13:38:48.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.261+0000 7f68aeffd640 1 -- 172.21.15.33:0/2460903838 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24996 (secure 0 0 0) 0x7f68a80c0b90 con 0x7f68b8101c10 2024-04-02T13:38:48.265 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:48.265 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":95,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:38:46.867341+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/80287983":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/395956530":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3947066882":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1212609914":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:38:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.263+0000 7f68bfb82640 1 -- 172.21.15.33:0/2460903838 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f68a40a39b0 msgr2=0x7f68a40a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.263+0000 7f68bfb82640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f68a40a39b0 0x7f68a40a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f68b8198220 tx=0x7f68b400a2d0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.263+0000 7f68bfb82640 1 -- 172.21.15.33:0/2460903838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8101c10 msgr2=0x7f68b8197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.263+0000 7f68bfb82640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8101c10 0x7f68b8197290 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009230 tx=0x7f68a8004ea0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.263+0000 7f68bfb82640 1 -- 172.21.15.33:0/2460903838 shutdown_connections 2024-04-02T13:38:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.263+0000 7f68bfb82640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f68a40a39b0 0x7f68a40a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.263+0000 7f68bfb82640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68b81025d0 0x7f68b819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.263+0000 7f68bfb82640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8101c10 0x7f68b8197290 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.263+0000 7f68bfb82640 1 --2- 172.21.15.33:0/2460903838 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68b8100a20 0x7f68b8196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.263+0000 7f68bfb82640 1 -- 172.21.15.33:0/2460903838 >> 172.21.15.33:0/2460903838 conn(0x7f68b80fbf90 msgr2=0x7f68b8104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:48.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.263+0000 7f68bfb82640 1 -- 172.21.15.33:0/2460903838 shutdown_connections 2024-04-02T13:38:48.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.263+0000 7f68bfb82640 1 -- 172.21.15.33:0/2460903838 wait complete. 2024-04-02T13:38:48.278 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:36:19.501121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:36:20.161569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:36:23.482134+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:38:48.279 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:38:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.381+0000 7fe6f203c640 1 -- 172.21.15.33:0/617029227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6ec101b30 msgr2=0x7fe6ec101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.381+0000 7fe6f203c640 1 --2- 172.21.15.33:0/617029227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6ec101b30 0x7fe6ec101f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009ca0 tx=0x7fe6dc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 -- 172.21.15.33:0/617029227 shutdown_connections 2024-04-02T13:38:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 --2- 172.21.15.33:0/617029227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec1036e0 0x7fe6ec109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 --2- 172.21.15.33:0/617029227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6ec102d20 0x7fe6ec1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 --2- 172.21.15.33:0/617029227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6ec101b30 0x7fe6ec101f50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 -- 172.21.15.33:0/617029227 >> 172.21.15.33:0/617029227 conn(0x7fe6ec0fd0a0 msgr2=0x7fe6ec0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 -- 172.21.15.33:0/617029227 shutdown_connections 2024-04-02T13:38:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 -- 172.21.15.33:0/617029227 wait complete. 2024-04-02T13:38:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 Processor -- start 2024-04-02T13:38:48.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 -- start start 2024-04-02T13:38:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec101b30 0x7fe6ec197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6ec102d20 0x7fe6ec198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6ec1036e0 0x7fe6ec19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec10d790 con 0x7fe6ec101b30 2024-04-02T13:38:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec10d610 con 0x7fe6ec102d20 2024-04-02T13:38:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.382+0000 7fe6f203c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec10d910 con 0x7fe6ec1036e0 2024-04-02T13:38:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6eb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec101b30 0x7fe6ec197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6eb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec101b30 0x7fe6ec197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59170/0 (socket says 172.21.15.33:59170) 2024-04-02T13:38:48.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6eb7fe640 1 -- 172.21.15.33:0/2251031258 learned_addr learned my addr 172.21.15.33:0/2251031258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6eaffd640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6ec102d20 0x7fe6ec198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6ebfff640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6ec1036e0 0x7fe6ec19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6eb7fe640 1 -- 172.21.15.33:0/2251031258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6ec1036e0 msgr2=0x7fe6ec19f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6eb7fe640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6ec1036e0 0x7fe6ec19f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6eb7fe640 1 -- 172.21.15.33:0/2251031258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6ec102d20 msgr2=0x7fe6ec198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6eb7fe640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6ec102d20 0x7fe6ec198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6eb7fe640 1 -- 172.21.15.33:0/2251031258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec19fae0 con 0x7fe6ec101b30 2024-04-02T13:38:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6eb7fe640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec101b30 0x7fe6ec197ec0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc02c630 tx=0x7fe6dc032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6e8ff9640 1 -- 172.21.15.33:0/2251031258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc039050 con 0x7fe6ec101b30 2024-04-02T13:38:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6e8ff9640 1 -- 172.21.15.33:0/2251031258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe6dc03a040 con 0x7fe6ec101b30 2024-04-02T13:38:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6f203c640 1 -- 172.21.15.33:0/2251031258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ec19fd70 con 0x7fe6ec101b30 2024-04-02T13:38:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6e8ff9640 1 -- 172.21.15.33:0/2251031258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc040450 con 0x7fe6ec101b30 2024-04-02T13:38:48.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.383+0000 7fe6f203c640 1 -- 172.21.15.33:0/2251031258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ec1a01b0 con 0x7fe6ec101b30 2024-04-02T13:38:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.384+0000 7fe6f203c640 1 -- 172.21.15.33:0/2251031258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6ec108ec0 con 0x7fe6ec101b30 2024-04-02T13:38:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.385+0000 7fe6e8ff9640 1 -- 172.21.15.33:0/2251031258 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fe6dc004040 con 0x7fe6ec101b30 2024-04-02T13:38:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.386+0000 7fe6e8ff9640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe6d80a39b0 0x7fe6d80a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.386+0000 7fe6eaffd640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe6d80a39b0 0x7fe6d80a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:48.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.386+0000 7fe6e8ff9640 1 -- 172.21.15.33:0/2251031258 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe6dc0f3110 con 0x7fe6ec101b30 2024-04-02T13:38:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.386+0000 7fe6eaffd640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe6d80a39b0 0x7fe6d80a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec1993d0 tx=0x7fe6d400b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:48.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.391+0000 7fe6e8ff9640 1 -- 172.21.15.33:0/2251031258 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe6dc0bf950 con 0x7fe6ec101b30 2024-04-02T13:38:48.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.578+0000 7fe6f203c640 1 -- 172.21.15.33:0/2251031258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe6ec0627a0 con 0x7fe6ec101b30 2024-04-02T13:38:48.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.579+0000 7fe6e8ff9640 1 -- 172.21.15.33:0/2251031258 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fe6dc0098b0 con 0x7fe6ec101b30 2024-04-02T13:38:48.582 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:38:48.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.581+0000 7fe6f203c640 1 -- 172.21.15.33:0/2251031258 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe6d80a39b0 msgr2=0x7fe6d80a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.581+0000 7fe6f203c640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe6d80a39b0 0x7fe6d80a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec1993d0 tx=0x7fe6d400b390 comp rx=0 tx=0).stop 2024-04-02T13:38:48.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.581+0000 7fe6f203c640 1 -- 172.21.15.33:0/2251031258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec101b30 msgr2=0x7fe6ec197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.581+0000 7fe6f203c640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec101b30 0x7fe6ec197ec0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc02c630 tx=0x7fe6dc032980 comp rx=0 tx=0).stop 2024-04-02T13:38:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.581+0000 7fe6f203c640 1 -- 172.21.15.33:0/2251031258 shutdown_connections 2024-04-02T13:38:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.581+0000 7fe6f203c640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe6d80a39b0 0x7fe6d80a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.581+0000 7fe6f203c640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe6ec1036e0 0x7fe6ec19f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.581+0000 7fe6f203c640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6ec102d20 0x7fe6ec198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.581+0000 7fe6f203c640 1 --2- 172.21.15.33:0/2251031258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec101b30 0x7fe6ec197ec0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.581+0000 7fe6f203c640 1 -- 172.21.15.33:0/2251031258 >> 172.21.15.33:0/2251031258 conn(0x7fe6ec0fd0a0 msgr2=0x7fe6ec106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.581+0000 7fe6f203c640 1 -- 172.21.15.33:0/2251031258 shutdown_connections 2024-04-02T13:38:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.581+0000 7fe6f203c640 1 -- 172.21.15.33:0/2251031258 wait complete. 2024-04-02T13:38:48.597 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:38:48.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.709+0000 7f8e4378d640 1 -- 172.21.15.33:0/3899810241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e3c104eb0 msgr2=0x7f8e3c1072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.709+0000 7f8e4378d640 1 --2- 172.21.15.33:0/3899810241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e3c104eb0 0x7f8e3c1072a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009900 tx=0x7f8e2c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.709+0000 7f8e4378d640 1 -- 172.21.15.33:0/3899810241 shutdown_connections 2024-04-02T13:38:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.709+0000 7f8e4378d640 1 --2- 172.21.15.33:0/3899810241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e3c1077e0 0x7f8e3c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.709+0000 7f8e4378d640 1 --2- 172.21.15.33:0/3899810241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e3c104eb0 0x7f8e3c1072a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.709+0000 7f8e4378d640 1 --2- 172.21.15.33:0/3899810241 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e3c102580 0x7f8e3c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.709+0000 7f8e4378d640 1 -- 172.21.15.33:0/3899810241 >> 172.21.15.33:0/3899810241 conn(0x7f8e3c0fbf90 msgr2=0x7f8e3c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.709+0000 7f8e4378d640 1 -- 172.21.15.33:0/3899810241 shutdown_connections 2024-04-02T13:38:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.709+0000 7f8e4378d640 1 -- 172.21.15.33:0/3899810241 wait complete. 2024-04-02T13:38:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.710+0000 7f8e4378d640 1 Processor -- start 2024-04-02T13:38:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.710+0000 7f8e4378d640 1 -- start start 2024-04-02T13:38:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.710+0000 7f8e4378d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e3c102580 0x7f8e3c101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.710+0000 7f8e4378d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e3c104eb0 0x7f8e3c100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.710+0000 7f8e4378d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e3c1077e0 0x7f8e3c100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.710+0000 7f8e4378d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c10d790 con 0x7f8e3c1077e0 2024-04-02T13:38:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.710+0000 7f8e4378d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c10d610 con 0x7f8e3c104eb0 2024-04-02T13:38:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.710+0000 7f8e4378d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c10d910 con 0x7f8e3c102580 2024-04-02T13:38:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.710+0000 7f8e41d2a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e3c1077e0 0x7f8e3c100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.710+0000 7f8e41d2a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e3c1077e0 0x7f8e3c100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59194/0 (socket says 172.21.15.33:59194) 2024-04-02T13:38:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.710+0000 7f8e41d2a640 1 -- 172.21.15.33:0/3865224050 learned_addr learned my addr 172.21.15.33:0/3865224050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.710+0000 7f8e41529640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e3c102580 0x7f8e3c101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.710+0000 7f8e40d28640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e3c104eb0 0x7f8e3c100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.711+0000 7f8e41d2a640 1 -- 172.21.15.33:0/3865224050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e3c102580 msgr2=0x7f8e3c101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.711+0000 7f8e41d2a640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e3c102580 0x7f8e3c101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.711+0000 7f8e41d2a640 1 -- 172.21.15.33:0/3865224050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e3c104eb0 msgr2=0x7f8e3c100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.711+0000 7f8e41d2a640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e3c104eb0 0x7f8e3c100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.711+0000 7f8e41d2a640 1 -- 172.21.15.33:0/3865224050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c101300 con 0x7f8e3c1077e0 2024-04-02T13:38:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.711+0000 7f8e41529640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e3c102580 0x7f8e3c101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.711+0000 7f8e41d2a640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e3c1077e0 0x7f8e3c100a70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400eb10 tx=0x7f8e3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.711+0000 7f8e327fc640 1 -- 172.21.15.33:0/3865224050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34004b20 con 0x7f8e3c1077e0 2024-04-02T13:38:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.711+0000 7f8e4378d640 1 -- 172.21.15.33:0/3865224050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3c1a1f60 con 0x7f8e3c1077e0 2024-04-02T13:38:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.711+0000 7f8e327fc640 1 -- 172.21.15.33:0/3865224050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8e34004cc0 con 0x7f8e3c1077e0 2024-04-02T13:38:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.711+0000 7f8e327fc640 1 -- 172.21.15.33:0/3865224050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34010700 con 0x7f8e3c1077e0 2024-04-02T13:38:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.711+0000 7f8e4378d640 1 -- 172.21.15.33:0/3865224050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3c1a23c0 con 0x7f8e3c1077e0 2024-04-02T13:38:48.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.712+0000 7f8e4378d640 1 -- 172.21.15.33:0/3865224050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3c0fda80 con 0x7f8e3c1077e0 2024-04-02T13:38:48.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.713+0000 7f8e327fc640 1 -- 172.21.15.33:0/3865224050 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f8e34004e70 con 0x7f8e3c1077e0 2024-04-02T13:38:48.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.714+0000 7f8e327fc640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8e280a39b0 0x7f8e280a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:48.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.714+0000 7f8e327fc640 1 -- 172.21.15.33:0/3865224050 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8e340d09a0 con 0x7f8e3c1077e0 2024-04-02T13:38:48.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.714+0000 7f8e41529640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8e280a39b0 0x7f8e280a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:48.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.715+0000 7f8e41529640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8e280a39b0 0x7f8e280a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800bfd0 tx=0x7f8e3800b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:48.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.722+0000 7f8e327fc640 1 -- 172.21.15.33:0/3865224050 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8e3409d1e0 con 0x7f8e3c1077e0 2024-04-02T13:38:48.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.892+0000 7f8e327fc640 1 -- 172.21.15.33:0/3865224050 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f8e34099150 con 0x7f8e3c1077e0 2024-04-02T13:38:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.939+0000 7f8e4378d640 1 -- 172.21.15.33:0/3865224050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8e3c062700 con 0x7f8e3c1077e0 2024-04-02T13:38:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.940+0000 7f8e327fc640 1 -- 172.21.15.33:0/3865224050 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f8e34065c60 con 0x7f8e3c1077e0 2024-04-02T13:38:48.943 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.942+0000 7f8e4378d640 1 -- 172.21.15.33:0/3865224050 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8e280a39b0 msgr2=0x7f8e280a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.942+0000 7f8e4378d640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8e280a39b0 0x7f8e280a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800bfd0 tx=0x7f8e3800b390 comp rx=0 tx=0).stop 2024-04-02T13:38:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.942+0000 7f8e4378d640 1 -- 172.21.15.33:0/3865224050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e3c1077e0 msgr2=0x7f8e3c100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.942+0000 7f8e4378d640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e3c1077e0 0x7f8e3c100a70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400eb10 tx=0x7f8e3400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.942+0000 7f8e4378d640 1 -- 172.21.15.33:0/3865224050 shutdown_connections 2024-04-02T13:38:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.942+0000 7f8e4378d640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8e280a39b0 0x7f8e280a5e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.942+0000 7f8e4378d640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e3c1077e0 0x7f8e3c100a70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.942+0000 7f8e4378d640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e3c104eb0 0x7f8e3c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.942+0000 7f8e4378d640 1 --2- 172.21.15.33:0/3865224050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e3c102580 0x7f8e3c101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.942+0000 7f8e4378d640 1 -- 172.21.15.33:0/3865224050 >> 172.21.15.33:0/3865224050 conn(0x7f8e3c0fbf90 msgr2=0x7f8e3c102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:48.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.942+0000 7f8e4378d640 1 -- 172.21.15.33:0/3865224050 shutdown_connections 2024-04-02T13:38:48.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:48.942+0000 7f8e4378d640 1 -- 172.21.15.33:0/3865224050 wait complete. 2024-04-02T13:38:48.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:38:49.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.081+0000 7fa5ca277640 1 -- 172.21.15.33:0/1157522260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5c41036e0 msgr2=0x7fa5c4109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:49.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.081+0000 7fa5ca277640 1 --2- 172.21.15.33:0/1157522260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5c41036e0 0x7fa5c4109f70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009ca0 tx=0x7fa5bc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.081+0000 7fa5ca277640 1 -- 172.21.15.33:0/1157522260 shutdown_connections 2024-04-02T13:38:49.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.081+0000 7fa5ca277640 1 --2- 172.21.15.33:0/1157522260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5c41036e0 0x7fa5c4109f70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.081+0000 7fa5ca277640 1 --2- 172.21.15.33:0/1157522260 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5c4102d20 0x7fa5c41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.081+0000 7fa5ca277640 1 --2- 172.21.15.33:0/1157522260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5c4101b30 0x7fa5c4101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.081+0000 7fa5ca277640 1 -- 172.21.15.33:0/1157522260 >> 172.21.15.33:0/1157522260 conn(0x7fa5c40fd0a0 msgr2=0x7fa5c40ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.082+0000 7fa5ca277640 1 -- 172.21.15.33:0/1157522260 shutdown_connections 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.082+0000 7fa5ca277640 1 -- 172.21.15.33:0/1157522260 wait complete. 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.082+0000 7fa5ca277640 1 Processor -- start 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.082+0000 7fa5ca277640 1 -- start start 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.082+0000 7fa5ca277640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5c4101b30 0x7fa5c4197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.082+0000 7fa5ca277640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5c4102d20 0x7fa5c41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.082+0000 7fa5ca277640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5c41036e0 0x7fa5c419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.082+0000 7fa5ca277640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410d790 con 0x7fa5c4102d20 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.082+0000 7fa5ca277640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410d610 con 0x7fa5c41036e0 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.082+0000 7fa5ca277640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410d910 con 0x7fa5c4101b30 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5c4102d20 0x7fa5c41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:49.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5c4102d20 0x7fa5c41982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59246/0 (socket says 172.21.15.33:59246) 2024-04-02T13:38:49.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c37fe640 1 -- 172.21.15.33:0/3611189374 learned_addr learned my addr 172.21.15.33:0/3611189374 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:49.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c8814640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5c41036e0 0x7fa5c419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:49.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c3fff640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5c4101b30 0x7fa5c4197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:49.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c37fe640 1 -- 172.21.15.33:0/3611189374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5c4101b30 msgr2=0x7fa5c4197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:49.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c37fe640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5c4101b30 0x7fa5c4197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c37fe640 1 -- 172.21.15.33:0/3611189374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5c41036e0 msgr2=0x7fa5c419f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:49.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c37fe640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5c41036e0 0x7fa5c419f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c37fe640 1 -- 172.21.15.33:0/3611189374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c419f9d0 con 0x7fa5c4102d20 2024-04-02T13:38:49.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c37fe640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5c4102d20 0x7fa5c41982f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400d950 tx=0x7fa5b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:49.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c17fa640 1 -- 172.21.15.33:0/3611189374 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4017030 con 0x7fa5c4102d20 2024-04-02T13:38:49.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5ca277640 1 -- 172.21.15.33:0/3611189374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c419fcc0 con 0x7fa5c4102d20 2024-04-02T13:38:49.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c17fa640 1 -- 172.21.15.33:0/3611189374 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa5b4004020 con 0x7fa5c4102d20 2024-04-02T13:38:49.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5c17fa640 1 -- 172.21.15.33:0/3611189374 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40047a0 con 0x7fa5c4102d20 2024-04-02T13:38:49.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.083+0000 7fa5ca277640 1 -- 172.21.15.33:0/3611189374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c41a01c0 con 0x7fa5c4102d20 2024-04-02T13:38:49.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.084+0000 7fa5ca277640 1 -- 172.21.15.33:0/3611189374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c4108e50 con 0x7fa5c4102d20 2024-04-02T13:38:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.085+0000 7fa5c17fa640 1 -- 172.21.15.33:0/3611189374 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fa5b40041d0 con 0x7fa5c4102d20 2024-04-02T13:38:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.086+0000 7fa5c17fa640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa5b00a3db0 0x7fa5b00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.086+0000 7fa5c3fff640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa5b00a3db0 0x7fa5b00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.086+0000 7fa5c17fa640 1 -- 172.21.15.33:0/3611189374 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa5b40d0d60 con 0x7fa5c4102d20 2024-04-02T13:38:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.087+0000 7fa5c3fff640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa5b00a3db0 0x7fa5b00a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4102b80 tx=0x7fa5ac005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.090+0000 7fa5c17fa640 1 -- 172.21.15.33:0/3611189374 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa5b409cee0 con 0x7fa5c4102d20 2024-04-02T13:38:49.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:48 smithi145 ceph-mon[35646]: mgrmap e22: x(active, since 1.02238s), standbys: z, y 2024-04-02T13:38:49.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:48 smithi145 ceph-mon[35646]: [02/Apr/2024:13:38:48] ENGINE Bus STARTING 2024-04-02T13:38:49.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:48 smithi145 ceph-mon[35646]: [02/Apr/2024:13:38:48] ENGINE Serving on https://172.21.15.33:7150 2024-04-02T13:38:49.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:48 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2460903838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:49.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:48 smithi145 ceph-mon[35646]: [02/Apr/2024:13:38:48] ENGINE Serving on http://172.21.15.33:8765 2024-04-02T13:38:49.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:48 smithi145 ceph-mon[35646]: [02/Apr/2024:13:38:48] ENGINE Bus STARTED 2024-04-02T13:38:49.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:48 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2251031258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:49.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:48 smithi050 ceph-mon[35524]: mgrmap e22: x(active, since 1.02238s), standbys: z, y 2024-04-02T13:38:49.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:48 smithi050 ceph-mon[35524]: [02/Apr/2024:13:38:48] ENGINE Bus STARTING 2024-04-02T13:38:49.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:48 smithi050 ceph-mon[35524]: [02/Apr/2024:13:38:48] ENGINE Serving on https://172.21.15.33:7150 2024-04-02T13:38:49.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:48 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2460903838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:49.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:48 smithi050 ceph-mon[35524]: [02/Apr/2024:13:38:48] ENGINE Serving on http://172.21.15.33:8765 2024-04-02T13:38:49.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:48 smithi050 ceph-mon[35524]: [02/Apr/2024:13:38:48] ENGINE Bus STARTED 2024-04-02T13:38:49.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:48 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2251031258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:49.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:48 smithi033 ceph-mon[32390]: mgrmap e22: x(active, since 1.02238s), standbys: z, y 2024-04-02T13:38:49.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:48 smithi033 ceph-mon[32390]: [02/Apr/2024:13:38:48] ENGINE Bus STARTING 2024-04-02T13:38:49.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:48 smithi033 ceph-mon[32390]: [02/Apr/2024:13:38:48] ENGINE Serving on https://172.21.15.33:7150 2024-04-02T13:38:49.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:48 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2460903838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:49.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:48 smithi033 ceph-mon[32390]: [02/Apr/2024:13:38:48] ENGINE Serving on http://172.21.15.33:8765 2024-04-02T13:38:49.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:48 smithi033 ceph-mon[32390]: [02/Apr/2024:13:38:48] ENGINE Bus STARTED 2024-04-02T13:38:49.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:48 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2251031258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.302+0000 7fa5ca277640 1 -- 172.21.15.33:0/3611189374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa5c402d370 con 0x7fa5c4102d20 2024-04-02T13:38:49.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.302+0000 7fa5c17fa640 1 -- 172.21.15.33:0/3611189374 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fa5b409c7e0 con 0x7fa5c4102d20 2024-04-02T13:38:49.305 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-02T13:38:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.305+0000 7fa5ca277640 1 -- 172.21.15.33:0/3611189374 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa5b00a3db0 msgr2=0x7fa5b00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.305+0000 7fa5ca277640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa5b00a3db0 0x7fa5b00a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4102b80 tx=0x7fa5ac005f90 comp rx=0 tx=0).stop 2024-04-02T13:38:49.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.305+0000 7fa5ca277640 1 -- 172.21.15.33:0/3611189374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5c4102d20 msgr2=0x7fa5c41982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.305+0000 7fa5ca277640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5c4102d20 0x7fa5c41982f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400d950 tx=0x7fa5b400de10 comp rx=0 tx=0).stop 2024-04-02T13:38:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.305+0000 7fa5ca277640 1 -- 172.21.15.33:0/3611189374 shutdown_connections 2024-04-02T13:38:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.305+0000 7fa5ca277640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa5b00a3db0 0x7fa5b00a6270 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.305+0000 7fa5ca277640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa5c41036e0 0x7fa5c419f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.305+0000 7fa5ca277640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5c4102d20 0x7fa5c41982f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.305+0000 7fa5ca277640 1 --2- 172.21.15.33:0/3611189374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5c4101b30 0x7fa5c4197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.305+0000 7fa5ca277640 1 -- 172.21.15.33:0/3611189374 >> 172.21.15.33:0/3611189374 conn(0x7fa5c40fd0a0 msgr2=0x7fa5c4106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.305+0000 7fa5ca277640 1 -- 172.21.15.33:0/3611189374 shutdown_connections 2024-04-02T13:38:49.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.305+0000 7fa5ca277640 1 -- 172.21.15.33:0/3611189374 wait complete. 2024-04-02T13:38:49.319 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:38:49.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.427+0000 7f2d6663a640 1 -- 172.21.15.33:0/2435510986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d60104f70 msgr2=0x7f2d60107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:49.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.427+0000 7f2d6663a640 1 --2- 172.21.15.33:0/2435510986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d60104f70 0x7f2d60107360 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009900 tx=0x7f2d5402bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:49.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.427+0000 7f2d6663a640 1 -- 172.21.15.33:0/2435510986 shutdown_connections 2024-04-02T13:38:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.427+0000 7f2d6663a640 1 --2- 172.21.15.33:0/2435510986 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d601078a0 0x7f2d60109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.427+0000 7f2d6663a640 1 --2- 172.21.15.33:0/2435510986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d60104f70 0x7f2d60107360 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.427+0000 7f2d6663a640 1 --2- 172.21.15.33:0/2435510986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d60102640 0x7f2d60104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.427+0000 7f2d6663a640 1 -- 172.21.15.33:0/2435510986 >> 172.21.15.33:0/2435510986 conn(0x7f2d600fbf90 msgr2=0x7f2d600fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.427+0000 7f2d6663a640 1 -- 172.21.15.33:0/2435510986 shutdown_connections 2024-04-02T13:38:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.427+0000 7f2d6663a640 1 -- 172.21.15.33:0/2435510986 wait complete. 2024-04-02T13:38:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d6663a640 1 Processor -- start 2024-04-02T13:38:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d6663a640 1 -- start start 2024-04-02T13:38:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d6663a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d60102640 0x7f2d60197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d6663a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d60104f70 0x7f2d60198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d6663a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d601078a0 0x7f2d6019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d6663a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6010d770 con 0x7f2d601078a0 2024-04-02T13:38:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d6663a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6010d5f0 con 0x7f2d60104f70 2024-04-02T13:38:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d6663a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6010d8f0 con 0x7f2d60102640 2024-04-02T13:38:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d64bd7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d601078a0 0x7f2d6019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d64bd7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d601078a0 0x7f2d6019f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59262/0 (socket says 172.21.15.33:59262) 2024-04-02T13:38:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d5ffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d60102640 0x7f2d60197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d64bd7640 1 -- 172.21.15.33:0/4288172239 learned_addr learned my addr 172.21.15.33:0/4288172239 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d5f7fe640 1 --2- 172.21.15.33:0/4288172239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d60104f70 0x7f2d60198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d64bd7640 1 -- 172.21.15.33:0/4288172239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d60102640 msgr2=0x7f2d60197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d64bd7640 1 --2- 172.21.15.33:0/4288172239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d60102640 0x7f2d60197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d64bd7640 1 -- 172.21.15.33:0/4288172239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d60104f70 msgr2=0x7f2d60198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d64bd7640 1 --2- 172.21.15.33:0/4288172239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d60104f70 0x7f2d60198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.428+0000 7f2d64bd7640 1 -- 172.21.15.33:0/4288172239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6019fac0 con 0x7f2d601078a0 2024-04-02T13:38:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.429+0000 7f2d64bd7640 1 --2- 172.21.15.33:0/4288172239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d601078a0 0x7f2d6019f450 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009fd0 tx=0x7f2d5800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.429+0000 7f2d5d7fa640 1 -- 172.21.15.33:0/4288172239 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58017070 con 0x7f2d601078a0 2024-04-02T13:38:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.429+0000 7f2d6663a640 1 -- 172.21.15.33:0/4288172239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d60100490 con 0x7f2d601078a0 2024-04-02T13:38:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.429+0000 7f2d5d7fa640 1 -- 172.21.15.33:0/4288172239 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2d58003e40 con 0x7f2d601078a0 2024-04-02T13:38:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.429+0000 7f2d5d7fa640 1 -- 172.21.15.33:0/4288172239 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d580106c0 con 0x7f2d601078a0 2024-04-02T13:38:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.429+0000 7f2d6663a640 1 -- 172.21.15.33:0/4288172239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d601009b0 con 0x7f2d601078a0 2024-04-02T13:38:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.430+0000 7f2d6663a640 1 -- 172.21.15.33:0/4288172239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d600fda80 con 0x7f2d601078a0 2024-04-02T13:38:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.431+0000 7f2d5d7fa640 1 -- 172.21.15.33:0/4288172239 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f2d58010860 con 0x7f2d601078a0 2024-04-02T13:38:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.432+0000 7f2d5d7fa640 1 --2- 172.21.15.33:0/4288172239 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2d480a3db0 0x7f2d480a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.432+0000 7f2d5ffff640 1 --2- 172.21.15.33:0/4288172239 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2d480a3db0 0x7f2d480a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.432+0000 7f2d5d7fa640 1 -- 172.21.15.33:0/4288172239 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2d580d0a60 con 0x7f2d601078a0 2024-04-02T13:38:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.432+0000 7f2d5ffff640 1 --2- 172.21.15.33:0/4288172239 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2d480a3db0 0x7f2d480a6270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009bd0 tx=0x7f2d50005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:49.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.441+0000 7f2d5d7fa640 1 -- 172.21.15.33:0/4288172239 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2d5809c590 con 0x7f2d601078a0 2024-04-02T13:38:49.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.681+0000 7f2d6663a640 1 -- 172.21.15.33:0/4288172239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2d60101d90 con 0x7f2d601078a0 2024-04-02T13:38:49.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.681+0000 7f2d5d7fa640 1 -- 172.21.15.33:0/4288172239 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f2d580a0890 con 0x7f2d601078a0 2024-04-02T13:38:49.684 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:49.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.684+0000 7f2d6663a640 1 -- 172.21.15.33:0/4288172239 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2d480a3db0 msgr2=0x7f2d480a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.684+0000 7f2d6663a640 1 --2- 172.21.15.33:0/4288172239 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2d480a3db0 0x7f2d480a6270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009bd0 tx=0x7f2d50005f90 comp rx=0 tx=0).stop 2024-04-02T13:38:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.684+0000 7f2d6663a640 1 -- 172.21.15.33:0/4288172239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d601078a0 msgr2=0x7f2d6019f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.684+0000 7f2d6663a640 1 --2- 172.21.15.33:0/4288172239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d601078a0 0x7f2d6019f450 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009fd0 tx=0x7f2d5800b260 comp rx=0 tx=0).stop 2024-04-02T13:38:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.684+0000 7f2d6663a640 1 -- 172.21.15.33:0/4288172239 shutdown_connections 2024-04-02T13:38:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.684+0000 7f2d6663a640 1 --2- 172.21.15.33:0/4288172239 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2d480a3db0 0x7f2d480a6270 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.684+0000 7f2d6663a640 1 --2- 172.21.15.33:0/4288172239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d601078a0 0x7f2d6019f450 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.684+0000 7f2d6663a640 1 --2- 172.21.15.33:0/4288172239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d60104f70 0x7f2d60198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.684+0000 7f2d6663a640 1 --2- 172.21.15.33:0/4288172239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d60102640 0x7f2d60197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.684+0000 7f2d6663a640 1 -- 172.21.15.33:0/4288172239 >> 172.21.15.33:0/4288172239 conn(0x7f2d600fbf90 msgr2=0x7f2d60103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.684+0000 7f2d6663a640 1 -- 172.21.15.33:0/4288172239 shutdown_connections 2024-04-02T13:38:49.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.684+0000 7f2d6663a640 1 -- 172.21.15.33:0/4288172239 wait complete. 2024-04-02T13:38:49.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:38:49.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.818+0000 7f00843db640 1 -- 172.21.15.33:0/3693820915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007c102640 msgr2=0x7f007c104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:49.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.818+0000 7f00843db640 1 --2- 172.21.15.33:0/3693820915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007c102640 0x7f007c104a30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0078009900 tx=0x7f007802bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:49.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.819+0000 7f00843db640 1 -- 172.21.15.33:0/3693820915 shutdown_connections 2024-04-02T13:38:49.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.819+0000 7f00843db640 1 --2- 172.21.15.33:0/3693820915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f007c1078a0 0x7f007c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.819+0000 7f00843db640 1 --2- 172.21.15.33:0/3693820915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f007c104f70 0x7f007c107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.819+0000 7f00843db640 1 --2- 172.21.15.33:0/3693820915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007c102640 0x7f007c104a30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.819+0000 7f00843db640 1 -- 172.21.15.33:0/3693820915 >> 172.21.15.33:0/3693820915 conn(0x7f007c0fbf90 msgr2=0x7f007c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:49.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.819+0000 7f00843db640 1 -- 172.21.15.33:0/3693820915 shutdown_connections 2024-04-02T13:38:49.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.819+0000 7f00843db640 1 -- 172.21.15.33:0/3693820915 wait complete. 2024-04-02T13:38:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.819+0000 7f00843db640 1 Processor -- start 2024-04-02T13:38:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f00843db640 1 -- start start 2024-04-02T13:38:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f00843db640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f007c102640 0x7f007c101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f00843db640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f007c104f70 0x7f007c1005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:49.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f00843db640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007c1078a0 0x7f007c100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f00843db640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f007c10d7a0 con 0x7f007c1078a0 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f00843db640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f007c10d620 con 0x7f007c104f70 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f00843db640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f007c10d920 con 0x7f007c102640 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f0082177640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f007c102640 0x7f007c101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f0082978640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007c1078a0 0x7f007c100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f0082177640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f007c102640 0x7f007c101fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:41882/0 (socket says 172.21.15.33:41882) 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f0082177640 1 -- 172.21.15.33:0/162554885 learned_addr learned my addr 172.21.15.33:0/162554885 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f0081976640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f007c104f70 0x7f007c1005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f0081976640 1 -- 172.21.15.33:0/162554885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f007c102640 msgr2=0x7f007c101fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f0081976640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f007c102640 0x7f007c101fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f0081976640 1 -- 172.21.15.33:0/162554885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007c1078a0 msgr2=0x7f007c100b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f0081976640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007c1078a0 0x7f007c100b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f0081976640 1 -- 172.21.15.33:0/162554885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007c1013c0 con 0x7f007c104f70 2024-04-02T13:38:49.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.820+0000 7f0082177640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f007c102640 0x7f007c101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:49.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.821+0000 7f0081976640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f007c104f70 0x7f007c1005f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f006c005fd0 tx=0x7f006c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:49.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.821+0000 7f00737fe640 1 -- 172.21.15.33:0/162554885 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c009c00 con 0x7f007c104f70 2024-04-02T13:38:49.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.821+0000 7f00737fe640 1 -- 172.21.15.33:0/162554885 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f006c009da0 con 0x7f007c104f70 2024-04-02T13:38:49.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.821+0000 7f00843db640 1 -- 172.21.15.33:0/162554885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007c1a1fa0 con 0x7f007c104f70 2024-04-02T13:38:49.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.821+0000 7f00737fe640 1 -- 172.21.15.33:0/162554885 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c010460 con 0x7f007c104f70 2024-04-02T13:38:49.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.821+0000 7f00843db640 1 -- 172.21.15.33:0/162554885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f007c1a2400 con 0x7f007c104f70 2024-04-02T13:38:49.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.822+0000 7f00843db640 1 -- 172.21.15.33:0/162554885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f007c0fda80 con 0x7f007c104f70 2024-04-02T13:38:49.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.823+0000 7f00737fe640 1 -- 172.21.15.33:0/162554885 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f006c003e90 con 0x7f007c104f70 2024-04-02T13:38:49.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.824+0000 7f00737fe640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f00680a3db0 0x7f00680a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:49.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.824+0000 7f00737fe640 1 -- 172.21.15.33:0/162554885 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f006c0d0830 con 0x7f007c104f70 2024-04-02T13:38:49.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.824+0000 7f0082177640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f00680a3db0 0x7f00680a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:49.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.824+0000 7f0082177640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f00680a3db0 0x7f00680a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f007802c710 tx=0x7f007802c820 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:49.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:49.829+0000 7f00737fe640 1 -- 172.21.15.33:0/162554885 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f006c017020 con 0x7f007c104f70 2024-04-02T13:38:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.022+0000 7f00843db640 1 -- 172.21.15.33:0/162554885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f007c02d570 con 0x7f007c104f70 2024-04-02T13:38:50.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.024+0000 7f00737fe640 1 -- 172.21.15.33:0/162554885 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24996 (secure 0 0 0) 0x7f006c09c3a0 con 0x7f007c104f70 2024-04-02T13:38:50.027 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:50.028 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":95,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:38:46.867341+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/80287983":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/395956530":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3947066882":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1212609914":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:38:50.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.026+0000 7f00843db640 1 -- 172.21.15.33:0/162554885 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f00680a3db0 msgr2=0x7f00680a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.026+0000 7f00843db640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f00680a3db0 0x7f00680a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f007802c710 tx=0x7f007802c820 comp rx=0 tx=0).stop 2024-04-02T13:38:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.026+0000 7f00843db640 1 -- 172.21.15.33:0/162554885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f007c104f70 msgr2=0x7f007c1005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.026+0000 7f00843db640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f007c104f70 0x7f007c1005f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f006c005fd0 tx=0x7f006c00dde0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.026+0000 7f00843db640 1 -- 172.21.15.33:0/162554885 shutdown_connections 2024-04-02T13:38:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.026+0000 7f00843db640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f00680a3db0 0x7f00680a6270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.026+0000 7f00843db640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f007c1078a0 0x7f007c100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.026+0000 7f00843db640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f007c104f70 0x7f007c1005f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.026+0000 7f00843db640 1 --2- 172.21.15.33:0/162554885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f007c102640 0x7f007c101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.026+0000 7f00843db640 1 -- 172.21.15.33:0/162554885 >> 172.21.15.33:0/162554885 conn(0x7f007c0fbf90 msgr2=0x7f007c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:50.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.026+0000 7f00843db640 1 -- 172.21.15.33:0/162554885 shutdown_connections 2024-04-02T13:38:50.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.026+0000 7f00843db640 1 -- 172.21.15.33:0/162554885 wait complete. 2024-04-02T13:38:50.042 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:36:19.501121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:36:20.161569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:36:23.482134+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:38:50.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:38:50.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.155+0000 7f086b59e640 1 -- 172.21.15.33:0/1850772905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f086c101a00 msgr2=0x7f086c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.155+0000 7f086b59e640 1 --2- 172.21.15.33:0/1850772905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f086c101a00 0x7f086c101e20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f085c009900 tx=0x7f085c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:50.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.155+0000 7f086b59e640 1 -- 172.21.15.33:0/1850772905 shutdown_connections 2024-04-02T13:38:50.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.155+0000 7f086b59e640 1 --2- 172.21.15.33:0/1850772905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f086c1035b0 0x7f086c109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.155+0000 7f086b59e640 1 --2- 172.21.15.33:0/1850772905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f086c102bf0 0x7f086c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.155+0000 7f086b59e640 1 --2- 172.21.15.33:0/1850772905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f086c101a00 0x7f086c101e20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.155+0000 7f086b59e640 1 -- 172.21.15.33:0/1850772905 >> 172.21.15.33:0/1850772905 conn(0x7f086c0fcf70 msgr2=0x7f086c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.155+0000 7f086b59e640 1 -- 172.21.15.33:0/1850772905 shutdown_connections 2024-04-02T13:38:50.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.155+0000 7f086b59e640 1 -- 172.21.15.33:0/1850772905 wait complete. 2024-04-02T13:38:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.156+0000 7f086b59e640 1 Processor -- start 2024-04-02T13:38:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.156+0000 7f086b59e640 1 -- start start 2024-04-02T13:38:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.156+0000 7f086b59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f086c101a00 0x7f086c197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.156+0000 7f086b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f086c102bf0 0x7f086c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.156+0000 7f086b59e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f086c1035b0 0x7f086c19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.156+0000 7f086b59e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f086c10d660 con 0x7f086c102bf0 2024-04-02T13:38:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.156+0000 7f086b59e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f086c10d4e0 con 0x7f086c101a00 2024-04-02T13:38:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.156+0000 7f086b59e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f086c10d7e0 con 0x7f086c1035b0 2024-04-02T13:38:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.156+0000 7f086a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f086c101a00 0x7f086c197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.156+0000 7f0869d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f086c102bf0 0x7f086c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.156+0000 7f086a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f086c101a00 0x7f086c197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:33462/0 (socket says 172.21.15.33:33462) 2024-04-02T13:38:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.156+0000 7f086a59c640 1 -- 172.21.15.33:0/1462762748 learned_addr learned my addr 172.21.15.33:0/1462762748 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.156+0000 7f086ad9d640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f086c1035b0 0x7f086c19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.157+0000 7f086a59c640 1 -- 172.21.15.33:0/1462762748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f086c1035b0 msgr2=0x7f086c19f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.157+0000 7f086a59c640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f086c1035b0 0x7f086c19f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.157+0000 7f086a59c640 1 -- 172.21.15.33:0/1462762748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f086c102bf0 msgr2=0x7f086c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.157+0000 7f086a59c640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f086c102bf0 0x7f086c198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.157+0000 7f086a59c640 1 -- 172.21.15.33:0/1462762748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086c19f930 con 0x7f086c101a00 2024-04-02T13:38:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.157+0000 7f0869d9b640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f086c102bf0 0x7f086c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.157+0000 7f086a59c640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f086c101a00 0x7f086c197d20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f085c02c880 tx=0x7f085c004e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.157+0000 7f085b7fe640 1 -- 172.21.15.33:0/1462762748 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c053070 con 0x7f086c101a00 2024-04-02T13:38:50.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.157+0000 7f085b7fe640 1 -- 172.21.15.33:0/1462762748 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f085c0044d0 con 0x7f086c101a00 2024-04-02T13:38:50.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.157+0000 7f085b7fe640 1 -- 172.21.15.33:0/1462762748 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c034440 con 0x7f086c101a00 2024-04-02T13:38:50.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.157+0000 7f086b59e640 1 -- 172.21.15.33:0/1462762748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086c073d30 con 0x7f086c101a00 2024-04-02T13:38:50.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.157+0000 7f086b59e640 1 -- 172.21.15.33:0/1462762748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086c074280 con 0x7f086c101a00 2024-04-02T13:38:50.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.158+0000 7f086b59e640 1 -- 172.21.15.33:0/1462762748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f086c108da0 con 0x7f086c101a00 2024-04-02T13:38:50.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.160+0000 7f085b7fe640 1 -- 172.21.15.33:0/1462762748 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f085c056070 con 0x7f086c101a00 2024-04-02T13:38:50.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.161+0000 7f085b7fe640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f08540a3db0 0x7f08540a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:50.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.161+0000 7f085b7fe640 1 -- 172.21.15.33:0/1462762748 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f085c103b50 con 0x7f086c101a00 2024-04-02T13:38:50.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.161+0000 7f0869d9b640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f08540a3db0 0x7f08540a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:50.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.162+0000 7f0869d9b640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f08540a3db0 0x7f08540a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0860005fd0 tx=0x7f086000b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:50.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.164+0000 7f085b7fe640 1 -- 172.21.15.33:0/1462762748 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f085c05d030 con 0x7f086c101a00 2024-04-02T13:38:50.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:49 smithi145 ceph-mon[35646]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:38:50.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:49 smithi145 ceph-mon[35646]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-02T13:38:50.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:49 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3865224050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:50.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:49 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:49 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:49 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:49 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:49 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:49 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:49 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3611189374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:50.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:49 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4288172239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:50.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:49 smithi050 ceph-mon[35524]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:38:50.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:49 smithi050 ceph-mon[35524]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-02T13:38:50.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:49 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3865224050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:50.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:49 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:49 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:49 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:49 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:49 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:49 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:49 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3611189374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:50.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:49 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4288172239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:50.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:49 smithi033 ceph-mon[32390]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:38:50.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:49 smithi033 ceph-mon[32390]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-02T13:38:50.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:49 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3865224050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:50.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:49 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:49 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:49 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:49 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:49 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:49 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:50.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:49 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3611189374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:50.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:49 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4288172239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:50.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.359+0000 7f086b59e640 1 -- 172.21.15.33:0/1462762748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f086c02d5c0 con 0x7f086c101a00 2024-04-02T13:38:50.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.360+0000 7f085b7fe640 1 -- 172.21.15.33:0/1462762748 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f085c0cfe20 con 0x7f086c101a00 2024-04-02T13:38:50.363 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:38:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.362+0000 7f086b59e640 1 -- 172.21.15.33:0/1462762748 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f08540a3db0 msgr2=0x7f08540a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.362+0000 7f086b59e640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f08540a3db0 0x7f08540a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0860005fd0 tx=0x7f086000b390 comp rx=0 tx=0).stop 2024-04-02T13:38:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.362+0000 7f086b59e640 1 -- 172.21.15.33:0/1462762748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f086c101a00 msgr2=0x7f086c197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.362+0000 7f086b59e640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f086c101a00 0x7f086c197d20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f085c02c880 tx=0x7f085c004e20 comp rx=0 tx=0).stop 2024-04-02T13:38:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.362+0000 7f086b59e640 1 -- 172.21.15.33:0/1462762748 shutdown_connections 2024-04-02T13:38:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.362+0000 7f086b59e640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f08540a3db0 0x7f08540a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.362+0000 7f086b59e640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f086c1035b0 0x7f086c19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.362+0000 7f086b59e640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f086c102bf0 0x7f086c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.362+0000 7f086b59e640 1 --2- 172.21.15.33:0/1462762748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f086c101a00 0x7f086c197d20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.362+0000 7f086b59e640 1 -- 172.21.15.33:0/1462762748 >> 172.21.15.33:0/1462762748 conn(0x7f086c0fcf70 msgr2=0x7f086c105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:50.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.362+0000 7f086b59e640 1 -- 172.21.15.33:0/1462762748 shutdown_connections 2024-04-02T13:38:50.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.362+0000 7f086b59e640 1 -- 172.21.15.33:0/1462762748 wait complete. 2024-04-02T13:38:50.377 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:38:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.487+0000 7f43a6dbc640 1 -- 172.21.15.33:0/2209011531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0102e70 msgr2=0x7f43a01032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.487+0000 7f43a6dbc640 1 --2- 172.21.15.33:0/2209011531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0102e70 0x7f43a01032f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f438c009930 tx=0x7f438c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.487+0000 7f43a6dbc640 1 -- 172.21.15.33:0/2209011531 shutdown_connections 2024-04-02T13:38:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.487+0000 7f43a6dbc640 1 --2- 172.21.15.33:0/2209011531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43a0103830 0x7f43a010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.487+0000 7f43a6dbc640 1 --2- 172.21.15.33:0/2209011531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0102e70 0x7f43a01032f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.487+0000 7f43a6dbc640 1 --2- 172.21.15.33:0/2209011531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a00ffe00 0x7f43a0100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.487+0000 7f43a6dbc640 1 -- 172.21.15.33:0/2209011531 >> 172.21.15.33:0/2209011531 conn(0x7f43a00fb6f0 msgr2=0x7f43a00fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.487+0000 7f43a6dbc640 1 -- 172.21.15.33:0/2209011531 shutdown_connections 2024-04-02T13:38:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.487+0000 7f43a6dbc640 1 -- 172.21.15.33:0/2209011531 wait complete. 2024-04-02T13:38:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f43a6dbc640 1 Processor -- start 2024-04-02T13:38:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f43a6dbc640 1 -- start start 2024-04-02T13:38:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f43a6dbc640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43a00ffe00 0x7f43a0197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f43a6dbc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a0102e70 0x7f43a0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f43a6dbc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0103830 0x7f43a019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f43a6dbc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43a010d790 con 0x7f43a0103830 2024-04-02T13:38:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f43a6dbc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43a010d610 con 0x7f43a0102e70 2024-04-02T13:38:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f43a6dbc640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f43a010d910 con 0x7f43a00ffe00 2024-04-02T13:38:50.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f43a5359640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0103830 0x7f43a019f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f439ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a0102e70 0x7f43a0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f43a5359640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0103830 0x7f43a019f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59306/0 (socket says 172.21.15.33:59306) 2024-04-02T13:38:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f43a5359640 1 -- 172.21.15.33:0/696990250 learned_addr learned my addr 172.21.15.33:0/696990250 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f43a4b58640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43a00ffe00 0x7f43a0197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f439ffff640 1 -- 172.21.15.33:0/696990250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43a00ffe00 msgr2=0x7f43a0197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f439ffff640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43a00ffe00 0x7f43a0197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f439ffff640 1 -- 172.21.15.33:0/696990250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0103830 msgr2=0x7f43a019f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.488+0000 7f439ffff640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0103830 0x7f43a019f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.489+0000 7f439ffff640 1 -- 172.21.15.33:0/696990250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a019fa60 con 0x7f43a0102e70 2024-04-02T13:38:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.489+0000 7f43a5359640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0103830 0x7f43a019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.489+0000 7f43a4b58640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43a00ffe00 0x7f43a0197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.489+0000 7f439ffff640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a0102e70 0x7f43a0198380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f438c02c170 tx=0x7f438c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:50.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.489+0000 7f439dffb640 1 -- 172.21.15.33:0/696990250 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c043020 con 0x7f43a0102e70 2024-04-02T13:38:50.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.489+0000 7f43a6dbc640 1 -- 172.21.15.33:0/696990250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a019fcf0 con 0x7f43a0102e70 2024-04-02T13:38:50.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.489+0000 7f439dffb640 1 -- 172.21.15.33:0/696990250 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f438c0054f0 con 0x7f43a0102e70 2024-04-02T13:38:50.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.489+0000 7f439dffb640 1 -- 172.21.15.33:0/696990250 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c003c30 con 0x7f43a0102e70 2024-04-02T13:38:50.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.489+0000 7f43a6dbc640 1 -- 172.21.15.33:0/696990250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a01a01b0 con 0x7f43a0102e70 2024-04-02T13:38:50.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.491+0000 7f43a6dbc640 1 -- 172.21.15.33:0/696990250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a0108fa0 con 0x7f43a0102e70 2024-04-02T13:38:50.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.492+0000 7f439dffb640 1 -- 172.21.15.33:0/696990250 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f438c046020 con 0x7f43a0102e70 2024-04-02T13:38:50.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.492+0000 7f439dffb640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43900a3db0 0x7f43900a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:50.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.493+0000 7f439dffb640 1 -- 172.21.15.33:0/696990250 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f438c0f2650 con 0x7f43a0102e70 2024-04-02T13:38:50.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.493+0000 7f43a4b58640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43900a3db0 0x7f43900a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:50.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.493+0000 7f43a4b58640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43900a3db0 0x7f43900a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f43a0069e10 tx=0x7f4394005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:50.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.497+0000 7f439dffb640 1 -- 172.21.15.33:0/696990250 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f438c04d070 con 0x7f43a0102e70 2024-04-02T13:38:50.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.717+0000 7f43a6dbc640 1 -- 172.21.15.33:0/696990250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f43a002d5b0 con 0x7f43a0102e70 2024-04-02T13:38:50.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.717+0000 7f439dffb640 1 -- 172.21.15.33:0/696990250 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f438c0be8e0 con 0x7f43a0102e70 2024-04-02T13:38:50.720 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:50.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.719+0000 7f43a6dbc640 1 -- 172.21.15.33:0/696990250 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43900a3db0 msgr2=0x7f43900a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.719+0000 7f43a6dbc640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43900a3db0 0x7f43900a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f43a0069e10 tx=0x7f4394005f90 comp rx=0 tx=0).stop 2024-04-02T13:38:50.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.719+0000 7f43a6dbc640 1 -- 172.21.15.33:0/696990250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a0102e70 msgr2=0x7f43a0198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.719+0000 7f43a6dbc640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a0102e70 0x7f43a0198380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f438c02c170 tx=0x7f438c005dc0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.719+0000 7f43a6dbc640 1 -- 172.21.15.33:0/696990250 shutdown_connections 2024-04-02T13:38:50.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.719+0000 7f43a6dbc640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43900a3db0 0x7f43900a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.719+0000 7f43a6dbc640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0103830 0x7f43a019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.719+0000 7f43a6dbc640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a0102e70 0x7f43a0198380 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.719+0000 7f43a6dbc640 1 --2- 172.21.15.33:0/696990250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43a00ffe00 0x7f43a0197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.719+0000 7f43a6dbc640 1 -- 172.21.15.33:0/696990250 >> 172.21.15.33:0/696990250 conn(0x7f43a00fb6f0 msgr2=0x7f43a0106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:50.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.719+0000 7f43a6dbc640 1 -- 172.21.15.33:0/696990250 shutdown_connections 2024-04-02T13:38:50.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.719+0000 7f43a6dbc640 1 -- 172.21.15.33:0/696990250 wait complete. 2024-04-02T13:38:50.734 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:38:50.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.840+0000 7f2c88b1c640 1 -- 172.21.15.33:0/2902141459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c841025b0 msgr2=0x7f2c84108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.840+0000 7f2c88b1c640 1 --2- 172.21.15.33:0/2902141459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c841025b0 0x7f2c84108e40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009ca0 tx=0x7f2c7c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 -- 172.21.15.33:0/2902141459 shutdown_connections 2024-04-02T13:38:50.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 --2- 172.21.15.33:0/2902141459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c841025b0 0x7f2c84108e40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 --2- 172.21.15.33:0/2902141459 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84101bf0 0x7f2c84102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 --2- 172.21.15.33:0/2902141459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c84100a00 0x7f2c84100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 -- 172.21.15.33:0/2902141459 >> 172.21.15.33:0/2902141459 conn(0x7f2c840fbf90 msgr2=0x7f2c840fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 -- 172.21.15.33:0/2902141459 shutdown_connections 2024-04-02T13:38:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 -- 172.21.15.33:0/2902141459 wait complete. 2024-04-02T13:38:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 Processor -- start 2024-04-02T13:38:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 -- start start 2024-04-02T13:38:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84100a00 0x7f2c84197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c84101bf0 0x7f2c84198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c841025b0 0x7f2c8419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8410c660 con 0x7f2c841025b0 2024-04-02T13:38:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8410c4e0 con 0x7f2c84101bf0 2024-04-02T13:38:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.841+0000 7f2c88b1c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8410c7e0 con 0x7f2c84100a00 2024-04-02T13:38:50.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c82d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c841025b0 0x7f2c8419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c7bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c84101bf0 0x7f2c84198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c7bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c84101bf0 0x7f2c84198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:33486/0 (socket says 172.21.15.33:33486) 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c8259c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84100a00 0x7f2c84197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c7bfff640 1 -- 172.21.15.33:0/3971081623 learned_addr learned my addr 172.21.15.33:0/3971081623 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c82d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c841025b0 0x7f2c8419f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59318/0 (socket says 172.21.15.33:59318) 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c8259c640 1 -- 172.21.15.33:0/3971081623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c84101bf0 msgr2=0x7f2c84198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c8259c640 1 --2- 172.21.15.33:0/3971081623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c84101bf0 0x7f2c84198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c8259c640 1 -- 172.21.15.33:0/3971081623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c841025b0 msgr2=0x7f2c8419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c8259c640 1 --2- 172.21.15.33:0/3971081623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c841025b0 0x7f2c8419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c8259c640 1 -- 172.21.15.33:0/3971081623 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8419fa50 con 0x7f2c84100a00 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c7bfff640 1 --2- 172.21.15.33:0/3971081623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c84101bf0 0x7f2c84198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c8259c640 1 --2- 172.21.15.33:0/3971081623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84100a00 0x7f2c84197e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009b90 tx=0x7f2c7400b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c7b7fe640 1 -- 172.21.15.33:0/3971081623 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400be60 con 0x7f2c84100a00 2024-04-02T13:38:50.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c88b1c640 1 -- 172.21.15.33:0/3971081623 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c84102e70 con 0x7f2c84100a00 2024-04-02T13:38:50.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c7b7fe640 1 -- 172.21.15.33:0/3971081623 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2c740035f0 con 0x7f2c84100a00 2024-04-02T13:38:50.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c7b7fe640 1 -- 172.21.15.33:0/3971081623 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74019670 con 0x7f2c84100a00 2024-04-02T13:38:50.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.842+0000 7f2c88b1c640 1 -- 172.21.15.33:0/3971081623 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c841033c0 con 0x7f2c84100a00 2024-04-02T13:38:50.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.843+0000 7f2c88b1c640 1 -- 172.21.15.33:0/3971081623 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c84103620 con 0x7f2c84100a00 2024-04-02T13:38:50.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.844+0000 7f2c7b7fe640 1 -- 172.21.15.33:0/3971081623 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f2c74020020 con 0x7f2c84100a00 2024-04-02T13:38:50.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.845+0000 7f2c7b7fe640 1 --2- 172.21.15.33:0/3971081623 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2c700a3db0 0x7f2c700a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:50.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.845+0000 7f2c7bfff640 1 --2- 172.21.15.33:0/3971081623 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2c700a3db0 0x7f2c700a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:50.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.845+0000 7f2c7b7fe640 1 -- 172.21.15.33:0/3971081623 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2c740d4800 con 0x7f2c84100a00 2024-04-02T13:38:50.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.846+0000 7f2c7bfff640 1 --2- 172.21.15.33:0/3971081623 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2c700a3db0 0x7f2c700a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c84199340 tx=0x7f2c6c005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:50.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:50.849+0000 7f2c7b7fe640 1 -- 172.21.15.33:0/3971081623 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2c740a09c0 con 0x7f2c84100a00 2024-04-02T13:38:51.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/162554885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:51.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi145 to 176.2M 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: Unable to set osd_memory_target on smithi145 to 184772403: error parsing value: Value '184772403' is below minimum 939524096 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-04-02T13:38:51.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: Unable to set osd_memory_target on smithi050 to 185898496: error parsing value: Value '185898496' is below minimum 939524096 2024-04-02T13:38:51.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:38:51.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:38:51.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-02T13:38:51.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: Updating smithi050:/etc/ceph/ceph.conf 2024-04-02T13:38:51.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: Updating smithi145:/etc/ceph/ceph.conf 2024-04-02T13:38:51.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1462762748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:51.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:38:51.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:38:51.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: Updating smithi033:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:38:51.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:50 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/696990250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:51.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.047+0000 7f2c88b1c640 1 -- 172.21.15.33:0/3971081623 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2c84103ee0 con 0x7f2c84100a00 2024-04-02T13:38:51.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.048+0000 7f2c7b7fe640 1 -- 172.21.15.33:0/3971081623 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f2c740a02c0 con 0x7f2c84100a00 2024-04-02T13:38:51.051 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-02T13:38:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.050+0000 7f2c88b1c640 1 -- 172.21.15.33:0/3971081623 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2c700a3db0 msgr2=0x7f2c700a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.050+0000 7f2c88b1c640 1 --2- 172.21.15.33:0/3971081623 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2c700a3db0 0x7f2c700a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c84199340 tx=0x7f2c6c005f90 comp rx=0 tx=0).stop 2024-04-02T13:38:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.050+0000 7f2c88b1c640 1 -- 172.21.15.33:0/3971081623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84100a00 msgr2=0x7f2c84197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.050+0000 7f2c88b1c640 1 --2- 172.21.15.33:0/3971081623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84100a00 0x7f2c84197e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009b90 tx=0x7f2c7400b690 comp rx=0 tx=0).stop 2024-04-02T13:38:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.050+0000 7f2c88b1c640 1 -- 172.21.15.33:0/3971081623 shutdown_connections 2024-04-02T13:38:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.050+0000 7f2c88b1c640 1 --2- 172.21.15.33:0/3971081623 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2c700a3db0 0x7f2c700a6270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.050+0000 7f2c88b1c640 1 --2- 172.21.15.33:0/3971081623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c841025b0 0x7f2c8419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.050+0000 7f2c88b1c640 1 --2- 172.21.15.33:0/3971081623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c84101bf0 0x7f2c84198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.050+0000 7f2c88b1c640 1 --2- 172.21.15.33:0/3971081623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c84100a00 0x7f2c84197e30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.050+0000 7f2c88b1c640 1 -- 172.21.15.33:0/3971081623 >> 172.21.15.33:0/3971081623 conn(0x7f2c840fbf90 msgr2=0x7f2c840fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.050+0000 7f2c88b1c640 1 -- 172.21.15.33:0/3971081623 shutdown_connections 2024-04-02T13:38:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.050+0000 7f2c88b1c640 1 -- 172.21.15.33:0/3971081623 wait complete. 2024-04-02T13:38:51.065 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:38:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.169+0000 7fad4788f640 1 -- 172.21.15.33:0/3662996081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad40107b50 msgr2=0x7fad40107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.169+0000 7fad4788f640 1 --2- 172.21.15.33:0/3662996081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad40107b50 0x7fad40107f30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009930 tx=0x7fad3c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 -- 172.21.15.33:0/3662996081 shutdown_connections 2024-04-02T13:38:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 --2- 172.21.15.33:0/3662996081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad400696c0 0x7fad4010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 --2- 172.21.15.33:0/3662996081 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad40068d00 0x7fad40069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 --2- 172.21.15.33:0/3662996081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad40107b50 0x7fad40107f30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 -- 172.21.15.33:0/3662996081 >> 172.21.15.33:0/3662996081 conn(0x7fad40077b00 msgr2=0x7fad40077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 -- 172.21.15.33:0/3662996081 shutdown_connections 2024-04-02T13:38:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 -- 172.21.15.33:0/3662996081 wait complete. 2024-04-02T13:38:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 Processor -- start 2024-04-02T13:38:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 -- start start 2024-04-02T13:38:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad40068d00 0x7fad4019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad400696c0 0x7fad4019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad40107b50 0x7fad4019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad40111ec0 con 0x7fad400696c0 2024-04-02T13:38:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fad40111d40 con 0x7fad40107b50 2024-04-02T13:38:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.170+0000 7fad4788f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fad40112040 con 0x7fad40068d00 2024-04-02T13:38:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad44e2a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad400696c0 0x7fad4019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:51.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad45e2c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad40107b50 0x7fad4019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:51.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad44e2a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad400696c0 0x7fad4019aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59346/0 (socket says 172.21.15.33:59346) 2024-04-02T13:38:51.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad44e2a640 1 -- 172.21.15.33:0/3637647845 learned_addr learned my addr 172.21.15.33:0/3637647845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:51.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad4562b640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad40068d00 0x7fad4019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:51.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad44e2a640 1 -- 172.21.15.33:0/3637647845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad40068d00 msgr2=0x7fad4019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad44e2a640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad40068d00 0x7fad4019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad44e2a640 1 -- 172.21.15.33:0/3637647845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad40107b50 msgr2=0x7fad4019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad44e2a640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad40107b50 0x7fad4019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad44e2a640 1 -- 172.21.15.33:0/3637647845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4019f550 con 0x7fad400696c0 2024-04-02T13:38:51.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad4562b640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad40068d00 0x7fad4019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:51.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad45e2c640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad40107b50 0x7fad4019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:51.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad44e2a640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad400696c0 0x7fad4019aaa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fad3000e9f0 tx=0x7fad3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:51.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad367fc640 1 -- 172.21.15.33:0/3637647845 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3000cd80 con 0x7fad400696c0 2024-04-02T13:38:51.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad4788f640 1 -- 172.21.15.33:0/3637647845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4019f840 con 0x7fad400696c0 2024-04-02T13:38:51.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad367fc640 1 -- 172.21.15.33:0/3637647845 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fad300054f0 con 0x7fad400696c0 2024-04-02T13:38:51.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad367fc640 1 -- 172.21.15.33:0/3637647845 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30010690 con 0x7fad400696c0 2024-04-02T13:38:51.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.171+0000 7fad4788f640 1 -- 172.21.15.33:0/3637647845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad401a70d0 con 0x7fad400696c0 2024-04-02T13:38:51.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.172+0000 7fad4788f640 1 -- 172.21.15.33:0/3637647845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad400688f0 con 0x7fad400696c0 2024-04-02T13:38:51.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.173+0000 7fad367fc640 1 -- 172.21.15.33:0/3637647845 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fad30004f20 con 0x7fad400696c0 2024-04-02T13:38:51.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.174+0000 7fad367fc640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fad2c0a3db0 0x7fad2c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:51.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.174+0000 7fad4562b640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fad2c0a3db0 0x7fad2c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:51.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.174+0000 7fad367fc640 1 -- 172.21.15.33:0/3637647845 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fad300d11b0 con 0x7fad400696c0 2024-04-02T13:38:51.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.175+0000 7fad4562b640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fad2c0a3db0 0x7fad2c0a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fad3c036040 tx=0x7fad3c005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:51.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.179+0000 7fad367fc640 1 -- 172.21.15.33:0/3637647845 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fad3009d370 con 0x7fad400696c0 2024-04-02T13:38:51.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/162554885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:51.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi145 to 176.2M 2024-04-02T13:38:51.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: Unable to set osd_memory_target on smithi145 to 184772403: error parsing value: Value '184772403' is below minimum 939524096 2024-04-02T13:38:51.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: Unable to set osd_memory_target on smithi050 to 185898496: error parsing value: Value '185898496' is below minimum 939524096 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:38:51.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:38:51.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-02T13:38:51.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: Updating smithi050:/etc/ceph/ceph.conf 2024-04-02T13:38:51.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: Updating smithi145:/etc/ceph/ceph.conf 2024-04-02T13:38:51.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1462762748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:51.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:38:51.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:38:51.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: Updating smithi033:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:38:51.231 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:50 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/696990250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:51.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/162554885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi145 to 176.2M 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: Unable to set osd_memory_target on smithi145 to 184772403: error parsing value: Value '184772403' is below minimum 939524096 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-02T13:38:51.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-02T13:38:51.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:51.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:38:51.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-04-02T13:38:51.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: Unable to set osd_memory_target on smithi050 to 185898496: error parsing value: Value '185898496' is below minimum 939524096 2024-04-02T13:38:51.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:38:51.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:38:51.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-02T13:38:51.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: Updating smithi050:/etc/ceph/ceph.conf 2024-04-02T13:38:51.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: Updating smithi145:/etc/ceph/ceph.conf 2024-04-02T13:38:51.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1462762748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:51.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:38:51.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:38:51.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: Updating smithi033:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:38:51.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:50 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/696990250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.316+0000 7fad367fc640 1 -- 172.21.15.33:0/3637647845 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fad300992e0 con 0x7fad400696c0 2024-04-02T13:38:51.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.382+0000 7fad4788f640 1 -- 172.21.15.33:0/3637647845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fad4002d370 con 0x7fad400696c0 2024-04-02T13:38:51.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.382+0000 7fad367fc640 1 -- 172.21.15.33:0/3637647845 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7fad3005cd50 con 0x7fad400696c0 2024-04-02T13:38:51.385 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.384+0000 7fad4788f640 1 -- 172.21.15.33:0/3637647845 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fad2c0a3db0 msgr2=0x7fad2c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.384+0000 7fad4788f640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fad2c0a3db0 0x7fad2c0a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fad3c036040 tx=0x7fad3c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.384+0000 7fad4788f640 1 -- 172.21.15.33:0/3637647845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad400696c0 msgr2=0x7fad4019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.384+0000 7fad4788f640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad400696c0 0x7fad4019aaa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fad3000e9f0 tx=0x7fad3000edf0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.384+0000 7fad4788f640 1 -- 172.21.15.33:0/3637647845 shutdown_connections 2024-04-02T13:38:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.384+0000 7fad4788f640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fad2c0a3db0 0x7fad2c0a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.384+0000 7fad4788f640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad40107b50 0x7fad4019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.384+0000 7fad4788f640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad400696c0 0x7fad4019aaa0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.384+0000 7fad4788f640 1 --2- 172.21.15.33:0/3637647845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad40068d00 0x7fad4019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.384+0000 7fad4788f640 1 -- 172.21.15.33:0/3637647845 >> 172.21.15.33:0/3637647845 conn(0x7fad40077b00 msgr2=0x7fad400fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.384+0000 7fad4788f640 1 -- 172.21.15.33:0/3637647845 shutdown_connections 2024-04-02T13:38:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.384+0000 7fad4788f640 1 -- 172.21.15.33:0/3637647845 wait complete. 2024-04-02T13:38:51.398 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-02T13:38:51.398 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:38:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.500+0000 7fcf24ce4640 1 -- 172.21.15.33:0/635802869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf20103650 msgr2=0x7fcf2010bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.501+0000 7fcf24ce4640 1 --2- 172.21.15.33:0/635802869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf20103650 0x7fcf2010bb30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009900 tx=0x7fcf0802bcd0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.501+0000 7fcf24ce4640 1 -- 172.21.15.33:0/635802869 shutdown_connections 2024-04-02T13:38:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.501+0000 7fcf24ce4640 1 --2- 172.21.15.33:0/635802869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf2010c100 0x7fcf2010e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.501+0000 7fcf24ce4640 1 --2- 172.21.15.33:0/635802869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf20103650 0x7fcf2010bb30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.501+0000 7fcf24ce4640 1 --2- 172.21.15.33:0/635802869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf20102d30 0x7fcf20103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.501+0000 7fcf24ce4640 1 -- 172.21.15.33:0/635802869 >> 172.21.15.33:0/635802869 conn(0x7fcf20076d80 msgr2=0x7fcf20077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.501+0000 7fcf24ce4640 1 -- 172.21.15.33:0/635802869 shutdown_connections 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.501+0000 7fcf24ce4640 1 -- 172.21.15.33:0/635802869 wait complete. 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.501+0000 7fcf24ce4640 1 Processor -- start 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.501+0000 7fcf24ce4640 1 -- start start 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf24ce4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf20102d30 0x7fcf20198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf24ce4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf20103650 0x7fcf20198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf24ce4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf2010c100 0x7fcf2019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf24ce4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf20111e60 con 0x7fcf20103650 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf24ce4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcf20111ce0 con 0x7fcf20102d30 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf24ce4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcf20111fe0 con 0x7fcf2010c100 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf20103650 0x7fcf20198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf1e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf20102d30 0x7fcf20198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf20103650 0x7fcf20198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59358/0 (socket says 172.21.15.33:59358) 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf17fff640 1 -- 172.21.15.33:0/4091482264 learned_addr learned my addr 172.21.15.33:0/4091482264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf1ed9d640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf2010c100 0x7fcf2019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf1e59c640 1 -- 172.21.15.33:0/4091482264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf2010c100 msgr2=0x7fcf2019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf1e59c640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf2010c100 0x7fcf2019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf1e59c640 1 -- 172.21.15.33:0/4091482264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf20103650 msgr2=0x7fcf20198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf1e59c640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf20103650 0x7fcf20198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf1e59c640 1 -- 172.21.15.33:0/4091482264 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2019d2e0 con 0x7fcf20102d30 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf17fff640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf20103650 0x7fcf20198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf1e59c640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf20102d30 0x7fcf20198340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000c970 tx=0x7fcf1000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf177fe640 1 -- 172.21.15.33:0/4091482264 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf10007bb0 con 0x7fcf20102d30 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf24ce4640 1 -- 172.21.15.33:0/4091482264 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2019d5d0 con 0x7fcf20102d30 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.502+0000 7fcf177fe640 1 -- 172.21.15.33:0/4091482264 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcf10007d50 con 0x7fcf20102d30 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.503+0000 7fcf177fe640 1 -- 172.21.15.33:0/4091482264 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf10018820 con 0x7fcf20102d30 2024-04-02T13:38:51.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.503+0000 7fcf24ce4640 1 -- 172.21.15.33:0/4091482264 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf201a4e60 con 0x7fcf20102d30 2024-04-02T13:38:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.503+0000 7fcf24ce4640 1 -- 172.21.15.33:0/4091482264 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf2006a950 con 0x7fcf20102d30 2024-04-02T13:38:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.505+0000 7fcf177fe640 1 -- 172.21.15.33:0/4091482264 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fcf100189c0 con 0x7fcf20102d30 2024-04-02T13:38:51.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.505+0000 7fcf177fe640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcf0c0a3e40 0x7fcf0c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:51.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.505+0000 7fcf17fff640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcf0c0a3e40 0x7fcf0c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:51.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.505+0000 7fcf177fe640 1 -- 172.21.15.33:0/4091482264 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcf100d0f60 con 0x7fcf20102d30 2024-04-02T13:38:51.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.506+0000 7fcf17fff640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcf0c0a3e40 0x7fcf0c0a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcf2010de70 tx=0x7fcf080058d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:51.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.509+0000 7fcf177fe640 1 -- 172.21.15.33:0/4091482264 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcf10014030 con 0x7fcf20102d30 2024-04-02T13:38:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.736+0000 7fcf24ce4640 1 -- 172.21.15.33:0/4091482264 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf20107250 con 0x7fcf20102d30 2024-04-02T13:38:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.737+0000 7fcf177fe640 1 -- 172.21.15.33:0/4091482264 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11873 (secure 0 0 0) 0x7fcf1009d150 con 0x7fcf20102d30 2024-04-02T13:38:51.740 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:51.740 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:38:45.531661+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24473,24512]},"id":1}]} 2024-04-02T13:38:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 25 2024-04-02T13:38:51.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.739+0000 7fcf24ce4640 1 -- 172.21.15.33:0/4091482264 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcf0c0a3e40 msgr2=0x7fcf0c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.739+0000 7fcf24ce4640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcf0c0a3e40 0x7fcf0c0a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcf2010de70 tx=0x7fcf080058d0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.739+0000 7fcf24ce4640 1 -- 172.21.15.33:0/4091482264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf20102d30 msgr2=0x7fcf20198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.739+0000 7fcf24ce4640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf20102d30 0x7fcf20198340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000c970 tx=0x7fcf1000ce30 comp rx=0 tx=0).stop 2024-04-02T13:38:51.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.739+0000 7fcf24ce4640 1 -- 172.21.15.33:0/4091482264 shutdown_connections 2024-04-02T13:38:51.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.739+0000 7fcf24ce4640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcf0c0a3e40 0x7fcf0c0a6300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.739+0000 7fcf24ce4640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcf2010c100 0x7fcf2019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.739+0000 7fcf24ce4640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf20103650 0x7fcf20198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.739+0000 7fcf24ce4640 1 --2- 172.21.15.33:0/4091482264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf20102d30 0x7fcf20198340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.739+0000 7fcf24ce4640 1 -- 172.21.15.33:0/4091482264 >> 172.21.15.33:0/4091482264 conn(0x7fcf20076d80 msgr2=0x7fcf200fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:51.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.739+0000 7fcf24ce4640 1 -- 172.21.15.33:0/4091482264 shutdown_connections 2024-04-02T13:38:51.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.739+0000 7fcf24ce4640 1 -- 172.21.15.33:0/4091482264 wait complete. 2024-04-02T13:38:51.753 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-02T13:38:51.753 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-02T13:38:51.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:38:51.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.853+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/1238744596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4101aa0 msgr2=0x7fd3e4101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.853+0000 7fd3e9c4a640 1 --2- 172.21.15.33:0/1238744596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4101aa0 0x7fd3e4101f20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009930 tx=0x7fd3cc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:51.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.853+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/1238744596 shutdown_connections 2024-04-02T13:38:51.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.853+0000 7fd3e9c4a640 1 --2- 172.21.15.33:0/1238744596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3e4102460 0x7fd3e410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.853+0000 7fd3e9c4a640 1 --2- 172.21.15.33:0/1238744596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4101aa0 0x7fd3e4101f20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.853+0000 7fd3e9c4a640 1 --2- 172.21.15.33:0/1238744596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3e4107ab0 0x7fd3e4107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.853+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/1238744596 >> 172.21.15.33:0/1238744596 conn(0x7fd3e40fd590 msgr2=0x7fd3e40ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.854+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/1238744596 shutdown_connections 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.854+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/1238744596 wait complete. 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.854+0000 7fd3e9c4a640 1 Processor -- start 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.854+0000 7fd3e9c4a640 1 -- start start 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.854+0000 7fd3e9c4a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3e4101aa0 0x7fd3e4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.854+0000 7fd3e9c4a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4102460 0x7fd3e41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.854+0000 7fd3e9c4a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3e4107ab0 0x7fd3e419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.854+0000 7fd3e9c4a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4111e90 con 0x7fd3e4102460 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.854+0000 7fd3e9c4a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4111d10 con 0x7fd3e4101aa0 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.854+0000 7fd3e9c4a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4112010 con 0x7fd3e4107ab0 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.854+0000 7fd3e2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4102460 0x7fd3e41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:51.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.854+0000 7fd3e2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4102460 0x7fd3e41988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59382/0 (socket says 172.21.15.33:59382) 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e2ffd640 1 -- 172.21.15.33:0/571827071 learned_addr learned my addr 172.21.15.33:0/571827071 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e37fe640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3e4101aa0 0x7fd3e4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e3fff640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3e4107ab0 0x7fd3e419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e2ffd640 1 -- 172.21.15.33:0/571827071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3e4107ab0 msgr2=0x7fd3e419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e2ffd640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3e4107ab0 0x7fd3e419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e2ffd640 1 -- 172.21.15.33:0/571827071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3e4101aa0 msgr2=0x7fd3e4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e2ffd640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3e4101aa0 0x7fd3e4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e2ffd640 1 -- 172.21.15.33:0/571827071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e419d360 con 0x7fd3e4102460 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e3fff640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3e4107ab0 0x7fd3e419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e2ffd640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4102460 0x7fd3e41988b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc000c00 tx=0x7fd3cc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e0ff9640 1 -- 172.21.15.33:0/571827071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc043020 con 0x7fd3e4102460 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/571827071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e419d5f0 con 0x7fd3e4102460 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e0ff9640 1 -- 172.21.15.33:0/571827071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd3cc02caa0 con 0x7fd3e4102460 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e0ff9640 1 -- 172.21.15.33:0/571827071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc003c20 con 0x7fd3e4102460 2024-04-02T13:38:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.855+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/571827071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e41a4ee0 con 0x7fd3e4102460 2024-04-02T13:38:51.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.856+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/571827071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e40688f0 con 0x7fd3e4102460 2024-04-02T13:38:51.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.857+0000 7fd3e0ff9640 1 -- 172.21.15.33:0/571827071 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd3cc046020 con 0x7fd3e4102460 2024-04-02T13:38:51.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.858+0000 7fd3e0ff9640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd3d00a3e40 0x7fd3d00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:51.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.858+0000 7fd3e37fe640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd3d00a3e40 0x7fd3d00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:51.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.858+0000 7fd3e0ff9640 1 -- 172.21.15.33:0/571827071 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd3cc0f30c0 con 0x7fd3e4102460 2024-04-02T13:38:51.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.858+0000 7fd3e37fe640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd3d00a3e40 0x7fd3d00a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc007e40 tx=0x7fd3dc0075b0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:51.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:51.862+0000 7fd3e0ff9640 1 -- 172.21.15.33:0/571827071 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd3cc04d070 con 0x7fd3e4102460 2024-04-02T13:38:52.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:38:52.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:38:52.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:38:52.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:38:52.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3971081623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:52.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:38:52.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:38:52.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: Updating smithi033:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3637647845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:51 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4091482264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:52.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.045+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/571827071 --> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd3e4061f40 con 0x7fd3d00a3e40 2024-04-02T13:38:52.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.055+0000 7fd3e0ff9640 1 -- 172.21.15.33:0/571827071 <== mgr.15456 v2:172.21.15.33:6800/619073725 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339690 (secure 0 0 0) 0x7fd3e4061f40 con 0x7fd3d00a3e40 2024-04-02T13:38:52.063 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:52.064 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-02T13:38:52.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.059+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/571827071 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd3d00a3e40 msgr2=0x7fd3d00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.059+0000 7fd3e9c4a640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd3d00a3e40 0x7fd3d00a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc007e40 tx=0x7fd3dc0075b0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.059+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/571827071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4102460 msgr2=0x7fd3e41988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.059+0000 7fd3e9c4a640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4102460 0x7fd3e41988b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc000c00 tx=0x7fd3cc0045e0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.059+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/571827071 shutdown_connections 2024-04-02T13:38:52.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.059+0000 7fd3e9c4a640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd3d00a3e40 0x7fd3d00a6300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.059+0000 7fd3e9c4a640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd3e4107ab0 0x7fd3e419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.059+0000 7fd3e9c4a640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4102460 0x7fd3e41988b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.059+0000 7fd3e9c4a640 1 --2- 172.21.15.33:0/571827071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3e4101aa0 0x7fd3e4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.059+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/571827071 >> 172.21.15.33:0/571827071 conn(0x7fd3e40fd590 msgr2=0x7fd3e40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:52.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.059+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/571827071 shutdown_connections 2024-04-02T13:38:52.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.059+0000 7fd3e9c4a640 1 -- 172.21.15.33:0/571827071 wait complete. 2024-04-02T13:38:52.072 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-02T13:38:50.885512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191445,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":995,"num_read_kb":662,"num_write":482,"num_write_kb":4390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":876,"ondisk_log_size":876,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374484,"kb_used_data":12812,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567340,"statfs":{"total":1151940427776,"available":1151556956160,"internally_reserved":0,"allocated":13119488,"data_stored":9930375,"data_compressed":541756,"data_compressed_allocated":733184,"data_compressed_original":4317184,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987578"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063585+0000","last_change":"2024-04-02T13:36:21.492436+0000","last_active":"2024-04-02T13:38:47.063585+0000","last_peered":"2024-04-02T13:38:47.063585+0000","last_clean":"2024-04-02T13:38:47.063585+0000","last_became_active":"2024-04-02T13:36:21.491831+0000","last_became_peered":"2024-04-02T13:36:21.491831+0000","last_unstale":"2024-04-02T13:38:47.063585+0000","last_undegraded":"2024-04-02T13:38:47.063585+0000","last_fullsized":"2024-04-02T13:38:47.063585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:50:57.919680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250790+0000","last_change":"2024-04-02T13:36:21.498089+0000","last_active":"2024-04-02T13:38:47.250790+0000","last_peered":"2024-04-02T13:38:47.250790+0000","last_clean":"2024-04-02T13:38:47.250790+0000","last_became_active":"2024-04-02T13:36:21.497784+0000","last_became_peered":"2024-04-02T13:36:21.497784+0000","last_unstale":"2024-04-02T13:38:47.250790+0000","last_undegraded":"2024-04-02T13:38:47.250790+0000","last_fullsized":"2024-04-02T13:38:47.250790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:42.137396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250172+0000","last_change":"2024-04-02T13:36:21.492207+0000","last_active":"2024-04-02T13:38:47.250172+0000","last_peered":"2024-04-02T13:38:47.250172+0000","last_clean":"2024-04-02T13:38:47.250172+0000","last_became_active":"2024-04-02T13:36:21.490605+0000","last_became_peered":"2024-04-02T13:36:21.490605+0000","last_unstale":"2024-04-02T13:38:47.250172+0000","last_undegraded":"2024-04-02T13:38:47.250172+0000","last_fullsized":"2024-04-02T13:38:47.250172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:03.984226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874419+0000","last_change":"2024-04-02T13:36:21.491225+0000","last_active":"2024-04-02T13:38:46.874419+0000","last_peered":"2024-04-02T13:38:46.874419+0000","last_clean":"2024-04-02T13:38:46.874419+0000","last_became_active":"2024-04-02T13:36:21.490886+0000","last_became_peered":"2024-04-02T13:36:21.490886+0000","last_unstale":"2024-04-02T13:38:46.874419+0000","last_undegraded":"2024-04-02T13:38:46.874419+0000","last_fullsized":"2024-04-02T13:38:46.874419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:55:53.316243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877507+0000","last_change":"2024-04-02T13:36:21.495808+0000","last_active":"2024-04-02T13:38:46.877507+0000","last_peered":"2024-04-02T13:38:46.877507+0000","last_clean":"2024-04-02T13:38:46.877507+0000","last_became_active":"2024-04-02T13:36:21.495468+0000","last_became_peered":"2024-04-02T13:36:21.495468+0000","last_unstale":"2024-04-02T13:38:46.877507+0000","last_undegraded":"2024-04-02T13:38:46.877507+0000","last_fullsized":"2024-04-02T13:38:46.877507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:17.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982665+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:38:46.982665+0000","last_peered":"2024-04-02T13:38:46.982665+0000","last_clean":"2024-04-02T13:38:46.982665+0000","last_became_active":"2024-04-02T13:36:21.499200+0000","last_became_peered":"2024-04-02T13:36:21.499200+0000","last_unstale":"2024-04-02T13:38:46.982665+0000","last_undegraded":"2024-04-02T13:38:46.982665+0000","last_fullsized":"2024-04-02T13:38:46.982665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:54:52.113844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062959+0000","last_change":"2024-04-02T13:36:21.493429+0000","last_active":"2024-04-02T13:38:47.062959+0000","last_peered":"2024-04-02T13:38:47.062959+0000","last_clean":"2024-04-02T13:38:47.062959+0000","last_became_active":"2024-04-02T13:36:21.493097+0000","last_became_peered":"2024-04-02T13:36:21.493097+0000","last_unstale":"2024-04-02T13:38:47.062959+0000","last_undegraded":"2024-04-02T13:38:47.062959+0000","last_fullsized":"2024-04-02T13:38:47.062959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:47:03.330448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.969983+0000","last_change":"2024-04-02T13:38:18.511280+0000","last_active":"2024-04-02T13:38:46.969983+0000","last_peered":"2024-04-02T13:38:46.969983+0000","last_clean":"2024-04-02T13:38:46.969983+0000","last_became_active":"2024-04-02T13:38:18.510953+0000","last_became_peered":"2024-04-02T13:38:18.510953+0000","last_unstale":"2024-04-02T13:38:46.969983+0000","last_undegraded":"2024-04-02T13:38:46.969983+0000","last_fullsized":"2024-04-02T13:38:46.969983+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:51:38.448445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063843+0000","last_change":"2024-04-02T13:36:21.490623+0000","last_active":"2024-04-02T13:38:47.063843+0000","last_peered":"2024-04-02T13:38:47.063843+0000","last_clean":"2024-04-02T13:38:47.063843+0000","last_became_active":"2024-04-02T13:36:21.490274+0000","last_became_peered":"2024-04-02T13:36:21.490274+0000","last_unstale":"2024-04-02T13:38:47.063843+0000","last_undegraded":"2024-04-02T13:38:47.063843+0000","last_fullsized":"2024-04-02T13:38:47.063843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:59:35.220430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982805+0000","last_change":"2024-04-02T13:37:17.631180+0000","last_active":"2024-04-02T13:38:46.982805+0000","last_peered":"2024-04-02T13:38:46.982805+0000","last_clean":"2024-04-02T13:38:46.982805+0000","last_became_active":"2024-04-02T13:37:17.630778+0000","last_became_peered":"2024-04-02T13:37:17.630778+0000","last_unstale":"2024-04-02T13:38:46.982805+0000","last_undegraded":"2024-04-02T13:38:46.982805+0000","last_fullsized":"2024-04-02T13:38:46.982805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:05:11.445074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063424+0000","last_change":"2024-04-02T13:36:21.492510+0000","last_active":"2024-04-02T13:38:47.063424+0000","last_peered":"2024-04-02T13:38:47.063424+0000","last_clean":"2024-04-02T13:38:47.063424+0000","last_became_active":"2024-04-02T13:36:21.492189+0000","last_became_peered":"2024-04-02T13:36:21.492189+0000","last_unstale":"2024-04-02T13:38:47.063424+0000","last_undegraded":"2024-04-02T13:38:47.063424+0000","last_fullsized":"2024-04-02T13:38:47.063424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:15.321621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874446+0000","last_change":"2024-04-02T13:36:21.498015+0000","last_active":"2024-04-02T13:38:46.874446+0000","last_peered":"2024-04-02T13:38:46.874446+0000","last_clean":"2024-04-02T13:38:46.874446+0000","last_became_active":"2024-04-02T13:36:21.497677+0000","last_became_peered":"2024-04-02T13:36:21.497677+0000","last_unstale":"2024-04-02T13:38:46.874446+0000","last_undegraded":"2024-04-02T13:38:46.874446+0000","last_fullsized":"2024-04-02T13:38:46.874446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:43:19.035902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063126+0000","last_change":"2024-04-02T13:36:21.497047+0000","last_active":"2024-04-02T13:38:47.063126+0000","last_peered":"2024-04-02T13:38:47.063126+0000","last_clean":"2024-04-02T13:38:47.063126+0000","last_became_active":"2024-04-02T13:36:21.496722+0000","last_became_peered":"2024-04-02T13:36:21.496722+0000","last_unstale":"2024-04-02T13:38:47.063126+0000","last_undegraded":"2024-04-02T13:38:47.063126+0000","last_fullsized":"2024-04-02T13:38:47.063126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:03:14.400687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250295+0000","last_change":"2024-04-02T13:36:21.496513+0000","last_active":"2024-04-02T13:38:47.250295+0000","last_peered":"2024-04-02T13:38:47.250295+0000","last_clean":"2024-04-02T13:38:47.250295+0000","last_became_active":"2024-04-02T13:36:21.496209+0000","last_became_peered":"2024-04-02T13:36:21.496209+0000","last_unstale":"2024-04-02T13:38:47.250295+0000","last_undegraded":"2024-04-02T13:38:47.250295+0000","last_fullsized":"2024-04-02T13:38:47.250295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:50.684206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250238+0000","last_change":"2024-04-02T13:36:21.493418+0000","last_active":"2024-04-02T13:38:47.250238+0000","last_peered":"2024-04-02T13:38:47.250238+0000","last_clean":"2024-04-02T13:38:47.250238+0000","last_became_active":"2024-04-02T13:36:21.492785+0000","last_became_peered":"2024-04-02T13:36:21.492785+0000","last_unstale":"2024-04-02T13:38:47.250238+0000","last_undegraded":"2024-04-02T13:38:47.250238+0000","last_fullsized":"2024-04-02T13:38:47.250238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:11.973683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062414+0000","last_change":"2024-04-02T13:36:21.490710+0000","last_active":"2024-04-02T13:38:47.062414+0000","last_peered":"2024-04-02T13:38:47.062414+0000","last_clean":"2024-04-02T13:38:47.062414+0000","last_became_active":"2024-04-02T13:36:21.489654+0000","last_became_peered":"2024-04-02T13:36:21.489654+0000","last_unstale":"2024-04-02T13:38:47.062414+0000","last_undegraded":"2024-04-02T13:38:47.062414+0000","last_fullsized":"2024-04-02T13:38:47.062414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:43:06.198540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063474+0000","last_change":"2024-04-02T13:36:21.493306+0000","last_active":"2024-04-02T13:38:47.063474+0000","last_peered":"2024-04-02T13:38:47.063474+0000","last_clean":"2024-04-02T13:38:47.063474+0000","last_became_active":"2024-04-02T13:36:21.492949+0000","last_became_peered":"2024-04-02T13:36:21.492949+0000","last_unstale":"2024-04-02T13:38:47.063474+0000","last_undegraded":"2024-04-02T13:38:47.063474+0000","last_fullsized":"2024-04-02T13:38:47.063474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:34.264790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970460+0000","last_change":"2024-04-02T13:36:21.496631+0000","last_active":"2024-04-02T13:38:46.970460+0000","last_peered":"2024-04-02T13:38:46.970460+0000","last_clean":"2024-04-02T13:38:46.970460+0000","last_became_active":"2024-04-02T13:36:21.496245+0000","last_became_peered":"2024-04-02T13:36:21.496245+0000","last_unstale":"2024-04-02T13:38:46.970460+0000","last_undegraded":"2024-04-02T13:38:46.970460+0000","last_fullsized":"2024-04-02T13:38:46.970460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:14:51.126381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063307+0000","last_change":"2024-04-02T13:36:21.491640+0000","last_active":"2024-04-02T13:38:47.063307+0000","last_peered":"2024-04-02T13:38:47.063307+0000","last_clean":"2024-04-02T13:38:47.063307+0000","last_became_active":"2024-04-02T13:36:21.487799+0000","last_became_peered":"2024-04-02T13:36:21.487799+0000","last_unstale":"2024-04-02T13:38:47.063307+0000","last_undegraded":"2024-04-02T13:38:47.063307+0000","last_fullsized":"2024-04-02T13:38:47.063307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:16.893295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062642+0000","last_change":"2024-04-02T13:36:25.499691+0000","last_active":"2024-04-02T13:38:47.062642+0000","last_peered":"2024-04-02T13:38:47.062642+0000","last_clean":"2024-04-02T13:38:47.062642+0000","last_became_active":"2024-04-02T13:36:25.499120+0000","last_became_peered":"2024-04-02T13:36:25.499120+0000","last_unstale":"2024-04-02T13:38:47.062642+0000","last_undegraded":"2024-04-02T13:38:47.062642+0000","last_fullsized":"2024-04-02T13:38:47.062642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:47:53.823456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970931+0000","last_change":"2024-04-02T13:37:17.878343+0000","last_active":"2024-04-02T13:38:46.970931+0000","last_peered":"2024-04-02T13:38:46.970931+0000","last_clean":"2024-04-02T13:38:46.970931+0000","last_became_active":"2024-04-02T13:37:17.871984+0000","last_became_peered":"2024-04-02T13:37:17.871984+0000","last_unstale":"2024-04-02T13:38:46.970931+0000","last_undegraded":"2024-04-02T13:38:46.970931+0000","last_fullsized":"2024-04-02T13:38:46.970931+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:28.804070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872842+0000","last_change":"2024-04-02T13:36:21.499019+0000","last_active":"2024-04-02T13:38:46.872842+0000","last_peered":"2024-04-02T13:38:46.872842+0000","last_clean":"2024-04-02T13:38:46.872842+0000","last_became_active":"2024-04-02T13:36:21.498684+0000","last_became_peered":"2024-04-02T13:36:21.498684+0000","last_unstale":"2024-04-02T13:38:46.872842+0000","last_undegraded":"2024-04-02T13:38:46.872842+0000","last_fullsized":"2024-04-02T13:38:46.872842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:15.618382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.873916+0000","last_change":"2024-04-02T13:36:25.498544+0000","last_active":"2024-04-02T13:38:46.873916+0000","last_peered":"2024-04-02T13:38:46.873916+0000","last_clean":"2024-04-02T13:38:46.873916+0000","last_became_active":"2024-04-02T13:36:25.498076+0000","last_became_peered":"2024-04-02T13:36:25.498076+0000","last_unstale":"2024-04-02T13:38:46.873916+0000","last_undegraded":"2024-04-02T13:38:46.873916+0000","last_fullsized":"2024-04-02T13:38:46.873916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:04:47.311056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.873130+0000","last_change":"2024-04-02T13:36:20.832649+0000","last_active":"2024-04-02T13:38:46.873130+0000","last_peered":"2024-04-02T13:38:46.873130+0000","last_clean":"2024-04-02T13:38:46.873130+0000","last_became_active":"2024-04-02T13:36:20.832184+0000","last_became_peered":"2024-04-02T13:36:20.832184+0000","last_unstale":"2024-04-02T13:38:46.873130+0000","last_undegraded":"2024-04-02T13:38:46.873130+0000","last_fullsized":"2024-04-02T13:38:46.873130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:37.639406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250670+0000","last_change":"2024-04-02T13:36:21.489988+0000","last_active":"2024-04-02T13:38:47.250670+0000","last_peered":"2024-04-02T13:38:47.250670+0000","last_clean":"2024-04-02T13:38:47.250670+0000","last_became_active":"2024-04-02T13:36:21.489636+0000","last_became_peered":"2024-04-02T13:36:21.489636+0000","last_unstale":"2024-04-02T13:38:47.250670+0000","last_undegraded":"2024-04-02T13:38:47.250670+0000","last_fullsized":"2024-04-02T13:38:47.250670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:52:36.670712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874051+0000","last_change":"2024-04-02T13:36:25.499441+0000","last_active":"2024-04-02T13:38:46.874051+0000","last_peered":"2024-04-02T13:38:46.874051+0000","last_clean":"2024-04-02T13:38:46.874051+0000","last_became_active":"2024-04-02T13:36:25.498512+0000","last_became_peered":"2024-04-02T13:36:25.498512+0000","last_unstale":"2024-04-02T13:38:46.874051+0000","last_undegraded":"2024-04-02T13:38:46.874051+0000","last_fullsized":"2024-04-02T13:38:46.874051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:48:59.291360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878544+0000","last_change":"2024-04-02T13:36:20.830008+0000","last_active":"2024-04-02T13:38:46.878544+0000","last_peered":"2024-04-02T13:38:46.878544+0000","last_clean":"2024-04-02T13:38:46.878544+0000","last_became_active":"2024-04-02T13:36:20.829048+0000","last_became_peered":"2024-04-02T13:36:20.829048+0000","last_unstale":"2024-04-02T13:38:46.878544+0000","last_undegraded":"2024-04-02T13:38:46.878544+0000","last_fullsized":"2024-04-02T13:38:46.878544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:08:34.405351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062882+0000","last_change":"2024-04-02T13:36:21.497296+0000","last_active":"2024-04-02T13:38:47.062882+0000","last_peered":"2024-04-02T13:38:47.062882+0000","last_clean":"2024-04-02T13:38:47.062882+0000","last_became_active":"2024-04-02T13:36:21.496802+0000","last_became_peered":"2024-04-02T13:36:21.496802+0000","last_unstale":"2024-04-02T13:38:47.062882+0000","last_undegraded":"2024-04-02T13:38:47.062882+0000","last_fullsized":"2024-04-02T13:38:47.062882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:03.995029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874086+0000","last_change":"2024-04-02T13:36:25.498661+0000","last_active":"2024-04-02T13:38:46.874086+0000","last_peered":"2024-04-02T13:38:46.874086+0000","last_clean":"2024-04-02T13:38:46.874086+0000","last_became_active":"2024-04-02T13:36:25.497866+0000","last_became_peered":"2024-04-02T13:36:25.497866+0000","last_unstale":"2024-04-02T13:38:46.874086+0000","last_undegraded":"2024-04-02T13:38:46.874086+0000","last_fullsized":"2024-04-02T13:38:46.874086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:47:05.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878040+0000","last_change":"2024-04-02T13:36:20.824586+0000","last_active":"2024-04-02T13:38:46.878040+0000","last_peered":"2024-04-02T13:38:46.878040+0000","last_clean":"2024-04-02T13:38:46.878040+0000","last_became_active":"2024-04-02T13:36:20.824254+0000","last_became_peered":"2024-04-02T13:36:20.824254+0000","last_unstale":"2024-04-02T13:38:46.878040+0000","last_undegraded":"2024-04-02T13:38:46.878040+0000","last_fullsized":"2024-04-02T13:38:46.878040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:08.774361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979403+0000","last_change":"2024-04-02T13:36:21.495125+0000","last_active":"2024-04-02T13:38:46.979403+0000","last_peered":"2024-04-02T13:38:46.979403+0000","last_clean":"2024-04-02T13:38:46.979403+0000","last_became_active":"2024-04-02T13:36:21.494724+0000","last_became_peered":"2024-04-02T13:36:21.494724+0000","last_unstale":"2024-04-02T13:38:46.979403+0000","last_undegraded":"2024-04-02T13:38:46.979403+0000","last_fullsized":"2024-04-02T13:38:46.979403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:27.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063127+0000","last_change":"2024-04-02T13:36:25.498708+0000","last_active":"2024-04-02T13:38:47.063127+0000","last_peered":"2024-04-02T13:38:47.063127+0000","last_clean":"2024-04-02T13:38:47.063127+0000","last_became_active":"2024-04-02T13:36:25.497834+0000","last_became_peered":"2024-04-02T13:36:25.497834+0000","last_unstale":"2024-04-02T13:38:47.063127+0000","last_undegraded":"2024-04-02T13:38:47.063127+0000","last_fullsized":"2024-04-02T13:38:47.063127+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:14.524569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":169,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.675130+0000","last_change":"2024-04-02T13:36:20.824901+0000","last_active":"2024-04-02T13:38:47.675130+0000","last_peered":"2024-04-02T13:38:47.675130+0000","last_clean":"2024-04-02T13:38:47.675130+0000","last_became_active":"2024-04-02T13:36:20.823973+0000","last_became_peered":"2024-04-02T13:36:20.823973+0000","last_unstale":"2024-04-02T13:38:47.675130+0000","last_undegraded":"2024-04-02T13:38:47.675130+0000","last_fullsized":"2024-04-02T13:38:47.675130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:17.636527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877414+0000","last_change":"2024-04-02T13:36:21.493411+0000","last_active":"2024-04-02T13:38:46.877414+0000","last_peered":"2024-04-02T13:38:46.877414+0000","last_clean":"2024-04-02T13:38:46.877414+0000","last_became_active":"2024-04-02T13:36:21.493061+0000","last_became_peered":"2024-04-02T13:36:21.493061+0000","last_unstale":"2024-04-02T13:38:46.877414+0000","last_undegraded":"2024-04-02T13:38:46.877414+0000","last_fullsized":"2024-04-02T13:38:46.877414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:15.902656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063044+0000","last_change":"2024-04-02T13:36:25.494817+0000","last_active":"2024-04-02T13:38:47.063044+0000","last_peered":"2024-04-02T13:38:47.063044+0000","last_clean":"2024-04-02T13:38:47.063044+0000","last_became_active":"2024-04-02T13:36:25.494185+0000","last_became_peered":"2024-04-02T13:36:25.494185+0000","last_unstale":"2024-04-02T13:38:47.063044+0000","last_undegraded":"2024-04-02T13:38:47.063044+0000","last_fullsized":"2024-04-02T13:38:47.063044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:04:32.719985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982429+0000","last_change":"2024-04-02T13:36:20.821981+0000","last_active":"2024-04-02T13:38:46.982429+0000","last_peered":"2024-04-02T13:38:46.982429+0000","last_clean":"2024-04-02T13:38:46.982429+0000","last_became_active":"2024-04-02T13:36:20.821074+0000","last_became_peered":"2024-04-02T13:36:20.821074+0000","last_unstale":"2024-04-02T13:38:46.982429+0000","last_undegraded":"2024-04-02T13:38:46.982429+0000","last_fullsized":"2024-04-02T13:38:46.982429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:22.094586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970127+0000","last_change":"2024-04-02T13:38:18.512127+0000","last_active":"2024-04-02T13:38:46.970127+0000","last_peered":"2024-04-02T13:38:46.970127+0000","last_clean":"2024-04-02T13:38:46.970127+0000","last_became_active":"2024-04-02T13:38:18.511729+0000","last_became_peered":"2024-04-02T13:38:18.511729+0000","last_unstale":"2024-04-02T13:38:46.970127+0000","last_undegraded":"2024-04-02T13:38:46.970127+0000","last_fullsized":"2024-04-02T13:38:46.970127+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:12:02.320973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062694+0000","last_change":"2024-04-02T13:36:25.503774+0000","last_active":"2024-04-02T13:38:47.062694+0000","last_peered":"2024-04-02T13:38:47.062694+0000","last_clean":"2024-04-02T13:38:47.062694+0000","last_became_active":"2024-04-02T13:36:25.503287+0000","last_became_peered":"2024-04-02T13:36:25.503287+0000","last_unstale":"2024-04-02T13:38:47.062694+0000","last_undegraded":"2024-04-02T13:38:47.062694+0000","last_fullsized":"2024-04-02T13:38:47.062694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:57:29.939484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063048+0000","last_change":"2024-04-02T13:36:20.816760+0000","last_active":"2024-04-02T13:38:47.063048+0000","last_peered":"2024-04-02T13:38:47.063048+0000","last_clean":"2024-04-02T13:38:47.063048+0000","last_became_active":"2024-04-02T13:36:20.816291+0000","last_became_peered":"2024-04-02T13:36:20.816291+0000","last_unstale":"2024-04-02T13:38:47.063048+0000","last_undegraded":"2024-04-02T13:38:47.063048+0000","last_fullsized":"2024-04-02T13:38:47.063048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:25.543946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872422+0000","last_change":"2024-04-02T13:36:21.498541+0000","last_active":"2024-04-02T13:38:46.872422+0000","last_peered":"2024-04-02T13:38:46.872422+0000","last_clean":"2024-04-02T13:38:46.872422+0000","last_became_active":"2024-04-02T13:36:21.497498+0000","last_became_peered":"2024-04-02T13:36:21.497498+0000","last_unstale":"2024-04-02T13:38:46.872422+0000","last_undegraded":"2024-04-02T13:38:46.872422+0000","last_fullsized":"2024-04-02T13:38:46.872422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:08.315938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250671+0000","last_change":"2024-04-02T13:36:25.500245+0000","last_active":"2024-04-02T13:38:47.250671+0000","last_peered":"2024-04-02T13:38:47.250671+0000","last_clean":"2024-04-02T13:38:47.250671+0000","last_became_active":"2024-04-02T13:36:25.499844+0000","last_became_peered":"2024-04-02T13:36:25.499844+0000","last_unstale":"2024-04-02T13:38:47.250671+0000","last_undegraded":"2024-04-02T13:38:47.250671+0000","last_fullsized":"2024-04-02T13:38:47.250671+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:17:01.105757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979010+0000","last_change":"2024-04-02T13:36:20.826851+0000","last_active":"2024-04-02T13:38:46.979010+0000","last_peered":"2024-04-02T13:38:46.979010+0000","last_clean":"2024-04-02T13:38:46.979010+0000","last_became_active":"2024-04-02T13:36:20.826227+0000","last_became_peered":"2024-04-02T13:36:20.826227+0000","last_unstale":"2024-04-02T13:38:46.979010+0000","last_undegraded":"2024-04-02T13:38:46.979010+0000","last_fullsized":"2024-04-02T13:38:46.979010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:45.841966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970174+0000","last_change":"2024-04-02T13:36:21.496520+0000","last_active":"2024-04-02T13:38:46.970174+0000","last_peered":"2024-04-02T13:38:46.970174+0000","last_clean":"2024-04-02T13:38:46.970174+0000","last_became_active":"2024-04-02T13:36:21.496082+0000","last_became_peered":"2024-04-02T13:36:21.496082+0000","last_unstale":"2024-04-02T13:38:46.970174+0000","last_undegraded":"2024-04-02T13:38:46.970174+0000","last_fullsized":"2024-04-02T13:38:46.970174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:12:06.517806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872513+0000","last_change":"2024-04-02T13:36:25.496064+0000","last_active":"2024-04-02T13:38:46.872513+0000","last_peered":"2024-04-02T13:38:46.872513+0000","last_clean":"2024-04-02T13:38:46.872513+0000","last_became_active":"2024-04-02T13:36:25.495464+0000","last_became_peered":"2024-04-02T13:36:25.495464+0000","last_unstale":"2024-04-02T13:38:46.872513+0000","last_undegraded":"2024-04-02T13:38:46.872513+0000","last_fullsized":"2024-04-02T13:38:46.872513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:06:15.206135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'5","reported_seq":161,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.674536+0000","last_change":"2024-04-02T13:36:20.821898+0000","last_active":"2024-04-02T13:38:47.674536+0000","last_peered":"2024-04-02T13:38:47.674536+0000","last_clean":"2024-04-02T13:38:47.674536+0000","last_became_active":"2024-04-02T13:36:20.821374+0000","last_became_peered":"2024-04-02T13:36:20.821374+0000","last_unstale":"2024-04-02T13:38:47.674536+0000","last_undegraded":"2024-04-02T13:38:47.674536+0000","last_fullsized":"2024-04-02T13:38:47.674536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:21.568976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063510+0000","last_change":"2024-04-02T13:36:21.497343+0000","last_active":"2024-04-02T13:38:47.063510+0000","last_peered":"2024-04-02T13:38:47.063510+0000","last_clean":"2024-04-02T13:38:47.063510+0000","last_became_active":"2024-04-02T13:36:21.496733+0000","last_became_peered":"2024-04-02T13:36:21.496733+0000","last_unstale":"2024-04-02T13:38:47.063510+0000","last_undegraded":"2024-04-02T13:38:47.063510+0000","last_fullsized":"2024-04-02T13:38:47.063510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:26.068717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970159+0000","last_change":"2024-04-02T13:36:25.503011+0000","last_active":"2024-04-02T13:38:46.970159+0000","last_peered":"2024-04-02T13:38:46.970159+0000","last_clean":"2024-04-02T13:38:46.970159+0000","last_became_active":"2024-04-02T13:36:25.502571+0000","last_became_peered":"2024-04-02T13:36:25.502571+0000","last_unstale":"2024-04-02T13:38:46.970159+0000","last_undegraded":"2024-04-02T13:38:46.970159+0000","last_fullsized":"2024-04-02T13:38:46.970159+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:48.672023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063668+0000","last_change":"2024-04-02T13:36:20.827705+0000","last_active":"2024-04-02T13:38:47.063668+0000","last_peered":"2024-04-02T13:38:47.063668+0000","last_clean":"2024-04-02T13:38:47.063668+0000","last_became_active":"2024-04-02T13:36:20.827285+0000","last_became_peered":"2024-04-02T13:36:20.827285+0000","last_unstale":"2024-04-02T13:38:47.063668+0000","last_undegraded":"2024-04-02T13:38:47.063668+0000","last_fullsized":"2024-04-02T13:38:47.063668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:12.175523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250885+0000","last_change":"2024-04-02T13:36:21.493365+0000","last_active":"2024-04-02T13:38:47.250885+0000","last_peered":"2024-04-02T13:38:47.250885+0000","last_clean":"2024-04-02T13:38:47.250885+0000","last_became_active":"2024-04-02T13:36:21.493011+0000","last_became_peered":"2024-04-02T13:36:21.493011+0000","last_unstale":"2024-04-02T13:38:47.250885+0000","last_undegraded":"2024-04-02T13:38:47.250885+0000","last_fullsized":"2024-04-02T13:38:47.250885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:53.165244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874354+0000","last_change":"2024-04-02T13:36:25.499017+0000","last_active":"2024-04-02T13:38:46.874354+0000","last_peered":"2024-04-02T13:38:46.874354+0000","last_clean":"2024-04-02T13:38:46.874354+0000","last_became_active":"2024-04-02T13:36:25.498293+0000","last_became_peered":"2024-04-02T13:36:25.498293+0000","last_unstale":"2024-04-02T13:38:46.874354+0000","last_undegraded":"2024-04-02T13:38:46.874354+0000","last_fullsized":"2024-04-02T13:38:46.874354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:55.910943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979425+0000","last_change":"2024-04-02T13:36:20.836345+0000","last_active":"2024-04-02T13:38:46.979425+0000","last_peered":"2024-04-02T13:38:46.979425+0000","last_clean":"2024-04-02T13:38:46.979425+0000","last_became_active":"2024-04-02T13:36:20.836037+0000","last_became_peered":"2024-04-02T13:36:20.836037+0000","last_unstale":"2024-04-02T13:38:46.979425+0000","last_undegraded":"2024-04-02T13:38:46.979425+0000","last_fullsized":"2024-04-02T13:38:46.979425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:47.070493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063302+0000","last_change":"2024-04-02T13:36:21.493209+0000","last_active":"2024-04-02T13:38:47.063302+0000","last_peered":"2024-04-02T13:38:47.063302+0000","last_clean":"2024-04-02T13:38:47.063302+0000","last_became_active":"2024-04-02T13:36:21.492894+0000","last_became_peered":"2024-04-02T13:36:21.492894+0000","last_unstale":"2024-04-02T13:38:47.063302+0000","last_undegraded":"2024-04-02T13:38:47.063302+0000","last_fullsized":"2024-04-02T13:38:47.063302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:47.162354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982294+0000","last_change":"2024-04-02T13:37:17.626933+0000","last_active":"2024-04-02T13:38:46.982294+0000","last_peered":"2024-04-02T13:38:46.982294+0000","last_clean":"2024-04-02T13:38:46.982294+0000","last_became_active":"2024-04-02T13:37:17.626411+0000","last_became_peered":"2024-04-02T13:37:17.626411+0000","last_unstale":"2024-04-02T13:38:46.982294+0000","last_undegraded":"2024-04-02T13:38:46.982294+0000","last_fullsized":"2024-04-02T13:38:46.982294+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:40:56.903871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":330,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.675659+0000","last_change":"2024-04-02T13:36:20.823563+0000","last_active":"2024-04-02T13:38:47.675659+0000","last_peered":"2024-04-02T13:38:47.675659+0000","last_clean":"2024-04-02T13:38:47.675659+0000","last_became_active":"2024-04-02T13:36:20.823186+0000","last_became_peered":"2024-04-02T13:36:20.823186+0000","last_unstale":"2024-04-02T13:38:47.675659+0000","last_undegraded":"2024-04-02T13:38:47.675659+0000","last_fullsized":"2024-04-02T13:38:47.675659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:50.983357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69194,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":255,"num_read_kb":195,"num_write":42,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878277+0000","last_change":"2024-04-02T13:36:21.500829+0000","last_active":"2024-04-02T13:38:46.878277+0000","last_peered":"2024-04-02T13:38:46.878277+0000","last_clean":"2024-04-02T13:38:46.878277+0000","last_became_active":"2024-04-02T13:36:21.500485+0000","last_became_peered":"2024-04-02T13:36:21.500485+0000","last_unstale":"2024-04-02T13:38:46.878277+0000","last_undegraded":"2024-04-02T13:38:46.878277+0000","last_fullsized":"2024-04-02T13:38:46.878277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:23:19.627565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979838+0000","last_change":"2024-04-02T13:36:25.506695+0000","last_active":"2024-04-02T13:38:46.979838+0000","last_peered":"2024-04-02T13:38:46.979838+0000","last_clean":"2024-04-02T13:38:46.979838+0000","last_became_active":"2024-04-02T13:36:25.506254+0000","last_became_peered":"2024-04-02T13:36:25.506254+0000","last_unstale":"2024-04-02T13:38:46.979838+0000","last_undegraded":"2024-04-02T13:38:46.979838+0000","last_fullsized":"2024-04-02T13:38:46.979838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:57:00.315005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063037+0000","last_change":"2024-04-02T13:36:20.825827+0000","last_active":"2024-04-02T13:38:47.063037+0000","last_peered":"2024-04-02T13:38:47.063037+0000","last_clean":"2024-04-02T13:38:47.063037+0000","last_became_active":"2024-04-02T13:36:20.825530+0000","last_became_peered":"2024-04-02T13:36:20.825530+0000","last_unstale":"2024-04-02T13:38:47.063037+0000","last_undegraded":"2024-04-02T13:38:47.063037+0000","last_fullsized":"2024-04-02T13:38:47.063037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:27:02.962513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063146+0000","last_change":"2024-04-02T13:36:21.486343+0000","last_active":"2024-04-02T13:38:47.063146+0000","last_peered":"2024-04-02T13:38:47.063146+0000","last_clean":"2024-04-02T13:38:47.063146+0000","last_became_active":"2024-04-02T13:36:21.486021+0000","last_became_peered":"2024-04-02T13:36:21.486021+0000","last_unstale":"2024-04-02T13:38:47.063146+0000","last_undegraded":"2024-04-02T13:38:47.063146+0000","last_fullsized":"2024-04-02T13:38:47.063146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:53:42.430173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062955+0000","last_change":"2024-04-02T13:37:17.631430+0000","last_active":"2024-04-02T13:38:47.062955+0000","last_peered":"2024-04-02T13:38:47.062955+0000","last_clean":"2024-04-02T13:38:47.062955+0000","last_became_active":"2024-04-02T13:37:17.630930+0000","last_became_peered":"2024-04-02T13:37:17.630930+0000","last_unstale":"2024-04-02T13:38:47.062955+0000","last_undegraded":"2024-04-02T13:38:47.062955+0000","last_fullsized":"2024-04-02T13:38:47.062955+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:10.514663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979556+0000","last_change":"2024-04-02T13:36:20.829303+0000","last_active":"2024-04-02T13:38:46.979556+0000","last_peered":"2024-04-02T13:38:46.979556+0000","last_clean":"2024-04-02T13:38:46.979556+0000","last_became_active":"2024-04-02T13:36:20.828952+0000","last_became_peered":"2024-04-02T13:36:20.828952+0000","last_unstale":"2024-04-02T13:38:46.979556+0000","last_undegraded":"2024-04-02T13:38:46.979556+0000","last_fullsized":"2024-04-02T13:38:46.979556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:14:04.385178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063176+0000","last_change":"2024-04-02T13:36:21.491767+0000","last_active":"2024-04-02T13:38:47.063176+0000","last_peered":"2024-04-02T13:38:47.063176+0000","last_clean":"2024-04-02T13:38:47.063176+0000","last_became_active":"2024-04-02T13:36:21.491369+0000","last_became_peered":"2024-04-02T13:36:21.491369+0000","last_unstale":"2024-04-02T13:38:47.063176+0000","last_undegraded":"2024-04-02T13:38:47.063176+0000","last_fullsized":"2024-04-02T13:38:47.063176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:14:35.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970825+0000","last_change":"2024-04-02T13:36:25.503900+0000","last_active":"2024-04-02T13:38:46.970825+0000","last_peered":"2024-04-02T13:38:46.970825+0000","last_clean":"2024-04-02T13:38:46.970825+0000","last_became_active":"2024-04-02T13:36:25.502840+0000","last_became_peered":"2024-04-02T13:36:25.502840+0000","last_unstale":"2024-04-02T13:38:46.970825+0000","last_undegraded":"2024-04-02T13:38:46.970825+0000","last_fullsized":"2024-04-02T13:38:46.970825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:31.106865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.712522+0000","last_change":"2024-04-02T13:36:20.823753+0000","last_active":"2024-04-02T13:38:47.712522+0000","last_peered":"2024-04-02T13:38:47.712522+0000","last_clean":"2024-04-02T13:38:47.712522+0000","last_became_active":"2024-04-02T13:36:20.823394+0000","last_became_peered":"2024-04-02T13:36:20.823394+0000","last_unstale":"2024-04-02T13:38:47.712522+0000","last_undegraded":"2024-04-02T13:38:47.712522+0000","last_fullsized":"2024-04-02T13:38:47.712522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:13.023125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2395,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063709+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:38:47.063709+0000","last_peered":"2024-04-02T13:38:47.063709+0000","last_clean":"2024-04-02T13:38:47.063709+0000","last_became_active":"2024-04-02T13:36:21.499197+0000","last_became_peered":"2024-04-02T13:36:21.499197+0000","last_unstale":"2024-04-02T13:38:47.063709+0000","last_undegraded":"2024-04-02T13:38:47.063709+0000","last_fullsized":"2024-04-02T13:38:47.063709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:26.435832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063525+0000","last_change":"2024-04-02T13:36:25.504643+0000","last_active":"2024-04-02T13:38:47.063525+0000","last_peered":"2024-04-02T13:38:47.063525+0000","last_clean":"2024-04-02T13:38:47.063525+0000","last_became_active":"2024-04-02T13:36:25.504249+0000","last_became_peered":"2024-04-02T13:36:25.504249+0000","last_unstale":"2024-04-02T13:38:47.063525+0000","last_undegraded":"2024-04-02T13:38:47.063525+0000","last_fullsized":"2024-04-02T13:38:47.063525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:15.818958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877236+0000","last_change":"2024-04-02T13:36:20.830097+0000","last_active":"2024-04-02T13:38:46.877236+0000","last_peered":"2024-04-02T13:38:46.877236+0000","last_clean":"2024-04-02T13:38:46.877236+0000","last_became_active":"2024-04-02T13:36:20.829779+0000","last_became_peered":"2024-04-02T13:36:20.829779+0000","last_unstale":"2024-04-02T13:38:46.877236+0000","last_undegraded":"2024-04-02T13:38:46.877236+0000","last_fullsized":"2024-04-02T13:38:46.877236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:59.477730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979210+0000","last_change":"2024-04-02T13:36:21.497480+0000","last_active":"2024-04-02T13:38:46.979210+0000","last_peered":"2024-04-02T13:38:46.979210+0000","last_clean":"2024-04-02T13:38:46.979210+0000","last_became_active":"2024-04-02T13:36:21.497106+0000","last_became_peered":"2024-04-02T13:36:21.497106+0000","last_unstale":"2024-04-02T13:38:46.979210+0000","last_undegraded":"2024-04-02T13:38:46.979210+0000","last_fullsized":"2024-04-02T13:38:46.979210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:30.394968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979152+0000","last_change":"2024-04-02T13:36:25.500040+0000","last_active":"2024-04-02T13:38:46.979152+0000","last_peered":"2024-04-02T13:38:46.979152+0000","last_clean":"2024-04-02T13:38:46.979152+0000","last_became_active":"2024-04-02T13:36:25.499576+0000","last_became_peered":"2024-04-02T13:36:25.499576+0000","last_unstale":"2024-04-02T13:38:46.979152+0000","last_undegraded":"2024-04-02T13:38:46.979152+0000","last_fullsized":"2024-04-02T13:38:46.979152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:15:47.475234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878479+0000","last_change":"2024-04-02T13:36:20.823225+0000","last_active":"2024-04-02T13:38:46.878479+0000","last_peered":"2024-04-02T13:38:46.878479+0000","last_clean":"2024-04-02T13:38:46.878479+0000","last_became_active":"2024-04-02T13:36:20.822865+0000","last_became_peered":"2024-04-02T13:36:20.822865+0000","last_unstale":"2024-04-02T13:38:46.878479+0000","last_undegraded":"2024-04-02T13:38:46.878479+0000","last_fullsized":"2024-04-02T13:38:46.878479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:00.488190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063314+0000","last_change":"2024-04-02T13:36:21.497692+0000","last_active":"2024-04-02T13:38:47.063314+0000","last_peered":"2024-04-02T13:38:47.063314+0000","last_clean":"2024-04-02T13:38:47.063314+0000","last_became_active":"2024-04-02T13:36:21.497322+0000","last_became_peered":"2024-04-02T13:36:21.497322+0000","last_unstale":"2024-04-02T13:38:47.063314+0000","last_undegraded":"2024-04-02T13:38:47.063314+0000","last_fullsized":"2024-04-02T13:38:47.063314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:17.590235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877317+0000","last_change":"2024-04-02T13:37:17.872416+0000","last_active":"2024-04-02T13:38:46.877317+0000","last_peered":"2024-04-02T13:38:46.877317+0000","last_clean":"2024-04-02T13:38:46.877317+0000","last_became_active":"2024-04-02T13:37:17.871932+0000","last_became_peered":"2024-04-02T13:37:17.871932+0000","last_unstale":"2024-04-02T13:38:46.877317+0000","last_undegraded":"2024-04-02T13:38:46.877317+0000","last_fullsized":"2024-04-02T13:38:46.877317+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:57:23.575737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970543+0000","last_change":"2024-04-02T13:36:20.835277+0000","last_active":"2024-04-02T13:38:46.970543+0000","last_peered":"2024-04-02T13:38:46.970543+0000","last_clean":"2024-04-02T13:38:46.970543+0000","last_became_active":"2024-04-02T13:36:20.834911+0000","last_became_peered":"2024-04-02T13:36:20.834911+0000","last_unstale":"2024-04-02T13:38:46.970543+0000","last_undegraded":"2024-04-02T13:38:46.970543+0000","last_fullsized":"2024-04-02T13:38:46.970543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:20.965544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982626+0000","last_change":"2024-04-02T13:36:21.495274+0000","last_active":"2024-04-02T13:38:46.982626+0000","last_peered":"2024-04-02T13:38:46.982626+0000","last_clean":"2024-04-02T13:38:46.982626+0000","last_became_active":"2024-04-02T13:36:21.494841+0000","last_became_peered":"2024-04-02T13:36:21.494841+0000","last_unstale":"2024-04-02T13:38:46.982626+0000","last_undegraded":"2024-04-02T13:38:46.982626+0000","last_fullsized":"2024-04-02T13:38:46.982626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:20.096507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063043+0000","last_change":"2024-04-02T13:36:25.499970+0000","last_active":"2024-04-02T13:38:47.063043+0000","last_peered":"2024-04-02T13:38:47.063043+0000","last_clean":"2024-04-02T13:38:47.063043+0000","last_became_active":"2024-04-02T13:36:25.499539+0000","last_became_peered":"2024-04-02T13:36:25.499539+0000","last_unstale":"2024-04-02T13:38:47.063043+0000","last_undegraded":"2024-04-02T13:38:47.063043+0000","last_fullsized":"2024-04-02T13:38:47.063043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:43.136635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063059+0000","last_change":"2024-04-02T13:36:20.831150+0000","last_active":"2024-04-02T13:38:47.063059+0000","last_peered":"2024-04-02T13:38:47.063059+0000","last_clean":"2024-04-02T13:38:47.063059+0000","last_became_active":"2024-04-02T13:36:20.830838+0000","last_became_peered":"2024-04-02T13:36:20.830838+0000","last_unstale":"2024-04-02T13:38:47.063059+0000","last_undegraded":"2024-04-02T13:38:47.063059+0000","last_fullsized":"2024-04-02T13:38:47.063059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:23:09.566911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982556+0000","last_change":"2024-04-02T13:36:21.503128+0000","last_active":"2024-04-02T13:38:46.982556+0000","last_peered":"2024-04-02T13:38:46.982556+0000","last_clean":"2024-04-02T13:38:46.982556+0000","last_became_active":"2024-04-02T13:36:21.502695+0000","last_became_peered":"2024-04-02T13:36:21.502695+0000","last_unstale":"2024-04-02T13:38:46.982556+0000","last_undegraded":"2024-04-02T13:38:46.982556+0000","last_fullsized":"2024-04-02T13:38:46.982556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:38.261472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063558+0000","last_change":"2024-04-02T13:36:25.530911+0000","last_active":"2024-04-02T13:38:47.063558+0000","last_peered":"2024-04-02T13:38:47.063558+0000","last_clean":"2024-04-02T13:38:47.063558+0000","last_became_active":"2024-04-02T13:36:25.530428+0000","last_became_peered":"2024-04-02T13:36:25.530428+0000","last_unstale":"2024-04-02T13:38:47.063558+0000","last_undegraded":"2024-04-02T13:38:47.063558+0000","last_fullsized":"2024-04-02T13:38:47.063558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:49.185987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979152+0000","last_change":"2024-04-02T13:36:21.492251+0000","last_active":"2024-04-02T13:38:46.979152+0000","last_peered":"2024-04-02T13:38:46.979152+0000","last_clean":"2024-04-02T13:38:46.979152+0000","last_became_active":"2024-04-02T13:36:21.491929+0000","last_became_peered":"2024-04-02T13:36:21.491929+0000","last_unstale":"2024-04-02T13:38:46.979152+0000","last_undegraded":"2024-04-02T13:38:46.979152+0000","last_fullsized":"2024-04-02T13:38:46.979152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:59.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":160,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062946+0000","last_change":"2024-04-02T13:36:20.821431+0000","last_active":"2024-04-02T13:38:47.062946+0000","last_peered":"2024-04-02T13:38:47.062946+0000","last_clean":"2024-04-02T13:38:47.062946+0000","last_became_active":"2024-04-02T13:36:20.821067+0000","last_became_peered":"2024-04-02T13:36:20.821067+0000","last_unstale":"2024-04-02T13:38:47.062946+0000","last_undegraded":"2024-04-02T13:38:47.062946+0000","last_fullsized":"2024-04-02T13:38:47.062946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:52.043907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063151+0000","last_change":"2024-04-02T13:36:25.496992+0000","last_active":"2024-04-02T13:38:47.063151+0000","last_peered":"2024-04-02T13:38:47.063151+0000","last_clean":"2024-04-02T13:38:47.063151+0000","last_became_active":"2024-04-02T13:36:25.496467+0000","last_became_peered":"2024-04-02T13:36:25.496467+0000","last_unstale":"2024-04-02T13:38:47.063151+0000","last_undegraded":"2024-04-02T13:38:47.063151+0000","last_fullsized":"2024-04-02T13:38:47.063151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:13:00.747716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872529+0000","last_change":"2024-04-02T13:36:21.491399+0000","last_active":"2024-04-02T13:38:46.872529+0000","last_peered":"2024-04-02T13:38:46.872529+0000","last_clean":"2024-04-02T13:38:46.872529+0000","last_became_active":"2024-04-02T13:36:21.491038+0000","last_became_peered":"2024-04-02T13:36:21.491038+0000","last_unstale":"2024-04-02T13:38:46.872529+0000","last_undegraded":"2024-04-02T13:38:46.872529+0000","last_fullsized":"2024-04-02T13:38:46.872529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:25.603086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874057+0000","last_change":"2024-04-02T13:36:20.822796+0000","last_active":"2024-04-02T13:38:46.874057+0000","last_peered":"2024-04-02T13:38:46.874057+0000","last_clean":"2024-04-02T13:38:46.874057+0000","last_became_active":"2024-04-02T13:36:20.822168+0000","last_became_peered":"2024-04-02T13:36:20.822168+0000","last_unstale":"2024-04-02T13:38:46.874057+0000","last_undegraded":"2024-04-02T13:38:46.874057+0000","last_fullsized":"2024-04-02T13:38:46.874057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:11.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062738+0000","last_change":"2024-04-02T13:35:08.678967+0000","last_active":"2024-04-02T13:38:47.062738+0000","last_peered":"2024-04-02T13:38:47.062738+0000","last_clean":"2024-04-02T13:38:47.062738+0000","last_became_active":"2024-04-02T13:35:08.678408+0000","last_became_peered":"2024-04-02T13:35:08.678408+0000","last_unstale":"2024-04-02T13:38:47.062738+0000","last_undegraded":"2024-04-02T13:38:47.062738+0000","last_fullsized":"2024-04-02T13:38:47.062738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_clean_scrub_stamp":"2024-04-02T13:31:42.453938+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:30:20.020809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979462+0000","last_change":"2024-04-02T13:36:25.532723+0000","last_active":"2024-04-02T13:38:46.979462+0000","last_peered":"2024-04-02T13:38:46.979462+0000","last_clean":"2024-04-02T13:38:46.979462+0000","last_became_active":"2024-04-02T13:36:25.532341+0000","last_became_peered":"2024-04-02T13:36:25.532341+0000","last_unstale":"2024-04-02T13:38:46.979462+0000","last_undegraded":"2024-04-02T13:38:46.979462+0000","last_fullsized":"2024-04-02T13:38:46.979462+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:53:43.190339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062721+0000","last_change":"2024-04-02T13:36:21.488983+0000","last_active":"2024-04-02T13:38:47.062721+0000","last_peered":"2024-04-02T13:38:47.062721+0000","last_clean":"2024-04-02T13:38:47.062721+0000","last_became_active":"2024-04-02T13:36:21.488580+0000","last_became_peered":"2024-04-02T13:36:21.488580+0000","last_unstale":"2024-04-02T13:38:47.062721+0000","last_undegraded":"2024-04-02T13:38:47.062721+0000","last_fullsized":"2024-04-02T13:38:47.062721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:39.382233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062580+0000","last_change":"2024-04-02T13:37:17.633840+0000","last_active":"2024-04-02T13:38:47.062580+0000","last_peered":"2024-04-02T13:38:47.062580+0000","last_clean":"2024-04-02T13:38:47.062580+0000","last_became_active":"2024-04-02T13:37:17.626842+0000","last_became_peered":"2024-04-02T13:37:17.626842+0000","last_unstale":"2024-04-02T13:38:47.062580+0000","last_undegraded":"2024-04-02T13:38:47.062580+0000","last_fullsized":"2024-04-02T13:38:47.062580+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:55:17.437862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062781+0000","last_change":"2024-04-02T13:36:25.502814+0000","last_active":"2024-04-02T13:38:47.062781+0000","last_peered":"2024-04-02T13:38:47.062781+0000","last_clean":"2024-04-02T13:38:47.062781+0000","last_became_active":"2024-04-02T13:36:25.502288+0000","last_became_peered":"2024-04-02T13:36:25.502288+0000","last_unstale":"2024-04-02T13:38:47.062781+0000","last_undegraded":"2024-04-02T13:38:47.062781+0000","last_fullsized":"2024-04-02T13:38:47.062781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:48.073286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878368+0000","last_change":"2024-04-02T13:36:20.830512+0000","last_active":"2024-04-02T13:38:46.878368+0000","last_peered":"2024-04-02T13:38:46.878368+0000","last_clean":"2024-04-02T13:38:46.878368+0000","last_became_active":"2024-04-02T13:36:20.830086+0000","last_became_peered":"2024-04-02T13:36:20.830086+0000","last_unstale":"2024-04-02T13:38:46.878368+0000","last_undegraded":"2024-04-02T13:38:46.878368+0000","last_fullsized":"2024-04-02T13:38:46.878368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:07.428363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063182+0000","last_change":"2024-04-02T13:36:21.478218+0000","last_active":"2024-04-02T13:38:47.063182+0000","last_peered":"2024-04-02T13:38:47.063182+0000","last_clean":"2024-04-02T13:38:47.063182+0000","last_became_active":"2024-04-02T13:36:21.477856+0000","last_became_peered":"2024-04-02T13:36:21.477856+0000","last_unstale":"2024-04-02T13:38:47.063182+0000","last_undegraded":"2024-04-02T13:38:47.063182+0000","last_fullsized":"2024-04-02T13:38:47.063182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:04.397606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872360+0000","last_change":"2024-04-02T13:36:25.530915+0000","last_active":"2024-04-02T13:38:46.872360+0000","last_peered":"2024-04-02T13:38:46.872360+0000","last_clean":"2024-04-02T13:38:46.872360+0000","last_became_active":"2024-04-02T13:36:25.530343+0000","last_became_peered":"2024-04-02T13:36:25.530343+0000","last_unstale":"2024-04-02T13:38:46.872360+0000","last_undegraded":"2024-04-02T13:38:46.872360+0000","last_fullsized":"2024-04-02T13:38:46.872360+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:32:41.181485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'45","reported_seq":102,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.095891+0000","last_change":"2024-04-02T13:36:20.828527+0000","last_active":"2024-04-02T13:38:47.095891+0000","last_peered":"2024-04-02T13:38:47.095891+0000","last_clean":"2024-04-02T13:38:47.095891+0000","last_became_active":"2024-04-02T13:36:20.828136+0000","last_became_peered":"2024-04-02T13:36:20.828136+0000","last_unstale":"2024-04-02T13:38:47.095891+0000","last_undegraded":"2024-04-02T13:38:47.095891+0000","last_fullsized":"2024-04-02T13:38:47.095891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:57.709400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872915+0000","last_change":"2024-04-02T13:36:21.492035+0000","last_active":"2024-04-02T13:38:46.872915+0000","last_peered":"2024-04-02T13:38:46.872915+0000","last_clean":"2024-04-02T13:38:46.872915+0000","last_became_active":"2024-04-02T13:36:21.491682+0000","last_became_peered":"2024-04-02T13:36:21.491682+0000","last_unstale":"2024-04-02T13:38:46.872915+0000","last_undegraded":"2024-04-02T13:38:46.872915+0000","last_fullsized":"2024-04-02T13:38:46.872915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:09:40.600616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062520+0000","last_change":"2024-04-02T13:36:25.493301+0000","last_active":"2024-04-02T13:38:47.062520+0000","last_peered":"2024-04-02T13:38:47.062520+0000","last_clean":"2024-04-02T13:38:47.062520+0000","last_became_active":"2024-04-02T13:36:25.492846+0000","last_became_peered":"2024-04-02T13:36:25.492846+0000","last_unstale":"2024-04-02T13:38:47.062520+0000","last_undegraded":"2024-04-02T13:38:47.062520+0000","last_fullsized":"2024-04-02T13:38:47.062520+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:12:53.350261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874377+0000","last_change":"2024-04-02T13:36:20.823559+0000","last_active":"2024-04-02T13:38:46.874377+0000","last_peered":"2024-04-02T13:38:46.874377+0000","last_clean":"2024-04-02T13:38:46.874377+0000","last_became_active":"2024-04-02T13:36:20.823178+0000","last_became_peered":"2024-04-02T13:36:20.823178+0000","last_unstale":"2024-04-02T13:38:46.874377+0000","last_undegraded":"2024-04-02T13:38:46.874377+0000","last_fullsized":"2024-04-02T13:38:46.874377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:42.628855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250679+0000","last_change":"2024-04-02T13:36:21.497258+0000","last_active":"2024-04-02T13:38:47.250679+0000","last_peered":"2024-04-02T13:38:47.250679+0000","last_clean":"2024-04-02T13:38:47.250679+0000","last_became_active":"2024-04-02T13:36:21.496943+0000","last_became_peered":"2024-04-02T13:36:21.496943+0000","last_unstale":"2024-04-02T13:38:47.250679+0000","last_undegraded":"2024-04-02T13:38:47.250679+0000","last_fullsized":"2024-04-02T13:38:47.250679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:51.989025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874385+0000","last_change":"2024-04-02T13:36:25.531251+0000","last_active":"2024-04-02T13:38:46.874385+0000","last_peered":"2024-04-02T13:38:46.874385+0000","last_clean":"2024-04-02T13:38:46.874385+0000","last_became_active":"2024-04-02T13:36:25.530505+0000","last_became_peered":"2024-04-02T13:36:25.530505+0000","last_unstale":"2024-04-02T13:38:46.874385+0000","last_undegraded":"2024-04-02T13:38:46.874385+0000","last_fullsized":"2024-04-02T13:38:46.874385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:02:10.711138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062408+0000","last_change":"2024-04-02T13:36:20.837865+0000","last_active":"2024-04-02T13:38:47.062408+0000","last_peered":"2024-04-02T13:38:47.062408+0000","last_clean":"2024-04-02T13:38:47.062408+0000","last_became_active":"2024-04-02T13:36:20.837567+0000","last_became_peered":"2024-04-02T13:36:20.837567+0000","last_unstale":"2024-04-02T13:38:47.062408+0000","last_undegraded":"2024-04-02T13:38:47.062408+0000","last_fullsized":"2024-04-02T13:38:47.062408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:11:59.365592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877191+0000","last_change":"2024-04-02T13:36:21.500085+0000","last_active":"2024-04-02T13:38:46.877191+0000","last_peered":"2024-04-02T13:38:46.877191+0000","last_clean":"2024-04-02T13:38:46.877191+0000","last_became_active":"2024-04-02T13:36:21.499706+0000","last_became_peered":"2024-04-02T13:36:21.499706+0000","last_unstale":"2024-04-02T13:38:46.877191+0000","last_undegraded":"2024-04-02T13:38:46.877191+0000","last_fullsized":"2024-04-02T13:38:46.877191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.378226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970930+0000","last_change":"2024-04-02T13:36:25.505797+0000","last_active":"2024-04-02T13:38:46.970930+0000","last_peered":"2024-04-02T13:38:46.970930+0000","last_clean":"2024-04-02T13:38:46.970930+0000","last_became_active":"2024-04-02T13:36:25.505409+0000","last_became_peered":"2024-04-02T13:36:25.505409+0000","last_unstale":"2024-04-02T13:38:46.970930+0000","last_undegraded":"2024-04-02T13:38:46.970930+0000","last_fullsized":"2024-04-02T13:38:46.970930+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:46:31.723248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872526+0000","last_change":"2024-04-02T13:36:20.821572+0000","last_active":"2024-04-02T13:38:46.872526+0000","last_peered":"2024-04-02T13:38:46.872526+0000","last_clean":"2024-04-02T13:38:46.872526+0000","last_became_active":"2024-04-02T13:36:20.821235+0000","last_became_peered":"2024-04-02T13:36:20.821235+0000","last_unstale":"2024-04-02T13:38:46.872526+0000","last_undegraded":"2024-04-02T13:38:46.872526+0000","last_fullsized":"2024-04-02T13:38:46.872526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:02.304145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062784+0000","last_change":"2024-04-02T13:36:21.485255+0000","last_active":"2024-04-02T13:38:47.062784+0000","last_peered":"2024-04-02T13:38:47.062784+0000","last_clean":"2024-04-02T13:38:47.062784+0000","last_became_active":"2024-04-02T13:36:21.484870+0000","last_became_peered":"2024-04-02T13:36:21.484870+0000","last_unstale":"2024-04-02T13:38:47.062784+0000","last_undegraded":"2024-04-02T13:38:47.062784+0000","last_fullsized":"2024-04-02T13:38:47.062784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:03.851161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874276+0000","last_change":"2024-04-02T13:36:25.494161+0000","last_active":"2024-04-02T13:38:46.874276+0000","last_peered":"2024-04-02T13:38:46.874276+0000","last_clean":"2024-04-02T13:38:46.874276+0000","last_became_active":"2024-04-02T13:36:25.493667+0000","last_became_peered":"2024-04-02T13:36:25.493667+0000","last_unstale":"2024-04-02T13:38:46.874276+0000","last_undegraded":"2024-04-02T13:38:46.874276+0000","last_fullsized":"2024-04-02T13:38:46.874276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:35:45.703988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062701+0000","last_change":"2024-04-02T13:37:17.636209+0000","last_active":"2024-04-02T13:38:47.062701+0000","last_peered":"2024-04-02T13:38:47.062701+0000","last_clean":"2024-04-02T13:38:47.062701+0000","last_became_active":"2024-04-02T13:37:17.631539+0000","last_became_peered":"2024-04-02T13:37:17.631539+0000","last_unstale":"2024-04-02T13:38:47.062701+0000","last_undegraded":"2024-04-02T13:38:47.062701+0000","last_fullsized":"2024-04-02T13:38:47.062701+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:05:52.619779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872963+0000","last_change":"2024-04-02T13:36:21.498640+0000","last_active":"2024-04-02T13:38:46.872963+0000","last_peered":"2024-04-02T13:38:46.872963+0000","last_clean":"2024-04-02T13:38:46.872963+0000","last_became_active":"2024-04-02T13:36:21.497996+0000","last_became_peered":"2024-04-02T13:36:21.497996+0000","last_unstale":"2024-04-02T13:38:46.872963+0000","last_undegraded":"2024-04-02T13:38:46.872963+0000","last_fullsized":"2024-04-02T13:38:46.872963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:56:45.748945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874223+0000","last_change":"2024-04-02T13:36:25.531119+0000","last_active":"2024-04-02T13:38:46.874223+0000","last_peered":"2024-04-02T13:38:46.874223+0000","last_clean":"2024-04-02T13:38:46.874223+0000","last_became_active":"2024-04-02T13:36:25.530307+0000","last_became_peered":"2024-04-02T13:36:25.530307+0000","last_unstale":"2024-04-02T13:38:46.874223+0000","last_undegraded":"2024-04-02T13:38:46.874223+0000","last_fullsized":"2024-04-02T13:38:46.874223+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:58:50.543386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877426+0000","last_change":"2024-04-02T13:36:20.829843+0000","last_active":"2024-04-02T13:38:46.877426+0000","last_peered":"2024-04-02T13:38:46.877426+0000","last_clean":"2024-04-02T13:38:46.877426+0000","last_became_active":"2024-04-02T13:36:20.829502+0000","last_became_peered":"2024-04-02T13:36:20.829502+0000","last_unstale":"2024-04-02T13:38:46.877426+0000","last_undegraded":"2024-04-02T13:38:46.877426+0000","last_fullsized":"2024-04-02T13:38:46.877426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:36:45.016174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063189+0000","last_change":"2024-04-02T13:36:21.491579+0000","last_active":"2024-04-02T13:38:47.063189+0000","last_peered":"2024-04-02T13:38:47.063189+0000","last_clean":"2024-04-02T13:38:47.063189+0000","last_became_active":"2024-04-02T13:36:21.490992+0000","last_became_peered":"2024-04-02T13:36:21.490992+0000","last_unstale":"2024-04-02T13:38:47.063189+0000","last_undegraded":"2024-04-02T13:38:47.063189+0000","last_fullsized":"2024-04-02T13:38:47.063189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:41.899766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062673+0000","last_change":"2024-04-02T13:36:25.502658+0000","last_active":"2024-04-02T13:38:47.062673+0000","last_peered":"2024-04-02T13:38:47.062673+0000","last_clean":"2024-04-02T13:38:47.062673+0000","last_became_active":"2024-04-02T13:36:25.502219+0000","last_became_peered":"2024-04-02T13:36:25.502219+0000","last_unstale":"2024-04-02T13:38:47.062673+0000","last_undegraded":"2024-04-02T13:38:47.062673+0000","last_fullsized":"2024-04-02T13:38:47.062673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:34.189381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063322+0000","last_change":"2024-04-02T13:36:20.835176+0000","last_active":"2024-04-02T13:38:47.063322+0000","last_peered":"2024-04-02T13:38:47.063322+0000","last_clean":"2024-04-02T13:38:47.063322+0000","last_became_active":"2024-04-02T13:36:20.834871+0000","last_became_peered":"2024-04-02T13:36:20.834871+0000","last_unstale":"2024-04-02T13:38:47.063322+0000","last_undegraded":"2024-04-02T13:38:47.063322+0000","last_fullsized":"2024-04-02T13:38:47.063322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:17:14.148762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063560+0000","last_change":"2024-04-02T13:36:21.490336+0000","last_active":"2024-04-02T13:38:47.063560+0000","last_peered":"2024-04-02T13:38:47.063560+0000","last_clean":"2024-04-02T13:38:47.063560+0000","last_became_active":"2024-04-02T13:36:21.489869+0000","last_became_peered":"2024-04-02T13:36:21.489869+0000","last_unstale":"2024-04-02T13:38:47.063560+0000","last_undegraded":"2024-04-02T13:38:47.063560+0000","last_fullsized":"2024-04-02T13:38:47.063560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:42.243209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250271+0000","last_change":"2024-04-02T13:36:25.503124+0000","last_active":"2024-04-02T13:38:47.250271+0000","last_peered":"2024-04-02T13:38:47.250271+0000","last_clean":"2024-04-02T13:38:47.250271+0000","last_became_active":"2024-04-02T13:36:25.502737+0000","last_became_peered":"2024-04-02T13:36:25.502737+0000","last_unstale":"2024-04-02T13:38:47.250271+0000","last_undegraded":"2024-04-02T13:38:47.250271+0000","last_fullsized":"2024-04-02T13:38:47.250271+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:02:56.830746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'39","reported_seq":80,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.094476+0000","last_change":"2024-04-02T13:36:20.828597+0000","last_active":"2024-04-02T13:38:47.094476+0000","last_peered":"2024-04-02T13:38:47.094476+0000","last_clean":"2024-04-02T13:38:47.094476+0000","last_became_active":"2024-04-02T13:36:20.828138+0000","last_became_peered":"2024-04-02T13:36:20.828138+0000","last_unstale":"2024-04-02T13:38:47.094476+0000","last_undegraded":"2024-04-02T13:38:47.094476+0000","last_fullsized":"2024-04-02T13:38:47.094476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:05.171448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063451+0000","last_change":"2024-04-02T13:36:21.498498+0000","last_active":"2024-04-02T13:38:47.063451+0000","last_peered":"2024-04-02T13:38:47.063451+0000","last_clean":"2024-04-02T13:38:47.063451+0000","last_became_active":"2024-04-02T13:36:21.498119+0000","last_became_peered":"2024-04-02T13:36:21.498119+0000","last_unstale":"2024-04-02T13:38:47.063451+0000","last_undegraded":"2024-04-02T13:38:47.063451+0000","last_fullsized":"2024-04-02T13:38:47.063451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:01.528228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877165+0000","last_change":"2024-04-02T13:37:17.872082+0000","last_active":"2024-04-02T13:38:46.877165+0000","last_peered":"2024-04-02T13:38:46.877165+0000","last_clean":"2024-04-02T13:38:46.877165+0000","last_became_active":"2024-04-02T13:37:17.871562+0000","last_became_peered":"2024-04-02T13:37:17.871562+0000","last_unstale":"2024-04-02T13:38:46.877165+0000","last_undegraded":"2024-04-02T13:38:46.877165+0000","last_fullsized":"2024-04-02T13:38:46.877165+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:32:39.849512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063400+0000","last_change":"2024-04-02T13:37:17.848534+0000","last_active":"2024-04-02T13:38:47.063400+0000","last_peered":"2024-04-02T13:38:47.063400+0000","last_clean":"2024-04-02T13:38:47.063400+0000","last_became_active":"2024-04-02T13:37:17.844875+0000","last_became_peered":"2024-04-02T13:37:17.844875+0000","last_unstale":"2024-04-02T13:38:47.063400+0000","last_undegraded":"2024-04-02T13:38:47.063400+0000","last_fullsized":"2024-04-02T13:38:47.063400+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:29:05.037405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063557+0000","last_change":"2024-04-02T13:36:21.494069+0000","last_active":"2024-04-02T13:38:47.063557+0000","last_peered":"2024-04-02T13:38:47.063557+0000","last_clean":"2024-04-02T13:38:47.063557+0000","last_became_active":"2024-04-02T13:36:21.493751+0000","last_became_peered":"2024-04-02T13:36:21.493751+0000","last_unstale":"2024-04-02T13:38:47.063557+0000","last_undegraded":"2024-04-02T13:38:47.063557+0000","last_fullsized":"2024-04-02T13:38:47.063557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:24:39.648318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062944+0000","last_change":"2024-04-02T13:36:21.494280+0000","last_active":"2024-04-02T13:38:47.062944+0000","last_peered":"2024-04-02T13:38:47.062944+0000","last_clean":"2024-04-02T13:38:47.062944+0000","last_became_active":"2024-04-02T13:36:21.491262+0000","last_became_peered":"2024-04-02T13:36:21.491262+0000","last_unstale":"2024-04-02T13:38:47.062944+0000","last_undegraded":"2024-04-02T13:38:47.062944+0000","last_fullsized":"2024-04-02T13:38:47.062944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:08:16.728646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062753+0000","last_change":"2024-04-02T13:36:21.497103+0000","last_active":"2024-04-02T13:38:47.062753+0000","last_peered":"2024-04-02T13:38:47.062753+0000","last_clean":"2024-04-02T13:38:47.062753+0000","last_became_active":"2024-04-02T13:36:21.496784+0000","last_became_peered":"2024-04-02T13:36:21.496784+0000","last_unstale":"2024-04-02T13:38:47.062753+0000","last_undegraded":"2024-04-02T13:38:47.062753+0000","last_fullsized":"2024-04-02T13:38:47.062753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:38:21.903362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062601+0000","last_change":"2024-04-02T13:36:21.491853+0000","last_active":"2024-04-02T13:38:47.062601+0000","last_peered":"2024-04-02T13:38:47.062601+0000","last_clean":"2024-04-02T13:38:47.062601+0000","last_became_active":"2024-04-02T13:36:21.487986+0000","last_became_peered":"2024-04-02T13:36:21.487986+0000","last_unstale":"2024-04-02T13:38:47.062601+0000","last_undegraded":"2024-04-02T13:38:47.062601+0000","last_fullsized":"2024-04-02T13:38:47.062601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:37:47.098681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250504+0000","last_change":"2024-04-02T13:37:17.631648+0000","last_active":"2024-04-02T13:38:47.250504+0000","last_peered":"2024-04-02T13:38:47.250504+0000","last_clean":"2024-04-02T13:38:47.250504+0000","last_became_active":"2024-04-02T13:37:17.631133+0000","last_became_peered":"2024-04-02T13:37:17.631133+0000","last_unstale":"2024-04-02T13:38:47.250504+0000","last_undegraded":"2024-04-02T13:38:47.250504+0000","last_fullsized":"2024-04-02T13:38:47.250504+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:10.791151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063190+0000","last_change":"2024-04-02T13:36:21.488723+0000","last_active":"2024-04-02T13:38:47.063190+0000","last_peered":"2024-04-02T13:38:47.063190+0000","last_clean":"2024-04-02T13:38:47.063190+0000","last_became_active":"2024-04-02T13:36:21.488391+0000","last_became_peered":"2024-04-02T13:36:21.488391+0000","last_unstale":"2024-04-02T13:38:47.063190+0000","last_undegraded":"2024-04-02T13:38:47.063190+0000","last_fullsized":"2024-04-02T13:38:47.063190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:23.768286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063129+0000","last_change":"2024-04-02T13:36:21.492676+0000","last_active":"2024-04-02T13:38:47.063129+0000","last_peered":"2024-04-02T13:38:47.063129+0000","last_clean":"2024-04-02T13:38:47.063129+0000","last_became_active":"2024-04-02T13:36:21.491663+0000","last_became_peered":"2024-04-02T13:36:21.491663+0000","last_unstale":"2024-04-02T13:38:47.063129+0000","last_undegraded":"2024-04-02T13:38:47.063129+0000","last_fullsized":"2024-04-02T13:38:47.063129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:00.131388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063384+0000","last_change":"2024-04-02T13:36:21.493380+0000","last_active":"2024-04-02T13:38:47.063384+0000","last_peered":"2024-04-02T13:38:47.063384+0000","last_clean":"2024-04-02T13:38:47.063384+0000","last_became_active":"2024-04-02T13:36:21.493073+0000","last_became_peered":"2024-04-02T13:36:21.493073+0000","last_unstale":"2024-04-02T13:38:47.063384+0000","last_undegraded":"2024-04-02T13:38:47.063384+0000","last_fullsized":"2024-04-02T13:38:47.063384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:59.412056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.873114+0000","last_change":"2024-04-02T13:36:21.500881+0000","last_active":"2024-04-02T13:38:46.873114+0000","last_peered":"2024-04-02T13:38:46.873114+0000","last_clean":"2024-04-02T13:38:46.873114+0000","last_became_active":"2024-04-02T13:36:21.500516+0000","last_became_peered":"2024-04-02T13:36:21.500516+0000","last_unstale":"2024-04-02T13:38:46.873114+0000","last_undegraded":"2024-04-02T13:38:46.873114+0000","last_fullsized":"2024-04-02T13:38:46.873114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:33.842494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979070+0000","last_change":"2024-04-02T13:36:21.494558+0000","last_active":"2024-04-02T13:38:46.979070+0000","last_peered":"2024-04-02T13:38:46.979070+0000","last_clean":"2024-04-02T13:38:46.979070+0000","last_became_active":"2024-04-02T13:36:21.494180+0000","last_became_peered":"2024-04-02T13:36:21.494180+0000","last_unstale":"2024-04-02T13:38:46.979070+0000","last_undegraded":"2024-04-02T13:38:46.979070+0000","last_fullsized":"2024-04-02T13:38:46.979070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:07:54.363504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872702+0000","last_change":"2024-04-02T13:36:21.500406+0000","last_active":"2024-04-02T13:38:46.872702+0000","last_peered":"2024-04-02T13:38:46.872702+0000","last_clean":"2024-04-02T13:38:46.872702+0000","last_became_active":"2024-04-02T13:36:21.499927+0000","last_became_peered":"2024-04-02T13:36:21.499927+0000","last_unstale":"2024-04-02T13:38:46.872702+0000","last_undegraded":"2024-04-02T13:38:46.872702+0000","last_fullsized":"2024-04-02T13:38:46.872702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:23.702776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062852+0000","last_change":"2024-04-02T13:36:21.486166+0000","last_active":"2024-04-02T13:38:47.062852+0000","last_peered":"2024-04-02T13:38:47.062852+0000","last_clean":"2024-04-02T13:38:47.062852+0000","last_became_active":"2024-04-02T13:36:21.485282+0000","last_became_peered":"2024-04-02T13:36:21.485282+0000","last_unstale":"2024-04-02T13:38:47.062852+0000","last_undegraded":"2024-04-02T13:38:47.062852+0000","last_fullsized":"2024-04-02T13:38:47.062852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:24:07.007675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063046+0000","last_change":"2024-04-02T13:36:21.484909+0000","last_active":"2024-04-02T13:38:47.063046+0000","last_peered":"2024-04-02T13:38:47.063046+0000","last_clean":"2024-04-02T13:38:47.063046+0000","last_became_active":"2024-04-02T13:36:21.484559+0000","last_became_peered":"2024-04-02T13:36:21.484559+0000","last_unstale":"2024-04-02T13:38:47.063046+0000","last_undegraded":"2024-04-02T13:38:47.063046+0000","last_fullsized":"2024-04-02T13:38:47.063046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:59:45.848407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063088+0000","last_change":"2024-04-02T13:36:21.493961+0000","last_active":"2024-04-02T13:38:47.063088+0000","last_peered":"2024-04-02T13:38:47.063088+0000","last_clean":"2024-04-02T13:38:47.063088+0000","last_became_active":"2024-04-02T13:36:21.493490+0000","last_became_peered":"2024-04-02T13:36:21.493490+0000","last_unstale":"2024-04-02T13:38:47.063088+0000","last_undegraded":"2024-04-02T13:38:47.063088+0000","last_fullsized":"2024-04-02T13:38:47.063088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:26:42.032550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108576,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600843,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":859,"num_read_kb":543,"num_write":299,"num_write_kb":1450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1802529,"data_compressed":80724,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":532498,"data_compressed":29620,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36516,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708636,"statfs":{"total":95995035648,"available":95957643264,"internally_reserved":0,"allocated":1208320,"data_stored":1322367,"data_compressed":73574,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1085440,"data_stored":930046,"data_compressed":48766,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":925288,"data_compressed":48766,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":1131395,"data_compressed":65812,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":527910,"data_compressed":29620,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1110016,"data_stored":597596,"data_compressed":31226,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":519432,"data_compressed":26636,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1130496,"data_stored":599780,"data_compressed":31226,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":1052920,"data_compressed":54922,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888141,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":534367,"data_compressed":29620,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1256776,"data_compressed":71968,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69776,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71960,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134492,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393372,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":204179,"data_compressed":7762,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:38:52.079 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:38:52.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.213+0000 7f9e18675640 1 -- 172.21.15.33:0/1956268947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e10100270 msgr2=0x7f9e10100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.213+0000 7f9e18675640 1 --2- 172.21.15.33:0/1956268947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e10100270 0x7f9e10100650 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009900 tx=0x7f9e0c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:52.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.213+0000 7f9e18675640 1 -- 172.21.15.33:0/1956268947 shutdown_connections 2024-04-02T13:38:52.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.213+0000 7f9e18675640 1 --2- 172.21.15.33:0/1956268947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e10106e80 0x7f9e1010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.213+0000 7f9e18675640 1 --2- 172.21.15.33:0/1956268947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e10100c20 0x7f9e10106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.213+0000 7f9e18675640 1 --2- 172.21.15.33:0/1956268947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e10100270 0x7f9e10100650 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.213+0000 7f9e18675640 1 -- 172.21.15.33:0/1956268947 >> 172.21.15.33:0/1956268947 conn(0x7f9e10076fc0 msgr2=0x7f9e100fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:52.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.213+0000 7f9e18675640 1 -- 172.21.15.33:0/1956268947 shutdown_connections 2024-04-02T13:38:52.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.213+0000 7f9e18675640 1 -- 172.21.15.33:0/1956268947 wait complete. 2024-04-02T13:38:52.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.213+0000 7f9e18675640 1 Processor -- start 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.213+0000 7f9e18675640 1 -- start start 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e18675640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e10100270 0x7f9e10198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e18675640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e10106e80 0x7f9e101988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e18675640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1019cc50 0x7f9e1019d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e18675640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10111ed0 con 0x7f9e10100270 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e18675640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10111d50 con 0x7f9e10106e80 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e18675640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10112050 con 0x7f9e1019cc50 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e16411640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e10100270 0x7f9e10198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e16411640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e10100270 0x7f9e10198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59396/0 (socket says 172.21.15.33:59396) 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e16411640 1 -- 172.21.15.33:0/1570363350 learned_addr learned my addr 172.21.15.33:0/1570363350 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e16c12640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1019cc50 0x7f9e1019d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e15c10640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e10106e80 0x7f9e101988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e16411640 1 -- 172.21.15.33:0/1570363350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1019cc50 msgr2=0x7f9e1019d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e16411640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1019cc50 0x7f9e1019d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e16411640 1 -- 172.21.15.33:0/1570363350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e10106e80 msgr2=0x7f9e101988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e16411640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e10106e80 0x7f9e101988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e16411640 1 -- 172.21.15.33:0/1570363350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1019d740 con 0x7f9e10100270 2024-04-02T13:38:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e15c10640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e10106e80 0x7f9e101988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e16411640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e10100270 0x7f9e10198390 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c005940 tx=0x7f9e0c004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e077fe640 1 -- 172.21.15.33:0/1570363350 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c043020 con 0x7f9e10100270 2024-04-02T13:38:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e18675640 1 -- 172.21.15.33:0/1570363350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1019d9d0 con 0x7f9e10100270 2024-04-02T13:38:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e077fe640 1 -- 172.21.15.33:0/1570363350 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9e0c02c5d0 con 0x7f9e10100270 2024-04-02T13:38:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e077fe640 1 -- 172.21.15.33:0/1570363350 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c003a80 con 0x7f9e10100270 2024-04-02T13:38:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.214+0000 7f9e18675640 1 -- 172.21.15.33:0/1570363350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e101a51d0 con 0x7f9e10100270 2024-04-02T13:38:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.215+0000 7f9e18675640 1 -- 172.21.15.33:0/1570363350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e10068d40 con 0x7f9e10100270 2024-04-02T13:38:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.217+0000 7f9e077fe640 1 -- 172.21.15.33:0/1570363350 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f9e0c046020 con 0x7f9e10100270 2024-04-02T13:38:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.217+0000 7f9e077fe640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9dfc0a3e40 0x7f9dfc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.217+0000 7f9e15c10640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9dfc0a3e40 0x7f9dfc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.217+0000 7f9e077fe640 1 -- 172.21.15.33:0/1570363350 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9e0c0f2e60 con 0x7f9e10100270 2024-04-02T13:38:52.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.218+0000 7f9e15c10640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9dfc0a3e40 0x7f9dfc0a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e100ff400 tx=0x7f9e00005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:52.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.221+0000 7f9e077fe640 1 -- 172.21.15.33:0/1570363350 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9e0c04d070 con 0x7f9e10100270 2024-04-02T13:38:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:38:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:38:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:38:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:38:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3971081623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:38:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:38:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: Updating smithi033:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:38:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-02T13:38:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3637647845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:52.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:38:52.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:38:52.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:38:52.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:51 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4091482264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:52.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:38:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:38:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:38:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:38:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3971081623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: Updating smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:38:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: Updating smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:38:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: Updating smithi033:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:38:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-02T13:38:52.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3637647845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:52.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:38:52.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:38:52.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:38:52.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:38:52.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:51 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4091482264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:52.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.402+0000 7f9e18675640 1 -- 172.21.15.33:0/1570363350 --> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e10100c20 con 0x7f9dfc0a3e40 2024-04-02T13:38:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.411+0000 7f9e077fe640 1 -- 172.21.15.33:0/1570363350 <== mgr.15456 v2:172.21.15.33:6800/619073725 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339690 (secure 0 0 0) 0x7f9e10100c20 con 0x7f9dfc0a3e40 2024-04-02T13:38:52.418 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:52.420 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-02T13:38:52.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.414+0000 7f9e18675640 1 -- 172.21.15.33:0/1570363350 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9dfc0a3e40 msgr2=0x7f9dfc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.414+0000 7f9e18675640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9dfc0a3e40 0x7f9dfc0a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e100ff400 tx=0x7f9e00005f90 comp rx=0 tx=0).stop 2024-04-02T13:38:52.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.414+0000 7f9e18675640 1 -- 172.21.15.33:0/1570363350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e10100270 msgr2=0x7f9e10198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.414+0000 7f9e18675640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e10100270 0x7f9e10198390 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c005940 tx=0x7f9e0c004bd0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.414+0000 7f9e18675640 1 -- 172.21.15.33:0/1570363350 shutdown_connections 2024-04-02T13:38:52.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.414+0000 7f9e18675640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9dfc0a3e40 0x7f9dfc0a6300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.414+0000 7f9e18675640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e1019cc50 0x7f9e1019d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.415+0000 7f9e18675640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e10106e80 0x7f9e101988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.415+0000 7f9e18675640 1 --2- 172.21.15.33:0/1570363350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e10100270 0x7f9e10198390 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.415+0000 7f9e18675640 1 -- 172.21.15.33:0/1570363350 >> 172.21.15.33:0/1570363350 conn(0x7f9e10076fc0 msgr2=0x7f9e10105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:52.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.415+0000 7f9e18675640 1 -- 172.21.15.33:0/1570363350 shutdown_connections 2024-04-02T13:38:52.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.415+0000 7f9e18675640 1 -- 172.21.15.33:0/1570363350 wait complete. 2024-04-02T13:38:52.428 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-02T13:38:50.885512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191445,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":995,"num_read_kb":662,"num_write":482,"num_write_kb":4390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":876,"ondisk_log_size":876,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374484,"kb_used_data":12812,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567340,"statfs":{"total":1151940427776,"available":1151556956160,"internally_reserved":0,"allocated":13119488,"data_stored":9930375,"data_compressed":541756,"data_compressed_allocated":733184,"data_compressed_original":4317184,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987578"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063585+0000","last_change":"2024-04-02T13:36:21.492436+0000","last_active":"2024-04-02T13:38:47.063585+0000","last_peered":"2024-04-02T13:38:47.063585+0000","last_clean":"2024-04-02T13:38:47.063585+0000","last_became_active":"2024-04-02T13:36:21.491831+0000","last_became_peered":"2024-04-02T13:36:21.491831+0000","last_unstale":"2024-04-02T13:38:47.063585+0000","last_undegraded":"2024-04-02T13:38:47.063585+0000","last_fullsized":"2024-04-02T13:38:47.063585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:50:57.919680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250790+0000","last_change":"2024-04-02T13:36:21.498089+0000","last_active":"2024-04-02T13:38:47.250790+0000","last_peered":"2024-04-02T13:38:47.250790+0000","last_clean":"2024-04-02T13:38:47.250790+0000","last_became_active":"2024-04-02T13:36:21.497784+0000","last_became_peered":"2024-04-02T13:36:21.497784+0000","last_unstale":"2024-04-02T13:38:47.250790+0000","last_undegraded":"2024-04-02T13:38:47.250790+0000","last_fullsized":"2024-04-02T13:38:47.250790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:42.137396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250172+0000","last_change":"2024-04-02T13:36:21.492207+0000","last_active":"2024-04-02T13:38:47.250172+0000","last_peered":"2024-04-02T13:38:47.250172+0000","last_clean":"2024-04-02T13:38:47.250172+0000","last_became_active":"2024-04-02T13:36:21.490605+0000","last_became_peered":"2024-04-02T13:36:21.490605+0000","last_unstale":"2024-04-02T13:38:47.250172+0000","last_undegraded":"2024-04-02T13:38:47.250172+0000","last_fullsized":"2024-04-02T13:38:47.250172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:03.984226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874419+0000","last_change":"2024-04-02T13:36:21.491225+0000","last_active":"2024-04-02T13:38:46.874419+0000","last_peered":"2024-04-02T13:38:46.874419+0000","last_clean":"2024-04-02T13:38:46.874419+0000","last_became_active":"2024-04-02T13:36:21.490886+0000","last_became_peered":"2024-04-02T13:36:21.490886+0000","last_unstale":"2024-04-02T13:38:46.874419+0000","last_undegraded":"2024-04-02T13:38:46.874419+0000","last_fullsized":"2024-04-02T13:38:46.874419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:55:53.316243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877507+0000","last_change":"2024-04-02T13:36:21.495808+0000","last_active":"2024-04-02T13:38:46.877507+0000","last_peered":"2024-04-02T13:38:46.877507+0000","last_clean":"2024-04-02T13:38:46.877507+0000","last_became_active":"2024-04-02T13:36:21.495468+0000","last_became_peered":"2024-04-02T13:36:21.495468+0000","last_unstale":"2024-04-02T13:38:46.877507+0000","last_undegraded":"2024-04-02T13:38:46.877507+0000","last_fullsized":"2024-04-02T13:38:46.877507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:17.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982665+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:38:46.982665+0000","last_peered":"2024-04-02T13:38:46.982665+0000","last_clean":"2024-04-02T13:38:46.982665+0000","last_became_active":"2024-04-02T13:36:21.499200+0000","last_became_peered":"2024-04-02T13:36:21.499200+0000","last_unstale":"2024-04-02T13:38:46.982665+0000","last_undegraded":"2024-04-02T13:38:46.982665+0000","last_fullsized":"2024-04-02T13:38:46.982665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:54:52.113844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062959+0000","last_change":"2024-04-02T13:36:21.493429+0000","last_active":"2024-04-02T13:38:47.062959+0000","last_peered":"2024-04-02T13:38:47.062959+0000","last_clean":"2024-04-02T13:38:47.062959+0000","last_became_active":"2024-04-02T13:36:21.493097+0000","last_became_peered":"2024-04-02T13:36:21.493097+0000","last_unstale":"2024-04-02T13:38:47.062959+0000","last_undegraded":"2024-04-02T13:38:47.062959+0000","last_fullsized":"2024-04-02T13:38:47.062959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:47:03.330448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.969983+0000","last_change":"2024-04-02T13:38:18.511280+0000","last_active":"2024-04-02T13:38:46.969983+0000","last_peered":"2024-04-02T13:38:46.969983+0000","last_clean":"2024-04-02T13:38:46.969983+0000","last_became_active":"2024-04-02T13:38:18.510953+0000","last_became_peered":"2024-04-02T13:38:18.510953+0000","last_unstale":"2024-04-02T13:38:46.969983+0000","last_undegraded":"2024-04-02T13:38:46.969983+0000","last_fullsized":"2024-04-02T13:38:46.969983+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:51:38.448445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063843+0000","last_change":"2024-04-02T13:36:21.490623+0000","last_active":"2024-04-02T13:38:47.063843+0000","last_peered":"2024-04-02T13:38:47.063843+0000","last_clean":"2024-04-02T13:38:47.063843+0000","last_became_active":"2024-04-02T13:36:21.490274+0000","last_became_peered":"2024-04-02T13:36:21.490274+0000","last_unstale":"2024-04-02T13:38:47.063843+0000","last_undegraded":"2024-04-02T13:38:47.063843+0000","last_fullsized":"2024-04-02T13:38:47.063843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:59:35.220430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982805+0000","last_change":"2024-04-02T13:37:17.631180+0000","last_active":"2024-04-02T13:38:46.982805+0000","last_peered":"2024-04-02T13:38:46.982805+0000","last_clean":"2024-04-02T13:38:46.982805+0000","last_became_active":"2024-04-02T13:37:17.630778+0000","last_became_peered":"2024-04-02T13:37:17.630778+0000","last_unstale":"2024-04-02T13:38:46.982805+0000","last_undegraded":"2024-04-02T13:38:46.982805+0000","last_fullsized":"2024-04-02T13:38:46.982805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:05:11.445074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063424+0000","last_change":"2024-04-02T13:36:21.492510+0000","last_active":"2024-04-02T13:38:47.063424+0000","last_peered":"2024-04-02T13:38:47.063424+0000","last_clean":"2024-04-02T13:38:47.063424+0000","last_became_active":"2024-04-02T13:36:21.492189+0000","last_became_peered":"2024-04-02T13:36:21.492189+0000","last_unstale":"2024-04-02T13:38:47.063424+0000","last_undegraded":"2024-04-02T13:38:47.063424+0000","last_fullsized":"2024-04-02T13:38:47.063424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:15.321621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874446+0000","last_change":"2024-04-02T13:36:21.498015+0000","last_active":"2024-04-02T13:38:46.874446+0000","last_peered":"2024-04-02T13:38:46.874446+0000","last_clean":"2024-04-02T13:38:46.874446+0000","last_became_active":"2024-04-02T13:36:21.497677+0000","last_became_peered":"2024-04-02T13:36:21.497677+0000","last_unstale":"2024-04-02T13:38:46.874446+0000","last_undegraded":"2024-04-02T13:38:46.874446+0000","last_fullsized":"2024-04-02T13:38:46.874446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:43:19.035902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063126+0000","last_change":"2024-04-02T13:36:21.497047+0000","last_active":"2024-04-02T13:38:47.063126+0000","last_peered":"2024-04-02T13:38:47.063126+0000","last_clean":"2024-04-02T13:38:47.063126+0000","last_became_active":"2024-04-02T13:36:21.496722+0000","last_became_peered":"2024-04-02T13:36:21.496722+0000","last_unstale":"2024-04-02T13:38:47.063126+0000","last_undegraded":"2024-04-02T13:38:47.063126+0000","last_fullsized":"2024-04-02T13:38:47.063126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:03:14.400687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250295+0000","last_change":"2024-04-02T13:36:21.496513+0000","last_active":"2024-04-02T13:38:47.250295+0000","last_peered":"2024-04-02T13:38:47.250295+0000","last_clean":"2024-04-02T13:38:47.250295+0000","last_became_active":"2024-04-02T13:36:21.496209+0000","last_became_peered":"2024-04-02T13:36:21.496209+0000","last_unstale":"2024-04-02T13:38:47.250295+0000","last_undegraded":"2024-04-02T13:38:47.250295+0000","last_fullsized":"2024-04-02T13:38:47.250295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:50.684206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250238+0000","last_change":"2024-04-02T13:36:21.493418+0000","last_active":"2024-04-02T13:38:47.250238+0000","last_peered":"2024-04-02T13:38:47.250238+0000","last_clean":"2024-04-02T13:38:47.250238+0000","last_became_active":"2024-04-02T13:36:21.492785+0000","last_became_peered":"2024-04-02T13:36:21.492785+0000","last_unstale":"2024-04-02T13:38:47.250238+0000","last_undegraded":"2024-04-02T13:38:47.250238+0000","last_fullsized":"2024-04-02T13:38:47.250238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:11.973683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062414+0000","last_change":"2024-04-02T13:36:21.490710+0000","last_active":"2024-04-02T13:38:47.062414+0000","last_peered":"2024-04-02T13:38:47.062414+0000","last_clean":"2024-04-02T13:38:47.062414+0000","last_became_active":"2024-04-02T13:36:21.489654+0000","last_became_peered":"2024-04-02T13:36:21.489654+0000","last_unstale":"2024-04-02T13:38:47.062414+0000","last_undegraded":"2024-04-02T13:38:47.062414+0000","last_fullsized":"2024-04-02T13:38:47.062414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:43:06.198540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063474+0000","last_change":"2024-04-02T13:36:21.493306+0000","last_active":"2024-04-02T13:38:47.063474+0000","last_peered":"2024-04-02T13:38:47.063474+0000","last_clean":"2024-04-02T13:38:47.063474+0000","last_became_active":"2024-04-02T13:36:21.492949+0000","last_became_peered":"2024-04-02T13:36:21.492949+0000","last_unstale":"2024-04-02T13:38:47.063474+0000","last_undegraded":"2024-04-02T13:38:47.063474+0000","last_fullsized":"2024-04-02T13:38:47.063474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:34.264790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970460+0000","last_change":"2024-04-02T13:36:21.496631+0000","last_active":"2024-04-02T13:38:46.970460+0000","last_peered":"2024-04-02T13:38:46.970460+0000","last_clean":"2024-04-02T13:38:46.970460+0000","last_became_active":"2024-04-02T13:36:21.496245+0000","last_became_peered":"2024-04-02T13:36:21.496245+0000","last_unstale":"2024-04-02T13:38:46.970460+0000","last_undegraded":"2024-04-02T13:38:46.970460+0000","last_fullsized":"2024-04-02T13:38:46.970460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:14:51.126381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063307+0000","last_change":"2024-04-02T13:36:21.491640+0000","last_active":"2024-04-02T13:38:47.063307+0000","last_peered":"2024-04-02T13:38:47.063307+0000","last_clean":"2024-04-02T13:38:47.063307+0000","last_became_active":"2024-04-02T13:36:21.487799+0000","last_became_peered":"2024-04-02T13:36:21.487799+0000","last_unstale":"2024-04-02T13:38:47.063307+0000","last_undegraded":"2024-04-02T13:38:47.063307+0000","last_fullsized":"2024-04-02T13:38:47.063307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:16.893295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062642+0000","last_change":"2024-04-02T13:36:25.499691+0000","last_active":"2024-04-02T13:38:47.062642+0000","last_peered":"2024-04-02T13:38:47.062642+0000","last_clean":"2024-04-02T13:38:47.062642+0000","last_became_active":"2024-04-02T13:36:25.499120+0000","last_became_peered":"2024-04-02T13:36:25.499120+0000","last_unstale":"2024-04-02T13:38:47.062642+0000","last_undegraded":"2024-04-02T13:38:47.062642+0000","last_fullsized":"2024-04-02T13:38:47.062642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:47:53.823456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970931+0000","last_change":"2024-04-02T13:37:17.878343+0000","last_active":"2024-04-02T13:38:46.970931+0000","last_peered":"2024-04-02T13:38:46.970931+0000","last_clean":"2024-04-02T13:38:46.970931+0000","last_became_active":"2024-04-02T13:37:17.871984+0000","last_became_peered":"2024-04-02T13:37:17.871984+0000","last_unstale":"2024-04-02T13:38:46.970931+0000","last_undegraded":"2024-04-02T13:38:46.970931+0000","last_fullsized":"2024-04-02T13:38:46.970931+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:28.804070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872842+0000","last_change":"2024-04-02T13:36:21.499019+0000","last_active":"2024-04-02T13:38:46.872842+0000","last_peered":"2024-04-02T13:38:46.872842+0000","last_clean":"2024-04-02T13:38:46.872842+0000","last_became_active":"2024-04-02T13:36:21.498684+0000","last_became_peered":"2024-04-02T13:36:21.498684+0000","last_unstale":"2024-04-02T13:38:46.872842+0000","last_undegraded":"2024-04-02T13:38:46.872842+0000","last_fullsized":"2024-04-02T13:38:46.872842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:15.618382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.873916+0000","last_change":"2024-04-02T13:36:25.498544+0000","last_active":"2024-04-02T13:38:46.873916+0000","last_peered":"2024-04-02T13:38:46.873916+0000","last_clean":"2024-04-02T13:38:46.873916+0000","last_became_active":"2024-04-02T13:36:25.498076+0000","last_became_peered":"2024-04-02T13:36:25.498076+0000","last_unstale":"2024-04-02T13:38:46.873916+0000","last_undegraded":"2024-04-02T13:38:46.873916+0000","last_fullsized":"2024-04-02T13:38:46.873916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:04:47.311056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.873130+0000","last_change":"2024-04-02T13:36:20.832649+0000","last_active":"2024-04-02T13:38:46.873130+0000","last_peered":"2024-04-02T13:38:46.873130+0000","last_clean":"2024-04-02T13:38:46.873130+0000","last_became_active":"2024-04-02T13:36:20.832184+0000","last_became_peered":"2024-04-02T13:36:20.832184+0000","last_unstale":"2024-04-02T13:38:46.873130+0000","last_undegraded":"2024-04-02T13:38:46.873130+0000","last_fullsized":"2024-04-02T13:38:46.873130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:37.639406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250670+0000","last_change":"2024-04-02T13:36:21.489988+0000","last_active":"2024-04-02T13:38:47.250670+0000","last_peered":"2024-04-02T13:38:47.250670+0000","last_clean":"2024-04-02T13:38:47.250670+0000","last_became_active":"2024-04-02T13:36:21.489636+0000","last_became_peered":"2024-04-02T13:36:21.489636+0000","last_unstale":"2024-04-02T13:38:47.250670+0000","last_undegraded":"2024-04-02T13:38:47.250670+0000","last_fullsized":"2024-04-02T13:38:47.250670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:52:36.670712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874051+0000","last_change":"2024-04-02T13:36:25.499441+0000","last_active":"2024-04-02T13:38:46.874051+0000","last_peered":"2024-04-02T13:38:46.874051+0000","last_clean":"2024-04-02T13:38:46.874051+0000","last_became_active":"2024-04-02T13:36:25.498512+0000","last_became_peered":"2024-04-02T13:36:25.498512+0000","last_unstale":"2024-04-02T13:38:46.874051+0000","last_undegraded":"2024-04-02T13:38:46.874051+0000","last_fullsized":"2024-04-02T13:38:46.874051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:48:59.291360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878544+0000","last_change":"2024-04-02T13:36:20.830008+0000","last_active":"2024-04-02T13:38:46.878544+0000","last_peered":"2024-04-02T13:38:46.878544+0000","last_clean":"2024-04-02T13:38:46.878544+0000","last_became_active":"2024-04-02T13:36:20.829048+0000","last_became_peered":"2024-04-02T13:36:20.829048+0000","last_unstale":"2024-04-02T13:38:46.878544+0000","last_undegraded":"2024-04-02T13:38:46.878544+0000","last_fullsized":"2024-04-02T13:38:46.878544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:08:34.405351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062882+0000","last_change":"2024-04-02T13:36:21.497296+0000","last_active":"2024-04-02T13:38:47.062882+0000","last_peered":"2024-04-02T13:38:47.062882+0000","last_clean":"2024-04-02T13:38:47.062882+0000","last_became_active":"2024-04-02T13:36:21.496802+0000","last_became_peered":"2024-04-02T13:36:21.496802+0000","last_unstale":"2024-04-02T13:38:47.062882+0000","last_undegraded":"2024-04-02T13:38:47.062882+0000","last_fullsized":"2024-04-02T13:38:47.062882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:03.995029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874086+0000","last_change":"2024-04-02T13:36:25.498661+0000","last_active":"2024-04-02T13:38:46.874086+0000","last_peered":"2024-04-02T13:38:46.874086+0000","last_clean":"2024-04-02T13:38:46.874086+0000","last_became_active":"2024-04-02T13:36:25.497866+0000","last_became_peered":"2024-04-02T13:36:25.497866+0000","last_unstale":"2024-04-02T13:38:46.874086+0000","last_undegraded":"2024-04-02T13:38:46.874086+0000","last_fullsized":"2024-04-02T13:38:46.874086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:47:05.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878040+0000","last_change":"2024-04-02T13:36:20.824586+0000","last_active":"2024-04-02T13:38:46.878040+0000","last_peered":"2024-04-02T13:38:46.878040+0000","last_clean":"2024-04-02T13:38:46.878040+0000","last_became_active":"2024-04-02T13:36:20.824254+0000","last_became_peered":"2024-04-02T13:36:20.824254+0000","last_unstale":"2024-04-02T13:38:46.878040+0000","last_undegraded":"2024-04-02T13:38:46.878040+0000","last_fullsized":"2024-04-02T13:38:46.878040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:08.774361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979403+0000","last_change":"2024-04-02T13:36:21.495125+0000","last_active":"2024-04-02T13:38:46.979403+0000","last_peered":"2024-04-02T13:38:46.979403+0000","last_clean":"2024-04-02T13:38:46.979403+0000","last_became_active":"2024-04-02T13:36:21.494724+0000","last_became_peered":"2024-04-02T13:36:21.494724+0000","last_unstale":"2024-04-02T13:38:46.979403+0000","last_undegraded":"2024-04-02T13:38:46.979403+0000","last_fullsized":"2024-04-02T13:38:46.979403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:27.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063127+0000","last_change":"2024-04-02T13:36:25.498708+0000","last_active":"2024-04-02T13:38:47.063127+0000","last_peered":"2024-04-02T13:38:47.063127+0000","last_clean":"2024-04-02T13:38:47.063127+0000","last_became_active":"2024-04-02T13:36:25.497834+0000","last_became_peered":"2024-04-02T13:36:25.497834+0000","last_unstale":"2024-04-02T13:38:47.063127+0000","last_undegraded":"2024-04-02T13:38:47.063127+0000","last_fullsized":"2024-04-02T13:38:47.063127+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:14.524569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":169,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.675130+0000","last_change":"2024-04-02T13:36:20.824901+0000","last_active":"2024-04-02T13:38:47.675130+0000","last_peered":"2024-04-02T13:38:47.675130+0000","last_clean":"2024-04-02T13:38:47.675130+0000","last_became_active":"2024-04-02T13:36:20.823973+0000","last_became_peered":"2024-04-02T13:36:20.823973+0000","last_unstale":"2024-04-02T13:38:47.675130+0000","last_undegraded":"2024-04-02T13:38:47.675130+0000","last_fullsized":"2024-04-02T13:38:47.675130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:17.636527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877414+0000","last_change":"2024-04-02T13:36:21.493411+0000","last_active":"2024-04-02T13:38:46.877414+0000","last_peered":"2024-04-02T13:38:46.877414+0000","last_clean":"2024-04-02T13:38:46.877414+0000","last_became_active":"2024-04-02T13:36:21.493061+0000","last_became_peered":"2024-04-02T13:36:21.493061+0000","last_unstale":"2024-04-02T13:38:46.877414+0000","last_undegraded":"2024-04-02T13:38:46.877414+0000","last_fullsized":"2024-04-02T13:38:46.877414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:15.902656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063044+0000","last_change":"2024-04-02T13:36:25.494817+0000","last_active":"2024-04-02T13:38:47.063044+0000","last_peered":"2024-04-02T13:38:47.063044+0000","last_clean":"2024-04-02T13:38:47.063044+0000","last_became_active":"2024-04-02T13:36:25.494185+0000","last_became_peered":"2024-04-02T13:36:25.494185+0000","last_unstale":"2024-04-02T13:38:47.063044+0000","last_undegraded":"2024-04-02T13:38:47.063044+0000","last_fullsized":"2024-04-02T13:38:47.063044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:04:32.719985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982429+0000","last_change":"2024-04-02T13:36:20.821981+0000","last_active":"2024-04-02T13:38:46.982429+0000","last_peered":"2024-04-02T13:38:46.982429+0000","last_clean":"2024-04-02T13:38:46.982429+0000","last_became_active":"2024-04-02T13:36:20.821074+0000","last_became_peered":"2024-04-02T13:36:20.821074+0000","last_unstale":"2024-04-02T13:38:46.982429+0000","last_undegraded":"2024-04-02T13:38:46.982429+0000","last_fullsized":"2024-04-02T13:38:46.982429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:22.094586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970127+0000","last_change":"2024-04-02T13:38:18.512127+0000","last_active":"2024-04-02T13:38:46.970127+0000","last_peered":"2024-04-02T13:38:46.970127+0000","last_clean":"2024-04-02T13:38:46.970127+0000","last_became_active":"2024-04-02T13:38:18.511729+0000","last_became_peered":"2024-04-02T13:38:18.511729+0000","last_unstale":"2024-04-02T13:38:46.970127+0000","last_undegraded":"2024-04-02T13:38:46.970127+0000","last_fullsized":"2024-04-02T13:38:46.970127+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:12:02.320973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062694+0000","last_change":"2024-04-02T13:36:25.503774+0000","last_active":"2024-04-02T13:38:47.062694+0000","last_peered":"2024-04-02T13:38:47.062694+0000","last_clean":"2024-04-02T13:38:47.062694+0000","last_became_active":"2024-04-02T13:36:25.503287+0000","last_became_peered":"2024-04-02T13:36:25.503287+0000","last_unstale":"2024-04-02T13:38:47.062694+0000","last_undegraded":"2024-04-02T13:38:47.062694+0000","last_fullsized":"2024-04-02T13:38:47.062694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:57:29.939484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063048+0000","last_change":"2024-04-02T13:36:20.816760+0000","last_active":"2024-04-02T13:38:47.063048+0000","last_peered":"2024-04-02T13:38:47.063048+0000","last_clean":"2024-04-02T13:38:47.063048+0000","last_became_active":"2024-04-02T13:36:20.816291+0000","last_became_peered":"2024-04-02T13:36:20.816291+0000","last_unstale":"2024-04-02T13:38:47.063048+0000","last_undegraded":"2024-04-02T13:38:47.063048+0000","last_fullsized":"2024-04-02T13:38:47.063048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:25.543946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872422+0000","last_change":"2024-04-02T13:36:21.498541+0000","last_active":"2024-04-02T13:38:46.872422+0000","last_peered":"2024-04-02T13:38:46.872422+0000","last_clean":"2024-04-02T13:38:46.872422+0000","last_became_active":"2024-04-02T13:36:21.497498+0000","last_became_peered":"2024-04-02T13:36:21.497498+0000","last_unstale":"2024-04-02T13:38:46.872422+0000","last_undegraded":"2024-04-02T13:38:46.872422+0000","last_fullsized":"2024-04-02T13:38:46.872422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:08.315938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250671+0000","last_change":"2024-04-02T13:36:25.500245+0000","last_active":"2024-04-02T13:38:47.250671+0000","last_peered":"2024-04-02T13:38:47.250671+0000","last_clean":"2024-04-02T13:38:47.250671+0000","last_became_active":"2024-04-02T13:36:25.499844+0000","last_became_peered":"2024-04-02T13:36:25.499844+0000","last_unstale":"2024-04-02T13:38:47.250671+0000","last_undegraded":"2024-04-02T13:38:47.250671+0000","last_fullsized":"2024-04-02T13:38:47.250671+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:17:01.105757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979010+0000","last_change":"2024-04-02T13:36:20.826851+0000","last_active":"2024-04-02T13:38:46.979010+0000","last_peered":"2024-04-02T13:38:46.979010+0000","last_clean":"2024-04-02T13:38:46.979010+0000","last_became_active":"2024-04-02T13:36:20.826227+0000","last_became_peered":"2024-04-02T13:36:20.826227+0000","last_unstale":"2024-04-02T13:38:46.979010+0000","last_undegraded":"2024-04-02T13:38:46.979010+0000","last_fullsized":"2024-04-02T13:38:46.979010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:45.841966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970174+0000","last_change":"2024-04-02T13:36:21.496520+0000","last_active":"2024-04-02T13:38:46.970174+0000","last_peered":"2024-04-02T13:38:46.970174+0000","last_clean":"2024-04-02T13:38:46.970174+0000","last_became_active":"2024-04-02T13:36:21.496082+0000","last_became_peered":"2024-04-02T13:36:21.496082+0000","last_unstale":"2024-04-02T13:38:46.970174+0000","last_undegraded":"2024-04-02T13:38:46.970174+0000","last_fullsized":"2024-04-02T13:38:46.970174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:12:06.517806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872513+0000","last_change":"2024-04-02T13:36:25.496064+0000","last_active":"2024-04-02T13:38:46.872513+0000","last_peered":"2024-04-02T13:38:46.872513+0000","last_clean":"2024-04-02T13:38:46.872513+0000","last_became_active":"2024-04-02T13:36:25.495464+0000","last_became_peered":"2024-04-02T13:36:25.495464+0000","last_unstale":"2024-04-02T13:38:46.872513+0000","last_undegraded":"2024-04-02T13:38:46.872513+0000","last_fullsized":"2024-04-02T13:38:46.872513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:06:15.206135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'5","reported_seq":161,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.674536+0000","last_change":"2024-04-02T13:36:20.821898+0000","last_active":"2024-04-02T13:38:47.674536+0000","last_peered":"2024-04-02T13:38:47.674536+0000","last_clean":"2024-04-02T13:38:47.674536+0000","last_became_active":"2024-04-02T13:36:20.821374+0000","last_became_peered":"2024-04-02T13:36:20.821374+0000","last_unstale":"2024-04-02T13:38:47.674536+0000","last_undegraded":"2024-04-02T13:38:47.674536+0000","last_fullsized":"2024-04-02T13:38:47.674536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:21.568976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063510+0000","last_change":"2024-04-02T13:36:21.497343+0000","last_active":"2024-04-02T13:38:47.063510+0000","last_peered":"2024-04-02T13:38:47.063510+0000","last_clean":"2024-04-02T13:38:47.063510+0000","last_became_active":"2024-04-02T13:36:21.496733+0000","last_became_peered":"2024-04-02T13:36:21.496733+0000","last_unstale":"2024-04-02T13:38:47.063510+0000","last_undegraded":"2024-04-02T13:38:47.063510+0000","last_fullsized":"2024-04-02T13:38:47.063510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:26.068717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970159+0000","last_change":"2024-04-02T13:36:25.503011+0000","last_active":"2024-04-02T13:38:46.970159+0000","last_peered":"2024-04-02T13:38:46.970159+0000","last_clean":"2024-04-02T13:38:46.970159+0000","last_became_active":"2024-04-02T13:36:25.502571+0000","last_became_peered":"2024-04-02T13:36:25.502571+0000","last_unstale":"2024-04-02T13:38:46.970159+0000","last_undegraded":"2024-04-02T13:38:46.970159+0000","last_fullsized":"2024-04-02T13:38:46.970159+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:48.672023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063668+0000","last_change":"2024-04-02T13:36:20.827705+0000","last_active":"2024-04-02T13:38:47.063668+0000","last_peered":"2024-04-02T13:38:47.063668+0000","last_clean":"2024-04-02T13:38:47.063668+0000","last_became_active":"2024-04-02T13:36:20.827285+0000","last_became_peered":"2024-04-02T13:36:20.827285+0000","last_unstale":"2024-04-02T13:38:47.063668+0000","last_undegraded":"2024-04-02T13:38:47.063668+0000","last_fullsized":"2024-04-02T13:38:47.063668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:12.175523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250885+0000","last_change":"2024-04-02T13:36:21.493365+0000","last_active":"2024-04-02T13:38:47.250885+0000","last_peered":"2024-04-02T13:38:47.250885+0000","last_clean":"2024-04-02T13:38:47.250885+0000","last_became_active":"2024-04-02T13:36:21.493011+0000","last_became_peered":"2024-04-02T13:36:21.493011+0000","last_unstale":"2024-04-02T13:38:47.250885+0000","last_undegraded":"2024-04-02T13:38:47.250885+0000","last_fullsized":"2024-04-02T13:38:47.250885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:53.165244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874354+0000","last_change":"2024-04-02T13:36:25.499017+0000","last_active":"2024-04-02T13:38:46.874354+0000","last_peered":"2024-04-02T13:38:46.874354+0000","last_clean":"2024-04-02T13:38:46.874354+0000","last_became_active":"2024-04-02T13:36:25.498293+0000","last_became_peered":"2024-04-02T13:36:25.498293+0000","last_unstale":"2024-04-02T13:38:46.874354+0000","last_undegraded":"2024-04-02T13:38:46.874354+0000","last_fullsized":"2024-04-02T13:38:46.874354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:55.910943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979425+0000","last_change":"2024-04-02T13:36:20.836345+0000","last_active":"2024-04-02T13:38:46.979425+0000","last_peered":"2024-04-02T13:38:46.979425+0000","last_clean":"2024-04-02T13:38:46.979425+0000","last_became_active":"2024-04-02T13:36:20.836037+0000","last_became_peered":"2024-04-02T13:36:20.836037+0000","last_unstale":"2024-04-02T13:38:46.979425+0000","last_undegraded":"2024-04-02T13:38:46.979425+0000","last_fullsized":"2024-04-02T13:38:46.979425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:47.070493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063302+0000","last_change":"2024-04-02T13:36:21.493209+0000","last_active":"2024-04-02T13:38:47.063302+0000","last_peered":"2024-04-02T13:38:47.063302+0000","last_clean":"2024-04-02T13:38:47.063302+0000","last_became_active":"2024-04-02T13:36:21.492894+0000","last_became_peered":"2024-04-02T13:36:21.492894+0000","last_unstale":"2024-04-02T13:38:47.063302+0000","last_undegraded":"2024-04-02T13:38:47.063302+0000","last_fullsized":"2024-04-02T13:38:47.063302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:47.162354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982294+0000","last_change":"2024-04-02T13:37:17.626933+0000","last_active":"2024-04-02T13:38:46.982294+0000","last_peered":"2024-04-02T13:38:46.982294+0000","last_clean":"2024-04-02T13:38:46.982294+0000","last_became_active":"2024-04-02T13:37:17.626411+0000","last_became_peered":"2024-04-02T13:37:17.626411+0000","last_unstale":"2024-04-02T13:38:46.982294+0000","last_undegraded":"2024-04-02T13:38:46.982294+0000","last_fullsized":"2024-04-02T13:38:46.982294+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:40:56.903871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":330,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.675659+0000","last_change":"2024-04-02T13:36:20.823563+0000","last_active":"2024-04-02T13:38:47.675659+0000","last_peered":"2024-04-02T13:38:47.675659+0000","last_clean":"2024-04-02T13:38:47.675659+0000","last_became_active":"2024-04-02T13:36:20.823186+0000","last_became_peered":"2024-04-02T13:36:20.823186+0000","last_unstale":"2024-04-02T13:38:47.675659+0000","last_undegraded":"2024-04-02T13:38:47.675659+0000","last_fullsized":"2024-04-02T13:38:47.675659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:50.983357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69194,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":255,"num_read_kb":195,"num_write":42,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878277+0000","last_change":"2024-04-02T13:36:21.500829+0000","last_active":"2024-04-02T13:38:46.878277+0000","last_peered":"2024-04-02T13:38:46.878277+0000","last_clean":"2024-04-02T13:38:46.878277+0000","last_became_active":"2024-04-02T13:36:21.500485+0000","last_became_peered":"2024-04-02T13:36:21.500485+0000","last_unstale":"2024-04-02T13:38:46.878277+0000","last_undegraded":"2024-04-02T13:38:46.878277+0000","last_fullsized":"2024-04-02T13:38:46.878277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:23:19.627565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979838+0000","last_change":"2024-04-02T13:36:25.506695+0000","last_active":"2024-04-02T13:38:46.979838+0000","last_peered":"2024-04-02T13:38:46.979838+0000","last_clean":"2024-04-02T13:38:46.979838+0000","last_became_active":"2024-04-02T13:36:25.506254+0000","last_became_peered":"2024-04-02T13:36:25.506254+0000","last_unstale":"2024-04-02T13:38:46.979838+0000","last_undegraded":"2024-04-02T13:38:46.979838+0000","last_fullsized":"2024-04-02T13:38:46.979838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:57:00.315005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063037+0000","last_change":"2024-04-02T13:36:20.825827+0000","last_active":"2024-04-02T13:38:47.063037+0000","last_peered":"2024-04-02T13:38:47.063037+0000","last_clean":"2024-04-02T13:38:47.063037+0000","last_became_active":"2024-04-02T13:36:20.825530+0000","last_became_peered":"2024-04-02T13:36:20.825530+0000","last_unstale":"2024-04-02T13:38:47.063037+0000","last_undegraded":"2024-04-02T13:38:47.063037+0000","last_fullsized":"2024-04-02T13:38:47.063037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:27:02.962513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063146+0000","last_change":"2024-04-02T13:36:21.486343+0000","last_active":"2024-04-02T13:38:47.063146+0000","last_peered":"2024-04-02T13:38:47.063146+0000","last_clean":"2024-04-02T13:38:47.063146+0000","last_became_active":"2024-04-02T13:36:21.486021+0000","last_became_peered":"2024-04-02T13:36:21.486021+0000","last_unstale":"2024-04-02T13:38:47.063146+0000","last_undegraded":"2024-04-02T13:38:47.063146+0000","last_fullsized":"2024-04-02T13:38:47.063146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:53:42.430173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062955+0000","last_change":"2024-04-02T13:37:17.631430+0000","last_active":"2024-04-02T13:38:47.062955+0000","last_peered":"2024-04-02T13:38:47.062955+0000","last_clean":"2024-04-02T13:38:47.062955+0000","last_became_active":"2024-04-02T13:37:17.630930+0000","last_became_peered":"2024-04-02T13:37:17.630930+0000","last_unstale":"2024-04-02T13:38:47.062955+0000","last_undegraded":"2024-04-02T13:38:47.062955+0000","last_fullsized":"2024-04-02T13:38:47.062955+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:10.514663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979556+0000","last_change":"2024-04-02T13:36:20.829303+0000","last_active":"2024-04-02T13:38:46.979556+0000","last_peered":"2024-04-02T13:38:46.979556+0000","last_clean":"2024-04-02T13:38:46.979556+0000","last_became_active":"2024-04-02T13:36:20.828952+0000","last_became_peered":"2024-04-02T13:36:20.828952+0000","last_unstale":"2024-04-02T13:38:46.979556+0000","last_undegraded":"2024-04-02T13:38:46.979556+0000","last_fullsized":"2024-04-02T13:38:46.979556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:14:04.385178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063176+0000","last_change":"2024-04-02T13:36:21.491767+0000","last_active":"2024-04-02T13:38:47.063176+0000","last_peered":"2024-04-02T13:38:47.063176+0000","last_clean":"2024-04-02T13:38:47.063176+0000","last_became_active":"2024-04-02T13:36:21.491369+0000","last_became_peered":"2024-04-02T13:36:21.491369+0000","last_unstale":"2024-04-02T13:38:47.063176+0000","last_undegraded":"2024-04-02T13:38:47.063176+0000","last_fullsized":"2024-04-02T13:38:47.063176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:14:35.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970825+0000","last_change":"2024-04-02T13:36:25.503900+0000","last_active":"2024-04-02T13:38:46.970825+0000","last_peered":"2024-04-02T13:38:46.970825+0000","last_clean":"2024-04-02T13:38:46.970825+0000","last_became_active":"2024-04-02T13:36:25.502840+0000","last_became_peered":"2024-04-02T13:36:25.502840+0000","last_unstale":"2024-04-02T13:38:46.970825+0000","last_undegraded":"2024-04-02T13:38:46.970825+0000","last_fullsized":"2024-04-02T13:38:46.970825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:31.106865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.712522+0000","last_change":"2024-04-02T13:36:20.823753+0000","last_active":"2024-04-02T13:38:47.712522+0000","last_peered":"2024-04-02T13:38:47.712522+0000","last_clean":"2024-04-02T13:38:47.712522+0000","last_became_active":"2024-04-02T13:36:20.823394+0000","last_became_peered":"2024-04-02T13:36:20.823394+0000","last_unstale":"2024-04-02T13:38:47.712522+0000","last_undegraded":"2024-04-02T13:38:47.712522+0000","last_fullsized":"2024-04-02T13:38:47.712522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:13.023125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2395,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063709+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:38:47.063709+0000","last_peered":"2024-04-02T13:38:47.063709+0000","last_clean":"2024-04-02T13:38:47.063709+0000","last_became_active":"2024-04-02T13:36:21.499197+0000","last_became_peered":"2024-04-02T13:36:21.499197+0000","last_unstale":"2024-04-02T13:38:47.063709+0000","last_undegraded":"2024-04-02T13:38:47.063709+0000","last_fullsized":"2024-04-02T13:38:47.063709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:26.435832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063525+0000","last_change":"2024-04-02T13:36:25.504643+0000","last_active":"2024-04-02T13:38:47.063525+0000","last_peered":"2024-04-02T13:38:47.063525+0000","last_clean":"2024-04-02T13:38:47.063525+0000","last_became_active":"2024-04-02T13:36:25.504249+0000","last_became_peered":"2024-04-02T13:36:25.504249+0000","last_unstale":"2024-04-02T13:38:47.063525+0000","last_undegraded":"2024-04-02T13:38:47.063525+0000","last_fullsized":"2024-04-02T13:38:47.063525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:15.818958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877236+0000","last_change":"2024-04-02T13:36:20.830097+0000","last_active":"2024-04-02T13:38:46.877236+0000","last_peered":"2024-04-02T13:38:46.877236+0000","last_clean":"2024-04-02T13:38:46.877236+0000","last_became_active":"2024-04-02T13:36:20.829779+0000","last_became_peered":"2024-04-02T13:36:20.829779+0000","last_unstale":"2024-04-02T13:38:46.877236+0000","last_undegraded":"2024-04-02T13:38:46.877236+0000","last_fullsized":"2024-04-02T13:38:46.877236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:59.477730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979210+0000","last_change":"2024-04-02T13:36:21.497480+0000","last_active":"2024-04-02T13:38:46.979210+0000","last_peered":"2024-04-02T13:38:46.979210+0000","last_clean":"2024-04-02T13:38:46.979210+0000","last_became_active":"2024-04-02T13:36:21.497106+0000","last_became_peered":"2024-04-02T13:36:21.497106+0000","last_unstale":"2024-04-02T13:38:46.979210+0000","last_undegraded":"2024-04-02T13:38:46.979210+0000","last_fullsized":"2024-04-02T13:38:46.979210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:30.394968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979152+0000","last_change":"2024-04-02T13:36:25.500040+0000","last_active":"2024-04-02T13:38:46.979152+0000","last_peered":"2024-04-02T13:38:46.979152+0000","last_clean":"2024-04-02T13:38:46.979152+0000","last_became_active":"2024-04-02T13:36:25.499576+0000","last_became_peered":"2024-04-02T13:36:25.499576+0000","last_unstale":"2024-04-02T13:38:46.979152+0000","last_undegraded":"2024-04-02T13:38:46.979152+0000","last_fullsized":"2024-04-02T13:38:46.979152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:15:47.475234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878479+0000","last_change":"2024-04-02T13:36:20.823225+0000","last_active":"2024-04-02T13:38:46.878479+0000","last_peered":"2024-04-02T13:38:46.878479+0000","last_clean":"2024-04-02T13:38:46.878479+0000","last_became_active":"2024-04-02T13:36:20.822865+0000","last_became_peered":"2024-04-02T13:36:20.822865+0000","last_unstale":"2024-04-02T13:38:46.878479+0000","last_undegraded":"2024-04-02T13:38:46.878479+0000","last_fullsized":"2024-04-02T13:38:46.878479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:00.488190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063314+0000","last_change":"2024-04-02T13:36:21.497692+0000","last_active":"2024-04-02T13:38:47.063314+0000","last_peered":"2024-04-02T13:38:47.063314+0000","last_clean":"2024-04-02T13:38:47.063314+0000","last_became_active":"2024-04-02T13:36:21.497322+0000","last_became_peered":"2024-04-02T13:36:21.497322+0000","last_unstale":"2024-04-02T13:38:47.063314+0000","last_undegraded":"2024-04-02T13:38:47.063314+0000","last_fullsized":"2024-04-02T13:38:47.063314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:17.590235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877317+0000","last_change":"2024-04-02T13:37:17.872416+0000","last_active":"2024-04-02T13:38:46.877317+0000","last_peered":"2024-04-02T13:38:46.877317+0000","last_clean":"2024-04-02T13:38:46.877317+0000","last_became_active":"2024-04-02T13:37:17.871932+0000","last_became_peered":"2024-04-02T13:37:17.871932+0000","last_unstale":"2024-04-02T13:38:46.877317+0000","last_undegraded":"2024-04-02T13:38:46.877317+0000","last_fullsized":"2024-04-02T13:38:46.877317+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:57:23.575737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970543+0000","last_change":"2024-04-02T13:36:20.835277+0000","last_active":"2024-04-02T13:38:46.970543+0000","last_peered":"2024-04-02T13:38:46.970543+0000","last_clean":"2024-04-02T13:38:46.970543+0000","last_became_active":"2024-04-02T13:36:20.834911+0000","last_became_peered":"2024-04-02T13:36:20.834911+0000","last_unstale":"2024-04-02T13:38:46.970543+0000","last_undegraded":"2024-04-02T13:38:46.970543+0000","last_fullsized":"2024-04-02T13:38:46.970543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:20.965544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982626+0000","last_change":"2024-04-02T13:36:21.495274+0000","last_active":"2024-04-02T13:38:46.982626+0000","last_peered":"2024-04-02T13:38:46.982626+0000","last_clean":"2024-04-02T13:38:46.982626+0000","last_became_active":"2024-04-02T13:36:21.494841+0000","last_became_peered":"2024-04-02T13:36:21.494841+0000","last_unstale":"2024-04-02T13:38:46.982626+0000","last_undegraded":"2024-04-02T13:38:46.982626+0000","last_fullsized":"2024-04-02T13:38:46.982626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:20.096507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063043+0000","last_change":"2024-04-02T13:36:25.499970+0000","last_active":"2024-04-02T13:38:47.063043+0000","last_peered":"2024-04-02T13:38:47.063043+0000","last_clean":"2024-04-02T13:38:47.063043+0000","last_became_active":"2024-04-02T13:36:25.499539+0000","last_became_peered":"2024-04-02T13:36:25.499539+0000","last_unstale":"2024-04-02T13:38:47.063043+0000","last_undegraded":"2024-04-02T13:38:47.063043+0000","last_fullsized":"2024-04-02T13:38:47.063043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:43.136635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063059+0000","last_change":"2024-04-02T13:36:20.831150+0000","last_active":"2024-04-02T13:38:47.063059+0000","last_peered":"2024-04-02T13:38:47.063059+0000","last_clean":"2024-04-02T13:38:47.063059+0000","last_became_active":"2024-04-02T13:36:20.830838+0000","last_became_peered":"2024-04-02T13:36:20.830838+0000","last_unstale":"2024-04-02T13:38:47.063059+0000","last_undegraded":"2024-04-02T13:38:47.063059+0000","last_fullsized":"2024-04-02T13:38:47.063059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:23:09.566911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982556+0000","last_change":"2024-04-02T13:36:21.503128+0000","last_active":"2024-04-02T13:38:46.982556+0000","last_peered":"2024-04-02T13:38:46.982556+0000","last_clean":"2024-04-02T13:38:46.982556+0000","last_became_active":"2024-04-02T13:36:21.502695+0000","last_became_peered":"2024-04-02T13:36:21.502695+0000","last_unstale":"2024-04-02T13:38:46.982556+0000","last_undegraded":"2024-04-02T13:38:46.982556+0000","last_fullsized":"2024-04-02T13:38:46.982556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:38.261472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063558+0000","last_change":"2024-04-02T13:36:25.530911+0000","last_active":"2024-04-02T13:38:47.063558+0000","last_peered":"2024-04-02T13:38:47.063558+0000","last_clean":"2024-04-02T13:38:47.063558+0000","last_became_active":"2024-04-02T13:36:25.530428+0000","last_became_peered":"2024-04-02T13:36:25.530428+0000","last_unstale":"2024-04-02T13:38:47.063558+0000","last_undegraded":"2024-04-02T13:38:47.063558+0000","last_fullsized":"2024-04-02T13:38:47.063558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:49.185987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979152+0000","last_change":"2024-04-02T13:36:21.492251+0000","last_active":"2024-04-02T13:38:46.979152+0000","last_peered":"2024-04-02T13:38:46.979152+0000","last_clean":"2024-04-02T13:38:46.979152+0000","last_became_active":"2024-04-02T13:36:21.491929+0000","last_became_peered":"2024-04-02T13:36:21.491929+0000","last_unstale":"2024-04-02T13:38:46.979152+0000","last_undegraded":"2024-04-02T13:38:46.979152+0000","last_fullsized":"2024-04-02T13:38:46.979152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:59.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":160,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062946+0000","last_change":"2024-04-02T13:36:20.821431+0000","last_active":"2024-04-02T13:38:47.062946+0000","last_peered":"2024-04-02T13:38:47.062946+0000","last_clean":"2024-04-02T13:38:47.062946+0000","last_became_active":"2024-04-02T13:36:20.821067+0000","last_became_peered":"2024-04-02T13:36:20.821067+0000","last_unstale":"2024-04-02T13:38:47.062946+0000","last_undegraded":"2024-04-02T13:38:47.062946+0000","last_fullsized":"2024-04-02T13:38:47.062946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:52.043907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063151+0000","last_change":"2024-04-02T13:36:25.496992+0000","last_active":"2024-04-02T13:38:47.063151+0000","last_peered":"2024-04-02T13:38:47.063151+0000","last_clean":"2024-04-02T13:38:47.063151+0000","last_became_active":"2024-04-02T13:36:25.496467+0000","last_became_peered":"2024-04-02T13:36:25.496467+0000","last_unstale":"2024-04-02T13:38:47.063151+0000","last_undegraded":"2024-04-02T13:38:47.063151+0000","last_fullsized":"2024-04-02T13:38:47.063151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:13:00.747716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872529+0000","last_change":"2024-04-02T13:36:21.491399+0000","last_active":"2024-04-02T13:38:46.872529+0000","last_peered":"2024-04-02T13:38:46.872529+0000","last_clean":"2024-04-02T13:38:46.872529+0000","last_became_active":"2024-04-02T13:36:21.491038+0000","last_became_peered":"2024-04-02T13:36:21.491038+0000","last_unstale":"2024-04-02T13:38:46.872529+0000","last_undegraded":"2024-04-02T13:38:46.872529+0000","last_fullsized":"2024-04-02T13:38:46.872529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:25.603086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874057+0000","last_change":"2024-04-02T13:36:20.822796+0000","last_active":"2024-04-02T13:38:46.874057+0000","last_peered":"2024-04-02T13:38:46.874057+0000","last_clean":"2024-04-02T13:38:46.874057+0000","last_became_active":"2024-04-02T13:36:20.822168+0000","last_became_peered":"2024-04-02T13:36:20.822168+0000","last_unstale":"2024-04-02T13:38:46.874057+0000","last_undegraded":"2024-04-02T13:38:46.874057+0000","last_fullsized":"2024-04-02T13:38:46.874057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:11.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062738+0000","last_change":"2024-04-02T13:35:08.678967+0000","last_active":"2024-04-02T13:38:47.062738+0000","last_peered":"2024-04-02T13:38:47.062738+0000","last_clean":"2024-04-02T13:38:47.062738+0000","last_became_active":"2024-04-02T13:35:08.678408+0000","last_became_peered":"2024-04-02T13:35:08.678408+0000","last_unstale":"2024-04-02T13:38:47.062738+0000","last_undegraded":"2024-04-02T13:38:47.062738+0000","last_fullsized":"2024-04-02T13:38:47.062738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_clean_scrub_stamp":"2024-04-02T13:31:42.453938+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:30:20.020809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979462+0000","last_change":"2024-04-02T13:36:25.532723+0000","last_active":"2024-04-02T13:38:46.979462+0000","last_peered":"2024-04-02T13:38:46.979462+0000","last_clean":"2024-04-02T13:38:46.979462+0000","last_became_active":"2024-04-02T13:36:25.532341+0000","last_became_peered":"2024-04-02T13:36:25.532341+0000","last_unstale":"2024-04-02T13:38:46.979462+0000","last_undegraded":"2024-04-02T13:38:46.979462+0000","last_fullsized":"2024-04-02T13:38:46.979462+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:53:43.190339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062721+0000","last_change":"2024-04-02T13:36:21.488983+0000","last_active":"2024-04-02T13:38:47.062721+0000","last_peered":"2024-04-02T13:38:47.062721+0000","last_clean":"2024-04-02T13:38:47.062721+0000","last_became_active":"2024-04-02T13:36:21.488580+0000","last_became_peered":"2024-04-02T13:36:21.488580+0000","last_unstale":"2024-04-02T13:38:47.062721+0000","last_undegraded":"2024-04-02T13:38:47.062721+0000","last_fullsized":"2024-04-02T13:38:47.062721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:39.382233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062580+0000","last_change":"2024-04-02T13:37:17.633840+0000","last_active":"2024-04-02T13:38:47.062580+0000","last_peered":"2024-04-02T13:38:47.062580+0000","last_clean":"2024-04-02T13:38:47.062580+0000","last_became_active":"2024-04-02T13:37:17.626842+0000","last_became_peered":"2024-04-02T13:37:17.626842+0000","last_unstale":"2024-04-02T13:38:47.062580+0000","last_undegraded":"2024-04-02T13:38:47.062580+0000","last_fullsized":"2024-04-02T13:38:47.062580+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:55:17.437862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062781+0000","last_change":"2024-04-02T13:36:25.502814+0000","last_active":"2024-04-02T13:38:47.062781+0000","last_peered":"2024-04-02T13:38:47.062781+0000","last_clean":"2024-04-02T13:38:47.062781+0000","last_became_active":"2024-04-02T13:36:25.502288+0000","last_became_peered":"2024-04-02T13:36:25.502288+0000","last_unstale":"2024-04-02T13:38:47.062781+0000","last_undegraded":"2024-04-02T13:38:47.062781+0000","last_fullsized":"2024-04-02T13:38:47.062781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:48.073286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878368+0000","last_change":"2024-04-02T13:36:20.830512+0000","last_active":"2024-04-02T13:38:46.878368+0000","last_peered":"2024-04-02T13:38:46.878368+0000","last_clean":"2024-04-02T13:38:46.878368+0000","last_became_active":"2024-04-02T13:36:20.830086+0000","last_became_peered":"2024-04-02T13:36:20.830086+0000","last_unstale":"2024-04-02T13:38:46.878368+0000","last_undegraded":"2024-04-02T13:38:46.878368+0000","last_fullsized":"2024-04-02T13:38:46.878368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:07.428363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063182+0000","last_change":"2024-04-02T13:36:21.478218+0000","last_active":"2024-04-02T13:38:47.063182+0000","last_peered":"2024-04-02T13:38:47.063182+0000","last_clean":"2024-04-02T13:38:47.063182+0000","last_became_active":"2024-04-02T13:36:21.477856+0000","last_became_peered":"2024-04-02T13:36:21.477856+0000","last_unstale":"2024-04-02T13:38:47.063182+0000","last_undegraded":"2024-04-02T13:38:47.063182+0000","last_fullsized":"2024-04-02T13:38:47.063182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:04.397606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872360+0000","last_change":"2024-04-02T13:36:25.530915+0000","last_active":"2024-04-02T13:38:46.872360+0000","last_peered":"2024-04-02T13:38:46.872360+0000","last_clean":"2024-04-02T13:38:46.872360+0000","last_became_active":"2024-04-02T13:36:25.530343+0000","last_became_peered":"2024-04-02T13:36:25.530343+0000","last_unstale":"2024-04-02T13:38:46.872360+0000","last_undegraded":"2024-04-02T13:38:46.872360+0000","last_fullsized":"2024-04-02T13:38:46.872360+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:32:41.181485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'45","reported_seq":102,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.095891+0000","last_change":"2024-04-02T13:36:20.828527+0000","last_active":"2024-04-02T13:38:47.095891+0000","last_peered":"2024-04-02T13:38:47.095891+0000","last_clean":"2024-04-02T13:38:47.095891+0000","last_became_active":"2024-04-02T13:36:20.828136+0000","last_became_peered":"2024-04-02T13:36:20.828136+0000","last_unstale":"2024-04-02T13:38:47.095891+0000","last_undegraded":"2024-04-02T13:38:47.095891+0000","last_fullsized":"2024-04-02T13:38:47.095891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:57.709400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872915+0000","last_change":"2024-04-02T13:36:21.492035+0000","last_active":"2024-04-02T13:38:46.872915+0000","last_peered":"2024-04-02T13:38:46.872915+0000","last_clean":"2024-04-02T13:38:46.872915+0000","last_became_active":"2024-04-02T13:36:21.491682+0000","last_became_peered":"2024-04-02T13:36:21.491682+0000","last_unstale":"2024-04-02T13:38:46.872915+0000","last_undegraded":"2024-04-02T13:38:46.872915+0000","last_fullsized":"2024-04-02T13:38:46.872915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:09:40.600616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062520+0000","last_change":"2024-04-02T13:36:25.493301+0000","last_active":"2024-04-02T13:38:47.062520+0000","last_peered":"2024-04-02T13:38:47.062520+0000","last_clean":"2024-04-02T13:38:47.062520+0000","last_became_active":"2024-04-02T13:36:25.492846+0000","last_became_peered":"2024-04-02T13:36:25.492846+0000","last_unstale":"2024-04-02T13:38:47.062520+0000","last_undegraded":"2024-04-02T13:38:47.062520+0000","last_fullsized":"2024-04-02T13:38:47.062520+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:12:53.350261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874377+0000","last_change":"2024-04-02T13:36:20.823559+0000","last_active":"2024-04-02T13:38:46.874377+0000","last_peered":"2024-04-02T13:38:46.874377+0000","last_clean":"2024-04-02T13:38:46.874377+0000","last_became_active":"2024-04-02T13:36:20.823178+0000","last_became_peered":"2024-04-02T13:36:20.823178+0000","last_unstale":"2024-04-02T13:38:46.874377+0000","last_undegraded":"2024-04-02T13:38:46.874377+0000","last_fullsized":"2024-04-02T13:38:46.874377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:42.628855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250679+0000","last_change":"2024-04-02T13:36:21.497258+0000","last_active":"2024-04-02T13:38:47.250679+0000","last_peered":"2024-04-02T13:38:47.250679+0000","last_clean":"2024-04-02T13:38:47.250679+0000","last_became_active":"2024-04-02T13:36:21.496943+0000","last_became_peered":"2024-04-02T13:36:21.496943+0000","last_unstale":"2024-04-02T13:38:47.250679+0000","last_undegraded":"2024-04-02T13:38:47.250679+0000","last_fullsized":"2024-04-02T13:38:47.250679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:51.989025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874385+0000","last_change":"2024-04-02T13:36:25.531251+0000","last_active":"2024-04-02T13:38:46.874385+0000","last_peered":"2024-04-02T13:38:46.874385+0000","last_clean":"2024-04-02T13:38:46.874385+0000","last_became_active":"2024-04-02T13:36:25.530505+0000","last_became_peered":"2024-04-02T13:36:25.530505+0000","last_unstale":"2024-04-02T13:38:46.874385+0000","last_undegraded":"2024-04-02T13:38:46.874385+0000","last_fullsized":"2024-04-02T13:38:46.874385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:02:10.711138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062408+0000","last_change":"2024-04-02T13:36:20.837865+0000","last_active":"2024-04-02T13:38:47.062408+0000","last_peered":"2024-04-02T13:38:47.062408+0000","last_clean":"2024-04-02T13:38:47.062408+0000","last_became_active":"2024-04-02T13:36:20.837567+0000","last_became_peered":"2024-04-02T13:36:20.837567+0000","last_unstale":"2024-04-02T13:38:47.062408+0000","last_undegraded":"2024-04-02T13:38:47.062408+0000","last_fullsized":"2024-04-02T13:38:47.062408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:11:59.365592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877191+0000","last_change":"2024-04-02T13:36:21.500085+0000","last_active":"2024-04-02T13:38:46.877191+0000","last_peered":"2024-04-02T13:38:46.877191+0000","last_clean":"2024-04-02T13:38:46.877191+0000","last_became_active":"2024-04-02T13:36:21.499706+0000","last_became_peered":"2024-04-02T13:36:21.499706+0000","last_unstale":"2024-04-02T13:38:46.877191+0000","last_undegraded":"2024-04-02T13:38:46.877191+0000","last_fullsized":"2024-04-02T13:38:46.877191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.378226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970930+0000","last_change":"2024-04-02T13:36:25.505797+0000","last_active":"2024-04-02T13:38:46.970930+0000","last_peered":"2024-04-02T13:38:46.970930+0000","last_clean":"2024-04-02T13:38:46.970930+0000","last_became_active":"2024-04-02T13:36:25.505409+0000","last_became_peered":"2024-04-02T13:36:25.505409+0000","last_unstale":"2024-04-02T13:38:46.970930+0000","last_undegraded":"2024-04-02T13:38:46.970930+0000","last_fullsized":"2024-04-02T13:38:46.970930+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:46:31.723248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872526+0000","last_change":"2024-04-02T13:36:20.821572+0000","last_active":"2024-04-02T13:38:46.872526+0000","last_peered":"2024-04-02T13:38:46.872526+0000","last_clean":"2024-04-02T13:38:46.872526+0000","last_became_active":"2024-04-02T13:36:20.821235+0000","last_became_peered":"2024-04-02T13:36:20.821235+0000","last_unstale":"2024-04-02T13:38:46.872526+0000","last_undegraded":"2024-04-02T13:38:46.872526+0000","last_fullsized":"2024-04-02T13:38:46.872526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:02.304145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062784+0000","last_change":"2024-04-02T13:36:21.485255+0000","last_active":"2024-04-02T13:38:47.062784+0000","last_peered":"2024-04-02T13:38:47.062784+0000","last_clean":"2024-04-02T13:38:47.062784+0000","last_became_active":"2024-04-02T13:36:21.484870+0000","last_became_peered":"2024-04-02T13:36:21.484870+0000","last_unstale":"2024-04-02T13:38:47.062784+0000","last_undegraded":"2024-04-02T13:38:47.062784+0000","last_fullsized":"2024-04-02T13:38:47.062784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:03.851161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874276+0000","last_change":"2024-04-02T13:36:25.494161+0000","last_active":"2024-04-02T13:38:46.874276+0000","last_peered":"2024-04-02T13:38:46.874276+0000","last_clean":"2024-04-02T13:38:46.874276+0000","last_became_active":"2024-04-02T13:36:25.493667+0000","last_became_peered":"2024-04-02T13:36:25.493667+0000","last_unstale":"2024-04-02T13:38:46.874276+0000","last_undegraded":"2024-04-02T13:38:46.874276+0000","last_fullsized":"2024-04-02T13:38:46.874276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:35:45.703988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062701+0000","last_change":"2024-04-02T13:37:17.636209+0000","last_active":"2024-04-02T13:38:47.062701+0000","last_peered":"2024-04-02T13:38:47.062701+0000","last_clean":"2024-04-02T13:38:47.062701+0000","last_became_active":"2024-04-02T13:37:17.631539+0000","last_became_peered":"2024-04-02T13:37:17.631539+0000","last_unstale":"2024-04-02T13:38:47.062701+0000","last_undegraded":"2024-04-02T13:38:47.062701+0000","last_fullsized":"2024-04-02T13:38:47.062701+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:05:52.619779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872963+0000","last_change":"2024-04-02T13:36:21.498640+0000","last_active":"2024-04-02T13:38:46.872963+0000","last_peered":"2024-04-02T13:38:46.872963+0000","last_clean":"2024-04-02T13:38:46.872963+0000","last_became_active":"2024-04-02T13:36:21.497996+0000","last_became_peered":"2024-04-02T13:36:21.497996+0000","last_unstale":"2024-04-02T13:38:46.872963+0000","last_undegraded":"2024-04-02T13:38:46.872963+0000","last_fullsized":"2024-04-02T13:38:46.872963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:56:45.748945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874223+0000","last_change":"2024-04-02T13:36:25.531119+0000","last_active":"2024-04-02T13:38:46.874223+0000","last_peered":"2024-04-02T13:38:46.874223+0000","last_clean":"2024-04-02T13:38:46.874223+0000","last_became_active":"2024-04-02T13:36:25.530307+0000","last_became_peered":"2024-04-02T13:36:25.530307+0000","last_unstale":"2024-04-02T13:38:46.874223+0000","last_undegraded":"2024-04-02T13:38:46.874223+0000","last_fullsized":"2024-04-02T13:38:46.874223+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:58:50.543386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877426+0000","last_change":"2024-04-02T13:36:20.829843+0000","last_active":"2024-04-02T13:38:46.877426+0000","last_peered":"2024-04-02T13:38:46.877426+0000","last_clean":"2024-04-02T13:38:46.877426+0000","last_became_active":"2024-04-02T13:36:20.829502+0000","last_became_peered":"2024-04-02T13:36:20.829502+0000","last_unstale":"2024-04-02T13:38:46.877426+0000","last_undegraded":"2024-04-02T13:38:46.877426+0000","last_fullsized":"2024-04-02T13:38:46.877426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:36:45.016174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063189+0000","last_change":"2024-04-02T13:36:21.491579+0000","last_active":"2024-04-02T13:38:47.063189+0000","last_peered":"2024-04-02T13:38:47.063189+0000","last_clean":"2024-04-02T13:38:47.063189+0000","last_became_active":"2024-04-02T13:36:21.490992+0000","last_became_peered":"2024-04-02T13:36:21.490992+0000","last_unstale":"2024-04-02T13:38:47.063189+0000","last_undegraded":"2024-04-02T13:38:47.063189+0000","last_fullsized":"2024-04-02T13:38:47.063189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:41.899766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062673+0000","last_change":"2024-04-02T13:36:25.502658+0000","last_active":"2024-04-02T13:38:47.062673+0000","last_peered":"2024-04-02T13:38:47.062673+0000","last_clean":"2024-04-02T13:38:47.062673+0000","last_became_active":"2024-04-02T13:36:25.502219+0000","last_became_peered":"2024-04-02T13:36:25.502219+0000","last_unstale":"2024-04-02T13:38:47.062673+0000","last_undegraded":"2024-04-02T13:38:47.062673+0000","last_fullsized":"2024-04-02T13:38:47.062673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:34.189381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063322+0000","last_change":"2024-04-02T13:36:20.835176+0000","last_active":"2024-04-02T13:38:47.063322+0000","last_peered":"2024-04-02T13:38:47.063322+0000","last_clean":"2024-04-02T13:38:47.063322+0000","last_became_active":"2024-04-02T13:36:20.834871+0000","last_became_peered":"2024-04-02T13:36:20.834871+0000","last_unstale":"2024-04-02T13:38:47.063322+0000","last_undegraded":"2024-04-02T13:38:47.063322+0000","last_fullsized":"2024-04-02T13:38:47.063322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:17:14.148762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063560+0000","last_change":"2024-04-02T13:36:21.490336+0000","last_active":"2024-04-02T13:38:47.063560+0000","last_peered":"2024-04-02T13:38:47.063560+0000","last_clean":"2024-04-02T13:38:47.063560+0000","last_became_active":"2024-04-02T13:36:21.489869+0000","last_became_peered":"2024-04-02T13:36:21.489869+0000","last_unstale":"2024-04-02T13:38:47.063560+0000","last_undegraded":"2024-04-02T13:38:47.063560+0000","last_fullsized":"2024-04-02T13:38:47.063560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:42.243209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250271+0000","last_change":"2024-04-02T13:36:25.503124+0000","last_active":"2024-04-02T13:38:47.250271+0000","last_peered":"2024-04-02T13:38:47.250271+0000","last_clean":"2024-04-02T13:38:47.250271+0000","last_became_active":"2024-04-02T13:36:25.502737+0000","last_became_peered":"2024-04-02T13:36:25.502737+0000","last_unstale":"2024-04-02T13:38:47.250271+0000","last_undegraded":"2024-04-02T13:38:47.250271+0000","last_fullsized":"2024-04-02T13:38:47.250271+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:02:56.830746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'39","reported_seq":80,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.094476+0000","last_change":"2024-04-02T13:36:20.828597+0000","last_active":"2024-04-02T13:38:47.094476+0000","last_peered":"2024-04-02T13:38:47.094476+0000","last_clean":"2024-04-02T13:38:47.094476+0000","last_became_active":"2024-04-02T13:36:20.828138+0000","last_became_peered":"2024-04-02T13:36:20.828138+0000","last_unstale":"2024-04-02T13:38:47.094476+0000","last_undegraded":"2024-04-02T13:38:47.094476+0000","last_fullsized":"2024-04-02T13:38:47.094476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:05.171448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063451+0000","last_change":"2024-04-02T13:36:21.498498+0000","last_active":"2024-04-02T13:38:47.063451+0000","last_peered":"2024-04-02T13:38:47.063451+0000","last_clean":"2024-04-02T13:38:47.063451+0000","last_became_active":"2024-04-02T13:36:21.498119+0000","last_became_peered":"2024-04-02T13:36:21.498119+0000","last_unstale":"2024-04-02T13:38:47.063451+0000","last_undegraded":"2024-04-02T13:38:47.063451+0000","last_fullsized":"2024-04-02T13:38:47.063451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:01.528228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877165+0000","last_change":"2024-04-02T13:37:17.872082+0000","last_active":"2024-04-02T13:38:46.877165+0000","last_peered":"2024-04-02T13:38:46.877165+0000","last_clean":"2024-04-02T13:38:46.877165+0000","last_became_active":"2024-04-02T13:37:17.871562+0000","last_became_peered":"2024-04-02T13:37:17.871562+0000","last_unstale":"2024-04-02T13:38:46.877165+0000","last_undegraded":"2024-04-02T13:38:46.877165+0000","last_fullsized":"2024-04-02T13:38:46.877165+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:32:39.849512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063400+0000","last_change":"2024-04-02T13:37:17.848534+0000","last_active":"2024-04-02T13:38:47.063400+0000","last_peered":"2024-04-02T13:38:47.063400+0000","last_clean":"2024-04-02T13:38:47.063400+0000","last_became_active":"2024-04-02T13:37:17.844875+0000","last_became_peered":"2024-04-02T13:37:17.844875+0000","last_unstale":"2024-04-02T13:38:47.063400+0000","last_undegraded":"2024-04-02T13:38:47.063400+0000","last_fullsized":"2024-04-02T13:38:47.063400+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:29:05.037405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063557+0000","last_change":"2024-04-02T13:36:21.494069+0000","last_active":"2024-04-02T13:38:47.063557+0000","last_peered":"2024-04-02T13:38:47.063557+0000","last_clean":"2024-04-02T13:38:47.063557+0000","last_became_active":"2024-04-02T13:36:21.493751+0000","last_became_peered":"2024-04-02T13:36:21.493751+0000","last_unstale":"2024-04-02T13:38:47.063557+0000","last_undegraded":"2024-04-02T13:38:47.063557+0000","last_fullsized":"2024-04-02T13:38:47.063557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:24:39.648318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062944+0000","last_change":"2024-04-02T13:36:21.494280+0000","last_active":"2024-04-02T13:38:47.062944+0000","last_peered":"2024-04-02T13:38:47.062944+0000","last_clean":"2024-04-02T13:38:47.062944+0000","last_became_active":"2024-04-02T13:36:21.491262+0000","last_became_peered":"2024-04-02T13:36:21.491262+0000","last_unstale":"2024-04-02T13:38:47.062944+0000","last_undegraded":"2024-04-02T13:38:47.062944+0000","last_fullsized":"2024-04-02T13:38:47.062944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:08:16.728646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062753+0000","last_change":"2024-04-02T13:36:21.497103+0000","last_active":"2024-04-02T13:38:47.062753+0000","last_peered":"2024-04-02T13:38:47.062753+0000","last_clean":"2024-04-02T13:38:47.062753+0000","last_became_active":"2024-04-02T13:36:21.496784+0000","last_became_peered":"2024-04-02T13:36:21.496784+0000","last_unstale":"2024-04-02T13:38:47.062753+0000","last_undegraded":"2024-04-02T13:38:47.062753+0000","last_fullsized":"2024-04-02T13:38:47.062753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:38:21.903362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062601+0000","last_change":"2024-04-02T13:36:21.491853+0000","last_active":"2024-04-02T13:38:47.062601+0000","last_peered":"2024-04-02T13:38:47.062601+0000","last_clean":"2024-04-02T13:38:47.062601+0000","last_became_active":"2024-04-02T13:36:21.487986+0000","last_became_peered":"2024-04-02T13:36:21.487986+0000","last_unstale":"2024-04-02T13:38:47.062601+0000","last_undegraded":"2024-04-02T13:38:47.062601+0000","last_fullsized":"2024-04-02T13:38:47.062601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:37:47.098681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250504+0000","last_change":"2024-04-02T13:37:17.631648+0000","last_active":"2024-04-02T13:38:47.250504+0000","last_peered":"2024-04-02T13:38:47.250504+0000","last_clean":"2024-04-02T13:38:47.250504+0000","last_became_active":"2024-04-02T13:37:17.631133+0000","last_became_peered":"2024-04-02T13:37:17.631133+0000","last_unstale":"2024-04-02T13:38:47.250504+0000","last_undegraded":"2024-04-02T13:38:47.250504+0000","last_fullsized":"2024-04-02T13:38:47.250504+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:10.791151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063190+0000","last_change":"2024-04-02T13:36:21.488723+0000","last_active":"2024-04-02T13:38:47.063190+0000","last_peered":"2024-04-02T13:38:47.063190+0000","last_clean":"2024-04-02T13:38:47.063190+0000","last_became_active":"2024-04-02T13:36:21.488391+0000","last_became_peered":"2024-04-02T13:36:21.488391+0000","last_unstale":"2024-04-02T13:38:47.063190+0000","last_undegraded":"2024-04-02T13:38:47.063190+0000","last_fullsized":"2024-04-02T13:38:47.063190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:23.768286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063129+0000","last_change":"2024-04-02T13:36:21.492676+0000","last_active":"2024-04-02T13:38:47.063129+0000","last_peered":"2024-04-02T13:38:47.063129+0000","last_clean":"2024-04-02T13:38:47.063129+0000","last_became_active":"2024-04-02T13:36:21.491663+0000","last_became_peered":"2024-04-02T13:36:21.491663+0000","last_unstale":"2024-04-02T13:38:47.063129+0000","last_undegraded":"2024-04-02T13:38:47.063129+0000","last_fullsized":"2024-04-02T13:38:47.063129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:00.131388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063384+0000","last_change":"2024-04-02T13:36:21.493380+0000","last_active":"2024-04-02T13:38:47.063384+0000","last_peered":"2024-04-02T13:38:47.063384+0000","last_clean":"2024-04-02T13:38:47.063384+0000","last_became_active":"2024-04-02T13:36:21.493073+0000","last_became_peered":"2024-04-02T13:36:21.493073+0000","last_unstale":"2024-04-02T13:38:47.063384+0000","last_undegraded":"2024-04-02T13:38:47.063384+0000","last_fullsized":"2024-04-02T13:38:47.063384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:59.412056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.873114+0000","last_change":"2024-04-02T13:36:21.500881+0000","last_active":"2024-04-02T13:38:46.873114+0000","last_peered":"2024-04-02T13:38:46.873114+0000","last_clean":"2024-04-02T13:38:46.873114+0000","last_became_active":"2024-04-02T13:36:21.500516+0000","last_became_peered":"2024-04-02T13:36:21.500516+0000","last_unstale":"2024-04-02T13:38:46.873114+0000","last_undegraded":"2024-04-02T13:38:46.873114+0000","last_fullsized":"2024-04-02T13:38:46.873114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:33.842494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979070+0000","last_change":"2024-04-02T13:36:21.494558+0000","last_active":"2024-04-02T13:38:46.979070+0000","last_peered":"2024-04-02T13:38:46.979070+0000","last_clean":"2024-04-02T13:38:46.979070+0000","last_became_active":"2024-04-02T13:36:21.494180+0000","last_became_peered":"2024-04-02T13:36:21.494180+0000","last_unstale":"2024-04-02T13:38:46.979070+0000","last_undegraded":"2024-04-02T13:38:46.979070+0000","last_fullsized":"2024-04-02T13:38:46.979070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:07:54.363504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872702+0000","last_change":"2024-04-02T13:36:21.500406+0000","last_active":"2024-04-02T13:38:46.872702+0000","last_peered":"2024-04-02T13:38:46.872702+0000","last_clean":"2024-04-02T13:38:46.872702+0000","last_became_active":"2024-04-02T13:36:21.499927+0000","last_became_peered":"2024-04-02T13:36:21.499927+0000","last_unstale":"2024-04-02T13:38:46.872702+0000","last_undegraded":"2024-04-02T13:38:46.872702+0000","last_fullsized":"2024-04-02T13:38:46.872702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:23.702776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062852+0000","last_change":"2024-04-02T13:36:21.486166+0000","last_active":"2024-04-02T13:38:47.062852+0000","last_peered":"2024-04-02T13:38:47.062852+0000","last_clean":"2024-04-02T13:38:47.062852+0000","last_became_active":"2024-04-02T13:36:21.485282+0000","last_became_peered":"2024-04-02T13:36:21.485282+0000","last_unstale":"2024-04-02T13:38:47.062852+0000","last_undegraded":"2024-04-02T13:38:47.062852+0000","last_fullsized":"2024-04-02T13:38:47.062852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:24:07.007675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063046+0000","last_change":"2024-04-02T13:36:21.484909+0000","last_active":"2024-04-02T13:38:47.063046+0000","last_peered":"2024-04-02T13:38:47.063046+0000","last_clean":"2024-04-02T13:38:47.063046+0000","last_became_active":"2024-04-02T13:36:21.484559+0000","last_became_peered":"2024-04-02T13:36:21.484559+0000","last_unstale":"2024-04-02T13:38:47.063046+0000","last_undegraded":"2024-04-02T13:38:47.063046+0000","last_fullsized":"2024-04-02T13:38:47.063046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:59:45.848407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063088+0000","last_change":"2024-04-02T13:36:21.493961+0000","last_active":"2024-04-02T13:38:47.063088+0000","last_peered":"2024-04-02T13:38:47.063088+0000","last_clean":"2024-04-02T13:38:47.063088+0000","last_became_active":"2024-04-02T13:36:21.493490+0000","last_became_peered":"2024-04-02T13:36:21.493490+0000","last_unstale":"2024-04-02T13:38:47.063088+0000","last_undegraded":"2024-04-02T13:38:47.063088+0000","last_fullsized":"2024-04-02T13:38:47.063088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:26:42.032550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108576,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600843,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":859,"num_read_kb":543,"num_write":299,"num_write_kb":1450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1802529,"data_compressed":80724,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":532498,"data_compressed":29620,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36516,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708636,"statfs":{"total":95995035648,"available":95957643264,"internally_reserved":0,"allocated":1208320,"data_stored":1322367,"data_compressed":73574,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1085440,"data_stored":930046,"data_compressed":48766,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":925288,"data_compressed":48766,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":1131395,"data_compressed":65812,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":527910,"data_compressed":29620,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1110016,"data_stored":597596,"data_compressed":31226,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":519432,"data_compressed":26636,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1130496,"data_stored":599780,"data_compressed":31226,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":1052920,"data_compressed":54922,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888141,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":534367,"data_compressed":29620,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1256776,"data_compressed":71968,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69776,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71960,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134492,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393372,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":204179,"data_compressed":7762,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:38:52.433 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-02T13:38:52.433 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:38:52.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.569+0000 7f1cffffa640 1 -- 172.21.15.33:0/824451521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf8100c20 msgr2=0x7f1cf8106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.569+0000 7f1cffffa640 1 --2- 172.21.15.33:0/824451521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf8100c20 0x7f1cf8106750 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009900 tx=0x7f1ce802bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:52.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.569+0000 7f1cffffa640 1 -- 172.21.15.33:0/824451521 shutdown_connections 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.569+0000 7f1cffffa640 1 --2- 172.21.15.33:0/824451521 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cf8106e80 0x7f1cf810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.569+0000 7f1cffffa640 1 --2- 172.21.15.33:0/824451521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf8100c20 0x7f1cf8106750 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.569+0000 7f1cffffa640 1 --2- 172.21.15.33:0/824451521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cf8100270 0x7f1cf8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.569+0000 7f1cffffa640 1 -- 172.21.15.33:0/824451521 >> 172.21.15.33:0/824451521 conn(0x7f1cf8076f90 msgr2=0x7f1cf80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.569+0000 7f1cffffa640 1 -- 172.21.15.33:0/824451521 shutdown_connections 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.569+0000 7f1cffffa640 1 -- 172.21.15.33:0/824451521 wait complete. 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cffffa640 1 Processor -- start 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cffffa640 1 -- start start 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cffffa640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cf8100270 0x7f1cf81011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cffffa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf8106e80 0x7f1cf8101720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cffffa640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cf8101c60 0x7f1cf81a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cffffa640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf8111eb0 con 0x7f1cf8106e80 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cffffa640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf8111d30 con 0x7f1cf8100270 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cffffa640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf8112030 con 0x7f1cf8101c60 2024-04-02T13:38:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cfd595640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf8106e80 0x7f1cf8101720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cfd595640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf8106e80 0x7f1cf8101720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59412/0 (socket says 172.21.15.33:59412) 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cfd595640 1 -- 172.21.15.33:0/2596015960 learned_addr learned my addr 172.21.15.33:0/2596015960 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cfdd96640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cf8100270 0x7f1cf81011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cfe597640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cf8101c60 0x7f1cf81a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cfd595640 1 -- 172.21.15.33:0/2596015960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cf8101c60 msgr2=0x7f1cf81a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cfd595640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cf8101c60 0x7f1cf81a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cfd595640 1 -- 172.21.15.33:0/2596015960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cf8100270 msgr2=0x7f1cf81011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cfd595640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cf8100270 0x7f1cf81011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.570+0000 7f1cfd595640 1 -- 172.21.15.33:0/2596015960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf81a6ce0 con 0x7f1cf8106e80 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.571+0000 7f1cfd595640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf8106e80 0x7f1cf8101720 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8005ab0 tx=0x7f1ce80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.571+0000 7f1ceeffd640 1 -- 172.21.15.33:0/2596015960 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8043020 con 0x7f1cf8106e80 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.571+0000 7f1ceeffd640 1 -- 172.21.15.33:0/2596015960 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1ce802caa0 con 0x7f1cf8106e80 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.571+0000 7f1cffffa640 1 -- 172.21.15.33:0/2596015960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf81a6f70 con 0x7f1cf8106e80 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.571+0000 7f1ceeffd640 1 -- 172.21.15.33:0/2596015960 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8003b50 con 0x7f1cf8106e80 2024-04-02T13:38:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.571+0000 7f1cffffa640 1 -- 172.21.15.33:0/2596015960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf81a7430 con 0x7f1cf8106e80 2024-04-02T13:38:52.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.572+0000 7f1cffffa640 1 -- 172.21.15.33:0/2596015960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf8103320 con 0x7f1cf8106e80 2024-04-02T13:38:52.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.573+0000 7f1ceeffd640 1 -- 172.21.15.33:0/2596015960 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f1ce8046020 con 0x7f1cf8106e80 2024-04-02T13:38:52.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.574+0000 7f1ceeffd640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1ce40a3e70 0x7f1ce40a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:52.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.574+0000 7f1cfdd96640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1ce40a3e70 0x7f1ce40a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:52.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.574+0000 7f1ceeffd640 1 -- 172.21.15.33:0/2596015960 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1ce80f2840 con 0x7f1cf8106e80 2024-04-02T13:38:52.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.574+0000 7f1cfdd96640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1ce40a3e70 0x7f1ce40a6330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400bbe0 tx=0x7f1cf400b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:52.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.578+0000 7f1ceeffd640 1 -- 172.21.15.33:0/2596015960 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1ce804d070 con 0x7f1cf8106e80 2024-04-02T13:38:52.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.764+0000 7f1cffffa640 1 -- 172.21.15.33:0/2596015960 --> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1cf8100c20 con 0x7f1ce40a3e70 2024-04-02T13:38:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.774+0000 7f1ceeffd640 1 -- 172.21.15.33:0/2596015960 <== mgr.15456 v2:172.21.15.33:6800/619073725 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339690 (secure 0 0 0) 0x7f1cf8100c20 con 0x7f1ce40a3e70 2024-04-02T13:38:52.780 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-02T13:38:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.777+0000 7f1cffffa640 1 -- 172.21.15.33:0/2596015960 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1ce40a3e70 msgr2=0x7f1ce40a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.777+0000 7f1cffffa640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1ce40a3e70 0x7f1ce40a6330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400bbe0 tx=0x7f1cf400b390 comp rx=0 tx=0).stop 2024-04-02T13:38:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.777+0000 7f1cffffa640 1 -- 172.21.15.33:0/2596015960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf8106e80 msgr2=0x7f1cf8101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.777+0000 7f1cffffa640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf8106e80 0x7f1cf8101720 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8005ab0 tx=0x7f1ce80045e0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.777+0000 7f1cffffa640 1 -- 172.21.15.33:0/2596015960 shutdown_connections 2024-04-02T13:38:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.777+0000 7f1cffffa640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1ce40a3e70 0x7f1ce40a6330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.777+0000 7f1cffffa640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1cf8101c60 0x7f1cf81a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.777+0000 7f1cffffa640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf8106e80 0x7f1cf8101720 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.777+0000 7f1cffffa640 1 --2- 172.21.15.33:0/2596015960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cf8100270 0x7f1cf81011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.777+0000 7f1cffffa640 1 -- 172.21.15.33:0/2596015960 >> 172.21.15.33:0/2596015960 conn(0x7f1cf8076f90 msgr2=0x7f1cf8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.777+0000 7f1cffffa640 1 -- 172.21.15.33:0/2596015960 shutdown_connections 2024-04-02T13:38:52.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.777+0000 7f1cffffa640 1 -- 172.21.15.33:0/2596015960 wait complete. 2024-04-02T13:38:52.791 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-02T13:38:50.885512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191445,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":995,"num_read_kb":662,"num_write":482,"num_write_kb":4390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":876,"ondisk_log_size":876,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374484,"kb_used_data":12812,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567340,"statfs":{"total":1151940427776,"available":1151556956160,"internally_reserved":0,"allocated":13119488,"data_stored":9930375,"data_compressed":541756,"data_compressed_allocated":733184,"data_compressed_original":4317184,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987578"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063585+0000","last_change":"2024-04-02T13:36:21.492436+0000","last_active":"2024-04-02T13:38:47.063585+0000","last_peered":"2024-04-02T13:38:47.063585+0000","last_clean":"2024-04-02T13:38:47.063585+0000","last_became_active":"2024-04-02T13:36:21.491831+0000","last_became_peered":"2024-04-02T13:36:21.491831+0000","last_unstale":"2024-04-02T13:38:47.063585+0000","last_undegraded":"2024-04-02T13:38:47.063585+0000","last_fullsized":"2024-04-02T13:38:47.063585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:50:57.919680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250790+0000","last_change":"2024-04-02T13:36:21.498089+0000","last_active":"2024-04-02T13:38:47.250790+0000","last_peered":"2024-04-02T13:38:47.250790+0000","last_clean":"2024-04-02T13:38:47.250790+0000","last_became_active":"2024-04-02T13:36:21.497784+0000","last_became_peered":"2024-04-02T13:36:21.497784+0000","last_unstale":"2024-04-02T13:38:47.250790+0000","last_undegraded":"2024-04-02T13:38:47.250790+0000","last_fullsized":"2024-04-02T13:38:47.250790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:42.137396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250172+0000","last_change":"2024-04-02T13:36:21.492207+0000","last_active":"2024-04-02T13:38:47.250172+0000","last_peered":"2024-04-02T13:38:47.250172+0000","last_clean":"2024-04-02T13:38:47.250172+0000","last_became_active":"2024-04-02T13:36:21.490605+0000","last_became_peered":"2024-04-02T13:36:21.490605+0000","last_unstale":"2024-04-02T13:38:47.250172+0000","last_undegraded":"2024-04-02T13:38:47.250172+0000","last_fullsized":"2024-04-02T13:38:47.250172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:03.984226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874419+0000","last_change":"2024-04-02T13:36:21.491225+0000","last_active":"2024-04-02T13:38:46.874419+0000","last_peered":"2024-04-02T13:38:46.874419+0000","last_clean":"2024-04-02T13:38:46.874419+0000","last_became_active":"2024-04-02T13:36:21.490886+0000","last_became_peered":"2024-04-02T13:36:21.490886+0000","last_unstale":"2024-04-02T13:38:46.874419+0000","last_undegraded":"2024-04-02T13:38:46.874419+0000","last_fullsized":"2024-04-02T13:38:46.874419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:55:53.316243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877507+0000","last_change":"2024-04-02T13:36:21.495808+0000","last_active":"2024-04-02T13:38:46.877507+0000","last_peered":"2024-04-02T13:38:46.877507+0000","last_clean":"2024-04-02T13:38:46.877507+0000","last_became_active":"2024-04-02T13:36:21.495468+0000","last_became_peered":"2024-04-02T13:36:21.495468+0000","last_unstale":"2024-04-02T13:38:46.877507+0000","last_undegraded":"2024-04-02T13:38:46.877507+0000","last_fullsized":"2024-04-02T13:38:46.877507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:19:17.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982665+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:38:46.982665+0000","last_peered":"2024-04-02T13:38:46.982665+0000","last_clean":"2024-04-02T13:38:46.982665+0000","last_became_active":"2024-04-02T13:36:21.499200+0000","last_became_peered":"2024-04-02T13:36:21.499200+0000","last_unstale":"2024-04-02T13:38:46.982665+0000","last_undegraded":"2024-04-02T13:38:46.982665+0000","last_fullsized":"2024-04-02T13:38:46.982665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:54:52.113844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062959+0000","last_change":"2024-04-02T13:36:21.493429+0000","last_active":"2024-04-02T13:38:47.062959+0000","last_peered":"2024-04-02T13:38:47.062959+0000","last_clean":"2024-04-02T13:38:47.062959+0000","last_became_active":"2024-04-02T13:36:21.493097+0000","last_became_peered":"2024-04-02T13:36:21.493097+0000","last_unstale":"2024-04-02T13:38:47.062959+0000","last_undegraded":"2024-04-02T13:38:47.062959+0000","last_fullsized":"2024-04-02T13:38:47.062959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:47:03.330448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.969983+0000","last_change":"2024-04-02T13:38:18.511280+0000","last_active":"2024-04-02T13:38:46.969983+0000","last_peered":"2024-04-02T13:38:46.969983+0000","last_clean":"2024-04-02T13:38:46.969983+0000","last_became_active":"2024-04-02T13:38:18.510953+0000","last_became_peered":"2024-04-02T13:38:18.510953+0000","last_unstale":"2024-04-02T13:38:46.969983+0000","last_undegraded":"2024-04-02T13:38:46.969983+0000","last_fullsized":"2024-04-02T13:38:46.969983+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:51:38.448445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063843+0000","last_change":"2024-04-02T13:36:21.490623+0000","last_active":"2024-04-02T13:38:47.063843+0000","last_peered":"2024-04-02T13:38:47.063843+0000","last_clean":"2024-04-02T13:38:47.063843+0000","last_became_active":"2024-04-02T13:36:21.490274+0000","last_became_peered":"2024-04-02T13:36:21.490274+0000","last_unstale":"2024-04-02T13:38:47.063843+0000","last_undegraded":"2024-04-02T13:38:47.063843+0000","last_fullsized":"2024-04-02T13:38:47.063843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:59:35.220430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982805+0000","last_change":"2024-04-02T13:37:17.631180+0000","last_active":"2024-04-02T13:38:46.982805+0000","last_peered":"2024-04-02T13:38:46.982805+0000","last_clean":"2024-04-02T13:38:46.982805+0000","last_became_active":"2024-04-02T13:37:17.630778+0000","last_became_peered":"2024-04-02T13:37:17.630778+0000","last_unstale":"2024-04-02T13:38:46.982805+0000","last_undegraded":"2024-04-02T13:38:46.982805+0000","last_fullsized":"2024-04-02T13:38:46.982805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:05:11.445074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063424+0000","last_change":"2024-04-02T13:36:21.492510+0000","last_active":"2024-04-02T13:38:47.063424+0000","last_peered":"2024-04-02T13:38:47.063424+0000","last_clean":"2024-04-02T13:38:47.063424+0000","last_became_active":"2024-04-02T13:36:21.492189+0000","last_became_peered":"2024-04-02T13:36:21.492189+0000","last_unstale":"2024-04-02T13:38:47.063424+0000","last_undegraded":"2024-04-02T13:38:47.063424+0000","last_fullsized":"2024-04-02T13:38:47.063424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:52:15.321621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874446+0000","last_change":"2024-04-02T13:36:21.498015+0000","last_active":"2024-04-02T13:38:46.874446+0000","last_peered":"2024-04-02T13:38:46.874446+0000","last_clean":"2024-04-02T13:38:46.874446+0000","last_became_active":"2024-04-02T13:36:21.497677+0000","last_became_peered":"2024-04-02T13:36:21.497677+0000","last_unstale":"2024-04-02T13:38:46.874446+0000","last_undegraded":"2024-04-02T13:38:46.874446+0000","last_fullsized":"2024-04-02T13:38:46.874446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:43:19.035902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063126+0000","last_change":"2024-04-02T13:36:21.497047+0000","last_active":"2024-04-02T13:38:47.063126+0000","last_peered":"2024-04-02T13:38:47.063126+0000","last_clean":"2024-04-02T13:38:47.063126+0000","last_became_active":"2024-04-02T13:36:21.496722+0000","last_became_peered":"2024-04-02T13:36:21.496722+0000","last_unstale":"2024-04-02T13:38:47.063126+0000","last_undegraded":"2024-04-02T13:38:47.063126+0000","last_fullsized":"2024-04-02T13:38:47.063126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:03:14.400687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250295+0000","last_change":"2024-04-02T13:36:21.496513+0000","last_active":"2024-04-02T13:38:47.250295+0000","last_peered":"2024-04-02T13:38:47.250295+0000","last_clean":"2024-04-02T13:38:47.250295+0000","last_became_active":"2024-04-02T13:36:21.496209+0000","last_became_peered":"2024-04-02T13:36:21.496209+0000","last_unstale":"2024-04-02T13:38:47.250295+0000","last_undegraded":"2024-04-02T13:38:47.250295+0000","last_fullsized":"2024-04-02T13:38:47.250295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:50.684206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250238+0000","last_change":"2024-04-02T13:36:21.493418+0000","last_active":"2024-04-02T13:38:47.250238+0000","last_peered":"2024-04-02T13:38:47.250238+0000","last_clean":"2024-04-02T13:38:47.250238+0000","last_became_active":"2024-04-02T13:36:21.492785+0000","last_became_peered":"2024-04-02T13:36:21.492785+0000","last_unstale":"2024-04-02T13:38:47.250238+0000","last_undegraded":"2024-04-02T13:38:47.250238+0000","last_fullsized":"2024-04-02T13:38:47.250238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:11.973683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062414+0000","last_change":"2024-04-02T13:36:21.490710+0000","last_active":"2024-04-02T13:38:47.062414+0000","last_peered":"2024-04-02T13:38:47.062414+0000","last_clean":"2024-04-02T13:38:47.062414+0000","last_became_active":"2024-04-02T13:36:21.489654+0000","last_became_peered":"2024-04-02T13:36:21.489654+0000","last_unstale":"2024-04-02T13:38:47.062414+0000","last_undegraded":"2024-04-02T13:38:47.062414+0000","last_fullsized":"2024-04-02T13:38:47.062414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:43:06.198540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063474+0000","last_change":"2024-04-02T13:36:21.493306+0000","last_active":"2024-04-02T13:38:47.063474+0000","last_peered":"2024-04-02T13:38:47.063474+0000","last_clean":"2024-04-02T13:38:47.063474+0000","last_became_active":"2024-04-02T13:36:21.492949+0000","last_became_peered":"2024-04-02T13:36:21.492949+0000","last_unstale":"2024-04-02T13:38:47.063474+0000","last_undegraded":"2024-04-02T13:38:47.063474+0000","last_fullsized":"2024-04-02T13:38:47.063474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:34.264790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970460+0000","last_change":"2024-04-02T13:36:21.496631+0000","last_active":"2024-04-02T13:38:46.970460+0000","last_peered":"2024-04-02T13:38:46.970460+0000","last_clean":"2024-04-02T13:38:46.970460+0000","last_became_active":"2024-04-02T13:36:21.496245+0000","last_became_peered":"2024-04-02T13:36:21.496245+0000","last_unstale":"2024-04-02T13:38:46.970460+0000","last_undegraded":"2024-04-02T13:38:46.970460+0000","last_fullsized":"2024-04-02T13:38:46.970460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:14:51.126381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063307+0000","last_change":"2024-04-02T13:36:21.491640+0000","last_active":"2024-04-02T13:38:47.063307+0000","last_peered":"2024-04-02T13:38:47.063307+0000","last_clean":"2024-04-02T13:38:47.063307+0000","last_became_active":"2024-04-02T13:36:21.487799+0000","last_became_peered":"2024-04-02T13:36:21.487799+0000","last_unstale":"2024-04-02T13:38:47.063307+0000","last_undegraded":"2024-04-02T13:38:47.063307+0000","last_fullsized":"2024-04-02T13:38:47.063307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:16.893295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062642+0000","last_change":"2024-04-02T13:36:25.499691+0000","last_active":"2024-04-02T13:38:47.062642+0000","last_peered":"2024-04-02T13:38:47.062642+0000","last_clean":"2024-04-02T13:38:47.062642+0000","last_became_active":"2024-04-02T13:36:25.499120+0000","last_became_peered":"2024-04-02T13:36:25.499120+0000","last_unstale":"2024-04-02T13:38:47.062642+0000","last_undegraded":"2024-04-02T13:38:47.062642+0000","last_fullsized":"2024-04-02T13:38:47.062642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:47:53.823456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970931+0000","last_change":"2024-04-02T13:37:17.878343+0000","last_active":"2024-04-02T13:38:46.970931+0000","last_peered":"2024-04-02T13:38:46.970931+0000","last_clean":"2024-04-02T13:38:46.970931+0000","last_became_active":"2024-04-02T13:37:17.871984+0000","last_became_peered":"2024-04-02T13:37:17.871984+0000","last_unstale":"2024-04-02T13:38:46.970931+0000","last_undegraded":"2024-04-02T13:38:46.970931+0000","last_fullsized":"2024-04-02T13:38:46.970931+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:28.804070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872842+0000","last_change":"2024-04-02T13:36:21.499019+0000","last_active":"2024-04-02T13:38:46.872842+0000","last_peered":"2024-04-02T13:38:46.872842+0000","last_clean":"2024-04-02T13:38:46.872842+0000","last_became_active":"2024-04-02T13:36:21.498684+0000","last_became_peered":"2024-04-02T13:36:21.498684+0000","last_unstale":"2024-04-02T13:38:46.872842+0000","last_undegraded":"2024-04-02T13:38:46.872842+0000","last_fullsized":"2024-04-02T13:38:46.872842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:15.618382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.873916+0000","last_change":"2024-04-02T13:36:25.498544+0000","last_active":"2024-04-02T13:38:46.873916+0000","last_peered":"2024-04-02T13:38:46.873916+0000","last_clean":"2024-04-02T13:38:46.873916+0000","last_became_active":"2024-04-02T13:36:25.498076+0000","last_became_peered":"2024-04-02T13:36:25.498076+0000","last_unstale":"2024-04-02T13:38:46.873916+0000","last_undegraded":"2024-04-02T13:38:46.873916+0000","last_fullsized":"2024-04-02T13:38:46.873916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:04:47.311056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.873130+0000","last_change":"2024-04-02T13:36:20.832649+0000","last_active":"2024-04-02T13:38:46.873130+0000","last_peered":"2024-04-02T13:38:46.873130+0000","last_clean":"2024-04-02T13:38:46.873130+0000","last_became_active":"2024-04-02T13:36:20.832184+0000","last_became_peered":"2024-04-02T13:36:20.832184+0000","last_unstale":"2024-04-02T13:38:46.873130+0000","last_undegraded":"2024-04-02T13:38:46.873130+0000","last_fullsized":"2024-04-02T13:38:46.873130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:37.639406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250670+0000","last_change":"2024-04-02T13:36:21.489988+0000","last_active":"2024-04-02T13:38:47.250670+0000","last_peered":"2024-04-02T13:38:47.250670+0000","last_clean":"2024-04-02T13:38:47.250670+0000","last_became_active":"2024-04-02T13:36:21.489636+0000","last_became_peered":"2024-04-02T13:36:21.489636+0000","last_unstale":"2024-04-02T13:38:47.250670+0000","last_undegraded":"2024-04-02T13:38:47.250670+0000","last_fullsized":"2024-04-02T13:38:47.250670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:52:36.670712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874051+0000","last_change":"2024-04-02T13:36:25.499441+0000","last_active":"2024-04-02T13:38:46.874051+0000","last_peered":"2024-04-02T13:38:46.874051+0000","last_clean":"2024-04-02T13:38:46.874051+0000","last_became_active":"2024-04-02T13:36:25.498512+0000","last_became_peered":"2024-04-02T13:36:25.498512+0000","last_unstale":"2024-04-02T13:38:46.874051+0000","last_undegraded":"2024-04-02T13:38:46.874051+0000","last_fullsized":"2024-04-02T13:38:46.874051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:48:59.291360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878544+0000","last_change":"2024-04-02T13:36:20.830008+0000","last_active":"2024-04-02T13:38:46.878544+0000","last_peered":"2024-04-02T13:38:46.878544+0000","last_clean":"2024-04-02T13:38:46.878544+0000","last_became_active":"2024-04-02T13:36:20.829048+0000","last_became_peered":"2024-04-02T13:36:20.829048+0000","last_unstale":"2024-04-02T13:38:46.878544+0000","last_undegraded":"2024-04-02T13:38:46.878544+0000","last_fullsized":"2024-04-02T13:38:46.878544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:08:34.405351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062882+0000","last_change":"2024-04-02T13:36:21.497296+0000","last_active":"2024-04-02T13:38:47.062882+0000","last_peered":"2024-04-02T13:38:47.062882+0000","last_clean":"2024-04-02T13:38:47.062882+0000","last_became_active":"2024-04-02T13:36:21.496802+0000","last_became_peered":"2024-04-02T13:36:21.496802+0000","last_unstale":"2024-04-02T13:38:47.062882+0000","last_undegraded":"2024-04-02T13:38:47.062882+0000","last_fullsized":"2024-04-02T13:38:47.062882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:03.995029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874086+0000","last_change":"2024-04-02T13:36:25.498661+0000","last_active":"2024-04-02T13:38:46.874086+0000","last_peered":"2024-04-02T13:38:46.874086+0000","last_clean":"2024-04-02T13:38:46.874086+0000","last_became_active":"2024-04-02T13:36:25.497866+0000","last_became_peered":"2024-04-02T13:36:25.497866+0000","last_unstale":"2024-04-02T13:38:46.874086+0000","last_undegraded":"2024-04-02T13:38:46.874086+0000","last_fullsized":"2024-04-02T13:38:46.874086+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:47:05.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878040+0000","last_change":"2024-04-02T13:36:20.824586+0000","last_active":"2024-04-02T13:38:46.878040+0000","last_peered":"2024-04-02T13:38:46.878040+0000","last_clean":"2024-04-02T13:38:46.878040+0000","last_became_active":"2024-04-02T13:36:20.824254+0000","last_became_peered":"2024-04-02T13:36:20.824254+0000","last_unstale":"2024-04-02T13:38:46.878040+0000","last_undegraded":"2024-04-02T13:38:46.878040+0000","last_fullsized":"2024-04-02T13:38:46.878040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:08.774361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979403+0000","last_change":"2024-04-02T13:36:21.495125+0000","last_active":"2024-04-02T13:38:46.979403+0000","last_peered":"2024-04-02T13:38:46.979403+0000","last_clean":"2024-04-02T13:38:46.979403+0000","last_became_active":"2024-04-02T13:36:21.494724+0000","last_became_peered":"2024-04-02T13:36:21.494724+0000","last_unstale":"2024-04-02T13:38:46.979403+0000","last_undegraded":"2024-04-02T13:38:46.979403+0000","last_fullsized":"2024-04-02T13:38:46.979403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:39:27.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063127+0000","last_change":"2024-04-02T13:36:25.498708+0000","last_active":"2024-04-02T13:38:47.063127+0000","last_peered":"2024-04-02T13:38:47.063127+0000","last_clean":"2024-04-02T13:38:47.063127+0000","last_became_active":"2024-04-02T13:36:25.497834+0000","last_became_peered":"2024-04-02T13:36:25.497834+0000","last_unstale":"2024-04-02T13:38:47.063127+0000","last_undegraded":"2024-04-02T13:38:47.063127+0000","last_fullsized":"2024-04-02T13:38:47.063127+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:14.524569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":169,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.675130+0000","last_change":"2024-04-02T13:36:20.824901+0000","last_active":"2024-04-02T13:38:47.675130+0000","last_peered":"2024-04-02T13:38:47.675130+0000","last_clean":"2024-04-02T13:38:47.675130+0000","last_became_active":"2024-04-02T13:36:20.823973+0000","last_became_peered":"2024-04-02T13:36:20.823973+0000","last_unstale":"2024-04-02T13:38:47.675130+0000","last_undegraded":"2024-04-02T13:38:47.675130+0000","last_fullsized":"2024-04-02T13:38:47.675130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:17.636527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877414+0000","last_change":"2024-04-02T13:36:21.493411+0000","last_active":"2024-04-02T13:38:46.877414+0000","last_peered":"2024-04-02T13:38:46.877414+0000","last_clean":"2024-04-02T13:38:46.877414+0000","last_became_active":"2024-04-02T13:36:21.493061+0000","last_became_peered":"2024-04-02T13:36:21.493061+0000","last_unstale":"2024-04-02T13:38:46.877414+0000","last_undegraded":"2024-04-02T13:38:46.877414+0000","last_fullsized":"2024-04-02T13:38:46.877414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:15.902656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063044+0000","last_change":"2024-04-02T13:36:25.494817+0000","last_active":"2024-04-02T13:38:47.063044+0000","last_peered":"2024-04-02T13:38:47.063044+0000","last_clean":"2024-04-02T13:38:47.063044+0000","last_became_active":"2024-04-02T13:36:25.494185+0000","last_became_peered":"2024-04-02T13:36:25.494185+0000","last_unstale":"2024-04-02T13:38:47.063044+0000","last_undegraded":"2024-04-02T13:38:47.063044+0000","last_fullsized":"2024-04-02T13:38:47.063044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:04:32.719985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982429+0000","last_change":"2024-04-02T13:36:20.821981+0000","last_active":"2024-04-02T13:38:46.982429+0000","last_peered":"2024-04-02T13:38:46.982429+0000","last_clean":"2024-04-02T13:38:46.982429+0000","last_became_active":"2024-04-02T13:36:20.821074+0000","last_became_peered":"2024-04-02T13:36:20.821074+0000","last_unstale":"2024-04-02T13:38:46.982429+0000","last_undegraded":"2024-04-02T13:38:46.982429+0000","last_fullsized":"2024-04-02T13:38:46.982429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:22.094586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970127+0000","last_change":"2024-04-02T13:38:18.512127+0000","last_active":"2024-04-02T13:38:46.970127+0000","last_peered":"2024-04-02T13:38:46.970127+0000","last_clean":"2024-04-02T13:38:46.970127+0000","last_became_active":"2024-04-02T13:38:18.511729+0000","last_became_peered":"2024-04-02T13:38:18.511729+0000","last_unstale":"2024-04-02T13:38:46.970127+0000","last_undegraded":"2024-04-02T13:38:46.970127+0000","last_fullsized":"2024-04-02T13:38:46.970127+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:12:02.320973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062694+0000","last_change":"2024-04-02T13:36:25.503774+0000","last_active":"2024-04-02T13:38:47.062694+0000","last_peered":"2024-04-02T13:38:47.062694+0000","last_clean":"2024-04-02T13:38:47.062694+0000","last_became_active":"2024-04-02T13:36:25.503287+0000","last_became_peered":"2024-04-02T13:36:25.503287+0000","last_unstale":"2024-04-02T13:38:47.062694+0000","last_undegraded":"2024-04-02T13:38:47.062694+0000","last_fullsized":"2024-04-02T13:38:47.062694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:57:29.939484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063048+0000","last_change":"2024-04-02T13:36:20.816760+0000","last_active":"2024-04-02T13:38:47.063048+0000","last_peered":"2024-04-02T13:38:47.063048+0000","last_clean":"2024-04-02T13:38:47.063048+0000","last_became_active":"2024-04-02T13:36:20.816291+0000","last_became_peered":"2024-04-02T13:36:20.816291+0000","last_unstale":"2024-04-02T13:38:47.063048+0000","last_undegraded":"2024-04-02T13:38:47.063048+0000","last_fullsized":"2024-04-02T13:38:47.063048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:25.543946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872422+0000","last_change":"2024-04-02T13:36:21.498541+0000","last_active":"2024-04-02T13:38:46.872422+0000","last_peered":"2024-04-02T13:38:46.872422+0000","last_clean":"2024-04-02T13:38:46.872422+0000","last_became_active":"2024-04-02T13:36:21.497498+0000","last_became_peered":"2024-04-02T13:36:21.497498+0000","last_unstale":"2024-04-02T13:38:46.872422+0000","last_undegraded":"2024-04-02T13:38:46.872422+0000","last_fullsized":"2024-04-02T13:38:46.872422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:48:08.315938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250671+0000","last_change":"2024-04-02T13:36:25.500245+0000","last_active":"2024-04-02T13:38:47.250671+0000","last_peered":"2024-04-02T13:38:47.250671+0000","last_clean":"2024-04-02T13:38:47.250671+0000","last_became_active":"2024-04-02T13:36:25.499844+0000","last_became_peered":"2024-04-02T13:36:25.499844+0000","last_unstale":"2024-04-02T13:38:47.250671+0000","last_undegraded":"2024-04-02T13:38:47.250671+0000","last_fullsized":"2024-04-02T13:38:47.250671+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:17:01.105757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979010+0000","last_change":"2024-04-02T13:36:20.826851+0000","last_active":"2024-04-02T13:38:46.979010+0000","last_peered":"2024-04-02T13:38:46.979010+0000","last_clean":"2024-04-02T13:38:46.979010+0000","last_became_active":"2024-04-02T13:36:20.826227+0000","last_became_peered":"2024-04-02T13:36:20.826227+0000","last_unstale":"2024-04-02T13:38:46.979010+0000","last_undegraded":"2024-04-02T13:38:46.979010+0000","last_fullsized":"2024-04-02T13:38:46.979010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:45.841966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970174+0000","last_change":"2024-04-02T13:36:21.496520+0000","last_active":"2024-04-02T13:38:46.970174+0000","last_peered":"2024-04-02T13:38:46.970174+0000","last_clean":"2024-04-02T13:38:46.970174+0000","last_became_active":"2024-04-02T13:36:21.496082+0000","last_became_peered":"2024-04-02T13:36:21.496082+0000","last_unstale":"2024-04-02T13:38:46.970174+0000","last_undegraded":"2024-04-02T13:38:46.970174+0000","last_fullsized":"2024-04-02T13:38:46.970174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:12:06.517806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872513+0000","last_change":"2024-04-02T13:36:25.496064+0000","last_active":"2024-04-02T13:38:46.872513+0000","last_peered":"2024-04-02T13:38:46.872513+0000","last_clean":"2024-04-02T13:38:46.872513+0000","last_became_active":"2024-04-02T13:36:25.495464+0000","last_became_peered":"2024-04-02T13:36:25.495464+0000","last_unstale":"2024-04-02T13:38:46.872513+0000","last_undegraded":"2024-04-02T13:38:46.872513+0000","last_fullsized":"2024-04-02T13:38:46.872513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:06:15.206135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'5","reported_seq":161,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.674536+0000","last_change":"2024-04-02T13:36:20.821898+0000","last_active":"2024-04-02T13:38:47.674536+0000","last_peered":"2024-04-02T13:38:47.674536+0000","last_clean":"2024-04-02T13:38:47.674536+0000","last_became_active":"2024-04-02T13:36:20.821374+0000","last_became_peered":"2024-04-02T13:36:20.821374+0000","last_unstale":"2024-04-02T13:38:47.674536+0000","last_undegraded":"2024-04-02T13:38:47.674536+0000","last_fullsized":"2024-04-02T13:38:47.674536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:21.568976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063510+0000","last_change":"2024-04-02T13:36:21.497343+0000","last_active":"2024-04-02T13:38:47.063510+0000","last_peered":"2024-04-02T13:38:47.063510+0000","last_clean":"2024-04-02T13:38:47.063510+0000","last_became_active":"2024-04-02T13:36:21.496733+0000","last_became_peered":"2024-04-02T13:36:21.496733+0000","last_unstale":"2024-04-02T13:38:47.063510+0000","last_undegraded":"2024-04-02T13:38:47.063510+0000","last_fullsized":"2024-04-02T13:38:47.063510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:16:26.068717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970159+0000","last_change":"2024-04-02T13:36:25.503011+0000","last_active":"2024-04-02T13:38:46.970159+0000","last_peered":"2024-04-02T13:38:46.970159+0000","last_clean":"2024-04-02T13:38:46.970159+0000","last_became_active":"2024-04-02T13:36:25.502571+0000","last_became_peered":"2024-04-02T13:36:25.502571+0000","last_unstale":"2024-04-02T13:38:46.970159+0000","last_undegraded":"2024-04-02T13:38:46.970159+0000","last_fullsized":"2024-04-02T13:38:46.970159+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:48.672023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063668+0000","last_change":"2024-04-02T13:36:20.827705+0000","last_active":"2024-04-02T13:38:47.063668+0000","last_peered":"2024-04-02T13:38:47.063668+0000","last_clean":"2024-04-02T13:38:47.063668+0000","last_became_active":"2024-04-02T13:36:20.827285+0000","last_became_peered":"2024-04-02T13:36:20.827285+0000","last_unstale":"2024-04-02T13:38:47.063668+0000","last_undegraded":"2024-04-02T13:38:47.063668+0000","last_fullsized":"2024-04-02T13:38:47.063668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:12.175523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250885+0000","last_change":"2024-04-02T13:36:21.493365+0000","last_active":"2024-04-02T13:38:47.250885+0000","last_peered":"2024-04-02T13:38:47.250885+0000","last_clean":"2024-04-02T13:38:47.250885+0000","last_became_active":"2024-04-02T13:36:21.493011+0000","last_became_peered":"2024-04-02T13:36:21.493011+0000","last_unstale":"2024-04-02T13:38:47.250885+0000","last_undegraded":"2024-04-02T13:38:47.250885+0000","last_fullsized":"2024-04-02T13:38:47.250885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:53.165244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874354+0000","last_change":"2024-04-02T13:36:25.499017+0000","last_active":"2024-04-02T13:38:46.874354+0000","last_peered":"2024-04-02T13:38:46.874354+0000","last_clean":"2024-04-02T13:38:46.874354+0000","last_became_active":"2024-04-02T13:36:25.498293+0000","last_became_peered":"2024-04-02T13:36:25.498293+0000","last_unstale":"2024-04-02T13:38:46.874354+0000","last_undegraded":"2024-04-02T13:38:46.874354+0000","last_fullsized":"2024-04-02T13:38:46.874354+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:55.910943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979425+0000","last_change":"2024-04-02T13:36:20.836345+0000","last_active":"2024-04-02T13:38:46.979425+0000","last_peered":"2024-04-02T13:38:46.979425+0000","last_clean":"2024-04-02T13:38:46.979425+0000","last_became_active":"2024-04-02T13:36:20.836037+0000","last_became_peered":"2024-04-02T13:36:20.836037+0000","last_unstale":"2024-04-02T13:38:46.979425+0000","last_undegraded":"2024-04-02T13:38:46.979425+0000","last_fullsized":"2024-04-02T13:38:46.979425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:37:47.070493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063302+0000","last_change":"2024-04-02T13:36:21.493209+0000","last_active":"2024-04-02T13:38:47.063302+0000","last_peered":"2024-04-02T13:38:47.063302+0000","last_clean":"2024-04-02T13:38:47.063302+0000","last_became_active":"2024-04-02T13:36:21.492894+0000","last_became_peered":"2024-04-02T13:36:21.492894+0000","last_unstale":"2024-04-02T13:38:47.063302+0000","last_undegraded":"2024-04-02T13:38:47.063302+0000","last_fullsized":"2024-04-02T13:38:47.063302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:47.162354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982294+0000","last_change":"2024-04-02T13:37:17.626933+0000","last_active":"2024-04-02T13:38:46.982294+0000","last_peered":"2024-04-02T13:38:46.982294+0000","last_clean":"2024-04-02T13:38:46.982294+0000","last_became_active":"2024-04-02T13:37:17.626411+0000","last_became_peered":"2024-04-02T13:37:17.626411+0000","last_unstale":"2024-04-02T13:38:46.982294+0000","last_undegraded":"2024-04-02T13:38:46.982294+0000","last_fullsized":"2024-04-02T13:38:46.982294+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:40:56.903871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":330,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.675659+0000","last_change":"2024-04-02T13:36:20.823563+0000","last_active":"2024-04-02T13:38:47.675659+0000","last_peered":"2024-04-02T13:38:47.675659+0000","last_clean":"2024-04-02T13:38:47.675659+0000","last_became_active":"2024-04-02T13:36:20.823186+0000","last_became_peered":"2024-04-02T13:36:20.823186+0000","last_unstale":"2024-04-02T13:38:47.675659+0000","last_undegraded":"2024-04-02T13:38:47.675659+0000","last_fullsized":"2024-04-02T13:38:47.675659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:50.983357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69194,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":255,"num_read_kb":195,"num_write":42,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878277+0000","last_change":"2024-04-02T13:36:21.500829+0000","last_active":"2024-04-02T13:38:46.878277+0000","last_peered":"2024-04-02T13:38:46.878277+0000","last_clean":"2024-04-02T13:38:46.878277+0000","last_became_active":"2024-04-02T13:36:21.500485+0000","last_became_peered":"2024-04-02T13:36:21.500485+0000","last_unstale":"2024-04-02T13:38:46.878277+0000","last_undegraded":"2024-04-02T13:38:46.878277+0000","last_fullsized":"2024-04-02T13:38:46.878277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:23:19.627565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979838+0000","last_change":"2024-04-02T13:36:25.506695+0000","last_active":"2024-04-02T13:38:46.979838+0000","last_peered":"2024-04-02T13:38:46.979838+0000","last_clean":"2024-04-02T13:38:46.979838+0000","last_became_active":"2024-04-02T13:36:25.506254+0000","last_became_peered":"2024-04-02T13:36:25.506254+0000","last_unstale":"2024-04-02T13:38:46.979838+0000","last_undegraded":"2024-04-02T13:38:46.979838+0000","last_fullsized":"2024-04-02T13:38:46.979838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:57:00.315005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063037+0000","last_change":"2024-04-02T13:36:20.825827+0000","last_active":"2024-04-02T13:38:47.063037+0000","last_peered":"2024-04-02T13:38:47.063037+0000","last_clean":"2024-04-02T13:38:47.063037+0000","last_became_active":"2024-04-02T13:36:20.825530+0000","last_became_peered":"2024-04-02T13:36:20.825530+0000","last_unstale":"2024-04-02T13:38:47.063037+0000","last_undegraded":"2024-04-02T13:38:47.063037+0000","last_fullsized":"2024-04-02T13:38:47.063037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:27:02.962513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063146+0000","last_change":"2024-04-02T13:36:21.486343+0000","last_active":"2024-04-02T13:38:47.063146+0000","last_peered":"2024-04-02T13:38:47.063146+0000","last_clean":"2024-04-02T13:38:47.063146+0000","last_became_active":"2024-04-02T13:36:21.486021+0000","last_became_peered":"2024-04-02T13:36:21.486021+0000","last_unstale":"2024-04-02T13:38:47.063146+0000","last_undegraded":"2024-04-02T13:38:47.063146+0000","last_fullsized":"2024-04-02T13:38:47.063146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:53:42.430173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062955+0000","last_change":"2024-04-02T13:37:17.631430+0000","last_active":"2024-04-02T13:38:47.062955+0000","last_peered":"2024-04-02T13:38:47.062955+0000","last_clean":"2024-04-02T13:38:47.062955+0000","last_became_active":"2024-04-02T13:37:17.630930+0000","last_became_peered":"2024-04-02T13:37:17.630930+0000","last_unstale":"2024-04-02T13:38:47.062955+0000","last_undegraded":"2024-04-02T13:38:47.062955+0000","last_fullsized":"2024-04-02T13:38:47.062955+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:44:10.514663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979556+0000","last_change":"2024-04-02T13:36:20.829303+0000","last_active":"2024-04-02T13:38:46.979556+0000","last_peered":"2024-04-02T13:38:46.979556+0000","last_clean":"2024-04-02T13:38:46.979556+0000","last_became_active":"2024-04-02T13:36:20.828952+0000","last_became_peered":"2024-04-02T13:36:20.828952+0000","last_unstale":"2024-04-02T13:38:46.979556+0000","last_undegraded":"2024-04-02T13:38:46.979556+0000","last_fullsized":"2024-04-02T13:38:46.979556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:14:04.385178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063176+0000","last_change":"2024-04-02T13:36:21.491767+0000","last_active":"2024-04-02T13:38:47.063176+0000","last_peered":"2024-04-02T13:38:47.063176+0000","last_clean":"2024-04-02T13:38:47.063176+0000","last_became_active":"2024-04-02T13:36:21.491369+0000","last_became_peered":"2024-04-02T13:36:21.491369+0000","last_unstale":"2024-04-02T13:38:47.063176+0000","last_undegraded":"2024-04-02T13:38:47.063176+0000","last_fullsized":"2024-04-02T13:38:47.063176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:14:35.717498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970825+0000","last_change":"2024-04-02T13:36:25.503900+0000","last_active":"2024-04-02T13:38:46.970825+0000","last_peered":"2024-04-02T13:38:46.970825+0000","last_clean":"2024-04-02T13:38:46.970825+0000","last_became_active":"2024-04-02T13:36:25.502840+0000","last_became_peered":"2024-04-02T13:36:25.502840+0000","last_unstale":"2024-04-02T13:38:46.970825+0000","last_undegraded":"2024-04-02T13:38:46.970825+0000","last_fullsized":"2024-04-02T13:38:46.970825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:31.106865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.712522+0000","last_change":"2024-04-02T13:36:20.823753+0000","last_active":"2024-04-02T13:38:47.712522+0000","last_peered":"2024-04-02T13:38:47.712522+0000","last_clean":"2024-04-02T13:38:47.712522+0000","last_became_active":"2024-04-02T13:36:20.823394+0000","last_became_peered":"2024-04-02T13:36:20.823394+0000","last_unstale":"2024-04-02T13:38:47.712522+0000","last_undegraded":"2024-04-02T13:38:47.712522+0000","last_fullsized":"2024-04-02T13:38:47.712522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:13.023125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2395,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063709+0000","last_change":"2024-04-02T13:36:21.499558+0000","last_active":"2024-04-02T13:38:47.063709+0000","last_peered":"2024-04-02T13:38:47.063709+0000","last_clean":"2024-04-02T13:38:47.063709+0000","last_became_active":"2024-04-02T13:36:21.499197+0000","last_became_peered":"2024-04-02T13:36:21.499197+0000","last_unstale":"2024-04-02T13:38:47.063709+0000","last_undegraded":"2024-04-02T13:38:47.063709+0000","last_fullsized":"2024-04-02T13:38:47.063709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:26.435832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063525+0000","last_change":"2024-04-02T13:36:25.504643+0000","last_active":"2024-04-02T13:38:47.063525+0000","last_peered":"2024-04-02T13:38:47.063525+0000","last_clean":"2024-04-02T13:38:47.063525+0000","last_became_active":"2024-04-02T13:36:25.504249+0000","last_became_peered":"2024-04-02T13:36:25.504249+0000","last_unstale":"2024-04-02T13:38:47.063525+0000","last_undegraded":"2024-04-02T13:38:47.063525+0000","last_fullsized":"2024-04-02T13:38:47.063525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:33:15.818958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877236+0000","last_change":"2024-04-02T13:36:20.830097+0000","last_active":"2024-04-02T13:38:46.877236+0000","last_peered":"2024-04-02T13:38:46.877236+0000","last_clean":"2024-04-02T13:38:46.877236+0000","last_became_active":"2024-04-02T13:36:20.829779+0000","last_became_peered":"2024-04-02T13:36:20.829779+0000","last_unstale":"2024-04-02T13:38:46.877236+0000","last_undegraded":"2024-04-02T13:38:46.877236+0000","last_fullsized":"2024-04-02T13:38:46.877236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:59.477730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979210+0000","last_change":"2024-04-02T13:36:21.497480+0000","last_active":"2024-04-02T13:38:46.979210+0000","last_peered":"2024-04-02T13:38:46.979210+0000","last_clean":"2024-04-02T13:38:46.979210+0000","last_became_active":"2024-04-02T13:36:21.497106+0000","last_became_peered":"2024-04-02T13:36:21.497106+0000","last_unstale":"2024-04-02T13:38:46.979210+0000","last_undegraded":"2024-04-02T13:38:46.979210+0000","last_fullsized":"2024-04-02T13:38:46.979210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:30.394968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979152+0000","last_change":"2024-04-02T13:36:25.500040+0000","last_active":"2024-04-02T13:38:46.979152+0000","last_peered":"2024-04-02T13:38:46.979152+0000","last_clean":"2024-04-02T13:38:46.979152+0000","last_became_active":"2024-04-02T13:36:25.499576+0000","last_became_peered":"2024-04-02T13:36:25.499576+0000","last_unstale":"2024-04-02T13:38:46.979152+0000","last_undegraded":"2024-04-02T13:38:46.979152+0000","last_fullsized":"2024-04-02T13:38:46.979152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:15:47.475234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878479+0000","last_change":"2024-04-02T13:36:20.823225+0000","last_active":"2024-04-02T13:38:46.878479+0000","last_peered":"2024-04-02T13:38:46.878479+0000","last_clean":"2024-04-02T13:38:46.878479+0000","last_became_active":"2024-04-02T13:36:20.822865+0000","last_became_peered":"2024-04-02T13:36:20.822865+0000","last_unstale":"2024-04-02T13:38:46.878479+0000","last_undegraded":"2024-04-02T13:38:46.878479+0000","last_fullsized":"2024-04-02T13:38:46.878479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:00.488190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063314+0000","last_change":"2024-04-02T13:36:21.497692+0000","last_active":"2024-04-02T13:38:47.063314+0000","last_peered":"2024-04-02T13:38:47.063314+0000","last_clean":"2024-04-02T13:38:47.063314+0000","last_became_active":"2024-04-02T13:36:21.497322+0000","last_became_peered":"2024-04-02T13:36:21.497322+0000","last_unstale":"2024-04-02T13:38:47.063314+0000","last_undegraded":"2024-04-02T13:38:47.063314+0000","last_fullsized":"2024-04-02T13:38:47.063314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:45:17.590235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877317+0000","last_change":"2024-04-02T13:37:17.872416+0000","last_active":"2024-04-02T13:38:46.877317+0000","last_peered":"2024-04-02T13:38:46.877317+0000","last_clean":"2024-04-02T13:38:46.877317+0000","last_became_active":"2024-04-02T13:37:17.871932+0000","last_became_peered":"2024-04-02T13:37:17.871932+0000","last_unstale":"2024-04-02T13:38:46.877317+0000","last_undegraded":"2024-04-02T13:38:46.877317+0000","last_fullsized":"2024-04-02T13:38:46.877317+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:57:23.575737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970543+0000","last_change":"2024-04-02T13:36:20.835277+0000","last_active":"2024-04-02T13:38:46.970543+0000","last_peered":"2024-04-02T13:38:46.970543+0000","last_clean":"2024-04-02T13:38:46.970543+0000","last_became_active":"2024-04-02T13:36:20.834911+0000","last_became_peered":"2024-04-02T13:36:20.834911+0000","last_unstale":"2024-04-02T13:38:46.970543+0000","last_undegraded":"2024-04-02T13:38:46.970543+0000","last_fullsized":"2024-04-02T13:38:46.970543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:54:20.965544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982626+0000","last_change":"2024-04-02T13:36:21.495274+0000","last_active":"2024-04-02T13:38:46.982626+0000","last_peered":"2024-04-02T13:38:46.982626+0000","last_clean":"2024-04-02T13:38:46.982626+0000","last_became_active":"2024-04-02T13:36:21.494841+0000","last_became_peered":"2024-04-02T13:36:21.494841+0000","last_unstale":"2024-04-02T13:38:46.982626+0000","last_undegraded":"2024-04-02T13:38:46.982626+0000","last_fullsized":"2024-04-02T13:38:46.982626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:06:20.096507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063043+0000","last_change":"2024-04-02T13:36:25.499970+0000","last_active":"2024-04-02T13:38:47.063043+0000","last_peered":"2024-04-02T13:38:47.063043+0000","last_clean":"2024-04-02T13:38:47.063043+0000","last_became_active":"2024-04-02T13:36:25.499539+0000","last_became_peered":"2024-04-02T13:36:25.499539+0000","last_unstale":"2024-04-02T13:38:47.063043+0000","last_undegraded":"2024-04-02T13:38:47.063043+0000","last_fullsized":"2024-04-02T13:38:47.063043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:43.136635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063059+0000","last_change":"2024-04-02T13:36:20.831150+0000","last_active":"2024-04-02T13:38:47.063059+0000","last_peered":"2024-04-02T13:38:47.063059+0000","last_clean":"2024-04-02T13:38:47.063059+0000","last_became_active":"2024-04-02T13:36:20.830838+0000","last_became_peered":"2024-04-02T13:36:20.830838+0000","last_unstale":"2024-04-02T13:38:47.063059+0000","last_undegraded":"2024-04-02T13:38:47.063059+0000","last_fullsized":"2024-04-02T13:38:47.063059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:23:09.566911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.982556+0000","last_change":"2024-04-02T13:36:21.503128+0000","last_active":"2024-04-02T13:38:46.982556+0000","last_peered":"2024-04-02T13:38:46.982556+0000","last_clean":"2024-04-02T13:38:46.982556+0000","last_became_active":"2024-04-02T13:36:21.502695+0000","last_became_peered":"2024-04-02T13:36:21.502695+0000","last_unstale":"2024-04-02T13:38:46.982556+0000","last_undegraded":"2024-04-02T13:38:46.982556+0000","last_fullsized":"2024-04-02T13:38:46.982556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:38.261472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063558+0000","last_change":"2024-04-02T13:36:25.530911+0000","last_active":"2024-04-02T13:38:47.063558+0000","last_peered":"2024-04-02T13:38:47.063558+0000","last_clean":"2024-04-02T13:38:47.063558+0000","last_became_active":"2024-04-02T13:36:25.530428+0000","last_became_peered":"2024-04-02T13:36:25.530428+0000","last_unstale":"2024-04-02T13:38:47.063558+0000","last_undegraded":"2024-04-02T13:38:47.063558+0000","last_fullsized":"2024-04-02T13:38:47.063558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:49.185987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979152+0000","last_change":"2024-04-02T13:36:21.492251+0000","last_active":"2024-04-02T13:38:46.979152+0000","last_peered":"2024-04-02T13:38:46.979152+0000","last_clean":"2024-04-02T13:38:46.979152+0000","last_became_active":"2024-04-02T13:36:21.491929+0000","last_became_peered":"2024-04-02T13:36:21.491929+0000","last_unstale":"2024-04-02T13:38:46.979152+0000","last_undegraded":"2024-04-02T13:38:46.979152+0000","last_fullsized":"2024-04-02T13:38:46.979152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:59.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":160,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062946+0000","last_change":"2024-04-02T13:36:20.821431+0000","last_active":"2024-04-02T13:38:47.062946+0000","last_peered":"2024-04-02T13:38:47.062946+0000","last_clean":"2024-04-02T13:38:47.062946+0000","last_became_active":"2024-04-02T13:36:20.821067+0000","last_became_peered":"2024-04-02T13:36:20.821067+0000","last_unstale":"2024-04-02T13:38:47.062946+0000","last_undegraded":"2024-04-02T13:38:47.062946+0000","last_fullsized":"2024-04-02T13:38:47.062946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:52.043907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063151+0000","last_change":"2024-04-02T13:36:25.496992+0000","last_active":"2024-04-02T13:38:47.063151+0000","last_peered":"2024-04-02T13:38:47.063151+0000","last_clean":"2024-04-02T13:38:47.063151+0000","last_became_active":"2024-04-02T13:36:25.496467+0000","last_became_peered":"2024-04-02T13:36:25.496467+0000","last_unstale":"2024-04-02T13:38:47.063151+0000","last_undegraded":"2024-04-02T13:38:47.063151+0000","last_fullsized":"2024-04-02T13:38:47.063151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:13:00.747716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872529+0000","last_change":"2024-04-02T13:36:21.491399+0000","last_active":"2024-04-02T13:38:46.872529+0000","last_peered":"2024-04-02T13:38:46.872529+0000","last_clean":"2024-04-02T13:38:46.872529+0000","last_became_active":"2024-04-02T13:36:21.491038+0000","last_became_peered":"2024-04-02T13:36:21.491038+0000","last_unstale":"2024-04-02T13:38:46.872529+0000","last_undegraded":"2024-04-02T13:38:46.872529+0000","last_fullsized":"2024-04-02T13:38:46.872529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:25.603086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874057+0000","last_change":"2024-04-02T13:36:20.822796+0000","last_active":"2024-04-02T13:38:46.874057+0000","last_peered":"2024-04-02T13:38:46.874057+0000","last_clean":"2024-04-02T13:38:46.874057+0000","last_became_active":"2024-04-02T13:36:20.822168+0000","last_became_peered":"2024-04-02T13:36:20.822168+0000","last_unstale":"2024-04-02T13:38:46.874057+0000","last_undegraded":"2024-04-02T13:38:46.874057+0000","last_fullsized":"2024-04-02T13:38:46.874057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:11.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062738+0000","last_change":"2024-04-02T13:35:08.678967+0000","last_active":"2024-04-02T13:38:47.062738+0000","last_peered":"2024-04-02T13:38:47.062738+0000","last_clean":"2024-04-02T13:38:47.062738+0000","last_became_active":"2024-04-02T13:35:08.678408+0000","last_became_peered":"2024-04-02T13:35:08.678408+0000","last_unstale":"2024-04-02T13:38:47.062738+0000","last_undegraded":"2024-04-02T13:38:47.062738+0000","last_fullsized":"2024-04-02T13:38:47.062738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:31:42.453938+0000","last_clean_scrub_stamp":"2024-04-02T13:31:42.453938+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:30:20.020809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979462+0000","last_change":"2024-04-02T13:36:25.532723+0000","last_active":"2024-04-02T13:38:46.979462+0000","last_peered":"2024-04-02T13:38:46.979462+0000","last_clean":"2024-04-02T13:38:46.979462+0000","last_became_active":"2024-04-02T13:36:25.532341+0000","last_became_peered":"2024-04-02T13:36:25.532341+0000","last_unstale":"2024-04-02T13:38:46.979462+0000","last_undegraded":"2024-04-02T13:38:46.979462+0000","last_fullsized":"2024-04-02T13:38:46.979462+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:53:43.190339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062721+0000","last_change":"2024-04-02T13:36:21.488983+0000","last_active":"2024-04-02T13:38:47.062721+0000","last_peered":"2024-04-02T13:38:47.062721+0000","last_clean":"2024-04-02T13:38:47.062721+0000","last_became_active":"2024-04-02T13:36:21.488580+0000","last_became_peered":"2024-04-02T13:36:21.488580+0000","last_unstale":"2024-04-02T13:38:47.062721+0000","last_undegraded":"2024-04-02T13:38:47.062721+0000","last_fullsized":"2024-04-02T13:38:47.062721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:01:39.382233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062580+0000","last_change":"2024-04-02T13:37:17.633840+0000","last_active":"2024-04-02T13:38:47.062580+0000","last_peered":"2024-04-02T13:38:47.062580+0000","last_clean":"2024-04-02T13:38:47.062580+0000","last_became_active":"2024-04-02T13:37:17.626842+0000","last_became_peered":"2024-04-02T13:37:17.626842+0000","last_unstale":"2024-04-02T13:38:47.062580+0000","last_undegraded":"2024-04-02T13:38:47.062580+0000","last_fullsized":"2024-04-02T13:38:47.062580+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:55:17.437862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062781+0000","last_change":"2024-04-02T13:36:25.502814+0000","last_active":"2024-04-02T13:38:47.062781+0000","last_peered":"2024-04-02T13:38:47.062781+0000","last_clean":"2024-04-02T13:38:47.062781+0000","last_became_active":"2024-04-02T13:36:25.502288+0000","last_became_peered":"2024-04-02T13:36:25.502288+0000","last_unstale":"2024-04-02T13:38:47.062781+0000","last_undegraded":"2024-04-02T13:38:47.062781+0000","last_fullsized":"2024-04-02T13:38:47.062781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:48.073286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.878368+0000","last_change":"2024-04-02T13:36:20.830512+0000","last_active":"2024-04-02T13:38:46.878368+0000","last_peered":"2024-04-02T13:38:46.878368+0000","last_clean":"2024-04-02T13:38:46.878368+0000","last_became_active":"2024-04-02T13:36:20.830086+0000","last_became_peered":"2024-04-02T13:36:20.830086+0000","last_unstale":"2024-04-02T13:38:46.878368+0000","last_undegraded":"2024-04-02T13:38:46.878368+0000","last_fullsized":"2024-04-02T13:38:46.878368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:46:07.428363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063182+0000","last_change":"2024-04-02T13:36:21.478218+0000","last_active":"2024-04-02T13:38:47.063182+0000","last_peered":"2024-04-02T13:38:47.063182+0000","last_clean":"2024-04-02T13:38:47.063182+0000","last_became_active":"2024-04-02T13:36:21.477856+0000","last_became_peered":"2024-04-02T13:36:21.477856+0000","last_unstale":"2024-04-02T13:38:47.063182+0000","last_undegraded":"2024-04-02T13:38:47.063182+0000","last_fullsized":"2024-04-02T13:38:47.063182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:58:04.397606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872360+0000","last_change":"2024-04-02T13:36:25.530915+0000","last_active":"2024-04-02T13:38:46.872360+0000","last_peered":"2024-04-02T13:38:46.872360+0000","last_clean":"2024-04-02T13:38:46.872360+0000","last_became_active":"2024-04-02T13:36:25.530343+0000","last_became_peered":"2024-04-02T13:36:25.530343+0000","last_unstale":"2024-04-02T13:38:46.872360+0000","last_undegraded":"2024-04-02T13:38:46.872360+0000","last_fullsized":"2024-04-02T13:38:46.872360+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:32:41.181485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'45","reported_seq":102,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.095891+0000","last_change":"2024-04-02T13:36:20.828527+0000","last_active":"2024-04-02T13:38:47.095891+0000","last_peered":"2024-04-02T13:38:47.095891+0000","last_clean":"2024-04-02T13:38:47.095891+0000","last_became_active":"2024-04-02T13:36:20.828136+0000","last_became_peered":"2024-04-02T13:36:20.828136+0000","last_unstale":"2024-04-02T13:38:47.095891+0000","last_undegraded":"2024-04-02T13:38:47.095891+0000","last_fullsized":"2024-04-02T13:38:47.095891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:57.709400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872915+0000","last_change":"2024-04-02T13:36:21.492035+0000","last_active":"2024-04-02T13:38:46.872915+0000","last_peered":"2024-04-02T13:38:46.872915+0000","last_clean":"2024-04-02T13:38:46.872915+0000","last_became_active":"2024-04-02T13:36:21.491682+0000","last_became_peered":"2024-04-02T13:36:21.491682+0000","last_unstale":"2024-04-02T13:38:46.872915+0000","last_undegraded":"2024-04-02T13:38:46.872915+0000","last_fullsized":"2024-04-02T13:38:46.872915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:09:40.600616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062520+0000","last_change":"2024-04-02T13:36:25.493301+0000","last_active":"2024-04-02T13:38:47.062520+0000","last_peered":"2024-04-02T13:38:47.062520+0000","last_clean":"2024-04-02T13:38:47.062520+0000","last_became_active":"2024-04-02T13:36:25.492846+0000","last_became_peered":"2024-04-02T13:36:25.492846+0000","last_unstale":"2024-04-02T13:38:47.062520+0000","last_undegraded":"2024-04-02T13:38:47.062520+0000","last_fullsized":"2024-04-02T13:38:47.062520+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:12:53.350261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874377+0000","last_change":"2024-04-02T13:36:20.823559+0000","last_active":"2024-04-02T13:38:46.874377+0000","last_peered":"2024-04-02T13:38:46.874377+0000","last_clean":"2024-04-02T13:38:46.874377+0000","last_became_active":"2024-04-02T13:36:20.823178+0000","last_became_peered":"2024-04-02T13:36:20.823178+0000","last_unstale":"2024-04-02T13:38:46.874377+0000","last_undegraded":"2024-04-02T13:38:46.874377+0000","last_fullsized":"2024-04-02T13:38:46.874377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:42.628855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250679+0000","last_change":"2024-04-02T13:36:21.497258+0000","last_active":"2024-04-02T13:38:47.250679+0000","last_peered":"2024-04-02T13:38:47.250679+0000","last_clean":"2024-04-02T13:38:47.250679+0000","last_became_active":"2024-04-02T13:36:21.496943+0000","last_became_peered":"2024-04-02T13:36:21.496943+0000","last_unstale":"2024-04-02T13:38:47.250679+0000","last_undegraded":"2024-04-02T13:38:47.250679+0000","last_fullsized":"2024-04-02T13:38:47.250679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:51.989025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874385+0000","last_change":"2024-04-02T13:36:25.531251+0000","last_active":"2024-04-02T13:38:46.874385+0000","last_peered":"2024-04-02T13:38:46.874385+0000","last_clean":"2024-04-02T13:38:46.874385+0000","last_became_active":"2024-04-02T13:36:25.530505+0000","last_became_peered":"2024-04-02T13:36:25.530505+0000","last_unstale":"2024-04-02T13:38:46.874385+0000","last_undegraded":"2024-04-02T13:38:46.874385+0000","last_fullsized":"2024-04-02T13:38:46.874385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:02:10.711138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062408+0000","last_change":"2024-04-02T13:36:20.837865+0000","last_active":"2024-04-02T13:38:47.062408+0000","last_peered":"2024-04-02T13:38:47.062408+0000","last_clean":"2024-04-02T13:38:47.062408+0000","last_became_active":"2024-04-02T13:36:20.837567+0000","last_became_peered":"2024-04-02T13:36:20.837567+0000","last_unstale":"2024-04-02T13:38:47.062408+0000","last_undegraded":"2024-04-02T13:38:47.062408+0000","last_fullsized":"2024-04-02T13:38:47.062408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:11:59.365592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877191+0000","last_change":"2024-04-02T13:36:21.500085+0000","last_active":"2024-04-02T13:38:46.877191+0000","last_peered":"2024-04-02T13:38:46.877191+0000","last_clean":"2024-04-02T13:38:46.877191+0000","last_became_active":"2024-04-02T13:36:21.499706+0000","last_became_peered":"2024-04-02T13:36:21.499706+0000","last_unstale":"2024-04-02T13:38:46.877191+0000","last_undegraded":"2024-04-02T13:38:46.877191+0000","last_fullsized":"2024-04-02T13:38:46.877191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.378226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.970930+0000","last_change":"2024-04-02T13:36:25.505797+0000","last_active":"2024-04-02T13:38:46.970930+0000","last_peered":"2024-04-02T13:38:46.970930+0000","last_clean":"2024-04-02T13:38:46.970930+0000","last_became_active":"2024-04-02T13:36:25.505409+0000","last_became_peered":"2024-04-02T13:36:25.505409+0000","last_unstale":"2024-04-02T13:38:46.970930+0000","last_undegraded":"2024-04-02T13:38:46.970930+0000","last_fullsized":"2024-04-02T13:38:46.970930+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:46:31.723248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872526+0000","last_change":"2024-04-02T13:36:20.821572+0000","last_active":"2024-04-02T13:38:46.872526+0000","last_peered":"2024-04-02T13:38:46.872526+0000","last_clean":"2024-04-02T13:38:46.872526+0000","last_became_active":"2024-04-02T13:36:20.821235+0000","last_became_peered":"2024-04-02T13:36:20.821235+0000","last_unstale":"2024-04-02T13:38:46.872526+0000","last_undegraded":"2024-04-02T13:38:46.872526+0000","last_fullsized":"2024-04-02T13:38:46.872526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:17:02.304145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062784+0000","last_change":"2024-04-02T13:36:21.485255+0000","last_active":"2024-04-02T13:38:47.062784+0000","last_peered":"2024-04-02T13:38:47.062784+0000","last_clean":"2024-04-02T13:38:47.062784+0000","last_became_active":"2024-04-02T13:36:21.484870+0000","last_became_peered":"2024-04-02T13:36:21.484870+0000","last_unstale":"2024-04-02T13:38:47.062784+0000","last_undegraded":"2024-04-02T13:38:47.062784+0000","last_fullsized":"2024-04-02T13:38:47.062784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:03.851161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874276+0000","last_change":"2024-04-02T13:36:25.494161+0000","last_active":"2024-04-02T13:38:46.874276+0000","last_peered":"2024-04-02T13:38:46.874276+0000","last_clean":"2024-04-02T13:38:46.874276+0000","last_became_active":"2024-04-02T13:36:25.493667+0000","last_became_peered":"2024-04-02T13:36:25.493667+0000","last_unstale":"2024-04-02T13:38:46.874276+0000","last_undegraded":"2024-04-02T13:38:46.874276+0000","last_fullsized":"2024-04-02T13:38:46.874276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:35:45.703988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062701+0000","last_change":"2024-04-02T13:37:17.636209+0000","last_active":"2024-04-02T13:38:47.062701+0000","last_peered":"2024-04-02T13:38:47.062701+0000","last_clean":"2024-04-02T13:38:47.062701+0000","last_became_active":"2024-04-02T13:37:17.631539+0000","last_became_peered":"2024-04-02T13:37:17.631539+0000","last_unstale":"2024-04-02T13:38:47.062701+0000","last_undegraded":"2024-04-02T13:38:47.062701+0000","last_fullsized":"2024-04-02T13:38:47.062701+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:05:52.619779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872963+0000","last_change":"2024-04-02T13:36:21.498640+0000","last_active":"2024-04-02T13:38:46.872963+0000","last_peered":"2024-04-02T13:38:46.872963+0000","last_clean":"2024-04-02T13:38:46.872963+0000","last_became_active":"2024-04-02T13:36:21.497996+0000","last_became_peered":"2024-04-02T13:36:21.497996+0000","last_unstale":"2024-04-02T13:38:46.872963+0000","last_undegraded":"2024-04-02T13:38:46.872963+0000","last_fullsized":"2024-04-02T13:38:46.872963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:56:45.748945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.874223+0000","last_change":"2024-04-02T13:36:25.531119+0000","last_active":"2024-04-02T13:38:46.874223+0000","last_peered":"2024-04-02T13:38:46.874223+0000","last_clean":"2024-04-02T13:38:46.874223+0000","last_became_active":"2024-04-02T13:36:25.530307+0000","last_became_peered":"2024-04-02T13:36:25.530307+0000","last_unstale":"2024-04-02T13:38:46.874223+0000","last_undegraded":"2024-04-02T13:38:46.874223+0000","last_fullsized":"2024-04-02T13:38:46.874223+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:58:50.543386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877426+0000","last_change":"2024-04-02T13:36:20.829843+0000","last_active":"2024-04-02T13:38:46.877426+0000","last_peered":"2024-04-02T13:38:46.877426+0000","last_clean":"2024-04-02T13:38:46.877426+0000","last_became_active":"2024-04-02T13:36:20.829502+0000","last_became_peered":"2024-04-02T13:36:20.829502+0000","last_unstale":"2024-04-02T13:38:46.877426+0000","last_undegraded":"2024-04-02T13:38:46.877426+0000","last_fullsized":"2024-04-02T13:38:46.877426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:36:45.016174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063189+0000","last_change":"2024-04-02T13:36:21.491579+0000","last_active":"2024-04-02T13:38:47.063189+0000","last_peered":"2024-04-02T13:38:47.063189+0000","last_clean":"2024-04-02T13:38:47.063189+0000","last_became_active":"2024-04-02T13:36:21.490992+0000","last_became_peered":"2024-04-02T13:36:21.490992+0000","last_unstale":"2024-04-02T13:38:47.063189+0000","last_undegraded":"2024-04-02T13:38:47.063189+0000","last_fullsized":"2024-04-02T13:38:47.063189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:27:41.899766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062673+0000","last_change":"2024-04-02T13:36:25.502658+0000","last_active":"2024-04-02T13:38:47.062673+0000","last_peered":"2024-04-02T13:38:47.062673+0000","last_clean":"2024-04-02T13:38:47.062673+0000","last_became_active":"2024-04-02T13:36:25.502219+0000","last_became_peered":"2024-04-02T13:36:25.502219+0000","last_unstale":"2024-04-02T13:38:47.062673+0000","last_undegraded":"2024-04-02T13:38:47.062673+0000","last_fullsized":"2024-04-02T13:38:47.062673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:23:34.189381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063322+0000","last_change":"2024-04-02T13:36:20.835176+0000","last_active":"2024-04-02T13:38:47.063322+0000","last_peered":"2024-04-02T13:38:47.063322+0000","last_clean":"2024-04-02T13:38:47.063322+0000","last_became_active":"2024-04-02T13:36:20.834871+0000","last_became_peered":"2024-04-02T13:36:20.834871+0000","last_unstale":"2024-04-02T13:38:47.063322+0000","last_undegraded":"2024-04-02T13:38:47.063322+0000","last_fullsized":"2024-04-02T13:38:47.063322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:17:14.148762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063560+0000","last_change":"2024-04-02T13:36:21.490336+0000","last_active":"2024-04-02T13:38:47.063560+0000","last_peered":"2024-04-02T13:38:47.063560+0000","last_clean":"2024-04-02T13:38:47.063560+0000","last_became_active":"2024-04-02T13:36:21.489869+0000","last_became_peered":"2024-04-02T13:36:21.489869+0000","last_unstale":"2024-04-02T13:38:47.063560+0000","last_undegraded":"2024-04-02T13:38:47.063560+0000","last_fullsized":"2024-04-02T13:38:47.063560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:42.243209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250271+0000","last_change":"2024-04-02T13:36:25.503124+0000","last_active":"2024-04-02T13:38:47.250271+0000","last_peered":"2024-04-02T13:38:47.250271+0000","last_clean":"2024-04-02T13:38:47.250271+0000","last_became_active":"2024-04-02T13:36:25.502737+0000","last_became_peered":"2024-04-02T13:36:25.502737+0000","last_unstale":"2024-04-02T13:38:47.250271+0000","last_undegraded":"2024-04-02T13:38:47.250271+0000","last_fullsized":"2024-04-02T13:38:47.250271+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:02:56.830746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'39","reported_seq":80,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.094476+0000","last_change":"2024-04-02T13:36:20.828597+0000","last_active":"2024-04-02T13:38:47.094476+0000","last_peered":"2024-04-02T13:38:47.094476+0000","last_clean":"2024-04-02T13:38:47.094476+0000","last_became_active":"2024-04-02T13:36:20.828138+0000","last_became_peered":"2024-04-02T13:36:20.828138+0000","last_unstale":"2024-04-02T13:38:47.094476+0000","last_undegraded":"2024-04-02T13:38:47.094476+0000","last_fullsized":"2024-04-02T13:38:47.094476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:05.171448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063451+0000","last_change":"2024-04-02T13:36:21.498498+0000","last_active":"2024-04-02T13:38:47.063451+0000","last_peered":"2024-04-02T13:38:47.063451+0000","last_clean":"2024-04-02T13:38:47.063451+0000","last_became_active":"2024-04-02T13:36:21.498119+0000","last_became_peered":"2024-04-02T13:36:21.498119+0000","last_unstale":"2024-04-02T13:38:47.063451+0000","last_undegraded":"2024-04-02T13:38:47.063451+0000","last_fullsized":"2024-04-02T13:38:47.063451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:01.528228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.877165+0000","last_change":"2024-04-02T13:37:17.872082+0000","last_active":"2024-04-02T13:38:46.877165+0000","last_peered":"2024-04-02T13:38:46.877165+0000","last_clean":"2024-04-02T13:38:46.877165+0000","last_became_active":"2024-04-02T13:37:17.871562+0000","last_became_peered":"2024-04-02T13:37:17.871562+0000","last_unstale":"2024-04-02T13:38:46.877165+0000","last_undegraded":"2024-04-02T13:38:46.877165+0000","last_fullsized":"2024-04-02T13:38:46.877165+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:24.473921+0000","last_clean_scrub_stamp":"2024-04-02T13:36:24.473921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:32:39.849512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063400+0000","last_change":"2024-04-02T13:37:17.848534+0000","last_active":"2024-04-02T13:38:47.063400+0000","last_peered":"2024-04-02T13:38:47.063400+0000","last_clean":"2024-04-02T13:38:47.063400+0000","last_became_active":"2024-04-02T13:37:17.844875+0000","last_became_peered":"2024-04-02T13:37:17.844875+0000","last_unstale":"2024-04-02T13:38:47.063400+0000","last_undegraded":"2024-04-02T13:38:47.063400+0000","last_fullsized":"2024-04-02T13:38:47.063400+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:19.804556+0000","last_clean_scrub_stamp":"2024-04-02T13:36:19.804556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:29:05.037405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063557+0000","last_change":"2024-04-02T13:36:21.494069+0000","last_active":"2024-04-02T13:38:47.063557+0000","last_peered":"2024-04-02T13:38:47.063557+0000","last_clean":"2024-04-02T13:38:47.063557+0000","last_became_active":"2024-04-02T13:36:21.493751+0000","last_became_peered":"2024-04-02T13:36:21.493751+0000","last_unstale":"2024-04-02T13:38:47.063557+0000","last_undegraded":"2024-04-02T13:38:47.063557+0000","last_fullsized":"2024-04-02T13:38:47.063557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:24:39.648318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062944+0000","last_change":"2024-04-02T13:36:21.494280+0000","last_active":"2024-04-02T13:38:47.062944+0000","last_peered":"2024-04-02T13:38:47.062944+0000","last_clean":"2024-04-02T13:38:47.062944+0000","last_became_active":"2024-04-02T13:36:21.491262+0000","last_became_peered":"2024-04-02T13:36:21.491262+0000","last_unstale":"2024-04-02T13:38:47.062944+0000","last_undegraded":"2024-04-02T13:38:47.062944+0000","last_fullsized":"2024-04-02T13:38:47.062944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:08:16.728646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062753+0000","last_change":"2024-04-02T13:36:21.497103+0000","last_active":"2024-04-02T13:38:47.062753+0000","last_peered":"2024-04-02T13:38:47.062753+0000","last_clean":"2024-04-02T13:38:47.062753+0000","last_became_active":"2024-04-02T13:36:21.496784+0000","last_became_peered":"2024-04-02T13:36:21.496784+0000","last_unstale":"2024-04-02T13:38:47.062753+0000","last_undegraded":"2024-04-02T13:38:47.062753+0000","last_fullsized":"2024-04-02T13:38:47.062753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:38:21.903362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062601+0000","last_change":"2024-04-02T13:36:21.491853+0000","last_active":"2024-04-02T13:38:47.062601+0000","last_peered":"2024-04-02T13:38:47.062601+0000","last_clean":"2024-04-02T13:38:47.062601+0000","last_became_active":"2024-04-02T13:36:21.487986+0000","last_became_peered":"2024-04-02T13:36:21.487986+0000","last_unstale":"2024-04-02T13:38:47.062601+0000","last_undegraded":"2024-04-02T13:38:47.062601+0000","last_fullsized":"2024-04-02T13:38:47.062601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:37:47.098681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.250504+0000","last_change":"2024-04-02T13:37:17.631648+0000","last_active":"2024-04-02T13:38:47.250504+0000","last_peered":"2024-04-02T13:38:47.250504+0000","last_clean":"2024-04-02T13:38:47.250504+0000","last_became_active":"2024-04-02T13:37:17.631133+0000","last_became_peered":"2024-04-02T13:37:17.631133+0000","last_unstale":"2024-04-02T13:38:47.250504+0000","last_undegraded":"2024-04-02T13:38:47.250504+0000","last_fullsized":"2024-04-02T13:38:47.250504+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:10.791151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063190+0000","last_change":"2024-04-02T13:36:21.488723+0000","last_active":"2024-04-02T13:38:47.063190+0000","last_peered":"2024-04-02T13:38:47.063190+0000","last_clean":"2024-04-02T13:38:47.063190+0000","last_became_active":"2024-04-02T13:36:21.488391+0000","last_became_peered":"2024-04-02T13:36:21.488391+0000","last_unstale":"2024-04-02T13:38:47.063190+0000","last_undegraded":"2024-04-02T13:38:47.063190+0000","last_fullsized":"2024-04-02T13:38:47.063190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:23.768286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063129+0000","last_change":"2024-04-02T13:36:21.492676+0000","last_active":"2024-04-02T13:38:47.063129+0000","last_peered":"2024-04-02T13:38:47.063129+0000","last_clean":"2024-04-02T13:38:47.063129+0000","last_became_active":"2024-04-02T13:36:21.491663+0000","last_became_peered":"2024-04-02T13:36:21.491663+0000","last_unstale":"2024-04-02T13:38:47.063129+0000","last_undegraded":"2024-04-02T13:38:47.063129+0000","last_fullsized":"2024-04-02T13:38:47.063129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:00.131388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063384+0000","last_change":"2024-04-02T13:36:21.493380+0000","last_active":"2024-04-02T13:38:47.063384+0000","last_peered":"2024-04-02T13:38:47.063384+0000","last_clean":"2024-04-02T13:38:47.063384+0000","last_became_active":"2024-04-02T13:36:21.493073+0000","last_became_peered":"2024-04-02T13:36:21.493073+0000","last_unstale":"2024-04-02T13:38:47.063384+0000","last_undegraded":"2024-04-02T13:38:47.063384+0000","last_fullsized":"2024-04-02T13:38:47.063384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:59.412056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.873114+0000","last_change":"2024-04-02T13:36:21.500881+0000","last_active":"2024-04-02T13:38:46.873114+0000","last_peered":"2024-04-02T13:38:46.873114+0000","last_clean":"2024-04-02T13:38:46.873114+0000","last_became_active":"2024-04-02T13:36:21.500516+0000","last_became_peered":"2024-04-02T13:36:21.500516+0000","last_unstale":"2024-04-02T13:38:46.873114+0000","last_undegraded":"2024-04-02T13:38:46.873114+0000","last_fullsized":"2024-04-02T13:38:46.873114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:41:33.842494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.979070+0000","last_change":"2024-04-02T13:36:21.494558+0000","last_active":"2024-04-02T13:38:46.979070+0000","last_peered":"2024-04-02T13:38:46.979070+0000","last_clean":"2024-04-02T13:38:46.979070+0000","last_became_active":"2024-04-02T13:36:21.494180+0000","last_became_peered":"2024-04-02T13:36:21.494180+0000","last_unstale":"2024-04-02T13:38:46.979070+0000","last_undegraded":"2024-04-02T13:38:46.979070+0000","last_fullsized":"2024-04-02T13:38:46.979070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:07:54.363504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:46.872702+0000","last_change":"2024-04-02T13:36:21.500406+0000","last_active":"2024-04-02T13:38:46.872702+0000","last_peered":"2024-04-02T13:38:46.872702+0000","last_clean":"2024-04-02T13:38:46.872702+0000","last_became_active":"2024-04-02T13:36:21.499927+0000","last_became_peered":"2024-04-02T13:36:21.499927+0000","last_unstale":"2024-04-02T13:38:46.872702+0000","last_undegraded":"2024-04-02T13:38:46.872702+0000","last_fullsized":"2024-04-02T13:38:46.872702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:47:23.702776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.062852+0000","last_change":"2024-04-02T13:36:21.486166+0000","last_active":"2024-04-02T13:38:47.062852+0000","last_peered":"2024-04-02T13:38:47.062852+0000","last_clean":"2024-04-02T13:38:47.062852+0000","last_became_active":"2024-04-02T13:36:21.485282+0000","last_became_peered":"2024-04-02T13:36:21.485282+0000","last_unstale":"2024-04-02T13:38:47.062852+0000","last_undegraded":"2024-04-02T13:38:47.062852+0000","last_fullsized":"2024-04-02T13:38:47.062852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:24:07.007675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063046+0000","last_change":"2024-04-02T13:36:21.484909+0000","last_active":"2024-04-02T13:38:47.063046+0000","last_peered":"2024-04-02T13:38:47.063046+0000","last_clean":"2024-04-02T13:38:47.063046+0000","last_became_active":"2024-04-02T13:36:21.484559+0000","last_became_peered":"2024-04-02T13:36:21.484559+0000","last_unstale":"2024-04-02T13:38:47.063046+0000","last_undegraded":"2024-04-02T13:38:47.063046+0000","last_fullsized":"2024-04-02T13:38:47.063046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:59:45.848407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:38:47.063088+0000","last_change":"2024-04-02T13:36:21.493961+0000","last_active":"2024-04-02T13:38:47.063088+0000","last_peered":"2024-04-02T13:38:47.063088+0000","last_clean":"2024-04-02T13:38:47.063088+0000","last_became_active":"2024-04-02T13:36:21.493490+0000","last_became_peered":"2024-04-02T13:36:21.493490+0000","last_unstale":"2024-04-02T13:38:47.063088+0000","last_undegraded":"2024-04-02T13:38:47.063088+0000","last_fullsized":"2024-04-02T13:38:47.063088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:36:20.806032+0000","last_clean_scrub_stamp":"2024-04-02T13:36:20.806032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:26:42.032550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108576,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600843,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":859,"num_read_kb":543,"num_write":299,"num_write_kb":1450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1802529,"data_compressed":80724,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":532498,"data_compressed":29620,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36516,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708636,"statfs":{"total":95995035648,"available":95957643264,"internally_reserved":0,"allocated":1208320,"data_stored":1322367,"data_compressed":73574,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1085440,"data_stored":930046,"data_compressed":48766,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":925288,"data_compressed":48766,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":1131395,"data_compressed":65812,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":527910,"data_compressed":29620,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1110016,"data_stored":597596,"data_compressed":31226,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":519432,"data_compressed":26636,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1130496,"data_stored":599780,"data_compressed":31226,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":1052920,"data_compressed":54922,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888141,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":534367,"data_compressed":29620,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1256776,"data_compressed":71968,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36192,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69776,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71960,"data_compressed":1606,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134492,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393372,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":204179,"data_compressed":7762,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:38:52.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:38:52.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.934+0000 7fe52a69a640 1 -- 172.21.15.33:0/2722764743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5241059b0 msgr2=0x7fe524105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.934+0000 7fe52a69a640 1 --2- 172.21.15.33:0/2722764743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5241059b0 0x7fe524105d90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe514009900 tx=0x7fe51402bc80 comp rx=0 tx=0).stop 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.934+0000 7fe52a69a640 1 -- 172.21.15.33:0/2722764743 shutdown_connections 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.934+0000 7fe52a69a640 1 --2- 172.21.15.33:0/2722764743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe524101af0 0x7fe52410d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.934+0000 7fe52a69a640 1 --2- 172.21.15.33:0/2722764743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe524101150 0x7fe5241015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.934+0000 7fe52a69a640 1 --2- 172.21.15.33:0/2722764743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5241059b0 0x7fe524105d90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.934+0000 7fe52a69a640 1 -- 172.21.15.33:0/2722764743 >> 172.21.15.33:0/2722764743 conn(0x7fe524076f90 msgr2=0x7fe5240fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.934+0000 7fe52a69a640 1 -- 172.21.15.33:0/2722764743 shutdown_connections 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.934+0000 7fe52a69a640 1 -- 172.21.15.33:0/2722764743 wait complete. 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.934+0000 7fe52a69a640 1 Processor -- start 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe52a69a640 1 -- start start 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe52a69a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe524101150 0x7fe524197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe52a69a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe524101af0 0x7fe524197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe52a69a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5241059b0 0x7fe52419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe52a69a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe524110f40 con 0x7fe5241059b0 2024-04-02T13:38:52.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe52a69a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe524110dc0 con 0x7fe524101af0 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe52a69a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241110c0 con 0x7fe524101150 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe528c37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5241059b0 0x7fe52419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe528c37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5241059b0 0x7fe52419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59424/0 (socket says 172.21.15.33:59424) 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe528c37640 1 -- 172.21.15.33:0/3353751010 learned_addr learned my addr 172.21.15.33:0/3353751010 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe523fff640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe524101150 0x7fe524197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe5237fe640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe524101af0 0x7fe524197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe528c37640 1 -- 172.21.15.33:0/3353751010 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe524101150 msgr2=0x7fe524197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe528c37640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe524101150 0x7fe524197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe528c37640 1 -- 172.21.15.33:0/3353751010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe524101af0 msgr2=0x7fe524197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe528c37640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe524101af0 0x7fe524197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe528c37640 1 -- 172.21.15.33:0/3353751010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52419c3f0 con 0x7fe5241059b0 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.935+0000 7fe5237fe640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe524101af0 0x7fe524197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.936+0000 7fe528c37640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5241059b0 0x7fe52419bcc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00b970 tx=0x7fe51c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.936+0000 7fe5217fa640 1 -- 172.21.15.33:0/3353751010 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c019070 con 0x7fe5241059b0 2024-04-02T13:38:52.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.936+0000 7fe5217fa640 1 -- 172.21.15.33:0/3353751010 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe51c00cb40 con 0x7fe5241059b0 2024-04-02T13:38:52.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.936+0000 7fe52a69a640 1 -- 172.21.15.33:0/3353751010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52419c6e0 con 0x7fe5241059b0 2024-04-02T13:38:52.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.936+0000 7fe5217fa640 1 -- 172.21.15.33:0/3353751010 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c007650 con 0x7fe5241059b0 2024-04-02T13:38:52.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.936+0000 7fe52a69a640 1 -- 172.21.15.33:0/3353751010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5241a3f70 con 0x7fe5241059b0 2024-04-02T13:38:52.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.937+0000 7fe52a69a640 1 -- 172.21.15.33:0/3353751010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe524068d40 con 0x7fe5241059b0 2024-04-02T13:38:52.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.938+0000 7fe5217fa640 1 -- 172.21.15.33:0/3353751010 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe51c01c070 con 0x7fe5241059b0 2024-04-02T13:38:52.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.939+0000 7fe5217fa640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe50c0a3e40 0x7fe50c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:52.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.939+0000 7fe523fff640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe50c0a3e40 0x7fe50c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:52.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.939+0000 7fe5217fa640 1 -- 172.21.15.33:0/3353751010 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe51c0d0820 con 0x7fe5241059b0 2024-04-02T13:38:52.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.939+0000 7fe523fff640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe50c0a3e40 0x7fe50c0a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe514005ab0 tx=0x7fe514005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:52.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:52.942+0000 7fe5217fa640 1 -- 172.21.15.33:0/3353751010 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe51c09c3c0 con 0x7fe5241059b0 2024-04-02T13:38:53.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.129+0000 7fe52a69a640 1 -- 172.21.15.33:0/3353751010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe524108930 con 0x7fe5241059b0 2024-04-02T13:38:53.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.130+0000 7fe5217fa640 1 -- 172.21.15.33:0/3353751010 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24996 (secure 0 0 0) 0x7fe51c0a06c0 con 0x7fe5241059b0 2024-04-02T13:38:53.133 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:53.133 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":95,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:38:46.867341+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/80287983":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/395956530":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3947066882":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1212609914":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:38:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.132+0000 7fe52a69a640 1 -- 172.21.15.33:0/3353751010 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe50c0a3e40 msgr2=0x7fe50c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.132+0000 7fe52a69a640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe50c0a3e40 0x7fe50c0a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe514005ab0 tx=0x7fe514005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.132+0000 7fe52a69a640 1 -- 172.21.15.33:0/3353751010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5241059b0 msgr2=0x7fe52419bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.132+0000 7fe52a69a640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5241059b0 0x7fe52419bcc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00b970 tx=0x7fe51c00be30 comp rx=0 tx=0).stop 2024-04-02T13:38:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.132+0000 7fe52a69a640 1 -- 172.21.15.33:0/3353751010 shutdown_connections 2024-04-02T13:38:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.132+0000 7fe52a69a640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe50c0a3e40 0x7fe50c0a6300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.133+0000 7fe52a69a640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5241059b0 0x7fe52419bcc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.133+0000 7fe52a69a640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe524101af0 0x7fe524197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.133+0000 7fe52a69a640 1 --2- 172.21.15.33:0/3353751010 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe524101150 0x7fe524197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.133+0000 7fe52a69a640 1 -- 172.21.15.33:0/3353751010 >> 172.21.15.33:0/3353751010 conn(0x7fe524076f90 msgr2=0x7fe524103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.133+0000 7fe52a69a640 1 -- 172.21.15.33:0/3353751010 shutdown_connections 2024-04-02T13:38:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.133+0000 7fe52a69a640 1 -- 172.21.15.33:0/3353751010 wait complete. 2024-04-02T13:38:53.145 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:52 smithi033 ceph-mon[32390]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:53.145 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:52 smithi033 ceph-mon[32390]: from='client.15618 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:53.146 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:36:19.501121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:36:20.161569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:36:23.482134+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:38:53.147 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:38:53.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:52 smithi145 ceph-mon[35646]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:53.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:52 smithi145 ceph-mon[35646]: from='client.15618 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:53.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:52 smithi050 ceph-mon[35524]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:53.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:52 smithi050 ceph-mon[35524]: from='client.15618 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.250+0000 7ff88a9cd640 1 -- 172.21.15.33:0/1630826313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8841008c0 msgr2=0x7ff884100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.250+0000 7ff88a9cd640 1 --2- 172.21.15.33:0/1630826313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8841008c0 0x7ff884100ca0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff874009900 tx=0x7ff87402bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.250+0000 7ff88a9cd640 1 -- 172.21.15.33:0/1630826313 shutdown_connections 2024-04-02T13:38:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.250+0000 7ff88a9cd640 1 --2- 172.21.15.33:0/1630826313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88410e470 0x7ff884110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.250+0000 7ff88a9cd640 1 --2- 172.21.15.33:0/1630826313 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8841011e0 0x7ff88410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.250+0000 7ff88a9cd640 1 --2- 172.21.15.33:0/1630826313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8841008c0 0x7ff884100ca0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.250+0000 7ff88a9cd640 1 -- 172.21.15.33:0/1630826313 >> 172.21.15.33:0/1630826313 conn(0x7ff884076d80 msgr2=0x7ff884077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.250+0000 7ff88a9cd640 1 -- 172.21.15.33:0/1630826313 shutdown_connections 2024-04-02T13:38:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.250+0000 7ff88a9cd640 1 -- 172.21.15.33:0/1630826313 wait complete. 2024-04-02T13:38:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.250+0000 7ff88a9cd640 1 Processor -- start 2024-04-02T13:38:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff88a9cd640 1 -- start start 2024-04-02T13:38:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff88a9cd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8841008c0 0x7ff88419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff88a9cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8841011e0 0x7ff88419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff88a9cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88410e470 0x7ff88419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff88a9cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8841140a0 con 0x7ff8841011e0 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff88a9cd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff884113f20 con 0x7ff8841008c0 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff88a9cd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff884114220 con 0x7ff88410e470 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff8837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8841011e0 0x7ff88419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff8837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8841011e0 0x7ff88419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59444/0 (socket says 172.21.15.33:59444) 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff8837fe640 1 -- 172.21.15.33:0/2410442770 learned_addr learned my addr 172.21.15.33:0/2410442770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff883fff640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8841008c0 0x7ff88419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff888f6a640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88410e470 0x7ff88419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff8837fe640 1 -- 172.21.15.33:0/2410442770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88410e470 msgr2=0x7ff88419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff8837fe640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88410e470 0x7ff88419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff8837fe640 1 -- 172.21.15.33:0/2410442770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8841008c0 msgr2=0x7ff88419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff8837fe640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8841008c0 0x7ff88419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff8837fe640 1 -- 172.21.15.33:0/2410442770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88419f510 con 0x7ff8841011e0 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff888f6a640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88410e470 0x7ff88419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.251+0000 7ff8837fe640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8841011e0 0x7ff88419aa60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c920 tx=0x7ff87800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:53.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.252+0000 7ff8817fa640 1 -- 172.21.15.33:0/2410442770 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878007c70 con 0x7ff8841011e0 2024-04-02T13:38:53.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.252+0000 7ff88a9cd640 1 -- 172.21.15.33:0/2410442770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88419f800 con 0x7ff8841011e0 2024-04-02T13:38:53.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.252+0000 7ff8817fa640 1 -- 172.21.15.33:0/2410442770 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff878007e10 con 0x7ff8841011e0 2024-04-02T13:38:53.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.252+0000 7ff8817fa640 1 -- 172.21.15.33:0/2410442770 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878018800 con 0x7ff8841011e0 2024-04-02T13:38:53.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.252+0000 7ff88a9cd640 1 -- 172.21.15.33:0/2410442770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8841a7090 con 0x7ff8841011e0 2024-04-02T13:38:53.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.253+0000 7ff88a9cd640 1 -- 172.21.15.33:0/2410442770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff88406a8f0 con 0x7ff8841011e0 2024-04-02T13:38:53.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.254+0000 7ff8817fa640 1 -- 172.21.15.33:0/2410442770 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff8780189c0 con 0x7ff8841011e0 2024-04-02T13:38:53.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.254+0000 7ff8817fa640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff86c0a3e40 0x7ff86c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:53.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.254+0000 7ff8817fa640 1 -- 172.21.15.33:0/2410442770 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff8780d11c0 con 0x7ff8841011e0 2024-04-02T13:38:53.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.254+0000 7ff883fff640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff86c0a3e40 0x7ff86c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:53.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.255+0000 7ff883fff640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff86c0a3e40 0x7ff86c0a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff874005ab0 tx=0x7ff874005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:53.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.258+0000 7ff8817fa640 1 -- 172.21.15.33:0/2410442770 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff878014030 con 0x7ff8841011e0 2024-04-02T13:38:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.444+0000 7ff88a9cd640 1 -- 172.21.15.33:0/2410442770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff88419b870 con 0x7ff8841011e0 2024-04-02T13:38:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.444+0000 7ff8817fa640 1 -- 172.21.15.33:0/2410442770 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7ff87809d330 con 0x7ff8841011e0 2024-04-02T13:38:53.447 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:38:53.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.446+0000 7ff88a9cd640 1 -- 172.21.15.33:0/2410442770 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff86c0a3e40 msgr2=0x7ff86c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.446+0000 7ff88a9cd640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff86c0a3e40 0x7ff86c0a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff874005ab0 tx=0x7ff874005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.446+0000 7ff88a9cd640 1 -- 172.21.15.33:0/2410442770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8841011e0 msgr2=0x7ff88419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.446+0000 7ff88a9cd640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8841011e0 0x7ff88419aa60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c920 tx=0x7ff87800cde0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.446+0000 7ff88a9cd640 1 -- 172.21.15.33:0/2410442770 shutdown_connections 2024-04-02T13:38:53.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.446+0000 7ff88a9cd640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff86c0a3e40 0x7ff86c0a6300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.446+0000 7ff88a9cd640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88410e470 0x7ff88419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.446+0000 7ff88a9cd640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8841011e0 0x7ff88419aa60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.446+0000 7ff88a9cd640 1 --2- 172.21.15.33:0/2410442770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8841008c0 0x7ff88419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.446+0000 7ff88a9cd640 1 -- 172.21.15.33:0/2410442770 >> 172.21.15.33:0/2410442770 conn(0x7ff884076d80 msgr2=0x7ff88410f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:53.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.447+0000 7ff88a9cd640 1 -- 172.21.15.33:0/2410442770 shutdown_connections 2024-04-02T13:38:53.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.447+0000 7ff88a9cd640 1 -- 172.21.15.33:0/2410442770 wait complete. 2024-04-02T13:38:53.460 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:38:53.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.566+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1904220959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f781009b0 msgr2=0x7f5f78100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.566+0000 7f5f7ecb7640 1 --2- 172.21.15.33:0/1904220959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f781009b0 0x7f5f78100e30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009930 tx=0x7f5f6402bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:53.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.566+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1904220959 shutdown_connections 2024-04-02T13:38:53.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.566+0000 7f5f7ecb7640 1 --2- 172.21.15.33:0/1904220959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f78101370 0x7f5f7810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.566+0000 7f5f7ecb7640 1 --2- 172.21.15.33:0/1904220959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f781009b0 0x7f5f78100e30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.566+0000 7f5f7ecb7640 1 --2- 172.21.15.33:0/1904220959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f781069c0 0x7f5f78106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.566+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1904220959 >> 172.21.15.33:0/1904220959 conn(0x7f5f78076d80 msgr2=0x7f5f78077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:53.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.566+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1904220959 shutdown_connections 2024-04-02T13:38:53.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.566+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1904220959 wait complete. 2024-04-02T13:38:53.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.566+0000 7f5f7ecb7640 1 Processor -- start 2024-04-02T13:38:53.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.566+0000 7f5f7ecb7640 1 -- start start 2024-04-02T13:38:53.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f7ecb7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f781009b0 0x7f5f7819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:53.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f7ecb7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78101370 0x7f5f7819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f7ecb7640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f781069c0 0x7f5f7819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f7ecb7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78111e60 con 0x7f5f78101370 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f7ecb7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78111ce0 con 0x7f5f781009b0 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f7ecb7640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78111fe0 con 0x7f5f781069c0 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78101370 0x7f5f7819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78101370 0x7f5f7819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33230/0 (socket says 172.21.15.33:33230) 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f77fff640 1 -- 172.21.15.33:0/1502544787 learned_addr learned my addr 172.21.15.33:0/1502544787 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f7ca53640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f781009b0 0x7f5f7819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f7d254640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f781069c0 0x7f5f7819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f77fff640 1 -- 172.21.15.33:0/1502544787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f781069c0 msgr2=0x7f5f7819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f77fff640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f781069c0 0x7f5f7819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f77fff640 1 -- 172.21.15.33:0/1502544787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f781009b0 msgr2=0x7f5f7819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f77fff640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f781009b0 0x7f5f7819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f77fff640 1 -- 172.21.15.33:0/1502544787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7819f4f0 con 0x7f5f78101370 2024-04-02T13:38:53.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f77fff640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78101370 0x7f5f7819aa40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5f64000c00 tx=0x7f5f640045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:53.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f75ffb640 1 -- 172.21.15.33:0/1502544787 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64043020 con 0x7f5f78101370 2024-04-02T13:38:53.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f75ffb640 1 -- 172.21.15.33:0/1502544787 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f6402caa0 con 0x7f5f78101370 2024-04-02T13:38:53.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f75ffb640 1 -- 172.21.15.33:0/1502544787 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64003a80 con 0x7f5f78101370 2024-04-02T13:38:53.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.567+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1502544787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7819f780 con 0x7f5f78101370 2024-04-02T13:38:53.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.568+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1502544787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f781a7070 con 0x7f5f78101370 2024-04-02T13:38:53.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.569+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1502544787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f781020f0 con 0x7f5f78101370 2024-04-02T13:38:53.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.570+0000 7f5f75ffb640 1 -- 172.21.15.33:0/1502544787 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5f64046020 con 0x7f5f78101370 2024-04-02T13:38:53.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.570+0000 7f5f75ffb640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f680a3ec0 0x7f5f680a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:53.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.570+0000 7f5f7ca53640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f680a3ec0 0x7f5f680a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:53.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.570+0000 7f5f75ffb640 1 -- 172.21.15.33:0/1502544787 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5f640f28d0 con 0x7f5f78101370 2024-04-02T13:38:53.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.571+0000 7f5f7ca53640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f680a3ec0 0x7f5f680a6380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00ab60 tx=0x7f5f6c00a2d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:53.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.575+0000 7f5f75ffb640 1 -- 172.21.15.33:0/1502544787 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5f6404d070 con 0x7f5f78101370 2024-04-02T13:38:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.761+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1502544787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5f781a72d0 con 0x7f5f78101370 2024-04-02T13:38:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.762+0000 7f5f75ffb640 1 -- 172.21.15.33:0/1502544787 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f5f640beac0 con 0x7f5f78101370 2024-04-02T13:38:53.764 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.764+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1502544787 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f680a3ec0 msgr2=0x7f5f680a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.764+0000 7f5f7ecb7640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f680a3ec0 0x7f5f680a6380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00ab60 tx=0x7f5f6c00a2d0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.764+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1502544787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78101370 msgr2=0x7f5f7819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.764+0000 7f5f7ecb7640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78101370 0x7f5f7819aa40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5f64000c00 tx=0x7f5f640045e0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.764+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1502544787 shutdown_connections 2024-04-02T13:38:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.764+0000 7f5f7ecb7640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f680a3ec0 0x7f5f680a6380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.764+0000 7f5f7ecb7640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f781069c0 0x7f5f7819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.764+0000 7f5f7ecb7640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78101370 0x7f5f7819aa40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.764+0000 7f5f7ecb7640 1 --2- 172.21.15.33:0/1502544787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f781009b0 0x7f5f7819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.764+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1502544787 >> 172.21.15.33:0/1502544787 conn(0x7f5f78076d80 msgr2=0x7f5f7810c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.764+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1502544787 shutdown_connections 2024-04-02T13:38:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.764+0000 7f5f7ecb7640 1 -- 172.21.15.33:0/1502544787 wait complete. 2024-04-02T13:38:53.777 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:38:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.881+0000 7ff680863640 1 -- 172.21.15.33:0/1095763167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff67c106990 msgr2=0x7ff67c106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.881+0000 7ff680863640 1 --2- 172.21.15.33:0/1095763167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff67c106990 0x7ff67c106d70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009930 tx=0x7ff66c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.881+0000 7ff680863640 1 -- 172.21.15.33:0/1095763167 shutdown_connections 2024-04-02T13:38:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.881+0000 7ff680863640 1 --2- 172.21.15.33:0/1095763167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff67c101340 0x7ff67c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.881+0000 7ff680863640 1 --2- 172.21.15.33:0/1095763167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff67c100980 0x7ff67c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.881+0000 7ff680863640 1 --2- 172.21.15.33:0/1095763167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff67c106990 0x7ff67c106d70 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.881+0000 7ff680863640 1 -- 172.21.15.33:0/1095763167 >> 172.21.15.33:0/1095763167 conn(0x7ff67c076d80 msgr2=0x7ff67c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.881+0000 7ff680863640 1 -- 172.21.15.33:0/1095763167 shutdown_connections 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.881+0000 7ff680863640 1 -- 172.21.15.33:0/1095763167 wait complete. 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff680863640 1 Processor -- start 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff680863640 1 -- start start 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff680863640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff67c100980 0x7ff67c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff680863640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff67c101340 0x7ff67c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff680863640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff67c106990 0x7ff67c1a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff680863640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c110d70 con 0x7ff67c101340 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff680863640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c110bf0 con 0x7ff67c106990 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff680863640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c110ef0 con 0x7ff67c100980 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff673fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff67c101340 0x7ff67c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff673fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff67c101340 0x7ff67c199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33246/0 (socket says 172.21.15.33:33246) 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff673fff640 1 -- 172.21.15.33:0/2827920431 learned_addr learned my addr 172.21.15.33:0/2827920431 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff67ad9d640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff67c106990 0x7ff67c1a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff67a59c640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff67c100980 0x7ff67c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff673fff640 1 -- 172.21.15.33:0/2827920431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff67c100980 msgr2=0x7ff67c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff673fff640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff67c100980 0x7ff67c199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.882+0000 7ff673fff640 1 -- 172.21.15.33:0/2827920431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff67c106990 msgr2=0x7ff67c1a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.883+0000 7ff673fff640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff67c106990 0x7ff67c1a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.883+0000 7ff673fff640 1 -- 172.21.15.33:0/2827920431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67c1a47f0 con 0x7ff67c101340 2024-04-02T13:38:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.883+0000 7ff673fff640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff67c101340 0x7ff67c199770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff66400d950 tx=0x7ff66400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.883+0000 7ff6737fe640 1 -- 172.21.15.33:0/2827920431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664017070 con 0x7ff67c101340 2024-04-02T13:38:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.883+0000 7ff6737fe640 1 -- 172.21.15.33:0/2827920431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff664009c30 con 0x7ff67c101340 2024-04-02T13:38:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.883+0000 7ff680863640 1 -- 172.21.15.33:0/2827920431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff67c1a49c0 con 0x7ff67c101340 2024-04-02T13:38:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.883+0000 7ff6737fe640 1 -- 172.21.15.33:0/2827920431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66400f6b0 con 0x7ff67c101340 2024-04-02T13:38:53.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.883+0000 7ff680863640 1 -- 172.21.15.33:0/2827920431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff67c1a4f10 con 0x7ff67c101340 2024-04-02T13:38:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.884+0000 7ff680863640 1 -- 172.21.15.33:0/2827920431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff67c1020c0 con 0x7ff67c101340 2024-04-02T13:38:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.886+0000 7ff6737fe640 1 -- 172.21.15.33:0/2827920431 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff664010a50 con 0x7ff67c101340 2024-04-02T13:38:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.886+0000 7ff6737fe640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff6680a3e40 0x7ff6680a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.886+0000 7ff6737fe640 1 -- 172.21.15.33:0/2827920431 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff6640d0b70 con 0x7ff67c101340 2024-04-02T13:38:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.886+0000 7ff67a59c640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff6680a3e40 0x7ff6680a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.887+0000 7ff67a59c640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff6680a3e40 0x7ff6680a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0096c0 tx=0x7ff66c005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:53.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:53.890+0000 7ff6737fe640 1 -- 172.21.15.33:0/2827920431 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff664009de0 con 0x7ff67c101340 2024-04-02T13:38:54.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.075+0000 7ff680863640 1 -- 172.21.15.33:0/2827920431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff67c108ef0 con 0x7ff67c101340 2024-04-02T13:38:54.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.075+0000 7ff6737fe640 1 -- 172.21.15.33:0/2827920431 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7ff66409cc10 con 0x7ff67c101340 2024-04-02T13:38:54.078 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-02T13:38:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.077+0000 7ff680863640 1 -- 172.21.15.33:0/2827920431 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff6680a3e40 msgr2=0x7ff6680a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.077+0000 7ff680863640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff6680a3e40 0x7ff6680a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0096c0 tx=0x7ff66c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.077+0000 7ff680863640 1 -- 172.21.15.33:0/2827920431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff67c101340 msgr2=0x7ff67c199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.077+0000 7ff680863640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff67c101340 0x7ff67c199770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff66400d950 tx=0x7ff66400de10 comp rx=0 tx=0).stop 2024-04-02T13:38:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.077+0000 7ff680863640 1 -- 172.21.15.33:0/2827920431 shutdown_connections 2024-04-02T13:38:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.077+0000 7ff680863640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff6680a3e40 0x7ff6680a6300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.077+0000 7ff680863640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff67c106990 0x7ff67c1a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.077+0000 7ff680863640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff67c101340 0x7ff67c199770 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.077+0000 7ff680863640 1 --2- 172.21.15.33:0/2827920431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff67c100980 0x7ff67c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.077+0000 7ff680863640 1 -- 172.21.15.33:0/2827920431 >> 172.21.15.33:0/2827920431 conn(0x7ff67c076d80 msgr2=0x7ff67c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.077+0000 7ff680863640 1 -- 172.21.15.33:0/2827920431 shutdown_connections 2024-04-02T13:38:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.077+0000 7ff680863640 1 -- 172.21.15.33:0/2827920431 wait complete. 2024-04-02T13:38:54.090 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:53 smithi033 ceph-mon[32390]: from='client.15624 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:54.090 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:53 smithi033 ceph-mon[32390]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:38:54.090 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:53 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3353751010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:54.090 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:53 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2410442770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:54.090 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:53 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1502544787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:54.091 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.195+0000 7f2715d96640 1 -- 172.21.15.33:0/3662447532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f271006ab50 msgr2=0x7f271010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.195+0000 7f2715d96640 1 --2- 172.21.15.33:0/3662447532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f271006ab50 0x7f271010e960 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2708009cd0 tx=0x7f270802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.195+0000 7f2715d96640 1 -- 172.21.15.33:0/3662447532 shutdown_connections 2024-04-02T13:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.195+0000 7f2715d96640 1 --2- 172.21.15.33:0/3662447532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f271006ab50 0x7f271010e960 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.195+0000 7f2715d96640 1 --2- 172.21.15.33:0/3662447532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f271006a190 0x7f271006a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.195+0000 7f2715d96640 1 --2- 172.21.15.33:0/3662447532 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2710107aa0 0x7f2710107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.195+0000 7f2715d96640 1 -- 172.21.15.33:0/3662447532 >> 172.21.15.33:0/3662447532 conn(0x7f2710077ad0 msgr2=0x7f2710077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.195+0000 7f2715d96640 1 -- 172.21.15.33:0/3662447532 shutdown_connections 2024-04-02T13:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.195+0000 7f2715d96640 1 -- 172.21.15.33:0/3662447532 wait complete. 2024-04-02T13:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.195+0000 7f2715d96640 1 Processor -- start 2024-04-02T13:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f2715d96640 1 -- start start 2024-04-02T13:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f2715d96640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f271006a190 0x7f2710198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f2715d96640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f271006ab50 0x7f27101988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f2715d96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2710107aa0 0x7f271019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f2715d96640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2710111e80 con 0x7f2710107aa0 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f2715d96640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2710111d00 con 0x7f271006a190 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f2715d96640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2710112000 con 0x7f271006ab50 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f270ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2710107aa0 0x7f271019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f270ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2710107aa0 0x7f271019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33268/0 (socket says 172.21.15.33:33268) 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f270ffff640 1 -- 172.21.15.33:0/370329908 learned_addr learned my addr 172.21.15.33:0/370329908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f270effd640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f271006ab50 0x7f27101988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f270f7fe640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f271006a190 0x7f2710198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f270ffff640 1 -- 172.21.15.33:0/370329908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f271006ab50 msgr2=0x7f27101988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f270ffff640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f271006ab50 0x7f27101988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f270ffff640 1 -- 172.21.15.33:0/370329908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f271006a190 msgr2=0x7f2710198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f270ffff640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f271006a190 0x7f2710198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f270ffff640 1 -- 172.21.15.33:0/370329908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271019d350 con 0x7f2710107aa0 2024-04-02T13:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f270ffff640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2710107aa0 0x7f271019cc20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2708004af0 tx=0x7f2708034d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.196+0000 7f270cff9640 1 -- 172.21.15.33:0/370329908 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708039050 con 0x7f2710107aa0 2024-04-02T13:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.197+0000 7f270cff9640 1 -- 172.21.15.33:0/370329908 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2708045070 con 0x7f2710107aa0 2024-04-02T13:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.197+0000 7f2715d96640 1 -- 172.21.15.33:0/370329908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271019d5e0 con 0x7f2710107aa0 2024-04-02T13:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.197+0000 7f270cff9640 1 -- 172.21.15.33:0/370329908 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27080046b0 con 0x7f2710107aa0 2024-04-02T13:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.197+0000 7f2715d96640 1 -- 172.21.15.33:0/370329908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27101a4ed0 con 0x7f2710107aa0 2024-04-02T13:38:54.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.198+0000 7f2715d96640 1 -- 172.21.15.33:0/370329908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27101031d0 con 0x7f2710107aa0 2024-04-02T13:38:54.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.199+0000 7f270cff9640 1 -- 172.21.15.33:0/370329908 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f2708003970 con 0x7f2710107aa0 2024-04-02T13:38:54.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.200+0000 7f270cff9640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f26f80a3e40 0x7f26f80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:54.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.200+0000 7f270f7fe640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f26f80a3e40 0x7f26f80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:54.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.200+0000 7f270cff9640 1 -- 172.21.15.33:0/370329908 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f27080f4230 con 0x7f2710107aa0 2024-04-02T13:38:54.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.200+0000 7f270f7fe640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f26f80a3e40 0x7f26f80a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2700005fd0 tx=0x7f270000b510 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:54.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.204+0000 7f270cff9640 1 -- 172.21.15.33:0/370329908 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f27080c0420 con 0x7f2710107aa0 2024-04-02T13:38:54.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:53 smithi145 ceph-mon[35646]: from='client.15624 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:54.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:53 smithi145 ceph-mon[35646]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:38:54.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:53 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3353751010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:54.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:53 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2410442770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:54.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:53 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1502544787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:54.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:53 smithi050 ceph-mon[35524]: from='client.15624 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:38:54.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:53 smithi050 ceph-mon[35524]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:38:54.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:53 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3353751010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:54.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:53 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2410442770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:38:54.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:53 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1502544787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:38:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.389+0000 7f2715d96640 1 -- 172.21.15.33:0/370329908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f27101996b0 con 0x7f2710107aa0 2024-04-02T13:38:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.390+0000 7f270cff9640 1 -- 172.21.15.33:0/370329908 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f27080bfd20 con 0x7f2710107aa0 2024-04-02T13:38:54.393 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:38:54.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.392+0000 7f2715d96640 1 -- 172.21.15.33:0/370329908 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f26f80a3e40 msgr2=0x7f26f80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.392+0000 7f2715d96640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f26f80a3e40 0x7f26f80a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2700005fd0 tx=0x7f270000b510 comp rx=0 tx=0).stop 2024-04-02T13:38:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.392+0000 7f2715d96640 1 -- 172.21.15.33:0/370329908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2710107aa0 msgr2=0x7f271019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.392+0000 7f2715d96640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2710107aa0 0x7f271019cc20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2708004af0 tx=0x7f2708034d70 comp rx=0 tx=0).stop 2024-04-02T13:38:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.392+0000 7f2715d96640 1 -- 172.21.15.33:0/370329908 shutdown_connections 2024-04-02T13:38:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.392+0000 7f2715d96640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f26f80a3e40 0x7f26f80a6300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.392+0000 7f2715d96640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2710107aa0 0x7f271019cc20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.392+0000 7f2715d96640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f271006ab50 0x7f27101988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.392+0000 7f2715d96640 1 --2- 172.21.15.33:0/370329908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f271006a190 0x7f2710198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.392+0000 7f2715d96640 1 -- 172.21.15.33:0/370329908 >> 172.21.15.33:0/370329908 conn(0x7f2710077ad0 msgr2=0x7f27100fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.392+0000 7f2715d96640 1 -- 172.21.15.33:0/370329908 shutdown_connections 2024-04-02T13:38:54.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.392+0000 7f2715d96640 1 -- 172.21.15.33:0/370329908 wait complete. 2024-04-02T13:38:54.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:38:54.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.505+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/2324846202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c8100270 msgr2=0x7fe7c8100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.505+0000 7fe7ce1e2640 1 --2- 172.21.15.33:0/2324846202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c8100270 0x7fe7c8100650 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009900 tx=0x7fe7b802bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:54.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.505+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/2324846202 shutdown_connections 2024-04-02T13:38:54.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.505+0000 7fe7ce1e2640 1 --2- 172.21.15.33:0/2324846202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7c8106e30 0x7fe7c810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.505+0000 7fe7ce1e2640 1 --2- 172.21.15.33:0/2324846202 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c8100c20 0x7fe7c8106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.505+0000 7fe7ce1e2640 1 --2- 172.21.15.33:0/2324846202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c8100270 0x7fe7c8100650 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.505+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/2324846202 >> 172.21.15.33:0/2324846202 conn(0x7fe7c8076f90 msgr2=0x7fe7c80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:54.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.506+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/2324846202 shutdown_connections 2024-04-02T13:38:54.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.506+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/2324846202 wait complete. 2024-04-02T13:38:54.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.506+0000 7fe7ce1e2640 1 Processor -- start 2024-04-02T13:38:54.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.506+0000 7fe7ce1e2640 1 -- start start 2024-04-02T13:38:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.506+0000 7fe7ce1e2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c8100270 0x7fe7c8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.506+0000 7fe7ce1e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c8106e30 0x7fe7c8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.506+0000 7fe7ce1e2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7c819cc70 0x7fe7c819d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.506+0000 7fe7ce1e2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8111e80 con 0x7fe7c8106e30 2024-04-02T13:38:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.506+0000 7fe7ce1e2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8111d00 con 0x7fe7c8100270 2024-04-02T13:38:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.506+0000 7fe7ce1e2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8112000 con 0x7fe7c819cc70 2024-04-02T13:38:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c8106e30 0x7fe7c8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c8106e30 0x7fe7c8198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33294/0 (socket says 172.21.15.33:33294) 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c6ffd640 1 -- 172.21.15.33:0/1261161985 learned_addr learned my addr 172.21.15.33:0/1261161985 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c77fe640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c8100270 0x7fe7c8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c7fff640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7c819cc70 0x7fe7c819d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c6ffd640 1 -- 172.21.15.33:0/1261161985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7c819cc70 msgr2=0x7fe7c819d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c6ffd640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7c819cc70 0x7fe7c819d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c6ffd640 1 -- 172.21.15.33:0/1261161985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c8100270 msgr2=0x7fe7c8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c6ffd640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c8100270 0x7fe7c8198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c6ffd640 1 -- 172.21.15.33:0/1261161985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c819d760 con 0x7fe7c8106e30 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c6ffd640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c8106e30 0x7fe7c8198940 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc005fd0 tx=0x7fe7bc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c4ff9640 1 -- 172.21.15.33:0/1261161985 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc017070 con 0x7fe7c8106e30 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c4ff9640 1 -- 172.21.15.33:0/1261161985 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe7bc009c30 con 0x7fe7c8106e30 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/1261161985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c819da50 con 0x7fe7c8106e30 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7c4ff9640 1 -- 172.21.15.33:0/1261161985 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc003e20 con 0x7fe7c8106e30 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.507+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/1261161985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c81a51e0 con 0x7fe7c8106e30 2024-04-02T13:38:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.508+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/1261161985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c8068d40 con 0x7fe7c8106e30 2024-04-02T13:38:54.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.509+0000 7fe7c4ff9640 1 -- 172.21.15.33:0/1261161985 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe7bc003fc0 con 0x7fe7c8106e30 2024-04-02T13:38:54.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.510+0000 7fe7c4ff9640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe7b00a3e40 0x7fe7b00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:54.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.510+0000 7fe7c77fe640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe7b00a3e40 0x7fe7b00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:54.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.510+0000 7fe7c4ff9640 1 -- 172.21.15.33:0/1261161985 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe7bc0d0ad0 con 0x7fe7c8106e30 2024-04-02T13:38:54.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.510+0000 7fe7c77fe640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe7b00a3e40 0x7fe7b00a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8005ab0 tx=0x7fe7b8005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:54.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.515+0000 7fe7c4ff9640 1 -- 172.21.15.33:0/1261161985 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe7bc020030 con 0x7fe7c8106e30 2024-04-02T13:38:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.741+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/1261161985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7c810a050 con 0x7fe7c8106e30 2024-04-02T13:38:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.742+0000 7fe7c4ff9640 1 -- 172.21.15.33:0/1261161985 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11873 (secure 0 0 0) 0x7fe7bc09cb70 con 0x7fe7c8106e30 2024-04-02T13:38:54.745 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:54.745 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:38:45.531661+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24473,24512]},"id":1}]} 2024-04-02T13:38:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 25 2024-04-02T13:38:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.744+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/1261161985 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe7b00a3e40 msgr2=0x7fe7b00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.744+0000 7fe7ce1e2640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe7b00a3e40 0x7fe7b00a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8005ab0 tx=0x7fe7b8005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.744+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/1261161985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c8106e30 msgr2=0x7fe7c8198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.744+0000 7fe7ce1e2640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c8106e30 0x7fe7c8198940 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc005fd0 tx=0x7fe7bc00dde0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.744+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/1261161985 shutdown_connections 2024-04-02T13:38:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.744+0000 7fe7ce1e2640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe7b00a3e40 0x7fe7b00a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.744+0000 7fe7ce1e2640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe7c819cc70 0x7fe7c819d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.744+0000 7fe7ce1e2640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c8106e30 0x7fe7c8198940 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.744+0000 7fe7ce1e2640 1 --2- 172.21.15.33:0/1261161985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7c8100270 0x7fe7c8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.744+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/1261161985 >> 172.21.15.33:0/1261161985 conn(0x7fe7c8076f90 msgr2=0x7fe7c8104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.744+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/1261161985 shutdown_connections 2024-04-02T13:38:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.744+0000 7fe7ce1e2640 1 -- 172.21.15.33:0/1261161985 wait complete. 2024-04-02T13:38:54.758 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:38:54.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.858+0000 7fcb487db640 1 -- 172.21.15.33:0/2979797618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb4006a150 msgr2=0x7fcb4006a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.858+0000 7fcb487db640 1 --2- 172.21.15.33:0/2979797618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb4006a150 0x7fcb4006a5b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009900 tx=0x7fcb3002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:54.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.858+0000 7fcb487db640 1 -- 172.21.15.33:0/2979797618 shutdown_connections 2024-04-02T13:38:54.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.858+0000 7fcb487db640 1 --2- 172.21.15.33:0/2979797618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb4006aaf0 0x7fcb4010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.858+0000 7fcb487db640 1 --2- 172.21.15.33:0/2979797618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb4006a150 0x7fcb4006a5b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.858+0000 7fcb487db640 1 --2- 172.21.15.33:0/2979797618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb40107b00 0x7fcb40107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.858+0000 7fcb487db640 1 -- 172.21.15.33:0/2979797618 >> 172.21.15.33:0/2979797618 conn(0x7fcb40076800 msgr2=0x7fcb40076c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.858+0000 7fcb487db640 1 -- 172.21.15.33:0/2979797618 shutdown_connections 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.858+0000 7fcb487db640 1 -- 172.21.15.33:0/2979797618 wait complete. 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb487db640 1 Processor -- start 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb487db640 1 -- start start 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb487db640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb4006a150 0x7fcb40198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb487db640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb4006aaf0 0x7fcb401989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb487db640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb40107b00 0x7fcb4019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb487db640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb40111eb0 con 0x7fcb4006a150 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb487db640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcb40111d30 con 0x7fcb40107b00 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb487db640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcb40112030 con 0x7fcb4006aaf0 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb46577640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb4006a150 0x7fcb40198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb46577640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb4006a150 0x7fcb40198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33302/0 (socket says 172.21.15.33:33302) 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb46577640 1 -- 172.21.15.33:0/2534116537 learned_addr learned my addr 172.21.15.33:0/2534116537 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb46d78640 1 --2- 172.21.15.33:0/2534116537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb40107b00 0x7fcb4019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:54.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb46577640 1 -- 172.21.15.33:0/2534116537 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb4006aaf0 msgr2=0x7fcb401989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb46577640 1 --2- 172.21.15.33:0/2534116537 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb4006aaf0 0x7fcb401989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb46577640 1 -- 172.21.15.33:0/2534116537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb40107b00 msgr2=0x7fcb4019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:54.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb46577640 1 --2- 172.21.15.33:0/2534116537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb40107b00 0x7fcb4019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:54.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.859+0000 7fcb46577640 1 -- 172.21.15.33:0/2534116537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4019d430 con 0x7fcb4006a150 2024-04-02T13:38:54.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.860+0000 7fcb46577640 1 --2- 172.21.15.33:0/2534116537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb4006a150 0x7fcb40198490 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00d950 tx=0x7fcb3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:54.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.860+0000 7fcb377fe640 1 -- 172.21.15.33:0/2534116537 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c017030 con 0x7fcb4006a150 2024-04-02T13:38:54.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.860+0000 7fcb487db640 1 -- 172.21.15.33:0/2534116537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4019d720 con 0x7fcb4006a150 2024-04-02T13:38:54.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.860+0000 7fcb377fe640 1 -- 172.21.15.33:0/2534116537 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcb3c004020 con 0x7fcb4006a150 2024-04-02T13:38:54.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.860+0000 7fcb377fe640 1 -- 172.21.15.33:0/2534116537 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c0047a0 con 0x7fcb4006a150 2024-04-02T13:38:54.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.860+0000 7fcb487db640 1 -- 172.21.15.33:0/2534116537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb401a4fb0 con 0x7fcb4006a150 2024-04-02T13:38:54.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.861+0000 7fcb487db640 1 -- 172.21.15.33:0/2534116537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb401031c0 con 0x7fcb4006a150 2024-04-02T13:38:54.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.862+0000 7fcb377fe640 1 -- 172.21.15.33:0/2534116537 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fcb3c0041d0 con 0x7fcb4006a150 2024-04-02T13:38:54.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.862+0000 7fcb377fe640 1 --2- 172.21.15.33:0/2534116537 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcb2c0a3e40 0x7fcb2c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:54.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.862+0000 7fcb45d76640 1 --2- 172.21.15.33:0/2534116537 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcb2c0a3e40 0x7fcb2c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:54.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.863+0000 7fcb377fe640 1 -- 172.21.15.33:0/2534116537 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcb3c0d0f30 con 0x7fcb4006a150 2024-04-02T13:38:54.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.863+0000 7fcb45d76640 1 --2- 172.21.15.33:0/2534116537 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcb2c0a3e40 0x7fcb2c0a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb300096c0 tx=0x7fcb30000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:54.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:54.866+0000 7fcb377fe640 1 -- 172.21.15.33:0/2534116537 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcb3c0fff20 con 0x7fcb4006a150 2024-04-02T13:38:55.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.055+0000 7fcb487db640 1 -- 172.21.15.33:0/2534116537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb40068d40 con 0x7fcb4006a150 2024-04-02T13:38:55.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.056+0000 7fcb377fe640 1 -- 172.21.15.33:0/2534116537 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24996 (secure 0 0 0) 0x7fcb3c09cfd0 con 0x7fcb4006a150 2024-04-02T13:38:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:38:55.059 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":95,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:38:46.867341+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/80287983":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/395956530":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3947066882":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1212609914":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:38:55.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.058+0000 7fcb487db640 1 -- 172.21.15.33:0/2534116537 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcb2c0a3e40 msgr2=0x7fcb2c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.058+0000 7fcb487db640 1 --2- 172.21.15.33:0/2534116537 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcb2c0a3e40 0x7fcb2c0a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb300096c0 tx=0x7fcb30000bc0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.058+0000 7fcb487db640 1 -- 172.21.15.33:0/2534116537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb4006a150 msgr2=0x7fcb40198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.058+0000 7fcb487db640 1 --2- 172.21.15.33:0/2534116537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb4006a150 0x7fcb40198490 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00d950 tx=0x7fcb3c00de10 comp rx=0 tx=0).stop 2024-04-02T13:38:55.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.058+0000 7fcb487db640 1 -- 172.21.15.33:0/2534116537 shutdown_connections 2024-04-02T13:38:55.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.058+0000 7fcb487db640 1 --2- 172.21.15.33:0/2534116537 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcb2c0a3e40 0x7fcb2c0a6300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.058+0000 7fcb487db640 1 --2- 172.21.15.33:0/2534116537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb40107b00 0x7fcb4019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.058+0000 7fcb487db640 1 --2- 172.21.15.33:0/2534116537 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb4006aaf0 0x7fcb401989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.058+0000 7fcb487db640 1 --2- 172.21.15.33:0/2534116537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb4006a150 0x7fcb40198490 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.058+0000 7fcb487db640 1 -- 172.21.15.33:0/2534116537 >> 172.21.15.33:0/2534116537 conn(0x7fcb40076800 msgr2=0x7fcb400fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.058+0000 7fcb487db640 1 -- 172.21.15.33:0/2534116537 shutdown_connections 2024-04-02T13:38:55.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.058+0000 7fcb487db640 1 -- 172.21.15.33:0/2534116537 wait complete. 2024-04-02T13:38:55.071 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:54 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2827920431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:55.071 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:54 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/370329908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:55.071 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:54 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1261161985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:55.072 DEBUG:tasks.fwd_scrub:Yielding 2024-04-02T13:38:55.073 INFO:teuthology.run_tasks:Running task exec... 2024-04-02T13:38:55.083 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T13:38:55.083 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-02T13:38:55.083 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-02T13:38:55.083 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-02T13:38:55.084 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:38:55.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:55.084 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:38:55.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.187+0000 7feec9029640 1 -- 172.21.15.33:0/2531992835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feec4101610 msgr2=0x7feec410d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.187+0000 7feec9029640 1 --2- 172.21.15.33:0/2531992835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feec4101610 0x7feec410d830 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7feebc009900 tx=0x7feebc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:55.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.187+0000 7feec9029640 1 -- 172.21.15.33:0/2531992835 shutdown_connections 2024-04-02T13:38:55.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.187+0000 7feec9029640 1 --2- 172.21.15.33:0/2531992835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feec4101610 0x7feec410d830 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.187+0000 7feec9029640 1 --2- 172.21.15.33:0/2531992835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec4100c50 0x7feec41010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.187+0000 7feec9029640 1 --2- 172.21.15.33:0/2531992835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feec41059a0 0x7feec4105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.187+0000 7feec9029640 1 -- 172.21.15.33:0/2531992835 >> 172.21.15.33:0/2531992835 conn(0x7feec4077850 msgr2=0x7feec4077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.187+0000 7feec9029640 1 -- 172.21.15.33:0/2531992835 shutdown_connections 2024-04-02T13:38:55.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.187+0000 7feec9029640 1 -- 172.21.15.33:0/2531992835 wait complete. 2024-04-02T13:38:55.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.188+0000 7f43a1449640 1 -- 172.21.15.33:0/1099035127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f439c068ce0 msgr2=0x7f439c069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.188+0000 7f43a1449640 1 --2- 172.21.15.33:0/1099035127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f439c068ce0 0x7f439c069160 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4390009930 tx=0x7f439002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:55.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.188+0000 7f43a1449640 1 -- 172.21.15.33:0/1099035127 shutdown_connections 2024-04-02T13:38:55.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.188+0000 7f43a1449640 1 --2- 172.21.15.33:0/1099035127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f439c0696a0 0x7f439c10e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.188+0000 7f43a1449640 1 --2- 172.21.15.33:0/1099035127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f439c068ce0 0x7f439c069160 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.188+0000 7f43a1449640 1 --2- 172.21.15.33:0/1099035127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f439c107a80 0x7f439c107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.188+0000 7f43a1449640 1 -- 172.21.15.33:0/1099035127 >> 172.21.15.33:0/1099035127 conn(0x7f439c077b00 msgr2=0x7f439c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.188+0000 7f43a1449640 1 -- 172.21.15.33:0/1099035127 shutdown_connections 2024-04-02T13:38:55.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.188+0000 7f43a1449640 1 -- 172.21.15.33:0/1099035127 wait complete. 2024-04-02T13:38:55.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.188+0000 7feec9029640 1 Processor -- start 2024-04-02T13:38:55.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.188+0000 7feec9029640 1 -- start start 2024-04-02T13:38:55.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec9029640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feec4100c50 0x7feec41990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec9029640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec4101610 0x7feec4199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec9029640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feec41059a0 0x7feec41a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec9029640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feec4110ee0 con 0x7feec41059a0 2024-04-02T13:38:55.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec9029640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feec4110d60 con 0x7feec4100c50 2024-04-02T13:38:55.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec9029640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feec4111060 con 0x7feec4101610 2024-04-02T13:38:55.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7f43a1449640 1 Processor -- start 2024-04-02T13:38:55.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feec41059a0 0x7feec41a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feec41059a0 0x7feec41a4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33322/0 (socket says 172.21.15.33:33322) 2024-04-02T13:38:55.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec359e640 1 -- 172.21.15.33:0/1232724474 learned_addr learned my addr 172.21.15.33:0/1232724474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:55.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec259c640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec4101610 0x7feec4199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7f43a1449640 1 -- start start 2024-04-02T13:38:55.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec2d9d640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feec4100c50 0x7feec41990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec359e640 1 -- 172.21.15.33:0/1232724474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec4101610 msgr2=0x7feec4199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec359e640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec4101610 0x7feec4199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec359e640 1 -- 172.21.15.33:0/1232724474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feec4100c50 msgr2=0x7feec41990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec359e640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feec4100c50 0x7feec41990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec359e640 1 -- 172.21.15.33:0/1232724474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec41a4680 con 0x7feec41059a0 2024-04-02T13:38:55.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7f43a1449640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f439c068ce0 0x7f439c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7f43a1449640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f439c0696a0 0x7f439c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7f43a1449640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f439c107a80 0x7f439c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7f43a1449640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f439c111e60 con 0x7f439c0696a0 2024-04-02T13:38:55.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7f43a1449640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f439c111ce0 con 0x7f439c068ce0 2024-04-02T13:38:55.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7f43a1449640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f439c111fe0 con 0x7f439c107a80 2024-04-02T13:38:55.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7feec359e640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feec41059a0 0x7feec41a4140 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7feebc005ab0 tx=0x7feebc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7f439a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f439c0696a0 0x7f439c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7f439a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f439c0696a0 0x7f439c198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33336/0 (socket says 172.21.15.33:33336) 2024-04-02T13:38:55.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7f439a7fc640 1 -- 172.21.15.33:0/2042808876 learned_addr learned my addr 172.21.15.33:0/2042808876 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:55.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7f439b7fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f439c107a80 0x7f439c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.189+0000 7f439affd640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f439c068ce0 0x7f439c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.190+0000 7feeabfff640 1 -- 172.21.15.33:0/1232724474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc043020 con 0x7feec41059a0 2024-04-02T13:38:55.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.190+0000 7feec9029640 1 -- 172.21.15.33:0/1232724474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec41a4850 con 0x7feec41059a0 2024-04-02T13:38:55.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.190+0000 7f439a7fc640 1 -- 172.21.15.33:0/2042808876 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f439c107a80 msgr2=0x7f439c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.190+0000 7f439a7fc640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f439c107a80 0x7f439c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.190+0000 7f439a7fc640 1 -- 172.21.15.33:0/2042808876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f439c068ce0 msgr2=0x7f439c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.190+0000 7f439a7fc640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f439c068ce0 0x7f439c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.190+0000 7feec9029640 1 -- 172.21.15.33:0/1232724474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec4197220 con 0x7feec41059a0 2024-04-02T13:38:55.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.190+0000 7f439a7fc640 1 -- 172.21.15.33:0/2042808876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c19d3f0 con 0x7f439c0696a0 2024-04-02T13:38:55.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.190+0000 7f439b7fe640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f439c107a80 0x7f439c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:55.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.190+0000 7feeabfff640 1 -- 172.21.15.33:0/1232724474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7feebc0054d0 con 0x7feec41059a0 2024-04-02T13:38:55.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.190+0000 7feeabfff640 1 -- 172.21.15.33:0/1232724474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc003b20 con 0x7feec41059a0 2024-04-02T13:38:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.190+0000 7f439a7fc640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f439c0696a0 0x7f439c198940 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4390000c00 tx=0x7f4390004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.191+0000 7f4383fff640 1 -- 172.21.15.33:0/2042808876 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390043020 con 0x7f439c0696a0 2024-04-02T13:38:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.191+0000 7f4383fff640 1 -- 172.21.15.33:0/2042808876 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f439002c5d0 con 0x7f439c0696a0 2024-04-02T13:38:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.191+0000 7f43a1449640 1 -- 172.21.15.33:0/2042808876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439c19d680 con 0x7f439c0696a0 2024-04-02T13:38:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.191+0000 7f4383fff640 1 -- 172.21.15.33:0/2042808876 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390003970 con 0x7f439c0696a0 2024-04-02T13:38:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.191+0000 7f43a1449640 1 -- 172.21.15.33:0/2042808876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439c1008f0 con 0x7f439c0696a0 2024-04-02T13:38:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.191+0000 7feec9029640 1 -- 172.21.15.33:0/1232724474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec4102110 con 0x7feec41059a0 2024-04-02T13:38:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.192+0000 7feeabfff640 1 -- 172.21.15.33:0/1232724474 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7feebc046020 con 0x7feec41059a0 2024-04-02T13:38:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.193+0000 7feeabfff640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7feeac0a3f60 0x7feeac0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.193+0000 7feec2d9d640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7feeac0a3f60 0x7feeac0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.193+0000 7feeabfff640 1 -- 172.21.15.33:0/1232724474 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7feebc0f3310 con 0x7feec41059a0 2024-04-02T13:38:55.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.193+0000 7f43a402a740 1 Processor -- start 2024-04-02T13:38:55.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.193+0000 7feec2d9d640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7feeac0a3f60 0x7feeac0a6420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feeb4007d50 tx=0x7feeb4007730 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.193+0000 7f43a402a740 1 -- start start 2024-04-02T13:38:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.193+0000 7f4383fff640 1 -- 172.21.15.33:0/2042808876 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4390046020 con 0x7f439c0696a0 2024-04-02T13:38:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f43a402a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf82c0 0x559f18cf86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f43a402a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18cf8be0 0x559f18d01280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f43a402a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18d017c0 0x559f18d03bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f43a402a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559f18c5c770 con 0x559f18d017c0 2024-04-02T13:38:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f43a402a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559f18c04960 con 0x559f18cf8be0 2024-04-02T13:38:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f43a402a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559f18c588c0 con 0x559f18cf82c0 2024-04-02T13:38:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f439a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18d017c0 0x559f18d03bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f439a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18d017c0 0x559f18d03bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33346/0 (socket says 172.21.15.33:33346) 2024-04-02T13:38:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f439a7fc640 1 -- 172.21.15.33:0/2375797278 learned_addr learned my addr 172.21.15.33:0/2375797278 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f439affd640 1 --2- 172.21.15.33:0/2375797278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18cf8be0 0x559f18d01280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f439b7fe640 1 --2- 172.21.15.33:0/2375797278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf82c0 0x559f18cf86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f4383fff640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43840a3ec0 0x7f43840a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f439a7fc640 1 -- 172.21.15.33:0/2375797278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf82c0 msgr2=0x559f18cf86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f439a7fc640 1 --2- 172.21.15.33:0/2375797278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf82c0 0x559f18cf86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f439a7fc640 1 -- 172.21.15.33:0/2375797278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18cf8be0 msgr2=0x559f18d01280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f439a7fc640 1 --2- 172.21.15.33:0/2375797278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18cf8be0 0x559f18d01280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f439a7fc640 1 -- 172.21.15.33:0/2375797278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f18a387e0 con 0x559f18d017c0 2024-04-02T13:38:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f439b7fe640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43840a3ec0 0x7f43840a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f4383fff640 1 -- 172.21.15.33:0/2042808876 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f43900f2590 con 0x7f439c0696a0 2024-04-02T13:38:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.194+0000 7f439a7fc640 1 --2- 172.21.15.33:0/2375797278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18d017c0 0x559f18d03bb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f43900ef8a0 tx=0x7f4390005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec50070e42c1f894 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.195+0000 7f439b7fe640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43840a3ec0 0x7f43840a6380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4394005fd0 tx=0x7f4394013460 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.197+0000 7f43817fa640 1 -- 172.21.15.33:0/2375797278 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900bdbc0 con 0x559f18d017c0 2024-04-02T13:38:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.197+0000 7f43817fa640 1 -- 172.21.15.33:0/2375797278 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f43900be780 con 0x559f18d017c0 2024-04-02T13:38:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.197+0000 7f43817fa640 1 -- 172.21.15.33:0/2375797278 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900bd9e0 con 0x559f18d017c0 2024-04-02T13:38:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.197+0000 7feeabfff640 1 -- 172.21.15.33:0/1232724474 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7feebc04d020 con 0x7feec41059a0 2024-04-02T13:38:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.198+0000 7f43a402a740 1 -- 172.21.15.33:0/2375797278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18d017c0 msgr2=0x559f18d03bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.198+0000 7f43a402a740 1 --2- 172.21.15.33:0/2375797278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18d017c0 0x559f18d03bb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f43900ef8a0 tx=0x7f4390005890 comp rx=0 tx=0).stop 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.198+0000 7f43a402a740 1 -- 172.21.15.33:0/2375797278 shutdown_connections 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.198+0000 7f43a402a740 1 --2- 172.21.15.33:0/2375797278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18d017c0 0x559f18d03bb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.198+0000 7f43a402a740 1 --2- 172.21.15.33:0/2375797278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18cf8be0 0x559f18d01280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.198+0000 7f43a402a740 1 --2- 172.21.15.33:0/2375797278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf82c0 0x559f18cf86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.198+0000 7f43a402a740 1 -- 172.21.15.33:0/2375797278 >> 172.21.15.33:0/2375797278 conn(0x559f18cf29f0 msgr2=0x559f18cf2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.198+0000 7f43a402a740 1 -- 172.21.15.33:0/2375797278 shutdown_connections 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.198+0000 7f43a402a740 1 -- 172.21.15.33:0/2375797278 wait complete. 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.198+0000 7f43a402a740 1 -- start start 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f43a402a740 1 Processor -- start 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f43a402a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 0x559f18cfe780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f43a402a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18cf8be0 0x559f18cfecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f43a402a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18d017c0 0x559f18cff200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f43a402a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559f18c5c770 con 0x559f18cf82c0 2024-04-02T13:38:55.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f43a402a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559f18c17d20 con 0x559f18cf8be0 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f43a402a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559f18d04370 con 0x559f18d017c0 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f43a402a740 10 client.0 _ll_register_callbacks cb 0x559f18be4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f439affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 0x559f18cfe780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f439affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 0x559f18cfe780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33352/0 (socket says 172.21.15.33:33352) 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f439affd640 1 -- 172.21.15.33:0/3106242670 learned_addr learned my addr 172.21.15.33:0/3106242670 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f439a7fc640 1 --2- 172.21.15.33:0/3106242670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18cf8be0 0x559f18cfecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f439b7fe640 1 --2- 172.21.15.33:0/3106242670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18d017c0 0x559f18cff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f439affd640 1 -- 172.21.15.33:0/3106242670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18d017c0 msgr2=0x559f18cff200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f439affd640 1 --2- 172.21.15.33:0/3106242670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18d017c0 0x559f18cff200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f439affd640 1 -- 172.21.15.33:0/3106242670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18cf8be0 msgr2=0x559f18cfecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f439affd640 1 --2- 172.21.15.33:0/3106242670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18cf8be0 0x559f18cfecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.199+0000 7f439affd640 1 -- 172.21.15.33:0/3106242670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f18a98f70 con 0x559f18cf82c0 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f439affd640 1 --2- 172.21.15.33:0/3106242670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 0x559f18cfe780 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f438c012710 tx=0x7f438c012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cec76b17d17d530 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f437eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f437eff5640 1 -- 172.21.15.33:0/3106242670 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c00f8c0 con 0x559f18cf82c0 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f437eff5640 1 -- 172.21.15.33:0/3106242670 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f438c00fa60 con 0x559f18cf82c0 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f43a402a740 1 -- 172.21.15.33:0/3106242670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43900be780 con 0x559f18cf82c0 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f437eff5640 1 -- 172.21.15.33:0/3106242670 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f438c00fc60 con 0x559f18cf82c0 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f43a402a740 10 client.15684 fetch_fsmap learned FSMap version 25 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f43a402a740 1 -- 172.21.15.33:0/3106242670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559f18930170 con 0x559f18cf82c0 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f437eff5640 1 -- 172.21.15.33:0/3106242670 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f43900be780 con 0x559f18cf82c0 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f437eff5640 1 -- 172.21.15.33:0/3106242670 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f438c01c420 con 0x559f18cf82c0 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f43a402a740 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f43a402a740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f43a402a740 20 client.15684 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f43a402a740 1 --2- 172.21.15.33:0/3106242670 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f18d11710 0x559f18d13b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f43a402a740 4 client.15684 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.200+0000 7f43a402a740 1 -- 172.21.15.33:0/3106242670 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f18c69cb0 con 0x559f18d11710 2024-04-02T13:38:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.201+0000 7f439a7fc640 1 --2- 172.21.15.33:0/3106242670 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f18d11710 0x559f18d13b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.201+0000 7f437eff5640 10 client.15684 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.201+0000 7f439a7fc640 1 --2- 172.21.15.33:0/3106242670 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f18d11710 0x559f18d13b20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c984d334387c083a server_cookie=2b55ed70ff58256a in_seq=0 out_seq=0 2024-04-02T13:38:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f437eff5640 1 -- 172.21.15.33:0/3106242670 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f18c69cb0 con 0x559f18d11710 2024-04-02T13:38:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f437eff5640 10 client.15684 handle_command_reply: tid=0 2024-04-02T13:38:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f43a402a740 1 client.15684 shutdown 2024-04-02T13:38:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f43a402a740 1 -- 172.21.15.33:0/3106242670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 msgr2=0x559f18cfe780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f43a402a740 1 --2- 172.21.15.33:0/3106242670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 0x559f18cfe780 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f438c012710 tx=0x7f438c012bd0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f43a402a740 1 -- 172.21.15.33:0/3106242670 shutdown_connections 2024-04-02T13:38:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f43a402a740 1 --2- 172.21.15.33:0/3106242670 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f18d11710 0x559f18d13b20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f437eff5640 0 client.15684 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f43a402a740 1 --2- 172.21.15.33:0/3106242670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18d017c0 0x559f18cff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f43a402a740 1 --2- 172.21.15.33:0/3106242670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18cf8be0 0x559f18cfecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f43a402a740 1 --2- 172.21.15.33:0/3106242670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 0x559f18cfe780 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f43a402a740 1 -- 172.21.15.33:0/3106242670 >> 172.21.15.33:0/3106242670 conn(0x559f18cf29f0 msgr2=0x559f18cf4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f43a402a740 1 -- 172.21.15.33:0/3106242670 shutdown_connections 2024-04-02T13:38:55.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f43a402a740 1 -- 172.21.15.33:0/3106242670 wait complete. 2024-04-02T13:38:55.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.202+0000 7f43a402a740 20 client.15684 trim_cache size 0 max 16384 2024-04-02T13:38:55.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.220+0000 7f43a402a740 1 Processor -- start 2024-04-02T13:38:55.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.220+0000 7f43a402a740 1 -- start start 2024-04-02T13:38:55.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f43a402a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 0x559f18d00eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f43a402a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf8be0 0x559f18d0d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f43a402a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18d017c0 0x559f18cfd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f43a402a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1b3ab0 con 0x559f18cf82c0 2024-04-02T13:38:55.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f43a402a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1b1380 con 0x559f18d017c0 2024-04-02T13:38:55.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f43a402a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1af1b0 con 0x559f18cf8be0 2024-04-02T13:38:55.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f439affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 0x559f18d00eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f439affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 0x559f18d00eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33356/0 (socket says 172.21.15.33:33356) 2024-04-02T13:38:55.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f439affd640 1 -- 172.21.15.33:0/2608106672 learned_addr learned my addr 172.21.15.33:0/2608106672 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:55.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f439a7fc640 1 --2- 172.21.15.33:0/2608106672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf8be0 0x559f18d0d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f439b7fe640 1 --2- 172.21.15.33:0/2608106672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18d017c0 0x559f18cfd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f439affd640 1 -- 172.21.15.33:0/2608106672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf8be0 msgr2=0x559f18d0d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f439affd640 1 --2- 172.21.15.33:0/2608106672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf8be0 0x559f18d0d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f439affd640 1 -- 172.21.15.33:0/2608106672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18d017c0 msgr2=0x559f18cfd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f439affd640 1 --2- 172.21.15.33:0/2608106672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18d017c0 0x559f18cfd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f439affd640 1 -- 172.21.15.33:0/2608106672 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f18cfa350 con 0x559f18cf82c0 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f439affd640 1 --2- 172.21.15.33:0/2608106672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 0x559f18d00eb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f438c012d30 tx=0x7f438c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd5de46f091ebf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.221+0000 7f4380ff9640 1 -- 172.21.15.33:0/2608106672 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c00fe80 con 0x559f18cf82c0 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.222+0000 7f4380ff9640 1 -- 172.21.15.33:0/2608106672 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f438c01cb50 con 0x559f18cf82c0 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.222+0000 7f4380ff9640 1 -- 172.21.15.33:0/2608106672 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c005540 con 0x559f18cf82c0 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.222+0000 7f43a402a740 1 -- 172.21.15.33:0/2608106672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 msgr2=0x559f18d00eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.222+0000 7f43a402a740 1 --2- 172.21.15.33:0/2608106672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 0x559f18d00eb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f438c012d30 tx=0x7f438c0096a0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.222+0000 7f43a402a740 1 -- 172.21.15.33:0/2608106672 shutdown_connections 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.222+0000 7f43a402a740 1 --2- 172.21.15.33:0/2608106672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18d017c0 0x559f18cfd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.222+0000 7f43a402a740 1 --2- 172.21.15.33:0/2608106672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf8be0 0x559f18d0d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.222+0000 7f43a402a740 1 --2- 172.21.15.33:0/2608106672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf82c0 0x559f18d00eb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.222+0000 7f43a402a740 1 -- 172.21.15.33:0/2608106672 >> 172.21.15.33:0/2608106672 conn(0x559f18cf29f0 msgr2=0x559f18cf1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.222+0000 7f43a402a740 1 -- 172.21.15.33:0/2608106672 shutdown_connections 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.222+0000 7f43a402a740 1 -- 172.21.15.33:0/2608106672 wait complete. 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f43a402a740 1 -- start start 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f43a402a740 1 Processor -- start 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f43a402a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf82c0 0x559f18d05620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f43a402a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf8be0 0x559f18d05b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f43a402a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18d017c0 0x559f18d76c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f43a402a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1af840 con 0x559f18cf8be0 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f43a402a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1b0cb0 con 0x559f18d017c0 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f43a402a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1aff10 con 0x559f18cf82c0 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f43a402a740 10 client.0 _ll_register_callbacks cb 0x7f438c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f439a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf8be0 0x559f18d05b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f439a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf8be0 0x559f18d05b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33366/0 (socket says 172.21.15.33:33366) 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f439a7fc640 1 -- 172.21.15.33:0/3558910728 learned_addr learned my addr 172.21.15.33:0/3558910728 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f439affd640 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf82c0 0x559f18d05620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f439a7fc640 1 -- 172.21.15.33:0/3558910728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf82c0 msgr2=0x559f18d05620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f439a7fc640 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf82c0 0x559f18d05620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f439b7fe640 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18d017c0 0x559f18d76c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f439a7fc640 1 -- 172.21.15.33:0/3558910728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18d017c0 msgr2=0x559f18d76c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f439a7fc640 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18d017c0 0x559f18d76c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f439a7fc640 1 -- 172.21.15.33:0/3558910728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f18d771a0 con 0x559f18cf8be0 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f439affd640 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf82c0 0x559f18d05620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f439a7fc640 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf8be0 0x559f18d05b60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f439002bc70 tx=0x7f43900f2850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6feab71d77355f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.223+0000 7f437eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f437eff5640 1 -- 172.21.15.33:0/3558910728 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900be780 con 0x559f18cf8be0 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f437eff5640 1 -- 172.21.15.33:0/3558910728 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f43900bc380 con 0x559f18cf8be0 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f43a402a740 1 -- 172.21.15.33:0/3558910728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f438c01cb50 con 0x559f18cf8be0 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f437eff5640 1 -- 172.21.15.33:0/3558910728 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f43900c2430 con 0x559f18cf8be0 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f43a402a740 10 client.15690 fetch_fsmap learned FSMap version 25 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f43a402a740 1 -- 172.21.15.33:0/3558910728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559f18d78480 con 0x559f18cf8be0 2024-04-02T13:38:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f437eff5640 1 -- 172.21.15.33:0/3558910728 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f438c01cb50 con 0x559f18cf8be0 2024-04-02T13:38:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f437eff5640 1 -- 172.21.15.33:0/3558910728 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f43900a3c70 con 0x559f18cf8be0 2024-04-02T13:38:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f43a402a740 10 client.15690 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:38:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f43a402a740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f43a402a740 20 client.15690 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f43a402a740 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f18d11710 0x559f18d7ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f43a402a740 4 client.15690 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:38:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f43a402a740 1 -- 172.21.15.33:0/3558910728 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f18c64000 con 0x559f18d11710 2024-04-02T13:38:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.224+0000 7f439affd640 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f18d11710 0x559f18d7ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.225+0000 7f439affd640 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f18d11710 0x559f18d7ac70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca18f72131d8b38 server_cookie=d49b7f0f86fb138d in_seq=0 out_seq=0 2024-04-02T13:38:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.225+0000 7f437eff5640 10 client.15690 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:55.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:54 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2827920431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:55.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:54 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/370329908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:55.234 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:54 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1261161985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f437eff5640 1 -- 172.21.15.33:0/3558910728 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f18c64000 con 0x559f18d11710 2024-04-02T13:38:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f437eff5640 10 client.15690 handle_command_reply: tid=0 2024-04-02T13:38:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f43a402a740 1 client.15690 shutdown 2024-04-02T13:38:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f43a402a740 1 -- 172.21.15.33:0/3558910728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf8be0 msgr2=0x559f18d05b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f43a402a740 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf8be0 0x559f18d05b60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f439002bc70 tx=0x7f43900f2850 comp rx=0 tx=0).stop 2024-04-02T13:38:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f43a402a740 1 -- 172.21.15.33:0/3558910728 shutdown_connections 2024-04-02T13:38:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f43a402a740 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f18d11710 0x559f18d7ac70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f43a402a740 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f18d017c0 0x559f18d76c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f43a402a740 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f18cf8be0 0x559f18d05b60 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f43a402a740 1 --2- 172.21.15.33:0/3558910728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f18cf82c0 0x559f18d05620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f43a402a740 1 -- 172.21.15.33:0/3558910728 >> 172.21.15.33:0/3558910728 conn(0x559f18cf29f0 msgr2=0x559f18cf1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f437eff5640 0 client.15690 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f43a402a740 1 -- 172.21.15.33:0/3558910728 shutdown_connections 2024-04-02T13:38:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.226+0000 7f43a402a740 1 -- 172.21.15.33:0/3558910728 wait complete. 2024-04-02T13:38:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.227+0000 7f43a402a740 20 client.15690 trim_cache size 0 max 16384 2024-04-02T13:38:55.237 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:38:55.237 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:38:55.237 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "84cad2fb-f96c-4155-a92b-c2a9747d034d", 2024-04-02T13:38:55.237 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:38:55.237 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:38:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.227+0000 7f43a1449640 1 -- 172.21.15.33:0/2042808876 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43840a3ec0 msgr2=0x7f43840a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.227+0000 7f43a1449640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43840a3ec0 0x7f43840a6380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4394005fd0 tx=0x7f4394013460 comp rx=0 tx=0).stop 2024-04-02T13:38:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.227+0000 7f43a1449640 1 -- 172.21.15.33:0/2042808876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f439c0696a0 msgr2=0x7f439c198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.227+0000 7f43a1449640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f439c0696a0 0x7f439c198940 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4390000c00 tx=0x7f4390004570 comp rx=0 tx=0).stop 2024-04-02T13:38:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.227+0000 7f43a1449640 1 -- 172.21.15.33:0/2042808876 shutdown_connections 2024-04-02T13:38:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.227+0000 7f43a1449640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43840a3ec0 0x7f43840a6380 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.227+0000 7f43a1449640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f439c107a80 0x7f439c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.227+0000 7f43a1449640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f439c0696a0 0x7f439c198940 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.227+0000 7f43a1449640 1 --2- 172.21.15.33:0/2042808876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f439c068ce0 0x7f439c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.227+0000 7f43a1449640 1 -- 172.21.15.33:0/2042808876 >> 172.21.15.33:0/2042808876 conn(0x7f439c077b00 msgr2=0x7f439c0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.227+0000 7f43a1449640 1 -- 172.21.15.33:0/2042808876 shutdown_connections 2024-04-02T13:38:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.227+0000 7f43a1449640 1 -- 172.21.15.33:0/2042808876 wait complete. 2024-04-02T13:38:55.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:38:55.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:38:55.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:54 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2827920431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:38:55.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:54 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/370329908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:38:55.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:54 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1261161985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:55.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.343+0000 7f9b3c9e2640 1 -- 172.21.15.33:0/3103196232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b38101330 msgr2=0x7f9b3810d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.343+0000 7f9b3c9e2640 1 --2- 172.21.15.33:0/3103196232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b38101330 0x7f9b3810d840 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009cd0 tx=0x7f9b3002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.343+0000 7f9b3c9e2640 1 -- 172.21.15.33:0/3103196232 shutdown_connections 2024-04-02T13:38:55.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.343+0000 7f9b3c9e2640 1 --2- 172.21.15.33:0/3103196232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b38101330 0x7f9b3810d840 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.343+0000 7f9b3c9e2640 1 --2- 172.21.15.33:0/3103196232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b38100970 0x7f9b38100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.343+0000 7f9b3c9e2640 1 --2- 172.21.15.33:0/3103196232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38106980 0x7f9b38106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.343+0000 7f9b3c9e2640 1 -- 172.21.15.33:0/3103196232 >> 172.21.15.33:0/3103196232 conn(0x7f9b38077b00 msgr2=0x7f9b38077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.343+0000 7f9b3c9e2640 1 -- 172.21.15.33:0/3103196232 shutdown_connections 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.343+0000 7f9b3c9e2640 1 -- 172.21.15.33:0/3103196232 wait complete. 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.344+0000 7f9b3c9e2640 1 Processor -- start 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.344+0000 7f9b3c9e2640 1 -- start start 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.344+0000 7f9b3c9e2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38100970 0x7f9b38102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.344+0000 7f9b3c9e2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b38101330 0x7f9b381033e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.344+0000 7f9b3c9e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b38106980 0x7f9b38103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.344+0000 7f9b3c9e2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38110d80 con 0x7f9b38106980 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.344+0000 7f9b3c9e2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38110c00 con 0x7f9b38101330 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.344+0000 7f9b3c9e2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38110f00 con 0x7f9b38100970 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.344+0000 7f9b36d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b38106980 0x7f9b38103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.344+0000 7f9b36d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b38106980 0x7f9b38103920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33396/0 (socket says 172.21.15.33:33396) 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.344+0000 7f9b36d9d640 1 -- 172.21.15.33:0/2584630663 learned_addr learned my addr 172.21.15.33:0/2584630663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b3659c640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38100970 0x7f9b38102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b38101330 0x7f9b381033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b36d9d640 1 -- 172.21.15.33:0/2584630663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38100970 msgr2=0x7f9b38102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b36d9d640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38100970 0x7f9b38102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b36d9d640 1 -- 172.21.15.33:0/2584630663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b38101330 msgr2=0x7f9b381033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b36d9d640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b38101330 0x7f9b381033e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b36d9d640 1 -- 172.21.15.33:0/2584630663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b381a68b0 con 0x7f9b38106980 2024-04-02T13:38:55.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b38101330 0x7f9b381033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:55.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b36d9d640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b38106980 0x7f9b38103920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9b30037040 tx=0x7f9b30034cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b2f7fe640 1 -- 172.21.15.33:0/2584630663 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30039050 con 0x7f9b38106980 2024-04-02T13:38:55.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b3c9e2640 1 -- 172.21.15.33:0/2584630663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b381a6b40 con 0x7f9b38106980 2024-04-02T13:38:55.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b2f7fe640 1 -- 172.21.15.33:0/2584630663 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9b30045070 con 0x7f9b38106980 2024-04-02T13:38:55.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b2f7fe640 1 -- 172.21.15.33:0/2584630663 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b300046f0 con 0x7f9b38106980 2024-04-02T13:38:55.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.345+0000 7f9b3c9e2640 1 -- 172.21.15.33:0/2584630663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b381a7080 con 0x7f9b38106980 2024-04-02T13:38:55.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.347+0000 7f9b2f7fe640 1 -- 172.21.15.33:0/2584630663 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f9b30003970 con 0x7f9b38106980 2024-04-02T13:38:55.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.347+0000 7f9b3f5c7740 1 Processor -- start 2024-04-02T13:38:55.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3f5c7740 1 -- start start 2024-04-02T13:38:55.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3f5c7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5d8d0 0x55dfaef5dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3f5c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef66890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3f5c7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 0x55dfaef691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3f5c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dfaeec1830 con 0x55dfaef5e1f0 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3f5c7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55dfaee69820 con 0x55dfaef66dd0 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3f5c7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55dfaeebd7e0 con 0x55dfaef5d8d0 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef66890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef66890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33400/0 (socket says 172.21.15.33:33400) 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3659c640 1 -- 172.21.15.33:0/1564011550 learned_addr learned my addr 172.21.15.33:0/1564011550 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b2f7fe640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9b240a3e40 0x7f9b240a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/1564011550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5d8d0 0x55dfaef5dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/1564011550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 0x55dfaef691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3659c640 1 -- 172.21.15.33:0/1564011550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5d8d0 msgr2=0x55dfaef5dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3659c640 1 --2- 172.21.15.33:0/1564011550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5d8d0 0x55dfaef5dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3659c640 1 -- 172.21.15.33:0/1564011550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 msgr2=0x55dfaef691c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3659c640 1 --2- 172.21.15.33:0/1564011550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 0x55dfaef691c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b3659c640 1 -- 172.21.15.33:0/1564011550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfaec9f7d0 con 0x55dfaef5e1f0 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b36d9d640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9b240a3e40 0x7f9b240a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/1564011550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 0x55dfaef691c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:55.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.348+0000 7f9b2f7fe640 1 -- 172.21.15.33:0/2584630663 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9b300f38d0 con 0x7f9b38106980 2024-04-02T13:38:55.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.349+0000 7f9b3659c640 1 --2- 172.21.15.33:0/1564011550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef66890 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9b28009a70 tx=0x7f9b28009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c902d2e88ce4714 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.349+0000 7f9b2cff9640 1 -- 172.21.15.33:0/1564011550 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28017070 con 0x55dfaef5e1f0 2024-04-02T13:38:55.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.349+0000 7f9b36d9d640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9b240a3e40 0x7f9b240a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b300f0ca0 tx=0x7f9b300bb040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.349+0000 7f9b2cff9640 1 -- 172.21.15.33:0/1564011550 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9b28013470 con 0x55dfaef5e1f0 2024-04-02T13:38:55.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.349+0000 7f9b2cff9640 1 -- 172.21.15.33:0/1564011550 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28012630 con 0x55dfaef5e1f0 2024-04-02T13:38:55.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/1564011550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 msgr2=0x55dfaef66890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/1564011550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef66890 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9b28009a70 tx=0x7f9b28009f30 comp rx=0 tx=0).stop 2024-04-02T13:38:55.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/1564011550 shutdown_connections 2024-04-02T13:38:55.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/1564011550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 0x55dfaef691c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/1564011550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef66890 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/1564011550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5d8d0 0x55dfaef5dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/1564011550 >> 172.21.15.33:0/1564011550 conn(0x55dfaef58000 msgr2=0x55dfaef583e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/1564011550 shutdown_connections 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/1564011550 wait complete. 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 -- start start 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 Processor -- start 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5d8d0 0x55dfaef666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef63a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 0x55dfaef63fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dfaef69bc0 con 0x55dfaef5e1f0 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55dfaee7cd30 con 0x55dfaef66dd0 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55dfaef69d40 con 0x55dfaef5d8d0 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3f5c7740 10 client.0 _ll_register_callbacks cb 0x55dfaee7c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.350+0000 7f9b3659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef63a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b3659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef63a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33412/0 (socket says 172.21.15.33:33412) 2024-04-02T13:38:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b3659c640 1 -- 172.21.15.33:0/3528944160 learned_addr learned my addr 172.21.15.33:0/3528944160 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5d8d0 0x55dfaef666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 0x55dfaef63fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b3659c640 1 -- 172.21.15.33:0/3528944160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5d8d0 msgr2=0x55dfaef666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b3659c640 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5d8d0 0x55dfaef666e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b3659c640 1 -- 172.21.15.33:0/3528944160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 msgr2=0x55dfaef63fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b3659c640 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 0x55dfaef63fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b3659c640 1 -- 172.21.15.33:0/3528944160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfaebd53f0 con 0x55dfaef5e1f0 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5d8d0 0x55dfaef666e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b3659c640 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef63a70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9b28009330 tx=0x7f9b28013e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4093cf4d4dcf9c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b1dffb640 1 -- 172.21.15.33:0/3528944160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28029060 con 0x55dfaef5e1f0 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b1dffb640 1 -- 172.21.15.33:0/3528944160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9b280139b0 con 0x55dfaef5e1f0 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/3528944160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b28013470 con 0x55dfaef5e1f0 2024-04-02T13:38:55.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b1dffb640 1 -- 172.21.15.33:0/3528944160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b28030050 con 0x55dfaef5e1f0 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b3f5c7740 10 client.15702 fetch_fsmap learned FSMap version 25 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.351+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/3528944160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfaeb950f0 con 0x55dfaef5e1f0 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.352+0000 7f9b1dffb640 1 -- 172.21.15.33:0/3528944160 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9b28013470 con 0x55dfaef5e1f0 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.352+0000 7f9b1dffb640 1 -- 172.21.15.33:0/3528944160 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9b280245b0 con 0x55dfaef5e1f0 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.352+0000 7f9b3f5c7740 10 client.15702 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.352+0000 7f9b3f5c7740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.352+0000 7f9b3f5c7740 20 client.15702 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.352+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dfaef76c00 0x55dfaef78ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.352+0000 7f9b3f5c7740 4 client.15702 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.352+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/3528944160 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfaeeced70 con 0x55dfaef76c00 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.352+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dfaef76c00 0x55dfaef78ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b1dffb640 10 client.15702 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dfaef76c00 0x55dfaef78ff0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d78c89dd39544be server_cookie=197db31930fc2dac in_seq=0 out_seq=0 2024-04-02T13:38:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b1dffb640 1 -- 172.21.15.33:0/3528944160 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfaeeced70 con 0x55dfaef76c00 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b1dffb640 10 client.15702 handle_command_reply: tid=0 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b3f5c7740 1 client.15702 shutdown 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/3528944160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 msgr2=0x55dfaef63a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef63a70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9b28009330 tx=0x7f9b28013e70 comp rx=0 tx=0).stop 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/3528944160 shutdown_connections 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dfaef76c00 0x55dfaef78ff0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 0x55dfaef63fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef63a70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b1dffb640 0 client.15702 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/3528944160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5d8d0 0x55dfaef666e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/3528944160 >> 172.21.15.33:0/3528944160 conn(0x55dfaef58000 msgr2=0x55dfaef59d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/3528944160 shutdown_connections 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.353+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/3528944160 wait complete. 2024-04-02T13:38:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.354+0000 7f9b3f5c7740 20 client.15702 trim_cache size 0 max 16384 2024-04-02T13:38:55.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.371+0000 7f9b3f5c7740 1 Processor -- start 2024-04-02T13:38:55.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.371+0000 7f9b3f5c7740 1 -- start start 2024-04-02T13:38:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.371+0000 7f9b3f5c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5d8d0 0x55dfaef59b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.371+0000 7f9b3f5c7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5e1f0 0x55dfaef661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.371+0000 7f9b3f5c7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 0x55dfaef666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.371+0000 7f9b3f5c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b381b3940 con 0x55dfaef5d8d0 2024-04-02T13:38:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.371+0000 7f9b3f5c7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9b381b1210 con 0x55dfaef66dd0 2024-04-02T13:38:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.371+0000 7f9b3f5c7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9b381af040 con 0x55dfaef5e1f0 2024-04-02T13:38:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.371+0000 7f9b2ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5d8d0 0x55dfaef59b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.371+0000 7f9b2ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5d8d0 0x55dfaef59b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33416/0 (socket says 172.21.15.33:33416) 2024-04-02T13:38:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.371+0000 7f9b2ffff640 1 -- 172.21.15.33:0/4204380050 learned_addr learned my addr 172.21.15.33:0/4204380050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.371+0000 7f9b3659c640 1 --2- 172.21.15.33:0/4204380050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5e1f0 0x55dfaef661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.372+0000 7f9b2ffff640 1 -- 172.21.15.33:0/4204380050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5e1f0 msgr2=0x55dfaef661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.372+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/4204380050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5e1f0 0x55dfaef661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.372+0000 7f9b2ffff640 1 -- 172.21.15.33:0/4204380050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 msgr2=0x55dfaef666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.372+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/4204380050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 0x55dfaef666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.372+0000 7f9b2ffff640 1 -- 172.21.15.33:0/4204380050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfaef59230 con 0x55dfaef5d8d0 2024-04-02T13:38:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.372+0000 7f9b3659c640 1 --2- 172.21.15.33:0/4204380050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5e1f0 0x55dfaef661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.372+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/4204380050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5d8d0 0x55dfaef59b40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000b370 tx=0x7f9b20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9cf124fec455c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.372+0000 7f9b1ffff640 1 -- 172.21.15.33:0/4204380050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20013390 con 0x55dfaef5d8d0 2024-04-02T13:38:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.372+0000 7f9b1ffff640 1 -- 172.21.15.33:0/4204380050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9b2001f900 con 0x55dfaef5d8d0 2024-04-02T13:38:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.372+0000 7f9b1ffff640 1 -- 172.21.15.33:0/4204380050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b200107b0 con 0x55dfaef5d8d0 2024-04-02T13:38:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/4204380050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5d8d0 msgr2=0x55dfaef59b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/4204380050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5d8d0 0x55dfaef59b40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000b370 tx=0x7f9b20004730 comp rx=0 tx=0).stop 2024-04-02T13:38:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/4204380050 shutdown_connections 2024-04-02T13:38:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/4204380050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef66dd0 0x55dfaef666e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/4204380050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef5e1f0 0x55dfaef661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/4204380050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5d8d0 0x55dfaef59b40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/4204380050 >> 172.21.15.33:0/4204380050 conn(0x55dfaef58000 msgr2=0x55dfaef59430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/4204380050 shutdown_connections 2024-04-02T13:38:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/4204380050 wait complete. 2024-04-02T13:38:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 -- start start 2024-04-02T13:38:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 Processor -- start 2024-04-02T13:38:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef5d8d0 0x55dfaef60b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef61060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef66dd0 0x55dfaefdbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b381af6d0 con 0x55dfaef5e1f0 2024-04-02T13:38:55.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9b381b0b40 con 0x55dfaef5d8d0 2024-04-02T13:38:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9b381afda0 con 0x55dfaef66dd0 2024-04-02T13:38:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.373+0000 7f9b3f5c7740 10 client.0 _ll_register_callbacks cb 0x7f9b28013d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b3659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef61060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b3659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef61060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33420/0 (socket says 172.21.15.33:33420) 2024-04-02T13:38:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b3659c640 1 -- 172.21.15.33:0/2670086781 learned_addr learned my addr 172.21.15.33:0/2670086781 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/2670086781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef5d8d0 0x55dfaef60b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/2670086781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef66dd0 0x55dfaefdbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b3659c640 1 -- 172.21.15.33:0/2670086781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef66dd0 msgr2=0x55dfaefdbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b3659c640 1 --2- 172.21.15.33:0/2670086781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef66dd0 0x55dfaefdbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b3659c640 1 -- 172.21.15.33:0/2670086781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef5d8d0 msgr2=0x55dfaef60b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b3659c640 1 --2- 172.21.15.33:0/2670086781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef5d8d0 0x55dfaef60b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b3659c640 1 -- 172.21.15.33:0/2670086781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfaef79450 con 0x55dfaef5e1f0 2024-04-02T13:38:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b3659c640 1 --2- 172.21.15.33:0/2670086781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef61060 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55dfaef65040 tx=0x7f9b28009180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2001065ec68ed6f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:38:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b1dffb640 1 -- 172.21.15.33:0/2670086781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28029070 con 0x55dfaef5e1f0 2024-04-02T13:38:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b1dffb640 1 -- 172.21.15.33:0/2670086781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9b280243f0 con 0x55dfaef5e1f0 2024-04-02T13:38:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/2670086781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b2001f900 con 0x55dfaef5e1f0 2024-04-02T13:38:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b1dffb640 1 -- 172.21.15.33:0/2670086781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b2803d050 con 0x55dfaef5e1f0 2024-04-02T13:38:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b3f5c7740 10 client.15708 fetch_fsmap learned FSMap version 25 2024-04-02T13:38:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.374+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/2670086781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfaef57e30 con 0x55dfaef5e1f0 2024-04-02T13:38:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.375+0000 7f9b1dffb640 1 -- 172.21.15.33:0/2670086781 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9b2001f900 con 0x55dfaef5e1f0 2024-04-02T13:38:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.375+0000 7f9b1dffb640 1 -- 172.21.15.33:0/2670086781 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9b2801ca40 con 0x55dfaef5e1f0 2024-04-02T13:38:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.375+0000 7f9b3f5c7740 10 client.15708 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:38:55.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.375+0000 7f9b3f5c7740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.375+0000 7f9b3f5c7740 20 client.15708 populate_metadata read hostname 'smithi033' 2024-04-02T13:38:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.375+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/2670086781 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dfaefdd8e0 0x55dfaefdfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.375+0000 7f9b3f5c7740 4 client.15708 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:38:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.375+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/2670086781 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dfaeec90c0 con 0x55dfaefdd8e0 2024-04-02T13:38:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.375+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/2670086781 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dfaefdd8e0 0x55dfaefdfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.375+0000 7f9b1dffb640 10 client.15708 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.375+0000 7f9b2ffff640 1 --2- 172.21.15.33:0/2670086781 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dfaefdd8e0 0x55dfaefdfd00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b77daf195af17347 server_cookie=6abb237c9d42ad36 in_seq=0 out_seq=0 2024-04-02T13:38:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b1dffb640 1 -- 172.21.15.33:0/2670086781 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dfaeec90c0 con 0x55dfaefdd8e0 2024-04-02T13:38:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b1dffb640 10 client.15708 handle_command_reply: tid=0 2024-04-02T13:38:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3f5c7740 1 client.15708 shutdown 2024-04-02T13:38:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/2670086781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 msgr2=0x55dfaef61060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/2670086781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef61060 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55dfaef65040 tx=0x7f9b28009180 comp rx=0 tx=0).stop 2024-04-02T13:38:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/2670086781 shutdown_connections 2024-04-02T13:38:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/2670086781 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dfaefdd8e0 0x55dfaefdfd00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/2670086781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dfaef66dd0 0x55dfaefdbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/2670086781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dfaef5e1f0 0x55dfaef61060 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b1dffb640 0 client.15708 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:38:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3f5c7740 1 --2- 172.21.15.33:0/2670086781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfaef5d8d0 0x55dfaef60b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/2670086781 >> 172.21.15.33:0/2670086781 conn(0x55dfaef58000 msgr2=0x55dfaef57240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/2670086781 shutdown_connections 2024-04-02T13:38:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3f5c7740 1 -- 172.21.15.33:0/2670086781 wait complete. 2024-04-02T13:38:55.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3f5c7740 20 client.15708 trim_cache size 0 max 16384 2024-04-02T13:38:55.387 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:38:55.387 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:38:55.387 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:38:55.387 INFO:teuthology.orchestra.run.smithi033.stdout: "84cad2fb-f96c-4155-a92b-c2a9747d034d": { 2024-04-02T13:38:55.387 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:38:55.387 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "84cad2fb-f96c-4155-a92b-c2a9747d034d", 2024-04-02T13:38:55.387 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:38:55.388 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:38:55.388 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:38:55.388 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:38:55.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3c9e2640 1 -- 172.21.15.33:0/2584630663 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9b240a3e40 msgr2=0x7f9b240a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3c9e2640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9b240a3e40 0x7f9b240a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b300f0ca0 tx=0x7f9b300bb040 comp rx=0 tx=0).stop 2024-04-02T13:38:55.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3c9e2640 1 -- 172.21.15.33:0/2584630663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b38106980 msgr2=0x7f9b38103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3c9e2640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b38106980 0x7f9b38103920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9b30037040 tx=0x7f9b30034cf0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3c9e2640 1 -- 172.21.15.33:0/2584630663 shutdown_connections 2024-04-02T13:38:55.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3c9e2640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9b240a3e40 0x7f9b240a6300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3c9e2640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b38106980 0x7f9b38103920 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3c9e2640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9b38101330 0x7f9b381033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3c9e2640 1 --2- 172.21.15.33:0/2584630663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b38100970 0x7f9b38102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.376+0000 7f9b3c9e2640 1 -- 172.21.15.33:0/2584630663 >> 172.21.15.33:0/2584630663 conn(0x7f9b38077b00 msgr2=0x7f9b38068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.377+0000 7f9b3c9e2640 1 -- 172.21.15.33:0/2584630663 shutdown_connections 2024-04-02T13:38:55.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.377+0000 7f9b3c9e2640 1 -- 172.21.15.33:0/2584630663 wait complete. 2024-04-02T13:38:55.391 INFO:tasks.cephfs.filesystem:scrub status for tag:84cad2fb-f96c-4155-a92b-c2a9747d034d - {'path': '/', 'tag': '84cad2fb-f96c-4155-a92b-c2a9747d034d', 'options': 'recursive,force'} 2024-04-02T13:38:55.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.414+0000 7feec9029640 1 -- 172.21.15.33:0/1232724474 --> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7feec40ff230 con 0x7feeac0a3f60 2024-04-02T13:38:55.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.480+0000 7feeabfff640 1 -- 172.21.15.33:0/1232724474 <== mgr.15456 v2:172.21.15.33:6800/619073725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7feec40ff230 con 0x7feeac0a3f60 2024-04-02T13:38:55.483 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path / 2024-04-02T13:38:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.482+0000 7feec9029640 1 -- 172.21.15.33:0/1232724474 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7feeac0a3f60 msgr2=0x7feeac0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.482+0000 7feec9029640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7feeac0a3f60 0x7feeac0a6420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feeb4007d50 tx=0x7feeb4007730 comp rx=0 tx=0).stop 2024-04-02T13:38:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.482+0000 7feec9029640 1 -- 172.21.15.33:0/1232724474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feec41059a0 msgr2=0x7feec41a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.482+0000 7feec9029640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feec41059a0 0x7feec41a4140 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7feebc005ab0 tx=0x7feebc0043b0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.482+0000 7feec9029640 1 -- 172.21.15.33:0/1232724474 shutdown_connections 2024-04-02T13:38:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.482+0000 7feec9029640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7feeac0a3f60 0x7feeac0a6420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.482+0000 7feec9029640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feec41059a0 0x7feec41a4140 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.482+0000 7feec9029640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feec4101610 0x7feec4199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.482+0000 7feec9029640 1 --2- 172.21.15.33:0/1232724474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feec4100c50 0x7feec41990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.482+0000 7feec9029640 1 -- 172.21.15.33:0/1232724474 >> 172.21.15.33:0/1232724474 conn(0x7feec4077850 msgr2=0x7feec40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.482+0000 7feec9029640 1 -- 172.21.15.33:0/1232724474 shutdown_connections 2024-04-02T13:38:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.482+0000 7feec9029640 1 -- 172.21.15.33:0/1232724474 wait complete. 2024-04-02T13:38:55.495 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-02T13:38:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.639+0000 7f899d0a5640 1 -- 172.21.15.33:0/141547019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89981059a0 msgr2=0x7f8998105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.639+0000 7f899d0a5640 1 --2- 172.21.15.33:0/141547019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89981059a0 0x7f8998105d80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8990009900 tx=0x7f899002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.639+0000 7f899d0a5640 1 -- 172.21.15.33:0/141547019 shutdown_connections 2024-04-02T13:38:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.639+0000 7f899d0a5640 1 --2- 172.21.15.33:0/141547019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8998101610 0x7f899810d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.639+0000 7f899d0a5640 1 --2- 172.21.15.33:0/141547019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8998100c50 0x7f89981010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.639+0000 7f899d0a5640 1 --2- 172.21.15.33:0/141547019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89981059a0 0x7f8998105d80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.639+0000 7f899d0a5640 1 -- 172.21.15.33:0/141547019 >> 172.21.15.33:0/141547019 conn(0x7f8998077850 msgr2=0x7f8998077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.639+0000 7f899d0a5640 1 -- 172.21.15.33:0/141547019 shutdown_connections 2024-04-02T13:38:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.639+0000 7f899d0a5640 1 -- 172.21.15.33:0/141547019 wait complete. 2024-04-02T13:38:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.639+0000 7f899d0a5640 1 Processor -- start 2024-04-02T13:38:55.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899d0a5640 1 -- start start 2024-04-02T13:38:55.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899d0a5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8998100c50 0x7f89981983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899d0a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8998101610 0x7f8998198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899d0a5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89981059a0 0x7f899819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899d0a5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8998110e90 con 0x7f8998101610 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899d0a5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8998110d10 con 0x7f8998100c50 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899d0a5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8998111010 con 0x7f89981059a0 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8998101610 0x7f8998198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8998101610 0x7f8998198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33442/0 (socket says 172.21.15.33:33442) 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899659c640 1 -- 172.21.15.33:0/296599630 learned_addr learned my addr 172.21.15.33:0/296599630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899759e640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89981059a0 0x7f899819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f8996d9d640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8998100c50 0x7f89981983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899659c640 1 -- 172.21.15.33:0/296599630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89981059a0 msgr2=0x7f899819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899659c640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89981059a0 0x7f899819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899659c640 1 -- 172.21.15.33:0/296599630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8998100c50 msgr2=0x7f89981983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899659c640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8998100c50 0x7f89981983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899659c640 1 -- 172.21.15.33:0/296599630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899819d3c0 con 0x7f8998101610 2024-04-02T13:38:55.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f8996d9d640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8998100c50 0x7f89981983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:55.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f899659c640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8998101610 0x7f8998198910 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f898c00d770 tx=0x7f898c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.640+0000 7f897ffff640 1 -- 172.21.15.33:0/296599630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c017070 con 0x7f8998101610 2024-04-02T13:38:55.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.641+0000 7f899d0a5640 1 -- 172.21.15.33:0/296599630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899819d6b0 con 0x7f8998101610 2024-04-02T13:38:55.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.641+0000 7f897ffff640 1 -- 172.21.15.33:0/296599630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f898c0054f0 con 0x7f8998101610 2024-04-02T13:38:55.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.641+0000 7f897ffff640 1 -- 172.21.15.33:0/296599630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c004d60 con 0x7f8998101610 2024-04-02T13:38:55.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.641+0000 7f899d0a5640 1 -- 172.21.15.33:0/296599630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89981a4f40 con 0x7f8998101610 2024-04-02T13:38:55.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.642+0000 7f899d0a5640 1 -- 172.21.15.33:0/296599630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899804f860 con 0x7f8998101610 2024-04-02T13:38:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.643+0000 7f897ffff640 1 -- 172.21.15.33:0/296599630 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f898c004f00 con 0x7f8998101610 2024-04-02T13:38:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.643+0000 7f897ffff640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f89800a3ee0 0x7f89800a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.643+0000 7f8996d9d640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f89800a3ee0 0x7f89800a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.643+0000 7f897ffff640 1 -- 172.21.15.33:0/296599630 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f898c0d08d0 con 0x7f8998101610 2024-04-02T13:38:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.644+0000 7f8996d9d640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f89800a3ee0 0x7f89800a63a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8990036040 tx=0x7f8990005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:55.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.648+0000 7f897ffff640 1 -- 172.21.15.33:0/296599630 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f898c09c9b0 con 0x7f8998101610 2024-04-02T13:38:55.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.859+0000 7f899d0a5640 1 -- 172.21.15.33:0/296599630 --> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f8998061fe0 con 0x7f89800a3ee0 2024-04-02T13:38:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.922+0000 7f897ffff640 1 -- 172.21.15.33:0/296599630 <== mgr.15456 v2:172.21.15.33:6800/619073725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f8998061fe0 con 0x7f89800a3ee0 2024-04-02T13:38:55.925 INFO:teuthology.orchestra.run.smithi033.stdout:Retention added to path / 2024-04-02T13:38:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.924+0000 7f899d0a5640 1 -- 172.21.15.33:0/296599630 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f89800a3ee0 msgr2=0x7f89800a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.924+0000 7f899d0a5640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f89800a3ee0 0x7f89800a63a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8990036040 tx=0x7f8990005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.924+0000 7f899d0a5640 1 -- 172.21.15.33:0/296599630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8998101610 msgr2=0x7f8998198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.924+0000 7f899d0a5640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8998101610 0x7f8998198910 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f898c00d770 tx=0x7f898c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:38:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.924+0000 7f899d0a5640 1 -- 172.21.15.33:0/296599630 shutdown_connections 2024-04-02T13:38:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.924+0000 7f899d0a5640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f89800a3ee0 0x7f89800a63a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.924+0000 7f899d0a5640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89981059a0 0x7f899819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.924+0000 7f899d0a5640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8998101610 0x7f8998198910 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.924+0000 7f899d0a5640 1 --2- 172.21.15.33:0/296599630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8998100c50 0x7f89981983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:55.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.924+0000 7f899d0a5640 1 -- 172.21.15.33:0/296599630 >> 172.21.15.33:0/296599630 conn(0x7f8998077850 msgr2=0x7f89980693c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:55.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.924+0000 7f899d0a5640 1 -- 172.21.15.33:0/296599630 shutdown_connections 2024-04-02T13:38:55.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:55.925+0000 7f899d0a5640 1 -- 172.21.15.33:0/296599630 wait complete. 2024-04-02T13:38:55.938 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-02T13:38:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.038+0000 7f5aad58f640 1 -- 172.21.15.33:0/2361141931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aa8100c50 msgr2=0x7f5aa81010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.038+0000 7f5aad58f640 1 --2- 172.21.15.33:0/2361141931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aa8100c50 0x7f5aa81010b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009900 tx=0x7f5a9c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.038+0000 7f5aad58f640 1 -- 172.21.15.33:0/2361141931 shutdown_connections 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.038+0000 7f5aad58f640 1 --2- 172.21.15.33:0/2361141931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5aa81015f0 0x7f5aa810d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.038+0000 7f5aad58f640 1 --2- 172.21.15.33:0/2361141931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aa8100c50 0x7f5aa81010b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.038+0000 7f5aad58f640 1 --2- 172.21.15.33:0/2361141931 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5aa8105980 0x7f5aa8105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.038+0000 7f5aad58f640 1 -- 172.21.15.33:0/2361141931 >> 172.21.15.33:0/2361141931 conn(0x7f5aa80777d0 msgr2=0x7f5aa8077be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.038+0000 7f5aad58f640 1 -- 172.21.15.33:0/2361141931 shutdown_connections 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.038+0000 7f5aad58f640 1 -- 172.21.15.33:0/2361141931 wait complete. 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.039+0000 7f5aad58f640 1 Processor -- start 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.039+0000 7f5aad58f640 1 -- start start 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.039+0000 7f5aad58f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aa8100c50 0x7f5aa8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.039+0000 7f5aad58f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5aa81015f0 0x7f5aa8199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.039+0000 7f5aad58f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5aa8105980 0x7f5aa81a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.039+0000 7f5aad58f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa8110e60 con 0x7f5aa8100c50 2024-04-02T13:38:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.039+0000 7f5aad58f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa8110ce0 con 0x7f5aa81015f0 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.039+0000 7f5aad58f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa8110fe0 con 0x7f5aa8105980 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.039+0000 7f5aa6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aa8100c50 0x7f5aa8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.039+0000 7f5aa6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aa8100c50 0x7f5aa8199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33462/0 (socket says 172.21.15.33:33462) 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.039+0000 7f5aa6ffd640 1 -- 172.21.15.33:0/2694323066 learned_addr learned my addr 172.21.15.33:0/2694323066 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.039+0000 7f5aa67fc640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5aa81015f0 0x7f5aa8199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.040+0000 7f5aa77fe640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5aa8105980 0x7f5aa81a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.040+0000 7f5aa6ffd640 1 -- 172.21.15.33:0/2694323066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5aa8105980 msgr2=0x7f5aa81a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.040+0000 7f5aa6ffd640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5aa8105980 0x7f5aa81a41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.040+0000 7f5aa6ffd640 1 -- 172.21.15.33:0/2694323066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5aa81015f0 msgr2=0x7f5aa8199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.040+0000 7f5aa6ffd640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5aa81015f0 0x7f5aa8199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.040+0000 7f5aa6ffd640 1 -- 172.21.15.33:0/2694323066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa81a46e0 con 0x7f5aa8100c50 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.040+0000 7f5aa67fc640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5aa81015f0 0x7f5aa8199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.040+0000 7f5aa6ffd640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aa8100c50 0x7f5aa8199120 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000cce0 tx=0x7f5aa000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.040+0000 7f5a8ffff640 1 -- 172.21.15.33:0/2694323066 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa000f650 con 0x7f5aa8100c50 2024-04-02T13:38:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.040+0000 7f5a8ffff640 1 -- 172.21.15.33:0/2694323066 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5aa000fcc0 con 0x7f5aa8100c50 2024-04-02T13:38:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.040+0000 7f5aad58f640 1 -- 172.21.15.33:0/2694323066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa81a48b0 con 0x7f5aa8100c50 2024-04-02T13:38:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.040+0000 7f5a8ffff640 1 -- 172.21.15.33:0/2694323066 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0018be0 con 0x7f5aa8100c50 2024-04-02T13:38:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.040+0000 7f5aad58f640 1 -- 172.21.15.33:0/2694323066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa81a4d70 con 0x7f5aa8100c50 2024-04-02T13:38:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.041+0000 7f5aad58f640 1 -- 172.21.15.33:0/2694323066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa8102110 con 0x7f5aa8100c50 2024-04-02T13:38:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.042+0000 7f5a8ffff640 1 -- 172.21.15.33:0/2694323066 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5aa000f7f0 con 0x7f5aa8100c50 2024-04-02T13:38:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.042+0000 7f5a8ffff640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5a900a3ee0 0x7f5a900a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.043+0000 7f5a8ffff640 1 -- 172.21.15.33:0/2694323066 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5aa00d1310 con 0x7f5aa8100c50 2024-04-02T13:38:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.043+0000 7f5aa67fc640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5a900a3ee0 0x7f5a900a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.043+0000 7f5aa67fc640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5a900a3ee0 0x7f5a900a63a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5aa806ae00 tx=0x7f5a9c000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:56.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.047+0000 7f5a8ffff640 1 -- 172.21.15.33:0/2694323066 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5aa0014030 con 0x7f5aa8100c50 2024-04-02T13:38:56.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:55 smithi145 ceph-mon[35646]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 23 op/s 2024-04-02T13:38:56.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:55 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2534116537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:56.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:55 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:38:56.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:55 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:56.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:55 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:38:56.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:55 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:56.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:55 smithi145 ceph-mon[35646]: from='client.15675 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:56.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:55 smithi050 ceph-mon[35524]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 23 op/s 2024-04-02T13:38:56.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:55 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2534116537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:56.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:55 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:38:56.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:55 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:56.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:55 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:38:56.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:55 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:56.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:55 smithi050 ceph-mon[35524]: from='client.15675 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:56.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:55 smithi033 ceph-mon[32390]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 23 op/s 2024-04-02T13:38:56.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:55 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2534116537' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:38:56.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:55 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:38:56.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:55 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:56.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:55 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:38:56.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:55 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:56.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:55 smithi033 ceph-mon[32390]: from='client.15675 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.261+0000 7f5aad58f640 1 -- 172.21.15.33:0/2694323066 --> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f5aa80fed40 con 0x7f5a900a3ee0 2024-04-02T13:38:56.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.268+0000 7f5a8ffff640 1 -- 172.21.15.33:0/2694323066 <== mgr.15456 v2:172.21.15.33:6800/619073725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f5aa80fed40 con 0x7f5a900a3ee0 2024-04-02T13:38:56.270 INFO:teuthology.orchestra.run.smithi033.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-02T00:00:00", "created": "2024-04-02T13:38:55", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-02T13:38:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.270+0000 7f5aad58f640 1 -- 172.21.15.33:0/2694323066 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5a900a3ee0 msgr2=0x7f5a900a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.270+0000 7f5aad58f640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5a900a3ee0 0x7f5a900a63a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5aa806ae00 tx=0x7f5a9c000bc0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.270+0000 7f5aad58f640 1 -- 172.21.15.33:0/2694323066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aa8100c50 msgr2=0x7f5aa8199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.270+0000 7f5aad58f640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aa8100c50 0x7f5aa8199120 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000cce0 tx=0x7f5aa000a620 comp rx=0 tx=0).stop 2024-04-02T13:38:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.270+0000 7f5aad58f640 1 -- 172.21.15.33:0/2694323066 shutdown_connections 2024-04-02T13:38:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.270+0000 7f5aad58f640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5a900a3ee0 0x7f5a900a63a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.270+0000 7f5aad58f640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5aa8105980 0x7f5aa81a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.270+0000 7f5aad58f640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5aa81015f0 0x7f5aa8199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.270+0000 7f5aad58f640 1 --2- 172.21.15.33:0/2694323066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5aa8100c50 0x7f5aa8199120 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.270+0000 7f5aad58f640 1 -- 172.21.15.33:0/2694323066 >> 172.21.15.33:0/2694323066 conn(0x7f5aa80777d0 msgr2=0x7f5aa80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.270+0000 7f5aad58f640 1 -- 172.21.15.33:0/2694323066 shutdown_connections 2024-04-02T13:38:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.270+0000 7f5aad58f640 1 -- 172.21.15.33:0/2694323066 wait complete. 2024-04-02T13:38:56.283 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-02T13:38:56.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.385+0000 7f6b790f2640 1 -- 172.21.15.33:0/1157199613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b741009d0 msgr2=0x7f6b74100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:56.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.385+0000 7f6b790f2640 1 --2- 172.21.15.33:0/1157199613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b741009d0 0x7f6b74100e50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009930 tx=0x7f6b5c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:56.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.385+0000 7f6b790f2640 1 -- 172.21.15.33:0/1157199613 shutdown_connections 2024-04-02T13:38:56.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.385+0000 7f6b790f2640 1 --2- 172.21.15.33:0/1157199613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b74101390 0x7f6b7410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.385+0000 7f6b790f2640 1 --2- 172.21.15.33:0/1157199613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b741009d0 0x7f6b74100e50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.385+0000 7f6b790f2640 1 --2- 172.21.15.33:0/1157199613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b741069e0 0x7f6b74106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.385+0000 7f6b790f2640 1 -- 172.21.15.33:0/1157199613 >> 172.21.15.33:0/1157199613 conn(0x7f6b74077b30 msgr2=0x7f6b74077f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.385+0000 7f6b790f2640 1 -- 172.21.15.33:0/1157199613 shutdown_connections 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.385+0000 7f6b790f2640 1 -- 172.21.15.33:0/1157199613 wait complete. 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.386+0000 7f6b790f2640 1 Processor -- start 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.386+0000 7f6b790f2640 1 -- start start 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.386+0000 7f6b790f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b741009d0 0x7f6b74102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.386+0000 7f6b790f2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b74101390 0x7f6b741034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.386+0000 7f6b790f2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b741069e0 0x7f6b74103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.386+0000 7f6b790f2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b74110dc0 con 0x7f6b741009d0 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.386+0000 7f6b790f2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6b74110c40 con 0x7f6b74101390 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.386+0000 7f6b790f2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6b74110f40 con 0x7f6b741069e0 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.386+0000 7f6b72d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b741009d0 0x7f6b74102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:56.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.386+0000 7f6b72d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b741009d0 0x7f6b74102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33472/0 (socket says 172.21.15.33:33472) 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.386+0000 7f6b72d9d640 1 -- 172.21.15.33:0/3965024163 learned_addr learned my addr 172.21.15.33:0/3965024163 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.387+0000 7f6b7359e640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b741069e0 0x7f6b74103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.387+0000 7f6b7259c640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b74101390 0x7f6b741034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.387+0000 7f6b72d9d640 1 -- 172.21.15.33:0/3965024163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b741069e0 msgr2=0x7f6b74103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.387+0000 7f6b72d9d640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b741069e0 0x7f6b74103a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.387+0000 7f6b72d9d640 1 -- 172.21.15.33:0/3965024163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b74101390 msgr2=0x7f6b741034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.387+0000 7f6b72d9d640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b74101390 0x7f6b741034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.387+0000 7f6b72d9d640 1 -- 172.21.15.33:0/3965024163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b741a67a0 con 0x7f6b741009d0 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.387+0000 7f6b72d9d640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b741009d0 0x7f6b74102f80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d950 tx=0x7f6b6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.387+0000 7f6b6bfff640 1 -- 172.21.15.33:0/3965024163 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c017070 con 0x7f6b741009d0 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.387+0000 7f6b790f2640 1 -- 172.21.15.33:0/3965024163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b741a6a90 con 0x7f6b741009d0 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.387+0000 7f6b6bfff640 1 -- 172.21.15.33:0/3965024163 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6b6c009c30 con 0x7f6b741009d0 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.387+0000 7f6b6bfff640 1 -- 172.21.15.33:0/3965024163 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00f6b0 con 0x7f6b741009d0 2024-04-02T13:38:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.387+0000 7f6b790f2640 1 -- 172.21.15.33:0/3965024163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b741a7000 con 0x7f6b741009d0 2024-04-02T13:38:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.388+0000 7f6b790f2640 1 -- 172.21.15.33:0/3965024163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b74102110 con 0x7f6b741009d0 2024-04-02T13:38:56.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.389+0000 7f6b6bfff640 1 -- 172.21.15.33:0/3965024163 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6b6c010a50 con 0x7f6b741009d0 2024-04-02T13:38:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.390+0000 7f6b6bfff640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6b600a3f10 0x7f6b600a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.390+0000 7f6b7259c640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6b600a3f10 0x7f6b600a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.390+0000 7f6b6bfff640 1 -- 172.21.15.33:0/3965024163 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6b6c0d0960 con 0x7f6b741009d0 2024-04-02T13:38:56.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.390+0000 7f6b7259c640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6b600a3f10 0x7f6b600a63d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c000c00 tx=0x7f6b5c005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:56.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.394+0000 7f6b6bfff640 1 -- 172.21.15.33:0/3965024163 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6b6c09c3b0 con 0x7f6b741009d0 2024-04-02T13:38:56.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.602+0000 7f6b790f2640 1 -- 172.21.15.33:0/3965024163 --> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f6b74061f40 con 0x7f6b600a3f10 2024-04-02T13:38:56.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.609+0000 7f6b6bfff640 1 -- 172.21.15.33:0/3965024163 <== mgr.15456 v2:172.21.15.33:6800/619073725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f6b74061f40 con 0x7f6b600a3f10 2024-04-02T13:38:56.612 INFO:teuthology.orchestra.run.smithi033.stdout:/ 1m 6m3h 2024-04-02T13:38:56.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.611+0000 7f6b790f2640 1 -- 172.21.15.33:0/3965024163 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6b600a3f10 msgr2=0x7f6b600a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:56.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.611+0000 7f6b790f2640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6b600a3f10 0x7f6b600a63d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c000c00 tx=0x7f6b5c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.611+0000 7f6b790f2640 1 -- 172.21.15.33:0/3965024163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b741009d0 msgr2=0x7f6b74102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:56.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.611+0000 7f6b790f2640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b741009d0 0x7f6b74102f80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d950 tx=0x7f6b6c00de10 comp rx=0 tx=0).stop 2024-04-02T13:38:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.611+0000 7f6b790f2640 1 -- 172.21.15.33:0/3965024163 shutdown_connections 2024-04-02T13:38:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.611+0000 7f6b790f2640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6b600a3f10 0x7f6b600a63d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.611+0000 7f6b790f2640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6b741069e0 0x7f6b74103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.611+0000 7f6b790f2640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b74101390 0x7f6b741034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.611+0000 7f6b790f2640 1 --2- 172.21.15.33:0/3965024163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b741009d0 0x7f6b74102f80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.611+0000 7f6b790f2640 1 -- 172.21.15.33:0/3965024163 >> 172.21.15.33:0/3965024163 conn(0x7f6b74077b30 msgr2=0x7f6b7406a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.611+0000 7f6b790f2640 1 -- 172.21.15.33:0/3965024163 shutdown_connections 2024-04-02T13:38:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:38:56.611+0000 7f6b790f2640 1 -- 172.21.15.33:0/3965024163 wait complete. 2024-04-02T13:38:56.625 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-02T13:38:56.652 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-02T13:38:56.663 INFO:teuthology.run_tasks:Running task workunit... 2024-04-02T13:38:56.680 INFO:tasks.workunit:Pulling workunits from ref 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:38:56.680 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-02T13:38:56.680 INFO:tasks.workunit:timeout=3h 2024-04-02T13:38:56.680 INFO:tasks.workunit:cleanup=True 2024-04-02T13:38:56.681 DEBUG:teuthology.orchestra.run.smithi033:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-02T13:38:56.707 INFO:teuthology.orchestra.run.smithi033.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-02T13:38:56.707 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-02T13:38:56.707 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 77h/119d Inode: 1 Links: 2 2024-04-02T13:38:56.707 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:38:56.707 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-02T13:38:56.707 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-02 13:36:48.217098227 +0000 2024-04-02T13:38:56.707 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-02 13:36:48.217098227 +0000 2024-04-02T13:38:56.707 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-02 13:38:45.141976849 +0000 2024-04-02T13:38:56.708 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-04-02T13:38:56.708 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-02T13:38:56.708 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-02T13:38:56.780 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:38:56.835 INFO:tasks.workunit.client.0.smithi033.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-02T13:38:57.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:56 smithi145 ceph-mon[35646]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:57.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:56 smithi145 ceph-mon[35646]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:57.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:56 smithi145 ceph-mon[35646]: from='client.15726 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:57.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:56 smithi050 ceph-mon[35524]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:57.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:56 smithi050 ceph-mon[35524]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:57.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:56 smithi050 ceph-mon[35524]: from='client.15726 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:57.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:56 smithi033 ceph-mon[32390]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:57.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:56 smithi033 ceph-mon[32390]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:57.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:56 smithi033 ceph-mon[32390]: from='client.15726 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:38:58.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:57 smithi145 ceph-mon[35646]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 17 op/s 2024-04-02T13:38:58.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:57 smithi050 ceph-mon[35524]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 17 op/s 2024-04-02T13:38:58.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:57 smithi033 ceph-mon[32390]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 17 op/s 2024-04-02T13:39:00.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:38:59 smithi145 ceph-mon[35646]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-02T13:39:00.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:38:59 smithi050 ceph-mon[35524]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-02T13:39:00.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:38:59 smithi033 ceph-mon[32390]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-02T13:39:01.254 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:39:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:39:00.932+0000 7f54b7415700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:39:01.254 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:39:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:39:00.932+0000 7f54b7415700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:39:01.254 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:39:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:39:00.932+0000 7f54b7415700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:39:01.254 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:39:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:39:00.932+0000 7f54b7415700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:39:01.254 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:39:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:39:00.932+0000 7f54b7415700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:39:02.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:01 smithi145 ceph-mon[35646]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-02T13:39:02.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:01 smithi050 ceph-mon[35524]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-02T13:39:02.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:01 smithi033 ceph-mon[32390]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-02T13:39:04.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:03 smithi145 ceph-mon[35646]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-02T13:39:04.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:03 smithi145 ceph-mon[35646]: mgrmap e25: x(active, since 16s), standbys: z, y 2024-04-02T13:39:04.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:03 smithi050 ceph-mon[35524]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-02T13:39:04.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:03 smithi050 ceph-mon[35524]: mgrmap e25: x(active, since 16s), standbys: z, y 2024-04-02T13:39:04.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:03 smithi033 ceph-mon[32390]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-02T13:39:04.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:03 smithi033 ceph-mon[32390]: mgrmap e25: x(active, since 16s), standbys: z, y 2024-04-02T13:39:06.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:05 smithi145 ceph-mon[35646]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-04-02T13:39:06.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:05 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:39:06.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:05 smithi050 ceph-mon[35524]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-04-02T13:39:06.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:05 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:39:06.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:05 smithi033 ceph-mon[32390]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-04-02T13:39:06.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:05 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:39:08.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:07 smithi145 ceph-mon[35646]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 52 op/s 2024-04-02T13:39:08.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:07 smithi050 ceph-mon[35524]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 52 op/s 2024-04-02T13:39:08.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:07 smithi033 ceph-mon[32390]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 52 op/s 2024-04-02T13:39:10.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:09 smithi145 ceph-mon[35646]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-04-02T13:39:10.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:09 smithi050 ceph-mon[35524]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-04-02T13:39:10.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:09 smithi033 ceph-mon[32390]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-04-02T13:39:12.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:11 smithi145 ceph-mon[35646]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-02T13:39:12.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:11 smithi050 ceph-mon[35524]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-02T13:39:12.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:11 smithi033 ceph-mon[32390]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-02T13:39:14.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:13 smithi145 ceph-mon[35646]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-02T13:39:14.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:13 smithi050 ceph-mon[35524]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-02T13:39:14.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:13 smithi033 ceph-mon[32390]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-02T13:39:16.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:15 smithi145 ceph-mon[35646]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-02T13:39:16.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:15 smithi050 ceph-mon[35524]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-02T13:39:16.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:15 smithi033 ceph-mon[32390]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-02T13:39:18.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:17 smithi145 ceph-mon[35646]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-02T13:39:18.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:17 smithi050 ceph-mon[35524]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-02T13:39:18.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:17 smithi033 ceph-mon[32390]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-02T13:39:20.229 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:19 smithi145 ceph-mon[35646]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-02T13:39:20.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:19 smithi050 ceph-mon[35524]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-02T13:39:20.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:19 smithi033 ceph-mon[32390]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-02T13:39:22.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:21 smithi145 ceph-mon[35646]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-02T13:39:22.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:21 smithi050 ceph-mon[35524]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-02T13:39:22.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:21 smithi033 ceph-mon[32390]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-02T13:39:24.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:23 smithi145 ceph-mon[35646]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-02T13:39:24.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:23 smithi050 ceph-mon[35524]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-02T13:39:24.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:23 smithi033 ceph-mon[32390]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-02T13:39:25.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:39:25.392 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:39:25.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.509+0000 7f5a6dbcd640 1 -- 172.21.15.33:0/943491488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a68101420 msgr2=0x7f5a6810d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.509+0000 7f5a6dbcd640 1 --2- 172.21.15.33:0/943491488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a68101420 0x7f5a6810d8e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009cd0 tx=0x7f5a6002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.510+0000 7f5a6dbcd640 1 -- 172.21.15.33:0/943491488 shutdown_connections 2024-04-02T13:39:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.510+0000 7f5a6dbcd640 1 --2- 172.21.15.33:0/943491488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a68101420 0x7f5a6810d8e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.510+0000 7f5a6dbcd640 1 --2- 172.21.15.33:0/943491488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a68100a60 0x7f5a68100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.510+0000 7f5a6dbcd640 1 --2- 172.21.15.33:0/943491488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a68106a20 0x7f5a68106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.510+0000 7f5a6dbcd640 1 -- 172.21.15.33:0/943491488 >> 172.21.15.33:0/943491488 conn(0x7f5a68077b60 msgr2=0x7f5a68077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.510+0000 7f5a6dbcd640 1 -- 172.21.15.33:0/943491488 shutdown_connections 2024-04-02T13:39:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.510+0000 7f5a6dbcd640 1 -- 172.21.15.33:0/943491488 wait complete. 2024-04-02T13:39:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.510+0000 7f5a6dbcd640 1 Processor -- start 2024-04-02T13:39:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.510+0000 7f5a6dbcd640 1 -- start start 2024-04-02T13:39:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a6dbcd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a68100a60 0x7f5a681983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a6dbcd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a68101420 0x7f5a68198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a6dbcd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a68106a20 0x7f5a6819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a6dbcd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68110e00 con 0x7f5a68100a60 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a6dbcd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68110c80 con 0x7f5a68101420 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a6dbcd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68110f80 con 0x7f5a68106a20 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a68100a60 0x7f5a681983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a68100a60 0x7f5a681983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54922/0 (socket says 172.21.15.33:54922) 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a677fe640 1 -- 172.21.15.33:0/1040927 learned_addr learned my addr 172.21.15.33:0/1040927 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a66ffd640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a68101420 0x7f5a68198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a67fff640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a68106a20 0x7f5a6819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a677fe640 1 -- 172.21.15.33:0/1040927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a68106a20 msgr2=0x7f5a6819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a677fe640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a68106a20 0x7f5a6819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a677fe640 1 -- 172.21.15.33:0/1040927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a68101420 msgr2=0x7f5a68198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a677fe640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a68101420 0x7f5a68198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a677fe640 1 -- 172.21.15.33:0/1040927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6819d3c0 con 0x7f5a68100a60 2024-04-02T13:39:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a677fe640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a68100a60 0x7f5a681983d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800d950 tx=0x7f5a5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:25.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a64ff9640 1 -- 172.21.15.33:0/1040927 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58017070 con 0x7f5a68100a60 2024-04-02T13:39:25.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a64ff9640 1 -- 172.21.15.33:0/1040927 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a58009c30 con 0x7f5a68100a60 2024-04-02T13:39:25.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a6dbcd640 1 -- 172.21.15.33:0/1040927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6819d6b0 con 0x7f5a68100a60 2024-04-02T13:39:25.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.511+0000 7f5a64ff9640 1 -- 172.21.15.33:0/1040927 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5800f6b0 con 0x7f5a68100a60 2024-04-02T13:39:25.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.512+0000 7f5a6dbcd640 1 -- 172.21.15.33:0/1040927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a681a4f40 con 0x7f5a68100a60 2024-04-02T13:39:25.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.514+0000 7f5a64ff9640 1 -- 172.21.15.33:0/1040927 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f5a58010a50 con 0x7f5a68100a60 2024-04-02T13:39:25.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.514+0000 7f5a707bb740 1 Processor -- start 2024-04-02T13:39:25.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.514+0000 7f5a64ff9640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5a500a3ed0 0x7f5a500a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.514+0000 7f5a707bb740 1 -- start start 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.514+0000 7f5a67fff640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5a500a3ed0 0x7f5a500a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.514+0000 7f5a707bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 0x55aee3b3acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.514+0000 7f5a707bb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b43890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.514+0000 7f5a64ff9640 1 -- 172.21.15.33:0/1040927 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5a580d09e0 con 0x7f5a68100a60 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.514+0000 7f5a707bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b43dd0 0x55aee3b461c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.514+0000 7f5a707bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aee3a9e830 con 0x55aee3b3a8d0 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.514+0000 7f5a707bb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55aee3a46820 con 0x55aee3b3b1f0 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.514+0000 7f5a707bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55aee3a9a7e0 con 0x55aee3b43dd0 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 0x55aee3b3acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 0x55aee3b3acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54932/0 (socket says 172.21.15.33:54932) 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a677fe640 1 -- 172.21.15.33:0/4203024320 learned_addr learned my addr 172.21.15.33:0/4203024320 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a66ffd640 1 --2- 172.21.15.33:0/4203024320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b43890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a677fe640 1 -- 172.21.15.33:0/4203024320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b43dd0 msgr2=0x55aee3b461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:25.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a677fe640 1 --2- 172.21.15.33:0/4203024320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b43dd0 0x55aee3b461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a677fe640 1 -- 172.21.15.33:0/4203024320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 msgr2=0x55aee3b43890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a677fe640 1 --2- 172.21.15.33:0/4203024320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b43890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a677fe640 1 -- 172.21.15.33:0/4203024320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee387c7d0 con 0x55aee3b3a8d0 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a67fff640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5a500a3ed0 0x7f5a500a6390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a60006010 tx=0x7f5a60005de0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a677fe640 1 --2- 172.21.15.33:0/4203024320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 0x55aee3b3acb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5a580a4b10 tx=0x7f5a58009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fd1fa0505c541a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a4dffb640 1 -- 172.21.15.33:0/4203024320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58020070 con 0x55aee3b3a8d0 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a4dffb640 1 -- 172.21.15.33:0/4203024320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a5809c3d0 con 0x55aee3b3a8d0 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.515+0000 7f5a4dffb640 1 -- 172.21.15.33:0/4203024320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58098120 con 0x55aee3b3a8d0 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.516+0000 7f5a707bb740 1 -- 172.21.15.33:0/4203024320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 msgr2=0x55aee3b3acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.516+0000 7f5a707bb740 1 --2- 172.21.15.33:0/4203024320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 0x55aee3b3acb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5a580a4b10 tx=0x7f5a58009de0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.516+0000 7f5a707bb740 1 -- 172.21.15.33:0/4203024320 shutdown_connections 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.516+0000 7f5a707bb740 1 --2- 172.21.15.33:0/4203024320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b43dd0 0x55aee3b461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.516+0000 7f5a707bb740 1 --2- 172.21.15.33:0/4203024320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b43890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.516+0000 7f5a707bb740 1 --2- 172.21.15.33:0/4203024320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 0x55aee3b3acb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.516+0000 7f5a707bb740 1 -- 172.21.15.33:0/4203024320 >> 172.21.15.33:0/4203024320 conn(0x55aee3b35000 msgr2=0x55aee3b353e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.516+0000 7f5a707bb740 1 -- 172.21.15.33:0/4203024320 shutdown_connections 2024-04-02T13:39:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.516+0000 7f5a707bb740 1 -- 172.21.15.33:0/4203024320 wait complete. 2024-04-02T13:39:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.516+0000 7f5a707bb740 1 -- start start 2024-04-02T13:39:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a707bb740 1 Processor -- start 2024-04-02T13:39:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a707bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 0x55aee3b436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a707bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3b1f0 0x55aee3b40a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a707bb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b43dd0 0x55aee3b40f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a707bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aee3b46bc0 con 0x55aee3b3a8d0 2024-04-02T13:39:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a707bb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55aee3a59d30 con 0x55aee3b43dd0 2024-04-02T13:39:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a707bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55aee3b46d40 con 0x55aee3b3b1f0 2024-04-02T13:39:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a707bb740 10 client.0 _ll_register_callbacks cb 0x55aee3a59720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 0x55aee3b436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 0x55aee3b436c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54938/0 (socket says 172.21.15.33:54938) 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a677fe640 1 -- 172.21.15.33:0/2519171420 learned_addr learned my addr 172.21.15.33:0/2519171420 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a66ffd640 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3b1f0 0x55aee3b40a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a67fff640 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b43dd0 0x55aee3b40f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a677fe640 1 -- 172.21.15.33:0/2519171420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3b1f0 msgr2=0x55aee3b40a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a677fe640 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3b1f0 0x55aee3b40a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a677fe640 1 -- 172.21.15.33:0/2519171420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b43dd0 msgr2=0x55aee3b40f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a677fe640 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b43dd0 0x55aee3b40f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a677fe640 1 -- 172.21.15.33:0/2519171420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aee37b23f0 con 0x55aee3b3a8d0 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a66ffd640 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3b1f0 0x55aee3b40a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a67fff640 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b43dd0 0x55aee3b40f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a677fe640 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 0x55aee3b436c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5a580cdc70 tx=0x7f5a58098c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec539f7b1afcca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.517+0000 7f5a46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a46ffd640 1 -- 172.21.15.33:0/2519171420 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58020070 con 0x55aee3b3a8d0 2024-04-02T13:39:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a46ffd640 1 -- 172.21.15.33:0/2519171420 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5a580ce180 con 0x55aee3b3a8d0 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a707bb740 1 -- 172.21.15.33:0/2519171420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a5809c3d0 con 0x55aee3b3a8d0 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a46ffd640 1 -- 172.21.15.33:0/2519171420 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a58083050 con 0x55aee3b3a8d0 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a707bb740 10 client.15744 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a707bb740 1 -- 172.21.15.33:0/2519171420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aee37720f0 con 0x55aee3b3a8d0 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a46ffd640 1 -- 172.21.15.33:0/2519171420 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a5809c3d0 con 0x55aee3b3a8d0 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a46ffd640 1 -- 172.21.15.33:0/2519171420 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5a580981b0 con 0x55aee3b3a8d0 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a707bb740 10 client.15744 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a707bb740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a707bb740 20 client.15744 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a707bb740 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aee3b53c80 0x55aee3b56070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a707bb740 4 client.15744 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a707bb740 1 -- 172.21.15.33:0/2519171420 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aee3aabd70 con 0x55aee3b53c80 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.518+0000 7f5a66ffd640 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aee3b53c80 0x55aee3b56070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.519+0000 7f5a66ffd640 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aee3b53c80 0x55aee3b56070 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e97cab2049c1bac server_cookie=30293c3585bb7caa in_seq=0 out_seq=0 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.519+0000 7f5a46ffd640 10 client.15744 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a46ffd640 1 -- 172.21.15.33:0/2519171420 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aee3aabd70 con 0x55aee3b53c80 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a46ffd640 10 client.15744 handle_command_reply: tid=0 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a707bb740 1 client.15744 shutdown 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a707bb740 1 -- 172.21.15.33:0/2519171420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 msgr2=0x55aee3b436c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a707bb740 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 0x55aee3b436c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5a580cdc70 tx=0x7f5a58098c70 comp rx=0 tx=0).stop 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a707bb740 1 -- 172.21.15.33:0/2519171420 shutdown_connections 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a707bb740 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aee3b53c80 0x55aee3b56070 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a707bb740 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b43dd0 0x55aee3b40f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a707bb740 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3b1f0 0x55aee3b40a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a707bb740 1 --2- 172.21.15.33:0/2519171420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b3a8d0 0x55aee3b436c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a46ffd640 0 client.15744 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a707bb740 1 -- 172.21.15.33:0/2519171420 >> 172.21.15.33:0/2519171420 conn(0x55aee3b35000 msgr2=0x55aee3b36d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a707bb740 1 -- 172.21.15.33:0/2519171420 shutdown_connections 2024-04-02T13:39:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a707bb740 1 -- 172.21.15.33:0/2519171420 wait complete. 2024-04-02T13:39:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.520+0000 7f5a707bb740 20 client.15744 trim_cache size 0 max 16384 2024-04-02T13:39:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a707bb740 1 Processor -- start 2024-04-02T13:39:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a707bb740 1 -- start start 2024-04-02T13:39:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a707bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3a8d0 0x55aee3b3be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a707bb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b43180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a707bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 0x55aee3b436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a707bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5a681b18a0 con 0x55aee3b43dd0 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a707bb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a681af170 con 0x55aee3b3b1f0 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a707bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5a681acfa0 con 0x55aee3b3a8d0 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 0x55aee3b436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 0x55aee3b436c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54940/0 (socket says 172.21.15.33:54940) 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a67fff640 1 -- 172.21.15.33:0/4166091315 learned_addr learned my addr 172.21.15.33:0/4166091315 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a677fe640 1 --2- 172.21.15.33:0/4166091315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3a8d0 0x55aee3b3be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a66ffd640 1 --2- 172.21.15.33:0/4166091315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b43180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a67fff640 1 -- 172.21.15.33:0/4166091315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3a8d0 msgr2=0x55aee3b3be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a67fff640 1 --2- 172.21.15.33:0/4166091315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3a8d0 0x55aee3b3be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a67fff640 1 -- 172.21.15.33:0/4166091315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 msgr2=0x55aee3b43180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a67fff640 1 --2- 172.21.15.33:0/4166091315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b43180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a67fff640 1 -- 172.21.15.33:0/4166091315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee3b35bd0 con 0x55aee3b43dd0 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.538+0000 7f5a66ffd640 1 --2- 172.21.15.33:0/4166091315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b43180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.539+0000 7f5a67fff640 1 --2- 172.21.15.33:0/4166091315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 0x55aee3b436c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009ca0 tx=0x7f5a6003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919a51720e4b41d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.539+0000 7f5a4d7fa640 1 -- 172.21.15.33:0/4166091315 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60051070 con 0x55aee3b43dd0 2024-04-02T13:39:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.539+0000 7f5a4d7fa640 1 -- 172.21.15.33:0/4166091315 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a60003970 con 0x55aee3b43dd0 2024-04-02T13:39:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.539+0000 7f5a4d7fa640 1 -- 172.21.15.33:0/4166091315 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60039040 con 0x55aee3b43dd0 2024-04-02T13:39:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.539+0000 7f5a707bb740 1 -- 172.21.15.33:0/4166091315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 msgr2=0x55aee3b436c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.539+0000 7f5a707bb740 1 --2- 172.21.15.33:0/4166091315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 0x55aee3b436c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009ca0 tx=0x7f5a6003f000 comp rx=0 tx=0).stop 2024-04-02T13:39:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.539+0000 7f5a707bb740 1 -- 172.21.15.33:0/4166091315 shutdown_connections 2024-04-02T13:39:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.539+0000 7f5a707bb740 1 --2- 172.21.15.33:0/4166091315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 0x55aee3b436c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.539+0000 7f5a707bb740 1 --2- 172.21.15.33:0/4166091315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b43180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.539+0000 7f5a707bb740 1 --2- 172.21.15.33:0/4166091315 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3a8d0 0x55aee3b3be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.539+0000 7f5a707bb740 1 -- 172.21.15.33:0/4166091315 >> 172.21.15.33:0/4166091315 conn(0x55aee3b35000 msgr2=0x55aee3b3b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.539+0000 7f5a707bb740 1 -- 172.21.15.33:0/4166091315 shutdown_connections 2024-04-02T13:39:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.540+0000 7f5a707bb740 1 -- 172.21.15.33:0/4166091315 wait complete. 2024-04-02T13:39:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.540+0000 7f5a707bb740 1 -- start start 2024-04-02T13:39:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.540+0000 7f5a707bb740 1 Processor -- start 2024-04-02T13:39:25.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.540+0000 7f5a707bb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3a8d0 0x55aee3b47920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.540+0000 7f5a707bb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b47e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.540+0000 7f5a707bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 0x55aee3b483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.540+0000 7f5a707bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5a681ad630 con 0x55aee3b43dd0 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.540+0000 7f5a707bb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a681aeaa0 con 0x55aee3b3b1f0 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.540+0000 7f5a707bb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5a681add00 con 0x55aee3b3a8d0 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.540+0000 7f5a707bb740 10 client.0 _ll_register_callbacks cb 0x7f5a580cdb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.540+0000 7f5a67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 0x55aee3b483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.540+0000 7f5a67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 0x55aee3b483a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54950/0 (socket says 172.21.15.33:54950) 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.540+0000 7f5a67fff640 1 -- 172.21.15.33:0/2212820485 learned_addr learned my addr 172.21.15.33:0/2212820485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a677fe640 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3a8d0 0x55aee3b47920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a66ffd640 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b47e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a67fff640 1 -- 172.21.15.33:0/2212820485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3a8d0 msgr2=0x55aee3b47920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a67fff640 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3a8d0 0x55aee3b47920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a67fff640 1 -- 172.21.15.33:0/2212820485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 msgr2=0x55aee3b47e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a67fff640 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b47e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a67fff640 1 -- 172.21.15.33:0/2212820485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aee3bb8f40 con 0x55aee3b43dd0 2024-04-02T13:39:25.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a677fe640 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3a8d0 0x55aee3b47920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a66ffd640 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b47e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a67fff640 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 0x55aee3b483a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5a6002c630 tx=0x7f5a60032880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9040b372d0ea1d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a46ffd640 1 -- 172.21.15.33:0/2212820485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60051030 con 0x55aee3b43dd0 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a46ffd640 1 -- 172.21.15.33:0/2212820485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5a60049040 con 0x55aee3b43dd0 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a707bb740 1 -- 172.21.15.33:0/2212820485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a60003970 con 0x55aee3b43dd0 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a46ffd640 1 -- 172.21.15.33:0/2212820485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a60039040 con 0x55aee3b43dd0 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a707bb740 10 client.15750 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a707bb740 1 -- 172.21.15.33:0/2212820485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aee3bba190 con 0x55aee3b43dd0 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.541+0000 7f5a46ffd640 1 -- 172.21.15.33:0/2212820485 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5a60003970 con 0x55aee3b43dd0 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.542+0000 7f5a46ffd640 1 -- 172.21.15.33:0/2212820485 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5a6003f070 con 0x55aee3b43dd0 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.542+0000 7f5a707bb740 10 client.15750 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.542+0000 7f5a707bb740 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.542+0000 7f5a707bb740 20 client.15750 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.542+0000 7f5a707bb740 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aee3bba800 0x55aee3bbcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.542+0000 7f5a707bb740 4 client.15750 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.542+0000 7f5a707bb740 1 -- 172.21.15.33:0/2212820485 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aee3aa60c0 con 0x55aee3bba800 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.542+0000 7f5a677fe640 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aee3bba800 0x55aee3bbcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.542+0000 7f5a46ffd640 10 client.15750 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.542+0000 7f5a677fe640 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aee3bba800 0x55aee3bbcc20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7dfedb15d5fa5ac server_cookie=e6c231519362b4e5 in_seq=0 out_seq=0 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a46ffd640 1 -- 172.21.15.33:0/2212820485 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aee3aa60c0 con 0x55aee3bba800 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a46ffd640 10 client.15750 handle_command_reply: tid=0 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a707bb740 1 client.15750 shutdown 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a707bb740 1 -- 172.21.15.33:0/2212820485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 msgr2=0x55aee3b483a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a707bb740 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 0x55aee3b483a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5a6002c630 tx=0x7f5a60032880 comp rx=0 tx=0).stop 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a707bb740 1 -- 172.21.15.33:0/2212820485 shutdown_connections 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a707bb740 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aee3bba800 0x55aee3bbcc20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a707bb740 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aee3b43dd0 0x55aee3b483a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a707bb740 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aee3b3b1f0 0x55aee3b47e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a46ffd640 0 client.15750 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:25.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a707bb740 1 --2- 172.21.15.33:0/2212820485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aee3b3a8d0 0x55aee3b47920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a707bb740 1 -- 172.21.15.33:0/2212820485 >> 172.21.15.33:0/2212820485 conn(0x55aee3b35000 msgr2=0x55aee3b3bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:25.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a707bb740 1 -- 172.21.15.33:0/2212820485 shutdown_connections 2024-04-02T13:39:25.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a707bb740 1 -- 172.21.15.33:0/2212820485 wait complete. 2024-04-02T13:39:25.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a707bb740 20 client.15750 trim_cache size 0 max 16384 2024-04-02T13:39:25.552 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:39:25.552 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:39:25.552 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:39:25.552 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:39:25.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a6dbcd640 1 -- 172.21.15.33:0/1040927 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5a500a3ed0 msgr2=0x7f5a500a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a6dbcd640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5a500a3ed0 0x7f5a500a6390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a60006010 tx=0x7f5a60005de0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a6dbcd640 1 -- 172.21.15.33:0/1040927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a68100a60 msgr2=0x7f5a681983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a6dbcd640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a68100a60 0x7f5a681983d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800d950 tx=0x7f5a5800de10 comp rx=0 tx=0).stop 2024-04-02T13:39:25.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a6dbcd640 1 -- 172.21.15.33:0/1040927 shutdown_connections 2024-04-02T13:39:25.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a6dbcd640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5a500a3ed0 0x7f5a500a6390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a6dbcd640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a68106a20 0x7f5a6819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a6dbcd640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a68101420 0x7f5a68198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a6dbcd640 1 --2- 172.21.15.33:0/1040927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a68100a60 0x7f5a681983d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.543+0000 7f5a6dbcd640 1 -- 172.21.15.33:0/1040927 >> 172.21.15.33:0/1040927 conn(0x7f5a68077b60 msgr2=0x7f5a6806a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:25.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.544+0000 7f5a6dbcd640 1 -- 172.21.15.33:0/1040927 shutdown_connections 2024-04-02T13:39:25.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.544+0000 7f5a6dbcd640 1 -- 172.21.15.33:0/1040927 wait complete. 2024-04-02T13:39:25.557 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:39:25.557 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:39:25.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 -- 172.21.15.33:0/2489826600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c106a10 msgr2=0x7fae5c106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 --2- 172.21.15.33:0/2489826600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c106a10 0x7fae5c106df0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fae58009930 tx=0x7fae5802bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 -- 172.21.15.33:0/2489826600 shutdown_connections 2024-04-02T13:39:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 --2- 172.21.15.33:0/2489826600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae5c1013c0 0x7fae5c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 --2- 172.21.15.33:0/2489826600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae5c100a00 0x7fae5c100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 --2- 172.21.15.33:0/2489826600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c106a10 0x7fae5c106df0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 -- 172.21.15.33:0/2489826600 >> 172.21.15.33:0/2489826600 conn(0x7fae5c077b00 msgr2=0x7fae5c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 -- 172.21.15.33:0/2489826600 shutdown_connections 2024-04-02T13:39:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 -- 172.21.15.33:0/2489826600 wait complete. 2024-04-02T13:39:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 Processor -- start 2024-04-02T13:39:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 -- start start 2024-04-02T13:39:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae5c100a00 0x7fae5c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae5c1013c0 0x7fae5c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c106a10 0x7fae5c1a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c110df0 con 0x7fae5c106a10 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c110c70 con 0x7fae5c100a00 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.658+0000 7fae64022640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c110f70 con 0x7fae5c1013c0 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae625bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c106a10 0x7fae5c1a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae625bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c106a10 0x7fae5c1a42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54968/0 (socket says 172.21.15.33:54968) 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae625bf640 1 -- 172.21.15.33:0/3702883747 learned_addr learned my addr 172.21.15.33:0/3702883747 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae61dbe640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae5c100a00 0x7fae5c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae615bd640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae5c1013c0 0x7fae5c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae625bf640 1 -- 172.21.15.33:0/3702883747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae5c1013c0 msgr2=0x7fae5c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae625bf640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae5c1013c0 0x7fae5c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae625bf640 1 -- 172.21.15.33:0/3702883747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae5c100a00 msgr2=0x7fae5c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae625bf640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae5c100a00 0x7fae5c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae625bf640 1 -- 172.21.15.33:0/3702883747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5c1a4830 con 0x7fae5c106a10 2024-04-02T13:39:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae625bf640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c106a10 0x7fae5c1a42f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fae5400eb10 tx=0x7fae5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae52ffd640 1 -- 172.21.15.33:0/3702883747 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54004b20 con 0x7fae5c106a10 2024-04-02T13:39:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae52ffd640 1 -- 172.21.15.33:0/3702883747 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fae54004cc0 con 0x7fae5c106a10 2024-04-02T13:39:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae64022640 1 -- 172.21.15.33:0/3702883747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae5c1a4a00 con 0x7fae5c106a10 2024-04-02T13:39:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae52ffd640 1 -- 172.21.15.33:0/3702883747 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54010700 con 0x7fae5c106a10 2024-04-02T13:39:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.659+0000 7fae64022640 1 -- 172.21.15.33:0/3702883747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae5c1a4ec0 con 0x7fae5c106a10 2024-04-02T13:39:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.660+0000 7fae64022640 1 -- 172.21.15.33:0/3702883747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae5c0688f0 con 0x7fae5c106a10 2024-04-02T13:39:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.661+0000 7fae52ffd640 1 -- 172.21.15.33:0/3702883747 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fae54004e70 con 0x7fae5c106a10 2024-04-02T13:39:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.662+0000 7fae52ffd640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fae480a3f50 0x7fae480a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.662+0000 7fae52ffd640 1 -- 172.21.15.33:0/3702883747 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fae540d1470 con 0x7fae5c106a10 2024-04-02T13:39:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.662+0000 7fae61dbe640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fae480a3f50 0x7fae480a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.662+0000 7fae61dbe640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fae480a3f50 0x7fae480a6410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fae58000c00 tx=0x7fae58005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:25.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.666+0000 7fae52ffd640 1 -- 172.21.15.33:0/3702883747 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fae54053f20 con 0x7fae5c106a10 2024-04-02T13:39:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.893+0000 7fae64022640 1 -- 172.21.15.33:0/3702883747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae5c069200 con 0x7fae5c106a10 2024-04-02T13:39:25.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.893+0000 7fae52ffd640 1 -- 172.21.15.33:0/3702883747 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11873 (secure 0 0 0) 0x7fae5c069200 con 0x7fae5c106a10 2024-04-02T13:39:25.896 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:39:25.896 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:38:45.531661+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24473,24512]},"id":1}]} 2024-04-02T13:39:25.896 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 25 2024-04-02T13:39:25.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.895+0000 7fae64022640 1 -- 172.21.15.33:0/3702883747 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fae480a3f50 msgr2=0x7fae480a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.895+0000 7fae64022640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fae480a3f50 0x7fae480a6410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fae58000c00 tx=0x7fae58005fd0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.895+0000 7fae64022640 1 -- 172.21.15.33:0/3702883747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c106a10 msgr2=0x7fae5c1a42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:25.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.895+0000 7fae64022640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c106a10 0x7fae5c1a42f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fae5400eb10 tx=0x7fae5400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.895+0000 7fae64022640 1 -- 172.21.15.33:0/3702883747 shutdown_connections 2024-04-02T13:39:25.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.895+0000 7fae64022640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fae480a3f50 0x7fae480a6410 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.895+0000 7fae64022640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c106a10 0x7fae5c1a42f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.895+0000 7fae64022640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae5c1013c0 0x7fae5c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.895+0000 7fae64022640 1 --2- 172.21.15.33:0/3702883747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae5c100a00 0x7fae5c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:25.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.895+0000 7fae64022640 1 -- 172.21.15.33:0/3702883747 >> 172.21.15.33:0/3702883747 conn(0x7fae5c077b00 msgr2=0x7fae5c1032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:25.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.895+0000 7fae64022640 1 -- 172.21.15.33:0/3702883747 shutdown_connections 2024-04-02T13:39:25.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:25.895+0000 7fae64022640 1 -- 172.21.15.33:0/3702883747 wait complete. 2024-04-02T13:39:25.909 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:39:26.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.013+0000 7f5773111640 1 -- 172.21.15.33:0/811501247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c1059a0 msgr2=0x7f576c105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.013+0000 7f5773111640 1 --2- 172.21.15.33:0/811501247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c1059a0 0x7f576c105d80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5760009930 tx=0x7f576002bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:26.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.013+0000 7f5773111640 1 -- 172.21.15.33:0/811501247 shutdown_connections 2024-04-02T13:39:26.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.013+0000 7f5773111640 1 --2- 172.21.15.33:0/811501247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f576c101ed0 0x7f576c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.013+0000 7f5773111640 1 --2- 172.21.15.33:0/811501247 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f576c101510 0x7f576c101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.013+0000 7f5773111640 1 --2- 172.21.15.33:0/811501247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c1059a0 0x7f576c105d80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.013+0000 7f5773111640 1 -- 172.21.15.33:0/811501247 >> 172.21.15.33:0/811501247 conn(0x7f576c075c60 msgr2=0x7f576c076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.013+0000 7f5773111640 1 -- 172.21.15.33:0/811501247 shutdown_connections 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.013+0000 7f5773111640 1 -- 172.21.15.33:0/811501247 wait complete. 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5773111640 1 Processor -- start 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5773111640 1 -- start start 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5773111640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c101510 0x7f576c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5773111640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f576c101ed0 0x7f576c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5773111640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f576c1059a0 0x7f576c1a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5773111640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f576c110da0 con 0x7f576c101510 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5773111640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f576c110c20 con 0x7f576c101ed0 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5773111640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f576c110f20 con 0x7f576c1059a0 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5770ead640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c101510 0x7f576c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5770ead640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c101510 0x7f576c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54992/0 (socket says 172.21.15.33:54992) 2024-04-02T13:39:26.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5770ead640 1 -- 172.21.15.33:0/3484575121 learned_addr learned my addr 172.21.15.33:0/3484575121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f57716ae640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f576c1059a0 0x7f576c1a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5770ead640 1 -- 172.21.15.33:0/3484575121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f576c1059a0 msgr2=0x7f576c1a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f576bfff640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f576c101ed0 0x7f576c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5770ead640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f576c1059a0 0x7f576c1a41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5770ead640 1 -- 172.21.15.33:0/3484575121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f576c101ed0 msgr2=0x7f576c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5770ead640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f576c101ed0 0x7f576c199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f5770ead640 1 -- 172.21.15.33:0/3484575121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c1a46e0 con 0x7f576c101510 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.014+0000 7f57716ae640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f576c1059a0 0x7f576c1a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.015+0000 7f5770ead640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c101510 0x7f576c199120 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5760009900 tx=0x7f576002cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.015+0000 7f5769ffb640 1 -- 172.21.15.33:0/3484575121 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760043020 con 0x7f576c101510 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.015+0000 7f5773111640 1 -- 172.21.15.33:0/3484575121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576c1a48b0 con 0x7f576c101510 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.015+0000 7f5769ffb640 1 -- 172.21.15.33:0/3484575121 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f57600054d0 con 0x7f576c101510 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.015+0000 7f5769ffb640 1 -- 172.21.15.33:0/3484575121 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760003a80 con 0x7f576c101510 2024-04-02T13:39:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.015+0000 7f5773111640 1 -- 172.21.15.33:0/3484575121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576c1a4da0 con 0x7f576c101510 2024-04-02T13:39:26.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.017+0000 7f5769ffb640 1 -- 172.21.15.33:0/3484575121 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f5760046020 con 0x7f576c101510 2024-04-02T13:39:26.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.017+0000 7f5775dd0740 1 Processor -- start 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f5775dd0740 1 -- start start 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f5775dd0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 0x5651ea72db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f5775dd0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 0x5651ea736760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f5775dd0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea736ca0 0x5651ea739090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f5775dd0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5651ea6917b0 con 0x5651ea72d7a0 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f5775dd0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5651ea630570 con 0x5651ea736ca0 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f5775dd0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5651ea68d760 con 0x5651ea72e0c0 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f57716ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 0x5651ea72db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f5769ffb640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f575c0a3ed0 0x7f575c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f57716ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 0x5651ea72db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54998/0 (socket says 172.21.15.33:54998) 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f57716ae640 1 -- 172.21.15.33:0/2251642894 learned_addr learned my addr 172.21.15.33:0/2251642894 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f576bfff640 1 --2- 172.21.15.33:0/2251642894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea736ca0 0x5651ea739090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f57716ae640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f575c0a3ed0 0x7f575c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f5769ffb640 1 -- 172.21.15.33:0/3484575121 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f57600f29c0 con 0x7f576c101510 2024-04-02T13:39:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f5770ead640 1 --2- 172.21.15.33:0/2251642894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 0x5651ea736760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f57716ae640 1 -- 172.21.15.33:0/2251642894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 msgr2=0x5651ea736760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f57716ae640 1 --2- 172.21.15.33:0/2251642894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 0x5651ea736760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f57716ae640 1 -- 172.21.15.33:0/2251642894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea736ca0 msgr2=0x5651ea739090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f57716ae640 1 --2- 172.21.15.33:0/2251642894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea736ca0 0x5651ea739090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f57716ae640 1 -- 172.21.15.33:0/2251642894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651ea3a53f0 con 0x5651ea72d7a0 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.018+0000 7f576bfff640 1 --2- 172.21.15.33:0/2251642894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea736ca0 0x5651ea739090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f57716ae640 1 --2- 172.21.15.33:0/2251642894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 0x5651ea72db80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5764012940 tx=0x7f5764012e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7fa6a903310590 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f5756ffd640 1 -- 172.21.15.33:0/2251642894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764022070 con 0x5651ea72d7a0 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f5756ffd640 1 -- 172.21.15.33:0/2251642894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5764012250 con 0x5651ea72d7a0 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f5756ffd640 1 -- 172.21.15.33:0/2251642894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576401dd20 con 0x5651ea72d7a0 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f57716ae640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f575c0a3ed0 0x7f575c0a6390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f576401f000 tx=0x7f576401d000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f5775dd0740 1 -- 172.21.15.33:0/2251642894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 msgr2=0x5651ea72db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f5775dd0740 1 --2- 172.21.15.33:0/2251642894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 0x5651ea72db80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5764012940 tx=0x7f5764012e00 comp rx=0 tx=0).stop 2024-04-02T13:39:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f5775dd0740 1 -- 172.21.15.33:0/2251642894 shutdown_connections 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f5775dd0740 1 --2- 172.21.15.33:0/2251642894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea736ca0 0x5651ea739090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f5775dd0740 1 --2- 172.21.15.33:0/2251642894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 0x5651ea736760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f5775dd0740 1 --2- 172.21.15.33:0/2251642894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 0x5651ea72db80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f5775dd0740 1 -- 172.21.15.33:0/2251642894 >> 172.21.15.33:0/2251642894 conn(0x5651ea727ed0 msgr2=0x5651ea7282b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f5775dd0740 1 -- 172.21.15.33:0/2251642894 shutdown_connections 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.019+0000 7f5775dd0740 1 -- 172.21.15.33:0/2251642894 wait complete. 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f5775dd0740 1 -- start start 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f5775dd0740 1 Processor -- start 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f5775dd0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 0x5651ea736710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f5775dd0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 0x5651ea733aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f5775dd0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea736ca0 0x5651ea7341e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f5775dd0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5651ea739a90 con 0x5651ea72d7a0 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f5775dd0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5651ea64ce80 con 0x5651ea736ca0 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f5775dd0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5651ea739c10 con 0x5651ea72e0c0 2024-04-02T13:39:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f5775dd0740 10 client.0 _ll_register_callbacks cb 0x5651ea688aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f5770ead640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 0x5651ea736710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f5770ead640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 0x5651ea736710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55002/0 (socket says 172.21.15.33:55002) 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f5770ead640 1 -- 172.21.15.33:0/1836947035 learned_addr learned my addr 172.21.15.33:0/1836947035 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f576bfff640 1 --2- 172.21.15.33:0/1836947035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 0x5651ea733aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.020+0000 7f57716ae640 1 --2- 172.21.15.33:0/1836947035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea736ca0 0x5651ea7341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f5770ead640 1 -- 172.21.15.33:0/1836947035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 msgr2=0x5651ea733aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f5770ead640 1 --2- 172.21.15.33:0/1836947035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 0x5651ea733aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f5770ead640 1 -- 172.21.15.33:0/1836947035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea736ca0 msgr2=0x5651ea7341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f5770ead640 1 --2- 172.21.15.33:0/1836947035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea736ca0 0x5651ea7341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f5770ead640 1 -- 172.21.15.33:0/1836947035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651ea46f7d0 con 0x5651ea72d7a0 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f5770ead640 1 --2- 172.21.15.33:0/1836947035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 0x5651ea736710 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f57600bae90 tx=0x7f5760009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8722bbe3d8ff95 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f574bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f574bfff640 1 -- 172.21.15.33:0/1836947035 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57600d6020 con 0x5651ea72d7a0 2024-04-02T13:39:26.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f574bfff640 1 -- 172.21.15.33:0/1836947035 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f57600be920 con 0x5651ea72d7a0 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f5775dd0740 1 -- 172.21.15.33:0/1836947035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5764012250 con 0x5651ea72d7a0 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f574bfff640 1 -- 172.21.15.33:0/1836947035 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f57600dc070 con 0x5651ea72d7a0 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f5775dd0740 10 client.15768 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f5775dd0740 1 -- 172.21.15.33:0/1836947035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5651ea739a90 con 0x5651ea72d7a0 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f574bfff640 1 -- 172.21.15.33:0/1836947035 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5764012250 con 0x5651ea72d7a0 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f574bfff640 1 -- 172.21.15.33:0/1836947035 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f57600dd290 con 0x5651ea72d7a0 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.021+0000 7f5775dd0740 10 client.15768 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.022+0000 7f5775dd0740 10 client.15768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.022+0000 7f5775dd0740 20 client.15768 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.022+0000 7f5775dd0740 1 --2- 172.21.15.33:0/1836947035 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5651ea746f40 0x5651ea749330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.022+0000 7f5775dd0740 4 client.15768 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.022+0000 7f5775dd0740 1 -- 172.21.15.33:0/1836947035 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651ea69ecf0 con 0x5651ea746f40 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.022+0000 7f576bfff640 1 --2- 172.21.15.33:0/1836947035 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5651ea746f40 0x5651ea749330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.022+0000 7f574bfff640 10 client.15768 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.022+0000 7f576bfff640 1 --2- 172.21.15.33:0/1836947035 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5651ea746f40 0x5651ea749330 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db35335496a8c6ea server_cookie=54e32acf4893ec27 in_seq=0 out_seq=0 2024-04-02T13:39:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f574bfff640 1 -- 172.21.15.33:0/1836947035 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651ea69ecf0 con 0x5651ea746f40 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f574bfff640 10 client.15768 handle_command_reply: tid=0 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f5775dd0740 1 client.15768 shutdown 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f5775dd0740 1 -- 172.21.15.33:0/1836947035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 msgr2=0x5651ea736710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f5775dd0740 1 --2- 172.21.15.33:0/1836947035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 0x5651ea736710 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f57600bae90 tx=0x7f5760009560 comp rx=0 tx=0).stop 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f5775dd0740 1 -- 172.21.15.33:0/1836947035 shutdown_connections 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f5775dd0740 1 --2- 172.21.15.33:0/1836947035 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5651ea746f40 0x5651ea749330 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f5775dd0740 1 --2- 172.21.15.33:0/1836947035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea736ca0 0x5651ea7341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f5775dd0740 1 --2- 172.21.15.33:0/1836947035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 0x5651ea733aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f5775dd0740 1 --2- 172.21.15.33:0/1836947035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea72d7a0 0x5651ea736710 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f574bfff640 0 client.15768 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f5775dd0740 1 -- 172.21.15.33:0/1836947035 >> 172.21.15.33:0/1836947035 conn(0x5651ea727ed0 msgr2=0x5651ea729c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f5775dd0740 1 -- 172.21.15.33:0/1836947035 shutdown_connections 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f5775dd0740 1 -- 172.21.15.33:0/1836947035 wait complete. 2024-04-02T13:39:26.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.023+0000 7f5775dd0740 20 client.15768 trim_cache size 0 max 16384 2024-04-02T13:39:26.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f5775dd0740 1 Processor -- start 2024-04-02T13:39:26.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f5775dd0740 1 -- start start 2024-04-02T13:39:26.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f5775dd0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72d7a0 0x5651ea742950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f5775dd0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea72e0c0 0x5651ea732a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f5775dd0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 0x5651ea7361d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f5775dd0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1b17b0 con 0x5651ea736ca0 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f5775dd0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1af080 con 0x5651ea72e0c0 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f5775dd0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1aceb0 con 0x5651ea72d7a0 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f57716ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 0x5651ea7361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f57716ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 0x5651ea7361d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55018/0 (socket says 172.21.15.33:55018) 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f57716ae640 1 -- 172.21.15.33:0/1412854125 learned_addr learned my addr 172.21.15.33:0/1412854125 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f5770ead640 1 --2- 172.21.15.33:0/1412854125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72d7a0 0x5651ea742950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f57716ae640 1 -- 172.21.15.33:0/1412854125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72d7a0 msgr2=0x5651ea742950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f576bfff640 1 --2- 172.21.15.33:0/1412854125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea72e0c0 0x5651ea732a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f57716ae640 1 --2- 172.21.15.33:0/1412854125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72d7a0 0x5651ea742950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f57716ae640 1 -- 172.21.15.33:0/1412854125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea72e0c0 msgr2=0x5651ea732a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f57716ae640 1 --2- 172.21.15.33:0/1412854125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea72e0c0 0x5651ea732a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f57716ae640 1 -- 172.21.15.33:0/1412854125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651ea72def0 con 0x5651ea736ca0 2024-04-02T13:39:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f5770ead640 1 --2- 172.21.15.33:0/1412854125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72d7a0 0x5651ea742950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f576bfff640 1 --2- 172.21.15.33:0/1412854125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea72e0c0 0x5651ea732a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.037+0000 7f57716ae640 1 --2- 172.21.15.33:0/1412854125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 0x5651ea7361d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5764010740 tx=0x7f576401d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c49959a350c08 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.038+0000 7f57567fc640 1 -- 172.21.15.33:0/1412854125 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764022070 con 0x5651ea736ca0 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.038+0000 7f57567fc640 1 -- 172.21.15.33:0/1412854125 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5764012520 con 0x5651ea736ca0 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.038+0000 7f57567fc640 1 -- 172.21.15.33:0/1412854125 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764028420 con 0x5651ea736ca0 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.038+0000 7f5775dd0740 1 -- 172.21.15.33:0/1412854125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 msgr2=0x5651ea7361d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.038+0000 7f5775dd0740 1 --2- 172.21.15.33:0/1412854125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 0x5651ea7361d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5764010740 tx=0x7f576401d9e0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.038+0000 7f5775dd0740 1 -- 172.21.15.33:0/1412854125 shutdown_connections 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.038+0000 7f5775dd0740 1 --2- 172.21.15.33:0/1412854125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 0x5651ea7361d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.038+0000 7f5775dd0740 1 --2- 172.21.15.33:0/1412854125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea72e0c0 0x5651ea732a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.038+0000 7f5775dd0740 1 --2- 172.21.15.33:0/1412854125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72d7a0 0x5651ea742950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.038+0000 7f5775dd0740 1 -- 172.21.15.33:0/1412854125 >> 172.21.15.33:0/1412854125 conn(0x5651ea727ed0 msgr2=0x5651ea732370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.038+0000 7f5775dd0740 1 -- 172.21.15.33:0/1412854125 shutdown_connections 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.038+0000 7f5775dd0740 1 -- 172.21.15.33:0/1412854125 wait complete. 2024-04-02T13:39:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.038+0000 7f5775dd0740 1 -- start start 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f5775dd0740 1 Processor -- start 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f5775dd0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea72d7a0 0x5651ea7374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f5775dd0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 0x5651ea7379f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f5775dd0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 0x5651ea73de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f5775dd0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1ad540 con 0x5651ea736ca0 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f5775dd0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1ae9b0 con 0x5651ea72d7a0 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f5775dd0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f576c1adc10 con 0x5651ea72e0c0 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f5775dd0740 10 client.0 _ll_register_callbacks cb 0x7f57600ba7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f57716ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 0x5651ea73de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f57716ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 0x5651ea73de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55034/0 (socket says 172.21.15.33:55034) 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f57716ae640 1 -- 172.21.15.33:0/3754588038 learned_addr learned my addr 172.21.15.33:0/3754588038 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f576bfff640 1 --2- 172.21.15.33:0/3754588038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 0x5651ea7379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f5770ead640 1 --2- 172.21.15.33:0/3754588038 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea72d7a0 0x5651ea7374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f57716ae640 1 -- 172.21.15.33:0/3754588038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 msgr2=0x5651ea7379f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f57716ae640 1 --2- 172.21.15.33:0/3754588038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 0x5651ea7379f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f57716ae640 1 -- 172.21.15.33:0/3754588038 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea72d7a0 msgr2=0x5651ea7374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f57716ae640 1 --2- 172.21.15.33:0/3754588038 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea72d7a0 0x5651ea7374b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.039+0000 7f57716ae640 1 -- 172.21.15.33:0/3754588038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651ea73e3a0 con 0x5651ea736ca0 2024-04-02T13:39:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f57716ae640 1 --2- 172.21.15.33:0/3754588038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 0x5651ea73de60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5764005c70 tx=0x7f57640122d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0bad963536ad06 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f574bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f574bfff640 1 -- 172.21.15.33:0/3754588038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764022070 con 0x5651ea736ca0 2024-04-02T13:39:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f574bfff640 1 -- 172.21.15.33:0/3754588038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f576402e050 con 0x5651ea736ca0 2024-04-02T13:39:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f5775dd0740 1 -- 172.21.15.33:0/3754588038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5764012520 con 0x5651ea736ca0 2024-04-02T13:39:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f574bfff640 1 -- 172.21.15.33:0/3754588038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5764045050 con 0x5651ea736ca0 2024-04-02T13:39:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f5775dd0740 10 client.15774 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f5775dd0740 1 -- 172.21.15.33:0/3754588038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5651ea73f4e0 con 0x5651ea736ca0 2024-04-02T13:39:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f574bfff640 1 -- 172.21.15.33:0/3754588038 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5764012520 con 0x5651ea736ca0 2024-04-02T13:39:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f574bfff640 1 -- 172.21.15.33:0/3754588038 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5764028b00 con 0x5651ea736ca0 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f5775dd0740 10 client.15774 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f5775dd0740 10 client.15774 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f5775dd0740 20 client.15774 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f5775dd0740 1 --2- 172.21.15.33:0/3754588038 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5651ea7ad4d0 0x5651ea7af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f5775dd0740 4 client.15774 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f5775dd0740 1 -- 172.21.15.33:0/3754588038 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651ea699040 con 0x5651ea7ad4d0 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.040+0000 7f5770ead640 1 --2- 172.21.15.33:0/3754588038 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5651ea7ad4d0 0x5651ea7af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f5770ead640 1 --2- 172.21.15.33:0/3754588038 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5651ea7ad4d0 0x5651ea7af8c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=badd575547c28c server_cookie=413abd1969bfbdbb in_seq=0 out_seq=0 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f574bfff640 10 client.15774 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f574bfff640 1 -- 172.21.15.33:0/3754588038 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651ea699040 con 0x5651ea7ad4d0 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f574bfff640 10 client.15774 handle_command_reply: tid=0 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f5775dd0740 1 client.15774 shutdown 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f5775dd0740 1 -- 172.21.15.33:0/3754588038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 msgr2=0x5651ea73de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f5775dd0740 1 --2- 172.21.15.33:0/3754588038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 0x5651ea73de60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5764005c70 tx=0x7f57640122d0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f5775dd0740 1 -- 172.21.15.33:0/3754588038 shutdown_connections 2024-04-02T13:39:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f5775dd0740 1 --2- 172.21.15.33:0/3754588038 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5651ea7ad4d0 0x5651ea7af8c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f5775dd0740 1 --2- 172.21.15.33:0/3754588038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ea736ca0 0x5651ea73de60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f5775dd0740 1 --2- 172.21.15.33:0/3754588038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5651ea72e0c0 0x5651ea7379f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f574bfff640 0 client.15774 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:39:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f5775dd0740 1 --2- 172.21.15.33:0/3754588038 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5651ea72d7a0 0x5651ea7374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f5775dd0740 1 -- 172.21.15.33:0/3754588038 >> 172.21.15.33:0/3754588038 conn(0x5651ea727ed0 msgr2=0x5651ea7266a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f5775dd0740 1 -- 172.21.15.33:0/3754588038 shutdown_connections 2024-04-02T13:39:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f5775dd0740 1 -- 172.21.15.33:0/3754588038 wait complete. 2024-04-02T13:39:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.041+0000 7f5775dd0740 20 client.15774 trim_cache size 0 max 16384 2024-04-02T13:39:26.051 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:39:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.042+0000 7f5773111640 1 -- 172.21.15.33:0/3484575121 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f575c0a3ed0 msgr2=0x7f575c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.042+0000 7f5773111640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f575c0a3ed0 0x7f575c0a6390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f576401f000 tx=0x7f576401d000 comp rx=0 tx=0).stop 2024-04-02T13:39:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.042+0000 7f5773111640 1 -- 172.21.15.33:0/3484575121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c101510 msgr2=0x7f576c199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.042+0000 7f5773111640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c101510 0x7f576c199120 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5760009900 tx=0x7f576002cb80 comp rx=0 tx=0).stop 2024-04-02T13:39:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.042+0000 7f5773111640 1 -- 172.21.15.33:0/3484575121 shutdown_connections 2024-04-02T13:39:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.042+0000 7f5773111640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f575c0a3ed0 0x7f575c0a6390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.042+0000 7f5773111640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f576c1059a0 0x7f576c1a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.042+0000 7f5773111640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f576c101ed0 0x7f576c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.042+0000 7f5773111640 1 --2- 172.21.15.33:0/3484575121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f576c101510 0x7f576c199120 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.042+0000 7f5773111640 1 -- 172.21.15.33:0/3484575121 >> 172.21.15.33:0/3484575121 conn(0x7f576c075c60 msgr2=0x7f576c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.042+0000 7f5773111640 1 -- 172.21.15.33:0/3484575121 shutdown_connections 2024-04-02T13:39:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.042+0000 7f5773111640 1 -- 172.21.15.33:0/3484575121 wait complete. 2024-04-02T13:39:26.055 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:39:26.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.157+0000 7f5cfe422640 1 -- 172.21.15.33:0/453480951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf8106980 msgr2=0x7f5cf8106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.157+0000 7f5cfe422640 1 --2- 172.21.15.33:0/453480951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf8106980 0x7f5cf8106d60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009930 tx=0x7f5ce802bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:26.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.157+0000 7f5cfe422640 1 -- 172.21.15.33:0/453480951 shutdown_connections 2024-04-02T13:39:26.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.157+0000 7f5cfe422640 1 --2- 172.21.15.33:0/453480951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf8101330 0x7f5cf810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.157+0000 7f5cfe422640 1 --2- 172.21.15.33:0/453480951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5cf8100970 0x7f5cf8100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.157+0000 7f5cfe422640 1 --2- 172.21.15.33:0/453480951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf8106980 0x7f5cf8106d60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.157+0000 7f5cfe422640 1 -- 172.21.15.33:0/453480951 >> 172.21.15.33:0/453480951 conn(0x7f5cf8077b00 msgr2=0x7f5cf8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.157+0000 7f5cfe422640 1 -- 172.21.15.33:0/453480951 shutdown_connections 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.157+0000 7f5cfe422640 1 -- 172.21.15.33:0/453480951 wait complete. 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.158+0000 7f5cfe422640 1 Processor -- start 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.158+0000 7f5cfe422640 1 -- start start 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.158+0000 7f5cfe422640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5cf8100970 0x7f5cf81972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.158+0000 7f5cfe422640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf8101330 0x7f5cf81977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.158+0000 7f5cfe422640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf8106980 0x7f5cf819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.158+0000 7f5cfe422640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8110d30 con 0x7f5cf8101330 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.158+0000 7f5cfe422640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8110bb0 con 0x7f5cf8106980 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.158+0000 7f5cfe422640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8110eb0 con 0x7f5cf8100970 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.158+0000 7f5cf77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf8101330 0x7f5cf81977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.158+0000 7f5cf77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf8101330 0x7f5cf81977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55054/0 (socket says 172.21.15.33:55054) 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.158+0000 7f5cf77fe640 1 -- 172.21.15.33:0/3389705093 learned_addr learned my addr 172.21.15.33:0/3389705093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.159+0000 7f5cfc9bf640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf8106980 0x7f5cf819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.159+0000 7f5cf7fff640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5cf8100970 0x7f5cf81972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.159+0000 7f5cf77fe640 1 -- 172.21.15.33:0/3389705093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5cf8100970 msgr2=0x7f5cf81972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.159+0000 7f5cf77fe640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5cf8100970 0x7f5cf81972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.159+0000 7f5cf77fe640 1 -- 172.21.15.33:0/3389705093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf8106980 msgr2=0x7f5cf819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.159+0000 7f5cf77fe640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf8106980 0x7f5cf819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.159+0000 7f5cf77fe640 1 -- 172.21.15.33:0/3389705093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf819c2a0 con 0x7f5cf8101330 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.159+0000 7f5cf77fe640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf8101330 0x7f5cf81977f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00d950 tx=0x7f5cec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.159+0000 7f5cf57fa640 1 -- 172.21.15.33:0/3389705093 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec017070 con 0x7f5cf8101330 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.159+0000 7f5cfe422640 1 -- 172.21.15.33:0/3389705093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf819c590 con 0x7f5cf8101330 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.159+0000 7f5cf57fa640 1 -- 172.21.15.33:0/3389705093 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5cec009c30 con 0x7f5cf8101330 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.159+0000 7f5cf57fa640 1 -- 172.21.15.33:0/3389705093 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec00f6b0 con 0x7f5cf8101330 2024-04-02T13:39:26.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.159+0000 7f5cfe422640 1 -- 172.21.15.33:0/3389705093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf81a3e20 con 0x7f5cf8101330 2024-04-02T13:39:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.161+0000 7f5cf57fa640 1 -- 172.21.15.33:0/3389705093 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f5cec010a50 con 0x7f5cf8101330 2024-04-02T13:39:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5d01009740 1 Processor -- start 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5d01009740 1 -- start start 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cf57fa640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5ce00a3ed0 0x7f5ce00a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5d01009740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350d95b0 0x5649350d9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5d01009740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350e4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5d01009740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 0x5649350e72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5d01009740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56493503d7b0 con 0x5649350e4eb0 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5d01009740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564934fdc570 con 0x5649350d95b0 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5d01009740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564935039760 con 0x5649350d9f60 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cfc9bf640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5ce00a3ed0 0x7f5ce00a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cf57fa640 1 -- 172.21.15.33:0/3389705093 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5cec0d0af0 con 0x7f5cf8101330 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cfc9bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 0x5649350e72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350d95b0 0x5649350d9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cfc9bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 0x5649350e72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55060/0 (socket says 172.21.15.33:55060) 2024-04-02T13:39:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cfc9bf640 1 -- 172.21.15.33:0/3820741551 learned_addr learned my addr 172.21.15.33:0/3820741551 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cf77fe640 1 --2- 172.21.15.33:0/3820741551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350e4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cf7fff640 1 -- 172.21.15.33:0/3820741551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 msgr2=0x5649350e4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cf7fff640 1 --2- 172.21.15.33:0/3820741551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350e4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cf7fff640 1 -- 172.21.15.33:0/3820741551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 msgr2=0x5649350e72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cf7fff640 1 --2- 172.21.15.33:0/3820741551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 0x5649350e72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cf7fff640 1 -- 172.21.15.33:0/3820741551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564934d513f0 con 0x5649350d95b0 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.162+0000 7f5cfc9bf640 1 --2- 172.21.15.33:0/3820741551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 0x5649350e72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.163+0000 7f5cfc9bf640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5ce00a3ed0 0x7f5ce00a6390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009fd0 tx=0x7f5cf0009d00 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.163+0000 7f5cf7fff640 1 --2- 172.21.15.33:0/3820741551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350d95b0 0x5649350d9990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8036040 tx=0x7f5ce8005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d558a47e07cf71c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.163+0000 7f5cde7fc640 1 -- 172.21.15.33:0/3820741551 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8042070 con 0x5649350d95b0 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.163+0000 7f5cde7fc640 1 -- 172.21.15.33:0/3820741551 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5ce8005520 con 0x5649350d95b0 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.163+0000 7f5cde7fc640 1 -- 172.21.15.33:0/3820741551 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80046c0 con 0x5649350d95b0 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.163+0000 7f5d01009740 1 -- 172.21.15.33:0/3820741551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350d95b0 msgr2=0x5649350d9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.163+0000 7f5d01009740 1 --2- 172.21.15.33:0/3820741551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350d95b0 0x5649350d9990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8036040 tx=0x7f5ce8005dd0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 -- 172.21.15.33:0/3820741551 shutdown_connections 2024-04-02T13:39:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 --2- 172.21.15.33:0/3820741551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 0x5649350e72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 --2- 172.21.15.33:0/3820741551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350e4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 --2- 172.21.15.33:0/3820741551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350d95b0 0x5649350d9990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 -- 172.21.15.33:0/3820741551 >> 172.21.15.33:0/3820741551 conn(0x5649350d3ed0 msgr2=0x5649350d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 -- 172.21.15.33:0/3820741551 shutdown_connections 2024-04-02T13:39:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 -- 172.21.15.33:0/3820741551 wait complete. 2024-04-02T13:39:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 -- start start 2024-04-02T13:39:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 Processor -- start 2024-04-02T13:39:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350d95b0 0x5649350e48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350e1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 0x5649350e23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5649350e7c10 con 0x5649350e4eb0 2024-04-02T13:39:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564934ff8e80 con 0x5649350d95b0 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5649350e7d90 con 0x5649350d9f60 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5d01009740 10 client.0 _ll_register_callbacks cb 0x564935034aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.164+0000 7f5cfc9bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 0x5649350e23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350d95b0 0x5649350e48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cfc9bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 0x5649350e23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55062/0 (socket says 172.21.15.33:55062) 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cfc9bf640 1 -- 172.21.15.33:0/2993280701 learned_addr learned my addr 172.21.15.33:0/2993280701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cf77fe640 1 --2- 172.21.15.33:0/2993280701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350e1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cfc9bf640 1 -- 172.21.15.33:0/2993280701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 msgr2=0x5649350e1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cfc9bf640 1 --2- 172.21.15.33:0/2993280701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350e1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cfc9bf640 1 -- 172.21.15.33:0/2993280701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350d95b0 msgr2=0x5649350e48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cfc9bf640 1 --2- 172.21.15.33:0/2993280701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350d95b0 0x5649350e48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cfc9bf640 1 -- 172.21.15.33:0/2993280701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564934e1b7d0 con 0x5649350e4eb0 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cf77fe640 1 --2- 172.21.15.33:0/2993280701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350e1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cfc9bf640 1 --2- 172.21.15.33:0/2993280701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 0x5649350e23c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0010d10 tx=0x7f5cf0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254c46abd036d9a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cd77fe640 1 -- 172.21.15.33:0/2993280701 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0026070 con 0x5649350e4eb0 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5d01009740 1 -- 172.21.15.33:0/2993280701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce8005520 con 0x5649350e4eb0 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cd77fe640 1 -- 172.21.15.33:0/2993280701 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5cf0022420 con 0x5649350e4eb0 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cd77fe640 1 -- 172.21.15.33:0/2993280701 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5cf0032050 con 0x5649350e4eb0 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5d01009740 10 client.15786 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5d01009740 1 -- 172.21.15.33:0/2993280701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5649350f3cd0 con 0x5649350e4eb0 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.165+0000 7f5cd77fe640 1 -- 172.21.15.33:0/2993280701 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5ce8005520 con 0x5649350e4eb0 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.166+0000 7f5cd77fe640 1 -- 172.21.15.33:0/2993280701 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5cf001f070 con 0x5649350e4eb0 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.166+0000 7f5d01009740 10 client.15786 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.166+0000 7f5d01009740 10 client.15786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.166+0000 7f5d01009740 20 client.15786 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.166+0000 7f5d01009740 1 --2- 172.21.15.33:0/2993280701 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x5649350f50e0 0x5649350f7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.166+0000 7f5d01009740 4 client.15786 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.166+0000 7f5d01009740 1 -- 172.21.15.33:0/2993280701 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56493504acf0 con 0x5649350f50e0 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.166+0000 7f5cf7fff640 1 --2- 172.21.15.33:0/2993280701 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x5649350f50e0 0x5649350f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.166+0000 7f5cf7fff640 1 --2- 172.21.15.33:0/2993280701 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x5649350f50e0 0x5649350f7510 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c024c090596abf server_cookie=2d271b339039e900 in_seq=0 out_seq=0 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.166+0000 7f5cd77fe640 10 client.15786 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.167+0000 7f5cd77fe640 1 -- 172.21.15.33:0/2993280701 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56493504acf0 con 0x5649350f50e0 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.167+0000 7f5cd77fe640 10 client.15786 handle_command_reply: tid=0 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.168+0000 7f5d01009740 1 client.15786 shutdown 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.168+0000 7f5d01009740 1 -- 172.21.15.33:0/2993280701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 msgr2=0x5649350e23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.168+0000 7f5d01009740 1 --2- 172.21.15.33:0/2993280701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 0x5649350e23c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0010d10 tx=0x7f5cf0005bd0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.168+0000 7f5d01009740 1 -- 172.21.15.33:0/2993280701 shutdown_connections 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.168+0000 7f5d01009740 1 --2- 172.21.15.33:0/2993280701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350e4eb0 0x5649350e23c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.168+0000 7f5d01009740 1 --2- 172.21.15.33:0/2993280701 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x5649350f50e0 0x5649350f7510 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.168+0000 7f5d01009740 1 --2- 172.21.15.33:0/2993280701 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350e1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.168+0000 7f5cd77fe640 0 client.15786 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.168+0000 7f5d01009740 1 --2- 172.21.15.33:0/2993280701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350d95b0 0x5649350e48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.168+0000 7f5d01009740 1 -- 172.21.15.33:0/2993280701 >> 172.21.15.33:0/2993280701 conn(0x5649350d3ed0 msgr2=0x5649350d5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.168+0000 7f5d01009740 1 -- 172.21.15.33:0/2993280701 shutdown_connections 2024-04-02T13:39:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.168+0000 7f5d01009740 1 -- 172.21.15.33:0/2993280701 wait complete. 2024-04-02T13:39:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.168+0000 7f5d01009740 20 client.15786 trim_cache size 0 max 16384 2024-04-02T13:39:26.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.181+0000 7f5d01009740 1 Processor -- start 2024-04-02T13:39:26.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.181+0000 7f5d01009740 1 -- start start 2024-04-02T13:39:26.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.181+0000 7f5d01009740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 0x5649350f0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.181+0000 7f5d01009740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.181+0000 7f5d01009740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350e4eb0 0x5649350fd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.181+0000 7f5d01009740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81af610 con 0x5649350d95b0 2024-04-02T13:39:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.181+0000 7f5d01009740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81acee0 con 0x5649350e4eb0 2024-04-02T13:39:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.181+0000 7f5d01009740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81aad10 con 0x5649350d9f60 2024-04-02T13:39:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.181+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 0x5649350f0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.181+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 0x5649350f0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55064/0 (socket says 172.21.15.33:55064) 2024-04-02T13:39:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.181+0000 7f5cf7fff640 1 -- 172.21.15.33:0/303999192 learned_addr learned my addr 172.21.15.33:0/303999192 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.181+0000 7f5cfc9bf640 1 --2- 172.21.15.33:0/303999192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350e4eb0 0x5649350fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5cf7fff640 1 -- 172.21.15.33:0/303999192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 msgr2=0x5649350fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5cf7fff640 1 --2- 172.21.15.33:0/303999192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5cf7fff640 1 -- 172.21.15.33:0/303999192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350e4eb0 msgr2=0x5649350fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5cf7fff640 1 --2- 172.21.15.33:0/303999192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350e4eb0 0x5649350fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5cf7fff640 1 -- 172.21.15.33:0/303999192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649350d8890 con 0x5649350d95b0 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5cf7fff640 1 --2- 172.21.15.33:0/303999192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 0x5649350f0b10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8005940 tx=0x7f5ce8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a1830e1ac2c52f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5cddffb640 1 -- 172.21.15.33:0/303999192 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8042040 con 0x5649350d95b0 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5cddffb640 1 -- 172.21.15.33:0/303999192 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5ce802c9f0 con 0x5649350d95b0 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5cddffb640 1 -- 172.21.15.33:0/303999192 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8003e50 con 0x5649350d95b0 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5d01009740 1 -- 172.21.15.33:0/303999192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 msgr2=0x5649350f0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5d01009740 1 --2- 172.21.15.33:0/303999192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 0x5649350f0b10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8005940 tx=0x7f5ce8000bc0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5d01009740 1 -- 172.21.15.33:0/303999192 shutdown_connections 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5d01009740 1 --2- 172.21.15.33:0/303999192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350e4eb0 0x5649350fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5d01009740 1 --2- 172.21.15.33:0/303999192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5d01009740 1 --2- 172.21.15.33:0/303999192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 0x5649350f0b10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.182+0000 7f5d01009740 1 -- 172.21.15.33:0/303999192 >> 172.21.15.33:0/303999192 conn(0x5649350d3ed0 msgr2=0x5649350e0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5d01009740 1 -- 172.21.15.33:0/303999192 shutdown_connections 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5d01009740 1 -- 172.21.15.33:0/303999192 wait complete. 2024-04-02T13:39:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5d01009740 1 -- start start 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5d01009740 1 Processor -- start 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5d01009740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 0x5649350ed550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5d01009740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350de790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5d01009740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350e4eb0 0x5649350deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5d01009740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81ab3a0 con 0x5649350d95b0 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5d01009740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81ac810 con 0x5649350e4eb0 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5d01009740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81aba70 con 0x5649350d9f60 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5d01009740 10 client.0 _ll_register_callbacks cb 0x7f5cf0012890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 0x5649350ed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5cf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 0x5649350ed550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55072/0 (socket says 172.21.15.33:55072) 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5cf7fff640 1 -- 172.21.15.33:0/3950758441 learned_addr learned my addr 172.21.15.33:0/3950758441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5cf77fe640 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350de790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.183+0000 7f5cfc9bf640 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350e4eb0 0x5649350deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5cf7fff640 1 -- 172.21.15.33:0/3950758441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 msgr2=0x5649350de790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5cf7fff640 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350de790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5cf7fff640 1 -- 172.21.15.33:0/3950758441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350e4eb0 msgr2=0x5649350deed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5cf7fff640 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350e4eb0 0x5649350deed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5cf7fff640 1 -- 172.21.15.33:0/3950758441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649350eb010 con 0x5649350d95b0 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5cfc9bf640 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350e4eb0 0x5649350deed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5cf7fff640 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 0x5649350ed550 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8006010 tx=0x7f5ce8009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eec08d1d8beb622 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5cd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5cd77fe640 1 -- 172.21.15.33:0/3950758441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8042070 con 0x5649350d95b0 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5cd77fe640 1 -- 172.21.15.33:0/3950758441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5ce8003800 con 0x5649350d95b0 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5d01009740 1 -- 172.21.15.33:0/3950758441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce802c9f0 con 0x5649350d95b0 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5cd77fe640 1 -- 172.21.15.33:0/3950758441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5ce8003ad0 con 0x5649350d95b0 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5d01009740 10 client.15792 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5d01009740 1 -- 172.21.15.33:0/3950758441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56493515b740 con 0x5649350d95b0 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5cd77fe640 1 -- 172.21.15.33:0/3950758441 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5ce802c9f0 con 0x5649350d95b0 2024-04-02T13:39:26.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.184+0000 7f5cd77fe640 1 -- 172.21.15.33:0/3950758441 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5ce80046c0 con 0x5649350d95b0 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.185+0000 7f5d01009740 10 client.15792 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.185+0000 7f5d01009740 10 client.15792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.185+0000 7f5d01009740 20 client.15792 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.185+0000 7f5d01009740 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56493515ba30 0x56493515de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.185+0000 7f5d01009740 4 client.15792 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.185+0000 7f5d01009740 1 -- 172.21.15.33:0/3950758441 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564935045040 con 0x56493515ba30 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.185+0000 7f5cf77fe640 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56493515ba30 0x56493515de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.185+0000 7f5cd77fe640 10 client.15792 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.185+0000 7f5cf77fe640 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56493515ba30 0x56493515de20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dcb6a8fb456eb07 server_cookie=d112f4c3a2be31f5 in_seq=0 out_seq=0 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.185+0000 7f5cd77fe640 1 -- 172.21.15.33:0/3950758441 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564935045040 con 0x56493515ba30 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.185+0000 7f5cd77fe640 10 client.15792 handle_command_reply: tid=0 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.185+0000 7f5d01009740 1 client.15792 shutdown 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5d01009740 1 -- 172.21.15.33:0/3950758441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 msgr2=0x5649350ed550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5d01009740 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 0x5649350ed550 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8006010 tx=0x7f5ce8009230 comp rx=0 tx=0).stop 2024-04-02T13:39:26.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5d01009740 1 -- 172.21.15.33:0/3950758441 shutdown_connections 2024-04-02T13:39:26.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5d01009740 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649350e4eb0 0x5649350deed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5d01009740 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56493515ba30 0x56493515de20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5d01009740 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5649350d9f60 0x5649350de790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5d01009740 1 --2- 172.21.15.33:0/3950758441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649350d95b0 0x5649350ed550 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5cd77fe640 0 client.15792 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:39:26.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5d01009740 1 -- 172.21.15.33:0/3950758441 >> 172.21.15.33:0/3950758441 conn(0x5649350d3ed0 msgr2=0x5649350d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5d01009740 1 -- 172.21.15.33:0/3950758441 shutdown_connections 2024-04-02T13:39:26.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5d01009740 1 -- 172.21.15.33:0/3950758441 wait complete. 2024-04-02T13:39:26.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5d01009740 20 client.15792 trim_cache size 0 max 16384 2024-04-02T13:39:26.195 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:39:26.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5cfe422640 1 -- 172.21.15.33:0/3389705093 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5ce00a3ed0 msgr2=0x7f5ce00a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5cfe422640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5ce00a3ed0 0x7f5ce00a6390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009fd0 tx=0x7f5cf0009d00 comp rx=0 tx=0).stop 2024-04-02T13:39:26.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5cfe422640 1 -- 172.21.15.33:0/3389705093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf8101330 msgr2=0x7f5cf81977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5cfe422640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf8101330 0x7f5cf81977f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00d950 tx=0x7f5cec00de10 comp rx=0 tx=0).stop 2024-04-02T13:39:26.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5cfe422640 1 -- 172.21.15.33:0/3389705093 shutdown_connections 2024-04-02T13:39:26.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5cfe422640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5ce00a3ed0 0x7f5ce00a6390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5cfe422640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf8106980 0x7f5cf819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5cfe422640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf8101330 0x7f5cf81977f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5cfe422640 1 --2- 172.21.15.33:0/3389705093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5cf8100970 0x7f5cf81972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5cfe422640 1 -- 172.21.15.33:0/3389705093 >> 172.21.15.33:0/3389705093 conn(0x7f5cf8077b00 msgr2=0x7f5cf8103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5cfe422640 1 -- 172.21.15.33:0/3389705093 shutdown_connections 2024-04-02T13:39:26.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.186+0000 7f5cfe422640 1 -- 172.21.15.33:0/3389705093 wait complete. 2024-04-02T13:39:26.198 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:25 smithi033 ceph-mon[32390]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-02T13:39:26.198 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:25 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3702883747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:39:26.200 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:39:26.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:25 smithi145 ceph-mon[35646]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-02T13:39:26.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:25 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3702883747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:39:26.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:25 smithi050 ceph-mon[35524]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-02T13:39:26.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:25 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3702883747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:39:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.301+0000 7f38c34ed640 1 -- 172.21.15.33:0/607969529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38bc101020 msgr2=0x7f38bc106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.301+0000 7f38c34ed640 1 --2- 172.21.15.33:0/607969529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38bc101020 0x7f38bc106750 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009900 tx=0x7f38b002bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.301+0000 7f38c34ed640 1 -- 172.21.15.33:0/607969529 shutdown_connections 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.301+0000 7f38c34ed640 1 --2- 172.21.15.33:0/607969529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38bc106e30 0x7f38bc10ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.301+0000 7f38c34ed640 1 --2- 172.21.15.33:0/607969529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38bc101020 0x7f38bc106750 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.301+0000 7f38c34ed640 1 --2- 172.21.15.33:0/607969529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38bc100670 0x7f38bc100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.301+0000 7f38c34ed640 1 -- 172.21.15.33:0/607969529 >> 172.21.15.33:0/607969529 conn(0x7f38bc075c60 msgr2=0x7f38bc076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c34ed640 1 -- 172.21.15.33:0/607969529 shutdown_connections 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c34ed640 1 -- 172.21.15.33:0/607969529 wait complete. 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c34ed640 1 Processor -- start 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c34ed640 1 -- start start 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c34ed640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38bc100670 0x7f38bc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c34ed640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38bc106e30 0x7f38bc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c34ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38bc19cd30 0x7f38bc19d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c34ed640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc111ef0 con 0x7f38bc19cd30 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c34ed640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc111d70 con 0x7f38bc106e30 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c34ed640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc112070 con 0x7f38bc100670 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c1a8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38bc19cd30 0x7f38bc19d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c1a8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38bc19cd30 0x7f38bc19d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55090/0 (socket says 172.21.15.33:55090) 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c1a8a640 1 -- 172.21.15.33:0/3292412536 learned_addr learned my addr 172.21.15.33:0/3292412536 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c0a88640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38bc106e30 0x7f38bc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.302+0000 7f38c1289640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38bc100670 0x7f38bc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.303+0000 7f38c1a8a640 1 -- 172.21.15.33:0/3292412536 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38bc100670 msgr2=0x7f38bc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.303+0000 7f38c1a8a640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38bc100670 0x7f38bc198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.303+0000 7f38c1a8a640 1 -- 172.21.15.33:0/3292412536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38bc106e30 msgr2=0x7f38bc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.303+0000 7f38c1a8a640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38bc106e30 0x7f38bc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.303+0000 7f38c1a8a640 1 -- 172.21.15.33:0/3292412536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc19d820 con 0x7f38bc19cd30 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.303+0000 7f38c1a8a640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38bc19cd30 0x7f38bc19d1e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00eb10 tx=0x7f38ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.303+0000 7f38ba7fc640 1 -- 172.21.15.33:0/3292412536 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac004b20 con 0x7f38bc19cd30 2024-04-02T13:39:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.303+0000 7f38ba7fc640 1 -- 172.21.15.33:0/3292412536 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f38ac004cc0 con 0x7f38bc19cd30 2024-04-02T13:39:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.303+0000 7f38c34ed640 1 -- 172.21.15.33:0/3292412536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc19db10 con 0x7f38bc19cd30 2024-04-02T13:39:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.303+0000 7f38ba7fc640 1 -- 172.21.15.33:0/3292412536 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac010700 con 0x7f38bc19cd30 2024-04-02T13:39:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.303+0000 7f38c34ed640 1 -- 172.21.15.33:0/3292412536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc1a52d0 con 0x7f38bc19cd30 2024-04-02T13:39:26.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.306+0000 7f38c61a8740 1 Processor -- start 2024-04-02T13:39:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.306+0000 7f38ba7fc640 1 -- 172.21.15.33:0/3292412536 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f38ac004e70 con 0x7f38bc19cd30 2024-04-02T13:39:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.306+0000 7f38c61a8740 1 -- start start 2024-04-02T13:39:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.306+0000 7f38c61a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd47a0 0x558408dd4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.306+0000 7f38c61a8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd50c0 0x558408ddd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.306+0000 7f38c61a8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 0x558408de0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.306+0000 7f38c61a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558408d387b0 con 0x558408dd47a0 2024-04-02T13:39:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.306+0000 7f38c61a8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558408cd7570 con 0x558408dd50c0 2024-04-02T13:39:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.306+0000 7f38c61a8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558408d34760 con 0x558408dddca0 2024-04-02T13:39:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38c0a88640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd47a0 0x558408dd4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38c0a88640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd47a0 0x558408dd4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55106/0 (socket says 172.21.15.33:55106) 2024-04-02T13:39:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38c0a88640 1 -- 172.21.15.33:0/4083814555 learned_addr learned my addr 172.21.15.33:0/4083814555 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38c1289640 1 --2- 172.21.15.33:0/4083814555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd50c0 0x558408ddd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38ba7fc640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f38a40a3f50 0x7f38a40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38c0a88640 1 -- 172.21.15.33:0/4083814555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 msgr2=0x558408de0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38c0a88640 1 --2- 172.21.15.33:0/4083814555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 0x558408de0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38c0a88640 1 -- 172.21.15.33:0/4083814555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd50c0 msgr2=0x558408ddd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38c0a88640 1 --2- 172.21.15.33:0/4083814555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd50c0 0x558408ddd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38c0a88640 1 -- 172.21.15.33:0/4083814555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558408a4c3f0 con 0x558408dd47a0 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38c1a8a640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f38a40a3f50 0x7f38a40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38ba7fc640 1 -- 172.21.15.33:0/3292412536 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f38ac0d1300 con 0x7f38bc19cd30 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38c0a88640 1 --2- 172.21.15.33:0/4083814555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd47a0 0x558408dd4b80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f38b0005ab0 tx=0x7f38b0009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7657c1ef1dc8745 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38a37fe640 1 -- 172.21.15.33:0/4083814555 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0044070 con 0x558408dd47a0 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38a37fe640 1 -- 172.21.15.33:0/4083814555 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f38b0005510 con 0x558408dd47a0 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.307+0000 7f38a37fe640 1 -- 172.21.15.33:0/4083814555 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b002cbd0 con 0x558408dd47a0 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.308+0000 7f38c61a8740 1 -- 172.21.15.33:0/4083814555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd47a0 msgr2=0x558408dd4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.308+0000 7f38c1a8a640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f38a40a3f50 0x7f38a40a6410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0ce500 tx=0x7f38ac06b6a0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.308+0000 7f38c61a8740 1 --2- 172.21.15.33:0/4083814555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd47a0 0x558408dd4b80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f38b0005ab0 tx=0x7f38b0009670 comp rx=0 tx=0).stop 2024-04-02T13:39:26.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.308+0000 7f38c61a8740 1 -- 172.21.15.33:0/4083814555 shutdown_connections 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.308+0000 7f38c61a8740 1 --2- 172.21.15.33:0/4083814555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 0x558408de0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.308+0000 7f38c61a8740 1 --2- 172.21.15.33:0/4083814555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd50c0 0x558408ddd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.308+0000 7f38c61a8740 1 --2- 172.21.15.33:0/4083814555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd47a0 0x558408dd4b80 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.308+0000 7f38c61a8740 1 -- 172.21.15.33:0/4083814555 >> 172.21.15.33:0/4083814555 conn(0x558408dceed0 msgr2=0x558408dcf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.308+0000 7f38c61a8740 1 -- 172.21.15.33:0/4083814555 shutdown_connections 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.308+0000 7f38c61a8740 1 -- 172.21.15.33:0/4083814555 wait complete. 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.308+0000 7f38c61a8740 1 -- start start 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.308+0000 7f38c61a8740 1 Processor -- start 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c61a8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd47a0 0x558408ddab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c61a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408ddb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c61a8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 0x558408ddb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c61a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558408de0a30 con 0x558408dd50c0 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c61a8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558408cf3e80 con 0x558408dd47a0 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c61a8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558408de0bb0 con 0x558408dddca0 2024-04-02T13:39:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c61a8740 10 client.0 _ll_register_callbacks cb 0x558408d2faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c1289640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408ddb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c1289640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408ddb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55112/0 (socket says 172.21.15.33:55112) 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c1289640 1 -- 172.21.15.33:0/700732480 learned_addr learned my addr 172.21.15.33:0/700732480 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c0a88640 1 --2- 172.21.15.33:0/700732480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 0x558408ddb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c0a88640 1 --2- 172.21.15.33:0/700732480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd47a0 0x558408ddab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c1289640 1 -- 172.21.15.33:0/700732480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 msgr2=0x558408ddb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c1289640 1 --2- 172.21.15.33:0/700732480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 0x558408ddb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c1289640 1 -- 172.21.15.33:0/700732480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd47a0 msgr2=0x558408ddab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c1289640 1 --2- 172.21.15.33:0/700732480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd47a0 0x558408ddab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c1289640 1 -- 172.21.15.33:0/700732480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558408b167d0 con 0x558408dd50c0 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38c1289640 1 --2- 172.21.15.33:0/700732480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408ddb040 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f38b4012790 tx=0x7f38b4012c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c997bf606ccf12ab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.309+0000 7f38a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38a0ff9640 1 -- 172.21.15.33:0/700732480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b401f060 con 0x558408dd50c0 2024-04-02T13:39:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38a0ff9640 1 -- 172.21.15.33:0/700732480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f38b4010dc0 con 0x558408dd50c0 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38c61a8740 1 -- 172.21.15.33:0/700732480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b0005510 con 0x558408dd50c0 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38a0ff9640 1 -- 172.21.15.33:0/700732480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f38b4005020 con 0x558408dd50c0 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38c61a8740 10 client.15804 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38c61a8740 1 -- 172.21.15.33:0/700732480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558408decb60 con 0x558408dd50c0 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38a0ff9640 1 -- 172.21.15.33:0/700732480 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f38b0005510 con 0x558408dd50c0 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38a0ff9640 1 -- 172.21.15.33:0/700732480 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f38b401a430 con 0x558408dd50c0 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38c61a8740 10 client.15804 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38c61a8740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38c61a8740 20 client.15804 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38c61a8740 1 --2- 172.21.15.33:0/700732480 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558408dee020 0x558408df0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38c61a8740 4 client.15804 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38c61a8740 1 -- 172.21.15.33:0/700732480 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558408d45cf0 con 0x558408dee020 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.310+0000 7f38c0a88640 1 --2- 172.21.15.33:0/700732480 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558408dee020 0x558408df0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.311+0000 7f38c0a88640 1 --2- 172.21.15.33:0/700732480 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558408dee020 0x558408df0450 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc06057e8e90b3d5 server_cookie=27500cff53caea55 in_seq=0 out_seq=0 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.311+0000 7f38a0ff9640 10 client.15804 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:26.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.311+0000 7f38a0ff9640 1 -- 172.21.15.33:0/700732480 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558408d45cf0 con 0x558408dee020 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.311+0000 7f38a0ff9640 10 client.15804 handle_command_reply: tid=0 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.312+0000 7f38c61a8740 1 client.15804 shutdown 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.312+0000 7f38c61a8740 1 -- 172.21.15.33:0/700732480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 msgr2=0x558408ddb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.312+0000 7f38c61a8740 1 --2- 172.21.15.33:0/700732480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408ddb040 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f38b4012790 tx=0x7f38b4012c50 comp rx=0 tx=0).stop 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.312+0000 7f38c61a8740 1 -- 172.21.15.33:0/700732480 shutdown_connections 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.312+0000 7f38c61a8740 1 --2- 172.21.15.33:0/700732480 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558408dee020 0x558408df0450 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.312+0000 7f38c61a8740 1 --2- 172.21.15.33:0/700732480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 0x558408ddb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.312+0000 7f38c61a8740 1 --2- 172.21.15.33:0/700732480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408ddb040 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.312+0000 7f38a0ff9640 0 client.15804 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.312+0000 7f38c61a8740 1 --2- 172.21.15.33:0/700732480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd47a0 0x558408ddab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.312+0000 7f38c61a8740 1 -- 172.21.15.33:0/700732480 >> 172.21.15.33:0/700732480 conn(0x558408dceed0 msgr2=0x558408dd0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.312+0000 7f38c61a8740 1 -- 172.21.15.33:0/700732480 shutdown_connections 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.312+0000 7f38c61a8740 1 -- 172.21.15.33:0/700732480 wait complete. 2024-04-02T13:39:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.312+0000 7f38c61a8740 20 client.15804 trim_cache size 0 max 16384 2024-04-02T13:39:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c61a8740 1 Processor -- start 2024-04-02T13:39:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c61a8740 1 -- start start 2024-04-02T13:39:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c61a8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dd47a0 0x558408ddd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c61a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408de98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c61a8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dddca0 0x558408de61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c61a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc1b16c0 con 0x558408dd50c0 2024-04-02T13:39:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c61a8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc1aef90 con 0x558408dddca0 2024-04-02T13:39:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c61a8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc1acdc0 con 0x558408dd47a0 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c1289640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408de98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c1289640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408de98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55126/0 (socket says 172.21.15.33:55126) 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c1289640 1 -- 172.21.15.33:0/3694732078 learned_addr learned my addr 172.21.15.33:0/3694732078 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c0a88640 1 --2- 172.21.15.33:0/3694732078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dd47a0 0x558408ddd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c1289640 1 -- 172.21.15.33:0/3694732078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dd47a0 msgr2=0x558408ddd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c1289640 1 --2- 172.21.15.33:0/3694732078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dd47a0 0x558408ddd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c0a88640 1 --2- 172.21.15.33:0/3694732078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dddca0 0x558408de61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c1289640 1 -- 172.21.15.33:0/3694732078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dddca0 msgr2=0x558408de61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.325+0000 7f38c1289640 1 --2- 172.21.15.33:0/3694732078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dddca0 0x558408de61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38c1289640 1 -- 172.21.15.33:0/3694732078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558408dd9050 con 0x558408dd50c0 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38c0a88640 1 --2- 172.21.15.33:0/3694732078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dddca0 0x558408de61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38c1289640 1 --2- 172.21.15.33:0/3694732078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408de98f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f38b401aa40 tx=0x7f38b400af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0ef3d36f1a81ad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38a2ffd640 1 -- 172.21.15.33:0/3694732078 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b401acc0 con 0x558408dd50c0 2024-04-02T13:39:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38a2ffd640 1 -- 172.21.15.33:0/3694732078 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f38b4005690 con 0x558408dd50c0 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38a2ffd640 1 -- 172.21.15.33:0/3694732078 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b40048c0 con 0x558408dd50c0 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38c61a8740 1 -- 172.21.15.33:0/3694732078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 msgr2=0x558408de98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38c61a8740 1 --2- 172.21.15.33:0/3694732078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408de98f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f38b401aa40 tx=0x7f38b400af30 comp rx=0 tx=0).stop 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38c61a8740 1 -- 172.21.15.33:0/3694732078 shutdown_connections 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38c61a8740 1 --2- 172.21.15.33:0/3694732078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dddca0 0x558408de61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38c61a8740 1 --2- 172.21.15.33:0/3694732078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408de98f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38c61a8740 1 --2- 172.21.15.33:0/3694732078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dd47a0 0x558408ddd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38c61a8740 1 -- 172.21.15.33:0/3694732078 >> 172.21.15.33:0/3694732078 conn(0x558408dceed0 msgr2=0x558408dd0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38c61a8740 1 -- 172.21.15.33:0/3694732078 shutdown_connections 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.326+0000 7f38c61a8740 1 -- 172.21.15.33:0/3694732078 wait complete. 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c61a8740 1 -- start start 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c61a8740 1 Processor -- start 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c61a8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd47a0 0x558408e4c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c61a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408e4ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c61a8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 0x558408e515e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c61a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc1ad450 con 0x558408dd50c0 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c61a8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc1ae8c0 con 0x558408dd47a0 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c61a8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc1adb20 con 0x558408dddca0 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c61a8740 10 client.0 _ll_register_callbacks cb 0x7f38b400a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c1289640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408e4ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c1289640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408e4ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55128/0 (socket says 172.21.15.33:55128) 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c1289640 1 -- 172.21.15.33:0/2050199784 learned_addr learned my addr 172.21.15.33:0/2050199784 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c0a88640 1 --2- 172.21.15.33:0/2050199784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 0x558408e515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c1289640 1 -- 172.21.15.33:0/2050199784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 msgr2=0x558408e515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c1289640 1 --2- 172.21.15.33:0/2050199784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 0x558408e515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c1289640 1 -- 172.21.15.33:0/2050199784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd47a0 msgr2=0x558408e4c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c1289640 1 --2- 172.21.15.33:0/2050199784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd47a0 0x558408e4c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c1289640 1 -- 172.21.15.33:0/2050199784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558408e51c20 con 0x558408dd50c0 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.327+0000 7f38c0a88640 1 --2- 172.21.15.33:0/2050199784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 0x558408e515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38c1289640 1 --2- 172.21.15.33:0/2050199784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408e4ccc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f38b401a8d0 tx=0x7f38b4010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47cfc68014ce3fa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38a0ff9640 1 -- 172.21.15.33:0/2050199784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b403f060 con 0x558408dd50c0 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38a0ff9640 1 -- 172.21.15.33:0/2050199784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f38b401aa70 con 0x558408dd50c0 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38c61a8740 1 -- 172.21.15.33:0/2050199784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b4005690 con 0x558408dd50c0 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38a0ff9640 1 -- 172.21.15.33:0/2050199784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f38b4046050 con 0x558408dd50c0 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38c61a8740 10 client.15810 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38c61a8740 1 -- 172.21.15.33:0/2050199784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558408e58f00 con 0x558408dd50c0 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38a0ff9640 1 -- 172.21.15.33:0/2050199784 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f38b4005690 con 0x558408dd50c0 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38a0ff9640 1 -- 172.21.15.33:0/2050199784 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f38b402a420 con 0x558408dd50c0 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38c61a8740 10 client.15810 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38c61a8740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38c61a8740 20 client.15810 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38c61a8740 1 --2- 172.21.15.33:0/2050199784 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558408e59590 0x558408e5b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38c61a8740 4 client.15810 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38c61a8740 1 -- 172.21.15.33:0/2050199784 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558408d40040 con 0x558408e59590 2024-04-02T13:39:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.328+0000 7f38c0a88640 1 --2- 172.21.15.33:0/2050199784 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558408e59590 0x558408e5b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38a0ff9640 10 client.15810 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38c0a88640 1 --2- 172.21.15.33:0/2050199784 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558408e59590 0x558408e5b9b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f8ceb43fb4851b server_cookie=44b7848dc5e2483 in_seq=0 out_seq=0 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38a0ff9640 1 -- 172.21.15.33:0/2050199784 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558408d40040 con 0x558408e59590 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38a0ff9640 10 client.15810 handle_command_reply: tid=0 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38c61a8740 1 client.15810 shutdown 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38c61a8740 1 -- 172.21.15.33:0/2050199784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 msgr2=0x558408e4ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38c61a8740 1 --2- 172.21.15.33:0/2050199784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408e4ccc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f38b401a8d0 tx=0x7f38b4010e70 comp rx=0 tx=0).stop 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38c61a8740 1 -- 172.21.15.33:0/2050199784 shutdown_connections 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38c61a8740 1 --2- 172.21.15.33:0/2050199784 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558408e59590 0x558408e5b9b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38c61a8740 1 --2- 172.21.15.33:0/2050199784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558408dddca0 0x558408e515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38c61a8740 1 --2- 172.21.15.33:0/2050199784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558408dd50c0 0x558408e4ccc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38c61a8740 1 --2- 172.21.15.33:0/2050199784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558408dd47a0 0x558408e4c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38a0ff9640 0 client.15810 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38c61a8740 1 -- 172.21.15.33:0/2050199784 >> 172.21.15.33:0/2050199784 conn(0x558408dceed0 msgr2=0x558408ddaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38c61a8740 1 -- 172.21.15.33:0/2050199784 shutdown_connections 2024-04-02T13:39:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38c61a8740 1 -- 172.21.15.33:0/2050199784 wait complete. 2024-04-02T13:39:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.329+0000 7f38c61a8740 20 client.15810 trim_cache size 0 max 16384 2024-04-02T13:39:26.339 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:39:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.330+0000 7f38c34ed640 1 -- 172.21.15.33:0/3292412536 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f38a40a3f50 msgr2=0x7f38a40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.330+0000 7f38c34ed640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f38a40a3f50 0x7f38a40a6410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0ce500 tx=0x7f38ac06b6a0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.330+0000 7f38c34ed640 1 -- 172.21.15.33:0/3292412536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38bc19cd30 msgr2=0x7f38bc19d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.330+0000 7f38c34ed640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38bc19cd30 0x7f38bc19d1e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00eb10 tx=0x7f38ac00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.330+0000 7f38c34ed640 1 -- 172.21.15.33:0/3292412536 shutdown_connections 2024-04-02T13:39:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.330+0000 7f38c34ed640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f38a40a3f50 0x7f38a40a6410 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.330+0000 7f38c34ed640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38bc19cd30 0x7f38bc19d1e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.330+0000 7f38c34ed640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38bc106e30 0x7f38bc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.330+0000 7f38c34ed640 1 --2- 172.21.15.33:0/3292412536 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38bc100670 0x7f38bc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.330+0000 7f38c34ed640 1 -- 172.21.15.33:0/3292412536 >> 172.21.15.33:0/3292412536 conn(0x7f38bc075c60 msgr2=0x7f38bc0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.330+0000 7f38c34ed640 1 -- 172.21.15.33:0/3292412536 shutdown_connections 2024-04-02T13:39:26.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:26.330+0000 7f38c34ed640 1 -- 172.21.15.33:0/3292412536 wait complete. 2024-04-02T13:39:27.343 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:39:27.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:39:27.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:39:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.461+0000 7effdea0d640 1 -- 172.21.15.33:0/2508770506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd81002e0 msgr2=0x7effd81006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.461+0000 7effdea0d640 1 --2- 172.21.15.33:0/2508770506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd81002e0 0x7effd81006c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7effc8009900 tx=0x7effc802bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.461+0000 7effdea0d640 1 -- 172.21.15.33:0/2508770506 shutdown_connections 2024-04-02T13:39:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.461+0000 7effdea0d640 1 --2- 172.21.15.33:0/2508770506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effd8106ef0 0x7effd810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.461+0000 7effdea0d640 1 --2- 172.21.15.33:0/2508770506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8100c90 0x7effd81067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.461+0000 7effdea0d640 1 --2- 172.21.15.33:0/2508770506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd81002e0 0x7effd81006c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.461+0000 7effdea0d640 1 -- 172.21.15.33:0/2508770506 >> 172.21.15.33:0/2508770506 conn(0x7effd8077030 msgr2=0x7effd80fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.461+0000 7effdea0d640 1 -- 172.21.15.33:0/2508770506 shutdown_connections 2024-04-02T13:39:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.462+0000 7effdea0d640 1 -- 172.21.15.33:0/2508770506 wait complete. 2024-04-02T13:39:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.462+0000 7effdea0d640 1 Processor -- start 2024-04-02T13:39:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.462+0000 7effdea0d640 1 -- start start 2024-04-02T13:39:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effdea0d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effd81002e0 0x7effd8101250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effdea0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd8106ef0 0x7effd8101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effdea0d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8101cd0 0x7effd81a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effdea0d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7effd8111f40 con 0x7effd8106ef0 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effdea0d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7effd8111dc0 con 0x7effd81002e0 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effdea0d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7effd81120c0 con 0x7effd8101cd0 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effd77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd8106ef0 0x7effd8101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effd77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd8106ef0 0x7effd8101790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55154/0 (socket says 172.21.15.33:55154) 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effd77fe640 1 -- 172.21.15.33:0/2647225813 learned_addr learned my addr 172.21.15.33:0/2647225813 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effd7fff640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effd81002e0 0x7effd8101250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effdcfaa640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8101cd0 0x7effd81a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effd77fe640 1 -- 172.21.15.33:0/2647225813 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8101cd0 msgr2=0x7effd81a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effd77fe640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8101cd0 0x7effd81a6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effd77fe640 1 -- 172.21.15.33:0/2647225813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effd81002e0 msgr2=0x7effd8101250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effd77fe640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effd81002e0 0x7effd8101250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effd77fe640 1 -- 172.21.15.33:0/2647225813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd81a6d70 con 0x7effd8106ef0 2024-04-02T13:39:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effd77fe640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd8106ef0 0x7effd8101790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7effcc00c970 tx=0x7effcc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effd57fa640 1 -- 172.21.15.33:0/2647225813 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc004220 con 0x7effd8106ef0 2024-04-02T13:39:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effd57fa640 1 -- 172.21.15.33:0/2647225813 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7effcc0043c0 con 0x7effd8106ef0 2024-04-02T13:39:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effdea0d640 1 -- 172.21.15.33:0/2647225813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd81a7060 con 0x7effd8106ef0 2024-04-02T13:39:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.463+0000 7effd57fa640 1 -- 172.21.15.33:0/2647225813 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc004a00 con 0x7effd8106ef0 2024-04-02T13:39:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.464+0000 7effdea0d640 1 -- 172.21.15.33:0/2647225813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd81a7560 con 0x7effd8106ef0 2024-04-02T13:39:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effd57fa640 1 -- 172.21.15.33:0/2647225813 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7effcc0079b0 con 0x7effd8106ef0 2024-04-02T13:39:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effe15f3740 1 Processor -- start 2024-04-02T13:39:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effe15f3740 1 -- start start 2024-04-02T13:39:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effe15f3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0c2b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effe15f3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0c34280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effe15f3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c36bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effe15f3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0b8f770 con 0x55a4c0c347c0 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effe15f3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0b37960 con 0x55a4c0c2b2c0 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effe15f3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0b8b8c0 con 0x55a4c0c2bbe0 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effd57fa640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7effc00a3f00 0x7effc00a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effd77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c36bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effdcfaa640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0c2b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effd77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c36bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55158/0 (socket says 172.21.15.33:55158) 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.466+0000 7effd77fe640 1 -- 172.21.15.33:0/1938156106 learned_addr learned my addr 172.21.15.33:0/1938156106 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effdcfaa640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7effc00a3f00 0x7effc00a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effd57fa640 1 -- 172.21.15.33:0/2647225813 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7effcc0d0b70 con 0x7effd8106ef0 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effd7fff640 1 --2- 172.21.15.33:0/1938156106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0c34280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effd77fe640 1 -- 172.21.15.33:0/1938156106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 msgr2=0x55a4c0c34280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effd77fe640 1 --2- 172.21.15.33:0/1938156106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0c34280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effd77fe640 1 -- 172.21.15.33:0/1938156106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 msgr2=0x55a4c0c2b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effd77fe640 1 --2- 172.21.15.33:0/1938156106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0c2b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effd77fe640 1 -- 172.21.15.33:0/1938156106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4c096b7e0 con 0x55a4c0c347c0 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effd7fff640 1 --2- 172.21.15.33:0/1938156106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0c34280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effd77fe640 1 --2- 172.21.15.33:0/1938156106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c36bb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7effcc004770 tx=0x7effcc00a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a5221ce23328d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effdcfaa640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7effc00a3f00 0x7effc00a63c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7effd0009fd0 tx=0x7effd0009240 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effbe7fc640 1 -- 172.21.15.33:0/1938156106 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc01d040 con 0x55a4c0c347c0 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effbe7fc640 1 -- 172.21.15.33:0/1938156106 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7effcc09c560 con 0x55a4c0c347c0 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.467+0000 7effbe7fc640 1 -- 172.21.15.33:0/1938156106 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc098510 con 0x55a4c0c347c0 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.468+0000 7effe15f3740 1 -- 172.21.15.33:0/1938156106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 msgr2=0x55a4c0c36bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.468+0000 7effe15f3740 1 --2- 172.21.15.33:0/1938156106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c36bb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7effcc004770 tx=0x7effcc00a520 comp rx=0 tx=0).stop 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.468+0000 7effe15f3740 1 -- 172.21.15.33:0/1938156106 shutdown_connections 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.468+0000 7effe15f3740 1 --2- 172.21.15.33:0/1938156106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c36bb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.468+0000 7effe15f3740 1 --2- 172.21.15.33:0/1938156106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0c34280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.468+0000 7effe15f3740 1 --2- 172.21.15.33:0/1938156106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0c2b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.468+0000 7effe15f3740 1 -- 172.21.15.33:0/1938156106 >> 172.21.15.33:0/1938156106 conn(0x55a4c0c259f0 msgr2=0x55a4c0c25dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.468+0000 7effe15f3740 1 -- 172.21.15.33:0/1938156106 shutdown_connections 2024-04-02T13:39:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.468+0000 7effe15f3740 1 -- 172.21.15.33:0/1938156106 wait complete. 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.468+0000 7effe15f3740 1 -- start start 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.468+0000 7effe15f3740 1 Processor -- start 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effe15f3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0c31790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effe15f3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0c31cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effe15f3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c32210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effe15f3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0c374f0 con 0x55a4c0c347c0 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effe15f3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0b4ad20 con 0x55a4c0c2b2c0 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effe15f3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0c37670 con 0x55a4c0c2bbe0 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effe15f3740 10 client.0 _ll_register_callbacks cb 0x55a4c0b17a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effdcfaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c32210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effdcfaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c32210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55162/0 (socket says 172.21.15.33:55162) 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effdcfaa640 1 -- 172.21.15.33:0/3583818861 learned_addr learned my addr 172.21.15.33:0/3583818861 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effd7fff640 1 --2- 172.21.15.33:0/3583818861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0c31790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effd77fe640 1 --2- 172.21.15.33:0/3583818861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0c31cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effdcfaa640 1 -- 172.21.15.33:0/3583818861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 msgr2=0x55a4c0c31cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effdcfaa640 1 --2- 172.21.15.33:0/3583818861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0c31cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effdcfaa640 1 -- 172.21.15.33:0/3583818861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 msgr2=0x55a4c0c31790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effdcfaa640 1 --2- 172.21.15.33:0/3583818861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0c31790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effdcfaa640 1 -- 172.21.15.33:0/3583818861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4c09cbf70 con 0x55a4c0c347c0 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effdcfaa640 1 --2- 172.21.15.33:0/3583818861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c32210 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7effd0003ef0 tx=0x7effd0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a88a26e169ee2e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.469+0000 7effb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effb77fe640 1 -- 172.21.15.33:0/3583818861 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0026070 con 0x55a4c0c347c0 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effb77fe640 1 -- 172.21.15.33:0/3583818861 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7effd00047c0 con 0x55a4c0c347c0 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effe15f3740 1 -- 172.21.15.33:0/3583818861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effcc09c560 con 0x55a4c0c347c0 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effb77fe640 1 -- 172.21.15.33:0/3583818861 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7effd0031050 con 0x55a4c0c347c0 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effe15f3740 10 client.15822 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effe15f3740 1 -- 172.21.15.33:0/3583818861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a4c0863170 con 0x55a4c0c347c0 2024-04-02T13:39:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effb77fe640 1 -- 172.21.15.33:0/3583818861 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7effcc09c560 con 0x55a4c0c347c0 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effb77fe640 1 -- 172.21.15.33:0/3583818861 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7effd001d050 con 0x55a4c0c347c0 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effe15f3740 10 client.15822 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effe15f3740 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effe15f3740 20 client.15822 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effe15f3740 1 --2- 172.21.15.33:0/3583818861 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a4c0c449f0 0x55a4c0c46e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effe15f3740 4 client.15822 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effe15f3740 1 -- 172.21.15.33:0/3583818861 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4c0b9ccb0 con 0x55a4c0c449f0 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.470+0000 7effd7fff640 1 --2- 172.21.15.33:0/3583818861 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a4c0c449f0 0x55a4c0c46e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.471+0000 7effb77fe640 10 client.15822 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.471+0000 7effd7fff640 1 --2- 172.21.15.33:0/3583818861 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a4c0c449f0 0x55a4c0c46e60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=601811de981bb0a1 server_cookie=3837c5c8ca55c411 in_seq=0 out_seq=0 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.471+0000 7effb77fe640 1 -- 172.21.15.33:0/3583818861 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4c0b9ccb0 con 0x55a4c0c449f0 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.471+0000 7effb77fe640 10 client.15822 handle_command_reply: tid=0 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.471+0000 7effe15f3740 1 client.15822 shutdown 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.472+0000 7effe15f3740 1 -- 172.21.15.33:0/3583818861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 msgr2=0x55a4c0c32210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.472+0000 7effe15f3740 1 --2- 172.21.15.33:0/3583818861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c32210 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7effd0003ef0 tx=0x7effd0004ab0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.472+0000 7effe15f3740 1 -- 172.21.15.33:0/3583818861 shutdown_connections 2024-04-02T13:39:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.472+0000 7effe15f3740 1 --2- 172.21.15.33:0/3583818861 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a4c0c449f0 0x55a4c0c46e60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.472+0000 7effe15f3740 1 --2- 172.21.15.33:0/3583818861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c32210 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.472+0000 7effe15f3740 1 --2- 172.21.15.33:0/3583818861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0c31cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.472+0000 7effb77fe640 0 client.15822 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.472+0000 7effe15f3740 1 --2- 172.21.15.33:0/3583818861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0c31790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.472+0000 7effe15f3740 1 -- 172.21.15.33:0/3583818861 >> 172.21.15.33:0/3583818861 conn(0x55a4c0c259f0 msgr2=0x55a4c0c27740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.472+0000 7effe15f3740 1 -- 172.21.15.33:0/3583818861 shutdown_connections 2024-04-02T13:39:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.472+0000 7effe15f3740 1 -- 172.21.15.33:0/3583818861 wait complete. 2024-04-02T13:39:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.472+0000 7effe15f3740 20 client.15822 trim_cache size 0 max 16384 2024-04-02T13:39:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effe15f3740 1 Processor -- start 2024-04-02T13:39:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effe15f3740 1 -- start start 2024-04-02T13:39:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effe15f3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0c33ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effe15f3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0c403e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effe15f3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c4c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effe15f3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7effd81b3ad0 con 0x55a4c0c347c0 2024-04-02T13:39:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effe15f3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7effd81b13a0 con 0x55a4c0c2b2c0 2024-04-02T13:39:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effe15f3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7effd81af1d0 con 0x55a4c0c2bbe0 2024-04-02T13:39:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effdcfaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c4c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effdcfaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c4c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55164/0 (socket says 172.21.15.33:55164) 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effdcfaa640 1 -- 172.21.15.33:0/2383061378 learned_addr learned my addr 172.21.15.33:0/2383061378 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effd77fe640 1 --2- 172.21.15.33:0/2383061378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0c403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effd7fff640 1 --2- 172.21.15.33:0/2383061378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0c33ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effdcfaa640 1 -- 172.21.15.33:0/2383061378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 msgr2=0x55a4c0c403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effdcfaa640 1 --2- 172.21.15.33:0/2383061378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0c403e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effdcfaa640 1 -- 172.21.15.33:0/2383061378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 msgr2=0x55a4c0c33ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effdcfaa640 1 --2- 172.21.15.33:0/2383061378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0c33ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effdcfaa640 1 -- 172.21.15.33:0/2383061378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4c0c2ba10 con 0x55a4c0c347c0 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.490+0000 7effdcfaa640 1 --2- 172.21.15.33:0/2383061378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c4c730 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7effd0014b10 tx=0x7effd0009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4588718feb8ed18c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.491+0000 7effbdffb640 1 -- 172.21.15.33:0/2383061378 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0039020 con 0x55a4c0c347c0 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.491+0000 7effbdffb640 1 -- 172.21.15.33:0/2383061378 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7effd000f910 con 0x55a4c0c347c0 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.491+0000 7effbdffb640 1 -- 172.21.15.33:0/2383061378 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0026070 con 0x55a4c0c347c0 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.491+0000 7effe15f3740 1 -- 172.21.15.33:0/2383061378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 msgr2=0x55a4c0c4c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.491+0000 7effe15f3740 1 --2- 172.21.15.33:0/2383061378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c4c730 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7effd0014b10 tx=0x7effd0009930 comp rx=0 tx=0).stop 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.491+0000 7effe15f3740 1 -- 172.21.15.33:0/2383061378 shutdown_connections 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.491+0000 7effe15f3740 1 --2- 172.21.15.33:0/2383061378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c347c0 0x55a4c0c4c730 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.491+0000 7effe15f3740 1 --2- 172.21.15.33:0/2383061378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0c403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.491+0000 7effe15f3740 1 --2- 172.21.15.33:0/2383061378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0c33ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.491+0000 7effe15f3740 1 -- 172.21.15.33:0/2383061378 >> 172.21.15.33:0/2383061378 conn(0x55a4c0c259f0 msgr2=0x55a4c0c24ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.491+0000 7effe15f3740 1 -- 172.21.15.33:0/2383061378 shutdown_connections 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.491+0000 7effe15f3740 1 -- 172.21.15.33:0/2383061378 wait complete. 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.491+0000 7effe15f3740 1 -- start start 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effe15f3740 1 Processor -- start 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effe15f3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0ca42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effe15f3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0ca47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effe15f3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c347c0 0x55a4c0c38b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effe15f3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7effd81af860 con 0x55a4c0c2bbe0 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effe15f3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7effd81b0cd0 con 0x55a4c0c2b2c0 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effe15f3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7effd81aff30 con 0x55a4c0c347c0 2024-04-02T13:39:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effe15f3740 10 client.0 _ll_register_callbacks cb 0x7effd0003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effd77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0ca47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effd77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0ca47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55166/0 (socket says 172.21.15.33:55166) 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effd77fe640 1 -- 172.21.15.33:0/2147805849 learned_addr learned my addr 172.21.15.33:0/2147805849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effd7fff640 1 --2- 172.21.15.33:0/2147805849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0ca42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effdcfaa640 1 --2- 172.21.15.33:0/2147805849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c347c0 0x55a4c0c38b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effd77fe640 1 -- 172.21.15.33:0/2147805849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c347c0 msgr2=0x55a4c0c38b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effd77fe640 1 --2- 172.21.15.33:0/2147805849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c347c0 0x55a4c0c38b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effd77fe640 1 -- 172.21.15.33:0/2147805849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 msgr2=0x55a4c0ca42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effd77fe640 1 --2- 172.21.15.33:0/2147805849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0ca42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.492+0000 7effd77fe640 1 -- 172.21.15.33:0/2147805849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4c0c39210 con 0x55a4c0c2bbe0 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effd77fe640 1 --2- 172.21.15.33:0/2147805849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0ca47f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7effcc0989c0 tx=0x7effcc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa598c9eb0dbd5e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effb77fe640 1 -- 172.21.15.33:0/2147805849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0b5060 con 0x55a4c0c2bbe0 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effb77fe640 1 -- 172.21.15.33:0/2147805849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7effcc09bbb0 con 0x55a4c0c2bbe0 2024-04-02T13:39:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effe15f3740 1 -- 172.21.15.33:0/2147805849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effd000f910 con 0x55a4c0c2bbe0 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effb77fe640 1 -- 172.21.15.33:0/2147805849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7effcc065030 con 0x55a4c0c2bbe0 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effe15f3740 10 client.15828 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effe15f3740 1 -- 172.21.15.33:0/2147805849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a4c0caba90 con 0x55a4c0c2bbe0 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effb77fe640 1 -- 172.21.15.33:0/2147805849 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7effd000f910 con 0x55a4c0c2bbe0 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effb77fe640 1 -- 172.21.15.33:0/2147805849 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7effcc0b5ce0 con 0x55a4c0c2bbe0 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effe15f3740 10 client.15828 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effe15f3740 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effe15f3740 20 client.15828 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effe15f3740 1 --2- 172.21.15.33:0/2147805849 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a4c0cabe20 0x55a4c0cae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effe15f3740 4 client.15828 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.493+0000 7effe15f3740 1 -- 172.21.15.33:0/2147805849 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a4c0b97000 con 0x55a4c0cabe20 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.494+0000 7effd7fff640 1 --2- 172.21.15.33:0/2147805849 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a4c0cabe20 0x55a4c0cae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.494+0000 7effb77fe640 10 client.15828 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.494+0000 7effd7fff640 1 --2- 172.21.15.33:0/2147805849 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a4c0cabe20 0x55a4c0cae240 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4603c4cfaace60e5 server_cookie=6e0ae0355fe1e916 in_seq=0 out_seq=0 2024-04-02T13:39:27.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.495+0000 7effb77fe640 1 -- 172.21.15.33:0/2147805849 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a4c0b97000 con 0x55a4c0cabe20 2024-04-02T13:39:27.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.495+0000 7effb77fe640 10 client.15828 handle_command_reply: tid=0 2024-04-02T13:39:27.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.495+0000 7effe15f3740 1 client.15828 shutdown 2024-04-02T13:39:27.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.495+0000 7effe15f3740 1 -- 172.21.15.33:0/2147805849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c2bbe0 msgr2=0x55a4c0ca47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.495+0000 7effe15f3740 1 --2- 172.21.15.33:0/2147805849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0ca47f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7effcc0989c0 tx=0x7effcc0058a0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.495+0000 7effe15f3740 1 -- 172.21.15.33:0/2147805849 shutdown_connections 2024-04-02T13:39:27.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.495+0000 7effe15f3740 1 --2- 172.21.15.33:0/2147805849 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a4c0cabe20 0x55a4c0cae240 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.495+0000 7effe15f3740 1 --2- 172.21.15.33:0/2147805849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a4c0c347c0 0x55a4c0c38b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.495+0000 7effe15f3740 1 --2- 172.21.15.33:0/2147805849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c0c2bbe0 0x55a4c0ca47f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.495+0000 7effe15f3740 1 --2- 172.21.15.33:0/2147805849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4c0c2b2c0 0x55a4c0ca42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.495+0000 7effe15f3740 1 -- 172.21.15.33:0/2147805849 >> 172.21.15.33:0/2147805849 conn(0x55a4c0c259f0 msgr2=0x55a4c0c24220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.495+0000 7effb77fe640 0 client.15828 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effe15f3740 1 -- 172.21.15.33:0/2147805849 shutdown_connections 2024-04-02T13:39:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effe15f3740 1 -- 172.21.15.33:0/2147805849 wait complete. 2024-04-02T13:39:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effe15f3740 20 client.15828 trim_cache size 0 max 16384 2024-04-02T13:39:27.504 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:39:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:39:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "0fd0b3c5-9bfd-4cf7-aff6-e563ae258e1d", 2024-04-02T13:39:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:39:27.505 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:39:27.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effdea0d640 1 -- 172.21.15.33:0/2647225813 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7effc00a3f00 msgr2=0x7effc00a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effdea0d640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7effc00a3f00 0x7effc00a63c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7effd0009fd0 tx=0x7effd0009240 comp rx=0 tx=0).stop 2024-04-02T13:39:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effdea0d640 1 -- 172.21.15.33:0/2647225813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd8106ef0 msgr2=0x7effd8101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effdea0d640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd8106ef0 0x7effd8101790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7effcc00c970 tx=0x7effcc00ce30 comp rx=0 tx=0).stop 2024-04-02T13:39:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effdea0d640 1 -- 172.21.15.33:0/2647225813 shutdown_connections 2024-04-02T13:39:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effdea0d640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7effc00a3f00 0x7effc00a63c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effdea0d640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8101cd0 0x7effd81a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effdea0d640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effd8106ef0 0x7effd8101790 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effdea0d640 1 --2- 172.21.15.33:0/2647225813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7effd81002e0 0x7effd8101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effdea0d640 1 -- 172.21.15.33:0/2647225813 >> 172.21.15.33:0/2647225813 conn(0x7effd8077030 msgr2=0x7effd80fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effdea0d640 1 -- 172.21.15.33:0/2647225813 shutdown_connections 2024-04-02T13:39:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.496+0000 7effdea0d640 1 -- 172.21.15.33:0/2647225813 wait complete. 2024-04-02T13:39:27.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:39:27.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:39:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.613+0000 7fbc3f767640 1 -- 172.21.15.33:0/2176977173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc381009b0 msgr2=0x7fbc38100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.613+0000 7fbc3f767640 1 --2- 172.21.15.33:0/2176977173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc381009b0 0x7fbc38100e30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009930 tx=0x7fbc2c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.613+0000 7fbc3f767640 1 -- 172.21.15.33:0/2176977173 shutdown_connections 2024-04-02T13:39:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.613+0000 7fbc3f767640 1 --2- 172.21.15.33:0/2176977173 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc38101370 0x7fbc3810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.613+0000 7fbc3f767640 1 --2- 172.21.15.33:0/2176977173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc381009b0 0x7fbc38100e30 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.613+0000 7fbc3f767640 1 --2- 172.21.15.33:0/2176977173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc381069c0 0x7fbc38106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.613+0000 7fbc3f767640 1 -- 172.21.15.33:0/2176977173 >> 172.21.15.33:0/2176977173 conn(0x7fbc38076d80 msgr2=0x7fbc38077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.613+0000 7fbc3f767640 1 -- 172.21.15.33:0/2176977173 shutdown_connections 2024-04-02T13:39:27.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.613+0000 7fbc3f767640 1 -- 172.21.15.33:0/2176977173 wait complete. 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3f767640 1 Processor -- start 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3f767640 1 -- start start 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3f767640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc381009b0 0x7fbc381983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3f767640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc38101370 0x7fbc381988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3f767640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc381069c0 0x7fbc3819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3f767640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbc38111ea0 con 0x7fbc38101370 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3f767640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbc38111d20 con 0x7fbc381009b0 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3f767640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbc38112020 con 0x7fbc381069c0 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3cd02640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc38101370 0x7fbc381988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3cd02640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc38101370 0x7fbc381988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55184/0 (socket says 172.21.15.33:55184) 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3cd02640 1 -- 172.21.15.33:0/4041172867 learned_addr learned my addr 172.21.15.33:0/4041172867 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3dd04640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc381069c0 0x7fbc3819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3d503640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc381009b0 0x7fbc381983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3cd02640 1 -- 172.21.15.33:0/4041172867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc381069c0 msgr2=0x7fbc3819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3cd02640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc381069c0 0x7fbc3819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3cd02640 1 -- 172.21.15.33:0/4041172867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc381009b0 msgr2=0x7fbc381983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3cd02640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc381009b0 0x7fbc381983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3cd02640 1 -- 172.21.15.33:0/4041172867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3819d390 con 0x7fbc38101370 2024-04-02T13:39:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3dd04640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc381069c0 0x7fbc3819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.614+0000 7fbc3cd02640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc38101370 0x7fbc381988e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c005ab0 tx=0x7fbc2c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.615+0000 7fbc367fc640 1 -- 172.21.15.33:0/4041172867 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c043020 con 0x7fbc38101370 2024-04-02T13:39:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.615+0000 7fbc367fc640 1 -- 172.21.15.33:0/4041172867 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc2c02c5d0 con 0x7fbc38101370 2024-04-02T13:39:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.615+0000 7fbc3f767640 1 -- 172.21.15.33:0/4041172867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3819d620 con 0x7fbc38101370 2024-04-02T13:39:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.615+0000 7fbc367fc640 1 -- 172.21.15.33:0/4041172867 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c0039f0 con 0x7fbc38101370 2024-04-02T13:39:27.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.615+0000 7fbc3f767640 1 -- 172.21.15.33:0/4041172867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc381a4f10 con 0x7fbc38101370 2024-04-02T13:39:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.617+0000 7fbc367fc640 1 -- 172.21.15.33:0/4041172867 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fbc2c046020 con 0x7fbc38101370 2024-04-02T13:39:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.617+0000 7fbc4242c740 1 Processor -- start 2024-04-02T13:39:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.617+0000 7fbc4242c740 1 -- start start 2024-04-02T13:39:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc4242c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6911c600 0x55ab6911c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc4242c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911cf20 0x55ab69129ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc4242c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 0x55ab6912c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc4242c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab69080830 con 0x55ab6912a1e0 2024-04-02T13:39:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc367fc640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbc200a3ed0 0x7fbc200a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc4242c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ab69028820 con 0x55ab6911cf20 2024-04-02T13:39:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc4242c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ab6907c7e0 con 0x55ab6911c600 2024-04-02T13:39:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc3cd02640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 0x55ab6912c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc3dd04640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbc200a3ed0 0x7fbc200a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc3cd02640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 0x55ab6912c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55196/0 (socket says 172.21.15.33:55196) 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc3cd02640 1 -- 172.21.15.33:0/762873306 learned_addr learned my addr 172.21.15.33:0/762873306 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc367fc640 1 -- 172.21.15.33:0/4041172867 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbc2c0f2820 con 0x7fbc38101370 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc3dd04640 1 --2- 172.21.15.33:0/762873306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6911c600 0x55ab6911c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc3d503640 1 --2- 172.21.15.33:0/762873306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911cf20 0x55ab69129ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc3cd02640 1 -- 172.21.15.33:0/762873306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6911c600 msgr2=0x55ab6911c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc3cd02640 1 --2- 172.21.15.33:0/762873306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6911c600 0x55ab6911c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc3cd02640 1 -- 172.21.15.33:0/762873306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911cf20 msgr2=0x55ab69129ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc3cd02640 1 --2- 172.21.15.33:0/762873306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911cf20 0x55ab69129ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc3cd02640 1 -- 172.21.15.33:0/762873306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab68e5e7d0 con 0x55ab6912a1e0 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc3dd04640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbc200a3ed0 0x7fbc200a6390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005fd0 tx=0x7fbc28009e30 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc3cd02640 1 --2- 172.21.15.33:0/762873306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 0x55ab6912c5d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0bacf0 tx=0x7fbc2c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d5e8d1f1fc8ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.618+0000 7fbc1f7fe640 1 -- 172.21.15.33:0/762873306 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c0bea20 con 0x55ab6912a1e0 2024-04-02T13:39:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.619+0000 7fbc1f7fe640 1 -- 172.21.15.33:0/762873306 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc2c0be320 con 0x55ab6912a1e0 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.619+0000 7fbc1f7fe640 1 -- 172.21.15.33:0/762873306 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c02c780 con 0x55ab6912a1e0 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.619+0000 7fbc4242c740 1 -- 172.21.15.33:0/762873306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 msgr2=0x55ab6912c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.619+0000 7fbc4242c740 1 --2- 172.21.15.33:0/762873306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 0x55ab6912c5d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0bacf0 tx=0x7fbc2c009260 comp rx=0 tx=0).stop 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.619+0000 7fbc4242c740 1 -- 172.21.15.33:0/762873306 shutdown_connections 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.619+0000 7fbc4242c740 1 --2- 172.21.15.33:0/762873306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 0x55ab6912c5d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.619+0000 7fbc4242c740 1 --2- 172.21.15.33:0/762873306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911cf20 0x55ab69129ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.619+0000 7fbc4242c740 1 --2- 172.21.15.33:0/762873306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6911c600 0x55ab6911c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.619+0000 7fbc4242c740 1 -- 172.21.15.33:0/762873306 >> 172.21.15.33:0/762873306 conn(0x55ab69117000 msgr2=0x55ab691173e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.619+0000 7fbc4242c740 1 -- 172.21.15.33:0/762873306 shutdown_connections 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.619+0000 7fbc4242c740 1 -- 172.21.15.33:0/762873306 wait complete. 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc4242c740 1 -- start start 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc4242c740 1 Processor -- start 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc4242c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911c600 0x55ab69129af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc4242c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6911cf20 0x55ab69126e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc4242c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6912a1e0 0x55ab691273c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc4242c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab69080830 con 0x55ab6911c600 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc4242c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ab6903bd30 con 0x55ab6912a1e0 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc4242c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ab6912cd60 con 0x55ab6911cf20 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc4242c740 10 client.0 _ll_register_callbacks cb 0x55ab6903b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc3d503640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911c600 0x55ab69129af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc3d503640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911c600 0x55ab69129af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55208/0 (socket says 172.21.15.33:55208) 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc3d503640 1 -- 172.21.15.33:0/3474772323 learned_addr learned my addr 172.21.15.33:0/3474772323 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc3dd04640 1 --2- 172.21.15.33:0/3474772323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6912a1e0 0x55ab691273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc3d503640 1 -- 172.21.15.33:0/3474772323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6911cf20 msgr2=0x55ab69126e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc3d503640 1 --2- 172.21.15.33:0/3474772323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6911cf20 0x55ab69126e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc3d503640 1 -- 172.21.15.33:0/3474772323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6912a1e0 msgr2=0x55ab691273c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc3d503640 1 --2- 172.21.15.33:0/3474772323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6912a1e0 0x55ab691273c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc3d503640 1 -- 172.21.15.33:0/3474772323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab68d943f0 con 0x55ab6911c600 2024-04-02T13:39:27.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.620+0000 7fbc3dd04640 1 --2- 172.21.15.33:0/3474772323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6912a1e0 0x55ab691273c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc3d503640 1 --2- 172.21.15.33:0/3474772323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911c600 0x55ab69129af0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000bef0 tx=0x7fbc30012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e9f4209b9099ad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc1cff9640 1 -- 172.21.15.33:0/3474772323 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc300108a0 con 0x55ab6911c600 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc4242c740 1 -- 172.21.15.33:0/3474772323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2c0be320 con 0x55ab6911c600 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc1cff9640 1 -- 172.21.15.33:0/3474772323 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbc30010a40 con 0x55ab6911c600 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc1cff9640 1 -- 172.21.15.33:0/3474772323 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc30010c40 con 0x55ab6911c600 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc4242c740 10 client.15840 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc4242c740 1 -- 172.21.15.33:0/3474772323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab68d540f0 con 0x55ab6911c600 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc1cff9640 1 -- 172.21.15.33:0/3474772323 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc2c0be320 con 0x55ab6911c600 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc1cff9640 1 -- 172.21.15.33:0/3474772323 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbc30019070 con 0x55ab6911c600 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc4242c740 10 client.15840 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc4242c740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc4242c740 20 client.15840 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc4242c740 1 --2- 172.21.15.33:0/3474772323 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ab69139e60 0x55ab6913c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc4242c740 4 client.15840 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.621+0000 7fbc4242c740 1 -- 172.21.15.33:0/3474772323 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab6908dd70 con 0x55ab69139e60 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.622+0000 7fbc3cd02640 1 --2- 172.21.15.33:0/3474772323 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ab69139e60 0x55ab6913c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.622+0000 7fbc3cd02640 1 --2- 172.21.15.33:0/3474772323 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ab69139e60 0x55ab6913c2e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3a231292c3fdd9 server_cookie=310bbfc73088fa21 in_seq=0 out_seq=0 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.622+0000 7fbc1cff9640 10 client.15840 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc1cff9640 1 -- 172.21.15.33:0/3474772323 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab6908dd70 con 0x55ab69139e60 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc1cff9640 10 client.15840 handle_command_reply: tid=0 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc4242c740 1 client.15840 shutdown 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc4242c740 1 -- 172.21.15.33:0/3474772323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911c600 msgr2=0x55ab69129af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc4242c740 1 --2- 172.21.15.33:0/3474772323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911c600 0x55ab69129af0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000bef0 tx=0x7fbc30012820 comp rx=0 tx=0).stop 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc4242c740 1 -- 172.21.15.33:0/3474772323 shutdown_connections 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc4242c740 1 --2- 172.21.15.33:0/3474772323 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ab69139e60 0x55ab6913c2e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc4242c740 1 --2- 172.21.15.33:0/3474772323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6912a1e0 0x55ab691273c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc4242c740 1 --2- 172.21.15.33:0/3474772323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6911cf20 0x55ab69126e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc1cff9640 0 client.15840 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:27.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc4242c740 1 --2- 172.21.15.33:0/3474772323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911c600 0x55ab69129af0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc4242c740 1 -- 172.21.15.33:0/3474772323 >> 172.21.15.33:0/3474772323 conn(0x55ab69117000 msgr2=0x55ab69118d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:27.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc4242c740 1 -- 172.21.15.33:0/3474772323 shutdown_connections 2024-04-02T13:39:27.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc4242c740 1 -- 172.21.15.33:0/3474772323 wait complete. 2024-04-02T13:39:27.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.623+0000 7fbc4242c740 20 client.15840 trim_cache size 0 max 16384 2024-04-02T13:39:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc4242c740 1 Processor -- start 2024-04-02T13:39:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc4242c740 1 -- start start 2024-04-02T13:39:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc4242c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6911c600 0x55ab69135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc4242c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911cf20 0x55ab691295b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc4242c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 0x55ab69129af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc4242c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbc381af5b0 con 0x55ab6912a1e0 2024-04-02T13:39:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc4242c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbc381ace80 con 0x55ab6911cf20 2024-04-02T13:39:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc4242c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbc381aacb0 con 0x55ab6911c600 2024-04-02T13:39:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc3dd04640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 0x55ab69129af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc3dd04640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 0x55ab69129af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55214/0 (socket says 172.21.15.33:55214) 2024-04-02T13:39:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc3dd04640 1 -- 172.21.15.33:0/2498787122 learned_addr learned my addr 172.21.15.33:0/2498787122 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc3cd02640 1 --2- 172.21.15.33:0/2498787122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911cf20 0x55ab691295b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc3dd04640 1 -- 172.21.15.33:0/2498787122 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6911c600 msgr2=0x55ab69135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc3dd04640 1 --2- 172.21.15.33:0/2498787122 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6911c600 0x55ab69135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc3dd04640 1 -- 172.21.15.33:0/2498787122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911cf20 msgr2=0x55ab691295b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc3dd04640 1 --2- 172.21.15.33:0/2498787122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911cf20 0x55ab691295b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc3dd04640 1 -- 172.21.15.33:0/2498787122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6911c240 con 0x55ab6912a1e0 2024-04-02T13:39:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.641+0000 7fbc3dd04640 1 --2- 172.21.15.33:0/2498787122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 0x55ab69129af0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbc28012760 tx=0x7fbc28004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea2b855df5e0e68 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.642+0000 7fbc1effd640 1 -- 172.21.15.33:0/2498787122 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2802d070 con 0x55ab6912a1e0 2024-04-02T13:39:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.642+0000 7fbc1effd640 1 -- 172.21.15.33:0/2498787122 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc280099b0 con 0x55ab6912a1e0 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.642+0000 7fbc1effd640 1 -- 172.21.15.33:0/2498787122 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2801e880 con 0x55ab6912a1e0 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.642+0000 7fbc4242c740 1 -- 172.21.15.33:0/2498787122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 msgr2=0x55ab69129af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.642+0000 7fbc4242c740 1 --2- 172.21.15.33:0/2498787122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 0x55ab69129af0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbc28012760 tx=0x7fbc28004ab0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.642+0000 7fbc4242c740 1 -- 172.21.15.33:0/2498787122 shutdown_connections 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.642+0000 7fbc4242c740 1 --2- 172.21.15.33:0/2498787122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6912a1e0 0x55ab69129af0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.642+0000 7fbc4242c740 1 --2- 172.21.15.33:0/2498787122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911cf20 0x55ab691295b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.642+0000 7fbc4242c740 1 --2- 172.21.15.33:0/2498787122 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6911c600 0x55ab69135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.642+0000 7fbc4242c740 1 -- 172.21.15.33:0/2498787122 >> 172.21.15.33:0/2498787122 conn(0x55ab69117000 msgr2=0x55ab691224d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.642+0000 7fbc4242c740 1 -- 172.21.15.33:0/2498787122 shutdown_connections 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.642+0000 7fbc4242c740 1 -- 172.21.15.33:0/2498787122 wait complete. 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc4242c740 1 -- start start 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc4242c740 1 Processor -- start 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc4242c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911c600 0x55ab69123af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc4242c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911cf20 0x55ab69124030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc4242c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6912a1e0 0x55ab69124570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc4242c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbc381ab340 con 0x55ab6911cf20 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc4242c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbc381ac7b0 con 0x55ab6911c600 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc4242c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbc381aba10 con 0x55ab6912a1e0 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc4242c740 10 client.0 _ll_register_callbacks cb 0x7fbc3000bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc3cd02640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911cf20 0x55ab69124030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc3cd02640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911cf20 0x55ab69124030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55220/0 (socket says 172.21.15.33:55220) 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc3cd02640 1 -- 172.21.15.33:0/175084285 learned_addr learned my addr 172.21.15.33:0/175084285 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc3d503640 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911c600 0x55ab69123af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc3dd04640 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6912a1e0 0x55ab69124570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc3cd02640 1 -- 172.21.15.33:0/175084285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6912a1e0 msgr2=0x55ab69124570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc3cd02640 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6912a1e0 0x55ab69124570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc3cd02640 1 -- 172.21.15.33:0/175084285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911c600 msgr2=0x55ab69123af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc3cd02640 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911c600 0x55ab69123af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc3cd02640 1 -- 172.21.15.33:0/175084285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6919f460 con 0x55ab6911cf20 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.643+0000 7fbc3dd04640 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6912a1e0 0x55ab69124570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc3cd02640 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911cf20 0x55ab69124030 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c005ec0 tx=0x7fbc2c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ca2283ba1b056e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc1cff9640 1 -- 172.21.15.33:0/175084285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c0bea20 con 0x55ab6911cf20 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc1cff9640 1 -- 172.21.15.33:0/175084285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbc2c0e3180 con 0x55ab6911cf20 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc4242c740 1 -- 172.21.15.33:0/175084285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc280099b0 con 0x55ab6911cf20 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc1cff9640 1 -- 172.21.15.33:0/175084285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc2c0c6f90 con 0x55ab6911cf20 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc4242c740 10 client.15846 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc4242c740 1 -- 172.21.15.33:0/175084285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab691a06c0 con 0x55ab6911cf20 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc1cff9640 1 -- 172.21.15.33:0/175084285 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbc280099b0 con 0x55ab6911cf20 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc1cff9640 1 -- 172.21.15.33:0/175084285 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbc2c02c780 con 0x55ab6911cf20 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc4242c740 10 client.15846 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc4242c740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc4242c740 20 client.15846 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc4242c740 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ab691a0c50 0x55ab691a3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc4242c740 4 client.15846 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.644+0000 7fbc4242c740 1 -- 172.21.15.33:0/175084285 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab690880c0 con 0x55ab691a0c50 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc3d503640 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ab691a0c50 0x55ab691a3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc1cff9640 10 client.15846 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc3d503640 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ab691a0c50 0x55ab691a3040 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17fa3908f98471d9 server_cookie=f4d7215ddc468c74 in_seq=0 out_seq=0 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc1cff9640 1 -- 172.21.15.33:0/175084285 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ab690880c0 con 0x55ab691a0c50 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc1cff9640 10 client.15846 handle_command_reply: tid=0 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc4242c740 1 client.15846 shutdown 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc4242c740 1 -- 172.21.15.33:0/175084285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911cf20 msgr2=0x55ab69124030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc4242c740 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911cf20 0x55ab69124030 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c005ec0 tx=0x7fbc2c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc4242c740 1 -- 172.21.15.33:0/175084285 shutdown_connections 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc4242c740 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ab691a0c50 0x55ab691a3040 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc4242c740 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ab6912a1e0 0x55ab69124570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc4242c740 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab6911cf20 0x55ab69124030 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc4242c740 1 --2- 172.21.15.33:0/175084285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ab6911c600 0x55ab69123af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc1cff9640 0 client.15846 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:27.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc4242c740 1 -- 172.21.15.33:0/175084285 >> 172.21.15.33:0/175084285 conn(0x55ab69117000 msgr2=0x55ab691159b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:27.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc4242c740 1 -- 172.21.15.33:0/175084285 shutdown_connections 2024-04-02T13:39:27.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc4242c740 1 -- 172.21.15.33:0/175084285 wait complete. 2024-04-02T13:39:27.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.645+0000 7fbc4242c740 20 client.15846 trim_cache size 0 max 16384 2024-04-02T13:39:27.655 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:39:27.655 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:39:27.655 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:39:27.655 INFO:teuthology.orchestra.run.smithi033.stdout: "0fd0b3c5-9bfd-4cf7-aff6-e563ae258e1d": { 2024-04-02T13:39:27.656 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:39:27.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "0fd0b3c5-9bfd-4cf7-aff6-e563ae258e1d", 2024-04-02T13:39:27.656 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:39:27.656 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:39:27.656 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:39:27.656 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:39:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.646+0000 7fbc3f767640 1 -- 172.21.15.33:0/4041172867 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbc200a3ed0 msgr2=0x7fbc200a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.646+0000 7fbc3f767640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbc200a3ed0 0x7fbc200a6390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005fd0 tx=0x7fbc28009e30 comp rx=0 tx=0).stop 2024-04-02T13:39:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.646+0000 7fbc3f767640 1 -- 172.21.15.33:0/4041172867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc38101370 msgr2=0x7fbc381988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.646+0000 7fbc3f767640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc38101370 0x7fbc381988e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c005ab0 tx=0x7fbc2c004600 comp rx=0 tx=0).stop 2024-04-02T13:39:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.646+0000 7fbc3f767640 1 -- 172.21.15.33:0/4041172867 shutdown_connections 2024-04-02T13:39:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.646+0000 7fbc3f767640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbc200a3ed0 0x7fbc200a6390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.646+0000 7fbc3f767640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbc381069c0 0x7fbc3819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.646+0000 7fbc3f767640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc38101370 0x7fbc381988e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.646+0000 7fbc3f767640 1 --2- 172.21.15.33:0/4041172867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbc381009b0 0x7fbc381983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.646+0000 7fbc3f767640 1 -- 172.21.15.33:0/4041172867 >> 172.21.15.33:0/4041172867 conn(0x7fbc38076d80 msgr2=0x7fbc380fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.646+0000 7fbc3f767640 1 -- 172.21.15.33:0/4041172867 shutdown_connections 2024-04-02T13:39:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:27.646+0000 7fbc3f767640 1 -- 172.21.15.33:0/4041172867 wait complete. 2024-04-02T13:39:27.659 INFO:tasks.cephfs.filesystem:scrub status for tag:0fd0b3c5-9bfd-4cf7-aff6-e563ae258e1d - {'path': '/', 'tag': '0fd0b3c5-9bfd-4cf7-aff6-e563ae258e1d', 'options': 'recursive,force'} 2024-04-02T13:39:28.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:27 smithi145 ceph-mon[35646]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:28.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:27 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:39:28.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:27 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:28.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:27 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:39:28.230 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:27 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:28.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:27 smithi050 ceph-mon[35524]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:28.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:27 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:39:28.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:27 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:28.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:27 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:39:28.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:27 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:28.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:27 smithi033 ceph-mon[32390]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:28.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:27 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:39:28.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:27 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:28.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:27 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:39:28.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:27 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:30.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:29 smithi145 ceph-mon[35646]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-02T13:39:30.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:29 smithi050 ceph-mon[35524]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-02T13:39:30.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:29 smithi033 ceph-mon[32390]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-02T13:39:31.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:30 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:39:31.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:30 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:39:31.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:30 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:39:32.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:31 smithi050 ceph-mon[35524]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:39:32.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:31 smithi033 ceph-mon[32390]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:39:32.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:31 smithi145 ceph-mon[35646]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:39:34.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:33 smithi050 ceph-mon[35524]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-02T13:39:34.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:33 smithi033 ceph-mon[32390]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-02T13:39:34.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:33 smithi145 ceph-mon[35646]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-02T13:39:36.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:35 smithi050 ceph-mon[35524]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-02T13:39:36.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:35 smithi033 ceph-mon[32390]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-02T13:39:36.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:35 smithi145 ceph-mon[35646]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-02T13:39:38.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:37 smithi050 ceph-mon[35524]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:39:38.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:37 smithi033 ceph-mon[32390]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:39:38.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:37 smithi145 ceph-mon[35646]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:39:40.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:39 smithi050 ceph-mon[35524]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-02T13:39:40.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:39 smithi033 ceph-mon[32390]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-02T13:39:40.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:39 smithi145 ceph-mon[35646]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-02T13:39:42.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:41 smithi050 ceph-mon[35524]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:42.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:41 smithi033 ceph-mon[32390]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:42.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:41 smithi145 ceph-mon[35646]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:44.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:43 smithi033 ceph-mon[32390]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:39:44.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:43 smithi145 ceph-mon[35646]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:39:44.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:43 smithi050 ceph-mon[35524]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:39:46.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:45 smithi033 ceph-mon[32390]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:46.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:45 smithi145 ceph-mon[35646]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:46.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:45 smithi050 ceph-mon[35524]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:48.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:47 smithi033 ceph-mon[32390]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:48.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:47 smithi145 ceph-mon[35646]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:48.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:47 smithi050 ceph-mon[35524]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:50.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:50 smithi033 ceph-mon[32390]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:39:50.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:50 smithi145 ceph-mon[35646]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:39:50.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:50 smithi050 ceph-mon[35524]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:39:52.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:52 smithi145 ceph-mon[35646]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:52.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:52 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:39:52.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:52 smithi050 ceph-mon[35524]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:52.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:52 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:39:52.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:52 smithi033 ceph-mon[32390]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:52.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:52 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:39:53.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:53 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:53 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:53 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:53 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:53 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:53 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:53 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:39:53.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:53 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:39:53.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:53 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:53 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:53 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:53 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:53 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:53 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:53 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:53 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:39:53.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:53 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:39:53.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:53 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:53 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:53 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:53 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:53 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:53 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:53 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:53.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:53 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:39:53.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:53 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:39:53.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:53 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:39:54.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:54 smithi145 ceph-mon[35646]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:39:54.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:54 smithi050 ceph-mon[35524]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:39:54.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:54 smithi033 ceph-mon[32390]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:39:56.531 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:56 smithi145 ceph-mon[35646]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:56.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:56 smithi050 ceph-mon[35524]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:56.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:56 smithi033 ceph-mon[32390]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:57.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:39:57.660 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:39:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.767+0000 7f5df051c640 1 -- 172.21.15.33:0/1603072389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de810e330 msgr2=0x7f5de8110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.767+0000 7f5df051c640 1 --2- 172.21.15.33:0/1603072389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de810e330 0x7f5de8110740 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009ca0 tx=0x7f5de002c180 comp rx=0 tx=0).stop 2024-04-02T13:39:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.767+0000 7f5df051c640 1 -- 172.21.15.33:0/1603072389 shutdown_connections 2024-04-02T13:39:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.767+0000 7f5df051c640 1 --2- 172.21.15.33:0/1603072389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de810e330 0x7f5de8110740 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.767+0000 7f5df051c640 1 --2- 172.21.15.33:0/1603072389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5de80747f0 0x7f5de810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.767+0000 7f5df051c640 1 --2- 172.21.15.33:0/1603072389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5de8073ed0 0x7f5de80742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.767+0000 7f5df051c640 1 -- 172.21.15.33:0/1603072389 >> 172.21.15.33:0/1603072389 conn(0x7f5de80fd5a0 msgr2=0x7f5de80ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.767+0000 7f5df051c640 1 -- 172.21.15.33:0/1603072389 shutdown_connections 2024-04-02T13:39:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.767+0000 7f5df051c640 1 -- 172.21.15.33:0/1603072389 wait complete. 2024-04-02T13:39:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.767+0000 7f5df051c640 1 Processor -- start 2024-04-02T13:39:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5df051c640 1 -- start start 2024-04-02T13:39:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5df051c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5de8073ed0 0x7f5de819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5df051c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5de80747f0 0x7f5de819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5df051c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de810e330 0x7f5de819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5df051c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8114060 con 0x7f5de810e330 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5df051c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8113ee0 con 0x7f5de80747f0 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5df051c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81141e0 con 0x7f5de8073ed0 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5dee2b8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5de8073ed0 0x7f5de819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5deeab9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de810e330 0x7f5de819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5dee2b8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5de8073ed0 0x7f5de819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:54406/0 (socket says 172.21.15.33:54406) 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5dee2b8640 1 -- 172.21.15.33:0/1033145783 learned_addr learned my addr 172.21.15.33:0/1033145783 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5dedab7640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5de80747f0 0x7f5de819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5dee2b8640 1 -- 172.21.15.33:0/1033145783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5de80747f0 msgr2=0x7f5de819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5de80747f0 0x7f5de819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5dee2b8640 1 -- 172.21.15.33:0/1033145783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de810e330 msgr2=0x7f5de819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de810e330 0x7f5de819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5dee2b8640 1 -- 172.21.15.33:0/1033145783 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de819f610 con 0x7f5de8073ed0 2024-04-02T13:39:57.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5dedab7640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5de80747f0 0x7f5de819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:57.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.768+0000 7f5deeab9640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de810e330 0x7f5de819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:57.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.769+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5de8073ed0 0x7f5de819a620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d950 tx=0x7f5de400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:57.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.769+0000 7f5ddf7fe640 1 -- 172.21.15.33:0/1033145783 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4017030 con 0x7f5de8073ed0 2024-04-02T13:39:57.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.769+0000 7f5df051c640 1 -- 172.21.15.33:0/1033145783 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de819f900 con 0x7f5de8073ed0 2024-04-02T13:39:57.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.769+0000 7f5df051c640 1 -- 172.21.15.33:0/1033145783 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de81a7190 con 0x7f5de8073ed0 2024-04-02T13:39:57.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.769+0000 7f5ddf7fe640 1 -- 172.21.15.33:0/1033145783 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5de4004020 con 0x7f5de8073ed0 2024-04-02T13:39:57.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.769+0000 7f5ddf7fe640 1 -- 172.21.15.33:0/1033145783 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40047a0 con 0x7f5de8073ed0 2024-04-02T13:39:57.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.772+0000 7f5df31e8740 1 Processor -- start 2024-04-02T13:39:57.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.772+0000 7f5ddf7fe640 1 -- 172.21.15.33:0/1033145783 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f5de40041d0 con 0x7f5de8073ed0 2024-04-02T13:39:57.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5df31e8740 1 -- start start 2024-04-02T13:39:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5df31e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64e912b0 0x55cc64e91690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5df31e8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5df31e8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 0x55cc64ea59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5df31e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cc64df5830 con 0x55cc64e912b0 2024-04-02T13:39:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5df31e8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55cc64d9d820 con 0x55cc64e91c60 2024-04-02T13:39:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5df31e8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55cc64df17e0 con 0x55cc64ea35e0 2024-04-02T13:39:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5ddf7fe640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5dd40a3fd0 0x7f5dd40a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5deeab9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64e912b0 0x55cc64e91690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5deeab9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64e912b0 0x55cc64e91690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52748/0 (socket says 172.21.15.33:52748) 2024-04-02T13:39:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5deeab9640 1 -- 172.21.15.33:0/1074539576 learned_addr learned my addr 172.21.15.33:0/1074539576 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5ddf7fe640 1 -- 172.21.15.33:0/1033145783 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5de40d0690 con 0x7f5de8073ed0 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/1074539576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.773+0000 7f5dedab7640 1 --2- 172.21.15.33:0/1074539576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 0x55cc64ea59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.774+0000 7f5deeab9640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5dd40a3fd0 0x7f5dd40a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.774+0000 7f5dee2b8640 1 -- 172.21.15.33:0/1074539576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 msgr2=0x55cc64ea59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.774+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/1074539576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 0x55cc64ea59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.774+0000 7f5dee2b8640 1 -- 172.21.15.33:0/1074539576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64e912b0 msgr2=0x55cc64e91690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.774+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/1074539576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64e912b0 0x55cc64e91690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.774+0000 7f5dee2b8640 1 -- 172.21.15.33:0/1074539576 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc64bd37d0 con 0x55cc64e91c60 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.774+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/1074539576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea30a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5de40cd830 tx=0x7f5de4098450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e6700dc7f387cab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.774+0000 7f5deeab9640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5dd40a3fd0 0x7f5dd40a6490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5de0004af0 tx=0x7f5de002c180 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.774+0000 7f5ddcbfe640 1 -- 172.21.15.33:0/1074539576 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de409c780 con 0x55cc64e91c60 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.774+0000 7f5ddcbfe640 1 -- 172.21.15.33:0/1074539576 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5de409c080 con 0x55cc64e91c60 2024-04-02T13:39:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.774+0000 7f5ddcbfe640 1 -- 172.21.15.33:0/1074539576 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4098880 con 0x55cc64e91c60 2024-04-02T13:39:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.775+0000 7f5df31e8740 1 -- 172.21.15.33:0/1074539576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 msgr2=0x55cc64ea30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.775+0000 7f5df31e8740 1 --2- 172.21.15.33:0/1074539576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea30a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5de40cd830 tx=0x7f5de4098450 comp rx=0 tx=0).stop 2024-04-02T13:39:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.775+0000 7f5df31e8740 1 -- 172.21.15.33:0/1074539576 shutdown_connections 2024-04-02T13:39:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.775+0000 7f5df31e8740 1 --2- 172.21.15.33:0/1074539576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 0x55cc64ea59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.775+0000 7f5df31e8740 1 --2- 172.21.15.33:0/1074539576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea30a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.775+0000 7f5df31e8740 1 --2- 172.21.15.33:0/1074539576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64e912b0 0x55cc64e91690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.775+0000 7f5df31e8740 1 -- 172.21.15.33:0/1074539576 >> 172.21.15.33:0/1074539576 conn(0x55cc64e8c000 msgr2=0x55cc64e8c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.775+0000 7f5df31e8740 1 -- 172.21.15.33:0/1074539576 shutdown_connections 2024-04-02T13:39:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.775+0000 7f5df31e8740 1 -- 172.21.15.33:0/1074539576 wait complete. 2024-04-02T13:39:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.776+0000 7f5df31e8740 1 -- start start 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.776+0000 7f5df31e8740 1 Processor -- start 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.776+0000 7f5df31e8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64e912b0 0x55cc64ea2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.776+0000 7f5df31e8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.776+0000 7f5df31e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64ea35e0 0x55cc64ea0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.776+0000 7f5df31e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cc64ea6340 con 0x55cc64ea35e0 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.776+0000 7f5df31e8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55cc64db0d30 con 0x55cc64e91c60 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.776+0000 7f5df31e8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55cc64ea64c0 con 0x55cc64e912b0 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.776+0000 7f5deeab9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64ea35e0 0x55cc64ea0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.776+0000 7f5dedab7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.777+0000 7f5dedab7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:44812/0 (socket says 172.21.15.33:44812) 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.777+0000 7f5dedab7640 1 -- 172.21.15.33:0/708732274 learned_addr learned my addr 172.21.15.33:0/708732274 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.777+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/708732274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64e912b0 0x55cc64ea2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.777+0000 7f5df31e8740 10 client.0 _ll_register_callbacks cb 0x55cc64db0720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.777+0000 7f5dedab7640 1 -- 172.21.15.33:0/708732274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64e912b0 msgr2=0x55cc64ea2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.777+0000 7f5dedab7640 1 --2- 172.21.15.33:0/708732274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64e912b0 0x55cc64ea2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.777+0000 7f5dedab7640 1 -- 172.21.15.33:0/708732274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64ea35e0 msgr2=0x55cc64ea0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.777+0000 7f5dedab7640 1 --2- 172.21.15.33:0/708732274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64ea35e0 0x55cc64ea0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.777+0000 7f5dedab7640 1 -- 172.21.15.33:0/708732274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc64b093f0 con 0x55cc64e91c60 2024-04-02T13:39:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.777+0000 7f5deeab9640 1 --2- 172.21.15.33:0/708732274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64ea35e0 0x55cc64ea0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.777+0000 7f5dedab7640 1 --2- 172.21.15.33:0/708732274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea0220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8013710 tx=0x7f5dd8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26805880ceb239 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.777+0000 7f5dd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T13:39:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.778+0000 7f5dd1ffb640 1 -- 172.21.15.33:0/708732274 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8011e30 con 0x55cc64e91c60 2024-04-02T13:39:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.778+0000 7f5df31e8740 1 -- 172.21.15.33:0/708732274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de409c080 con 0x55cc64e91c60 2024-04-02T13:39:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.778+0000 7f5dd1ffb640 1 -- 172.21.15.33:0/708732274 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5dd8005020 con 0x55cc64e91c60 2024-04-02T13:39:57.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.778+0000 7f5dd1ffb640 1 -- 172.21.15.33:0/708732274 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5dd8026050 con 0x55cc64e91c60 2024-04-02T13:39:57.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.778+0000 7f5df31e8740 10 client.25466 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:57.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.778+0000 7f5df31e8740 1 -- 172.21.15.33:0/708732274 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cc64ac90f0 con 0x55cc64e91c60 2024-04-02T13:39:57.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.778+0000 7f5dd1ffb640 1 -- 172.21.15.33:0/708732274 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5de409c080 con 0x55cc64e91c60 2024-04-02T13:39:57.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.779+0000 7f5dd1ffb640 1 -- 172.21.15.33:0/708732274 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5dd801a030 con 0x55cc64e91c60 2024-04-02T13:39:57.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.779+0000 7f5df31e8740 10 client.25466 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:57.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.779+0000 7f5df31e8740 10 client.25466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:57.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.779+0000 7f5df31e8740 20 client.25466 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:57.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.779+0000 7f5df31e8740 1 --2- 172.21.15.33:0/708732274 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cc64eb35f0 0x55cc64eb5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.779+0000 7f5df31e8740 4 client.25466 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:57.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.779+0000 7f5df31e8740 1 -- 172.21.15.33:0/708732274 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc64e02d70 con 0x55cc64eb35f0 2024-04-02T13:39:57.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.779+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/708732274 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cc64eb35f0 0x55cc64eb5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/708732274 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cc64eb35f0 0x55cc64eb5a30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2718bd3275f83380 server_cookie=7299f8206a7c18b1 in_seq=0 out_seq=0 2024-04-02T13:39:57.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5dd1ffb640 10 client.25466 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:57.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5dd1ffb640 1 -- 172.21.15.33:0/708732274 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc64e02d70 con 0x55cc64eb35f0 2024-04-02T13:39:57.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5dd1ffb640 10 client.25466 handle_command_reply: tid=0 2024-04-02T13:39:57.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5df31e8740 1 client.25466 shutdown 2024-04-02T13:39:57.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5df31e8740 1 -- 172.21.15.33:0/708732274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 msgr2=0x55cc64ea0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:57.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5df31e8740 1 --2- 172.21.15.33:0/708732274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea0220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8013710 tx=0x7f5dd8009f90 comp rx=0 tx=0).stop 2024-04-02T13:39:57.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5df31e8740 1 -- 172.21.15.33:0/708732274 shutdown_connections 2024-04-02T13:39:57.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5df31e8740 1 --2- 172.21.15.33:0/708732274 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cc64eb35f0 0x55cc64eb5a30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5df31e8740 1 --2- 172.21.15.33:0/708732274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64ea35e0 0x55cc64ea0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5df31e8740 1 --2- 172.21.15.33:0/708732274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea0220 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5dd1ffb640 0 client.25466 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:57.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5df31e8740 1 --2- 172.21.15.33:0/708732274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64e912b0 0x55cc64ea2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5df31e8740 1 -- 172.21.15.33:0/708732274 >> 172.21.15.33:0/708732274 conn(0x55cc64e8c000 msgr2=0x55cc64e8dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:57.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.780+0000 7f5df31e8740 1 -- 172.21.15.33:0/708732274 shutdown_connections 2024-04-02T13:39:57.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.781+0000 7f5df31e8740 1 -- 172.21.15.33:0/708732274 wait complete. 2024-04-02T13:39:57.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.781+0000 7f5df31e8740 20 client.25466 trim_cache size 0 max 16384 2024-04-02T13:39:57.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.798+0000 7f5df31e8740 1 Processor -- start 2024-04-02T13:39:57.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.798+0000 7f5df31e8740 1 -- start start 2024-04-02T13:39:57.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.798+0000 7f5df31e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64e912b0 0x55cc64eaf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.798+0000 7f5df31e8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64e9b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.798+0000 7f5df31e8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 0x55cc64ea2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.798+0000 7f5df31e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc012700 con 0x55cc64e912b0 2024-04-02T13:39:57.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.798+0000 7f5df31e8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc00ffd0 con 0x55cc64e91c60 2024-04-02T13:39:57.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.798+0000 7f5df31e8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc00de00 con 0x55cc64ea35e0 2024-04-02T13:39:57.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.798+0000 7f5dee2b8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64e912b0 0x55cc64eaf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.799+0000 7f5dee2b8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64e912b0 0x55cc64eaf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52768/0 (socket says 172.21.15.33:52768) 2024-04-02T13:39:57.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.799+0000 7f5dee2b8640 1 -- 172.21.15.33:0/3580529680 learned_addr learned my addr 172.21.15.33:0/3580529680 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:57.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.799+0000 7f5dedab7640 1 --2- 172.21.15.33:0/3580529680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64e9b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.799+0000 7f5dee2b8640 1 -- 172.21.15.33:0/3580529680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 msgr2=0x55cc64ea2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:57.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.799+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/3580529680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 0x55cc64ea2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.799+0000 7f5dee2b8640 1 -- 172.21.15.33:0/3580529680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 msgr2=0x55cc64e9b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:57.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.799+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/3580529680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64e9b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.799+0000 7f5dee2b8640 1 -- 172.21.15.33:0/3580529680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc64e8be30 con 0x55cc64e912b0 2024-04-02T13:39:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.799+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/3580529680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64e912b0 0x55cc64eaf270 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5de40052f0 tx=0x7f5de4098450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36362c99e47c6d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.799+0000 7f5dd3fff640 1 -- 172.21.15.33:0/3580529680 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40bb020 con 0x55cc64e912b0 2024-04-02T13:39:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.799+0000 7f5dd3fff640 1 -- 172.21.15.33:0/3580529680 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5de409c780 con 0x55cc64e912b0 2024-04-02T13:39:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.799+0000 7f5dd3fff640 1 -- 172.21.15.33:0/3580529680 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40d02e0 con 0x55cc64e912b0 2024-04-02T13:39:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.800+0000 7f5df31e8740 1 -- 172.21.15.33:0/3580529680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64e912b0 msgr2=0x55cc64eaf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.800+0000 7f5df31e8740 1 --2- 172.21.15.33:0/3580529680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64e912b0 0x55cc64eaf270 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5de40052f0 tx=0x7f5de4098450 comp rx=0 tx=0).stop 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.800+0000 7f5df31e8740 1 -- 172.21.15.33:0/3580529680 shutdown_connections 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.800+0000 7f5df31e8740 1 --2- 172.21.15.33:0/3580529680 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 0x55cc64ea2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.800+0000 7f5df31e8740 1 --2- 172.21.15.33:0/3580529680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64e9b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.800+0000 7f5df31e8740 1 --2- 172.21.15.33:0/3580529680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64e912b0 0x55cc64eaf270 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.800+0000 7f5df31e8740 1 -- 172.21.15.33:0/3580529680 >> 172.21.15.33:0/3580529680 conn(0x55cc64e8c000 msgr2=0x55cc64e9f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.800+0000 7f5df31e8740 1 -- 172.21.15.33:0/3580529680 shutdown_connections 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.800+0000 7f5df31e8740 1 -- 172.21.15.33:0/3580529680 wait complete. 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.800+0000 7f5df31e8740 1 -- start start 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.800+0000 7f5df31e8740 1 Processor -- start 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5df31e8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5df31e8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 0x55cc64ea7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5df31e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64f16ba0 0x55cc64f18f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5df31e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc00e490 con 0x55cc64f16ba0 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5df31e8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc00f900 con 0x55cc64e91c60 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5df31e8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc00eb60 con 0x55cc64ea35e0 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5df31e8740 10 client.0 _ll_register_callbacks cb 0x7f5dd8009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5deeab9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64f16ba0 0x55cc64f18f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5dee2b8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5deeab9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64f16ba0 0x55cc64f18f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52776/0 (socket says 172.21.15.33:52776) 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5dee2b8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:44830/0 (socket says 172.21.15.33:44830) 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5deeab9640 1 -- 172.21.15.33:0/2191171663 learned_addr learned my addr 172.21.15.33:0/2191171663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5dedab7640 1 --2- 172.21.15.33:0/2191171663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 0x55cc64ea7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5dee2b8640 1 -- 172.21.15.33:0/2191171663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 msgr2=0x55cc64ea7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/2191171663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 0x55cc64ea7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5dee2b8640 1 -- 172.21.15.33:0/2191171663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64f16ba0 msgr2=0x55cc64f18f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/2191171663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64f16ba0 0x55cc64f18f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5dee2b8640 1 -- 172.21.15.33:0/2191171663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc64eb2180 con 0x55cc64e91c60 2024-04-02T13:39:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.801+0000 7f5deeab9640 1 --2- 172.21.15.33:0/2191171663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64f16ba0 0x55cc64f18f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.802+0000 7f5dee2b8640 1 --2- 172.21.15.33:0/2191171663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea7880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5de40b6000 tx=0x7f5de4009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47cf980a7cd0a2cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.802+0000 7f5dd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T13:39:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.802+0000 7f5dd1ffb640 1 -- 172.21.15.33:0/2191171663 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40986f0 con 0x55cc64e91c60 2024-04-02T13:39:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.802+0000 7f5dd1ffb640 1 -- 172.21.15.33:0/2191171663 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5de40d02e0 con 0x55cc64e91c60 2024-04-02T13:39:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.802+0000 7f5df31e8740 1 -- 172.21.15.33:0/2191171663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de409c780 con 0x55cc64e91c60 2024-04-02T13:39:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.802+0000 7f5dd1ffb640 1 -- 172.21.15.33:0/2191171663 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5de409c080 con 0x55cc64e91c60 2024-04-02T13:39:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.802+0000 7f5df31e8740 10 client.25472 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.802+0000 7f5df31e8740 1 -- 172.21.15.33:0/2191171663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cc64f1a660 con 0x55cc64e91c60 2024-04-02T13:39:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.802+0000 7f5dd1ffb640 1 -- 172.21.15.33:0/2191171663 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5de40a0430 con 0x55cc64e91c60 2024-04-02T13:39:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.803+0000 7f5dd1ffb640 1 -- 172.21.15.33:0/2191171663 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5de4097dd0 con 0x55cc64e91c60 2024-04-02T13:39:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.803+0000 7f5df31e8740 10 client.25472 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.803+0000 7f5df31e8740 10 client.25472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.803+0000 7f5df31e8740 20 client.25472 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.803+0000 7f5df31e8740 1 --2- 172.21.15.33:0/2191171663 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cc64f1a890 0x55cc64f1cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.803+0000 7f5df31e8740 4 client.25472 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.803+0000 7f5df31e8740 1 -- 172.21.15.33:0/2191171663 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5de409c780 con 0x55cc64f1a890 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.803+0000 7f5dedab7640 1 --2- 172.21.15.33:0/2191171663 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cc64f1a890 0x55cc64f1cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.803+0000 7f5dd1ffb640 10 client.25472 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.803+0000 7f5dedab7640 1 --2- 172.21.15.33:0/2191171663 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cc64f1a890 0x55cc64f1cc80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c21c3f0c93bd6c server_cookie=eed7c6a075075306 in_seq=0 out_seq=0 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5dd1ffb640 1 -- 172.21.15.33:0/2191171663 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f5de409c780 con 0x55cc64f1a890 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5dd1ffb640 10 client.25472 handle_command_reply: tid=0 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5df31e8740 1 client.25472 shutdown 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5df31e8740 1 -- 172.21.15.33:0/2191171663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 msgr2=0x55cc64ea7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5df31e8740 1 --2- 172.21.15.33:0/2191171663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea7880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5de40b6000 tx=0x7f5de4009740 comp rx=0 tx=0).stop 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5df31e8740 1 -- 172.21.15.33:0/2191171663 shutdown_connections 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5df31e8740 1 --2- 172.21.15.33:0/2191171663 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cc64f1a890 0x55cc64f1cc80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5df31e8740 1 --2- 172.21.15.33:0/2191171663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc64f16ba0 0x55cc64f18f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5df31e8740 1 --2- 172.21.15.33:0/2191171663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cc64ea35e0 0x55cc64ea7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5df31e8740 1 --2- 172.21.15.33:0/2191171663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cc64e91c60 0x55cc64ea7880 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5df31e8740 1 -- 172.21.15.33:0/2191171663 >> 172.21.15.33:0/2191171663 conn(0x55cc64e8c000 msgr2=0x55cc64e8a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:57.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5dd1ffb640 0 client.25472 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:57.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5df31e8740 1 -- 172.21.15.33:0/2191171663 shutdown_connections 2024-04-02T13:39:57.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5df31e8740 1 -- 172.21.15.33:0/2191171663 wait complete. 2024-04-02T13:39:57.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.804+0000 7f5df31e8740 20 client.25472 trim_cache size 0 max 16384 2024-04-02T13:39:57.812 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:39:57.812 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:39:57.812 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:39:57.812 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:39:57.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.805+0000 7f5df051c640 1 -- 172.21.15.33:0/1033145783 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5dd40a3fd0 msgr2=0x7f5dd40a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:57.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.805+0000 7f5df051c640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5dd40a3fd0 0x7f5dd40a6490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5de0004af0 tx=0x7f5de002c180 comp rx=0 tx=0).stop 2024-04-02T13:39:57.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.805+0000 7f5df051c640 1 -- 172.21.15.33:0/1033145783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5de8073ed0 msgr2=0x7f5de819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:57.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.805+0000 7f5df051c640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5de8073ed0 0x7f5de819a620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d950 tx=0x7f5de400de10 comp rx=0 tx=0).stop 2024-04-02T13:39:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.805+0000 7f5df051c640 1 -- 172.21.15.33:0/1033145783 shutdown_connections 2024-04-02T13:39:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.805+0000 7f5df051c640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5dd40a3fd0 0x7f5dd40a6490 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.805+0000 7f5df051c640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5de810e330 0x7f5de819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.805+0000 7f5df051c640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5de80747f0 0x7f5de819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.805+0000 7f5df051c640 1 --2- 172.21.15.33:0/1033145783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5de8073ed0 0x7f5de819a620 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.805+0000 7f5df051c640 1 -- 172.21.15.33:0/1033145783 >> 172.21.15.33:0/1033145783 conn(0x7f5de80fd5a0 msgr2=0x7f5de80fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.805+0000 7f5df051c640 1 -- 172.21.15.33:0/1033145783 shutdown_connections 2024-04-02T13:39:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.805+0000 7f5df051c640 1 -- 172.21.15.33:0/1033145783 wait complete. 2024-04-02T13:39:57.819 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:39:57.819 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:39:57.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.960+0000 7f9852a9a640 1 -- 172.21.15.33:0/2046843181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c10e510 msgr2=0x7f984c110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:57.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.960+0000 7f9852a9a640 1 --2- 172.21.15.33:0/2046843181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c10e510 0x7f984c110900 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9844009ca0 tx=0x7f984402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.960+0000 7f9852a9a640 1 -- 172.21.15.33:0/2046843181 shutdown_connections 2024-04-02T13:39:57.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.960+0000 7f9852a9a640 1 --2- 172.21.15.33:0/2046843181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c10e510 0x7f984c110900 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.960+0000 7f9852a9a640 1 --2- 172.21.15.33:0/2046843181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f984c06aa50 0x7f984c10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.960+0000 7f9852a9a640 1 --2- 172.21.15.33:0/2046843181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f984c06a130 0x7f984c06a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.960+0000 7f9852a9a640 1 -- 172.21.15.33:0/2046843181 >> 172.21.15.33:0/2046843181 conn(0x7f984c077aa0 msgr2=0x7f984c077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.960+0000 7f9852a9a640 1 -- 172.21.15.33:0/2046843181 shutdown_connections 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.960+0000 7f9852a9a640 1 -- 172.21.15.33:0/2046843181 wait complete. 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.961+0000 7f9852a9a640 1 Processor -- start 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.961+0000 7f9852a9a640 1 -- start start 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.961+0000 7f9852a9a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f984c06a130 0x7f984c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.961+0000 7f9852a9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c06aa50 0x7f984c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.961+0000 7f9852a9a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f984c10e510 0x7f984c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.961+0000 7f9852a9a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f984c1140a0 con 0x7f984c06aa50 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.961+0000 7f9852a9a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f984c113f20 con 0x7f984c06a130 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.961+0000 7f9852a9a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f984c114220 con 0x7f984c10e510 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.961+0000 7f984bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c06aa50 0x7f984c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.961+0000 7f984bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c06aa50 0x7f984c19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52784/0 (socket says 172.21.15.33:52784) 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.961+0000 7f984bfff640 1 -- 172.21.15.33:0/2041297396 learned_addr learned my addr 172.21.15.33:0/2041297396 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:57.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.961+0000 7f9850836640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f984c06a130 0x7f984c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.962+0000 7f9851037640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f984c10e510 0x7f984c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.962+0000 7f984bfff640 1 -- 172.21.15.33:0/2041297396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f984c10e510 msgr2=0x7f984c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.962+0000 7f984bfff640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f984c10e510 0x7f984c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.962+0000 7f984bfff640 1 -- 172.21.15.33:0/2041297396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f984c06a130 msgr2=0x7f984c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.962+0000 7f984bfff640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f984c06a130 0x7f984c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.962+0000 7f984bfff640 1 -- 172.21.15.33:0/2041297396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984c19f5d0 con 0x7f984c06aa50 2024-04-02T13:39:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.962+0000 7f984bfff640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c06aa50 0x7f984c19ab20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f983800e9f0 tx=0x7f983800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.962+0000 7f9849ffb640 1 -- 172.21.15.33:0/2041297396 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983800cd80 con 0x7f984c06aa50 2024-04-02T13:39:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.962+0000 7f9849ffb640 1 -- 172.21.15.33:0/2041297396 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9838010900 con 0x7f984c06aa50 2024-04-02T13:39:57.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.962+0000 7f9852a9a640 1 -- 172.21.15.33:0/2041297396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984c19f8c0 con 0x7f984c06aa50 2024-04-02T13:39:57.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.962+0000 7f9849ffb640 1 -- 172.21.15.33:0/2041297396 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98380189e0 con 0x7f984c06aa50 2024-04-02T13:39:57.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.963+0000 7f9852a9a640 1 -- 172.21.15.33:0/2041297396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984c1a7150 con 0x7f984c06aa50 2024-04-02T13:39:57.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.964+0000 7f9852a9a640 1 -- 172.21.15.33:0/2041297396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984c100ff0 con 0x7f984c06aa50 2024-04-02T13:39:57.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.965+0000 7f9849ffb640 1 -- 172.21.15.33:0/2041297396 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f9838010ab0 con 0x7f984c06aa50 2024-04-02T13:39:57.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.966+0000 7f9849ffb640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f983c0a3f50 0x7f983c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:57.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.966+0000 7f9850836640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f983c0a3f50 0x7f983c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:57.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.966+0000 7f9849ffb640 1 -- 172.21.15.33:0/2041297396 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f98380d1290 con 0x7f984c06aa50 2024-04-02T13:39:57.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.966+0000 7f9850836640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f983c0a3f50 0x7f983c0a6410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f984000bbe0 tx=0x7f984000b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:57.970+0000 7f9849ffb640 1 -- 172.21.15.33:0/2041297396 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9838053f20 con 0x7f984c06aa50 2024-04-02T13:39:58.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.428+0000 7f9852a9a640 1 -- 172.21.15.33:0/2041297396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f984c1096a0 con 0x7f984c06aa50 2024-04-02T13:39:58.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.429+0000 7f9849ffb640 1 -- 172.21.15.33:0/2041297396 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11873 (secure 0 0 0) 0x7f983809d380 con 0x7f984c06aa50 2024-04-02T13:39:58.432 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:39:58.432 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:38:45.531661+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24473,24512]},"id":1}]} 2024-04-02T13:39:58.432 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 25 2024-04-02T13:39:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.432+0000 7f9852a9a640 1 -- 172.21.15.33:0/2041297396 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f983c0a3f50 msgr2=0x7f983c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.432+0000 7f9852a9a640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f983c0a3f50 0x7f983c0a6410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f984000bbe0 tx=0x7f984000b390 comp rx=0 tx=0).stop 2024-04-02T13:39:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.432+0000 7f9852a9a640 1 -- 172.21.15.33:0/2041297396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c06aa50 msgr2=0x7f984c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.432+0000 7f9852a9a640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c06aa50 0x7f984c19ab20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f983800e9f0 tx=0x7f983800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.432+0000 7f9852a9a640 1 -- 172.21.15.33:0/2041297396 shutdown_connections 2024-04-02T13:39:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.432+0000 7f9852a9a640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f983c0a3f50 0x7f983c0a6410 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.432+0000 7f9852a9a640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f984c10e510 0x7f984c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.432+0000 7f9852a9a640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f984c06aa50 0x7f984c19ab20 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.432+0000 7f9852a9a640 1 --2- 172.21.15.33:0/2041297396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f984c06a130 0x7f984c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.432+0000 7f9852a9a640 1 -- 172.21.15.33:0/2041297396 >> 172.21.15.33:0/2041297396 conn(0x7f984c077aa0 msgr2=0x7f984c10f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.432+0000 7f9852a9a640 1 -- 172.21.15.33:0/2041297396 shutdown_connections 2024-04-02T13:39:58.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.432+0000 7f9852a9a640 1 -- 172.21.15.33:0/2041297396 wait complete. 2024-04-02T13:39:58.449 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:39:58.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.558+0000 7fd0be63a640 1 -- 172.21.15.33:0/998504989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b8101350 msgr2=0x7fd0b810d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.558+0000 7fd0be63a640 1 --2- 172.21.15.33:0/998504989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b8101350 0x7fd0b810d850 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009cd0 tx=0x7fd0b002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.558+0000 7fd0be63a640 1 -- 172.21.15.33:0/998504989 shutdown_connections 2024-04-02T13:39:58.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.558+0000 7fd0be63a640 1 --2- 172.21.15.33:0/998504989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b8101350 0x7fd0b810d850 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.558+0000 7fd0be63a640 1 --2- 172.21.15.33:0/998504989 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0b8100990 0x7fd0b8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.558+0000 7fd0be63a640 1 --2- 172.21.15.33:0/998504989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b8106990 0x7fd0b8106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.558+0000 7fd0be63a640 1 -- 172.21.15.33:0/998504989 >> 172.21.15.33:0/998504989 conn(0x7fd0b8077ad0 msgr2=0x7fd0b8077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.558+0000 7fd0be63a640 1 -- 172.21.15.33:0/998504989 shutdown_connections 2024-04-02T13:39:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.558+0000 7fd0be63a640 1 -- 172.21.15.33:0/998504989 wait complete. 2024-04-02T13:39:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.559+0000 7fd0be63a640 1 Processor -- start 2024-04-02T13:39:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.559+0000 7fd0be63a640 1 -- start start 2024-04-02T13:39:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.559+0000 7fd0be63a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b8100990 0x7fd0b81981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.559+0000 7fd0be63a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b8101350 0x7fd0b8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.559+0000 7fd0be63a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0b8106990 0x7fd0b819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.559+0000 7fd0be63a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b8110d70 con 0x7fd0b8100990 2024-04-02T13:39:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.559+0000 7fd0be63a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b8110bf0 con 0x7fd0b8101350 2024-04-02T13:39:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.559+0000 7fd0be63a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b8110ef0 con 0x7fd0b8106990 2024-04-02T13:39:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b8100990 0x7fd0b81981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b8100990 0x7fd0b81981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52812/0 (socket says 172.21.15.33:52812) 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0b7fff640 1 -- 172.21.15.33:0/3564559034 learned_addr learned my addr 172.21.15.33:0/3564559034 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b8101350 0x7fd0b8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0bcbd7640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0b8106990 0x7fd0b819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0b7fff640 1 -- 172.21.15.33:0/3564559034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0b8106990 msgr2=0x7fd0b819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0b8106990 0x7fd0b819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0b7fff640 1 -- 172.21.15.33:0/3564559034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b8101350 msgr2=0x7fd0b8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b8101350 0x7fd0b8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0b7fff640 1 -- 172.21.15.33:0/3564559034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b819d160 con 0x7fd0b8100990 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b8101350 0x7fd0b8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0bcbd7640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0b8106990 0x7fd0b819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b8100990 0x7fd0b81981c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800e9f0 tx=0x7fd0a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.560+0000 7fd0b57fa640 1 -- 172.21.15.33:0/3564559034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a800cd80 con 0x7fd0b8100990 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.561+0000 7fd0b57fa640 1 -- 172.21.15.33:0/3564559034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd0a8010900 con 0x7fd0b8100990 2024-04-02T13:39:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.561+0000 7fd0b57fa640 1 -- 172.21.15.33:0/3564559034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a80189e0 con 0x7fd0b8100990 2024-04-02T13:39:58.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.561+0000 7fd0be63a640 1 -- 172.21.15.33:0/3564559034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b819d450 con 0x7fd0b8100990 2024-04-02T13:39:58.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.561+0000 7fd0be63a640 1 -- 172.21.15.33:0/3564559034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b81a4ce0 con 0x7fd0b8100990 2024-04-02T13:39:58.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.563+0000 7fd0b57fa640 1 -- 172.21.15.33:0/3564559034 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fd0a8010ab0 con 0x7fd0b8100990 2024-04-02T13:39:58.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.563+0000 7fd0b57fa640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd0a00a3f50 0x7fd0a00a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.564+0000 7fd0b57fa640 1 -- 172.21.15.33:0/3564559034 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd0a80d1110 con 0x7fd0b8100990 2024-04-02T13:39:58.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.564+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd0a00a3f50 0x7fd0a00a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.564+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd0a00a3f50 0x7fd0a00a6410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80780d0 tx=0x7fd0ac00b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.564+0000 7fd0c121a740 1 Processor -- start 2024-04-02T13:39:58.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.564+0000 7fd0c121a740 1 -- start start 2024-04-02T13:39:58.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0c121a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 0x55e161a95b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0c121a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a960c0 0x55e161a9e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0c121a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a9eca0 0x55e161aa1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0c121a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e1619f97b0 con 0x55e161a957a0 2024-04-02T13:39:58.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0c121a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e161998570 con 0x55e161a9eca0 2024-04-02T13:39:58.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0c121a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e1619f5760 con 0x55e161a960c0 2024-04-02T13:39:58.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 0x55e161a95b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 0x55e161a95b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52818/0 (socket says 172.21.15.33:52818) 2024-04-02T13:39:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0b7fff640 1 -- 172.21.15.33:0/1296713178 learned_addr learned my addr 172.21.15.33:0/1296713178 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0bcbd7640 1 --2- 172.21.15.33:0/1296713178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a9eca0 0x55e161aa1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/1296713178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a960c0 0x55e161a9e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0b7fff640 1 -- 172.21.15.33:0/1296713178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a960c0 msgr2=0x55e161a9e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/1296713178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a960c0 0x55e161a9e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0b7fff640 1 -- 172.21.15.33:0/1296713178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a9eca0 msgr2=0x55e161aa1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/1296713178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a9eca0 0x55e161aa1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0b7fff640 1 -- 172.21.15.33:0/1296713178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e16170d3f0 con 0x55e161a957a0 2024-04-02T13:39:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0bcbd7640 1 --2- 172.21.15.33:0/1296713178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a9eca0 0x55e161aa1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.565+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/1296713178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 0x55e161a95b80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800c6a0 tx=0x7fd0a8010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d57a090528879b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.566+0000 7fd09e7fc640 1 -- 172.21.15.33:0/1296713178 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a80b1320 con 0x55e161a957a0 2024-04-02T13:39:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.566+0000 7fd09e7fc640 1 -- 172.21.15.33:0/1296713178 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd0a80a0eb0 con 0x55e161a957a0 2024-04-02T13:39:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.566+0000 7fd09e7fc640 1 -- 172.21.15.33:0/1296713178 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8018b80 con 0x55e161a957a0 2024-04-02T13:39:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.567+0000 7fd0c121a740 1 -- 172.21.15.33:0/1296713178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 msgr2=0x55e161a95b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.567+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1296713178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 0x55e161a95b80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800c6a0 tx=0x7fd0a8010e70 comp rx=0 tx=0).stop 2024-04-02T13:39:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.567+0000 7fd0c121a740 1 -- 172.21.15.33:0/1296713178 shutdown_connections 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.567+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1296713178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a9eca0 0x55e161aa1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.567+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1296713178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a960c0 0x55e161a9e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.567+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1296713178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 0x55e161a95b80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.567+0000 7fd0c121a740 1 -- 172.21.15.33:0/1296713178 >> 172.21.15.33:0/1296713178 conn(0x55e161a8fed0 msgr2=0x55e161a902b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.567+0000 7fd0c121a740 1 -- 172.21.15.33:0/1296713178 shutdown_connections 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.567+0000 7fd0c121a740 1 -- 172.21.15.33:0/1296713178 wait complete. 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.567+0000 7fd0c121a740 1 -- start start 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.568+0000 7fd0c121a740 1 Processor -- start 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.568+0000 7fd0c121a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 0x55e161a9e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.568+0000 7fd0c121a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a960c0 0x55e161a9ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.568+0000 7fd0c121a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a9eca0 0x55e161a9c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.568+0000 7fd0c121a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e161aa1a30 con 0x55e161a957a0 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.568+0000 7fd0c121a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e1619b4e80 con 0x55e161a960c0 2024-04-02T13:39:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.568+0000 7fd0b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 0x55e161a9e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.568+0000 7fd0c121a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e161aa1bb0 con 0x55e161a9eca0 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.568+0000 7fd0c121a740 10 client.0 _ll_register_callbacks cb 0x55e1619f0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.568+0000 7fd0b7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 0x55e161a9e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52828/0 (socket says 172.21.15.33:52828) 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.568+0000 7fd0b7fff640 1 -- 172.21.15.33:0/1936236004 learned_addr learned my addr 172.21.15.33:0/1936236004 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.568+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a960c0 0x55e161a9ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.568+0000 7fd0bcbd7640 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a9eca0 0x55e161a9c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0b7fff640 1 -- 172.21.15.33:0/1936236004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a9eca0 msgr2=0x55e161a9c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a9eca0 0x55e161a9c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0b7fff640 1 -- 172.21.15.33:0/1936236004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a960c0 msgr2=0x55e161a9ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a960c0 0x55e161a9ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0b7fff640 1 -- 172.21.15.33:0/1936236004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1617d77d0 con 0x55e161a957a0 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a960c0 0x55e161a9ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 0x55e161a9e6d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800e9c0 tx=0x7fd0a80bd280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4585e52d9d2832c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0977fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0977fe640 1 -- 172.21.15.33:0/1936236004 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a80a5770 con 0x55e161a957a0 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0977fe640 1 -- 172.21.15.33:0/1936236004 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd0a8018d90 con 0x55e161a957a0 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0c121a740 1 -- 172.21.15.33:0/1936236004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0a80a0eb0 con 0x55e161a957a0 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0977fe640 1 -- 172.21.15.33:0/1936236004 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0a808e050 con 0x55e161a957a0 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0c121a740 10 client.15882 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.569+0000 7fd0c121a740 1 -- 172.21.15.33:0/1936236004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e161aad840 con 0x55e161a957a0 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.570+0000 7fd0977fe640 1 -- 172.21.15.33:0/1936236004 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd0a80a0eb0 con 0x55e161a957a0 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.570+0000 7fd0977fe640 1 -- 172.21.15.33:0/1936236004 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd0a8098850 con 0x55e161a957a0 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.570+0000 7fd0c121a740 10 client.15882 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.570+0000 7fd0c121a740 10 client.15882 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.570+0000 7fd0c121a740 20 client.15882 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.570+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55e161aaeb90 0x55e161ab0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.570+0000 7fd0c121a740 4 client.15882 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.570+0000 7fd0c121a740 1 -- 172.21.15.33:0/1936236004 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e161a06cf0 con 0x55e161aaeb90 2024-04-02T13:39:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.570+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55e161aaeb90 0x55e161ab0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.571+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55e161aaeb90 0x55e161ab0f80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75595bcca5cfef3a server_cookie=16f17c74347f7a5 in_seq=0 out_seq=0 2024-04-02T13:39:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.571+0000 7fd0977fe640 10 client.15882 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:39:58.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0977fe640 1 -- 172.21.15.33:0/1936236004 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e161a06cf0 con 0x55e161aaeb90 2024-04-02T13:39:58.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0977fe640 10 client.15882 handle_command_reply: tid=0 2024-04-02T13:39:58.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0c121a740 1 client.15882 shutdown 2024-04-02T13:39:58.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0c121a740 1 -- 172.21.15.33:0/1936236004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 msgr2=0x55e161a9e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 0x55e161a9e6d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800e9c0 tx=0x7fd0a80bd280 comp rx=0 tx=0).stop 2024-04-02T13:39:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0c121a740 1 -- 172.21.15.33:0/1936236004 shutdown_connections 2024-04-02T13:39:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55e161aaeb90 0x55e161ab0f80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a9eca0 0x55e161a9c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0977fe640 0 client.15882 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:39:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a960c0 0x55e161a9ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1936236004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a957a0 0x55e161a9e6d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0c121a740 1 -- 172.21.15.33:0/1936236004 >> 172.21.15.33:0/1936236004 conn(0x55e161a8fed0 msgr2=0x55e161a91c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0c121a740 1 -- 172.21.15.33:0/1936236004 shutdown_connections 2024-04-02T13:39:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0c121a740 1 -- 172.21.15.33:0/1936236004 wait complete. 2024-04-02T13:39:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.572+0000 7fd0c121a740 20 client.15882 trim_cache size 0 max 16384 2024-04-02T13:39:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.599+0000 7fd0c121a740 1 Processor -- start 2024-04-02T13:39:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.599+0000 7fd0c121a740 1 -- start start 2024-04-02T13:39:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.599+0000 7fd0c121a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a957a0 0x55e161a9c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0c121a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 0x55e161a902b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0c121a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a9eca0 0x55e161a9e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0c121a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b81a6cb0 con 0x55e161a960c0 2024-04-02T13:39:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0c121a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b80733a0 con 0x55e161a9eca0 2024-04-02T13:39:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0c121a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b80711d0 con 0x55e161a957a0 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0b77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 0x55e161a902b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0b77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 0x55e161a902b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52832/0 (socket says 172.21.15.33:52832) 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0b77fe640 1 -- 172.21.15.33:0/293846644 learned_addr learned my addr 172.21.15.33:0/293846644 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/293846644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a957a0 0x55e161a9c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0bcbd7640 1 --2- 172.21.15.33:0/293846644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a9eca0 0x55e161a9e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0b77fe640 1 -- 172.21.15.33:0/293846644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a957a0 msgr2=0x55e161a9c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/293846644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a957a0 0x55e161a9c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0b77fe640 1 -- 172.21.15.33:0/293846644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a9eca0 msgr2=0x55e161a9e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/293846644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a9eca0 0x55e161a9e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0b77fe640 1 -- 172.21.15.33:0/293846644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e161a907f0 con 0x55e161a960c0 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.600+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/293846644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 0x55e161a902b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00bfd0 tx=0x7fd0ac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89b4b08344b8273 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.601+0000 7fd09dffb640 1 -- 172.21.15.33:0/293846644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0320d0 con 0x55e161a960c0 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.601+0000 7fd09dffb640 1 -- 172.21.15.33:0/293846644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd0ac032e80 con 0x55e161a960c0 2024-04-02T13:39:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.601+0000 7fd09dffb640 1 -- 172.21.15.33:0/293846644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac004a00 con 0x55e161a960c0 2024-04-02T13:39:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.601+0000 7fd0c121a740 1 -- 172.21.15.33:0/293846644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 msgr2=0x55e161a902b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.601+0000 7fd0c121a740 1 --2- 172.21.15.33:0/293846644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 0x55e161a902b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00bfd0 tx=0x7fd0ac013000 comp rx=0 tx=0).stop 2024-04-02T13:39:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.601+0000 7fd0c121a740 1 -- 172.21.15.33:0/293846644 shutdown_connections 2024-04-02T13:39:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.601+0000 7fd0c121a740 1 --2- 172.21.15.33:0/293846644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a9eca0 0x55e161a9e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.601+0000 7fd0c121a740 1 --2- 172.21.15.33:0/293846644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 0x55e161a902b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.601+0000 7fd0c121a740 1 --2- 172.21.15.33:0/293846644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a957a0 0x55e161a9c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.601+0000 7fd0c121a740 1 -- 172.21.15.33:0/293846644 >> 172.21.15.33:0/293846644 conn(0x55e161a8fed0 msgr2=0x55e161a9b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.602+0000 7fd0c121a740 1 -- 172.21.15.33:0/293846644 shutdown_connections 2024-04-02T13:39:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.602+0000 7fd0c121a740 1 -- 172.21.15.33:0/293846644 wait complete. 2024-04-02T13:39:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.602+0000 7fd0c121a740 1 -- start start 2024-04-02T13:39:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.602+0000 7fd0c121a740 1 Processor -- start 2024-04-02T13:39:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.602+0000 7fd0c121a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a957a0 0x55e161aaf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.602+0000 7fd0c121a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 0x55e161aaf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.602+0000 7fd0c121a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a9eca0 0x55e161b0cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.602+0000 7fd0c121a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b8071860 con 0x55e161a960c0 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.602+0000 7fd0c121a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b8072cd0 con 0x55e161a957a0 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.602+0000 7fd0c121a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b8071f30 con 0x55e161a9eca0 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.602+0000 7fd0c121a740 10 client.0 _ll_register_callbacks cb 0x7fd0a80ce250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.602+0000 7fd0b77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 0x55e161aaf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0b77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 0x55e161aaf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52842/0 (socket says 172.21.15.33:52842) 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0b77fe640 1 -- 172.21.15.33:0/1153359181 learned_addr learned my addr 172.21.15.33:0/1153359181 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a957a0 0x55e161aaf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0bcbd7640 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a9eca0 0x55e161b0cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0b7fff640 1 -- 172.21.15.33:0/1153359181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a9eca0 msgr2=0x55e161b0cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a9eca0 0x55e161b0cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0b7fff640 1 -- 172.21.15.33:0/1153359181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 msgr2=0x55e161aaf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 0x55e161aaf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0b7fff640 1 -- 172.21.15.33:0/1153359181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e161b0d2e0 con 0x55e161a957a0 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 0x55e161aaf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0b7fff640 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a957a0 0x55e161aaf200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80a5930 tx=0x7fd0a80ce7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e6c6ff968a8d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0977fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0977fe640 1 -- 172.21.15.33:0/1153359181 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a80a5770 con 0x55e161a957a0 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0c121a740 1 -- 172.21.15.33:0/1153359181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e161a01040 con 0x55e161a957a0 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.603+0000 7fd0977fe640 1 -- 172.21.15.33:0/1153359181 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd0a8010430 con 0x55e161a957a0 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.604+0000 7fd0977fe640 1 -- 172.21.15.33:0/1153359181 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0a809bd30 con 0x55e161a957a0 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.604+0000 7fd0c121a740 10 client.25496 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.604+0000 7fd0c121a740 1 -- 172.21.15.33:0/1153359181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e161b0d2e0 con 0x55e161a957a0 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.604+0000 7fd0977fe640 1 -- 172.21.15.33:0/1153359181 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55e161a01040 con 0x55e161a957a0 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.604+0000 7fd0977fe640 1 -- 172.21.15.33:0/1153359181 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd0a80a2040 con 0x55e161a957a0 2024-04-02T13:39:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.604+0000 7fd0c121a740 10 client.25496 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.604+0000 7fd0c121a740 10 client.25496 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.604+0000 7fd0c121a740 20 client.25496 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.604+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55e161aaeb90 0x55e161b16a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.604+0000 7fd0c121a740 4 client.25496 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.604+0000 7fd0c121a740 1 -- 172.21.15.33:0/1153359181 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e16197e6b0 con 0x55e161aaeb90 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.604+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55e161aaeb90 0x55e161b16a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0977fe640 10 client.25496 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0b77fe640 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55e161aaeb90 0x55e161b16a20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbe9eb4bc5095a09 server_cookie=abd712afbabdf2af in_seq=0 out_seq=0 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0977fe640 1 -- 172.21.15.33:0/1153359181 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e16197e6b0 con 0x55e161aaeb90 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0977fe640 10 client.25496 handle_command_reply: tid=0 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0c121a740 1 client.25496 shutdown 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0c121a740 1 -- 172.21.15.33:0/1153359181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a957a0 msgr2=0x55e161aaf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a957a0 0x55e161aaf200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80a5930 tx=0x7fd0a80ce7d0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0c121a740 1 -- 172.21.15.33:0/1153359181 shutdown_connections 2024-04-02T13:39:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55e161aaeb90 0x55e161b16a20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e161a9eca0 0x55e161b0cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0977fe640 0 client.25496 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:39:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e161a960c0 0x55e161aaf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0c121a740 1 --2- 172.21.15.33:0/1153359181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e161a957a0 0x55e161aaf200 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0c121a740 1 -- 172.21.15.33:0/1153359181 >> 172.21.15.33:0/1153359181 conn(0x55e161a8fed0 msgr2=0x55e161a8e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0c121a740 1 -- 172.21.15.33:0/1153359181 shutdown_connections 2024-04-02T13:39:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.605+0000 7fd0c121a740 1 -- 172.21.15.33:0/1153359181 wait complete. 2024-04-02T13:39:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.606+0000 7fd0c121a740 20 client.25496 trim_cache size 0 max 16384 2024-04-02T13:39:58.613 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:39:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.606+0000 7fd0be63a640 1 -- 172.21.15.33:0/3564559034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd0a00a3f50 msgr2=0x7fd0a00a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.606+0000 7fd0be63a640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd0a00a3f50 0x7fd0a00a6410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80780d0 tx=0x7fd0ac00b390 comp rx=0 tx=0).stop 2024-04-02T13:39:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.606+0000 7fd0be63a640 1 -- 172.21.15.33:0/3564559034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b8100990 msgr2=0x7fd0b81981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.606+0000 7fd0be63a640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b8100990 0x7fd0b81981c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800e9f0 tx=0x7fd0a800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.606+0000 7fd0be63a640 1 -- 172.21.15.33:0/3564559034 shutdown_connections 2024-04-02T13:39:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.606+0000 7fd0be63a640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd0a00a3f50 0x7fd0a00a6410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.606+0000 7fd0be63a640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0b8106990 0x7fd0b819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.606+0000 7fd0be63a640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0b8101350 0x7fd0b8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.606+0000 7fd0be63a640 1 --2- 172.21.15.33:0/3564559034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b8100990 0x7fd0b81981c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.606+0000 7fd0be63a640 1 -- 172.21.15.33:0/3564559034 >> 172.21.15.33:0/3564559034 conn(0x7fd0b8077ad0 msgr2=0x7fd0b806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.606+0000 7fd0be63a640 1 -- 172.21.15.33:0/3564559034 shutdown_connections 2024-04-02T13:39:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.606+0000 7fd0be63a640 1 -- 172.21.15.33:0/3564559034 wait complete. 2024-04-02T13:39:58.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:58 smithi033 ceph-mon[32390]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:58.624 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:39:58.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:58 smithi145 ceph-mon[35646]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.734+0000 7f490f3d1640 1 -- 172.21.15.33:0/2692544772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4908107b50 msgr2=0x7f4908107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.734+0000 7f490f3d1640 1 --2- 172.21.15.33:0/2692544772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4908107b50 0x7f4908107f30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4904009930 tx=0x7f490402bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.734+0000 7f490f3d1640 1 -- 172.21.15.33:0/2692544772 shutdown_connections 2024-04-02T13:39:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.734+0000 7f490f3d1640 1 --2- 172.21.15.33:0/2692544772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f49080696c0 0x7f490810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.734+0000 7f490f3d1640 1 --2- 172.21.15.33:0/2692544772 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4908068d00 0x7f4908069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.734+0000 7f490f3d1640 1 --2- 172.21.15.33:0/2692544772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4908107b50 0x7f4908107f30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.734+0000 7f490f3d1640 1 -- 172.21.15.33:0/2692544772 >> 172.21.15.33:0/2692544772 conn(0x7f4908077b00 msgr2=0x7f4908077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.734+0000 7f490f3d1640 1 -- 172.21.15.33:0/2692544772 shutdown_connections 2024-04-02T13:39:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.734+0000 7f490f3d1640 1 -- 172.21.15.33:0/2692544772 wait complete. 2024-04-02T13:39:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.735+0000 7f490f3d1640 1 Processor -- start 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.735+0000 7f490f3d1640 1 -- start start 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.735+0000 7f490f3d1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4908068d00 0x7f4908198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.735+0000 7f490f3d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49080696c0 0x7f49081988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.735+0000 7f490f3d1640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4908107b50 0x7f490819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.735+0000 7f490f3d1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4908111ec0 con 0x7f49080696c0 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.735+0000 7f490f3d1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4908111d40 con 0x7f4908068d00 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.735+0000 7f490f3d1640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4908112040 con 0x7f4908107b50 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.735+0000 7f490c96c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49080696c0 0x7f49081988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.735+0000 7f490c96c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49080696c0 0x7f49081988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52862/0 (socket says 172.21.15.33:52862) 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.735+0000 7f490c96c640 1 -- 172.21.15.33:0/3855829412 learned_addr learned my addr 172.21.15.33:0/3855829412 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f490d16d640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4908068d00 0x7f4908198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f490d96e640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4908107b50 0x7f490819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f490c96c640 1 -- 172.21.15.33:0/3855829412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4908107b50 msgr2=0x7f490819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f490c96c640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4908107b50 0x7f490819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f490c96c640 1 -- 172.21.15.33:0/3855829412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4908068d00 msgr2=0x7f4908198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f490c96c640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4908068d00 0x7f4908198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f490c96c640 1 -- 172.21.15.33:0/3855829412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490819d320 con 0x7f49080696c0 2024-04-02T13:39:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f490d16d640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4908068d00 0x7f4908198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f490d96e640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4908107b50 0x7f490819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f490c96c640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49080696c0 0x7f49081988c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f48f800e9f0 tx=0x7f48f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f48fe7fc640 1 -- 172.21.15.33:0/3855829412 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f800cd80 con 0x7f49080696c0 2024-04-02T13:39:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f490f3d1640 1 -- 172.21.15.33:0/3855829412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490819d610 con 0x7f49080696c0 2024-04-02T13:39:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f48fe7fc640 1 -- 172.21.15.33:0/3855829412 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f48f80054f0 con 0x7f49080696c0 2024-04-02T13:39:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f48fe7fc640 1 -- 172.21.15.33:0/3855829412 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8010690 con 0x7f49080696c0 2024-04-02T13:39:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.736+0000 7f490f3d1640 1 -- 172.21.15.33:0/3855829412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49081a4ea0 con 0x7f49080696c0 2024-04-02T13:39:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.739+0000 7f48fe7fc640 1 -- 172.21.15.33:0/3855829412 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f48f8004f20 con 0x7f49080696c0 2024-04-02T13:39:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.740+0000 7f4911fb0740 1 Processor -- start 2024-04-02T13:39:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.740+0000 7f4911fb0740 1 -- start start 2024-04-02T13:39:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.740+0000 7f4911fb0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 0x55f8fe378b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.740+0000 7f4911fb0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe381760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.740+0000 7f4911fb0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe381ca0 0x55f8fe384090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.740+0000 7f4911fb0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f8fe2dc7b0 con 0x55f8fe3787a0 2024-04-02T13:39:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.740+0000 7f4911fb0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f8fe27b570 con 0x55f8fe3790c0 2024-04-02T13:39:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.740+0000 7f4911fb0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f8fe2d8760 con 0x55f8fe381ca0 2024-04-02T13:39:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.740+0000 7f490d96e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 0x55f8fe378b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.740+0000 7f490d96e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 0x55f8fe378b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52876/0 (socket says 172.21.15.33:52876) 2024-04-02T13:39:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.740+0000 7f490d96e640 1 -- 172.21.15.33:0/1736380869 learned_addr learned my addr 172.21.15.33:0/1736380869 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.740+0000 7f490d16d640 1 --2- 172.21.15.33:0/1736380869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe381760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.740+0000 7f48fe7fc640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f48f40a3f50 0x7f48f40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f490d96e640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f48f40a3f50 0x7f48f40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f490c96c640 1 --2- 172.21.15.33:0/1736380869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe381ca0 0x55f8fe384090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f48fe7fc640 1 -- 172.21.15.33:0/3855829412 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f48f80d1350 con 0x7f49080696c0 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f490d16d640 1 -- 172.21.15.33:0/1736380869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe381ca0 msgr2=0x55f8fe384090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f490d16d640 1 --2- 172.21.15.33:0/1736380869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe381ca0 0x55f8fe384090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f490d16d640 1 -- 172.21.15.33:0/1736380869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 msgr2=0x55f8fe378b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f490d16d640 1 --2- 172.21.15.33:0/1736380869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 0x55f8fe378b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f490d16d640 1 -- 172.21.15.33:0/1736380869 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8fdff03f0 con 0x55f8fe3790c0 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f490d96e640 1 --2- 172.21.15.33:0/1736380869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 0x55f8fe378b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f490c96c640 1 --2- 172.21.15.33:0/1736380869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe381ca0 0x55f8fe384090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f490d16d640 1 --2- 172.21.15.33:0/1736380869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe381760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f490402c170 tx=0x7f4904005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38f141bfea62ff35 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f490d96e640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f48f40a3f50 0x7f48f40a6410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4900009fd0 tx=0x7f4900004ab0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f48f37fe640 1 -- 172.21.15.33:0/1736380869 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904004240 con 0x55f8fe3790c0 2024-04-02T13:39:58.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f48f37fe640 1 -- 172.21.15.33:0/1736380869 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f49040043e0 con 0x55f8fe3790c0 2024-04-02T13:39:58.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.741+0000 7f48f37fe640 1 -- 172.21.15.33:0/1736380869 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904004900 con 0x55f8fe3790c0 2024-04-02T13:39:58.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:58 smithi050 ceph-mon[35524]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:39:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.742+0000 7f4911fb0740 1 -- 172.21.15.33:0/1736380869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 msgr2=0x55f8fe381760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.742+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1736380869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe381760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f490402c170 tx=0x7f4904005e50 comp rx=0 tx=0).stop 2024-04-02T13:39:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.743+0000 7f4911fb0740 1 -- 172.21.15.33:0/1736380869 shutdown_connections 2024-04-02T13:39:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.743+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1736380869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe381ca0 0x55f8fe384090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.743+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1736380869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe381760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.743+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1736380869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 0x55f8fe378b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.743+0000 7f4911fb0740 1 -- 172.21.15.33:0/1736380869 >> 172.21.15.33:0/1736380869 conn(0x55f8fe372ed0 msgr2=0x55f8fe3732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.743+0000 7f4911fb0740 1 -- 172.21.15.33:0/1736380869 shutdown_connections 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.743+0000 7f4911fb0740 1 -- 172.21.15.33:0/1736380869 wait complete. 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.743+0000 7f4911fb0740 1 -- start start 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.743+0000 7f4911fb0740 1 Processor -- start 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.743+0000 7f4911fb0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe3787a0 0x55f8fe381330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f4911fb0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe37a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f4911fb0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe381ca0 0x55f8fe37abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f4911fb0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f8fe388e40 con 0x55f8fe381ca0 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f4911fb0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f8fe297e80 con 0x55f8fe3790c0 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f4911fb0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f8fe388fc0 con 0x55f8fe3787a0 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f4911fb0740 10 client.0 _ll_register_callbacks cb 0x55f8fe2d3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f490d96e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe381ca0 0x55f8fe37abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f490d16d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe3787a0 0x55f8fe381330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f490d96e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe381ca0 0x55f8fe37abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52888/0 (socket says 172.21.15.33:52888) 2024-04-02T13:39:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f490d96e640 1 -- 172.21.15.33:0/3618079441 learned_addr learned my addr 172.21.15.33:0/3618079441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f490c96c640 1 --2- 172.21.15.33:0/3618079441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe37a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f490d96e640 1 -- 172.21.15.33:0/3618079441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe3787a0 msgr2=0x55f8fe381330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f490d96e640 1 --2- 172.21.15.33:0/3618079441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe3787a0 0x55f8fe381330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f490d96e640 1 -- 172.21.15.33:0/3618079441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 msgr2=0x55f8fe37a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f490d96e640 1 --2- 172.21.15.33:0/3618079441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe37a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f490d96e640 1 -- 172.21.15.33:0/3618079441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8fe0ba7d0 con 0x55f8fe381ca0 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f490d96e640 1 --2- 172.21.15.33:0/3618079441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe381ca0 0x55f8fe37abc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f49000133c0 tx=0x7f4900004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa748b28e902127 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f48f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.744+0000 7f48f0ff9640 1 -- 172.21.15.33:0/3618079441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900025020 con 0x55f8fe381ca0 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.745+0000 7f48f0ff9640 1 -- 172.21.15.33:0/3618079441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4900011d40 con 0x55f8fe381ca0 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.745+0000 7f4911fb0740 1 -- 172.21.15.33:0/3618079441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49040043e0 con 0x55f8fe381ca0 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.745+0000 7f48f0ff9640 1 -- 172.21.15.33:0/3618079441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4900030050 con 0x55f8fe381ca0 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.745+0000 7f4911fb0740 10 client.15900 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.745+0000 7f4911fb0740 1 -- 172.21.15.33:0/3618079441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8fe3992f0 con 0x55f8fe381ca0 2024-04-02T13:39:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.745+0000 7f48f0ff9640 1 -- 172.21.15.33:0/3618079441 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f49040043e0 con 0x55f8fe381ca0 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.745+0000 7f48f0ff9640 1 -- 172.21.15.33:0/3618079441 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f49000103d0 con 0x55f8fe381ca0 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.745+0000 7f4911fb0740 10 client.15900 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.745+0000 7f4911fb0740 10 client.15900 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.745+0000 7f4911fb0740 20 client.15900 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.745+0000 7f4911fb0740 1 --2- 172.21.15.33:0/3618079441 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f8fe39a800 0x55f8fe39cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.745+0000 7f4911fb0740 4 client.15900 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.745+0000 7f4911fb0740 1 -- 172.21.15.33:0/3618079441 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8fe2e9cf0 con 0x55f8fe39a800 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.746+0000 7f490d16d640 1 --2- 172.21.15.33:0/3618079441 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f8fe39a800 0x55f8fe39cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.746+0000 7f490d16d640 1 --2- 172.21.15.33:0/3618079441 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f8fe39a800 0x55f8fe39cc30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61778bca5918729c server_cookie=bed5eef42321011e in_seq=0 out_seq=0 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.746+0000 7f48f0ff9640 10 client.15900 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f48f0ff9640 1 -- 172.21.15.33:0/3618079441 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f8fe2e9cf0 con 0x55f8fe39a800 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f48f0ff9640 10 client.15900 handle_command_reply: tid=0 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f4911fb0740 1 client.15900 shutdown 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f4911fb0740 1 -- 172.21.15.33:0/3618079441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe381ca0 msgr2=0x55f8fe37abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f4911fb0740 1 --2- 172.21.15.33:0/3618079441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe381ca0 0x55f8fe37abc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f49000133c0 tx=0x7f4900004a40 comp rx=0 tx=0).stop 2024-04-02T13:39:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f4911fb0740 1 -- 172.21.15.33:0/3618079441 shutdown_connections 2024-04-02T13:39:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f4911fb0740 1 --2- 172.21.15.33:0/3618079441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe381ca0 0x55f8fe37abc0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f4911fb0740 1 --2- 172.21.15.33:0/3618079441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe37a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f4911fb0740 1 --2- 172.21.15.33:0/3618079441 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f8fe39a800 0x55f8fe39cc30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f48f0ff9640 0 client.15900 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:39:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f4911fb0740 1 --2- 172.21.15.33:0/3618079441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe3787a0 0x55f8fe381330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f4911fb0740 1 -- 172.21.15.33:0/3618079441 >> 172.21.15.33:0/3618079441 conn(0x55f8fe372ed0 msgr2=0x55f8fe374c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f4911fb0740 1 -- 172.21.15.33:0/3618079441 shutdown_connections 2024-04-02T13:39:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f4911fb0740 1 -- 172.21.15.33:0/3618079441 wait complete. 2024-04-02T13:39:58.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.747+0000 7f4911fb0740 20 client.15900 trim_cache size 0 max 16384 2024-04-02T13:39:58.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.772+0000 7f4911fb0740 1 Processor -- start 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f4911fb0740 1 -- start start 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f4911fb0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe372ed0 0x55f8fe37eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f4911fb0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 0x55f8fe380e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f4911fb0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe3a2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f4911fb0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49081af490 con 0x55f8fe3787a0 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f4911fb0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f49081acd60 con 0x55f8fe3790c0 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f4911fb0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f49081aab90 con 0x55f8fe372ed0 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f490c96c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 0x55f8fe380e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f490c96c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 0x55f8fe380e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52904/0 (socket says 172.21.15.33:52904) 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f490c96c640 1 -- 172.21.15.33:0/1501495098 learned_addr learned my addr 172.21.15.33:0/1501495098 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f490d96e640 1 --2- 172.21.15.33:0/1501495098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe3a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f490d16d640 1 --2- 172.21.15.33:0/1501495098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe372ed0 0x55f8fe37eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f490c96c640 1 -- 172.21.15.33:0/1501495098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe372ed0 msgr2=0x55f8fe37eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f490c96c640 1 --2- 172.21.15.33:0/1501495098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe372ed0 0x55f8fe37eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f490c96c640 1 -- 172.21.15.33:0/1501495098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 msgr2=0x55f8fe3a2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f490c96c640 1 --2- 172.21.15.33:0/1501495098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe3a2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.773+0000 7f490c96c640 1 -- 172.21.15.33:0/1501495098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8fe378ef0 con 0x55f8fe3787a0 2024-04-02T13:39:58.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.774+0000 7f490c96c640 1 --2- 172.21.15.33:0/1501495098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 0x55f8fe380e60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f48f80049d0 tx=0x7f48f80bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa55292ad0d2f9b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.774+0000 7f48f2ffd640 1 -- 172.21.15.33:0/1501495098 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f80b6070 con 0x55f8fe3787a0 2024-04-02T13:39:58.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.774+0000 7f48f2ffd640 1 -- 172.21.15.33:0/1501495098 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f48f80a10f0 con 0x55f8fe3787a0 2024-04-02T13:39:58.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.774+0000 7f48f2ffd640 1 -- 172.21.15.33:0/1501495098 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8005210 con 0x55f8fe3787a0 2024-04-02T13:39:58.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.774+0000 7f4911fb0740 1 -- 172.21.15.33:0/1501495098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 msgr2=0x55f8fe380e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.774+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1501495098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 0x55f8fe380e60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f48f80049d0 tx=0x7f48f80bd040 comp rx=0 tx=0).stop 2024-04-02T13:39:58.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.775+0000 7f4911fb0740 1 -- 172.21.15.33:0/1501495098 shutdown_connections 2024-04-02T13:39:58.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.775+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1501495098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe3a2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.775+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1501495098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe3787a0 0x55f8fe380e60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.775+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1501495098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe372ed0 0x55f8fe37eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.775+0000 7f4911fb0740 1 -- 172.21.15.33:0/1501495098 >> 172.21.15.33:0/1501495098 conn(0x55f8fe3886d0 msgr2=0x55f8fe3732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.775+0000 7f4911fb0740 1 -- 172.21.15.33:0/1501495098 shutdown_connections 2024-04-02T13:39:58.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.775+0000 7f4911fb0740 1 -- 172.21.15.33:0/1501495098 wait complete. 2024-04-02T13:39:58.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.775+0000 7f4911fb0740 1 -- start start 2024-04-02T13:39:58.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.775+0000 7f4911fb0740 1 Processor -- start 2024-04-02T13:39:58.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f4911fb0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe372ed0 0x55f8fe387500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f4911fb0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe3787a0 0x55f8fe396aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f4911fb0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe396fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f4911fb0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49081ab220 con 0x55f8fe372ed0 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f4911fb0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f49081ac690 con 0x55f8fe3790c0 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f4911fb0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f49081ab8f0 con 0x55f8fe3787a0 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f490d16d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe372ed0 0x55f8fe387500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f4911fb0740 10 client.0 _ll_register_callbacks cb 0x7f49000134a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f490d16d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe372ed0 0x55f8fe387500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52916/0 (socket says 172.21.15.33:52916) 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f490d16d640 1 -- 172.21.15.33:0/1546356775 learned_addr learned my addr 172.21.15.33:0/1546356775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f490c96c640 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe3787a0 0x55f8fe396aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f490d96e640 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe396fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f490d16d640 1 -- 172.21.15.33:0/1546356775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe3787a0 msgr2=0x55f8fe396aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f490d16d640 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe3787a0 0x55f8fe396aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f490d16d640 1 -- 172.21.15.33:0/1546356775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 msgr2=0x55f8fe396fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f490d16d640 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe396fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f490d16d640 1 -- 172.21.15.33:0/1546356775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8fe3976d0 con 0x55f8fe372ed0 2024-04-02T13:39:58.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f490d96e640 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe396fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:58.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f490d16d640 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe372ed0 0x55f8fe387500 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4904009930 tx=0x7f49040056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77065a6ea8596cf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.776+0000 7f48f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:58.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f48f0ff9640 1 -- 172.21.15.33:0/1546356775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904050dc0 con 0x55f8fe372ed0 2024-04-02T13:39:58.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f48f0ff9640 1 -- 172.21.15.33:0/1546356775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f490402c810 con 0x55f8fe372ed0 2024-04-02T13:39:58.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f4911fb0740 1 -- 172.21.15.33:0/1546356775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48f80a10f0 con 0x55f8fe372ed0 2024-04-02T13:39:58.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f48f0ff9640 1 -- 172.21.15.33:0/1546356775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f490402ca10 con 0x55f8fe372ed0 2024-04-02T13:39:58.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f4911fb0740 10 client.15906 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:58.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f4911fb0740 1 -- 172.21.15.33:0/1546356775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8fe400d70 con 0x55f8fe372ed0 2024-04-02T13:39:58.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f48f0ff9640 1 -- 172.21.15.33:0/1546356775 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f48f80a10f0 con 0x55f8fe372ed0 2024-04-02T13:39:58.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f48f0ff9640 1 -- 172.21.15.33:0/1546356775 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4904058020 con 0x55f8fe372ed0 2024-04-02T13:39:58.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f4911fb0740 10 client.15906 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:58.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f4911fb0740 10 client.15906 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:39:58.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f4911fb0740 20 client.15906 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f8fe381ca0 0x55f8fe4030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f4911fb0740 4 client.15906 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.777+0000 7f4911fb0740 1 -- 172.21.15.33:0/1546356775 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8fe2e4040 con 0x55f8fe381ca0 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.778+0000 7f490c96c640 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f8fe381ca0 0x55f8fe4030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.778+0000 7f48f0ff9640 10 client.15906 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.778+0000 7f490c96c640 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f8fe381ca0 0x55f8fe4030d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=423c60e78b0bc3f5 server_cookie=13bc516e121b29b1 in_seq=0 out_seq=0 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.778+0000 7f48f0ff9640 1 -- 172.21.15.33:0/1546356775 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8fe2e4040 con 0x55f8fe381ca0 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.778+0000 7f48f0ff9640 10 client.15906 handle_command_reply: tid=0 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.778+0000 7f4911fb0740 1 client.15906 shutdown 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.778+0000 7f4911fb0740 1 -- 172.21.15.33:0/1546356775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe372ed0 msgr2=0x55f8fe387500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.778+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe372ed0 0x55f8fe387500 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4904009930 tx=0x7f49040056a0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.778+0000 7f4911fb0740 1 -- 172.21.15.33:0/1546356775 shutdown_connections 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.778+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f8fe3790c0 0x55f8fe396fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.779+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f8fe381ca0 0x55f8fe4030d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.779+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f8fe3787a0 0x55f8fe396aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.779+0000 7f4911fb0740 1 --2- 172.21.15.33:0/1546356775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8fe372ed0 0x55f8fe387500 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.779+0000 7f48f0ff9640 0 client.15906 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:39:58.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.779+0000 7f4911fb0740 1 -- 172.21.15.33:0/1546356775 >> 172.21.15.33:0/1546356775 conn(0x55f8fe3886d0 msgr2=0x55f8fe3732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.779+0000 7f4911fb0740 1 -- 172.21.15.33:0/1546356775 shutdown_connections 2024-04-02T13:39:58.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.779+0000 7f4911fb0740 1 -- 172.21.15.33:0/1546356775 wait complete. 2024-04-02T13:39:58.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.779+0000 7f4911fb0740 20 client.15906 trim_cache size 0 max 16384 2024-04-02T13:39:58.787 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:39:58.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.779+0000 7f490f3d1640 1 -- 172.21.15.33:0/3855829412 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f48f40a3f50 msgr2=0x7f48f40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.779+0000 7f490f3d1640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f48f40a3f50 0x7f48f40a6410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4900009fd0 tx=0x7f4900004ab0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.779+0000 7f490f3d1640 1 -- 172.21.15.33:0/3855829412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49080696c0 msgr2=0x7f49081988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.779+0000 7f490f3d1640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49080696c0 0x7f49081988c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f48f800e9f0 tx=0x7f48f800edf0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.780+0000 7f490f3d1640 1 -- 172.21.15.33:0/3855829412 shutdown_connections 2024-04-02T13:39:58.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.780+0000 7f490f3d1640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f48f40a3f50 0x7f48f40a6410 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.780+0000 7f490f3d1640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4908107b50 0x7f490819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.780+0000 7f490f3d1640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49080696c0 0x7f49081988c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.780+0000 7f490f3d1640 1 --2- 172.21.15.33:0/3855829412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4908068d00 0x7f4908198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.780+0000 7f490f3d1640 1 -- 172.21.15.33:0/3855829412 >> 172.21.15.33:0/3855829412 conn(0x7f4908077b00 msgr2=0x7f49080fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.780+0000 7f490f3d1640 1 -- 172.21.15.33:0/3855829412 shutdown_connections 2024-04-02T13:39:58.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.780+0000 7f490f3d1640 1 -- 172.21.15.33:0/3855829412 wait complete. 2024-04-02T13:39:58.793 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:39:58.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.900+0000 7fa47243c640 1 -- 172.21.15.33:0/2526391325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c107b50 msgr2=0x7fa46c107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.900+0000 7fa47243c640 1 --2- 172.21.15.33:0/2526391325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c107b50 0x7fa46c107f30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009930 tx=0x7fa45c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.900+0000 7fa47243c640 1 -- 172.21.15.33:0/2526391325 shutdown_connections 2024-04-02T13:39:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.900+0000 7fa47243c640 1 --2- 172.21.15.33:0/2526391325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa46c0696c0 0x7fa46c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.900+0000 7fa47243c640 1 --2- 172.21.15.33:0/2526391325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa46c068d00 0x7fa46c069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.900+0000 7fa47243c640 1 --2- 172.21.15.33:0/2526391325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c107b50 0x7fa46c107f30 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.900+0000 7fa47243c640 1 -- 172.21.15.33:0/2526391325 >> 172.21.15.33:0/2526391325 conn(0x7fa46c077b00 msgr2=0x7fa46c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.900+0000 7fa47243c640 1 -- 172.21.15.33:0/2526391325 shutdown_connections 2024-04-02T13:39:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.900+0000 7fa47243c640 1 -- 172.21.15.33:0/2526391325 wait complete. 2024-04-02T13:39:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.901+0000 7fa47243c640 1 Processor -- start 2024-04-02T13:39:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.901+0000 7fa47243c640 1 -- start start 2024-04-02T13:39:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.901+0000 7fa47243c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa46c068d00 0x7fa46c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.901+0000 7fa47243c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c0696c0 0x7fa46c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.901+0000 7fa47243c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa46c107b50 0x7fa46c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.901+0000 7fa47243c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c111e40 con 0x7fa46c0696c0 2024-04-02T13:39:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.901+0000 7fa47243c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c111cc0 con 0x7fa46c068d00 2024-04-02T13:39:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.901+0000 7fa47243c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c111fc0 con 0x7fa46c107b50 2024-04-02T13:39:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.902+0000 7fa46b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c0696c0 0x7fa46c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.902+0000 7fa46b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c0696c0 0x7fa46c198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52936/0 (socket says 172.21.15.33:52936) 2024-04-02T13:39:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.902+0000 7fa46b7fe640 1 -- 172.21.15.33:0/1682085438 learned_addr learned my addr 172.21.15.33:0/1682085438 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.902+0000 7fa46bfff640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa46c068d00 0x7fa46c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.902+0000 7fa4709d9640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa46c107b50 0x7fa46c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.902+0000 7fa46b7fe640 1 -- 172.21.15.33:0/1682085438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa46c107b50 msgr2=0x7fa46c19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.902+0000 7fa46b7fe640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa46c107b50 0x7fa46c19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.902+0000 7fa46b7fe640 1 -- 172.21.15.33:0/1682085438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa46c068d00 msgr2=0x7fa46c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.902+0000 7fa46b7fe640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa46c068d00 0x7fa46c198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.902+0000 7fa46b7fe640 1 -- 172.21.15.33:0/1682085438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46c19d310 con 0x7fa46c0696c0 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.902+0000 7fa4709d9640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa46c107b50 0x7fa46c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.902+0000 7fa46bfff640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa46c068d00 0x7fa46c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.902+0000 7fa46b7fe640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c0696c0 0x7fa46c198860 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa46000d950 tx=0x7fa46000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.903+0000 7fa4697fa640 1 -- 172.21.15.33:0/1682085438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460017070 con 0x7fa46c0696c0 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.903+0000 7fa4697fa640 1 -- 172.21.15.33:0/1682085438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa460009c30 con 0x7fa46c0696c0 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.903+0000 7fa47243c640 1 -- 172.21.15.33:0/1682085438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa46c19d600 con 0x7fa46c0696c0 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.903+0000 7fa4697fa640 1 -- 172.21.15.33:0/1682085438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46000f6b0 con 0x7fa46c0696c0 2024-04-02T13:39:58.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.903+0000 7fa47243c640 1 -- 172.21.15.33:0/1682085438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa46c1008f0 con 0x7fa46c0696c0 2024-04-02T13:39:58.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.905+0000 7fa4697fa640 1 -- 172.21.15.33:0/1682085438 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fa460010a50 con 0x7fa46c0696c0 2024-04-02T13:39:58.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.906+0000 7fa4697fa640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa4540a4020 0x7fa4540a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.906+0000 7fa46bfff640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa4540a4020 0x7fa4540a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.907+0000 7fa46bfff640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa4540a4020 0x7fa4540a64e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa45c000c00 tx=0x7fa45c005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.907+0000 7fa4697fa640 1 -- 172.21.15.33:0/1682085438 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa4600d0980 con 0x7fa46c0696c0 2024-04-02T13:39:58.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.907+0000 7fa475023740 1 Processor -- start 2024-04-02T13:39:58.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa475023740 1 -- start start 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa475023740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f1b7a0 0x5637a5f1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa475023740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f24760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa475023740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f27090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa475023740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5637a5e7f7b0 con 0x5637a5f24ca0 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa475023740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5637a5e1e570 con 0x5637a5f1b7a0 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa475023740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5637a5e7b760 con 0x5637a5f1c0c0 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa4709d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f27090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa4709d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f27090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52950/0 (socket says 172.21.15.33:52950) 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa4709d9640 1 -- 172.21.15.33:0/1165992690 learned_addr learned my addr 172.21.15.33:0/1165992690 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa46b7fe640 1 --2- 172.21.15.33:0/1165992690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f24760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa46bfff640 1 --2- 172.21.15.33:0/1165992690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f1b7a0 0x5637a5f1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa4709d9640 1 -- 172.21.15.33:0/1165992690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 msgr2=0x5637a5f24760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa4709d9640 1 --2- 172.21.15.33:0/1165992690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f24760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa4709d9640 1 -- 172.21.15.33:0/1165992690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f1b7a0 msgr2=0x5637a5f1bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa4709d9640 1 --2- 172.21.15.33:0/1165992690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f1b7a0 0x5637a5f1bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.908+0000 7fa4709d9640 1 -- 172.21.15.33:0/1165992690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a5b933f0 con 0x5637a5f24ca0 2024-04-02T13:39:58.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.909+0000 7fa4709d9640 1 --2- 172.21.15.33:0/1165992690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f27090 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa464009fd0 tx=0x7fa4640108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11577085c1db6c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.909+0000 7fa4527fc640 1 -- 172.21.15.33:0/1165992690 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464011760 con 0x5637a5f24ca0 2024-04-02T13:39:58.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.909+0000 7fa4527fc640 1 -- 172.21.15.33:0/1165992690 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa464011dd0 con 0x5637a5f24ca0 2024-04-02T13:39:58.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.909+0000 7fa4527fc640 1 -- 172.21.15.33:0/1165992690 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46400e3c0 con 0x5637a5f24ca0 2024-04-02T13:39:58.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.910+0000 7fa475023740 1 -- 172.21.15.33:0/1165992690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 msgr2=0x5637a5f27090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.910+0000 7fa475023740 1 --2- 172.21.15.33:0/1165992690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f27090 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa464009fd0 tx=0x7fa4640108e0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.910+0000 7fa475023740 1 -- 172.21.15.33:0/1165992690 shutdown_connections 2024-04-02T13:39:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.910+0000 7fa475023740 1 --2- 172.21.15.33:0/1165992690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f27090 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.910+0000 7fa475023740 1 --2- 172.21.15.33:0/1165992690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f24760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.910+0000 7fa475023740 1 --2- 172.21.15.33:0/1165992690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f1b7a0 0x5637a5f1bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.910+0000 7fa475023740 1 -- 172.21.15.33:0/1165992690 >> 172.21.15.33:0/1165992690 conn(0x5637a5f15ed0 msgr2=0x5637a5f162b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.910+0000 7fa475023740 1 -- 172.21.15.33:0/1165992690 shutdown_connections 2024-04-02T13:39:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.910+0000 7fa475023740 1 -- 172.21.15.33:0/1165992690 wait complete. 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.910+0000 7fa475023740 1 -- start start 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.910+0000 7fa475023740 1 Processor -- start 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa475023740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f1b7a0 0x5637a5f246b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa475023740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f21a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa475023740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f22050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa475023740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5637a5f27a90 con 0x5637a5f24ca0 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa475023740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5637a5e3ae80 con 0x5637a5f1b7a0 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa475023740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5637a5f27c10 con 0x5637a5f1c0c0 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa475023740 10 client.0 _ll_register_callbacks cb 0x5637a5e76aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa4709d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f22050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa4709d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f22050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52954/0 (socket says 172.21.15.33:52954) 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa4709d9640 1 -- 172.21.15.33:0/1917369950 learned_addr learned my addr 172.21.15.33:0/1917369950 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa46bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f1b7a0 0x5637a5f246b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa46b7fe640 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f21a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa4709d9640 1 -- 172.21.15.33:0/1917369950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 msgr2=0x5637a5f21a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa4709d9640 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f21a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa4709d9640 1 -- 172.21.15.33:0/1917369950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f1b7a0 msgr2=0x5637a5f246b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa4709d9640 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f1b7a0 0x5637a5f246b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa4709d9640 1 -- 172.21.15.33:0/1917369950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a5c5d7d0 con 0x5637a5f24ca0 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa46b7fe640 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f21a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa46bfff640 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f1b7a0 0x5637a5f246b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa4709d9640 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f22050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa4640092f0 tx=0x7fa46401b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f158069fcefcb5d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.911+0000 7fa44b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.912+0000 7fa44b7fe640 1 -- 172.21.15.33:0/1917369950 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46402a060 con 0x5637a5f24ca0 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.912+0000 7fa44b7fe640 1 -- 172.21.15.33:0/1917369950 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa464004dd0 con 0x5637a5f24ca0 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.912+0000 7fa475023740 1 -- 172.21.15.33:0/1917369950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa464011dd0 con 0x5637a5f24ca0 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.912+0000 7fa44b7fe640 1 -- 172.21.15.33:0/1917369950 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa464011560 con 0x5637a5f24ca0 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.912+0000 7fa475023740 10 client.15918 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.912+0000 7fa475023740 1 -- 172.21.15.33:0/1917369950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5637a5f33ac0 con 0x5637a5f24ca0 2024-04-02T13:39:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.912+0000 7fa44b7fe640 1 -- 172.21.15.33:0/1917369950 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa464011dd0 con 0x5637a5f24ca0 2024-04-02T13:39:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.912+0000 7fa44b7fe640 1 -- 172.21.15.33:0/1917369950 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa464005790 con 0x5637a5f24ca0 2024-04-02T13:39:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.913+0000 7fa475023740 10 client.15918 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.913+0000 7fa475023740 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.913+0000 7fa475023740 20 client.15918 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.913+0000 7fa475023740 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5637a5f34e70 0x5637a5f37260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.913+0000 7fa475023740 4 client.15918 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.913+0000 7fa475023740 1 -- 172.21.15.33:0/1917369950 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637a5e8ccf0 con 0x5637a5f34e70 2024-04-02T13:39:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.913+0000 7fa46bfff640 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5637a5f34e70 0x5637a5f37260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.914+0000 7fa44b7fe640 10 client.15918 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:58.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.914+0000 7fa46bfff640 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5637a5f34e70 0x5637a5f37260 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c10de75decdb01 server_cookie=23817ab7853a0743 in_seq=0 out_seq=0 2024-04-02T13:39:58.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.914+0000 7fa44b7fe640 1 -- 172.21.15.33:0/1917369950 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637a5e8ccf0 con 0x5637a5f34e70 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.914+0000 7fa44b7fe640 10 client.15918 handle_command_reply: tid=0 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.914+0000 7fa475023740 1 client.15918 shutdown 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.915+0000 7fa475023740 1 -- 172.21.15.33:0/1917369950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 msgr2=0x5637a5f22050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.915+0000 7fa475023740 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f22050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa4640092f0 tx=0x7fa46401b9f0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.915+0000 7fa475023740 1 -- 172.21.15.33:0/1917369950 shutdown_connections 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.915+0000 7fa475023740 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5637a5f34e70 0x5637a5f37260 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.915+0000 7fa475023740 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f22050 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.915+0000 7fa475023740 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f21a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.915+0000 7fa475023740 1 --2- 172.21.15.33:0/1917369950 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f1b7a0 0x5637a5f246b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.915+0000 7fa475023740 1 -- 172.21.15.33:0/1917369950 >> 172.21.15.33:0/1917369950 conn(0x5637a5f15ed0 msgr2=0x5637a5f17c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.915+0000 7fa44b7fe640 0 client.15918 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.915+0000 7fa475023740 1 -- 172.21.15.33:0/1917369950 shutdown_connections 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.915+0000 7fa475023740 1 -- 172.21.15.33:0/1917369950 wait complete. 2024-04-02T13:39:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.915+0000 7fa475023740 20 client.15918 trim_cache size 0 max 16384 2024-04-02T13:39:58.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.928+0000 7fa475023740 1 Processor -- start 2024-04-02T13:39:58.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.928+0000 7fa475023740 1 -- start start 2024-04-02T13:39:58.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa475023740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f1b7a0 0x5637a5f24170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa475023740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f246b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa475023740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f24ca0 0x5637a5f1ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa475023740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1b39c0 con 0x5637a5f1b7a0 2024-04-02T13:39:58.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa475023740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1b1290 con 0x5637a5f24ca0 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa475023740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1af0c0 con 0x5637a5f1c0c0 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa46bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f1b7a0 0x5637a5f24170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa46bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f1b7a0 0x5637a5f24170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52956/0 (socket says 172.21.15.33:52956) 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa46bfff640 1 -- 172.21.15.33:0/4079934834 learned_addr learned my addr 172.21.15.33:0/4079934834 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa46b7fe640 1 --2- 172.21.15.33:0/4079934834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f246b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa4709d9640 1 --2- 172.21.15.33:0/4079934834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f24ca0 0x5637a5f1ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa46bfff640 1 -- 172.21.15.33:0/4079934834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 msgr2=0x5637a5f246b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa46bfff640 1 --2- 172.21.15.33:0/4079934834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f246b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa46bfff640 1 -- 172.21.15.33:0/4079934834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f24ca0 msgr2=0x5637a5f1ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa46bfff640 1 --2- 172.21.15.33:0/4079934834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f24ca0 0x5637a5f1ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa46bfff640 1 -- 172.21.15.33:0/4079934834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a5f1b590 con 0x5637a5f1b7a0 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.929+0000 7fa46b7fe640 1 --2- 172.21.15.33:0/4079934834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f246b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.930+0000 7fa46bfff640 1 --2- 172.21.15.33:0/4079934834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f1b7a0 0x5637a5f24170 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009230 tx=0x7fa45c043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2395c521f0ceea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.930+0000 7fa451ffb640 1 -- 172.21.15.33:0/4079934834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c057330 con 0x5637a5f1b7a0 2024-04-02T13:39:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.930+0000 7fa451ffb640 1 -- 172.21.15.33:0/4079934834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa45c05d070 con 0x5637a5f1b7a0 2024-04-02T13:39:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.930+0000 7fa451ffb640 1 -- 172.21.15.33:0/4079934834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c03f440 con 0x5637a5f1b7a0 2024-04-02T13:39:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.931+0000 7fa475023740 1 -- 172.21.15.33:0/4079934834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f1b7a0 msgr2=0x5637a5f24170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.931+0000 7fa475023740 1 --2- 172.21.15.33:0/4079934834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f1b7a0 0x5637a5f24170 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009230 tx=0x7fa45c043000 comp rx=0 tx=0).stop 2024-04-02T13:39:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.931+0000 7fa475023740 1 -- 172.21.15.33:0/4079934834 shutdown_connections 2024-04-02T13:39:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.931+0000 7fa475023740 1 --2- 172.21.15.33:0/4079934834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f24ca0 0x5637a5f1ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.931+0000 7fa475023740 1 --2- 172.21.15.33:0/4079934834 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f246b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.931+0000 7fa475023740 1 --2- 172.21.15.33:0/4079934834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f1b7a0 0x5637a5f24170 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.931+0000 7fa475023740 1 -- 172.21.15.33:0/4079934834 >> 172.21.15.33:0/4079934834 conn(0x5637a5f15ed0 msgr2=0x5637a5f21a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.931+0000 7fa475023740 1 -- 172.21.15.33:0/4079934834 shutdown_connections 2024-04-02T13:39:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.931+0000 7fa475023740 1 -- 172.21.15.33:0/4079934834 wait complete. 2024-04-02T13:39:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.931+0000 7fa475023740 1 -- start start 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.931+0000 7fa475023740 1 Processor -- start 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.931+0000 7fa475023740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f3a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa475023740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa475023740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f35d30 0x5637a5f361b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa475023740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1af750 con 0x5637a5f24ca0 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa475023740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1b0bc0 con 0x5637a5f35d30 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa475023740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c1afe20 con 0x5637a5f1c0c0 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa475023740 10 client.0 _ll_register_callbacks cb 0x7fa45c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa46b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa46b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f357f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52958/0 (socket says 172.21.15.33:52958) 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa46b7fe640 1 -- 172.21.15.33:0/3724739156 learned_addr learned my addr 172.21.15.33:0/3724739156 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa46bfff640 1 --2- 172.21.15.33:0/3724739156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f3a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa4709d9640 1 --2- 172.21.15.33:0/3724739156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f35d30 0x5637a5f361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa46b7fe640 1 -- 172.21.15.33:0/3724739156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 msgr2=0x5637a5f3a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa46b7fe640 1 --2- 172.21.15.33:0/3724739156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f3a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa46b7fe640 1 -- 172.21.15.33:0/3724739156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f35d30 msgr2=0x5637a5f361b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa46b7fe640 1 --2- 172.21.15.33:0/3724739156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f35d30 0x5637a5f361b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa46b7fe640 1 -- 172.21.15.33:0/3724739156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a5fa2850 con 0x5637a5f24ca0 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa46b7fe640 1 --2- 172.21.15.33:0/3724739156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f357f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa46009c300 tx=0x7fa46009bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bad2abf7d13d69 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa44b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.932+0000 7fa44b7fe640 1 -- 172.21.15.33:0/3724739156 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460020040 con 0x5637a5f24ca0 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.933+0000 7fa44b7fe640 1 -- 172.21.15.33:0/3724739156 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa46009b9c0 con 0x5637a5f24ca0 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.933+0000 7fa475023740 1 -- 172.21.15.33:0/3724739156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa45c05d070 con 0x5637a5f24ca0 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.933+0000 7fa44b7fe640 1 -- 172.21.15.33:0/3724739156 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa4600a4fe0 con 0x5637a5f24ca0 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.933+0000 7fa475023740 10 client.15924 fetch_fsmap learned FSMap version 25 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.933+0000 7fa475023740 1 -- 172.21.15.33:0/3724739156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5637a5fa3a90 con 0x5637a5f24ca0 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.933+0000 7fa44b7fe640 1 -- 172.21.15.33:0/3724739156 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa45c05d070 con 0x5637a5f24ca0 2024-04-02T13:39:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.933+0000 7fa44b7fe640 1 -- 172.21.15.33:0/3724739156 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa4600a0170 con 0x5637a5f24ca0 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.933+0000 7fa475023740 10 client.15924 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.933+0000 7fa475023740 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.933+0000 7fa475023740 20 client.15924 populate_metadata read hostname 'smithi033' 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.933+0000 7fa475023740 1 --2- 172.21.15.33:0/3724739156 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5637a5fa4290 0x5637a5fa66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.933+0000 7fa475023740 4 client.15924 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.933+0000 7fa475023740 1 -- 172.21.15.33:0/3724739156 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637a5e87040 con 0x5637a5fa4290 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa46bfff640 1 --2- 172.21.15.33:0/3724739156 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5637a5fa4290 0x5637a5fa66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa44b7fe640 10 client.15924 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa46bfff640 1 --2- 172.21.15.33:0/3724739156 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5637a5fa4290 0x5637a5fa66b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d6d8fd572476bf server_cookie=9bec8b799dc22ac0 in_seq=0 out_seq=0 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa44b7fe640 1 -- 172.21.15.33:0/3724739156 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637a5e87040 con 0x5637a5fa4290 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa44b7fe640 10 client.15924 handle_command_reply: tid=0 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa475023740 1 client.15924 shutdown 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa475023740 1 -- 172.21.15.33:0/3724739156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 msgr2=0x5637a5f357f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa475023740 1 --2- 172.21.15.33:0/3724739156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f357f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa46009c300 tx=0x7fa46009bdf0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa475023740 1 -- 172.21.15.33:0/3724739156 shutdown_connections 2024-04-02T13:39:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa475023740 1 --2- 172.21.15.33:0/3724739156 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5637a5fa4290 0x5637a5fa66b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa44b7fe640 0 client.15924 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:39:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa475023740 1 --2- 172.21.15.33:0/3724739156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637a5f35d30 0x5637a5f361b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa475023740 1 --2- 172.21.15.33:0/3724739156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637a5f24ca0 0x5637a5f357f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa475023740 1 --2- 172.21.15.33:0/3724739156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637a5f1c0c0 0x5637a5f3a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.934+0000 7fa475023740 1 -- 172.21.15.33:0/3724739156 >> 172.21.15.33:0/3724739156 conn(0x5637a5f15ed0 msgr2=0x5637a5f20310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa475023740 1 -- 172.21.15.33:0/3724739156 shutdown_connections 2024-04-02T13:39:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa475023740 1 -- 172.21.15.33:0/3724739156 wait complete. 2024-04-02T13:39:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa475023740 20 client.15924 trim_cache size 0 max 16384 2024-04-02T13:39:58.942 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:39:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa47243c640 1 -- 172.21.15.33:0/1682085438 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa4540a4020 msgr2=0x7fa4540a64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa47243c640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa4540a4020 0x7fa4540a64e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa45c000c00 tx=0x7fa45c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa47243c640 1 -- 172.21.15.33:0/1682085438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c0696c0 msgr2=0x7fa46c198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa47243c640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c0696c0 0x7fa46c198860 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa46000d950 tx=0x7fa46000de10 comp rx=0 tx=0).stop 2024-04-02T13:39:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa47243c640 1 -- 172.21.15.33:0/1682085438 shutdown_connections 2024-04-02T13:39:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa47243c640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa4540a4020 0x7fa4540a64e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa47243c640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa46c107b50 0x7fa46c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa47243c640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c0696c0 0x7fa46c198860 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa47243c640 1 --2- 172.21.15.33:0/1682085438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa46c068d00 0x7fa46c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa47243c640 1 -- 172.21.15.33:0/1682085438 >> 172.21.15.33:0/1682085438 conn(0x7fa46c077b00 msgr2=0x7fa46c0fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa47243c640 1 -- 172.21.15.33:0/1682085438 shutdown_connections 2024-04-02T13:39:58.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:39:58.935+0000 7fa47243c640 1 -- 172.21.15.33:0/1682085438 wait complete. 2024-04-02T13:39:59.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:39:59 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2041297396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:39:59.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:39:59 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2041297396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:39:59.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:39:59 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2041297396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:39:59.950 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:39:59.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:39:59.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:40:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.064+0000 7f6c88c7b640 1 -- 172.21.15.33:0/3915857157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c84102da0 msgr2=0x7f6c841004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.064+0000 7f6c88c7b640 1 --2- 172.21.15.33:0/3915857157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c84102da0 0x7f6c841004e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009900 tx=0x7f6c7802bcd0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.064+0000 7f6c88c7b640 1 -- 172.21.15.33:0/3915857157 shutdown_connections 2024-04-02T13:40:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.064+0000 7f6c88c7b640 1 --2- 172.21.15.33:0/3915857157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c84100b00 0x7f6c84110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.064+0000 7f6c88c7b640 1 --2- 172.21.15.33:0/3915857157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c84102da0 0x7f6c841004e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.064+0000 7f6c88c7b640 1 --2- 172.21.15.33:0/3915857157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c841029c0 0x7f6c840fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.064+0000 7f6c88c7b640 1 -- 172.21.15.33:0/3915857157 >> 172.21.15.33:0/3915857157 conn(0x7f6c84076f20 msgr2=0x7f6c840fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:00.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.064+0000 7f6c88c7b640 1 -- 172.21.15.33:0/3915857157 shutdown_connections 2024-04-02T13:40:00.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.064+0000 7f6c88c7b640 1 -- 172.21.15.33:0/3915857157 wait complete. 2024-04-02T13:40:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.065+0000 7f6c88c7b640 1 Processor -- start 2024-04-02T13:40:00.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.065+0000 7f6c88c7b640 1 -- start start 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.065+0000 7f6c88c7b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c84100b00 0x7f6c8419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.065+0000 7f6c88c7b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c841029c0 0x7f6c8419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.065+0000 7f6c88c7b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c84102da0 0x7f6c8419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.065+0000 7f6c88c7b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84114040 con 0x7f6c841029c0 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.065+0000 7f6c88c7b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84113ec0 con 0x7f6c84102da0 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.065+0000 7f6c88c7b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841141c0 con 0x7f6c84100b00 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.065+0000 7f6c81d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c841029c0 0x7f6c8419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.065+0000 7f6c81d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c841029c0 0x7f6c8419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52978/0 (socket says 172.21.15.33:52978) 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.065+0000 7f6c81d9b640 1 -- 172.21.15.33:0/1015874161 learned_addr learned my addr 172.21.15.33:0/1015874161 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.065+0000 7f6c82d9d640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c84102da0 0x7f6c8419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.065+0000 7f6c8259c640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c84100b00 0x7f6c8419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.066+0000 7f6c81d9b640 1 -- 172.21.15.33:0/1015874161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c84100b00 msgr2=0x7f6c8419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.066+0000 7f6c81d9b640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c84100b00 0x7f6c8419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.066+0000 7f6c81d9b640 1 -- 172.21.15.33:0/1015874161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c84102da0 msgr2=0x7f6c8419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.066+0000 7f6c81d9b640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c84102da0 0x7f6c8419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.066+0000 7f6c81d9b640 1 -- 172.21.15.33:0/1015874161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8419f5a0 con 0x7f6c841029c0 2024-04-02T13:40:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.066+0000 7f6c81d9b640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c841029c0 0x7f6c8419ab40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005940 tx=0x7f6c78004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.066+0000 7f6c737fe640 1 -- 172.21.15.33:0/1015874161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78043020 con 0x7f6c841029c0 2024-04-02T13:40:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.066+0000 7f6c88c7b640 1 -- 172.21.15.33:0/1015874161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8419f830 con 0x7f6c841029c0 2024-04-02T13:40:00.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.066+0000 7f6c88c7b640 1 -- 172.21.15.33:0/1015874161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c841a7120 con 0x7f6c841029c0 2024-04-02T13:40:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.068+0000 7f6c737fe640 1 -- 172.21.15.33:0/1015874161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6c7802c670 con 0x7f6c841029c0 2024-04-02T13:40:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.068+0000 7f6c737fe640 1 -- 172.21.15.33:0/1015874161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78003ab0 con 0x7f6c841029c0 2024-04-02T13:40:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.069+0000 7f6c8b866740 1 Processor -- start 2024-04-02T13:40:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.069+0000 7f6c8b866740 1 -- start start 2024-04-02T13:40:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.069+0000 7f6c8b866740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d6e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.069+0000 7f6c8b866740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d77280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.069+0000 7f6c8b866740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 0x55f7e8d79bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.069+0000 7f6c8b866740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f7e8cd2770 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.069+0000 7f6c8b866740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f7e8c7a960 con 0x55f7e8d6e2c0 2024-04-02T13:40:00.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.069+0000 7f6c8b866740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f7e8cce8c0 con 0x55f7e8d777c0 2024-04-02T13:40:00.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.070+0000 7f6c8259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d77280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.070+0000 7f6c8259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d77280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52980/0 (socket says 172.21.15.33:52980) 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.070+0000 7f6c8259c640 1 -- 172.21.15.33:0/2316641050 learned_addr learned my addr 172.21.15.33:0/2316641050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.070+0000 7f6c82d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d6e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.070+0000 7f6c8259c640 1 -- 172.21.15.33:0/2316641050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 msgr2=0x55f7e8d79bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.070+0000 7f6c8259c640 1 --2- 172.21.15.33:0/2316641050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 0x55f7e8d79bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.070+0000 7f6c8259c640 1 -- 172.21.15.33:0/2316641050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d6e2c0 msgr2=0x55f7e8d6e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.070+0000 7f6c8259c640 1 --2- 172.21.15.33:0/2316641050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d6e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.070+0000 7f6c8259c640 1 -- 172.21.15.33:0/2316641050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7e8aae7e0 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.070+0000 7f6c737fe640 1 -- 172.21.15.33:0/1015874161 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f6c78046020 con 0x7f6c841029c0 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.070+0000 7f6c8259c640 1 --2- 172.21.15.33:0/2316641050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d77280 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009c10 tx=0x7f6c740117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d4bc8a75fabf33 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.070+0000 7f6c70ff9640 1 -- 172.21.15.33:0/2316641050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74012450 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.071+0000 7f6c70ff9640 1 -- 172.21.15.33:0/2316641050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6c74012ac0 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.071+0000 7f6c70ff9640 1 -- 172.21.15.33:0/2316641050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400fb50 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.071+0000 7f6c737fe640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6c6c0a3f50 0x7f6c6c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.071+0000 7f6c82d9d640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6c6c0a3f50 0x7f6c6c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.071+0000 7f6c737fe640 1 -- 172.21.15.33:0/1015874161 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6c780f2bd0 con 0x7f6c841029c0 2024-04-02T13:40:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 -- 172.21.15.33:0/2316641050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 msgr2=0x55f7e8d77280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 --2- 172.21.15.33:0/2316641050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d77280 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009c10 tx=0x7f6c740117b0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c82d9d640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6c6c0a3f50 0x7f6c6c0a6410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c005fd0 tx=0x7f6c7c009600 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 -- 172.21.15.33:0/2316641050 shutdown_connections 2024-04-02T13:40:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 --2- 172.21.15.33:0/2316641050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 0x55f7e8d79bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 --2- 172.21.15.33:0/2316641050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d77280 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 --2- 172.21.15.33:0/2316641050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d6e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 -- 172.21.15.33:0/2316641050 >> 172.21.15.33:0/2316641050 conn(0x55f7e8d689f0 msgr2=0x55f7e8d68dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 -- 172.21.15.33:0/2316641050 shutdown_connections 2024-04-02T13:40:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 -- 172.21.15.33:0/2316641050 wait complete. 2024-04-02T13:40:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 -- start start 2024-04-02T13:40:00.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 Processor -- start 2024-04-02T13:40:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d74790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d74cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.072+0000 7f6c8b866740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d777c0 0x55f7e8d75210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c8b866740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f7e8d7a600 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c8b866740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f7e8c8dd20 con 0x55f7e8d777c0 2024-04-02T13:40:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c8b866740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f7e8d7a780 con 0x55f7e8d6e2c0 2024-04-02T13:40:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c8b866740 10 client.0 _ll_register_callbacks cb 0x55f7e8c5aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c81d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d74cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c81d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d74cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52984/0 (socket says 172.21.15.33:52984) 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c81d9b640 1 -- 172.21.15.33:0/1564815745 learned_addr learned my addr 172.21.15.33:0/1564815745 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c8259c640 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d74790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c82d9d640 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d777c0 0x55f7e8d75210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c81d9b640 1 -- 172.21.15.33:0/1564815745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d6e2c0 msgr2=0x55f7e8d74790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c81d9b640 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d74790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c81d9b640 1 -- 172.21.15.33:0/1564815745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d777c0 msgr2=0x55f7e8d75210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c81d9b640 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d777c0 0x55f7e8d75210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c81d9b640 1 -- 172.21.15.33:0/1564815745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7e8b0ef70 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c8259c640 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d74790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c81d9b640 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d74cd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6c78003700 tx=0x7f6c780be210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5449c8dc1ff14f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c69ffb640 1 -- 172.21.15.33:0/1564815745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78003e60 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.073+0000 7f6c69ffb640 1 -- 172.21.15.33:0/1564815745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6c780c70b0 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.074+0000 7f6c8b866740 1 -- 172.21.15.33:0/1564815745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c74012ac0 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.074+0000 7f6c69ffb640 1 -- 172.21.15.33:0/1564815745 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6c780d4030 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.074+0000 7f6c8b866740 10 client.15936 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.074+0000 7f6c8b866740 1 -- 172.21.15.33:0/1564815745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f7e89a6170 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.074+0000 7f6c69ffb640 1 -- 172.21.15.33:0/1564815745 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6c780d7e50 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.074+0000 7f6c69ffb640 1 -- 172.21.15.33:0/1564815745 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6c780d7020 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.074+0000 7f6c8b866740 10 client.15936 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.074+0000 7f6c8b866740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.074+0000 7f6c8b866740 20 client.15936 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.074+0000 7f6c8b866740 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f7e8d87b80 0x55f7e8d89fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.074+0000 7f6c8b866740 4 client.15936 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.074+0000 7f6c8b866740 1 -- 172.21.15.33:0/1564815745 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7e8cdfcb0 con 0x55f7e8d87b80 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.075+0000 7f6c8259c640 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f7e8d87b80 0x55f7e8d89fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.075+0000 7f6c69ffb640 10 client.15936 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.075+0000 7f6c8259c640 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f7e8d87b80 0x55f7e8d89fa0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=830ddfe47b96f786 server_cookie=d5c236b72c27234c in_seq=0 out_seq=0 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.076+0000 7f6c69ffb640 1 -- 172.21.15.33:0/1564815745 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f7e8cdfcb0 con 0x55f7e8d87b80 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.076+0000 7f6c69ffb640 10 client.15936 handle_command_reply: tid=0 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.076+0000 7f6c8b866740 1 client.15936 shutdown 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.076+0000 7f6c8b866740 1 -- 172.21.15.33:0/1564815745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 msgr2=0x55f7e8d74cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.076+0000 7f6c8b866740 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d74cd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6c78003700 tx=0x7f6c780be210 comp rx=0 tx=0).stop 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.076+0000 7f6c8b866740 1 -- 172.21.15.33:0/1564815745 shutdown_connections 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.076+0000 7f6c8b866740 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f7e8d87b80 0x55f7e8d89fa0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.076+0000 7f6c69ffb640 0 client.15936 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.076+0000 7f6c8b866740 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d777c0 0x55f7e8d75210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.076+0000 7f6c8b866740 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d74cd0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.076+0000 7f6c8b866740 1 --2- 172.21.15.33:0/1564815745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d74790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.076+0000 7f6c8b866740 1 -- 172.21.15.33:0/1564815745 >> 172.21.15.33:0/1564815745 conn(0x55f7e8d689f0 msgr2=0x55f7e8d6a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.077+0000 7f6c8b866740 1 -- 172.21.15.33:0/1564815745 shutdown_connections 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.077+0000 7f6c8b866740 1 -- 172.21.15.33:0/1564815745 wait complete. 2024-04-02T13:40:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.077+0000 7f6c8b866740 20 client.15936 trim_cache size 0 max 16384 2024-04-02T13:40:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.112+0000 7f6c8b866740 1 Processor -- start 2024-04-02T13:40:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.112+0000 7f6c8b866740 1 -- start start 2024-04-02T13:40:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.112+0000 7f6c8b866740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d76ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.112+0000 7f6c8b866740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d83520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.112+0000 7f6c8b866740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 0x55f7e8d6a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.112+0000 7f6c8b866740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64012750 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.112+0000 7f6c8b866740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64010020 con 0x55f7e8d6e2c0 2024-04-02T13:40:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.112+0000 7f6c8b866740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6400de50 con 0x55f7e8d777c0 2024-04-02T13:40:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.112+0000 7f6c8259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d76ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.112+0000 7f6c81d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d83520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.113+0000 7f6c8259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d76ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:45046/0 (socket says 172.21.15.33:45046) 2024-04-02T13:40:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.113+0000 7f6c8259c640 1 -- 172.21.15.33:0/3576366259 learned_addr learned my addr 172.21.15.33:0/3576366259 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.113+0000 7f6c82d9d640 1 --2- 172.21.15.33:0/3576366259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 0x55f7e8d6a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.113+0000 7f6c81d9b640 1 -- 172.21.15.33:0/3576366259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 msgr2=0x55f7e8d6a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.113+0000 7f6c81d9b640 1 --2- 172.21.15.33:0/3576366259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 0x55f7e8d6a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.113+0000 7f6c81d9b640 1 -- 172.21.15.33:0/3576366259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d6e2c0 msgr2=0x55f7e8d76ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.113+0000 7f6c81d9b640 1 --2- 172.21.15.33:0/3576366259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d76ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.113+0000 7f6c81d9b640 1 -- 172.21.15.33:0/3576366259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7e8d6ea10 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.113+0000 7f6c81d9b640 1 --2- 172.21.15.33:0/3576366259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d83520 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6c78000c00 tx=0x7f6c780be340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d962b89c5c4c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.113+0000 7f6c6bfff640 1 -- 172.21.15.33:0/3576366259 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78003e60 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.113+0000 7f6c6bfff640 1 -- 172.21.15.33:0/3576366259 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6c780d7e50 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.113+0000 7f6c6bfff640 1 -- 172.21.15.33:0/3576366259 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c780be030 con 0x55f7e8d6ebe0 2024-04-02T13:40:00.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.114+0000 7f6c8b866740 1 -- 172.21.15.33:0/3576366259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 msgr2=0x55f7e8d83520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.114+0000 7f6c8b866740 1 --2- 172.21.15.33:0/3576366259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d83520 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6c78000c00 tx=0x7f6c780be340 comp rx=0 tx=0).stop 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.114+0000 7f6c8b866740 1 -- 172.21.15.33:0/3576366259 shutdown_connections 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.114+0000 7f6c8b866740 1 --2- 172.21.15.33:0/3576366259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 0x55f7e8d6a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.114+0000 7f6c8b866740 1 --2- 172.21.15.33:0/3576366259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6ebe0 0x55f7e8d83520 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.114+0000 7f6c8b866740 1 --2- 172.21.15.33:0/3576366259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d76ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.114+0000 7f6c8b866740 1 -- 172.21.15.33:0/3576366259 >> 172.21.15.33:0/3576366259 conn(0x55f7e8d689f0 msgr2=0x55f7e8d69e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.114+0000 7f6c8b866740 1 -- 172.21.15.33:0/3576366259 shutdown_connections 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.114+0000 7f6c8b866740 1 -- 172.21.15.33:0/3576366259 wait complete. 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.115+0000 7f6c8b866740 1 -- start start 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.115+0000 7f6c8b866740 1 Processor -- start 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.115+0000 7f6c8b866740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d7f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.115+0000 7f6c8b866740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 0x55f7e8d7fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.115+0000 7f6c8259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d7f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.115+0000 7f6c8259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d7f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53008/0 (socket says 172.21.15.33:53008) 2024-04-02T13:40:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.115+0000 7f6c8b866740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d87b80 0x55f7e8d7bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.115+0000 7f6c8259c640 1 -- 172.21.15.33:0/1805316794 learned_addr learned my addr 172.21.15.33:0/1805316794 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.115+0000 7f6c8b866740 1 -- 172.21.15.33:0/1805316794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6400e4e0 con 0x55f7e8d6e2c0 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.115+0000 7f6c8b866740 1 -- 172.21.15.33:0/1805316794 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6400f950 con 0x55f7e8d87b80 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.115+0000 7f6c8b866740 1 -- 172.21.15.33:0/1805316794 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6400ebb0 con 0x55f7e8d777c0 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.115+0000 7f6c81d9b640 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 0x55f7e8d7fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c82d9d640 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d87b80 0x55f7e8d7bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c8b866740 10 client.0 _ll_register_callbacks cb 0x7f6c780ba970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c8259c640 1 -- 172.21.15.33:0/1805316794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 msgr2=0x55f7e8d7fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c8259c640 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 0x55f7e8d7fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c8259c640 1 -- 172.21.15.33:0/1805316794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d87b80 msgr2=0x55f7e8d7bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c8259c640 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d87b80 0x55f7e8d7bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c8259c640 1 -- 172.21.15.33:0/1805316794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7e8d7c2b0 con 0x55f7e8d6e2c0 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c82d9d640 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d87b80 0x55f7e8d7bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:40:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c8259c640 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d7f740 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6c7401c040 tx=0x7f6c7400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5612410838b9519 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c69ffb640 1 -- 172.21.15.33:0/1805316794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74034020 con 0x55f7e8d6e2c0 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c69ffb640 1 -- 172.21.15.33:0/1805316794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6c74012860 con 0x55f7e8d6e2c0 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c8b866740 1 -- 172.21.15.33:0/1805316794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c780d7e50 con 0x55f7e8d6e2c0 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.116+0000 7f6c69ffb640 1 -- 172.21.15.33:0/1805316794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6c7403c050 con 0x55f7e8d6e2c0 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.117+0000 7f6c8b866740 10 client.15942 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.117+0000 7f6c8b866740 1 -- 172.21.15.33:0/1805316794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f7e8deeb50 con 0x55f7e8d6e2c0 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.117+0000 7f6c69ffb640 1 -- 172.21.15.33:0/1805316794 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6c7400fcd0 con 0x55f7e8d6e2c0 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.117+0000 7f6c69ffb640 1 -- 172.21.15.33:0/1805316794 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6c740035f0 con 0x55f7e8d6e2c0 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.117+0000 7f6c8b866740 10 client.15942 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.117+0000 7f6c8b866740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.117+0000 7f6c8b866740 20 client.15942 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.117+0000 7f6c8b866740 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f7e8def050 0x55f7e8df1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.117+0000 7f6c8b866740 4 client.15942 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.117+0000 7f6c8b866740 1 -- 172.21.15.33:0/1805316794 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f6c780d7e50 con 0x55f7e8def050 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.118+0000 7f6c81d9b640 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f7e8def050 0x55f7e8df1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.118+0000 7f6c81d9b640 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f7e8def050 0x55f7e8df1470 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ffad4ab795694b6 server_cookie=a0aaf123ead190c1 in_seq=0 out_seq=0 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.118+0000 7f6c69ffb640 10 client.15942 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.119+0000 7f6c69ffb640 1 -- 172.21.15.33:0/1805316794 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f6c780d7e50 con 0x55f7e8def050 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.119+0000 7f6c69ffb640 10 client.15942 handle_command_reply: tid=0 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.119+0000 7f6c8b866740 1 client.15942 shutdown 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.120+0000 7f6c8b866740 1 -- 172.21.15.33:0/1805316794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6e2c0 msgr2=0x55f7e8d7f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.120+0000 7f6c8b866740 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d7f740 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6c7401c040 tx=0x7f6c7400f520 comp rx=0 tx=0).stop 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.120+0000 7f6c8b866740 1 -- 172.21.15.33:0/1805316794 shutdown_connections 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.120+0000 7f6c8b866740 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f7e8def050 0x55f7e8df1470 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.120+0000 7f6c8b866740 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f7e8d87b80 0x55f7e8d7bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.120+0000 7f6c69ffb640 0 client.15942 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.120+0000 7f6c8b866740 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f7e8d777c0 0x55f7e8d7fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.120+0000 7f6c8b866740 1 --2- 172.21.15.33:0/1805316794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7e8d6e2c0 0x55f7e8d7f740 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.120+0000 7f6c8b866740 1 -- 172.21.15.33:0/1805316794 >> 172.21.15.33:0/1805316794 conn(0x55f7e8d689f0 msgr2=0x55f7e8d6a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.120+0000 7f6c8b866740 1 -- 172.21.15.33:0/1805316794 shutdown_connections 2024-04-02T13:40:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.120+0000 7f6c8b866740 1 -- 172.21.15.33:0/1805316794 wait complete. 2024-04-02T13:40:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.120+0000 7f6c8b866740 20 client.15942 trim_cache size 0 max 16384 2024-04-02T13:40:00.128 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:40:00.128 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:40:00.128 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "f3ba1026-9561-4542-8653-673cd6fb9f46", 2024-04-02T13:40:00.128 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:40:00.128 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:40:00.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.121+0000 7f6c88c7b640 1 -- 172.21.15.33:0/1015874161 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6c6c0a3f50 msgr2=0x7f6c6c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.121+0000 7f6c88c7b640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6c6c0a3f50 0x7f6c6c0a6410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c005fd0 tx=0x7f6c7c009600 comp rx=0 tx=0).stop 2024-04-02T13:40:00.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.121+0000 7f6c88c7b640 1 -- 172.21.15.33:0/1015874161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c841029c0 msgr2=0x7f6c8419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.121+0000 7f6c88c7b640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c841029c0 0x7f6c8419ab40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005940 tx=0x7f6c78004e90 comp rx=0 tx=0).stop 2024-04-02T13:40:00.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.121+0000 7f6c88c7b640 1 -- 172.21.15.33:0/1015874161 shutdown_connections 2024-04-02T13:40:00.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.121+0000 7f6c88c7b640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6c6c0a3f50 0x7f6c6c0a6410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.121+0000 7f6c88c7b640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c84102da0 0x7f6c8419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.121+0000 7f6c88c7b640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c841029c0 0x7f6c8419ab40 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.121+0000 7f6c88c7b640 1 --2- 172.21.15.33:0/1015874161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6c84100b00 0x7f6c8419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.121+0000 7f6c88c7b640 1 -- 172.21.15.33:0/1015874161 >> 172.21.15.33:0/1015874161 conn(0x7f6c84076f20 msgr2=0x7f6c840fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:00.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.121+0000 7f6c88c7b640 1 -- 172.21.15.33:0/1015874161 shutdown_connections 2024-04-02T13:40:00.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.121+0000 7f6c88c7b640 1 -- 172.21.15.33:0/1015874161 wait complete. 2024-04-02T13:40:00.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:40:00.135 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:40:00.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.244+0000 7fbd222ba640 1 -- 172.21.15.33:0/2835596540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd1c100210 msgr2=0x7fbd1c1005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.244+0000 7fbd222ba640 1 --2- 172.21.15.33:0/2835596540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd1c100210 0x7fbd1c1005f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009860 tx=0x7fbd0c02bb70 comp rx=0 tx=0).stop 2024-04-02T13:40:00.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.244+0000 7fbd222ba640 1 -- 172.21.15.33:0/2835596540 shutdown_connections 2024-04-02T13:40:00.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.244+0000 7fbd222ba640 1 --2- 172.21.15.33:0/2835596540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd1c106dd0 0x7fbd1c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.244+0000 7fbd222ba640 1 --2- 172.21.15.33:0/2835596540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c100bc0 0x7fbd1c1066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.244+0000 7fbd222ba640 1 --2- 172.21.15.33:0/2835596540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd1c100210 0x7fbd1c1005f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.244+0000 7fbd222ba640 1 -- 172.21.15.33:0/2835596540 >> 172.21.15.33:0/2835596540 conn(0x7fbd1c076f20 msgr2=0x7fbd1c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:00.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.244+0000 7fbd222ba640 1 -- 172.21.15.33:0/2835596540 shutdown_connections 2024-04-02T13:40:00.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.244+0000 7fbd222ba640 1 -- 172.21.15.33:0/2835596540 wait complete. 2024-04-02T13:40:00.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.245+0000 7fbd222ba640 1 Processor -- start 2024-04-02T13:40:00.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.245+0000 7fbd222ba640 1 -- start start 2024-04-02T13:40:00.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.245+0000 7fbd222ba640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd1c100210 0x7fbd1c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.245+0000 7fbd222ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c106dd0 0x7fbd1c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.245+0000 7fbd222ba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd1c19cc40 0x7fbd1c19d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.245+0000 7fbd222ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c111e00 con 0x7fbd1c106dd0 2024-04-02T13:40:00.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd222ba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c111c80 con 0x7fbd1c19cc40 2024-04-02T13:40:00.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd222ba640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c111f80 con 0x7fbd1c100210 2024-04-02T13:40:00.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c106dd0 0x7fbd1c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c106dd0 0x7fbd1c1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53014/0 (socket says 172.21.15.33:53014) 2024-04-02T13:40:00.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd1b7fe640 1 -- 172.21.15.33:0/4053894212 learned_addr learned my addr 172.21.15.33:0/4053894212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd20857640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd1c19cc40 0x7fbd1c19d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd1c100210 0x7fbd1c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd1b7fe640 1 -- 172.21.15.33:0/4053894212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd1c100210 msgr2=0x7fbd1c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd1b7fe640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd1c100210 0x7fbd1c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd1b7fe640 1 -- 172.21.15.33:0/4053894212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd1c19cc40 msgr2=0x7fbd1c19d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd1b7fe640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd1c19cc40 0x7fbd1c19d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd1b7fe640 1 -- 172.21.15.33:0/4053894212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c19d730 con 0x7fbd1c106dd0 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd1c100210 0x7fbd1c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd1b7fe640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c106dd0 0x7fbd1c1988c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000e9a0 tx=0x7fbd1000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd197fa640 1 -- 172.21.15.33:0/4053894212 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1000cd80 con 0x7fbd1c106dd0 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd222ba640 1 -- 172.21.15.33:0/4053894212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1c19da20 con 0x7fbd1c106dd0 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.246+0000 7fbd197fa640 1 -- 172.21.15.33:0/4053894212 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd100035f0 con 0x7fbd1c106dd0 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.247+0000 7fbd197fa640 1 -- 172.21.15.33:0/4053894212 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10010690 con 0x7fbd1c106dd0 2024-04-02T13:40:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.247+0000 7fbd222ba640 1 -- 172.21.15.33:0/4053894212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1c1a5200 con 0x7fbd1c106dd0 2024-04-02T13:40:00.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.250+0000 7fbd24f7d740 1 Processor -- start 2024-04-02T13:40:00.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.250+0000 7fbd24f7d740 1 -- start start 2024-04-02T13:40:00.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.250+0000 7fbd24f7d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f1096690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.250+0000 7fbd24f7d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f1096c60 0x5621f10a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.250+0000 7fbd24f7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f10a41e0 0x5621f10a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.250+0000 7fbd24f7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5621f0ffa830 con 0x5621f10a41e0 2024-04-02T13:40:00.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.250+0000 7fbd24f7d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5621f0fa2820 con 0x5621f10962b0 2024-04-02T13:40:00.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.250+0000 7fbd24f7d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5621f0ff67e0 con 0x5621f1096c60 2024-04-02T13:40:00.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.250+0000 7fbd197fa640 1 -- 172.21.15.33:0/4053894212 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fbd100037a0 con 0x7fbd1c106dd0 2024-04-02T13:40:00.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.250+0000 7fbd20857640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f1096690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.250+0000 7fbd1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f10a41e0 0x5621f10a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.250+0000 7fbd20857640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f1096690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:45088/0 (socket says 172.21.15.33:45088) 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.250+0000 7fbd20857640 1 -- 172.21.15.33:0/3246477731 learned_addr learned my addr 172.21.15.33:0/3246477731 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.251+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/3246477731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f1096c60 0x5621f10a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.251+0000 7fbd20857640 1 -- 172.21.15.33:0/3246477731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f1096c60 msgr2=0x5621f10a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.251+0000 7fbd20857640 1 --2- 172.21.15.33:0/3246477731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f1096c60 0x5621f10a3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.251+0000 7fbd20857640 1 -- 172.21.15.33:0/3246477731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f10a41e0 msgr2=0x5621f10a65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.251+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/3246477731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f1096c60 0x5621f10a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.251+0000 7fbd20857640 1 --2- 172.21.15.33:0/3246477731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f10a41e0 0x5621f10a65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.251+0000 7fbd20857640 1 -- 172.21.15.33:0/3246477731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621f0dd87d0 con 0x5621f10962b0 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.251+0000 7fbd197fa640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbd040a3f50 0x7fbd040a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.251+0000 7fbd1b7fe640 1 --2- 172.21.15.33:0/3246477731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f10a41e0 0x5621f10a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.251+0000 7fbd197fa640 1 -- 172.21.15.33:0/4053894212 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbd100d1180 con 0x7fbd1c106dd0 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.251+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbd040a3f50 0x7fbd040a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.252+0000 7fbd20857640 1 --2- 172.21.15.33:0/3246477731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f1096690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007ec0 tx=0x7fbd1400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2892471c94da347 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:00.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.252+0000 7fbd027fc640 1 -- 172.21.15.33:0/3246477731 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14010840 con 0x5621f10962b0 2024-04-02T13:40:00.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.252+0000 7fbd027fc640 1 -- 172.21.15.33:0/3246477731 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd140183f0 con 0x5621f10962b0 2024-04-02T13:40:00.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.252+0000 7fbd027fc640 1 -- 172.21.15.33:0/3246477731 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14017600 con 0x5621f10962b0 2024-04-02T13:40:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.253+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbd040a3f50 0x7fbd040a6410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c000c00 tx=0x7fbd0c005da0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.253+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3246477731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 msgr2=0x5621f1096690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.253+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/3246477731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f1096690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007ec0 tx=0x7fbd1400f790 comp rx=0 tx=0).stop 2024-04-02T13:40:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.253+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3246477731 shutdown_connections 2024-04-02T13:40:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.253+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/3246477731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f10a41e0 0x5621f10a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.253+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/3246477731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f1096c60 0x5621f10a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.253+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/3246477731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f1096690 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.253+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3246477731 >> 172.21.15.33:0/3246477731 conn(0x5621f1091000 msgr2=0x5621f10913e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.253+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3246477731 shutdown_connections 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.253+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3246477731 wait complete. 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.253+0000 7fbd24f7d740 1 -- start start 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd24f7d740 1 Processor -- start 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd24f7d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f10962b0 0x5621f10a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd24f7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 0x5621f10a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 0x5621f10a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd1bfff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f10962b0 0x5621f10a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd1bfff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f10962b0 0x5621f10a3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:54702/0 (socket says 172.21.15.33:54702) 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 0x5621f10a0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53034/0 (socket says 172.21.15.33:53034) 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd1bfff640 1 -- 172.21.15.33:0/3429908450 learned_addr learned my addr 172.21.15.33:0/3429908450 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10a41e0 0x5621f10a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3429908450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5621f10a6fa0 con 0x5621f1096c60 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3429908450 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5621f0fb5d30 con 0x5621f10a41e0 2024-04-02T13:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3429908450 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5621f10a7120 con 0x5621f10962b0 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd20857640 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10a41e0 0x5621f10a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.254+0000 7fbd1bfff640 1 -- 172.21.15.33:0/3429908450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10a41e0 msgr2=0x5621f10a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10a41e0 0x5621f10a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbd1bfff640 1 -- 172.21.15.33:0/3429908450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 msgr2=0x5621f10a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 0x5621f10a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbd1bfff640 1 -- 172.21.15.33:0/3429908450 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621f0d0e3f0 con 0x5621f10962b0 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbd1b7fe640 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 0x5621f10a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbd24f7d740 10 client.0 _ll_register_callbacks cb 0x5621f0fb5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3429908450 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd140183f0 con 0x5621f10962b0 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f10962b0 0x5621f10a3b10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c036000 tx=0x7fbd0c02bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2a521c6a4071b50 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbcfb7fe640 10 client.25429 ms_handle_connect on v2:172.21.15.145:3300/0 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbcfb7fe640 1 -- 172.21.15.33:0/3429908450 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c039040 con 0x5621f10962b0 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbcfb7fe640 1 -- 172.21.15.33:0/3429908450 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbd0c003af0 con 0x5621f10962b0 2024-04-02T13:40:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbcfb7fe640 1 -- 172.21.15.33:0/3429908450 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd0c003cf0 con 0x5621f10962b0 2024-04-02T13:40:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbd24f7d740 10 client.25429 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.255+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3429908450 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5621f0cce0f0 con 0x5621f10962b0 2024-04-02T13:40:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.256+0000 7fbcfb7fe640 1 -- 172.21.15.33:0/3429908450 <== mon.1 v2:172.21.15.145:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbd0c053030 con 0x5621f10962b0 2024-04-02T13:40:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.256+0000 7fbd24f7d740 10 client.25429 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.256+0000 7fbd24f7d740 10 client.25429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.256+0000 7fbd24f7d740 20 client.25429 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.256+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5621f10b3fb0 0x5621f10b6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.256+0000 7fbd24f7d740 4 client.25429 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.256+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3429908450 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621f1007d70 con 0x5621f10b3fb0 2024-04-02T13:40:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.256+0000 7fbd1b7fe640 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5621f10b3fb0 0x5621f10b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.256+0000 7fbd1b7fe640 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5621f10b3fb0 0x5621f10b6400 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f50852c496c647c7 server_cookie=b79036af5bae66b6 in_seq=0 out_seq=0 2024-04-02T13:40:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.257+0000 7fbcfb7fe640 10 client.25429 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.257+0000 7fbcfb7fe640 1 -- 172.21.15.33:0/3429908450 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621f1007d70 con 0x5621f10b3fb0 2024-04-02T13:40:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.257+0000 7fbcfb7fe640 10 client.25429 handle_command_reply: tid=0 2024-04-02T13:40:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.257+0000 7fbd24f7d740 1 client.25429 shutdown 2024-04-02T13:40:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.257+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3429908450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f10962b0 msgr2=0x5621f10a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.257+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f10962b0 0x5621f10a3b10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c036000 tx=0x7fbd0c02bb70 comp rx=0 tx=0).stop 2024-04-02T13:40:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.257+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3429908450 shutdown_connections 2024-04-02T13:40:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.257+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5621f10b3fb0 0x5621f10b6400 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.257+0000 7fbcfb7fe640 0 client.25429 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.257+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10a41e0 0x5621f10a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.257+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 0x5621f10a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.257+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/3429908450 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f10962b0 0x5621f10a3b10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.257+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3429908450 >> 172.21.15.33:0/3429908450 conn(0x5621f1091000 msgr2=0x5621f1092d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:00.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.258+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3429908450 shutdown_connections 2024-04-02T13:40:00.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.258+0000 7fbd24f7d740 1 -- 172.21.15.33:0/3429908450 wait complete. 2024-04-02T13:40:00.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.258+0000 7fbd24f7d740 20 client.25429 trim_cache size 0 max 16384 2024-04-02T13:40:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.277+0000 7fbd24f7d740 1 Processor -- start 2024-04-02T13:40:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.277+0000 7fbd24f7d740 1 -- start start 2024-04-02T13:40:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.277+0000 7fbd24f7d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f1090240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.277+0000 7fbd24f7d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f1096c60 0x5621f1092ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.277+0000 7fbd24f7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f10a41e0 0x5621f10a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.277+0000 7fbd24f7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1af5c0 con 0x5621f10a41e0 2024-04-02T13:40:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.277+0000 7fbd24f7d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1ace90 con 0x5621f10962b0 2024-04-02T13:40:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.277+0000 7fbd24f7d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1aacc0 con 0x5621f1096c60 2024-04-02T13:40:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.277+0000 7fbd20857640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f10a41e0 0x5621f10a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.277+0000 7fbd20857640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f10a41e0 0x5621f10a35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53048/0 (socket says 172.21.15.33:53048) 2024-04-02T13:40:00.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.277+0000 7fbd20857640 1 -- 172.21.15.33:0/2094663580 learned_addr learned my addr 172.21.15.33:0/2094663580 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:00.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.277+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/2094663580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f1090240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.278+0000 7fbd1b7fe640 1 --2- 172.21.15.33:0/2094663580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f1096c60 0x5621f1092ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.278+0000 7fbd20857640 1 -- 172.21.15.33:0/2094663580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f1096c60 msgr2=0x5621f1092ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.278+0000 7fbd20857640 1 --2- 172.21.15.33:0/2094663580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f1096c60 0x5621f1092ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.278+0000 7fbd20857640 1 -- 172.21.15.33:0/2094663580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 msgr2=0x5621f1090240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.278+0000 7fbd20857640 1 --2- 172.21.15.33:0/2094663580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f1090240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.278+0000 7fbd20857640 1 -- 172.21.15.33:0/2094663580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621f108cc50 con 0x5621f10a41e0 2024-04-02T13:40:00.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.278+0000 7fbd20857640 1 --2- 172.21.15.33:0/2094663580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f10a41e0 0x5621f10a35d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007de0 tx=0x7fbd140072e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0c87bb3916acc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:00.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.278+0000 7fbd01ffb640 1 -- 172.21.15.33:0/2094663580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14027020 con 0x5621f10a41e0 2024-04-02T13:40:00.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.278+0000 7fbd01ffb640 1 -- 172.21.15.33:0/2094663580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd14003e40 con 0x5621f10a41e0 2024-04-02T13:40:00.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.278+0000 7fbd01ffb640 1 -- 172.21.15.33:0/2094663580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14018800 con 0x5621f10a41e0 2024-04-02T13:40:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.279+0000 7fbd24f7d740 1 -- 172.21.15.33:0/2094663580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f10a41e0 msgr2=0x5621f10a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.279+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/2094663580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f10a41e0 0x5621f10a35d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007de0 tx=0x7fbd140072e0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.279+0000 7fbd24f7d740 1 -- 172.21.15.33:0/2094663580 shutdown_connections 2024-04-02T13:40:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.279+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/2094663580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f10a41e0 0x5621f10a35d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.279+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/2094663580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f1096c60 0x5621f1092ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.279+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/2094663580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f1090240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.279+0000 7fbd24f7d740 1 -- 172.21.15.33:0/2094663580 >> 172.21.15.33:0/2094663580 conn(0x5621f1091000 msgr2=0x5621f10923c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.279+0000 7fbd24f7d740 1 -- 172.21.15.33:0/2094663580 shutdown_connections 2024-04-02T13:40:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.279+0000 7fbd24f7d740 1 -- 172.21.15.33:0/2094663580 wait complete. 2024-04-02T13:40:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.279+0000 7fbd24f7d740 1 -- start start 2024-04-02T13:40:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.280+0000 7fbd24f7d740 1 Processor -- start 2024-04-02T13:40:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.280+0000 7fbd24f7d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f10a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.280+0000 7fbd24f7d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 0x5621f10a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.280+0000 7fbd24f7d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f10a41e0 0x5621f10a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.280+0000 7fbd24f7d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1ab350 con 0x5621f1096c60 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.280+0000 7fbd24f7d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1ac7c0 con 0x5621f10962b0 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.280+0000 7fbd24f7d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1aba20 con 0x5621f10a41e0 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.280+0000 7fbd1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 0x5621f10a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.280+0000 7fbd24f7d740 10 client.0 _ll_register_callbacks cb 0x7fbd0c0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.280+0000 7fbd1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 0x5621f10a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53054/0 (socket says 172.21.15.33:53054) 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.280+0000 7fbd1b7fe640 1 -- 172.21.15.33:0/1103575352 learned_addr learned my addr 172.21.15.33:0/1103575352 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.280+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f10a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.280+0000 7fbd20857640 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f10a41e0 0x5621f10a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbd1b7fe640 1 -- 172.21.15.33:0/1103575352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f10a41e0 msgr2=0x5621f10a8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbd1b7fe640 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f10a41e0 0x5621f10a8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbd1b7fe640 1 -- 172.21.15.33:0/1103575352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 msgr2=0x5621f10a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbd1b7fe640 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f10a8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbd1b7fe640 1 -- 172.21.15.33:0/1103575352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621f1119790 con 0x5621f1096c60 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbd20857640 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f10a41e0 0x5621f10a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f10a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbd1b7fe640 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 0x5621f10a8540 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000c6a0 tx=0x7fbd100d4c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8363a31cf7474fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbcfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbcfb7fe640 1 -- 172.21.15.33:0/1103575352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd100b5030 con 0x5621f1096c60 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbcfb7fe640 1 -- 172.21.15.33:0/1103575352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbd100bd0b0 con 0x5621f1096c60 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbd24f7d740 1 -- 172.21.15.33:0/1103575352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5621f10020c0 con 0x5621f1096c60 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbcfb7fe640 1 -- 172.21.15.33:0/1103575352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd1009c6b0 con 0x5621f1096c60 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbd24f7d740 10 client.15960 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbd24f7d740 1 -- 172.21.15.33:0/1103575352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5621f111aac0 con 0x5621f1096c60 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.281+0000 7fbcfb7fe640 1 -- 172.21.15.33:0/1103575352 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd10003b80 con 0x5621f1096c60 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.282+0000 7fbcfb7fe640 1 -- 172.21.15.33:0/1103575352 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbd100b7020 con 0x5621f1096c60 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.282+0000 7fbd24f7d740 10 client.15960 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.282+0000 7fbd24f7d740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:00.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.282+0000 7fbd24f7d740 20 client.15960 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.282+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5621f10b3fb0 0x5621f111d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.282+0000 7fbd24f7d740 4 client.15960 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.282+0000 7fbd24f7d740 1 -- 172.21.15.33:0/1103575352 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5621f10020c0 con 0x5621f10b3fb0 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.282+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5621f10b3fb0 0x5621f111d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.282+0000 7fbcfb7fe640 10 client.15960 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.282+0000 7fbd1bfff640 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5621f10b3fb0 0x5621f111d1a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25fddfa7e4d6072f server_cookie=3303093ccc9b9a42 in_seq=0 out_seq=0 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.282+0000 7fbcfb7fe640 1 -- 172.21.15.33:0/1103575352 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5621f10020c0 con 0x5621f10b3fb0 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.282+0000 7fbcfb7fe640 10 client.15960 handle_command_reply: tid=0 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd24f7d740 1 client.15960 shutdown 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd24f7d740 1 -- 172.21.15.33:0/1103575352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 msgr2=0x5621f10a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 0x5621f10a8540 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000c6a0 tx=0x7fbd100d4c10 comp rx=0 tx=0).stop 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd24f7d740 1 -- 172.21.15.33:0/1103575352 shutdown_connections 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5621f10b3fb0 0x5621f111d1a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbcfb7fe640 0 client.15960 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5621f10a41e0 0x5621f10a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621f1096c60 0x5621f10a8540 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd24f7d740 1 --2- 172.21.15.33:0/1103575352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621f10962b0 0x5621f10a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd24f7d740 1 -- 172.21.15.33:0/1103575352 >> 172.21.15.33:0/1103575352 conn(0x5621f1091000 msgr2=0x5621f1092c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:00.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd24f7d740 1 -- 172.21.15.33:0/1103575352 shutdown_connections 2024-04-02T13:40:00.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd24f7d740 1 -- 172.21.15.33:0/1103575352 wait complete. 2024-04-02T13:40:00.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd24f7d740 20 client.15960 trim_cache size 0 max 16384 2024-04-02T13:40:00.291 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:40:00.291 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:40:00.291 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:40:00.291 INFO:teuthology.orchestra.run.smithi033.stdout: "f3ba1026-9561-4542-8653-673cd6fb9f46": { 2024-04-02T13:40:00.291 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:40:00.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "f3ba1026-9561-4542-8653-673cd6fb9f46", 2024-04-02T13:40:00.292 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:40:00.292 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:40:00.292 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:40:00.292 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:40:00.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd222ba640 1 -- 172.21.15.33:0/4053894212 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbd040a3f50 msgr2=0x7fbd040a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.283+0000 7fbd222ba640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbd040a3f50 0x7fbd040a6410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c000c00 tx=0x7fbd0c005da0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.284+0000 7fbd222ba640 1 -- 172.21.15.33:0/4053894212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c106dd0 msgr2=0x7fbd1c1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.284+0000 7fbd222ba640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c106dd0 0x7fbd1c1988c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000e9a0 tx=0x7fbd1000ee60 comp rx=0 tx=0).stop 2024-04-02T13:40:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.284+0000 7fbd222ba640 1 -- 172.21.15.33:0/4053894212 shutdown_connections 2024-04-02T13:40:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.284+0000 7fbd222ba640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbd040a3f50 0x7fbd040a6410 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.284+0000 7fbd222ba640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd1c19cc40 0x7fbd1c19d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.284+0000 7fbd222ba640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c106dd0 0x7fbd1c1988c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.284+0000 7fbd222ba640 1 --2- 172.21.15.33:0/4053894212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd1c100210 0x7fbd1c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.284+0000 7fbd222ba640 1 -- 172.21.15.33:0/4053894212 >> 172.21.15.33:0/4053894212 conn(0x7fbd1c076f20 msgr2=0x7fbd1c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.284+0000 7fbd222ba640 1 -- 172.21.15.33:0/4053894212 shutdown_connections 2024-04-02T13:40:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:00.284+0000 7fbd222ba640 1 -- 172.21.15.33:0/4053894212 wait complete. 2024-04-02T13:40:00.298 INFO:tasks.cephfs.filesystem:scrub status for tag:f3ba1026-9561-4542-8653-673cd6fb9f46 - {'path': '/', 'tag': 'f3ba1026-9561-4542-8653-673cd6fb9f46', 'options': 'recursive,force'} 2024-04-02T13:40:00.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:00 smithi050 ceph-mon[35524]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:40:00.646 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:00 smithi050 ceph-mon[35524]: overall HEALTH_OK 2024-04-02T13:40:00.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:00 smithi145 ceph-mon[35646]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:40:00.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:00 smithi145 ceph-mon[35646]: overall HEALTH_OK 2024-04-02T13:40:00.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:00 smithi033 ceph-mon[32390]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:40:00.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:00 smithi033 ceph-mon[32390]: overall HEALTH_OK 2024-04-02T13:40:01.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:01 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:40:01.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:01 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:01.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:01 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:40:01.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:01 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:01.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:01 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:40:01.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:01 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:40:01.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:01 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:01.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:01 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:40:01.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:01 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:01.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:01 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:40:01.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:01 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:40:01.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:01 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:01.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:01 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:40:01.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:01 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:01.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:01 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:40:02.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:02 smithi145 ceph-mon[35646]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:40:02.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:02 smithi050 ceph-mon[35524]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:40:02.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:02 smithi033 ceph-mon[32390]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-02T13:40:04.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:04 smithi145 ceph-mon[35646]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-02T13:40:04.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:04 smithi050 ceph-mon[35524]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-02T13:40:04.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:04 smithi033 ceph-mon[32390]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-02T13:40:06.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:06 smithi145 ceph-mon[35646]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:40:06.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:06 smithi050 ceph-mon[35524]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:40:06.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:06 smithi033 ceph-mon[32390]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:40:08.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:08 smithi145 ceph-mon[35646]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:40:08.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:08 smithi050 ceph-mon[35524]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:40:08.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:08 smithi033 ceph-mon[32390]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:40:10.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:10 smithi145 ceph-mon[35646]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-02T13:40:10.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:10 smithi050 ceph-mon[35524]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-02T13:40:10.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:10 smithi033 ceph-mon[32390]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-02T13:40:12.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:12 smithi145 ceph-mon[35646]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:40:12.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:12 smithi050 ceph-mon[35524]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:40:12.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:12 smithi033 ceph-mon[32390]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:40:14.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:14 smithi145 ceph-mon[35646]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-02T13:40:14.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:14 smithi050 ceph-mon[35524]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-02T13:40:14.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:14 smithi033 ceph-mon[32390]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-02T13:40:15.624 INFO:tasks.workunit.client.0.smithi033.stderr:Updating files: 92% (11354/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-02T13:40:16.316 INFO:tasks.workunit.client.0.smithi033.stderr:Note: switching to '191d963ac9de7322da8aff20b567b3fa458e2ef6'. 2024-04-02T13:40:16.316 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-02T13:40:16.316 INFO:tasks.workunit.client.0.smithi033.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-02T13:40:16.316 INFO:tasks.workunit.client.0.smithi033.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-02T13:40:16.317 INFO:tasks.workunit.client.0.smithi033.stderr:state without impacting any branches by switching back to a branch. 2024-04-02T13:40:16.317 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-02T13:40:16.317 INFO:tasks.workunit.client.0.smithi033.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-02T13:40:16.317 INFO:tasks.workunit.client.0.smithi033.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-02T13:40:16.317 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-02T13:40:16.317 INFO:tasks.workunit.client.0.smithi033.stderr: git switch -c 2024-04-02T13:40:16.317 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-02T13:40:16.317 INFO:tasks.workunit.client.0.smithi033.stderr:Or undo this operation with: 2024-04-02T13:40:16.317 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-02T13:40:16.317 INFO:tasks.workunit.client.0.smithi033.stderr: git switch - 2024-04-02T13:40:16.317 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-02T13:40:16.318 INFO:tasks.workunit.client.0.smithi033.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-02T13:40:16.318 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-02T13:40:16.318 INFO:tasks.workunit.client.0.smithi033.stderr:HEAD is now at 191d963ac9d Revert "Merge pull request #55436 from tchaikov/mgr-python-3.12" 2024-04-02T13:40:16.322 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-02T13:40:16.349 INFO:tasks.workunit.client.0.smithi033.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-02T13:40:16.351 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T13:40:16.351 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-02T13:40:16.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:16 smithi145 ceph-mon[35646]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T13:40:16.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:16 smithi050 ceph-mon[35524]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T13:40:16.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:16 smithi033 ceph-mon[32390]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T13:40:17.090 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-02T13:40:17.157 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-02T13:40:17.197 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T13:40:17.198 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T13:40:17.198 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-02T13:40:17.237 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T13:40:17.240 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:40:17.240 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-02T13:40:17.295 INFO:tasks.workunit:Running workunits matching fs/norstats on client.0... 2024-04-02T13:40:17.297 INFO:tasks.workunit:Running workunit fs/norstats/kernel_untar_tar.sh... 2024-04-02T13:40:17.297 DEBUG:teuthology.orchestra.run.smithi033:workunit test fs/norstats/kernel_untar_tar.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=191d963ac9de7322da8aff20b567b3fa458e2ef6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh 2024-04-02T13:40:17.359 INFO:tasks.workunit.client.0.smithi033.stderr:+ KERNEL=linux-4.0.5 2024-04-02T13:40:17.360 INFO:tasks.workunit.client.0.smithi033.stderr:+ wget -q http://download.ceph.com/qa/linux-4.0.5.tar.xz 2024-04-02T13:40:18.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:18 smithi145 ceph-mon[35646]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T13:40:18.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:18 smithi050 ceph-mon[35524]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T13:40:18.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:18 smithi033 ceph-mon[32390]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-02T13:40:20.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:20 smithi145 ceph-mon[35646]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-02T13:40:20.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:20 smithi145 ceph-mon[35646]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-02T13:40:20.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:20 smithi050 ceph-mon[35524]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-02T13:40:20.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:20 smithi050 ceph-mon[35524]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-02T13:40:20.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:20 smithi033 ceph-mon[32390]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-02T13:40:20.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:20 smithi033 ceph-mon[32390]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-02T13:40:20.881 INFO:tasks.workunit.client.0.smithi033.stderr:+ mkdir untar_tar 2024-04-02T13:40:20.884 INFO:tasks.workunit.client.0.smithi033.stderr:+ cd untar_tar 2024-04-02T13:40:20.885 INFO:tasks.workunit.client.0.smithi033.stderr:+ tar Jxvf ../linux-4.0.5.tar.xz linux-4.0.5/Documentation/ 2024-04-02T13:40:20.917 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ 2024-04-02T13:40:20.920 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/00-INDEX 2024-04-02T13:40:20.922 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/ 2024-04-02T13:40:20.924 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/README 2024-04-02T13:40:20.926 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/obsolete/ 2024-04-02T13:40:20.928 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/obsolete/proc-sys-vm-nr_pdflush_threads 2024-04-02T13:40:20.930 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-bus-usb 2024-04-02T13:40:20.933 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-class-rfkill 2024-04-02T13:40:20.936 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-koneplus 2024-04-02T13:40:20.938 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-kovaplus 2024-04-02T13:40:20.941 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra 2024-04-02T13:40:20.943 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/removed/ 2024-04-02T13:40:20.945 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/removed/devfs 2024-04-02T13:40:20.947 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/removed/dv1394 2024-04-02T13:40:20.950 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/removed/ip_queue 2024-04-02T13:40:20.952 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/removed/net_dma 2024-04-02T13:40:20.954 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/removed/o2cb 2024-04-02T13:40:20.957 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/removed/raw1394 2024-04-02T13:40:20.959 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/removed/video1394 2024-04-02T13:40:20.961 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/ 2024-04-02T13:40:20.964 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/firewire-cdev 2024-04-02T13:40:20.966 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/o2cb 2024-04-02T13:40:20.968 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/syscalls 2024-04-02T13:40:20.970 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-acpi-pmprofile 2024-04-02T13:40:20.973 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-firewire 2024-04-02T13:40:20.975 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-usb 2024-04-02T13:40:20.977 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-xen-backend 2024-04-02T13:40:20.982 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-backlight 2024-04-02T13:40:20.982 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-rfkill 2024-04-02T13:40:20.984 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-tpm 2024-04-02T13:40:20.987 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-ubi 2024-04-02T13:40:20.989 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-udc 2024-04-02T13:40:20.992 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-node 2024-04-02T13:40:20.994 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-cpu 2024-04-02T13:40:20.996 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-xen_memory 2024-04-02T13:40:20.999 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-ib_srp 2024-04-02T13:40:21.002 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-qla2xxx 2024-04-02T13:40:21.004 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-usb-usbtmc 2024-04-02T13:40:21.007 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-w1_ds28e04 2024-04-02T13:40:21.009 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-efi-vars 2024-04-02T13:40:21.011 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-dump 2024-04-02T13:40:21.014 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-elog 2024-04-02T13:40:21.016 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-module 2024-04-02T13:40:21.019 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-transport-srp 2024-04-02T13:40:21.021 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/thermal-notification 2024-04-02T13:40:21.023 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/stable/vdso 2024-04-02T13:40:21.025 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/ 2024-04-02T13:40:21.027 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-spear-pcie-gadget 2024-04-02T13:40:21.029 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget 2024-04-02T13:40:21.032 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-acm 2024-04-02T13:40:21.034 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ecm 2024-04-02T13:40:21.037 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-eem 2024-04-02T13:40:21.039 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ffs 2024-04-02T13:40:21.041 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-hid 2024-04-02T13:40:21.043 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-loopback 2024-04-02T13:40:21.045 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-mass-storage 2024-04-02T13:40:21.050 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-midi 2024-04-02T13:40:21.050 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ncm 2024-04-02T13:40:21.052 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-obex 2024-04-02T13:40:21.054 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-phonet 2024-04-02T13:40:21.056 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-rndis 2024-04-02T13:40:21.059 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-serial 2024-04-02T13:40:21.061 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-sourcesink 2024-04-02T13:40:21.063 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-subset 2024-04-02T13:40:21.065 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac1 2024-04-02T13:40:21.068 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac2 2024-04-02T13:40:21.070 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uvc 2024-04-02T13:40:21.072 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-driver-genwqe 2024-04-02T13:40:21.075 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ec 2024-04-02T13:40:21.077 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ideapad 2024-04-02T13:40:21.079 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-olpc 2024-04-02T13:40:21.082 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pfo-nx-crypto 2024-04-02T13:40:21.083 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pktcdvd 2024-04-02T13:40:21.086 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/dev-kmsg 2024-04-02T13:40:21.088 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/evm 2024-04-02T13:40:21.090 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/ima_policy 2024-04-02T13:40:21.092 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/procfs-diskstats 2024-04-02T13:40:21.095 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/pstore 2024-04-02T13:40:21.097 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ata 2024-04-02T13:40:21.099 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block 2024-04-02T13:40:21.102 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-bcache 2024-04-02T13:40:21.105 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-dm 2024-04-02T13:40:21.107 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-rssd 2024-04-02T13:40:21.109 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-zram 2024-04-02T13:40:21.112 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-acpi 2024-04-02T13:40:21.114 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-amba 2024-04-02T13:40:21.117 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-bcma 2024-04-02T13:40:21.119 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10 2024-04-02T13:40:21.122 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm3x 2024-04-02T13:40:21.124 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-funnel 2024-04-02T13:40:21.126 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc 2024-04-02T13:40:21.129 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-css 2024-04-02T13:40:21.131 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-events 2024-04-02T13:40:21.133 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-format 2024-04-02T13:40:21.136 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7 2024-04-02T13:40:21.138 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_gpci 2024-04-02T13:40:21.141 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-fcoe 2024-04-02T13:40:21.143 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-hsi 2024-04-02T13:40:21.145 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-fsa9480 2024-04-02T13:40:21.148 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 2024-04-02T13:40:21.150 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-lm3533 2024-04-02T13:40:21.152 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio 2024-04-02T13:40:21.156 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-accel-bmc150 2024-04-02T13:40:21.159 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-ad9523 2024-04-02T13:40:21.161 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-adf4350 2024-04-02T13:40:21.163 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-gyro-bmg160 2024-04-02T13:40:21.165 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-light-lm3533-als 2024-04-02T13:40:21.168 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-mpu6050 2024-04-02T13:40:21.170 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 2024-04-02T13:40:21.172 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-trigger-sysfs 2024-04-02T13:40:21.175 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mdio 2024-04-02T13:40:21.177 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-media 2024-04-02T13:40:21.179 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mei 2024-04-02T13:40:21.181 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci 2024-04-02T13:40:21.186 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss 2024-04-02T13:40:21.187 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-drivers-ehci_hcd 2024-04-02T13:40:21.189 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-platform 2024-04-02T13:40:21.191 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rbd 2024-04-02T13:40:21.193 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rpmsg 2024-04-02T13:40:21.196 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-umc 2024-04-02T13:40:21.198 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb 2024-04-02T13:40:21.200 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg 2024-04-02T13:40:21.202 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-lvstest 2024-04-02T13:40:21.204 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-c2port 2024-04-02T13:40:21.207 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-cfq-target-latency 2024-04-02T13:40:21.209 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class 2024-04-02T13:40:21.211 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-adp8870 2024-04-02T13:40:21.214 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533 2024-04-02T13:40:21.216 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-bdi 2024-04-02T13:40:21.218 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-cxl 2024-04-02T13:40:21.221 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-devfreq 2024-04-02T13:40:21.223 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-extcon 2024-04-02T13:40:21.225 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu 2024-04-02T13:40:21.227 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-amd-iommu 2024-04-02T13:40:21.230 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-intel-iommu 2024-04-02T13:40:21.232 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-lcd 2024-04-02T13:40:21.234 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led 2024-04-02T13:40:21.237 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led-driver-lm3533 2024-04-02T13:40:21.239 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-leds-gt683r 2024-04-02T13:40:21.241 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mei 2024-04-02T13:40:21.244 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mic.txt 2024-04-02T13:40:21.246 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mtd 2024-04-02T13:40:21.249 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net 2024-04-02T13:40:21.251 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-batman-adv 2024-04-02T13:40:21.254 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-cdc_ncm 2024-04-02T13:40:21.256 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-grcan 2024-04-02T13:40:21.258 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-mesh 2024-04-02T13:40:21.260 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-queues 2024-04-02T13:40:21.263 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-statistics 2024-04-02T13:40:21.265 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pktcdvd 2024-04-02T13:40:21.267 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-power 2024-04-02T13:40:21.270 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-powercap 2024-04-02T13:40:21.272 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pwm 2024-04-02T13:40:21.274 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rc 2024-04-02T13:40:21.276 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-regulator 2024-04-02T13:40:21.279 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rtc-rtc0-device-rtc_calibration 2024-04-02T13:40:21.281 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-scsi_host 2024-04-02T13:40:21.284 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc 2024-04-02T13:40:21.286 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc-wusbhc 2024-04-02T13:40:21.288 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-dev 2024-04-02T13:40:21.290 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices 2024-04-02T13:40:21.293 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-edac 2024-04-02T13:40:21.295 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-firmware_node 2024-04-02T13:40:21.297 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-lpss_ltr 2024-04-02T13:40:21.299 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-memory 2024-04-02T13:40:21.302 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-mmc 2024-04-02T13:40:21.304 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-online 2024-04-02T13:40:21.306 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-_UDC_-gadget 2024-04-02T13:40:21.308 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-docg3 2024-04-02T13:40:21.311 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-sh_mobile_lcdc_fb 2024-04-02T13:40:21.314 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power 2024-04-02T13:40:21.316 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D0 2024-04-02T13:40:21.318 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D1 2024-04-02T13:40:21.320 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D2 2024-04-02T13:40:21.323 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D3hot 2024-04-02T13:40:21.325 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_wakeup 2024-04-02T13:40:21.327 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_state 2024-04-02T13:40:21.329 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-real_power_state 2024-04-02T13:40:21.332 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-resource_in_use 2024-04-02T13:40:21.334 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-soc 2024-04-02T13:40:21.337 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-sun 2024-04-02T13:40:21.339 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-cpu 2024-04-02T13:40:21.341 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-ibm-rtl 2024-04-02T13:40:21.343 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-xen_cpu 2024-04-02T13:40:21.346 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-genwqe 2024-04-02T13:40:21.348 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid 2024-04-02T13:40:21.350 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-lenovo 2024-04-02T13:40:21.352 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-logitech-lg4ff 2024-04-02T13:40:21.355 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-multitouch 2024-04-02T13:40:21.357 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-picolcd 2024-04-02T13:40:21.359 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-prodikeys 2024-04-02T13:40:21.361 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-arvo 2024-04-02T13:40:21.364 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-isku 2024-04-02T13:40:21.366 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone 2024-04-02T13:40:21.368 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-koneplus 2024-04-02T13:40:21.371 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-konepure 2024-04-02T13:40:21.373 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kovaplus 2024-04-02T13:40:21.375 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-lua 2024-04-02T13:40:21.378 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-pyra 2024-04-02T13:40:22.784 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:22 smithi033 ceph-mon[32390]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-04-02T13:40:22.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:22 smithi145 ceph-mon[35646]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-04-02T13:40:22.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:22 smithi050 ceph-mon[35524]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-04-02T13:40:24.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:24 smithi145 ceph-mon[35646]: pgmap v51: 129 pgs: 129 active+clean; 80 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 MiB/s wr, 28 op/s 2024-04-02T13:40:24.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:24 smithi050 ceph-mon[35524]: pgmap v51: 129 pgs: 129 active+clean; 80 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 MiB/s wr, 28 op/s 2024-04-02T13:40:25.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:24 smithi033 ceph-mon[32390]: pgmap v51: 129 pgs: 129 active+clean; 80 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 MiB/s wr, 28 op/s 2024-04-02T13:40:25.788 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-ryos 2024-04-02T13:40:25.791 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-savu 2024-04-02T13:40:25.794 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-srws1 2024-04-02T13:40:25.797 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-wiimote 2024-04-02T13:40:25.799 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-input-axp-pek 2024-04-02T13:40:25.801 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-intel-rapid-start 2024-04-02T13:40:25.804 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-pciback 2024-04-02T13:40:25.807 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-ppi 2024-04-02T13:40:25.809 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-samsung-laptop 2024-04-02T13:40:25.812 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-sunxi-sid 2024-04-02T13:40:25.815 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-tegra-fuse 2024-04-02T13:40:25.817 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-toshiba_acpi 2024-04-02T13:40:25.820 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-wacom 2024-04-02T13:40:25.822 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkback 2024-04-02T13:40:25.824 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkfront 2024-04-02T13:40:25.827 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-acpi 2024-04-02T13:40:25.830 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-dmi 2024-04-02T13:40:25.832 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi 2024-04-02T13:40:25.835 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi-runtime-map 2024-04-02T13:40:25.837 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-gsmi 2024-04-02T13:40:25.839 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-log 2024-04-02T13:40:25.842 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-memmap 2024-04-02T13:40:25.844 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-ofw 2024-04-02T13:40:25.846 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sfi 2024-04-02T13:40:25.849 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sgi_uv 2024-04-02T13:40:25.851 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-ext4 2024-04-02T13:40:25.854 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-f2fs 2024-04-02T13:40:25.856 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-nilfs2 2024-04-02T13:40:25.859 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-xfs 2024-04-02T13:40:25.861 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-gpio 2024-04-02T13:40:25.863 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-i2c-bmp085 2024-04-02T13:40:25.866 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ibft 2024-04-02T13:40:25.869 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-boot_params 2024-04-02T13:40:25.871 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-fscaps 2024-04-02T13:40:25.874 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-iommu_groups 2024-04-02T13:40:25.879 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-livepatch 2024-04-02T13:40:25.879 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm 2024-04-02T13:40:25.880 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-hugepages 2024-04-02T13:40:25.883 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-ksm 2024-04-02T13:40:25.885 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-slab 2024-04-02T13:40:25.888 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-uids 2024-04-02T13:40:25.890 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo 2024-04-02T13:40:25.892 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-memory-page-offline 2024-04-02T13:40:25.895 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-module 2024-04-02T13:40:25.898 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ocfs2 2024-04-02T13:40:25.900 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-laptop 2024-04-02T13:40:25.903 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-wmi 2024-04-02T13:40:25.905 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-at91 2024-04-02T13:40:25.908 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-brcmstb-gisb-arb 2024-04-02T13:40:25.911 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-chipidea-usb-otg 2024-04-02T13:40:25.913 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-eeepc-laptop 2024-04-02T13:40:25.915 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ideapad-laptop 2024-04-02T13:40:25.917 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-kim 2024-04-02T13:40:25.920 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-msi-laptop 2024-04-02T13:40:25.922 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-tahvo-usb 2024-04-02T13:40:25.925 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ts5500 2024-04-02T13:40:25.927 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-power 2024-04-02T13:40:25.929 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-pps 2024-04-02T13:40:25.932 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-profiling 2024-04-02T13:40:25.934 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ptp 2024-04-02T13:40:25.936 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-tty 2024-04-02T13:40:25.939 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-wusb_cbaf 2024-04-02T13:40:25.941 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/BUG-HUNTING 2024-04-02T13:40:25.944 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/Changes 2024-04-02T13:40:25.947 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/CodeOfConflict 2024-04-02T13:40:25.950 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/CodingStyle 2024-04-02T13:40:25.952 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DMA-API-HOWTO.txt 2024-04-02T13:40:25.955 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DMA-API.txt 2024-04-02T13:40:25.958 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DMA-ISA-LPC.txt 2024-04-02T13:40:25.960 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DMA-attributes.txt 2024-04-02T13:40:25.963 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/ 2024-04-02T13:40:25.964 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/.gitignore 2024-04-02T13:40:25.967 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/80211.tmpl 2024-04-02T13:40:25.969 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/Makefile 2024-04-02T13:40:25.971 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/alsa-driver-api.tmpl 2024-04-02T13:40:25.974 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/crypto-API.tmpl 2024-04-02T13:40:25.977 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/debugobjects.tmpl 2024-04-02T13:40:25.979 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/device-drivers.tmpl 2024-04-02T13:40:25.982 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/deviceiobook.tmpl 2024-04-02T13:40:25.984 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/drm.tmpl 2024-04-02T13:40:25.990 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/filesystems.tmpl 2024-04-02T13:40:25.994 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/gadget.tmpl 2024-04-02T13:40:25.997 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/genericirq.tmpl 2024-04-02T13:40:26.000 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/kernel-api.tmpl 2024-04-02T13:40:26.002 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/kernel-hacking.tmpl 2024-04-02T13:40:26.005 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/kernel-locking.tmpl 2024-04-02T13:40:26.008 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/kgdb.tmpl 2024-04-02T13:40:26.012 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/libata.tmpl 2024-04-02T13:40:26.016 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/librs.tmpl 2024-04-02T13:40:26.019 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/lsm.tmpl 2024-04-02T13:40:26.021 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/ 2024-04-02T13:40:26.022 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/Makefile 2024-04-02T13:40:26.025 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/bayer.png.b64 2024-04-02T13:40:26.027 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/constraints.png.b64 2024-04-02T13:40:26.029 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/crop.gif.b64 2024-04-02T13:40:26.032 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ 2024-04-02T13:40:26.033 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/.gitignore 2024-04-02T13:40:26.035 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/audio.xml 2024-04-02T13:40:26.038 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ca.xml 2024-04-02T13:40:26.041 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/demux.xml 2024-04-02T13:40:26.044 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbapi.xml 2024-04-02T13:40:26.046 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbproperty.xml 2024-04-02T13:40:26.049 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbstb.pdf 2024-04-02T13:40:26.052 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/examples.xml 2024-04-02T13:40:26.054 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/frontend.xml 2024-04-02T13:40:26.057 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/intro.xml 2024-04-02T13:40:26.061 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/kdapi.xml 2024-04-02T13:40:26.064 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/net.xml 2024-04-02T13:40:26.066 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/video.xml 2024-04-02T13:40:26.070 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/dvbstb.png.b64 2024-04-02T13:40:26.074 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_bt.gif.b64 2024-04-02T13:40:26.077 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_tb.gif.b64 2024-04-02T13:40:26.082 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt.gif.b64 2024-04-02T13:40:26.083 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt_example.gif.b64 2024-04-02T13:40:26.085 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/pipeline.png.b64 2024-04-02T13:40:26.087 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/selection.png.b64 2024-04-02T13:40:26.090 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/ 2024-04-02T13:40:26.092 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/.gitignore 2024-04-02T13:40:26.094 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/biblio.xml 2024-04-02T13:40:26.097 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/capture.c.xml 2024-04-02T13:40:26.099 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/common.xml 2024-04-02T13:40:26.102 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/compat.xml 2024-04-02T13:40:26.107 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/controls.xml 2024-04-02T13:40:26.114 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/crop.pdf 2024-04-02T13:40:26.116 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-capture.xml 2024-04-02T13:40:26.119 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-codec.xml 2024-04-02T13:40:26.122 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-effect.xml 2024-04-02T13:40:26.124 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-event.xml 2024-04-02T13:40:26.126 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-osd.xml 2024-04-02T13:40:26.129 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-output.xml 2024-04-02T13:40:26.132 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-overlay.xml 2024-04-02T13:40:26.134 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-radio.xml 2024-04-02T13:40:26.136 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-raw-vbi.xml 2024-04-02T13:40:26.139 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-rds.xml 2024-04-02T13:40:26.142 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sdr.xml 2024-04-02T13:40:26.144 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sliced-vbi.xml 2024-04-02T13:40:26.147 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-subdev.xml 2024-04-02T13:40:26.150 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-teletext.xml 2024-04-02T13:40:26.153 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/driver.xml 2024-04-02T13:40:26.155 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fdl-appendix.xml 2024-04-02T13:40:26.158 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_bt.pdf 2024-04-02T13:40:26.160 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_tb.pdf 2024-04-02T13:40:26.163 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-close.xml 2024-04-02T13:40:26.165 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-ioctl.xml 2024-04-02T13:40:26.168 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-mmap.xml 2024-04-02T13:40:26.170 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-munmap.xml 2024-04-02T13:40:26.173 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-open.xml 2024-04-02T13:40:26.175 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-poll.xml 2024-04-02T13:40:26.178 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-read.xml 2024-04-02T13:40:26.180 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-select.xml 2024-04-02T13:40:26.183 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-write.xml 2024-04-02T13:40:26.185 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/gen-errors.xml 2024-04-02T13:40:26.187 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/io.xml 2024-04-02T13:40:26.191 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/keytable.c.xml 2024-04-02T13:40:26.193 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/libv4l.xml 2024-04-02T13:40:26.196 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/lirc_device_interface.xml 2024-04-02T13:40:26.199 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-controller.xml 2024-04-02T13:40:26.201 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-close.xml 2024-04-02T13:40:26.203 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-ioctl.xml 2024-04-02T13:40:26.206 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-open.xml 2024-04-02T13:40:26.208 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-device-info.xml 2024-04-02T13:40:26.211 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-entities.xml 2024-04-02T13:40:26.214 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-links.xml 2024-04-02T13:40:26.216 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-setup-link.xml 2024-04-02T13:40:26.219 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pipeline.pdf 2024-04-02T13:40:26.221 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-grey.xml 2024-04-02T13:40:26.224 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-m420.xml 2024-04-02T13:40:26.226 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12.xml 2024-04-02T13:40:26.229 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12m.xml 2024-04-02T13:40:26.231 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12mt.xml 2024-04-02T13:40:26.233 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16.xml 2024-04-02T13:40:26.236 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16m.xml 2024-04-02T13:40:26.238 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv24.xml 2024-04-02T13:40:26.241 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-rgb.xml 2024-04-02T13:40:26.244 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-yuv.xml 2024-04-02T13:40:26.247 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr16.xml 2024-04-02T13:40:26.249 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr8.xml 2024-04-02T13:40:26.251 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs08.xml 2024-04-02T13:40:26.253 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs14le.xml 2024-04-02T13:40:26.256 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu08.xml 2024-04-02T13:40:26.258 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu16le.xml 2024-04-02T13:40:26.261 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-ru12le.xml 2024-04-02T13:40:26.263 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgbrg8.xml 2024-04-02T13:40:26.265 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgrbg8.xml 2024-04-02T13:40:26.268 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10.xml 2024-04-02T13:40:26.270 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10alaw8.xml 2024-04-02T13:40:26.273 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10dpcm8.xml 2024-04-02T13:40:26.275 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10p.xml 2024-04-02T13:40:26.278 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb12.xml 2024-04-02T13:40:26.280 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb8.xml 2024-04-02T13:40:26.282 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uv8.xml 2024-04-02T13:40:26.286 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uyvy.xml 2024-04-02T13:40:26.286 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-vyuy.xml 2024-04-02T13:40:26.289 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10.xml 2024-04-02T13:40:26.291 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10b.xml 2024-04-02T13:40:26.293 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y12.xml 2024-04-02T13:40:26.296 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y16.xml 2024-04-02T13:40:26.298 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y41p.xml 2024-04-02T13:40:26.300 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv410.xml 2024-04-02T13:40:26.303 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv411p.xml 2024-04-02T13:40:26.305 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420.xml 2024-04-02T13:40:26.308 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420m.xml 2024-04-02T13:40:26.311 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv422p.xml 2024-04-02T13:40:26.314 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuyv.xml 2024-04-02T13:40:26.316 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvu420m.xml 2024-04-02T13:40:26.319 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvyu.xml 2024-04-02T13:40:26.321 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt.xml 2024-04-02T13:40:26.325 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/planar-apis.xml 2024-04-02T13:40:26.328 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/remote_controllers.xml 2024-04-02T13:40:26.330 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selection-api.xml 2024-04-02T13:40:26.333 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selections-common.xml 2024-04-02T13:40:26.335 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-formats.xml 2024-04-02T13:40:26.341 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.dia 2024-04-02T13:40:26.344 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.svg 2024-04-02T13:40:26.346 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.dia 2024-04-02T13:40:26.351 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.svg 2024-04-02T13:40:26.353 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.dia 2024-04-02T13:40:26.357 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.svg 2024-04-02T13:40:26.360 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2.xml 2024-04-02T13:40:26.363 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2grab.c.xml 2024-04-02T13:40:26.366 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_525.pdf 2024-04-02T13:40:26.368 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_625.pdf 2024-04-02T13:40:26.371 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_hsync.pdf 2024-04-02T13:40:26.373 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml 2024-04-02T13:40:26.375 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-cropcap.xml 2024-04-02T13:40:26.378 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-chip-info.xml 2024-04-02T13:40:26.380 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-register.xml 2024-04-02T13:40:26.383 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-decoder-cmd.xml 2024-04-02T13:40:26.386 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dqevent.xml 2024-04-02T13:40:26.388 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml 2024-04-02T13:40:26.391 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-encoder-cmd.xml 2024-04-02T13:40:26.393 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml 2024-04-02T13:40:26.395 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-fmt.xml 2024-04-02T13:40:26.398 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-frameintervals.xml 2024-04-02T13:40:26.401 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-framesizes.xml 2024-04-02T13:40:26.403 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml 2024-04-02T13:40:26.405 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudio.xml 2024-04-02T13:40:26.407 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudioout.xml 2024-04-02T13:40:26.410 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enuminput.xml 2024-04-02T13:40:26.412 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumoutput.xml 2024-04-02T13:40:26.415 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumstd.xml 2024-04-02T13:40:26.418 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-expbuf.xml 2024-04-02T13:40:26.420 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audio.xml 2024-04-02T13:40:26.422 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audioout.xml 2024-04-02T13:40:26.424 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-crop.xml 2024-04-02T13:40:26.426 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ctrl.xml 2024-04-02T13:40:26.429 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-dv-timings.xml 2024-04-02T13:40:26.432 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-edid.xml 2024-04-02T13:40:26.434 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-enc-index.xml 2024-04-02T13:40:26.436 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ext-ctrls.xml 2024-04-02T13:40:26.439 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fbuf.xml 2024-04-02T13:40:26.442 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fmt.xml 2024-04-02T13:40:26.444 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-frequency.xml 2024-04-02T13:40:26.446 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-input.xml 2024-04-02T13:40:26.449 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-jpegcomp.xml 2024-04-02T13:40:26.451 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-modulator.xml 2024-04-02T13:40:26.454 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-output.xml 2024-04-02T13:40:26.457 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-parm.xml 2024-04-02T13:40:26.459 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-priority.xml 2024-04-02T13:40:26.462 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-selection.xml 2024-04-02T13:40:26.465 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-sliced-vbi-cap.xml 2024-04-02T13:40:26.467 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-std.xml 2024-04-02T13:40:26.469 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-tuner.xml 2024-04-02T13:40:26.472 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-log-status.xml 2024-04-02T13:40:26.475 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-overlay.xml 2024-04-02T13:40:26.477 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml 2024-04-02T13:40:26.480 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-qbuf.xml 2024-04-02T13:40:26.482 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml 2024-04-02T13:40:26.485 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querybuf.xml 2024-04-02T13:40:26.793 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:26 smithi145 ceph-mon[35646]: pgmap v52: 129 pgs: 129 active+clean; 80 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.6 MiB/s wr, 25 op/s 2024-04-02T13:40:26.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:26 smithi050 ceph-mon[35524]: pgmap v52: 129 pgs: 129 active+clean; 80 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.6 MiB/s wr, 25 op/s 2024-04-02T13:40:27.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:26 smithi033 ceph-mon[32390]: pgmap v52: 129 pgs: 129 active+clean; 80 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.6 MiB/s wr, 25 op/s 2024-04-02T13:40:28.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:28 smithi145 ceph-mon[35646]: pgmap v53: 129 pgs: 129 active+clean; 80 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.6 MiB/s wr, 25 op/s 2024-04-02T13:40:28.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:28 smithi050 ceph-mon[35524]: pgmap v53: 129 pgs: 129 active+clean; 80 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.6 MiB/s wr, 25 op/s 2024-04-02T13:40:29.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:28 smithi033 ceph-mon[32390]: pgmap v53: 129 pgs: 129 active+clean; 80 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.6 MiB/s wr, 25 op/s 2024-04-02T13:40:30.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:40:30.299 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:40:30.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.417+0000 7f5f1d72b640 1 -- 172.21.15.33:0/498283495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18102450 msgr2=0x7f5f1810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.417+0000 7f5f1d72b640 1 --2- 172.21.15.33:0/498283495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18102450 0x7f5f1810e960 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009cd0 tx=0x7f5f1002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.417+0000 7f5f1d72b640 1 -- 172.21.15.33:0/498283495 shutdown_connections 2024-04-02T13:40:30.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.417+0000 7f5f1d72b640 1 --2- 172.21.15.33:0/498283495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18102450 0x7f5f1810e960 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.417+0000 7f5f1d72b640 1 --2- 172.21.15.33:0/498283495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f18101a90 0x7f5f18101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.417+0000 7f5f1d72b640 1 --2- 172.21.15.33:0/498283495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f18107aa0 0x7f5f18107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.417+0000 7f5f1d72b640 1 -- 172.21.15.33:0/498283495 >> 172.21.15.33:0/498283495 conn(0x7f5f18078f00 msgr2=0x7f5f18079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:30.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.417+0000 7f5f1d72b640 1 -- 172.21.15.33:0/498283495 shutdown_connections 2024-04-02T13:40:30.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.417+0000 7f5f1d72b640 1 -- 172.21.15.33:0/498283495 wait complete. 2024-04-02T13:40:30.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.418+0000 7f5f1d72b640 1 Processor -- start 2024-04-02T13:40:30.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.418+0000 7f5f1d72b640 1 -- start start 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.418+0000 7f5f1d72b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f18101a90 0x7f5f18198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.418+0000 7f5f1d72b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18102450 0x7f5f181988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.418+0000 7f5f1d72b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f18107aa0 0x7f5f1819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.418+0000 7f5f1d72b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18111e80 con 0x7f5f18102450 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.418+0000 7f5f1d72b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18111d00 con 0x7f5f18107aa0 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.418+0000 7f5f1d72b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18112000 con 0x7f5f18101a90 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.418+0000 7f5f167fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18102450 0x7f5f181988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.418+0000 7f5f167fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18102450 0x7f5f181988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47040/0 (socket says 172.21.15.33:47040) 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.418+0000 7f5f167fc640 1 -- 172.21.15.33:0/2208336700 learned_addr learned my addr 172.21.15.33:0/2208336700 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.418+0000 7f5f16ffd640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f18101a90 0x7f5f18198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.418+0000 7f5f177fe640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f18107aa0 0x7f5f1819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.419+0000 7f5f167fc640 1 -- 172.21.15.33:0/2208336700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f18101a90 msgr2=0x7f5f18198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.419+0000 7f5f167fc640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f18101a90 0x7f5f18198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.419+0000 7f5f167fc640 1 -- 172.21.15.33:0/2208336700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f18107aa0 msgr2=0x7f5f1819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.419+0000 7f5f167fc640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f18107aa0 0x7f5f1819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.419+0000 7f5f167fc640 1 -- 172.21.15.33:0/2208336700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1819d370 con 0x7f5f18102450 2024-04-02T13:40:30.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.419+0000 7f5f167fc640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18102450 0x7f5f181988c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00d770 tx=0x7f5f0c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.419+0000 7f5efffff640 1 -- 172.21.15.33:0/2208336700 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c017070 con 0x7f5f18102450 2024-04-02T13:40:30.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.419+0000 7f5efffff640 1 -- 172.21.15.33:0/2208336700 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f0c0054f0 con 0x7f5f18102450 2024-04-02T13:40:30.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.419+0000 7f5f1d72b640 1 -- 172.21.15.33:0/2208336700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1819d660 con 0x7f5f18102450 2024-04-02T13:40:30.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.419+0000 7f5efffff640 1 -- 172.21.15.33:0/2208336700 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c004d60 con 0x7f5f18102450 2024-04-02T13:40:30.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.419+0000 7f5f1d72b640 1 -- 172.21.15.33:0/2208336700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f181a4ef0 con 0x7f5f18102450 2024-04-02T13:40:30.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.422+0000 7f5efffff640 1 -- 172.21.15.33:0/2208336700 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5f0c004f00 con 0x7f5f18102450 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.422+0000 7f5f203ec740 1 Processor -- start 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.422+0000 7f5f203ec740 1 -- start start 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.422+0000 7f5f203ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fd8d0 0x5613f47fdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.422+0000 7f5f203ec740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 0x5613f4806890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.422+0000 7f5f203ec740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f4806dd0 0x5613f48091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.422+0000 7f5f203ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5613f4761830 con 0x5613f47fd8d0 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.422+0000 7f5f203ec740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5613f4709820 con 0x5613f4806dd0 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.422+0000 7f5f203ec740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5613f475d7e0 con 0x5613f47fe1f0 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.422+0000 7f5f177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fd8d0 0x5613f47fdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.422+0000 7f5f177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fd8d0 0x5613f47fdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47050/0 (socket says 172.21.15.33:47050) 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.422+0000 7f5f177fe640 1 -- 172.21.15.33:0/274571948 learned_addr learned my addr 172.21.15.33:0/274571948 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.422+0000 7f5efffff640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f000a3d20 0x7f5f000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.423+0000 7f5f167fc640 1 --2- 172.21.15.33:0/274571948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f4806dd0 0x5613f48091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.423+0000 7f5f16ffd640 1 --2- 172.21.15.33:0/274571948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 0x5613f4806890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.423+0000 7f5f177fe640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f000a3d20 0x7f5f000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.423+0000 7f5f177fe640 1 -- 172.21.15.33:0/274571948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 msgr2=0x5613f4806890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.423+0000 7f5f177fe640 1 --2- 172.21.15.33:0/274571948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 0x5613f4806890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.423+0000 7f5f177fe640 1 -- 172.21.15.33:0/274571948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f4806dd0 msgr2=0x5613f48091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.423+0000 7f5f177fe640 1 --2- 172.21.15.33:0/274571948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f4806dd0 0x5613f48091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.423+0000 7f5f177fe640 1 -- 172.21.15.33:0/274571948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613f453f7d0 con 0x5613f47fd8d0 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.423+0000 7f5efffff640 1 -- 172.21.15.33:0/2208336700 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5f0c0d05b0 con 0x7f5f18102450 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.423+0000 7f5f177fe640 1 --2- 172.21.15.33:0/274571948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fd8d0 0x5613f47fdcb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5f10004af0 tx=0x7f5f10032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5a3ba77e9fb97e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.423+0000 7f5efd7fa640 1 -- 172.21.15.33:0/274571948 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10039040 con 0x5613f47fd8d0 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.423+0000 7f5efd7fa640 1 -- 172.21.15.33:0/274571948 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f1003a040 con 0x5613f47fd8d0 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.423+0000 7f5efd7fa640 1 -- 172.21.15.33:0/274571948 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10004600 con 0x5613f47fd8d0 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.424+0000 7f5f203ec740 1 -- 172.21.15.33:0/274571948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fd8d0 msgr2=0x5613f47fdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.424+0000 7f5f203ec740 1 --2- 172.21.15.33:0/274571948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fd8d0 0x5613f47fdcb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5f10004af0 tx=0x7f5f10032ea0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.424+0000 7f5f177fe640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f000a3d20 0x7f5f000a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5f1004e000 tx=0x7f5f1004c000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.424+0000 7f5f203ec740 1 -- 172.21.15.33:0/274571948 shutdown_connections 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.424+0000 7f5f203ec740 1 --2- 172.21.15.33:0/274571948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f4806dd0 0x5613f48091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.424+0000 7f5f203ec740 1 --2- 172.21.15.33:0/274571948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 0x5613f4806890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.424+0000 7f5f203ec740 1 --2- 172.21.15.33:0/274571948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fd8d0 0x5613f47fdcb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.424+0000 7f5f203ec740 1 -- 172.21.15.33:0/274571948 >> 172.21.15.33:0/274571948 conn(0x5613f47f8000 msgr2=0x5613f47f83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.424+0000 7f5f203ec740 1 -- 172.21.15.33:0/274571948 shutdown_connections 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.424+0000 7f5f203ec740 1 -- 172.21.15.33:0/274571948 wait complete. 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.424+0000 7f5f203ec740 1 -- start start 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.424+0000 7f5f203ec740 1 Processor -- start 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f203ec740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 0x5613f48065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f203ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fe1f0 0x5613f48039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f203ec740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f4806dd0 0x5613f4803f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f203ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5613f4809bc0 con 0x5613f47fe1f0 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f203ec740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5613f471cd30 con 0x5613f47fd8d0 2024-04-02T13:40:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f203ec740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5613f4809d40 con 0x5613f4806dd0 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f203ec740 10 client.0 _ll_register_callbacks cb 0x5613f471c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f167fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fe1f0 0x5613f48039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f167fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fe1f0 0x5613f48039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47056/0 (socket says 172.21.15.33:47056) 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f167fc640 1 -- 172.21.15.33:0/1552732182 learned_addr learned my addr 172.21.15.33:0/1552732182 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f16ffd640 1 --2- 172.21.15.33:0/1552732182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 0x5613f48065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f177fe640 1 --2- 172.21.15.33:0/1552732182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f4806dd0 0x5613f4803f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f167fc640 1 -- 172.21.15.33:0/1552732182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f4806dd0 msgr2=0x5613f4803f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f167fc640 1 --2- 172.21.15.33:0/1552732182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f4806dd0 0x5613f4803f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f167fc640 1 -- 172.21.15.33:0/1552732182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 msgr2=0x5613f48065c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f167fc640 1 --2- 172.21.15.33:0/1552732182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 0x5613f48065c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f167fc640 1 -- 172.21.15.33:0/1552732182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613f44753f0 con 0x5613f47fe1f0 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f167fc640 1 --2- 172.21.15.33:0/1552732182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fe1f0 0x5613f48039c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00d740 tx=0x7f5f0c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e56a0eb5964d069 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5efaff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5efaff5640 1 -- 172.21.15.33:0/1552732182 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c020070 con 0x5613f47fe1f0 2024-04-02T13:40:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5efaff5640 1 -- 172.21.15.33:0/1552732182 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5f0c0cdbd0 con 0x5613f47fe1f0 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.425+0000 7f5f203ec740 1 -- 172.21.15.33:0/1552732182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f1003a040 con 0x5613f47fe1f0 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.426+0000 7f5efaff5640 1 -- 172.21.15.33:0/1552732182 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f0c0cddd0 con 0x5613f47fe1f0 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.426+0000 7f5f203ec740 10 client.15972 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.426+0000 7f5f203ec740 1 -- 172.21.15.33:0/1552732182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5613f44350f0 con 0x5613f47fe1f0 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.426+0000 7f5efaff5640 1 -- 172.21.15.33:0/1552732182 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f1003a040 con 0x5613f47fe1f0 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.426+0000 7f5efaff5640 1 -- 172.21.15.33:0/1552732182 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5f0c0d01a0 con 0x5613f47fe1f0 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.426+0000 7f5f203ec740 10 client.15972 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.426+0000 7f5f203ec740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.426+0000 7f5f203ec740 20 client.15972 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.426+0000 7f5f203ec740 1 --2- 172.21.15.33:0/1552732182 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5613f4816dc0 0x5613f48191d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.426+0000 7f5f203ec740 4 client.15972 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.426+0000 7f5f203ec740 1 -- 172.21.15.33:0/1552732182 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613f476ed70 con 0x5613f4816dc0 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.426+0000 7f5f16ffd640 1 --2- 172.21.15.33:0/1552732182 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5613f4816dc0 0x5613f48191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.427+0000 7f5f16ffd640 1 --2- 172.21.15.33:0/1552732182 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5613f4816dc0 0x5613f48191d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbc36dec87c13dc6 server_cookie=49a5f7b437bc082c in_seq=0 out_seq=0 2024-04-02T13:40:30.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.427+0000 7f5efaff5640 10 client.15972 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.427+0000 7f5efaff5640 1 -- 172.21.15.33:0/1552732182 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613f476ed70 con 0x5613f4816dc0 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.427+0000 7f5efaff5640 10 client.15972 handle_command_reply: tid=0 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.427+0000 7f5f203ec740 1 client.15972 shutdown 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.428+0000 7f5f203ec740 1 -- 172.21.15.33:0/1552732182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fe1f0 msgr2=0x5613f48039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.428+0000 7f5f203ec740 1 --2- 172.21.15.33:0/1552732182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fe1f0 0x5613f48039c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00d740 tx=0x7f5f0c0056a0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.428+0000 7f5f203ec740 1 -- 172.21.15.33:0/1552732182 shutdown_connections 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.428+0000 7f5f203ec740 1 --2- 172.21.15.33:0/1552732182 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5613f4816dc0 0x5613f48191d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.428+0000 7f5f203ec740 1 --2- 172.21.15.33:0/1552732182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f4806dd0 0x5613f4803f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.428+0000 7f5f203ec740 1 --2- 172.21.15.33:0/1552732182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f47fe1f0 0x5613f48039c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.428+0000 7f5f203ec740 1 --2- 172.21.15.33:0/1552732182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 0x5613f48065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.428+0000 7f5efaff5640 0 client.15972 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.428+0000 7f5f203ec740 1 -- 172.21.15.33:0/1552732182 >> 172.21.15.33:0/1552732182 conn(0x5613f47f8000 msgr2=0x5613f47f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.428+0000 7f5f203ec740 1 -- 172.21.15.33:0/1552732182 shutdown_connections 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.428+0000 7f5f203ec740 1 -- 172.21.15.33:0/1552732182 wait complete. 2024-04-02T13:40:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.428+0000 7f5f203ec740 20 client.15972 trim_cache size 0 max 16384 2024-04-02T13:40:30.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.445+0000 7f5f203ec740 1 Processor -- start 2024-04-02T13:40:30.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.445+0000 7f5f203ec740 1 -- start start 2024-04-02T13:40:30.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.445+0000 7f5f203ec740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 0x5613f4812ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.445+0000 7f5f203ec740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 0x5613f481eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.445+0000 7f5f203ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 0x5613f481efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f203ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181b1690 con 0x5613f4806dd0 2024-04-02T13:40:30.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f203ec740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181aef60 con 0x5613f47fd8d0 2024-04-02T13:40:30.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f203ec740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181acd90 con 0x5613f47fe1f0 2024-04-02T13:40:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 0x5613f481efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 0x5613f481efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47066/0 (socket says 172.21.15.33:47066) 2024-04-02T13:40:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f177fe640 1 -- 172.21.15.33:0/4047320259 learned_addr learned my addr 172.21.15.33:0/4047320259 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f167fc640 1 --2- 172.21.15.33:0/4047320259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 0x5613f481eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f177fe640 1 -- 172.21.15.33:0/4047320259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 msgr2=0x5613f481eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f177fe640 1 --2- 172.21.15.33:0/4047320259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 0x5613f481eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f177fe640 1 -- 172.21.15.33:0/4047320259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 msgr2=0x5613f4812ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f16ffd640 1 --2- 172.21.15.33:0/4047320259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 0x5613f4812ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f177fe640 1 --2- 172.21.15.33:0/4047320259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 0x5613f4812ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f177fe640 1 -- 172.21.15.33:0/4047320259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613f47f7e30 con 0x5613f4806dd0 2024-04-02T13:40:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5f177fe640 1 --2- 172.21.15.33:0/4047320259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 0x5613f481efe0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5f10004af0 tx=0x7f5f1003a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831e5667ecfe4274 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5efcff9640 1 -- 172.21.15.33:0/4047320259 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10039030 con 0x5613f4806dd0 2024-04-02T13:40:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5efcff9640 1 -- 172.21.15.33:0/4047320259 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f100328f0 con 0x5613f4806dd0 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.446+0000 7f5efcff9640 1 -- 172.21.15.33:0/4047320259 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10003e30 con 0x5613f4806dd0 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 -- 172.21.15.33:0/4047320259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 msgr2=0x5613f481efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 --2- 172.21.15.33:0/4047320259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 0x5613f481efe0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5f10004af0 tx=0x7f5f1003a040 comp rx=0 tx=0).stop 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 -- 172.21.15.33:0/4047320259 shutdown_connections 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 --2- 172.21.15.33:0/4047320259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 0x5613f481efe0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 --2- 172.21.15.33:0/4047320259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 0x5613f481eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 --2- 172.21.15.33:0/4047320259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 0x5613f4812ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 -- 172.21.15.33:0/4047320259 >> 172.21.15.33:0/4047320259 conn(0x5613f47f8000 msgr2=0x5613f4802290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 -- 172.21.15.33:0/4047320259 shutdown_connections 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 -- 172.21.15.33:0/4047320259 wait complete. 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 -- start start 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 Processor -- start 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 0x5613f480f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 0x5613f480a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 0x5613f480aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181ad420 con 0x5613f4806dd0 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181ae890 con 0x5613f47fd8d0 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181adaf0 con 0x5613f47fe1f0 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.447+0000 7f5f203ec740 10 client.0 _ll_register_callbacks cb 0x7f5f0c0bcae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 0x5613f480aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 0x5613f480aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47078/0 (socket says 172.21.15.33:47078) 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f177fe640 1 -- 172.21.15.33:0/2279262628 learned_addr learned my addr 172.21.15.33:0/2279262628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f16ffd640 1 --2- 172.21.15.33:0/2279262628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 0x5613f480f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f167fc640 1 --2- 172.21.15.33:0/2279262628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 0x5613f480a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f177fe640 1 -- 172.21.15.33:0/2279262628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 msgr2=0x5613f480a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f177fe640 1 --2- 172.21.15.33:0/2279262628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 0x5613f480a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f177fe640 1 -- 172.21.15.33:0/2279262628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 msgr2=0x5613f480f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f177fe640 1 --2- 172.21.15.33:0/2279262628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 0x5613f480f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f177fe640 1 -- 172.21.15.33:0/2279262628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613f480b110 con 0x5613f4806dd0 2024-04-02T13:40:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f177fe640 1 --2- 172.21.15.33:0/2279262628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 0x5613f480aad0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5f1002c130 tx=0x7f5f10009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c824a07698b9b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5efaff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5efaff5640 1 -- 172.21.15.33:0/2279262628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10062070 con 0x5613f4806dd0 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5efaff5640 1 -- 172.21.15.33:0/2279262628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5f10034440 con 0x5613f4806dd0 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f203ec740 1 -- 172.21.15.33:0/2279262628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f100328f0 con 0x5613f4806dd0 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5efaff5640 1 -- 172.21.15.33:0/2279262628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5f10039040 con 0x5613f4806dd0 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f203ec740 10 client.15978 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5f203ec740 1 -- 172.21.15.33:0/2279262628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5613f487d190 con 0x5613f4806dd0 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.448+0000 7f5efaff5640 1 -- 172.21.15.33:0/2279262628 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f100328f0 con 0x5613f4806dd0 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.449+0000 7f5efaff5640 1 -- 172.21.15.33:0/2279262628 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5f10067460 con 0x5613f4806dd0 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.449+0000 7f5f203ec740 10 client.15978 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.449+0000 7f5f203ec740 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.449+0000 7f5f203ec740 20 client.15978 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.449+0000 7f5f203ec740 1 --2- 172.21.15.33:0/2279262628 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5613f487d460 0x5613f487f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.449+0000 7f5f203ec740 4 client.15978 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:40:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.449+0000 7f5f203ec740 1 -- 172.21.15.33:0/2279262628 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5613f47690c0 con 0x5613f487d460 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.449+0000 7f5f16ffd640 1 --2- 172.21.15.33:0/2279262628 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5613f487d460 0x5613f487f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.449+0000 7f5efaff5640 10 client.15978 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.449+0000 7f5f16ffd640 1 --2- 172.21.15.33:0/2279262628 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5613f487d460 0x5613f487f850 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca15107c3189979e server_cookie=fadd69c351237079 in_seq=0 out_seq=0 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5efaff5640 1 -- 172.21.15.33:0/2279262628 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5613f47690c0 con 0x5613f487d460 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5efaff5640 10 client.15978 handle_command_reply: tid=0 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f203ec740 1 client.15978 shutdown 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f203ec740 1 -- 172.21.15.33:0/2279262628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 msgr2=0x5613f480aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f203ec740 1 --2- 172.21.15.33:0/2279262628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 0x5613f480aad0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5f1002c130 tx=0x7f5f10009c20 comp rx=0 tx=0).stop 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f203ec740 1 -- 172.21.15.33:0/2279262628 shutdown_connections 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f203ec740 1 --2- 172.21.15.33:0/2279262628 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5613f487d460 0x5613f487f850 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f203ec740 1 --2- 172.21.15.33:0/2279262628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f4806dd0 0x5613f480aad0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f203ec740 1 --2- 172.21.15.33:0/2279262628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5613f47fe1f0 0x5613f480a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f203ec740 1 --2- 172.21.15.33:0/2279262628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613f47fd8d0 0x5613f480f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f203ec740 1 -- 172.21.15.33:0/2279262628 >> 172.21.15.33:0/2279262628 conn(0x5613f47f8000 msgr2=0x5613f48026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:30.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5efaff5640 0 client.15978 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f203ec740 1 -- 172.21.15.33:0/2279262628 shutdown_connections 2024-04-02T13:40:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f203ec740 1 -- 172.21.15.33:0/2279262628 wait complete. 2024-04-02T13:40:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f203ec740 20 client.15978 trim_cache size 0 max 16384 2024-04-02T13:40:30.459 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:40:30.459 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:40:30.459 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:40:30.459 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:40:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f1d72b640 1 -- 172.21.15.33:0/2208336700 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f000a3d20 msgr2=0x7f5f000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f1d72b640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f000a3d20 0x7f5f000a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5f1004e000 tx=0x7f5f1004c000 comp rx=0 tx=0).stop 2024-04-02T13:40:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f1d72b640 1 -- 172.21.15.33:0/2208336700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18102450 msgr2=0x7f5f181988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f1d72b640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18102450 0x7f5f181988c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00d770 tx=0x7f5f0c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:40:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f1d72b640 1 -- 172.21.15.33:0/2208336700 shutdown_connections 2024-04-02T13:40:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f1d72b640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f000a3d20 0x7f5f000a61e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f1d72b640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f18107aa0 0x7f5f1819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f1d72b640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f18102450 0x7f5f181988c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f1d72b640 1 --2- 172.21.15.33:0/2208336700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f18101a90 0x7f5f18198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f1d72b640 1 -- 172.21.15.33:0/2208336700 >> 172.21.15.33:0/2208336700 conn(0x7f5f18078f00 msgr2=0x7f5f18104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:30.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.450+0000 7f5f1d72b640 1 -- 172.21.15.33:0/2208336700 shutdown_connections 2024-04-02T13:40:30.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.451+0000 7f5f1d72b640 1 -- 172.21.15.33:0/2208336700 wait complete. 2024-04-02T13:40:30.464 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:40:30.464 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:40:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.564+0000 7f71484a5640 1 -- 172.21.15.33:0/341062771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7140106a40 msgr2=0x7f7140106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.564+0000 7f71484a5640 1 --2- 172.21.15.33:0/341062771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7140106a40 0x7f7140106e20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7130009cd0 tx=0x7f713002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.564+0000 7f71484a5640 1 -- 172.21.15.33:0/341062771 shutdown_connections 2024-04-02T13:40:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.564+0000 7f71484a5640 1 --2- 172.21.15.33:0/341062771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7140101440 0x7f714010d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.564+0000 7f71484a5640 1 --2- 172.21.15.33:0/341062771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7140100a80 0x7f7140100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.564+0000 7f71484a5640 1 --2- 172.21.15.33:0/341062771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7140106a40 0x7f7140106e20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.564+0000 7f71484a5640 1 -- 172.21.15.33:0/341062771 >> 172.21.15.33:0/341062771 conn(0x7f7140077b60 msgr2=0x7f7140077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.564+0000 7f71484a5640 1 -- 172.21.15.33:0/341062771 shutdown_connections 2024-04-02T13:40:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.564+0000 7f71484a5640 1 -- 172.21.15.33:0/341062771 wait complete. 2024-04-02T13:40:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.564+0000 7f71484a5640 1 Processor -- start 2024-04-02T13:40:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f71484a5640 1 -- start start 2024-04-02T13:40:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f71484a5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7140100a80 0x7f71401983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f71484a5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7140101440 0x7f7140198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f71484a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7140106a40 0x7f714019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f71484a5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7140110df0 con 0x7f7140106a40 2024-04-02T13:40:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f71484a5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7140110c70 con 0x7f7140100a80 2024-04-02T13:40:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f71484a5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7140110f70 con 0x7f7140101440 2024-04-02T13:40:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f7146a42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7140106a40 0x7f714019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f7146a42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7140106a40 0x7f714019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47086/0 (socket says 172.21.15.33:47086) 2024-04-02T13:40:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f7146a42640 1 -- 172.21.15.33:0/3773219112 learned_addr learned my addr 172.21.15.33:0/3773219112 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f7146241640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7140100a80 0x7f71401983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f7145a40640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7140101440 0x7f7140198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f7146a42640 1 -- 172.21.15.33:0/3773219112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7140101440 msgr2=0x7f7140198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f7146a42640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7140101440 0x7f7140198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f7146a42640 1 -- 172.21.15.33:0/3773219112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7140100a80 msgr2=0x7f71401983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f7146a42640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7140100a80 0x7f71401983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f7146a42640 1 -- 172.21.15.33:0/3773219112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714019d3b0 con 0x7f7140106a40 2024-04-02T13:40:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f7145a40640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7140101440 0x7f7140198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:40:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f7146a42640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7140106a40 0x7f714019cc80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f713800c970 tx=0x7f713800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.565+0000 7f713f7fe640 1 -- 172.21.15.33:0/3773219112 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138007bb0 con 0x7f7140106a40 2024-04-02T13:40:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.566+0000 7f713f7fe640 1 -- 172.21.15.33:0/3773219112 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7138007d50 con 0x7f7140106a40 2024-04-02T13:40:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.566+0000 7f71484a5640 1 -- 172.21.15.33:0/3773219112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714019d6a0 con 0x7f7140106a40 2024-04-02T13:40:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.566+0000 7f713f7fe640 1 -- 172.21.15.33:0/3773219112 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138018790 con 0x7f7140106a40 2024-04-02T13:40:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.566+0000 7f71484a5640 1 -- 172.21.15.33:0/3773219112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71401a4f30 con 0x7f7140106a40 2024-04-02T13:40:30.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.566+0000 7f71484a5640 1 -- 172.21.15.33:0/3773219112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7140068d40 con 0x7f7140106a40 2024-04-02T13:40:30.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.568+0000 7f713f7fe640 1 -- 172.21.15.33:0/3773219112 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f713800fab0 con 0x7f7140106a40 2024-04-02T13:40:30.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.568+0000 7f713f7fe640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f71280a3d20 0x7f71280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.568+0000 7f7146241640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f71280a3d20 0x7f71280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.569+0000 7f713f7fe640 1 -- 172.21.15.33:0/3773219112 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f71380d0bd0 con 0x7f7140106a40 2024-04-02T13:40:30.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.569+0000 7f7146241640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f71280a3d20 0x7f71280a61e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7130005cc0 tx=0x7f7130005c50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.573+0000 7f713f7fe640 1 -- 172.21.15.33:0/3773219112 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7138014030 con 0x7f7140106a40 2024-04-02T13:40:30.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:30 smithi033 ceph-mon[32390]: pgmap v54: 129 pgs: 129 active+clean; 84 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.9 MiB/s wr, 52 op/s 2024-04-02T13:40:30.798 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:30 smithi050 ceph-mon[35524]: pgmap v54: 129 pgs: 129 active+clean; 84 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.9 MiB/s wr, 52 op/s 2024-04-02T13:40:30.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.814+0000 7f71484a5640 1 -- 172.21.15.33:0/3773219112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f714010a010 con 0x7f7140106a40 2024-04-02T13:40:30.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.814+0000 7f713f7fe640 1 -- 172.21.15.33:0/3773219112 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11873 (secure 0 0 0) 0x7f713809cf00 con 0x7f7140106a40 2024-04-02T13:40:30.817 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:40:30.817 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:38:45.531661+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24473,24512]},"id":1}]} 2024-04-02T13:40:30.817 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 25 2024-04-02T13:40:30.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.817+0000 7f71484a5640 1 -- 172.21.15.33:0/3773219112 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f71280a3d20 msgr2=0x7f71280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.817+0000 7f71484a5640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f71280a3d20 0x7f71280a61e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7130005cc0 tx=0x7f7130005c50 comp rx=0 tx=0).stop 2024-04-02T13:40:30.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.817+0000 7f71484a5640 1 -- 172.21.15.33:0/3773219112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7140106a40 msgr2=0x7f714019cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.817+0000 7f71484a5640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7140106a40 0x7f714019cc80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f713800c970 tx=0x7f713800ce30 comp rx=0 tx=0).stop 2024-04-02T13:40:30.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.817+0000 7f71484a5640 1 -- 172.21.15.33:0/3773219112 shutdown_connections 2024-04-02T13:40:30.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.817+0000 7f71484a5640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f71280a3d20 0x7f71280a61e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.817+0000 7f71484a5640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7140106a40 0x7f714019cc80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.817+0000 7f71484a5640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7140101440 0x7f7140198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.817+0000 7f71484a5640 1 --2- 172.21.15.33:0/3773219112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7140100a80 0x7f71401983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.817+0000 7f71484a5640 1 -- 172.21.15.33:0/3773219112 >> 172.21.15.33:0/3773219112 conn(0x7f7140077b60 msgr2=0x7f714006a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:30.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.817+0000 7f71484a5640 1 -- 172.21.15.33:0/3773219112 shutdown_connections 2024-04-02T13:40:30.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.817+0000 7f71484a5640 1 -- 172.21.15.33:0/3773219112 wait complete. 2024-04-02T13:40:30.831 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:40:30.867 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querycap.xml 2024-04-02T13:40:30.870 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-queryctrl.xml 2024-04-02T13:40:30.873 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querystd.xml 2024-04-02T13:40:30.876 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-reqbufs.xml 2024-04-02T13:40:30.878 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-s-hw-freq-seek.xml 2024-04-02T13:40:30.882 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-streamon.xml 2024-04-02T13:40:30.884 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml 2024-04-02T13:40:30.888 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml 2024-04-02T13:40:30.890 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml 2024-04-02T13:40:30.893 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-crop.xml 2024-04-02T13:40:30.896 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml 2024-04-02T13:40:30.899 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml 2024-04-02T13:40:30.902 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml 2024-04-02T13:40:30.905 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml 2024-04-02T13:40:30.907 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_525.gif.b64 2024-04-02T13:40:30.911 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_625.gif.b64 2024-04-02T13:40:30.914 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_hsync.gif.b64 2024-04-02T13:40:30.916 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/media_api.tmpl 2024-04-02T13:40:30.919 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/mtdnand.tmpl 2024-04-02T13:40:30.923 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/networking.tmpl 2024-04-02T13:40:30.925 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/rapidio.tmpl 2024-04-02T13:40:30.927 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/regulator.tmpl 2024-04-02T13:40:30.931 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/s390-drivers.tmpl 2024-04-02T13:40:30.934 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/scsi.tmpl 2024-04-02T13:40:30.936 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/sh.tmpl 2024-04-02T13:40:30.939 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/stylesheet.xsl 2024-04-02T13:40:30.941 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/tracepoint.tmpl 2024-04-02T13:40:30.944 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/uio-howto.tmpl 2024-04-02T13:40:30.947 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/usb.tmpl 2024-04-02T13:40:30.951 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/w1.tmpl 2024-04-02T13:40:30.953 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/writing-an-alsa-driver.tmpl 2024-04-02T13:40:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.955+0000 7f8f260d9640 1 -- 172.21.15.33:0/3756887737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f20068d00 msgr2=0x7f8f20069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.955+0000 7f8f260d9640 1 --2- 172.21.15.33:0/3756887737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f20068d00 0x7f8f20069180 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009930 tx=0x7f8f1402bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.955+0000 7f8f260d9640 1 -- 172.21.15.33:0/3756887737 shutdown_connections 2024-04-02T13:40:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.955+0000 7f8f260d9640 1 --2- 172.21.15.33:0/3756887737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f200696c0 0x7f8f2010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.955+0000 7f8f260d9640 1 --2- 172.21.15.33:0/3756887737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f20068d00 0x7f8f20069180 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.955+0000 7f8f260d9640 1 --2- 172.21.15.33:0/3756887737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f20107b50 0x7f8f20107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.955+0000 7f8f260d9640 1 -- 172.21.15.33:0/3756887737 >> 172.21.15.33:0/3756887737 conn(0x7f8f20077b00 msgr2=0x7f8f20077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.955+0000 7f8f260d9640 1 -- 172.21.15.33:0/3756887737 shutdown_connections 2024-04-02T13:40:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.955+0000 7f8f260d9640 1 -- 172.21.15.33:0/3756887737 wait complete. 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.955+0000 7f8f260d9640 1 Processor -- start 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f260d9640 1 -- start start 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f260d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f20068d00 0x7f8f201983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f260d9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f200696c0 0x7f8f201988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f260d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f20107b50 0x7f8f2019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f260d9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f20111e70 con 0x7f8f20107b50 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f260d9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f20111cf0 con 0x7f8f20068d00 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f260d9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8f20111ff0 con 0x7f8f200696c0 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f1ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f20107b50 0x7f8f2019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f1ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f20107b50 0x7f8f2019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47100/0 (socket says 172.21.15.33:47100) 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f1ffff640 1 -- 172.21.15.33:0/3505497390 learned_addr learned my addr 172.21.15.33:0/3505497390 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f1effd640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f200696c0 0x7f8f201988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f1f7fe640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f20068d00 0x7f8f201983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f1ffff640 1 -- 172.21.15.33:0/3505497390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f200696c0 msgr2=0x7f8f201988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f1ffff640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f200696c0 0x7f8f201988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f1ffff640 1 -- 172.21.15.33:0/3505497390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f20068d00 msgr2=0x7f8f201983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f1ffff640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f20068d00 0x7f8f201983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.956+0000 7f8f1ffff640 1 -- 172.21.15.33:0/3505497390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2019d350 con 0x7f8f20107b50 2024-04-02T13:40:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.957+0000 7f8f1ffff640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f20107b50 0x7f8f2019cc20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800eb10 tx=0x7f8f1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.957+0000 7f8f1cff9640 1 -- 172.21.15.33:0/3505497390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18004b20 con 0x7f8f20107b50 2024-04-02T13:40:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.957+0000 7f8f260d9640 1 -- 172.21.15.33:0/3505497390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2019d640 con 0x7f8f20107b50 2024-04-02T13:40:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.957+0000 7f8f1cff9640 1 -- 172.21.15.33:0/3505497390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8f18004cc0 con 0x7f8f20107b50 2024-04-02T13:40:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.957+0000 7f8f1cff9640 1 -- 172.21.15.33:0/3505497390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18010700 con 0x7f8f20107b50 2024-04-02T13:40:30.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.957+0000 7f8f260d9640 1 -- 172.21.15.33:0/3505497390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f201a4ed0 con 0x7f8f20107b50 2024-04-02T13:40:30.963 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/writing_musb_glue_layer.tmpl 2024-04-02T13:40:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.959+0000 7f8f1cff9640 1 -- 172.21.15.33:0/3505497390 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8f18004e70 con 0x7f8f20107b50 2024-04-02T13:40:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f28da1740 1 Processor -- start 2024-04-02T13:40:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f28da1740 1 -- start start 2024-04-02T13:40:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f1cff9640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8f080a3d20 0x7f8f080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f28da1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7d7a0 0x558e2bc7db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f28da1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7e0c0 0x558e2bc86760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f28da1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 0x558e2bc89090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f28da1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558e2bbe17b0 con 0x558e2bc7e0c0 2024-04-02T13:40:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f28da1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558e2bb80570 con 0x558e2bc7d7a0 2024-04-02T13:40:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f28da1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558e2bbdd760 con 0x558e2bc86ca0 2024-04-02T13:40:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f1effd640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8f080a3d20 0x7f8f080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f1cff9640 1 -- 172.21.15.33:0/3505497390 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8f180d10a0 con 0x7f8f20107b50 2024-04-02T13:40:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f1f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7d7a0 0x558e2bc7db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f1f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7d7a0 0x558e2bc7db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:59664/0 (socket says 172.21.15.33:59664) 2024-04-02T13:40:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.960+0000 7f8f1f7fe640 1 -- 172.21.15.33:0/129167420 learned_addr learned my addr 172.21.15.33:0/129167420 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.961+0000 7f8f1effd640 1 --2- 172.21.15.33:0/129167420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7e0c0 0x558e2bc86760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.961+0000 7f8f1ffff640 1 --2- 172.21.15.33:0/129167420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 0x558e2bc89090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.961+0000 7f8f1f7fe640 1 -- 172.21.15.33:0/129167420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 msgr2=0x558e2bc89090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.961+0000 7f8f1f7fe640 1 --2- 172.21.15.33:0/129167420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 0x558e2bc89090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.961+0000 7f8f1f7fe640 1 -- 172.21.15.33:0/129167420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7e0c0 msgr2=0x558e2bc86760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.961+0000 7f8f1f7fe640 1 --2- 172.21.15.33:0/129167420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7e0c0 0x558e2bc86760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.961+0000 7f8f1f7fe640 1 -- 172.21.15.33:0/129167420 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e2b8f53f0 con 0x558e2bc7d7a0 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.961+0000 7f8f1effd640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8f080a3d20 0x7f8f080a61e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8f14000c00 tx=0x7f8f1402c5d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.961+0000 7f8f1f7fe640 1 --2- 172.21.15.33:0/129167420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7d7a0 0x558e2bc7db80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009cf0 tx=0x7f8f10011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4c7380ad346ecde server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.961+0000 7f8f05ffb640 1 -- 172.21.15.33:0/129167420 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1001e070 con 0x558e2bc7d7a0 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.961+0000 7f8f05ffb640 1 -- 172.21.15.33:0/129167420 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8f10009de0 con 0x558e2bc7d7a0 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.961+0000 7f8f05ffb640 1 -- 172.21.15.33:0/129167420 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f100196b0 con 0x558e2bc7d7a0 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.962+0000 7f8f28da1740 1 -- 172.21.15.33:0/129167420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7d7a0 msgr2=0x558e2bc7db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.962+0000 7f8f28da1740 1 --2- 172.21.15.33:0/129167420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7d7a0 0x558e2bc7db80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009cf0 tx=0x7f8f10011810 comp rx=0 tx=0).stop 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.962+0000 7f8f28da1740 1 -- 172.21.15.33:0/129167420 shutdown_connections 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.962+0000 7f8f28da1740 1 --2- 172.21.15.33:0/129167420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 0x558e2bc89090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.962+0000 7f8f28da1740 1 --2- 172.21.15.33:0/129167420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7e0c0 0x558e2bc86760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.962+0000 7f8f28da1740 1 --2- 172.21.15.33:0/129167420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7d7a0 0x558e2bc7db80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.962+0000 7f8f28da1740 1 -- 172.21.15.33:0/129167420 >> 172.21.15.33:0/129167420 conn(0x558e2bc77ed0 msgr2=0x558e2bc782b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.962+0000 7f8f28da1740 1 -- 172.21.15.33:0/129167420 shutdown_connections 2024-04-02T13:40:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.962+0000 7f8f28da1740 1 -- 172.21.15.33:0/129167420 wait complete. 2024-04-02T13:40:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f28da1740 1 -- start start 2024-04-02T13:40:30.968 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/writing_usb_driver.tmpl 2024-04-02T13:40:30.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f28da1740 1 Processor -- start 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f28da1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc83b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f28da1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc7e0c0 0x558e2bc84080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f28da1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc86ca0 0x558e2bc845c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f28da1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558e2bc89a90 con 0x558e2bc7d7a0 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f28da1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558e2bb9ce80 con 0x558e2bc86ca0 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f28da1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558e2bc89c10 con 0x558e2bc7e0c0 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f28da1740 10 client.0 _ll_register_callbacks cb 0x558e2bbd8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f1f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc83b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f1f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc83b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47116/0 (socket says 172.21.15.33:47116) 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f1f7fe640 1 -- 172.21.15.33:0/2589335460 learned_addr learned my addr 172.21.15.33:0/2589335460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f1effd640 1 --2- 172.21.15.33:0/2589335460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc7e0c0 0x558e2bc84080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.963+0000 7f8f1ffff640 1 --2- 172.21.15.33:0/2589335460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc86ca0 0x558e2bc845c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.964+0000 7f8f1f7fe640 1 -- 172.21.15.33:0/2589335460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc7e0c0 msgr2=0x558e2bc84080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.964+0000 7f8f1f7fe640 1 --2- 172.21.15.33:0/2589335460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc7e0c0 0x558e2bc84080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.964+0000 7f8f1f7fe640 1 -- 172.21.15.33:0/2589335460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc86ca0 msgr2=0x558e2bc845c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.964+0000 7f8f1f7fe640 1 --2- 172.21.15.33:0/2589335460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc86ca0 0x558e2bc845c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.964+0000 7f8f1f7fe640 1 -- 172.21.15.33:0/2589335460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e2b9bf7d0 con 0x558e2bc7d7a0 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.964+0000 7f8f1f7fe640 1 --2- 172.21.15.33:0/2589335460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc83b40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8f10004060 tx=0x7f8f10004090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed41d0a542e1bfe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.964+0000 7f8efeffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.964+0000 7f8efeffd640 1 -- 172.21.15.33:0/2589335460 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10029060 con 0x558e2bc7d7a0 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.964+0000 7f8efeffd640 1 -- 172.21.15.33:0/2589335460 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8f100042b0 con 0x558e2bc7d7a0 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.964+0000 7f8f28da1740 1 -- 172.21.15.33:0/2589335460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f10009de0 con 0x558e2bc7d7a0 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.964+0000 7f8efeffd640 1 -- 172.21.15.33:0/2589335460 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f10030050 con 0x558e2bc7d7a0 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.964+0000 7f8f28da1740 10 client.15996 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.964+0000 7f8f28da1740 1 -- 172.21.15.33:0/2589335460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558e2bc95ba0 con 0x558e2bc7d7a0 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.965+0000 7f8efeffd640 1 -- 172.21.15.33:0/2589335460 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8f100044b0 con 0x558e2bc7d7a0 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.965+0000 7f8efeffd640 1 -- 172.21.15.33:0/2589335460 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8f100047e0 con 0x558e2bc7d7a0 2024-04-02T13:40:30.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.965+0000 7f8f28da1740 10 client.15996 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.965+0000 7f8f28da1740 10 client.15996 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:40:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.965+0000 7f8f28da1740 20 client.15996 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.965+0000 7f8f28da1740 1 --2- 172.21.15.33:0/2589335460 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x558e2bc97190 0x558e2bc99580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.965+0000 7f8f28da1740 4 client.15996 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.965+0000 7f8f28da1740 1 -- 172.21.15.33:0/2589335460 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8f10009de0 con 0x558e2bc97190 2024-04-02T13:40:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.965+0000 7f8f1effd640 1 --2- 172.21.15.33:0/2589335460 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x558e2bc97190 0x558e2bc99580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.966+0000 7f8f1effd640 1 --2- 172.21.15.33:0/2589335460 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x558e2bc97190 0x558e2bc99580 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e4d53a4342976ea server_cookie=f4a599abf5853ccd in_seq=0 out_seq=0 2024-04-02T13:40:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.966+0000 7f8efeffd640 10 client.15996 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:40:30.973 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/DocBook/z8530book.tmpl 2024-04-02T13:40:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.966+0000 7f8efeffd640 1 -- 172.21.15.33:0/2589335460 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f8f10009de0 con 0x558e2bc97190 2024-04-02T13:40:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.966+0000 7f8efeffd640 10 client.15996 handle_command_reply: tid=0 2024-04-02T13:40:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.966+0000 7f8f28da1740 1 client.15996 shutdown 2024-04-02T13:40:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.967+0000 7f8f28da1740 1 -- 172.21.15.33:0/2589335460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 msgr2=0x558e2bc83b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.967+0000 7f8f28da1740 1 --2- 172.21.15.33:0/2589335460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc83b40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8f10004060 tx=0x7f8f10004090 comp rx=0 tx=0).stop 2024-04-02T13:40:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.967+0000 7f8f28da1740 1 -- 172.21.15.33:0/2589335460 shutdown_connections 2024-04-02T13:40:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.967+0000 7f8f28da1740 1 --2- 172.21.15.33:0/2589335460 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x558e2bc97190 0x558e2bc99580 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.967+0000 7f8f28da1740 1 --2- 172.21.15.33:0/2589335460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc86ca0 0x558e2bc845c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.967+0000 7f8efeffd640 0 client.15996 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:40:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.967+0000 7f8f28da1740 1 --2- 172.21.15.33:0/2589335460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc7e0c0 0x558e2bc84080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.967+0000 7f8f28da1740 1 --2- 172.21.15.33:0/2589335460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc83b40 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.967+0000 7f8f28da1740 1 -- 172.21.15.33:0/2589335460 >> 172.21.15.33:0/2589335460 conn(0x558e2bc77ed0 msgr2=0x558e2bc79c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.967+0000 7f8f28da1740 1 -- 172.21.15.33:0/2589335460 shutdown_connections 2024-04-02T13:40:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.968+0000 7f8f28da1740 1 -- 172.21.15.33:0/2589335460 wait complete. 2024-04-02T13:40:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.968+0000 7f8f28da1740 20 client.15996 trim_cache size 0 max 16384 2024-04-02T13:40:30.976 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/EDID/ 2024-04-02T13:40:30.976 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/EDID/1024x768.S 2024-04-02T13:40:30.976 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/EDID/1280x1024.S 2024-04-02T13:40:30.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:30 smithi145 ceph-mon[35646]: pgmap v54: 129 pgs: 129 active+clean; 84 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.9 MiB/s wr, 52 op/s 2024-04-02T13:40:30.979 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/EDID/1600x1200.S 2024-04-02T13:40:30.981 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/EDID/1680x1050.S 2024-04-02T13:40:30.983 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/EDID/1920x1080.S 2024-04-02T13:40:30.986 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/EDID/800x600.S 2024-04-02T13:40:30.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.986+0000 7f8f28da1740 1 Processor -- start 2024-04-02T13:40:30.989 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/EDID/HOWTO.txt 2024-04-02T13:40:30.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.986+0000 7f8f28da1740 1 -- start start 2024-04-02T13:40:30.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.986+0000 7f8f28da1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc86270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.986+0000 7f8f28da1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7e0c0 0x558e2bc92950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.986+0000 7f8f28da1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 0x558e2bc82ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.986+0000 7f8f28da1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f201b1830 con 0x558e2bc7d7a0 2024-04-02T13:40:30.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.986+0000 7f8f28da1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f201af100 con 0x558e2bc7e0c0 2024-04-02T13:40:30.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.986+0000 7f8f28da1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8f201acf30 con 0x558e2bc86ca0 2024-04-02T13:40:30.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f1f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc86270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f1effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7e0c0 0x558e2bc92950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f1f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc86270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47124/0 (socket says 172.21.15.33:47124) 2024-04-02T13:40:30.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f1f7fe640 1 -- 172.21.15.33:0/2604180473 learned_addr learned my addr 172.21.15.33:0/2604180473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f1ffff640 1 --2- 172.21.15.33:0/2604180473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 0x558e2bc82ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f1f7fe640 1 -- 172.21.15.33:0/2604180473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 msgr2=0x558e2bc82ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f1f7fe640 1 --2- 172.21.15.33:0/2604180473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 0x558e2bc82ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f1f7fe640 1 -- 172.21.15.33:0/2604180473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7e0c0 msgr2=0x558e2bc92950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f1f7fe640 1 --2- 172.21.15.33:0/2604180473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7e0c0 0x558e2bc92950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f1f7fe640 1 -- 172.21.15.33:0/2604180473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e2bc7def0 con 0x558e2bc7d7a0 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f1f7fe640 1 --2- 172.21.15.33:0/2604180473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc86270 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009c40 tx=0x7f8f100096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102c540e5bd409ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f057fa640 1 -- 172.21.15.33:0/2604180473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10029070 con 0x558e2bc7d7a0 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f057fa640 1 -- 172.21.15.33:0/2604180473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8f100054d0 con 0x558e2bc7d7a0 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.987+0000 7f8f057fa640 1 -- 172.21.15.33:0/2604180473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1001e040 con 0x558e2bc7d7a0 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.988+0000 7f8f28da1740 1 -- 172.21.15.33:0/2604180473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 msgr2=0x558e2bc86270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.988+0000 7f8f28da1740 1 --2- 172.21.15.33:0/2604180473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc86270 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009c40 tx=0x7f8f100096e0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.988+0000 7f8f28da1740 1 -- 172.21.15.33:0/2604180473 shutdown_connections 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.988+0000 7f8f28da1740 1 --2- 172.21.15.33:0/2604180473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 0x558e2bc82ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.988+0000 7f8f28da1740 1 --2- 172.21.15.33:0/2604180473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7e0c0 0x558e2bc92950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.988+0000 7f8f28da1740 1 --2- 172.21.15.33:0/2604180473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc86270 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.988+0000 7f8f28da1740 1 -- 172.21.15.33:0/2604180473 >> 172.21.15.33:0/2604180473 conn(0x558e2bc77ed0 msgr2=0x558e2bc766a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:30.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.988+0000 7f8f28da1740 1 -- 172.21.15.33:0/2604180473 shutdown_connections 2024-04-02T13:40:30.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.988+0000 7f8f28da1740 1 -- 172.21.15.33:0/2604180473 wait complete. 2024-04-02T13:40:30.993 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/EDID/Makefile 2024-04-02T13:40:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.989+0000 7f8f28da1740 1 -- start start 2024-04-02T13:40:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.989+0000 7f8f28da1740 1 Processor -- start 2024-04-02T13:40:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.989+0000 7f8f28da1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc8f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.989+0000 7f8f28da1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7e0c0 0x558e2bc80390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.989+0000 7f8f28da1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 0x558e2bc808d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.989+0000 7f8f28da1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f201ad5c0 con 0x558e2bc7d7a0 2024-04-02T13:40:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.989+0000 7f8f28da1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f201aea30 con 0x558e2bc7e0c0 2024-04-02T13:40:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.989+0000 7f8f28da1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8f201adc90 con 0x558e2bc86ca0 2024-04-02T13:40:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.989+0000 7f8f28da1740 10 client.0 _ll_register_callbacks cb 0x7f8f1001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.989+0000 7f8f1effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7e0c0 0x558e2bc80390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.989+0000 7f8f1f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc8f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.990+0000 7f8f1effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7e0c0 0x558e2bc80390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:59692/0 (socket says 172.21.15.33:59692) 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.990+0000 7f8f1effd640 1 -- 172.21.15.33:0/93459530 learned_addr learned my addr 172.21.15.33:0/93459530 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.990+0000 7f8f1ffff640 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 0x558e2bc808d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.990+0000 7f8f1effd640 1 -- 172.21.15.33:0/93459530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 msgr2=0x558e2bc808d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.990+0000 7f8f1effd640 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 0x558e2bc808d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.990+0000 7f8f1effd640 1 -- 172.21.15.33:0/93459530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 msgr2=0x558e2bc8f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.990+0000 7f8f1effd640 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc8f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.990+0000 7f8f1effd640 1 -- 172.21.15.33:0/93459530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e2bc80e10 con 0x558e2bc7e0c0 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.990+0000 7f8f1f7fe640 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc8f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.990+0000 7f8f1ffff640 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 0x558e2bc808d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.990+0000 7f8f1effd640 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7e0c0 0x558e2bc80390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009230 tx=0x7f8f14046040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b74488a98bc106ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.990+0000 7f8efeffd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8efeffd640 1 -- 172.21.15.33:0/93459530 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1405c070 con 0x558e2bc7e0c0 2024-04-02T13:40:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8efeffd640 1 -- 172.21.15.33:0/93459530 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8f140551c0 con 0x558e2bc7e0c0 2024-04-02T13:40:30.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8f28da1740 1 -- 172.21.15.33:0/93459530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f100054d0 con 0x558e2bc7e0c0 2024-04-02T13:40:30.997 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/EDID/edid.S 2024-04-02T13:40:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8efeffd640 1 -- 172.21.15.33:0/93459530 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f14066050 con 0x558e2bc7e0c0 2024-04-02T13:40:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8f28da1740 10 client.25607 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8f28da1740 1 -- 172.21.15.33:0/93459530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558e2bcfd130 con 0x558e2bc7e0c0 2024-04-02T13:40:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8efeffd640 1 -- 172.21.15.33:0/93459530 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8f100054d0 con 0x558e2bc7e0c0 2024-04-02T13:40:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8efeffd640 1 -- 172.21.15.33:0/93459530 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8f1404f040 con 0x558e2bc7e0c0 2024-04-02T13:40:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8f28da1740 10 client.25607 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8f28da1740 10 client.25607 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8f28da1740 20 client.25607 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8f28da1740 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x558e2bc97190 0x558e2bcff970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8f28da1740 4 client.25607 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.991+0000 7f8f28da1740 1 -- 172.21.15.33:0/93459530 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e2bbeecf0 con 0x558e2bc97190 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.992+0000 7f8f1f7fe640 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x558e2bc97190 0x558e2bcff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.992+0000 7f8efeffd640 10 client.25607 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.992+0000 7f8f1f7fe640 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x558e2bc97190 0x558e2bcff970 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2015fae285bd112f server_cookie=a604bdac6a708f4 in_seq=0 out_seq=0 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.992+0000 7f8efeffd640 1 -- 172.21.15.33:0/93459530 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e2bbeecf0 con 0x558e2bc97190 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.992+0000 7f8efeffd640 10 client.25607 handle_command_reply: tid=0 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.993+0000 7f8f28da1740 1 client.25607 shutdown 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.993+0000 7f8f28da1740 1 -- 172.21.15.33:0/93459530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7e0c0 msgr2=0x558e2bc80390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.993+0000 7f8f28da1740 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7e0c0 0x558e2bc80390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009230 tx=0x7f8f14046040 comp rx=0 tx=0).stop 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.993+0000 7f8f28da1740 1 -- 172.21.15.33:0/93459530 shutdown_connections 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.993+0000 7f8f28da1740 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x558e2bc97190 0x558e2bcff970 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.993+0000 7f8f28da1740 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558e2bc86ca0 0x558e2bc808d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.993+0000 7f8f28da1740 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e2bc7e0c0 0x558e2bc80390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.993+0000 7f8f28da1740 1 --2- 172.21.15.33:0/93459530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e2bc7d7a0 0x558e2bc8f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.993+0000 7f8f28da1740 1 -- 172.21.15.33:0/93459530 >> 172.21.15.33:0/93459530 conn(0x558e2bc77ed0 msgr2=0x558e2bc766a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.993+0000 7f8efeffd640 0 client.25607 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:40:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.993+0000 7f8f28da1740 1 -- 172.21.15.33:0/93459530 shutdown_connections 2024-04-02T13:40:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.993+0000 7f8f28da1740 1 -- 172.21.15.33:0/93459530 wait complete. 2024-04-02T13:40:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.993+0000 7f8f28da1740 20 client.25607 trim_cache size 0 max 16384 2024-04-02T13:40:31.004 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:40:31.004 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/EDID/hex 2024-04-02T13:40:31.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.994+0000 7f8f260d9640 1 -- 172.21.15.33:0/3505497390 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8f080a3d20 msgr2=0x7f8f080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.994+0000 7f8f260d9640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8f080a3d20 0x7f8f080a61e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8f14000c00 tx=0x7f8f1402c5d0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.994+0000 7f8f260d9640 1 -- 172.21.15.33:0/3505497390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f20107b50 msgr2=0x7f8f2019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.994+0000 7f8f260d9640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f20107b50 0x7f8f2019cc20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800eb10 tx=0x7f8f1800c2d0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.994+0000 7f8f260d9640 1 -- 172.21.15.33:0/3505497390 shutdown_connections 2024-04-02T13:40:31.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.994+0000 7f8f260d9640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8f080a3d20 0x7f8f080a61e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.994+0000 7f8f260d9640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f20107b50 0x7f8f2019cc20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.994+0000 7f8f260d9640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f200696c0 0x7f8f201988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.994+0000 7f8f260d9640 1 --2- 172.21.15.33:0/3505497390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f20068d00 0x7f8f201983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.994+0000 7f8f260d9640 1 -- 172.21.15.33:0/3505497390 >> 172.21.15.33:0/3505497390 conn(0x7f8f20077b00 msgr2=0x7f8f200fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.994+0000 7f8f260d9640 1 -- 172.21.15.33:0/3505497390 shutdown_connections 2024-04-02T13:40:31.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:30.994+0000 7f8f260d9640 1 -- 172.21.15.33:0/3505497390 wait complete. 2024-04-02T13:40:31.007 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/HOWTO 2024-04-02T13:40:31.007 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/IPMI.txt 2024-04-02T13:40:31.007 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/IRQ-affinity.txt 2024-04-02T13:40:31.007 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/IRQ-domain.txt 2024-04-02T13:40:31.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:40:31.010 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/IRQ.txt 2024-04-02T13:40:31.012 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/Intel-IOMMU.txt 2024-04-02T13:40:31.014 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/Makefile 2024-04-02T13:40:31.017 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ManagementStyle 2024-04-02T13:40:31.020 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/PCI/ 2024-04-02T13:40:31.021 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/PCI/00-INDEX 2024-04-02T13:40:31.024 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/PCI/MSI-HOWTO.txt 2024-04-02T13:40:31.026 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/PCI/PCIEBUS-HOWTO.txt 2024-04-02T13:40:31.029 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/PCI/pci-error-recovery.txt 2024-04-02T13:40:31.032 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/PCI/pci-iov-howto.txt 2024-04-02T13:40:31.035 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/PCI/pci.txt 2024-04-02T13:40:31.038 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/PCI/pcieaer-howto.txt 2024-04-02T13:40:31.040 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/ 2024-04-02T13:40:31.044 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/00-INDEX 2024-04-02T13:40:31.045 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/NMI-RCU.txt 2024-04-02T13:40:31.048 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/RTFP.txt 2024-04-02T13:40:31.053 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/UP.txt 2024-04-02T13:40:31.056 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/arrayRCU.txt 2024-04-02T13:40:31.058 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/checklist.txt 2024-04-02T13:40:31.061 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/listRCU.txt 2024-04-02T13:40:31.063 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/lockdep-splat.txt 2024-04-02T13:40:31.066 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/lockdep.txt 2024-04-02T13:40:31.068 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/rcu.txt 2024-04-02T13:40:31.071 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/rcu_dereference.txt 2024-04-02T13:40:31.074 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/rcubarrier.txt 2024-04-02T13:40:31.076 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/rculist_nulls.txt 2024-04-02T13:40:31.079 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/rcuref.txt 2024-04-02T13:40:31.082 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/stallwarn.txt 2024-04-02T13:40:31.084 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/torture.txt 2024-04-02T13:40:31.088 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/trace.txt 2024-04-02T13:40:31.090 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/RCU/whatisRCU.txt 2024-04-02T13:40:31.094 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/SAK.txt 2024-04-02T13:40:31.097 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/SM501.txt 2024-04-02T13:40:31.099 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/SecurityBugs 2024-04-02T13:40:31.102 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/SubmitChecklist 2024-04-02T13:40:31.104 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/SubmittingDrivers 2024-04-02T13:40:31.107 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/SubmittingPatches 2024-04-02T13:40:31.110 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/VGA-softcursor.txt 2024-04-02T13:40:31.113 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/accounting/ 2024-04-02T13:40:31.115 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/accounting/.gitignore 2024-04-02T13:40:31.117 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/accounting/Makefile 2024-04-02T13:40:31.119 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/accounting/cgroupstats.txt 2024-04-02T13:40:31.121 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/accounting/delay-accounting.txt 2024-04-02T13:40:31.124 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/accounting/getdelays.c 2024-04-02T13:40:31.126 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/accounting/taskstats-struct.txt 2024-04-02T13:40:31.129 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/accounting/taskstats.txt 2024-04-02T13:40:31.132 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/ 2024-04-02T13:40:31.134 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/apei/ 2024-04-02T13:40:31.135 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/apei/einj.txt 2024-04-02T13:40:31.138 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/apei/output_format.txt 2024-04-02T13:40:31.140 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/debug.txt 2024-04-02T13:40:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.139+0000 7f47cee82640 1 -- 172.21.15.33:0/1723249756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c8102460 msgr2=0x7f47c810e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.139+0000 7f47cee82640 1 --2- 172.21.15.33:0/1723249756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c8102460 0x7f47c810e970 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009cd0 tx=0x7f47c002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.139+0000 7f47cee82640 1 -- 172.21.15.33:0/1723249756 shutdown_connections 2024-04-02T13:40:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.139+0000 7f47cee82640 1 --2- 172.21.15.33:0/1723249756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c8102460 0x7f47c810e970 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.139+0000 7f47cee82640 1 --2- 172.21.15.33:0/1723249756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c8101aa0 0x7f47c8101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.139+0000 7f47cee82640 1 --2- 172.21.15.33:0/1723249756 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47c8107ab0 0x7f47c8107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.139+0000 7f47cee82640 1 -- 172.21.15.33:0/1723249756 >> 172.21.15.33:0/1723249756 conn(0x7f47c80fd530 msgr2=0x7f47c80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.139+0000 7f47cee82640 1 -- 172.21.15.33:0/1723249756 shutdown_connections 2024-04-02T13:40:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.140+0000 7f47cee82640 1 -- 172.21.15.33:0/1723249756 wait complete. 2024-04-02T13:40:31.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.140+0000 7f47cee82640 1 Processor -- start 2024-04-02T13:40:31.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.140+0000 7f47cee82640 1 -- start start 2024-04-02T13:40:31.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47cee82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c8101aa0 0x7f47c819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47cee82640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47c8102460 0x7f47c819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47cee82640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c8107ab0 0x7f47c819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47cee82640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8111e90 con 0x7f47c8101aa0 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47cee82640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8111d10 con 0x7f47c8107ab0 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47cee82640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8112010 con 0x7f47c8102460 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47ccc1e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c8101aa0 0x7f47c819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47ccc1e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c8101aa0 0x7f47c819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47160/0 (socket says 172.21.15.33:47160) 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47ccc1e640 1 -- 172.21.15.33:0/1178623151 learned_addr learned my addr 172.21.15.33:0/1178623151 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47cd41f640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c8107ab0 0x7f47c819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47c7fff640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47c8102460 0x7f47c819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47ccc1e640 1 -- 172.21.15.33:0/1178623151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47c8102460 msgr2=0x7f47c819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47c8102460 0x7f47c819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47ccc1e640 1 -- 172.21.15.33:0/1178623151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c8107ab0 msgr2=0x7f47c819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c8107ab0 0x7f47c819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47ccc1e640 1 -- 172.21.15.33:0/1178623151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c819f580 con 0x7f47c8101aa0 2024-04-02T13:40:31.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47cd41f640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c8107ab0 0x7f47c819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:31.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c8101aa0 0x7f47c819a590 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00e9f0 tx=0x7f47bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:31.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47c5ffb640 1 -- 172.21.15.33:0/1178623151 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00cd80 con 0x7f47c8101aa0 2024-04-02T13:40:31.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.141+0000 7f47c5ffb640 1 -- 172.21.15.33:0/1178623151 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f47bc0054f0 con 0x7f47c8101aa0 2024-04-02T13:40:31.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.142+0000 7f47cee82640 1 -- 172.21.15.33:0/1178623151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c819f870 con 0x7f47c8101aa0 2024-04-02T13:40:31.146 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/dsdt-override.txt 2024-04-02T13:40:31.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.142+0000 7f47c5ffb640 1 -- 172.21.15.33:0/1178623151 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc010690 con 0x7f47c8101aa0 2024-04-02T13:40:31.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.142+0000 7f47cee82640 1 -- 172.21.15.33:0/1178623151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c81a7100 con 0x7f47c8101aa0 2024-04-02T13:40:31.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.144+0000 7f47c5ffb640 1 -- 172.21.15.33:0/1178623151 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f47bc004f20 con 0x7f47c8101aa0 2024-04-02T13:40:31.147 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/enumeration.txt 2024-04-02T13:40:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.145+0000 7f47c5ffb640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f47b80a3d20 0x7f47b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.145+0000 7f47d1a60740 1 Processor -- start 2024-04-02T13:40:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.145+0000 7f47c7fff640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f47b80a3d20 0x7f47b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.145+0000 7f47d1a60740 1 -- start start 2024-04-02T13:40:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.145+0000 7f47c5ffb640 1 -- 172.21.15.33:0/1178623151 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f47bc0d0f10 con 0x7f47c8101aa0 2024-04-02T13:40:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47d1a60740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323a0180 0x564f323a0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47d1a60740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0b30 0x564f323b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47d1a60740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323b24b0 0x564f323b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47d1a60740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564f323047b0 con 0x564f323a0b30 2024-04-02T13:40:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47d1a60740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564f322a3570 con 0x564f323b24b0 2024-04-02T13:40:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47d1a60740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564f32300760 con 0x564f323a0180 2024-04-02T13:40:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47ccc1e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323a0180 0x564f323a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47ccc1e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323a0180 0x564f323a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:56318/0 (socket says 172.21.15.33:56318) 2024-04-02T13:40:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47ccc1e640 1 -- 172.21.15.33:0/3539103051 learned_addr learned my addr 172.21.15.33:0/3539103051 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47cd41f640 1 --2- 172.21.15.33:0/3539103051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323b24b0 0x564f323b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47c7fff640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f47b80a3d20 0x7f47b80a61e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f47c810dd40 tx=0x7f47b4005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47c7fff640 1 --2- 172.21.15.33:0/3539103051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0b30 0x564f323b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47ccc1e640 1 -- 172.21.15.33:0/3539103051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323b24b0 msgr2=0x564f323b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/3539103051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323b24b0 0x564f323b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47ccc1e640 1 -- 172.21.15.33:0/3539103051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0b30 msgr2=0x564f323b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/3539103051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0b30 0x564f323b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47ccc1e640 1 -- 172.21.15.33:0/3539103051 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f320183f0 con 0x564f323a0180 2024-04-02T13:40:31.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.146+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/3539103051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323a0180 0x564f323a0560 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47bc099510 tx=0x7f47bc09d240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6478a5b647adb7e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:31.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.147+0000 7f47b2ffd640 1 -- 172.21.15.33:0/3539103051 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0b6070 con 0x564f323a0180 2024-04-02T13:40:31.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.147+0000 7f47b2ffd640 1 -- 172.21.15.33:0/3539103051 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f47bc0a57b0 con 0x564f323a0180 2024-04-02T13:40:31.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.147+0000 7f47b2ffd640 1 -- 172.21.15.33:0/3539103051 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0991b0 con 0x564f323a0180 2024-04-02T13:40:31.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.147+0000 7f47d1a60740 1 -- 172.21.15.33:0/3539103051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323a0180 msgr2=0x564f323a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.148+0000 7f47d1a60740 1 --2- 172.21.15.33:0/3539103051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323a0180 0x564f323a0560 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47bc099510 tx=0x7f47bc09d240 comp rx=0 tx=0).stop 2024-04-02T13:40:31.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.148+0000 7f47d1a60740 1 -- 172.21.15.33:0/3539103051 shutdown_connections 2024-04-02T13:40:31.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.148+0000 7f47d1a60740 1 --2- 172.21.15.33:0/3539103051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323b24b0 0x564f323b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.148+0000 7f47d1a60740 1 --2- 172.21.15.33:0/3539103051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0b30 0x564f323b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.148+0000 7f47d1a60740 1 --2- 172.21.15.33:0/3539103051 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323a0180 0x564f323a0560 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.148+0000 7f47d1a60740 1 -- 172.21.15.33:0/3539103051 >> 172.21.15.33:0/3539103051 conn(0x564f3239aed0 msgr2=0x564f3239b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.148+0000 7f47d1a60740 1 -- 172.21.15.33:0/3539103051 shutdown_connections 2024-04-02T13:40:31.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.148+0000 7f47d1a60740 1 -- 172.21.15.33:0/3539103051 wait complete. 2024-04-02T13:40:31.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.148+0000 7f47d1a60740 1 -- start start 2024-04-02T13:40:31.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.148+0000 7f47d1a60740 1 Processor -- start 2024-04-02T13:40:31.153 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/gpio-properties.txt 2024-04-02T13:40:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.148+0000 7f47d1a60740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0180 0x564f323af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.148+0000 7f47d1a60740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0b30 0x564f323af860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47d1a60740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 0x564f323afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47d1a60740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564f323b52c0 con 0x564f323a0b30 2024-04-02T13:40:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47d1a60740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564f322bfe80 con 0x564f323a0180 2024-04-02T13:40:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47d1a60740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564f323b5440 con 0x564f323b24b0 2024-04-02T13:40:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0b30 0x564f323af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47ccc1e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0180 0x564f323af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0b30 0x564f323af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47170/0 (socket says 172.21.15.33:47170) 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47ccc1e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0180 0x564f323af320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:59710/0 (socket says 172.21.15.33:59710) 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47c7fff640 1 -- 172.21.15.33:0/3861605708 learned_addr learned my addr 172.21.15.33:0/3861605708 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47d1a60740 10 client.0 _ll_register_callbacks cb 0x564f322fbaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47cd41f640 1 --2- 172.21.15.33:0/3861605708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 0x564f323afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47c7fff640 1 -- 172.21.15.33:0/3861605708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 msgr2=0x564f323afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47c7fff640 1 --2- 172.21.15.33:0/3861605708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 0x564f323afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47c7fff640 1 -- 172.21.15.33:0/3861605708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0180 msgr2=0x564f323af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47c7fff640 1 --2- 172.21.15.33:0/3861605708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0180 0x564f323af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47c7fff640 1 -- 172.21.15.33:0/3861605708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f320e27d0 con 0x564f323a0b30 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/3861605708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0180 0x564f323af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47c7fff640 1 --2- 172.21.15.33:0/3861605708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0b30 0x564f323af860 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f47b4003970 tx=0x7f47b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8829a000a8ead8db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:40:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47a7fff640 1 -- 172.21.15.33:0/3861605708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b400e740 con 0x564f323a0b30 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47a7fff640 1 -- 172.21.15.33:0/3861605708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f47b400e8e0 con 0x564f323a0b30 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.149+0000 7f47d1a60740 1 -- 172.21.15.33:0/3861605708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47bc0a57b0 con 0x564f323a0b30 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.150+0000 7f47a7fff640 1 -- 172.21.15.33:0/3861605708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f47b400eae0 con 0x564f323a0b30 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.150+0000 7f47d1a60740 10 client.16014 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.150+0000 7f47d1a60740 1 -- 172.21.15.33:0/3861605708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564f323c1430 con 0x564f323a0b30 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.150+0000 7f47a7fff640 1 -- 172.21.15.33:0/3861605708 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f47bc0a57b0 con 0x564f323a0b30 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.150+0000 7f47a7fff640 1 -- 172.21.15.33:0/3861605708 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f47b401d410 con 0x564f323a0b30 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.150+0000 7f47d1a60740 10 client.16014 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.150+0000 7f47d1a60740 10 client.16014 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.150+0000 7f47d1a60740 20 client.16014 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.150+0000 7f47d1a60740 1 --2- 172.21.15.33:0/3861605708 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564f323c28f0 0x564f323c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.150+0000 7f47d1a60740 4 client.16014 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.150+0000 7f47d1a60740 1 -- 172.21.15.33:0/3861605708 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f32311cf0 con 0x564f323c28f0 2024-04-02T13:40:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.150+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/3861605708 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564f323c28f0 0x564f323c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.158 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/initrd_table_override.txt 2024-04-02T13:40:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.151+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/3861605708 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564f323c28f0 0x564f323c4d40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ac74096d3a65e2e server_cookie=d75d3bb8f6865be0 in_seq=0 out_seq=0 2024-04-02T13:40:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.151+0000 7f47a7fff640 10 client.16014 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:40:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.151+0000 7f47a7fff640 1 -- 172.21.15.33:0/3861605708 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f32311cf0 con 0x564f323c28f0 2024-04-02T13:40:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.151+0000 7f47a7fff640 10 client.16014 handle_command_reply: tid=0 2024-04-02T13:40:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.151+0000 7f47d1a60740 1 client.16014 shutdown 2024-04-02T13:40:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.152+0000 7f47d1a60740 1 -- 172.21.15.33:0/3861605708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0b30 msgr2=0x564f323af860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.152+0000 7f47d1a60740 1 --2- 172.21.15.33:0/3861605708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0b30 0x564f323af860 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f47b4003970 tx=0x7f47b4004730 comp rx=0 tx=0).stop 2024-04-02T13:40:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.152+0000 7f47d1a60740 1 -- 172.21.15.33:0/3861605708 shutdown_connections 2024-04-02T13:40:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.152+0000 7f47d1a60740 1 --2- 172.21.15.33:0/3861605708 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564f323c28f0 0x564f323c4d40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.152+0000 7f47d1a60740 1 --2- 172.21.15.33:0/3861605708 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 0x564f323afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.152+0000 7f47d1a60740 1 --2- 172.21.15.33:0/3861605708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0b30 0x564f323af860 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.152+0000 7f47d1a60740 1 --2- 172.21.15.33:0/3861605708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0180 0x564f323af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.152+0000 7f47d1a60740 1 -- 172.21.15.33:0/3861605708 >> 172.21.15.33:0/3861605708 conn(0x564f3239aed0 msgr2=0x564f3239cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.152+0000 7f47a7fff640 0 client.16014 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:40:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.152+0000 7f47d1a60740 1 -- 172.21.15.33:0/3861605708 shutdown_connections 2024-04-02T13:40:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.152+0000 7f47d1a60740 1 -- 172.21.15.33:0/3861605708 wait complete. 2024-04-02T13:40:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.152+0000 7f47d1a60740 20 client.16014 trim_cache size 0 max 16384 2024-04-02T13:40:31.160 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/method-customizing.txt 2024-04-02T13:40:31.160 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/method-tracing.txt 2024-04-02T13:40:31.161 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/namespace.txt 2024-04-02T13:40:31.163 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/scan_handlers.txt 2024-04-02T13:40:31.166 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/acpi/video_extension.txt 2024-04-02T13:40:31.169 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/aoe/ 2024-04-02T13:40:31.171 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/aoe/aoe.txt 2024-04-02T13:40:31.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.171+0000 7f47d1a60740 1 Processor -- start 2024-04-02T13:40:31.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.171+0000 7f47d1a60740 1 -- start start 2024-04-02T13:40:31.174 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/aoe/autoload.sh 2024-04-02T13:40:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.171+0000 7f47d1a60740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 0x564f323b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.171+0000 7f47d1a60740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0b30 0x564f323be1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.171+0000 7f47d1a60740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 0x564f323ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.171+0000 7f47d1a60740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81af6a0 con 0x564f323a0180 2024-04-02T13:40:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.171+0000 7f47d1a60740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81acf70 con 0x564f323a0b30 2024-04-02T13:40:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.171+0000 7f47d1a60740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81aae00 con 0x564f323b24b0 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47ccc1e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 0x564f323b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47ccc1e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 0x564f323b1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47172/0 (socket says 172.21.15.33:47172) 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47ccc1e640 1 -- 172.21.15.33:0/2348991155 learned_addr learned my addr 172.21.15.33:0/2348991155 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47cd41f640 1 --2- 172.21.15.33:0/2348991155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 0x564f323ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47c7fff640 1 --2- 172.21.15.33:0/2348991155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0b30 0x564f323be1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47ccc1e640 1 -- 172.21.15.33:0/2348991155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 msgr2=0x564f323ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/2348991155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 0x564f323ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47ccc1e640 1 -- 172.21.15.33:0/2348991155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0b30 msgr2=0x564f323be1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/2348991155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0b30 0x564f323be1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47ccc1e640 1 -- 172.21.15.33:0/2348991155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f3239f890 con 0x564f323a0180 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47cd41f640 1 --2- 172.21.15.33:0/2348991155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 0x564f323ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/2348991155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 0x564f323b1a50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f47bc09c840 tx=0x7f47bc09ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79659ed017d0f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47b27fc640 1 -- 172.21.15.33:0/2348991155 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0b6040 con 0x564f323a0180 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47b27fc640 1 -- 172.21.15.33:0/2348991155 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f47bc0a57b0 con 0x564f323a0180 2024-04-02T13:40:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.172+0000 7f47b27fc640 1 -- 172.21.15.33:0/2348991155 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0a5320 con 0x564f323a0180 2024-04-02T13:40:31.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.173+0000 7f47d1a60740 1 -- 172.21.15.33:0/2348991155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 msgr2=0x564f323b1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.173+0000 7f47d1a60740 1 --2- 172.21.15.33:0/2348991155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 0x564f323b1a50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f47bc09c840 tx=0x7f47bc09ca40 comp rx=0 tx=0).stop 2024-04-02T13:40:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.173+0000 7f47d1a60740 1 -- 172.21.15.33:0/2348991155 shutdown_connections 2024-04-02T13:40:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.173+0000 7f47d1a60740 1 --2- 172.21.15.33:0/2348991155 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 0x564f323ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.173+0000 7f47d1a60740 1 --2- 172.21.15.33:0/2348991155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0b30 0x564f323be1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.173+0000 7f47d1a60740 1 --2- 172.21.15.33:0/2348991155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 0x564f323b1a50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.173+0000 7f47d1a60740 1 -- 172.21.15.33:0/2348991155 >> 172.21.15.33:0/2348991155 conn(0x564f3239aed0 msgr2=0x564f323adbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.173+0000 7f47d1a60740 1 -- 172.21.15.33:0/2348991155 shutdown_connections 2024-04-02T13:40:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.173+0000 7f47d1a60740 1 -- 172.21.15.33:0/2348991155 wait complete. 2024-04-02T13:40:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.174+0000 7f47d1a60740 1 -- start start 2024-04-02T13:40:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.174+0000 7f47d1a60740 1 Processor -- start 2024-04-02T13:40:31.179 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/aoe/status.sh 2024-04-02T13:40:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.174+0000 7f47d1a60740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 0x564f323ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.174+0000 7f47d1a60740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0b30 0x564f323acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.174+0000 7f47d1a60740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 0x564f323ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.174+0000 7f47d1a60740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81ab450 con 0x564f323a0180 2024-04-02T13:40:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.174+0000 7f47d1a60740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81ac8a0 con 0x564f323a0b30 2024-04-02T13:40:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.174+0000 7f47d1a60740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81abb00 con 0x564f323b24b0 2024-04-02T13:40:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.174+0000 7f47d1a60740 10 client.0 _ll_register_callbacks cb 0x564f323b2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.174+0000 7f47ccc1e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 0x564f323ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.174+0000 7f47ccc1e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 0x564f323ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47184/0 (socket says 172.21.15.33:47184) 2024-04-02T13:40:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.174+0000 7f47ccc1e640 1 -- 172.21.15.33:0/2663400002 learned_addr learned my addr 172.21.15.33:0/2663400002 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.174+0000 7f47c7fff640 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0b30 0x564f323acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47cd41f640 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 0x564f323ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47ccc1e640 1 -- 172.21.15.33:0/2663400002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 msgr2=0x564f323ab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 0x564f323ab590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47ccc1e640 1 -- 172.21.15.33:0/2663400002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0b30 msgr2=0x564f323acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0b30 0x564f323acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47ccc1e640 1 -- 172.21.15.33:0/2663400002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f323badf0 con 0x564f323a0180 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47c7fff640 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0b30 0x564f323acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47ccc1e640 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 0x564f323ac890 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f47bc09adf0 tx=0x7f47bc09d240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4180f8efeee05f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47a7fff640 1 -- 172.21.15.33:0/2663400002 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0b6040 con 0x564f323a0180 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47a7fff640 1 -- 172.21.15.33:0/2663400002 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f47bc005210 con 0x564f323a0180 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47d1a60740 1 -- 172.21.15.33:0/2663400002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47bc0a57b0 con 0x564f323a0180 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47a7fff640 1 -- 172.21.15.33:0/2663400002 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f47bc0b0050 con 0x564f323a0180 2024-04-02T13:40:31.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47d1a60740 10 client.16020 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.175+0000 7f47d1a60740 1 -- 172.21.15.33:0/2663400002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564f32428320 con 0x564f323a0180 2024-04-02T13:40:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.176+0000 7f47a7fff640 1 -- 172.21.15.33:0/2663400002 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f47bc0a57b0 con 0x564f323a0180 2024-04-02T13:40:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.176+0000 7f47a7fff640 1 -- 172.21.15.33:0/2663400002 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f47bc0a53b0 con 0x564f323a0180 2024-04-02T13:40:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.176+0000 7f47d1a60740 10 client.16020 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.176+0000 7f47d1a60740 10 client.16020 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:40:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.176+0000 7f47d1a60740 20 client.16020 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.176+0000 7f47d1a60740 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564f32428540 0x564f3242a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.176+0000 7f47d1a60740 4 client.16020 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:40:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.176+0000 7f47d1a60740 1 -- 172.21.15.33:0/2663400002 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f3230c040 con 0x564f32428540 2024-04-02T13:40:31.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.176+0000 7f47c7fff640 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564f32428540 0x564f3242a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.183 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/aoe/todo.txt 2024-04-02T13:40:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47c7fff640 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564f32428540 0x564f3242a930 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df9045d4f373a461 server_cookie=9b842fb198a192d in_seq=0 out_seq=0 2024-04-02T13:40:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47a7fff640 10 client.16020 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:40:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47a7fff640 1 -- 172.21.15.33:0/2663400002 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f3230c040 con 0x564f32428540 2024-04-02T13:40:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47a7fff640 10 client.16020 handle_command_reply: tid=0 2024-04-02T13:40:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47d1a60740 1 client.16020 shutdown 2024-04-02T13:40:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47d1a60740 1 -- 172.21.15.33:0/2663400002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 msgr2=0x564f323ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47d1a60740 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 0x564f323ac890 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f47bc09adf0 tx=0x7f47bc09d240 comp rx=0 tx=0).stop 2024-04-02T13:40:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47d1a60740 1 -- 172.21.15.33:0/2663400002 shutdown_connections 2024-04-02T13:40:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47d1a60740 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564f32428540 0x564f3242a930 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47d1a60740 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564f323b24b0 0x564f323ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47d1a60740 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564f323a0b30 0x564f323acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47d1a60740 1 --2- 172.21.15.33:0/2663400002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f323a0180 0x564f323ac890 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47a7fff640 0 client.16020 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:40:31.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47d1a60740 1 -- 172.21.15.33:0/2663400002 >> 172.21.15.33:0/2663400002 conn(0x564f3239aed0 msgr2=0x564f323996a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47d1a60740 1 -- 172.21.15.33:0/2663400002 shutdown_connections 2024-04-02T13:40:31.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47d1a60740 1 -- 172.21.15.33:0/2663400002 wait complete. 2024-04-02T13:40:31.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.177+0000 7f47d1a60740 20 client.16020 trim_cache size 0 max 16384 2024-04-02T13:40:31.186 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:40:31.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.178+0000 7f47cee82640 1 -- 172.21.15.33:0/1178623151 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f47b80a3d20 msgr2=0x7f47b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.178+0000 7f47cee82640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f47b80a3d20 0x7f47b80a61e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f47c810dd40 tx=0x7f47b4005f90 comp rx=0 tx=0).stop 2024-04-02T13:40:31.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.178+0000 7f47cee82640 1 -- 172.21.15.33:0/1178623151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c8101aa0 msgr2=0x7f47c819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.178+0000 7f47cee82640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c8101aa0 0x7f47c819a590 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00e9f0 tx=0x7f47bc00edf0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.178+0000 7f47cee82640 1 -- 172.21.15.33:0/1178623151 shutdown_connections 2024-04-02T13:40:31.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.178+0000 7f47cee82640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f47b80a3d20 0x7f47b80a61e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.178+0000 7f47cee82640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c8107ab0 0x7f47c819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.178+0000 7f47cee82640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f47c8102460 0x7f47c819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.178+0000 7f47cee82640 1 --2- 172.21.15.33:0/1178623151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47c8101aa0 0x7f47c819a590 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.178+0000 7f47cee82640 1 -- 172.21.15.33:0/1178623151 >> 172.21.15.33:0/1178623151 conn(0x7f47c80fd530 msgr2=0x7f47c8106d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.178+0000 7f47cee82640 1 -- 172.21.15.33:0/1178623151 shutdown_connections 2024-04-02T13:40:31.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.178+0000 7f47cee82640 1 -- 172.21.15.33:0/1178623151 wait complete. 2024-04-02T13:40:31.188 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/aoe/udev-install.sh 2024-04-02T13:40:31.188 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/aoe/udev.txt 2024-04-02T13:40:31.189 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/applying-patches.txt 2024-04-02T13:40:31.189 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/ 2024-04-02T13:40:31.191 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/00-INDEX 2024-04-02T13:40:31.193 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:40:31.194 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Atmel/ 2024-04-02T13:40:31.195 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Atmel/README 2024-04-02T13:40:31.197 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Booting 2024-04-02T13:40:31.200 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/CCN.txt 2024-04-02T13:40:31.206 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/IXP4xx 2024-04-02T13:40:31.206 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Interrupts 2024-04-02T13:40:31.208 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Makefile 2024-04-02T13:40:31.211 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Marvell/ 2024-04-02T13:40:31.211 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Marvell/README 2024-04-02T13:40:31.214 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Netwinder 2024-04-02T13:40:31.217 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/OMAP/ 2024-04-02T13:40:31.219 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/OMAP/DSS 2024-04-02T13:40:31.221 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/OMAP/omap_pm 2024-04-02T13:40:31.224 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Porting 2024-04-02T13:40:31.227 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/README 2024-04-02T13:40:31.230 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/ 2024-04-02T13:40:31.232 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/ADSBitsy 2024-04-02T13:40:31.234 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/Assabet 2024-04-02T13:40:31.237 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/Brutus 2024-04-02T13:40:31.239 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/CERF 2024-04-02T13:40:31.242 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/FreeBird 2024-04-02T13:40:31.244 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsClient 2024-04-02T13:40:31.246 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsMaster 2024-04-02T13:40:31.249 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/HUW_WEBPANEL 2024-04-02T13:40:31.251 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/Itsy 2024-04-02T13:40:31.253 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/LART 2024-04-02T13:40:31.256 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/PLEB 2024-04-02T13:40:31.258 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/Pangolin 2024-04-02T13:40:31.261 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/Tifon 2024-04-02T13:40:31.263 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/Victor 2024-04-02T13:40:31.266 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/Yopy 2024-04-02T13:40:31.269 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/empeg 2024-04-02T13:40:31.272 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/nanoEngine 2024-04-02T13:40:31.274 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SA1100/serial_UART 2024-04-02T13:40:31.277 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/ 2024-04-02T13:40:31.279 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/.gitignore 2024-04-02T13:40:31.281 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/Makefile 2024-04-02T13:40:31.284 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/vrl4.c 2024-04-02T13:40:31.286 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-mmcif.txt 2024-04-02T13:40:31.288 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-sdhi.txt 2024-04-02T13:40:31.291 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SPEAr/ 2024-04-02T13:40:31.293 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/SPEAr/overview.txt 2024-04-02T13:40:31.295 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/ 2024-04-02T13:40:31.298 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/CPUfreq.txt 2024-04-02T13:40:31.300 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/EB2410ITX.txt 2024-04-02T13:40:31.302 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/GPIO.txt 2024-04-02T13:40:31.305 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/H1940.txt 2024-04-02T13:40:31.308 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/NAND.txt 2024-04-02T13:40:31.310 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Overview.txt 2024-04-02T13:40:31.312 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2412.txt 2024-04-02T13:40:31.315 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2413.txt 2024-04-02T13:40:31.317 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/SMDK2440.txt 2024-04-02T13:40:31.320 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Suspend.txt 2024-04-02T13:40:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.319+0000 7fe92e655640 1 -- 172.21.15.33:0/4159994506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe928100970 msgr2=0x7fe928100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.319+0000 7fe92e655640 1 --2- 172.21.15.33:0/4159994506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe928100970 0x7fe928100df0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009930 tx=0x7fe91c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:31.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.319+0000 7fe92e655640 1 -- 172.21.15.33:0/4159994506 shutdown_connections 2024-04-02T13:40:31.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.319+0000 7fe92e655640 1 --2- 172.21.15.33:0/4159994506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe928101330 0x7fe92810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.319+0000 7fe92e655640 1 --2- 172.21.15.33:0/4159994506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe928100970 0x7fe928100df0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.319+0000 7fe92e655640 1 --2- 172.21.15.33:0/4159994506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe928106980 0x7fe928106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.319+0000 7fe92e655640 1 -- 172.21.15.33:0/4159994506 >> 172.21.15.33:0/4159994506 conn(0x7fe928077b00 msgr2=0x7fe928077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.319+0000 7fe92e655640 1 -- 172.21.15.33:0/4159994506 shutdown_connections 2024-04-02T13:40:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.319+0000 7fe92e655640 1 -- 172.21.15.33:0/4159994506 wait complete. 2024-04-02T13:40:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.319+0000 7fe92e655640 1 Processor -- start 2024-04-02T13:40:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe92e655640 1 -- start start 2024-04-02T13:40:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe92e655640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe928100970 0x7fe92819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe92e655640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe928101330 0x7fe92819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe92e655640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe928106980 0x7fe92819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe92e655640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe928110d30 con 0x7fe928100970 2024-04-02T13:40:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe92e655640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe928110bb0 con 0x7fe928101330 2024-04-02T13:40:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe92e655640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe928110eb0 con 0x7fe928106980 2024-04-02T13:40:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe927fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe928100970 0x7fe92819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe9277fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe928101330 0x7fe92819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe92cbf2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe928106980 0x7fe92819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe9277fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe928101330 0x7fe92819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:59766/0 (socket says 172.21.15.33:59766) 2024-04-02T13:40:31.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe9277fe640 1 -- 172.21.15.33:0/4275329650 learned_addr learned my addr 172.21.15.33:0/4275329650 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:31.328 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/USB-Host.txt 2024-04-02T13:40:31.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe9277fe640 1 -- 172.21.15.33:0/4275329650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe928106980 msgr2=0x7fe92819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe9277fe640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe928106980 0x7fe92819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe9277fe640 1 -- 172.21.15.33:0/4275329650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe928100970 msgr2=0x7fe92819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe9277fe640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe928100970 0x7fe92819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe9277fe640 1 -- 172.21.15.33:0/4275329650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92819f580 con 0x7fe928101330 2024-04-02T13:40:31.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe92cbf2640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe928106980 0x7fe92819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:31.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.320+0000 7fe927fff640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe928100970 0x7fe92819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:31.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.321+0000 7fe9277fe640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe928101330 0x7fe92819aad0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe91c000c00 tx=0x7fe91c004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:31.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.321+0000 7fe9257fa640 1 -- 172.21.15.33:0/4275329650 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c043020 con 0x7fe928101330 2024-04-02T13:40:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.321+0000 7fe9257fa640 1 -- 172.21.15.33:0/4275329650 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe91c02c5d0 con 0x7fe928101330 2024-04-02T13:40:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.321+0000 7fe92e655640 1 -- 172.21.15.33:0/4275329650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92819f810 con 0x7fe928101330 2024-04-02T13:40:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.321+0000 7fe92e655640 1 -- 172.21.15.33:0/4275329650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9281a7100 con 0x7fe928101330 2024-04-02T13:40:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.321+0000 7fe9257fa640 1 -- 172.21.15.33:0/4275329650 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c003970 con 0x7fe928101330 2024-04-02T13:40:31.333 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung/ 2024-04-02T13:40:31.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.324+0000 7fe9257fa640 1 -- 172.21.15.33:0/4275329650 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe91c046020 con 0x7fe928101330 2024-04-02T13:40:31.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.324+0000 7fe93123c740 1 Processor -- start 2024-04-02T13:40:31.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.324+0000 7fe93123c740 1 -- start start 2024-04-02T13:40:31.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.324+0000 7fe93123c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601db7a0 0x563f601dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.324+0000 7fe93123c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601dc0c0 0x563f601e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.324+0000 7fe93123c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601e4ca0 0x563f601e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.324+0000 7fe93123c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563f6013f7b0 con 0x563f601db7a0 2024-04-02T13:40:31.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.324+0000 7fe92cbf2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601db7a0 0x563f601dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe927fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601dc0c0 0x563f601e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.324+0000 7fe93123c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563f600de570 con 0x563f601e4ca0 2024-04-02T13:40:31.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe93123c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563f6013b760 con 0x563f601dc0c0 2024-04-02T13:40:31.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe92cbf2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601db7a0 0x563f601dbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47222/0 (socket says 172.21.15.33:47222) 2024-04-02T13:40:31.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe92cbf2640 1 -- 172.21.15.33:0/4225776165 learned_addr learned my addr 172.21.15.33:0/4225776165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:31.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe9257fa640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe9100a3d20 0x7fe9100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe92cbf2640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe9100a3d20 0x7fe9100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe9277fe640 1 --2- 172.21.15.33:0/4225776165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601e4ca0 0x563f601e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe927fff640 1 -- 172.21.15.33:0/4225776165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601e4ca0 msgr2=0x563f601e7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe927fff640 1 --2- 172.21.15.33:0/4225776165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601e4ca0 0x563f601e7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe927fff640 1 -- 172.21.15.33:0/4225776165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601db7a0 msgr2=0x563f601dbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe927fff640 1 --2- 172.21.15.33:0/4225776165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601db7a0 0x563f601dbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe927fff640 1 -- 172.21.15.33:0/4225776165 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f5fe533f0 con 0x563f601dc0c0 2024-04-02T13:40:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe9277fe640 1 --2- 172.21.15.33:0/4225776165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601e4ca0 0x563f601e7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe9257fa640 1 -- 172.21.15.33:0/4275329650 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe91c0f2250 con 0x7fe928101330 2024-04-02T13:40:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe927fff640 1 --2- 172.21.15.33:0/4225776165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601dc0c0 0x563f601e4760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe918009f50 tx=0x7fe9180118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55062dd9bd412cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe92cbf2640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe9100a3d20 0x7fe9100a61e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe920005c40 tx=0x7fe920004ab0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe90e7fc640 1 -- 172.21.15.33:0/4225776165 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918004030 con 0x563f601dc0c0 2024-04-02T13:40:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe90e7fc640 1 -- 172.21.15.33:0/4225776165 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe9180054f0 con 0x563f601dc0c0 2024-04-02T13:40:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.325+0000 7fe90e7fc640 1 -- 172.21.15.33:0/4225776165 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918004d70 con 0x563f601dc0c0 2024-04-02T13:40:31.339 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung/GPIO.txt 2024-04-02T13:40:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.326+0000 7fe93123c740 1 -- 172.21.15.33:0/4225776165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601dc0c0 msgr2=0x563f601e4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.326+0000 7fe93123c740 1 --2- 172.21.15.33:0/4225776165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601dc0c0 0x563f601e4760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe918009f50 tx=0x7fe9180118d0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.326+0000 7fe93123c740 1 -- 172.21.15.33:0/4225776165 shutdown_connections 2024-04-02T13:40:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.326+0000 7fe93123c740 1 --2- 172.21.15.33:0/4225776165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601e4ca0 0x563f601e7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.326+0000 7fe93123c740 1 --2- 172.21.15.33:0/4225776165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601dc0c0 0x563f601e4760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.326+0000 7fe93123c740 1 --2- 172.21.15.33:0/4225776165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601db7a0 0x563f601dbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.326+0000 7fe93123c740 1 -- 172.21.15.33:0/4225776165 >> 172.21.15.33:0/4225776165 conn(0x563f601d5ed0 msgr2=0x563f601d62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.326+0000 7fe93123c740 1 -- 172.21.15.33:0/4225776165 shutdown_connections 2024-04-02T13:40:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.326+0000 7fe93123c740 1 -- 172.21.15.33:0/4225776165 wait complete. 2024-04-02T13:40:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe93123c740 1 -- start start 2024-04-02T13:40:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe93123c740 1 Processor -- start 2024-04-02T13:40:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe93123c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601db7a0 0x563f601e45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe93123c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601dc0c0 0x563f601e1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe93123c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601e4ca0 0x563f601e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe93123c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563f601e7a90 con 0x563f601dc0c0 2024-04-02T13:40:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe93123c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563f600fae80 con 0x563f601e4ca0 2024-04-02T13:40:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe93123c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563f601e7c10 con 0x563f601db7a0 2024-04-02T13:40:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe93123c740 10 client.0 _ll_register_callbacks cb 0x563f60136aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe9277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601dc0c0 0x563f601e1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe9277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601dc0c0 0x563f601e1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47234/0 (socket says 172.21.15.33:47234) 2024-04-02T13:40:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe9277fe640 1 -- 172.21.15.33:0/2053067663 learned_addr learned my addr 172.21.15.33:0/2053067663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe92cbf2640 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601e4ca0 0x563f601e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.327+0000 7fe927fff640 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601db7a0 0x563f601e45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe92cbf2640 1 -- 172.21.15.33:0/2053067663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601db7a0 msgr2=0x563f601e45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe92cbf2640 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601db7a0 0x563f601e45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe92cbf2640 1 -- 172.21.15.33:0/2053067663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601dc0c0 msgr2=0x563f601e1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe92cbf2640 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601dc0c0 0x563f601e1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe92cbf2640 1 -- 172.21.15.33:0/2053067663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f5ff1d7d0 con 0x563f601e4ca0 2024-04-02T13:40:31.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe9277fe640 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601dc0c0 0x563f601e1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:31.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe92cbf2640 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601e4ca0 0x563f601e1e90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe920009fd0 tx=0x7fe920004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc0594a4d4b0c876 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:31.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe9077fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T13:40:31.347 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung/Overview.txt 2024-04-02T13:40:31.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe9077fe640 1 -- 172.21.15.33:0/2053067663 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920026020 con 0x563f601e4ca0 2024-04-02T13:40:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe9077fe640 1 -- 172.21.15.33:0/2053067663 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe9200223f0 con 0x563f601e4ca0 2024-04-02T13:40:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe93123c740 1 -- 172.21.15.33:0/2053067663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9180054f0 con 0x563f601e4ca0 2024-04-02T13:40:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe9077fe640 1 -- 172.21.15.33:0/2053067663 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe9200225f0 con 0x563f601e4ca0 2024-04-02T13:40:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe93123c740 10 client.25634 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe93123c740 1 -- 172.21.15.33:0/2053067663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563f601f3a10 con 0x563f601e4ca0 2024-04-02T13:40:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.328+0000 7fe9077fe640 1 -- 172.21.15.33:0/2053067663 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe9180054f0 con 0x563f601e4ca0 2024-04-02T13:40:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.329+0000 7fe9077fe640 1 -- 172.21.15.33:0/2053067663 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe92002a970 con 0x563f601e4ca0 2024-04-02T13:40:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.329+0000 7fe93123c740 10 client.25634 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:31.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.329+0000 7fe93123c740 10 client.25634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.329+0000 7fe93123c740 20 client.25634 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.329+0000 7fe93123c740 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563f601f4fc0 0x563f601f7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.329+0000 7fe93123c740 4 client.25634 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.329+0000 7fe93123c740 1 -- 172.21.15.33:0/2053067663 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f6014ccf0 con 0x563f601f4fc0 2024-04-02T13:40:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.329+0000 7fe927fff640 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563f601f4fc0 0x563f601f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.330+0000 7fe927fff640 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563f601f4fc0 0x563f601f7420 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ffbec9428e2c2e3 server_cookie=542e77ad1cad7bf4 in_seq=0 out_seq=0 2024-04-02T13:40:31.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.330+0000 7fe9077fe640 10 client.25634 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:31.350 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Samsung/clksrc-change-registers.awk 2024-04-02T13:40:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe9077fe640 1 -- 172.21.15.33:0/2053067663 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f6014ccf0 con 0x563f601f4fc0 2024-04-02T13:40:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe9077fe640 10 client.25634 handle_command_reply: tid=0 2024-04-02T13:40:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe93123c740 1 client.25634 shutdown 2024-04-02T13:40:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe93123c740 1 -- 172.21.15.33:0/2053067663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601e4ca0 msgr2=0x563f601e1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe93123c740 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601e4ca0 0x563f601e1e90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe920009fd0 tx=0x7fe920004a40 comp rx=0 tx=0).stop 2024-04-02T13:40:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe93123c740 1 -- 172.21.15.33:0/2053067663 shutdown_connections 2024-04-02T13:40:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe93123c740 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563f601f4fc0 0x563f601f7420 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe93123c740 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601e4ca0 0x563f601e1e90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe93123c740 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601dc0c0 0x563f601e1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe93123c740 1 --2- 172.21.15.33:0/2053067663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601db7a0 0x563f601e45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe93123c740 1 -- 172.21.15.33:0/2053067663 >> 172.21.15.33:0/2053067663 conn(0x563f601d5ed0 msgr2=0x563f601d7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe9077fe640 0 client.25634 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe93123c740 1 -- 172.21.15.33:0/2053067663 shutdown_connections 2024-04-02T13:40:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.331+0000 7fe93123c740 1 -- 172.21.15.33:0/2053067663 wait complete. 2024-04-02T13:40:31.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.332+0000 7fe93123c740 20 client.25634 trim_cache size 0 max 16384 2024-04-02T13:40:31.354 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/Setup 2024-04-02T13:40:31.354 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/VFP/ 2024-04-02T13:40:31.355 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/VFP/release-notes.txt 2024-04-02T13:40:31.355 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/cluster-pm-race-avoidance.txt 2024-04-02T13:40:31.355 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/firmware.txt 2024-04-02T13:40:31.355 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/kernel_mode_neon.txt 2024-04-02T13:40:31.355 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/kernel_user_helpers.txt 2024-04-02T13:40:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.349+0000 7fe93123c740 1 Processor -- start 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.349+0000 7fe93123c740 1 -- start start 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.349+0000 7fe93123c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601db7a0 0x563f601f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.349+0000 7fe93123c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601dc0c0 0x563f601e4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.349+0000 7fe93123c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601e4ca0 0x563f601e45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.349+0000 7fe93123c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe914012740 con 0x563f601e4ca0 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.349+0000 7fe93123c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe914010010 con 0x563f601db7a0 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.349+0000 7fe93123c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe91400de40 con 0x563f601dc0c0 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.349+0000 7fe92cbf2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601e4ca0 0x563f601e45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.349+0000 7fe92cbf2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601e4ca0 0x563f601e45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47242/0 (socket says 172.21.15.33:47242) 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.349+0000 7fe92cbf2640 1 -- 172.21.15.33:0/1407115187 learned_addr learned my addr 172.21.15.33:0/1407115187 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.349+0000 7fe9277fe640 1 --2- 172.21.15.33:0/1407115187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601dc0c0 0x563f601e4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.349+0000 7fe927fff640 1 --2- 172.21.15.33:0/1407115187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601db7a0 0x563f601f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.350+0000 7fe9277fe640 1 -- 172.21.15.33:0/1407115187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601db7a0 msgr2=0x563f601f09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.350+0000 7fe9277fe640 1 --2- 172.21.15.33:0/1407115187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601db7a0 0x563f601f09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.350+0000 7fe9277fe640 1 -- 172.21.15.33:0/1407115187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601e4ca0 msgr2=0x563f601e45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.350+0000 7fe9277fe640 1 --2- 172.21.15.33:0/1407115187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601e4ca0 0x563f601e45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.350+0000 7fe9277fe640 1 -- 172.21.15.33:0/1407115187 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f601dbeb0 con 0x563f601dc0c0 2024-04-02T13:40:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.350+0000 7fe9277fe640 1 --2- 172.21.15.33:0/1407115187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601dc0c0 0x563f601e4080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe91c005ab0 tx=0x7fe91c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfb8dd4b24b0be3d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.350+0000 7fe90dffb640 1 -- 172.21.15.33:0/1407115187 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c0d7070 con 0x563f601dc0c0 2024-04-02T13:40:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.350+0000 7fe90dffb640 1 -- 172.21.15.33:0/1407115187 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe91c0be690 con 0x563f601dc0c0 2024-04-02T13:40:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.350+0000 7fe90dffb640 1 -- 172.21.15.33:0/1407115187 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c0c6770 con 0x563f601dc0c0 2024-04-02T13:40:31.358 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/mem_alignment 2024-04-02T13:40:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.351+0000 7fe93123c740 1 -- 172.21.15.33:0/1407115187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601dc0c0 msgr2=0x563f601e4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.351+0000 7fe93123c740 1 --2- 172.21.15.33:0/1407115187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601dc0c0 0x563f601e4080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe91c005ab0 tx=0x7fe91c009260 comp rx=0 tx=0).stop 2024-04-02T13:40:31.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.351+0000 7fe93123c740 1 -- 172.21.15.33:0/1407115187 shutdown_connections 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.351+0000 7fe93123c740 1 --2- 172.21.15.33:0/1407115187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601e4ca0 0x563f601e45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.351+0000 7fe93123c740 1 --2- 172.21.15.33:0/1407115187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601dc0c0 0x563f601e4080 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.351+0000 7fe93123c740 1 --2- 172.21.15.33:0/1407115187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601db7a0 0x563f601f09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.351+0000 7fe93123c740 1 -- 172.21.15.33:0/1407115187 >> 172.21.15.33:0/1407115187 conn(0x563f601d5ed0 msgr2=0x563f601e1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.351+0000 7fe93123c740 1 -- 172.21.15.33:0/1407115187 shutdown_connections 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.351+0000 7fe93123c740 1 -- 172.21.15.33:0/1407115187 wait complete. 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.351+0000 7fe93123c740 1 -- start start 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe93123c740 1 Processor -- start 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe93123c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601db7a0 0x563f601e6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe93123c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601dc0c0 0x563f601e5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe93123c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601e4ca0 0x563f601e5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe93123c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe91400e4d0 con 0x563f601e4ca0 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe93123c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe91400f940 con 0x563f601dc0c0 2024-04-02T13:40:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe93123c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe91400eba0 con 0x563f601db7a0 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe93123c740 10 client.0 _ll_register_callbacks cb 0x7fe92000f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe92cbf2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601e4ca0 0x563f601e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe927fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601db7a0 0x563f601e6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe92cbf2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601e4ca0 0x563f601e5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47244/0 (socket says 172.21.15.33:47244) 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe92cbf2640 1 -- 172.21.15.33:0/3373751605 learned_addr learned my addr 172.21.15.33:0/3373751605 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe927fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601db7a0 0x563f601e6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:56412/0 (socket says 172.21.15.33:56412) 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe9277fe640 1 --2- 172.21.15.33:0/3373751605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601dc0c0 0x563f601e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe927fff640 1 -- 172.21.15.33:0/3373751605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601dc0c0 msgr2=0x563f601e5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe927fff640 1 --2- 172.21.15.33:0/3373751605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601dc0c0 0x563f601e5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe927fff640 1 -- 172.21.15.33:0/3373751605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601e4ca0 msgr2=0x563f601e5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe927fff640 1 --2- 172.21.15.33:0/3373751605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601e4ca0 0x563f601e5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.352+0000 7fe927fff640 1 -- 172.21.15.33:0/3373751605 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f601e5e30 con 0x563f601db7a0 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe927fff640 1 --2- 172.21.15.33:0/3373751605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601db7a0 0x563f601e6850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe91801b040 tx=0x7fe9180048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94874d012b86292f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe9077fe640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe9077fe640 1 -- 172.21.15.33:0/3373751605 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918031070 con 0x563f601db7a0 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe9077fe640 1 -- 172.21.15.33:0/3373751605 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe91802cdd0 con 0x563f601db7a0 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe93123c740 1 -- 172.21.15.33:0/3373751605 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe91c0be690 con 0x563f601db7a0 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe9077fe640 1 -- 172.21.15.33:0/3373751605 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe91803d050 con 0x563f601db7a0 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe93123c740 10 client.25507 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe93123c740 1 -- 172.21.15.33:0/3373751605 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563f601ecf30 con 0x563f601db7a0 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe9077fe640 1 -- 172.21.15.33:0/3373751605 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe91c0be690 con 0x563f601db7a0 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe9077fe640 1 -- 172.21.15.33:0/3373751605 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe9180035f0 con 0x563f601db7a0 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe93123c740 10 client.25507 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe93123c740 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe93123c740 20 client.25507 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe93123c740 1 --2- 172.21.15.33:0/3373751605 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563f601ed4c0 0x563f60265dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe93123c740 4 client.25507 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:40:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.353+0000 7fe93123c740 1 -- 172.21.15.33:0/3373751605 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f60147040 con 0x563f601ed4c0 2024-04-02T13:40:31.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.354+0000 7fe9277fe640 1 --2- 172.21.15.33:0/3373751605 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563f601ed4c0 0x563f60265dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:31.365 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/memory.txt 2024-04-02T13:40:31.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.354+0000 7fe9277fe640 1 --2- 172.21.15.33:0/3373751605 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563f601ed4c0 0x563f60265dc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d23e9698cd2b3b60 server_cookie=4181ad34cb1f3877 in_seq=0 out_seq=0 2024-04-02T13:40:31.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.354+0000 7fe9077fe640 10 client.25507 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:31.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.354+0000 7fe9077fe640 1 -- 172.21.15.33:0/3373751605 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f60147040 con 0x563f601ed4c0 2024-04-02T13:40:31.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.354+0000 7fe9077fe640 10 client.25507 handle_command_reply: tid=0 2024-04-02T13:40:31.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.354+0000 7fe93123c740 1 client.25507 shutdown 2024-04-02T13:40:31.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.355+0000 7fe93123c740 1 -- 172.21.15.33:0/3373751605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601db7a0 msgr2=0x563f601e6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.355+0000 7fe93123c740 1 --2- 172.21.15.33:0/3373751605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601db7a0 0x563f601e6850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe91801b040 tx=0x7fe9180048d0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.355+0000 7fe93123c740 1 -- 172.21.15.33:0/3373751605 shutdown_connections 2024-04-02T13:40:31.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.355+0000 7fe93123c740 1 --2- 172.21.15.33:0/3373751605 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563f601ed4c0 0x563f60265dc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.355+0000 7fe93123c740 1 --2- 172.21.15.33:0/3373751605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f601e4ca0 0x563f601e5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.355+0000 7fe93123c740 1 --2- 172.21.15.33:0/3373751605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f601dc0c0 0x563f601e5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.355+0000 7fe93123c740 1 --2- 172.21.15.33:0/3373751605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563f601db7a0 0x563f601e6850 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.355+0000 7fe9077fe640 0 client.25507 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:31.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.355+0000 7fe93123c740 1 -- 172.21.15.33:0/3373751605 >> 172.21.15.33:0/3373751605 conn(0x563f601d5ed0 msgr2=0x563f601dcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.355+0000 7fe93123c740 1 -- 172.21.15.33:0/3373751605 shutdown_connections 2024-04-02T13:40:31.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.355+0000 7fe93123c740 1 -- 172.21.15.33:0/3373751605 wait complete. 2024-04-02T13:40:31.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.355+0000 7fe93123c740 20 client.25507 trim_cache size 0 max 16384 2024-04-02T13:40:31.368 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:40:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.356+0000 7fe92e655640 1 -- 172.21.15.33:0/4275329650 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe9100a3d20 msgr2=0x7fe9100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.356+0000 7fe92e655640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe9100a3d20 0x7fe9100a61e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe920005c40 tx=0x7fe920004ab0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.356+0000 7fe92e655640 1 -- 172.21.15.33:0/4275329650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe928101330 msgr2=0x7fe92819aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.356+0000 7fe92e655640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe928101330 0x7fe92819aad0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe91c000c00 tx=0x7fe91c004570 comp rx=0 tx=0).stop 2024-04-02T13:40:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.356+0000 7fe92e655640 1 -- 172.21.15.33:0/4275329650 shutdown_connections 2024-04-02T13:40:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.356+0000 7fe92e655640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe9100a3d20 0x7fe9100a61e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.356+0000 7fe92e655640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe928106980 0x7fe92819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.356+0000 7fe92e655640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe928101330 0x7fe92819aad0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.356+0000 7fe92e655640 1 --2- 172.21.15.33:0/4275329650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe928100970 0x7fe92819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.356+0000 7fe92e655640 1 -- 172.21.15.33:0/4275329650 >> 172.21.15.33:0/4275329650 conn(0x7fe928077b00 msgr2=0x7fe928104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.357+0000 7fe92e655640 1 -- 172.21.15.33:0/4275329650 shutdown_connections 2024-04-02T13:40:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:31.357+0000 7fe92e655640 1 -- 172.21.15.33:0/4275329650 wait complete. 2024-04-02T13:40:31.370 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/msm/ 2024-04-02T13:40:31.370 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/msm/gpiomux.txt 2024-04-02T13:40:31.370 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/nwfpe/ 2024-04-02T13:40:31.371 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/nwfpe/NOTES 2024-04-02T13:40:31.371 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/nwfpe/README 2024-04-02T13:40:31.371 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/nwfpe/README.FPE 2024-04-02T13:40:31.372 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/nwfpe/TODO 2024-04-02T13:40:31.375 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/pxa/ 2024-04-02T13:40:31.378 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/pxa/mfp.txt 2024-04-02T13:40:31.381 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/sti/ 2024-04-02T13:40:31.384 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/sti/overview.txt 2024-04-02T13:40:31.385 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/sti/stih407-overview.txt 2024-04-02T13:40:31.388 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/sti/stih415-overview.txt 2024-04-02T13:40:31.390 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/sti/stih416-overview.txt 2024-04-02T13:40:31.393 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/sti/stih418-overview.txt 2024-04-02T13:40:31.396 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/sunxi/ 2024-04-02T13:40:31.398 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/sunxi/README 2024-04-02T13:40:31.400 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/sunxi/clocks.txt 2024-04-02T13:40:31.402 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/swp_emulation 2024-04-02T13:40:31.405 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/tcm.txt 2024-04-02T13:40:31.407 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/uefi.txt 2024-04-02T13:40:31.409 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm/vlocks.txt 2024-04-02T13:40:31.414 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm64/ 2024-04-02T13:40:31.414 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm64/booting.txt 2024-04-02T13:40:31.416 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm64/legacy_instructions.txt 2024-04-02T13:40:31.418 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm64/memory.txt 2024-04-02T13:40:31.420 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/arm64/tagged-pointers.txt 2024-04-02T13:40:31.422 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/assoc_array.txt 2024-04-02T13:40:31.425 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/atomic_ops.txt 2024-04-02T13:40:31.428 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/auxdisplay/ 2024-04-02T13:40:31.430 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/auxdisplay/.gitignore 2024-04-02T13:40:31.432 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/auxdisplay/Makefile 2024-04-02T13:40:31.434 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b 2024-04-02T13:40:31.436 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b-example.c 2024-04-02T13:40:31.438 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/auxdisplay/ks0108 2024-04-02T13:40:31.441 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/backlight/ 2024-04-02T13:40:31.443 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/backlight/lp855x-driver.txt 2024-04-02T13:40:31.445 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/bad_memory.txt 2024-04-02T13:40:31.447 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/basic_profiling.txt 2024-04-02T13:40:31.449 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/bcache.txt 2024-04-02T13:40:31.452 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/binfmt_misc.txt 2024-04-02T13:40:31.455 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blackfin/ 2024-04-02T13:40:31.456 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blackfin/00-INDEX 2024-04-02T13:40:31.459 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blackfin/Makefile 2024-04-02T13:40:31.461 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blackfin/bfin-gpio-notes.txt 2024-04-02T13:40:31.463 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blackfin/bfin-spi-notes.txt 2024-04-02T13:40:31.465 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blackfin/gptimers-example.c 2024-04-02T13:40:31.467 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/ 2024-04-02T13:40:31.470 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/00-INDEX 2024-04-02T13:40:31.472 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/biodoc.txt 2024-04-02T13:40:31.475 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/biovecs.txt 2024-04-02T13:40:31.477 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/capability.txt 2024-04-02T13:40:31.480 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/cfq-iosched.txt 2024-04-02T13:40:31.481 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/cmdline-partition.txt 2024-04-02T13:40:31.484 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/data-integrity.txt 2024-04-02T13:40:31.486 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/deadline-iosched.txt 2024-04-02T13:40:31.489 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/ioprio.txt 2024-04-02T13:40:31.492 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/null_blk.txt 2024-04-02T13:40:31.494 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/queue-sysfs.txt 2024-04-02T13:40:31.496 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/request.txt 2024-04-02T13:40:31.499 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/stat.txt 2024-04-02T13:40:31.501 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/switching-sched.txt 2024-04-02T13:40:31.503 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/block/writeback_cache_control.txt 2024-04-02T13:40:31.506 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/ 2024-04-02T13:40:31.508 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/00-INDEX 2024-04-02T13:40:31.510 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/README.DAC960 2024-04-02T13:40:31.513 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/cciss.txt 2024-04-02T13:40:31.515 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/cpqarray.txt 2024-04-02T13:40:31.518 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/drbd/ 2024-04-02T13:40:31.519 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-8.3-data-packets.svg 2024-04-02T13:40:31.522 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-data-packets.svg 2024-04-02T13:40:31.524 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/drbd/README.txt 2024-04-02T13:40:31.527 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/drbd/conn-states-8.dot 2024-04-02T13:40:31.529 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/drbd/data-structure-v9.txt 2024-04-02T13:40:31.531 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/drbd/disk-states-8.dot 2024-04-02T13:40:31.533 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/drbd/drbd-connection-state-overview.dot 2024-04-02T13:40:31.536 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/drbd/node-states-8.dot 2024-04-02T13:40:31.538 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/floppy.txt 2024-04-02T13:40:31.541 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/mflash.txt 2024-04-02T13:40:31.543 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/nbd.txt 2024-04-02T13:40:31.545 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/paride.txt 2024-04-02T13:40:31.548 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/ramdisk.txt 2024-04-02T13:40:31.551 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/blockdev/zram.txt 2024-04-02T13:40:31.554 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/braille-console.txt 2024-04-02T13:40:31.557 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/bt8xxgpio.txt 2024-04-02T13:40:31.560 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/btmrvl.txt 2024-04-02T13:40:31.563 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/bus-devices/ 2024-04-02T13:40:31.565 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt 2024-04-02T13:40:31.567 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/bus-virt-phys-mapping.txt 2024-04-02T13:40:31.570 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cachetlb.txt 2024-04-02T13:40:31.572 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cdrom/ 2024-04-02T13:40:31.574 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cdrom/00-INDEX 2024-04-02T13:40:31.577 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cdrom/Makefile 2024-04-02T13:40:31.579 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cdrom/cdrom-standard.tex 2024-04-02T13:40:31.582 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cdrom/ide-cd 2024-04-02T13:40:31.585 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cdrom/packet-writing.txt 2024-04-02T13:40:31.587 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/ 2024-04-02T13:40:31.590 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/00-INDEX 2024-04-02T13:40:31.591 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/blkio-controller.txt 2024-04-02T13:40:31.594 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/cgroups.txt 2024-04-02T13:40:31.598 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/cpuacct.txt 2024-04-02T13:40:31.600 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/cpusets.txt 2024-04-02T13:40:31.603 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/devices.txt 2024-04-02T13:40:31.606 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/freezer-subsystem.txt 2024-04-02T13:40:31.609 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/hugetlb.txt 2024-04-02T13:40:31.611 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/memcg_test.txt 2024-04-02T13:40:31.613 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/memory.txt 2024-04-02T13:40:31.618 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/net_cls.txt 2024-04-02T13:40:31.619 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/net_prio.txt 2024-04-02T13:40:31.621 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cgroups/unified-hierarchy.txt 2024-04-02T13:40:31.625 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/circular-buffers.txt 2024-04-02T13:40:31.628 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/clk.txt 2024-04-02T13:40:31.631 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/coccinelle.txt 2024-04-02T13:40:31.634 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/connector/ 2024-04-02T13:40:31.635 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/connector/.gitignore 2024-04-02T13:40:31.638 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/connector/Makefile 2024-04-02T13:40:31.640 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/connector/cn_test.c 2024-04-02T13:40:31.642 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/connector/connector.txt 2024-04-02T13:40:31.645 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/connector/ucon.c 2024-04-02T13:40:31.647 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/console/ 2024-04-02T13:40:31.649 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/console/console.txt 2024-04-02T13:40:31.651 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-freq/ 2024-04-02T13:40:31.653 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-freq/amd-powernow.txt 2024-04-02T13:40:31.655 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-freq/boost.txt 2024-04-02T13:40:31.657 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-freq/core.txt 2024-04-02T13:40:31.660 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-freq/cpu-drivers.txt 2024-04-02T13:40:31.662 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-nforce2.txt 2024-04-02T13:40:31.664 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-stats.txt 2024-04-02T13:40:31.667 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-freq/governors.txt 2024-04-02T13:40:31.669 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-freq/index.txt 2024-04-02T13:40:31.672 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-freq/intel-pstate.txt 2024-04-02T13:40:31.674 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-freq/pcc-cpufreq.txt 2024-04-02T13:40:31.676 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-freq/user-guide.txt 2024-04-02T13:40:31.679 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-hotplug.txt 2024-04-02T13:40:31.681 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpu-load.txt 2024-04-02T13:40:31.684 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpuidle/ 2024-04-02T13:40:31.686 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpuidle/core.txt 2024-04-02T13:40:31.688 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpuidle/driver.txt 2024-04-02T13:40:31.690 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpuidle/governor.txt 2024-04-02T13:40:31.692 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cpuidle/sysfs.txt 2024-04-02T13:40:31.695 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cputopology.txt 2024-04-02T13:40:31.698 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/crc32.txt 2024-04-02T13:40:31.700 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cris/ 2024-04-02T13:40:31.702 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/cris/README 2024-04-02T13:40:31.704 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/crypto/ 2024-04-02T13:40:31.706 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/crypto/api-intro.txt 2024-04-02T13:40:31.708 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/crypto/asymmetric-keys.txt 2024-04-02T13:40:31.710 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/crypto/async-tx-api.txt 2024-04-02T13:40:31.713 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/crypto/crypto-API-userspace.txt 2024-04-02T13:40:31.715 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/crypto/descore-readme.txt 2024-04-02T13:40:31.718 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dcdbas.txt 2024-04-02T13:40:31.720 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/debugging-modules.txt 2024-04-02T13:40:31.723 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/debugging-via-ohci1394.txt 2024-04-02T13:40:31.725 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dell_rbu.txt 2024-04-02T13:40:31.727 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/development-process/ 2024-04-02T13:40:31.729 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/development-process/1.Intro 2024-04-02T13:40:31.731 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/development-process/2.Process 2024-04-02T13:40:31.734 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/development-process/3.Early-stage 2024-04-02T13:40:31.737 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/development-process/4.Coding 2024-04-02T13:40:31.739 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/development-process/5.Posting 2024-04-02T13:40:31.742 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/development-process/6.Followthrough 2024-04-02T13:40:31.745 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/development-process/7.AdvancedTopics 2024-04-02T13:40:31.747 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/development-process/8.Conclusion 2024-04-02T13:40:31.749 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/ 2024-04-02T13:40:31.751 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/cache-policies.txt 2024-04-02T13:40:31.753 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/cache.txt 2024-04-02T13:40:31.769 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/delay.txt 2024-04-02T13:40:31.873 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/dm-crypt.txt 2024-04-02T13:40:31.875 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/dm-flakey.txt 2024-04-02T13:40:31.877 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/dm-io.txt 2024-04-02T13:40:31.879 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/dm-log.txt 2024-04-02T13:40:31.882 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/dm-queue-length.txt 2024-04-02T13:40:31.884 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/dm-raid.txt 2024-04-02T13:40:31.888 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/dm-service-time.txt 2024-04-02T13:40:31.890 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/dm-uevent.txt 2024-04-02T13:40:31.893 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/era.txt 2024-04-02T13:40:31.894 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/kcopyd.txt 2024-04-02T13:40:31.897 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/linear.txt 2024-04-02T13:40:31.900 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/persistent-data.txt 2024-04-02T13:40:31.901 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/snapshot.txt 2024-04-02T13:40:31.904 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/statistics.txt 2024-04-02T13:40:31.906 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/striped.txt 2024-04-02T13:40:31.909 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/switch.txt 2024-04-02T13:40:31.911 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/thin-provisioning.txt 2024-04-02T13:40:31.913 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/verity.txt 2024-04-02T13:40:31.916 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/device-mapper/zero.txt 2024-04-02T13:40:31.918 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devices.txt 2024-04-02T13:40:31.924 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/ 2024-04-02T13:40:31.926 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/00-INDEX 2024-04-02T13:40:31.928 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ 2024-04-02T13:40:31.930 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ABI.txt 2024-04-02T13:40:31.932 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/ 2024-04-02T13:40:31.934 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/interrupts.txt 2024-04-02T13:40:31.936 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/pmu.txt 2024-04-02T13:40:31.938 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ 2024-04-02T13:40:31.940 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/adapteva.txt 2024-04-02T13:40:31.942 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/ 2024-04-02T13:40:31.945 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-clk-manager.txt 2024-04-02T13:40:31.947 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-sdram-edac.txt 2024-04-02T13:40:31.949 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-system.txt 2024-04-02T13:40:31.951 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/amlogic.txt 2024-04-02T13:40:31.954 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arch_timer.txt 2024-04-02T13:40:31.956 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arm-boards 2024-04-02T13:40:31.959 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt 2024-04-02T13:40:31.961 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp.txt 2024-04-02T13:40:31.964 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-375.txt 2024-04-02T13:40:31.966 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-380-mpcore-soc-ctrl.txt 2024-04-02T13:40:31.969 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-38x.txt 2024-04-02T13:40:31.971 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-cpu-reset.txt 2024-04-02T13:40:31.974 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armadeus.txt 2024-04-02T13:40:31.976 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-at91.txt 2024-04-02T13:40:31.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:31 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3773219112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:40:31.978 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-pmc.txt 2024-04-02T13:40:31.980 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/axxia.txt 2024-04-02T13:40:31.983 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/ 2024-04-02T13:40:31.985 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm11351.txt 2024-04-02T13:40:31.987 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm21664.txt 2024-04-02T13:40:31.989 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm63138.txt 2024-04-02T13:40:31.992 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/brcm,bcm11351-cpu-method 2024-04-02T13:40:31.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:31 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3773219112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:40:31.995 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/cygnus.txt 2024-04-02T13:40:31.997 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-resetmgr.txt 2024-04-02T13:40:32.000 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-timer.txt 2024-04-02T13:40:32.002 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-wdt.txt 2024-04-02T13:40:32.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:31 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3773219112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:40:32.005 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm2835.txt 2024-04-02T13:40:32.008 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm4708.txt 2024-04-02T13:40:32.011 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/brcm-brcmstb.txt 2024-04-02T13:40:32.013 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda.txt 2024-04-02T13:40:32.015 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/ 2024-04-02T13:40:32.017 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/combophy.txt 2024-04-02T13:40:32.020 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/l2ecc.txt 2024-04-02T13:40:32.022 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/mem-ctrlr.txt 2024-04-02T13:40:32.026 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cavium-thunder.txt 2024-04-02T13:40:32.027 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cci.txt 2024-04-02T13:40:32.030 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ccn.txt 2024-04-02T13:40:32.032 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coherency-fabric.txt 2024-04-02T13:40:32.035 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coresight.txt 2024-04-02T13:40:32.037 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/ 2024-04-02T13:40:32.039 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/marvell,berlin-smp 2024-04-02T13:40:32.042 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpus.txt 2024-04-02T13:40:32.045 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci.txt 2024-04-02T13:40:32.047 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/ 2024-04-02T13:40:32.050 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/cp-intc.txt 2024-04-02T13:40:32.053 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/digicolor.txt 2024-04-02T13:40:32.056 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/ 2024-04-02T13:40:32.058 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/power_domain.txt 2024-04-02T13:40:32.060 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/smp-sysram.txt 2024-04-02T13:40:32.063 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/ 2024-04-02T13:40:32.065 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/tlm,trusted-foundations.txt 2024-04-02T13:40:32.067 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fsl.txt 2024-04-02T13:40:32.071 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fw-cfg.txt 2024-04-02T13:40:32.074 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/geniatech.txt 2024-04-02T13:40:32.076 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic-v3.txt 2024-04-02T13:40:32.078 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic.txt 2024-04-02T13:40:32.081 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/global_timer.txt 2024-04-02T13:40:32.083 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/ 2024-04-02T13:40:32.085 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt 2024-04-02T13:40:32.087 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/idle-states.txt 2024-04-02T13:40:32.090 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/insignal-boards.txt 2024-04-02T13:40:32.093 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/ 2024-04-02T13:40:32.095 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/keystone.txt 2024-04-02T13:40:32.097 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/kirkwood.txt 2024-04-02T13:40:32.100 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/l2cc.txt 2024-04-02T13:40:32.102 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx-mic.txt 2024-04-02T13:40:32.104 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx.txt 2024-04-02T13:40:32.107 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,berlin.txt 2024-04-02T13:40:32.109 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,dove.txt 2024-04-02T13:40:32.112 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,kirkwood.txt 2024-04-02T13:40:32.114 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek.txt 2024-04-02T13:40:32.117 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/ 2024-04-02T13:40:32.119 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/mediatek,sysirq.txt 2024-04-02T13:40:32.122 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/moxart.txt 2024-04-02T13:40:32.125 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/ 2024-04-02T13:40:32.127 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/feroceon.txt 2024-04-02T13:40:32.130 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/intc.txt 2024-04-02T13:40:32.133 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/mrvl.txt 2024-04-02T13:40:32.135 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/tauros2.txt 2024-04-02T13:40:32.137 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/timer.txt 2024-04-02T13:40:32.140 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ 2024-04-02T13:40:32.143 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,kpss-acc.txt 2024-04-02T13:40:32.145 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,saw2.txt 2024-04-02T13:40:32.148 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ssbi.txt 2024-04-02T13:40:32.151 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/timer.txt 2024-04-02T13:40:32.153 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mvebu-system-controller.txt 2024-04-02T13:40:32.157 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/nspire.txt 2024-04-02T13:40:32.160 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/olimex.txt 2024-04-02T13:40:32.163 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/ 2024-04-02T13:40:32.165 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/counter.txt 2024-04-02T13:40:32.168 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/crossbar.txt 2024-04-02T13:40:32.170 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dmm.txt 2024-04-02T13:40:32.173 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dsp.txt 2024-04-02T13:40:32.175 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/intc.txt 2024-04-02T13:40:32.177 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/iva.txt 2024-04-02T13:40:32.180 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/l3-noc.txt 2024-04-02T13:40:32.182 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/mpu.txt 2024-04-02T13:40:32.185 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/omap.txt 2024-04-02T13:40:32.187 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/prcm.txt 2024-04-02T13:40:32.190 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/timer.txt 2024-04-02T13:40:32.193 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/picoxcell.txt 2024-04-02T13:40:32.196 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/pmu.txt 2024-04-02T13:40:32.199 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/primecell.txt 2024-04-02T13:40:32.201 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/psci.txt 2024-04-02T13:40:32.203 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip.txt 2024-04-02T13:40:32.205 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/ 2024-04-02T13:40:32.207 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu-sram.txt 2024-04-02T13:40:32.209 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu.txt 2024-04-02T13:40:32.212 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/smp-sram.txt 2024-04-02T13:40:32.214 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rtsm-dcscb.txt 2024-04-02T13:40:32.217 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung-boards.txt 2024-04-02T13:40:32.219 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/ 2024-04-02T13:40:32.221 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-adc.txt 2024-04-02T13:40:32.224 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-chipid.txt 2024-04-02T13:40:32.226 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/interrupt-combiner.txt 2024-04-02T13:40:32.230 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/pmu.txt 2024-04-02T13:40:32.231 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/sysreg.txt 2024-04-02T13:40:32.234 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/shmobile.txt 2024-04-02T13:40:32.237 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sirf.txt 2024-04-02T13:40:32.240 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-misc.txt 2024-04-02T13:40:32.242 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-timer.txt 2024-04-02T13:40:32.244 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear.txt 2024-04-02T13:40:32.247 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/ 2024-04-02T13:40:32.249 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/shirq.txt 2024-04-02T13:40:32.251 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sprd.txt 2024-04-02T13:40:32.255 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-nomadik.txt 2024-04-02T13:40:32.257 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-u300.txt 2024-04-02T13:40:32.260 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sti.txt 2024-04-02T13:40:32.262 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sunxi.txt 2024-04-02T13:40:32.265 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra.txt 2024-04-02T13:40:32.267 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/ 2024-04-02T13:40:32.270 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-ahb.txt 2024-04-02T13:40:32.272 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-emc.txt 2024-04-02T13:40:32.275 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-flowctrl.txt 2024-04-02T13:40:32.278 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-mc.txt 2024-04-02T13:40:32.280 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-pmc.txt 2024-04-02T13:40:32.282 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra30-mc.txt 2024-04-02T13:40:32.284 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/topology.txt 2024-04-02T13:40:32.287 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/twd.txt 2024-04-02T13:40:32.290 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/ 2024-04-02T13:40:32.292 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/power_domain.txt 2024-04-02T13:40:32.294 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-fpga-irq.txt 2024-04-02T13:40:32.297 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-sysreg.txt 2024-04-02T13:40:32.300 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-scc.txt 2024-04-02T13:40:32.302 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-sysreg.txt 2024-04-02T13:40:32.305 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress.txt 2024-04-02T13:40:32.307 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vic.txt 2024-04-02T13:40:32.310 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500.txt 2024-04-02T13:40:32.313 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/ 2024-04-02T13:40:32.315 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-intc.txt 2024-04-02T13:40:32.372 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:40:32.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:40:32.373 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:40:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.479+0000 7fd4f9fcd640 1 -- 172.21.15.33:0/4168622397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f4101330 msgr2=0x7fd4f410d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.479+0000 7fd4f9fcd640 1 --2- 172.21.15.33:0/4168622397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f4101330 0x7fd4f410d840 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009cd0 tx=0x7fd4ec02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.479+0000 7fd4f9fcd640 1 -- 172.21.15.33:0/4168622397 shutdown_connections 2024-04-02T13:40:32.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.479+0000 7fd4f9fcd640 1 --2- 172.21.15.33:0/4168622397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f4101330 0x7fd4f410d840 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.479+0000 7fd4f9fcd640 1 --2- 172.21.15.33:0/4168622397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4f4100970 0x7fd4f4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.479+0000 7fd4f9fcd640 1 --2- 172.21.15.33:0/4168622397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4f4106980 0x7fd4f4106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.479+0000 7fd4f9fcd640 1 -- 172.21.15.33:0/4168622397 >> 172.21.15.33:0/4168622397 conn(0x7fd4f4077b00 msgr2=0x7fd4f4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.479+0000 7fd4f9fcd640 1 -- 172.21.15.33:0/4168622397 shutdown_connections 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.479+0000 7fd4f9fcd640 1 -- 172.21.15.33:0/4168622397 wait complete. 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.480+0000 7fd4f9fcd640 1 Processor -- start 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.480+0000 7fd4f9fcd640 1 -- start start 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.480+0000 7fd4f9fcd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f4100970 0x7fd4f4102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.480+0000 7fd4f9fcd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4f4101330 0x7fd4f4103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.480+0000 7fd4f9fcd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4f4106980 0x7fd4f4103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.480+0000 7fd4f9fcd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4110d60 con 0x7fd4f4100970 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.480+0000 7fd4f9fcd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4110be0 con 0x7fd4f4101330 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.480+0000 7fd4f9fcd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4110ee0 con 0x7fd4f4106980 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.480+0000 7fd4f37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f4100970 0x7fd4f4102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.480+0000 7fd4f37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f4100970 0x7fd4f4102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47258/0 (socket says 172.21.15.33:47258) 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f37fe640 1 -- 172.21.15.33:0/2373784323 learned_addr learned my addr 172.21.15.33:0/2373784323 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4f4101330 0x7fd4f4103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f3fff640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4f4106980 0x7fd4f4103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f37fe640 1 -- 172.21.15.33:0/2373784323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4f4106980 msgr2=0x7fd4f4103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f37fe640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4f4106980 0x7fd4f4103970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f37fe640 1 -- 172.21.15.33:0/2373784323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4f4101330 msgr2=0x7fd4f4103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f37fe640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4f4101330 0x7fd4f4103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f37fe640 1 -- 172.21.15.33:0/2373784323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f41a68a0 con 0x7fd4f4100970 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4f4101330 0x7fd4f4103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f37fe640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f4100970 0x7fd4f4102ef0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400e9f0 tx=0x7fd4e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f0ff9640 1 -- 172.21.15.33:0/2373784323 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400cd80 con 0x7fd4f4100970 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f0ff9640 1 -- 172.21.15.33:0/2373784323 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd4e4010900 con 0x7fd4f4100970 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f9fcd640 1 -- 172.21.15.33:0/2373784323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f41a6b90 con 0x7fd4f4100970 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f0ff9640 1 -- 172.21.15.33:0/2373784323 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e40189e0 con 0x7fd4f4100970 2024-04-02T13:40:32.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.481+0000 7fd4f9fcd640 1 -- 172.21.15.33:0/2373784323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f41a70e0 con 0x7fd4f4100970 2024-04-02T13:40:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4f0ff9640 1 -- 172.21.15.33:0/2373784323 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd4e4010ab0 con 0x7fd4f4100970 2024-04-02T13:40:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4fcbba740 1 Processor -- start 2024-04-02T13:40:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4fcbba740 1 -- start start 2024-04-02T13:40:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4fcbba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 0x55858eb766a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4fcbba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb76be0 0x55858eb7f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4fcbba740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb7f7c0 0x55858eb81bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4fcbba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55858eada770 con 0x55858eb762c0 2024-04-02T13:40:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4fcbba740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55858ea82960 con 0x55858eb7f7c0 2024-04-02T13:40:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4fcbba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55858ead68c0 con 0x55858eb76be0 2024-04-02T13:40:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4f3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 0x55858eb766a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4f3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 0x55858eb766a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47268/0 (socket says 172.21.15.33:47268) 2024-04-02T13:40:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4f3fff640 1 -- 172.21.15.33:0/1353740649 learned_addr learned my addr 172.21.15.33:0/1353740649 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/1353740649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb7f7c0 0x55858eb81bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4f0ff9640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd4dc0a3d20 0x7fd4dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4f37fe640 1 --2- 172.21.15.33:0/1353740649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb76be0 0x55858eb7f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4f2ffd640 1 -- 172.21.15.33:0/1353740649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb76be0 msgr2=0x55858eb7f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/1353740649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb76be0 0x55858eb7f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4f2ffd640 1 -- 172.21.15.33:0/1353740649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 msgr2=0x55858eb766a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/1353740649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 0x55858eb766a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4f2ffd640 1 -- 172.21.15.33:0/1353740649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858e8b67e0 con 0x55858eb7f7c0 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.484+0000 7fd4f3fff640 1 --2- 172.21.15.33:0/1353740649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 0x55858eb766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.485+0000 7fd4f3fff640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd4dc0a3d20 0x7fd4dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.485+0000 7fd4f0ff9640 1 -- 172.21.15.33:0/2373784323 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd4e40d0ea0 con 0x7fd4f4100970 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.485+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/1353740649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb7f7c0 0x55858eb81bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8005fd0 tx=0x7fd4e8010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79101ba03ccce252 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.485+0000 7fd4d9ffb640 1 -- 172.21.15.33:0/1353740649 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8011700 con 0x55858eb7f7c0 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.485+0000 7fd4d9ffb640 1 -- 172.21.15.33:0/1353740649 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd4e8011d70 con 0x55858eb7f7c0 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.485+0000 7fd4d9ffb640 1 -- 172.21.15.33:0/1353740649 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e801ada0 con 0x55858eb7f7c0 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4fcbba740 1 -- 172.21.15.33:0/1353740649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb7f7c0 msgr2=0x55858eb81bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4f3fff640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd4dc0a3d20 0x7fd4dc0a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec02c700 tx=0x7fd4ec02e040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/1353740649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb7f7c0 0x55858eb81bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8005fd0 tx=0x7fd4e8010860 comp rx=0 tx=0).stop 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4fcbba740 1 -- 172.21.15.33:0/1353740649 shutdown_connections 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/1353740649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb7f7c0 0x55858eb81bb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/1353740649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb76be0 0x55858eb7f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/1353740649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 0x55858eb766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4fcbba740 1 -- 172.21.15.33:0/1353740649 >> 172.21.15.33:0/1353740649 conn(0x55858eb709f0 msgr2=0x55858eb70dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4fcbba740 1 -- 172.21.15.33:0/1353740649 shutdown_connections 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4fcbba740 1 -- 172.21.15.33:0/1353740649 wait complete. 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4fcbba740 1 -- start start 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4fcbba740 1 Processor -- start 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4fcbba740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb762c0 0x55858eb7c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.486+0000 7fd4fcbba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb76be0 0x55858eb7cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4fcbba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb7f7c0 0x55858eb7d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4fcbba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55858eb825b0 con 0x55858eb76be0 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4fcbba740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55858ea95d20 con 0x55858eb762c0 2024-04-02T13:40:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4fcbba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55858eb82730 con 0x55858eb7f7c0 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4fcbba740 10 client.0 _ll_register_callbacks cb 0x55858ea62a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4f2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb76be0 0x55858eb7cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4f37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb762c0 0x55858eb7c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4f37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb762c0 0x55858eb7c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:59840/0 (socket says 172.21.15.33:59840) 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4f37fe640 1 -- 172.21.15.33:0/4223930977 learned_addr learned my addr 172.21.15.33:0/4223930977 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4f3fff640 1 --2- 172.21.15.33:0/4223930977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb7f7c0 0x55858eb7d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4f37fe640 1 -- 172.21.15.33:0/4223930977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb7f7c0 msgr2=0x55858eb7d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4f37fe640 1 --2- 172.21.15.33:0/4223930977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb7f7c0 0x55858eb7d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4f37fe640 1 -- 172.21.15.33:0/4223930977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb76be0 msgr2=0x55858eb7cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4f37fe640 1 --2- 172.21.15.33:0/4223930977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb76be0 0x55858eb7cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4f37fe640 1 -- 172.21.15.33:0/4223930977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858e916f70 con 0x55858eb762c0 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4f37fe640 1 --2- 172.21.15.33:0/4223930977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb762c0 0x55858eb7c760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009830 tx=0x7fd4e409c610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c758e07c1e0973 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4d2ffd640 1 -- 172.21.15.33:0/4223930977 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e409cad0 con 0x55858eb762c0 2024-04-02T13:40:32.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4fcbba740 1 -- 172.21.15.33:0/4223930977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e8011d70 con 0x55858eb762c0 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.487+0000 7fd4d2ffd640 1 -- 172.21.15.33:0/4223930977 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd4e40bd0b0 con 0x55858eb762c0 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.488+0000 7fd4d2ffd640 1 -- 172.21.15.33:0/4223930977 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd4e40a0e80 con 0x55858eb762c0 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.488+0000 7fd4fcbba740 10 client.25652 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.488+0000 7fd4fcbba740 1 -- 172.21.15.33:0/4223930977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55858e7ae170 con 0x55858eb762c0 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.488+0000 7fd4d2ffd640 1 -- 172.21.15.33:0/4223930977 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd4e8011d70 con 0x55858eb762c0 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.488+0000 7fd4d2ffd640 1 -- 172.21.15.33:0/4223930977 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd4e40b1150 con 0x55858eb762c0 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.488+0000 7fd4fcbba740 10 client.25652 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.488+0000 7fd4fcbba740 10 client.25652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.488+0000 7fd4fcbba740 20 client.25652 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.488+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/4223930977 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55858eb8fa70 0x55858eb91e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.488+0000 7fd4fcbba740 4 client.25652 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.488+0000 7fd4fcbba740 1 -- 172.21.15.33:0/4223930977 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55858eae7cb0 con 0x55858eb8fa70 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.488+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/4223930977 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55858eb8fa70 0x55858eb91e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.489+0000 7fd4d2ffd640 10 client.25652 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.489+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/4223930977 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55858eb8fa70 0x55858eb91e60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d04d5093c72fba server_cookie=1ac132f598ecfa35 in_seq=0 out_seq=0 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4d2ffd640 1 -- 172.21.15.33:0/4223930977 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55858eae7cb0 con 0x55858eb8fa70 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4d2ffd640 10 client.25652 handle_command_reply: tid=0 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4fcbba740 1 client.25652 shutdown 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4fcbba740 1 -- 172.21.15.33:0/4223930977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb762c0 msgr2=0x55858eb7c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/4223930977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb762c0 0x55858eb7c760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009830 tx=0x7fd4e409c610 comp rx=0 tx=0).stop 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4fcbba740 1 -- 172.21.15.33:0/4223930977 shutdown_connections 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/4223930977 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55858eb8fa70 0x55858eb91e60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/4223930977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb7f7c0 0x55858eb7d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4d2ffd640 0 client.25652 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/4223930977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb76be0 0x55858eb7cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/4223930977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb762c0 0x55858eb7c760 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4fcbba740 1 -- 172.21.15.33:0/4223930977 >> 172.21.15.33:0/4223930977 conn(0x55858eb709f0 msgr2=0x55858eb72740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4fcbba740 1 -- 172.21.15.33:0/4223930977 shutdown_connections 2024-04-02T13:40:32.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.490+0000 7fd4fcbba740 1 -- 172.21.15.33:0/4223930977 wait complete. 2024-04-02T13:40:32.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.491+0000 7fd4fcbba740 20 client.25652 trim_cache size 0 max 16384 2024-04-02T13:40:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.511+0000 7fd4fcbba740 1 Processor -- start 2024-04-02T13:40:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.511+0000 7fd4fcbba740 1 -- start start 2024-04-02T13:40:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.511+0000 7fd4fcbba740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb762c0 0x55858eb7ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.511+0000 7fd4fcbba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb76be0 0x55858eb8b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.511+0000 7fd4fcbba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb7f7c0 0x55858eb97700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.511+0000 7fd4fcbba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41b39e0 con 0x55858eb76be0 2024-04-02T13:40:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.511+0000 7fd4fcbba740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41b12b0 con 0x55858eb762c0 2024-04-02T13:40:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.511+0000 7fd4fcbba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41af0e0 con 0x55858eb7f7c0 2024-04-02T13:40:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.511+0000 7fd4f2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb76be0 0x55858eb8b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.511+0000 7fd4f2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb76be0 0x55858eb8b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47272/0 (socket says 172.21.15.33:47272) 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.511+0000 7fd4f2ffd640 1 -- 172.21.15.33:0/1012724630 learned_addr learned my addr 172.21.15.33:0/1012724630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.511+0000 7fd4f37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb762c0 0x55858eb7ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.511+0000 7fd4f3fff640 1 --2- 172.21.15.33:0/1012724630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb7f7c0 0x55858eb97700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.512+0000 7fd4f2ffd640 1 -- 172.21.15.33:0/1012724630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb7f7c0 msgr2=0x55858eb97700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.512+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/1012724630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb7f7c0 0x55858eb97700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.512+0000 7fd4f2ffd640 1 -- 172.21.15.33:0/1012724630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb762c0 msgr2=0x55858eb7ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.512+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/1012724630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb762c0 0x55858eb7ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.512+0000 7fd4f2ffd640 1 -- 172.21.15.33:0/1012724630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858eb76a10 con 0x55858eb76be0 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.512+0000 7fd4f37fe640 1 --2- 172.21.15.33:0/1012724630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb762c0 0x55858eb7ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.512+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/1012724630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb76be0 0x55858eb8b4a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800bdc0 tx=0x7fd4e8011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ac216d38ddbe92 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.512+0000 7fd4d97fa640 1 -- 172.21.15.33:0/1012724630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e801a9b0 con 0x55858eb76be0 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.512+0000 7fd4d97fa640 1 -- 172.21.15.33:0/1012724630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd4e8005510 con 0x55858eb76be0 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.512+0000 7fd4d97fa640 1 -- 172.21.15.33:0/1012724630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e802b450 con 0x55858eb76be0 2024-04-02T13:40:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 -- 172.21.15.33:0/1012724630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb76be0 msgr2=0x55858eb8b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/1012724630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb76be0 0x55858eb8b4a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800bdc0 tx=0x7fd4e8011f20 comp rx=0 tx=0).stop 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 -- 172.21.15.33:0/1012724630 shutdown_connections 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/1012724630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb7f7c0 0x55858eb97700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/1012724630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb76be0 0x55858eb8b4a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/1012724630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb762c0 0x55858eb7ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 -- 172.21.15.33:0/1012724630 >> 172.21.15.33:0/1012724630 conn(0x55858eb709f0 msgr2=0x55858eb6fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 -- 172.21.15.33:0/1012724630 shutdown_connections 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 -- 172.21.15.33:0/1012724630 wait complete. 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 -- start start 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 Processor -- start 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 0x55858eb83be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb76be0 0x55858eb84120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb7f7c0 0x55858ebf4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41af770 con 0x55858eb762c0 2024-04-02T13:40:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41b0be0 con 0x55858eb7f7c0 2024-04-02T13:40:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.513+0000 7fd4fcbba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41afe40 con 0x55858eb76be0 2024-04-02T13:40:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4fcbba740 10 client.0 _ll_register_callbacks cb 0x7fd4e40986d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4f37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 0x55858eb83be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4f37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 0x55858eb83be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47280/0 (socket says 172.21.15.33:47280) 2024-04-02T13:40:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4f37fe640 1 -- 172.21.15.33:0/3319038875 learned_addr learned my addr 172.21.15.33:0/3319038875 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4f3fff640 1 --2- 172.21.15.33:0/3319038875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb7f7c0 0x55858ebf4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/3319038875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb76be0 0x55858eb84120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4f37fe640 1 -- 172.21.15.33:0/3319038875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb76be0 msgr2=0x55858eb84120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4f37fe640 1 --2- 172.21.15.33:0/3319038875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb76be0 0x55858eb84120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4f37fe640 1 -- 172.21.15.33:0/3319038875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb7f7c0 msgr2=0x55858ebf4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4f37fe640 1 --2- 172.21.15.33:0/3319038875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb7f7c0 0x55858ebf4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4f37fe640 1 -- 172.21.15.33:0/3319038875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858ebf5560 con 0x55858eb762c0 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4f37fe640 1 --2- 172.21.15.33:0/3319038875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 0x55858eb83be0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd4e409ca60 tx=0x7fd4e4098d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa920a7fa99a5c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4d2ffd640 1 -- 172.21.15.33:0/3319038875 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e409c430 con 0x55858eb762c0 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4d2ffd640 1 -- 172.21.15.33:0/3319038875 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd4e409bf20 con 0x55858eb762c0 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4fcbba740 1 -- 172.21.15.33:0/3319038875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e8005510 con 0x55858eb762c0 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4d2ffd640 1 -- 172.21.15.33:0/3319038875 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd4e40a5740 con 0x55858eb762c0 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4fcbba740 10 client.16056 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.514+0000 7fd4fcbba740 1 -- 172.21.15.33:0/3319038875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55858ebf6880 con 0x55858eb762c0 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.515+0000 7fd4d2ffd640 1 -- 172.21.15.33:0/3319038875 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd4e8005510 con 0x55858eb762c0 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.515+0000 7fd4d2ffd640 1 -- 172.21.15.33:0/3319038875 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd4e409b990 con 0x55858eb762c0 2024-04-02T13:40:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.515+0000 7fd4fcbba740 10 client.16056 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.515+0000 7fd4fcbba740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.515+0000 7fd4fcbba740 20 client.16056 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.515+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/3319038875 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55858ebf6ea0 0x55858ebf92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.515+0000 7fd4fcbba740 4 client.16056 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:40:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.515+0000 7fd4fcbba740 1 -- 172.21.15.33:0/3319038875 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55858eae2000 con 0x55858ebf6ea0 2024-04-02T13:40:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.515+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/3319038875 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55858ebf6ea0 0x55858ebf92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.515+0000 7fd4d2ffd640 10 client.16056 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.515+0000 7fd4f2ffd640 1 --2- 172.21.15.33:0/3319038875 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55858ebf6ea0 0x55858ebf92c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=305cfe5691087d69 server_cookie=baf99e0b480a5ece in_seq=0 out_seq=0 2024-04-02T13:40:32.629 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-pmc.txt 2024-04-02T13:40:32.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.626+0000 7fd4d2ffd640 1 -- 172.21.15.33:0/3319038875 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55858eae2000 con 0x55858ebf6ea0 2024-04-02T13:40:32.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.626+0000 7fd4d2ffd640 10 client.16056 handle_command_reply: tid=0 2024-04-02T13:40:32.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.626+0000 7fd4fcbba740 1 client.16056 shutdown 2024-04-02T13:40:32.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.627+0000 7fd4fcbba740 1 -- 172.21.15.33:0/3319038875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 msgr2=0x55858eb83be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.627+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/3319038875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 0x55858eb83be0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd4e409ca60 tx=0x7fd4e4098d70 comp rx=0 tx=0).stop 2024-04-02T13:40:32.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.627+0000 7fd4fcbba740 1 -- 172.21.15.33:0/3319038875 shutdown_connections 2024-04-02T13:40:32.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.627+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/3319038875 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55858ebf6ea0 0x55858ebf92c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.627+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/3319038875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55858eb7f7c0 0x55858ebf4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.627+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/3319038875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55858eb76be0 0x55858eb84120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.627+0000 7fd4d2ffd640 0 client.16056 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:32.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.627+0000 7fd4fcbba740 1 --2- 172.21.15.33:0/3319038875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55858eb762c0 0x55858eb83be0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.627+0000 7fd4fcbba740 1 -- 172.21.15.33:0/3319038875 >> 172.21.15.33:0/3319038875 conn(0x55858eb709f0 msgr2=0x55858eb7c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:32.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.627+0000 7fd4fcbba740 1 -- 172.21.15.33:0/3319038875 shutdown_connections 2024-04-02T13:40:32.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.627+0000 7fd4fcbba740 1 -- 172.21.15.33:0/3319038875 wait complete. 2024-04-02T13:40:32.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.627+0000 7fd4fcbba740 20 client.16056 trim_cache size 0 max 16384 2024-04-02T13:40:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:40:32.632 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:40:32.632 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "d3d02c43-90ea-4711-8ee0-b9b22907f1bb", 2024-04-02T13:40:32.632 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:40:32.632 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:40:32.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.628+0000 7fd4f9fcd640 1 -- 172.21.15.33:0/2373784323 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd4dc0a3d20 msgr2=0x7fd4dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.628+0000 7fd4f9fcd640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd4dc0a3d20 0x7fd4dc0a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec02c700 tx=0x7fd4ec02e040 comp rx=0 tx=0).stop 2024-04-02T13:40:32.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.628+0000 7fd4f9fcd640 1 -- 172.21.15.33:0/2373784323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f4100970 msgr2=0x7fd4f4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.628+0000 7fd4f9fcd640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f4100970 0x7fd4f4102ef0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400e9f0 tx=0x7fd4e400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.628+0000 7fd4f9fcd640 1 -- 172.21.15.33:0/2373784323 shutdown_connections 2024-04-02T13:40:32.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.628+0000 7fd4f9fcd640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd4dc0a3d20 0x7fd4dc0a61e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.628+0000 7fd4f9fcd640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4f4106980 0x7fd4f4103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.628+0000 7fd4f9fcd640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4f4101330 0x7fd4f4103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.628+0000 7fd4f9fcd640 1 --2- 172.21.15.33:0/2373784323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f4100970 0x7fd4f4102ef0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.628+0000 7fd4f9fcd640 1 -- 172.21.15.33:0/2373784323 >> 172.21.15.33:0/2373784323 conn(0x7fd4f4077b00 msgr2=0x7fd4f4068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:32.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.628+0000 7fd4f9fcd640 1 -- 172.21.15.33:0/2373784323 shutdown_connections 2024-04-02T13:40:32.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.628+0000 7fd4f9fcd640 1 -- 172.21.15.33:0/2373784323 wait complete. 2024-04-02T13:40:32.634 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-timer.txt 2024-04-02T13:40:32.639 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xen.txt 2024-04-02T13:40:32.639 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xilinx.txt 2024-04-02T13:40:32.641 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ 2024-04-02T13:40:32.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:40:32.644 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:40:32.644 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-platform.txt 2024-04-02T13:40:32.647 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-st.txt 2024-04-02T13:40:32.650 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/apm-xgene.txt 2024-04-02T13:40:32.653 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/atmel-at91_cf.txt 2024-04-02T13:40:32.656 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/cavium-compact-flash.txt 2024-04-02T13:40:32.658 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/exynos-sata.txt 2024-04-02T13:40:32.661 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/fsl-sata.txt 2024-04-02T13:40:32.664 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-pata.txt 2024-04-02T13:40:32.667 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-sata.txt 2024-04-02T13:40:32.669 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/marvell.txt 2024-04-02T13:40:32.672 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/pata-arasan.txt 2024-04-02T13:40:32.675 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/qcom-sata.txt 2024-04-02T13:40:32.678 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_highbank.txt 2024-04-02T13:40:32.680 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_rcar.txt 2024-04-02T13:40:32.683 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/tegra-sata.txt 2024-04-02T13:40:32.687 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/btmrvl.txt 2024-04-02T13:40:32.691 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ 2024-04-02T13:40:32.694 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/bcma.txt 2024-04-02T13:40:32.699 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/brcm,gisb-arb.txt 2024-04-02T13:40:32.701 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/imx-weim.txt 2024-04-02T13:40:32.704 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/mvebu-mbus.txt 2024-04-02T13:40:32.707 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/omap-ocp2scp.txt 2024-04-02T13:40:32.710 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ti-gpmc.txt 2024-04-02T13:40:32.713 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/ 2024-04-02T13:40:32.717 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/clocks.txt 2024-04-02T13:40:32.721 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/dscr.txt 2024-04-02T13:40:32.726 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/emifa.txt 2024-04-02T13:40:32.729 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/interrupt.txt 2024-04-02T13:40:32.732 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/soc.txt 2024-04-02T13:40:32.735 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/timer64.txt 2024-04-02T13:40:32.738 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/chosen.txt 2024-04-02T13:40:32.741 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ 2024-04-02T13:40:32.744 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/alphascale,acc.txt 2024-04-02T13:40:32.747 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/altr_socfpga.txt 2024-04-02T13:40:32.750 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/arm-integrator.txt 2024-04-02T13:40:32.752 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/at91-clock.txt 2024-04-02T13:40:32.755 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/axi-clkgen.txt 2024-04-02T13:40:32.758 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-cygnus-clock.txt 2024-04-02T13:40:32.761 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt 2024-04-02T13:40:32.764 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/calxeda.txt 2024-04-02T13:40:32.766 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-exynos-audss.txt 2024-04-02T13:40:32.774 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-palmas-clk32kg-clocks.txt 2024-04-02T13:40:32.777 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-s5pv210-audss.txt 2024-04-02T13:40:32.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.776+0000 7fa2df4ed640 1 -- 172.21.15.33:0/2115277441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2d8106980 msgr2=0x7fa2d8106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.776+0000 7fa2df4ed640 1 --2- 172.21.15.33:0/2115277441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2d8106980 0x7fa2d8106d60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009900 tx=0x7fa2d402bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.776+0000 7fa2df4ed640 1 -- 172.21.15.33:0/2115277441 shutdown_connections 2024-04-02T13:40:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.776+0000 7fa2df4ed640 1 --2- 172.21.15.33:0/2115277441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d8101330 0x7fa2d810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.776+0000 7fa2df4ed640 1 --2- 172.21.15.33:0/2115277441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8100970 0x7fa2d8100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.776+0000 7fa2df4ed640 1 --2- 172.21.15.33:0/2115277441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2d8106980 0x7fa2d8106d60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.776+0000 7fa2df4ed640 1 -- 172.21.15.33:0/2115277441 >> 172.21.15.33:0/2115277441 conn(0x7fa2d8077b00 msgr2=0x7fa2d8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.776+0000 7fa2df4ed640 1 -- 172.21.15.33:0/2115277441 shutdown_connections 2024-04-02T13:40:32.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.776+0000 7fa2df4ed640 1 -- 172.21.15.33:0/2115277441 wait complete. 2024-04-02T13:40:32.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2df4ed640 1 Processor -- start 2024-04-02T13:40:32.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2df4ed640 1 -- start start 2024-04-02T13:40:32.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2df4ed640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8100970 0x7fa2d8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2df4ed640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2d8101330 0x7fa2d81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2df4ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d8106980 0x7fa2d819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2df4ed640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8110d30 con 0x7fa2d8106980 2024-04-02T13:40:32.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2df4ed640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8110bb0 con 0x7fa2d8101330 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2df4ed640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8110eb0 con 0x7fa2d8100970 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2dda8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d8106980 0x7fa2d819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2dd289640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8100970 0x7fa2d8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2dda8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d8106980 0x7fa2d819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47294/0 (socket says 172.21.15.33:47294) 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2dd289640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8100970 0x7fa2d8198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:56506/0 (socket says 172.21.15.33:56506) 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2dda8a640 1 -- 172.21.15.33:0/106199807 learned_addr learned my addr 172.21.15.33:0/106199807 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2dca88640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2d8101330 0x7fa2d81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2dda8a640 1 -- 172.21.15.33:0/106199807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8100970 msgr2=0x7fa2d8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2dda8a640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8100970 0x7fa2d8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2dda8a640 1 -- 172.21.15.33:0/106199807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2d8101330 msgr2=0x7fa2d81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2dda8a640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2d8101330 0x7fa2d81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2dda8a640 1 -- 172.21.15.33:0/106199807 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d819d370 con 0x7fa2d8106980 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2dd289640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8100970 0x7fa2d8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.777+0000 7fa2dda8a640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d8106980 0x7fa2d819cc40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000efd0 tx=0x7fa2d000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:32.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.778+0000 7fa2ce7fc640 1 -- 172.21.15.33:0/106199807 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0003b70 con 0x7fa2d8106980 2024-04-02T13:40:32.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.778+0000 7fa2df4ed640 1 -- 172.21.15.33:0/106199807 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d819d660 con 0x7fa2d8106980 2024-04-02T13:40:32.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.778+0000 7fa2ce7fc640 1 -- 172.21.15.33:0/106199807 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa2d0003d10 con 0x7fa2d8106980 2024-04-02T13:40:32.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.778+0000 7fa2ce7fc640 1 -- 172.21.15.33:0/106199807 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00109f0 con 0x7fa2d8106980 2024-04-02T13:40:32.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.778+0000 7fa2df4ed640 1 -- 172.21.15.33:0/106199807 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d81a4ef0 con 0x7fa2d8106980 2024-04-02T13:40:32.783 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clock-bindings.txt 2024-04-02T13:40:32.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.780+0000 7fa2ce7fc640 1 -- 172.21.15.33:0/106199807 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa2d0003ec0 con 0x7fa2d8106980 2024-04-02T13:40:32.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.781+0000 7fa2e21a8740 1 Processor -- start 2024-04-02T13:40:32.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.781+0000 7fa2e21a8740 1 -- start start 2024-04-02T13:40:32.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.781+0000 7fa2ce7fc640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa2c40a3d20 0x7fa2c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.781+0000 7fa2e21a8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538600 0x5597575389e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.781+0000 7fa2e21a8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538f20 0x559757545ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.781+0000 7fa2e21a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x5597575485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.785 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clps711x-clock.txt 2024-04-02T13:40:32.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.781+0000 7fa2e21a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55975749c830 con 0x5597575461e0 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.781+0000 7fa2e21a8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559757444820 con 0x559757538600 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.781+0000 7fa2e21a8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5597574987e0 con 0x559757538f20 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.781+0000 7fa2dca88640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa2c40a3d20 0x7fa2c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.782+0000 7fa2dd289640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538600 0x5597575389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.782+0000 7fa2dd289640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538600 0x5597575389e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:59884/0 (socket says 172.21.15.33:59884) 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.782+0000 7fa2dd289640 1 -- 172.21.15.33:0/562494715 learned_addr learned my addr 172.21.15.33:0/562494715 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.782+0000 7fa2dca88640 1 --2- 172.21.15.33:0/562494715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538f20 0x559757545ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.782+0000 7fa2ce7fc640 1 -- 172.21.15.33:0/106199807 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa2d00d0e80 con 0x7fa2d8106980 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.782+0000 7fa2dda8a640 1 --2- 172.21.15.33:0/562494715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x5597575485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.782+0000 7fa2dda8a640 1 -- 172.21.15.33:0/562494715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538f20 msgr2=0x559757545ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.782+0000 7fa2dda8a640 1 --2- 172.21.15.33:0/562494715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538f20 0x559757545ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.782+0000 7fa2dda8a640 1 -- 172.21.15.33:0/562494715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538600 msgr2=0x5597575389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.782+0000 7fa2dda8a640 1 --2- 172.21.15.33:0/562494715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538600 0x5597575389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.782+0000 7fa2dda8a640 1 -- 172.21.15.33:0/562494715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55975727a7d0 con 0x5597575461e0 2024-04-02T13:40:32.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.782+0000 7fa2dd289640 1 --2- 172.21.15.33:0/562494715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538600 0x5597575389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:32.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.783+0000 7fa2dca88640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa2c40a3d20 0x7fa2c40a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8100590 tx=0x7fa2c8009b60 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:32.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.783+0000 7fa2dda8a640 1 --2- 172.21.15.33:0/562494715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x5597575485d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00ce260 tx=0x7fa2d009c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571c58ccafa3f236 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:32.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.783+0000 7fa2c37fe640 1 -- 172.21.15.33:0/562494715 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d009d1b0 con 0x5597575461e0 2024-04-02T13:40:32.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.783+0000 7fa2c37fe640 1 -- 172.21.15.33:0/562494715 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa2d009cab0 con 0x5597575461e0 2024-04-02T13:40:32.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.783+0000 7fa2c37fe640 1 -- 172.21.15.33:0/562494715 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00992b0 con 0x5597575461e0 2024-04-02T13:40:32.788 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/efm32-clock.txt 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.784+0000 7fa2e21a8740 1 -- 172.21.15.33:0/562494715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 msgr2=0x5597575485d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.784+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/562494715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x5597575485d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00ce260 tx=0x7fa2d009c4d0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.784+0000 7fa2e21a8740 1 -- 172.21.15.33:0/562494715 shutdown_connections 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.784+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/562494715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x5597575485d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.784+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/562494715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538f20 0x559757545ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.784+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/562494715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538600 0x5597575389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.784+0000 7fa2e21a8740 1 -- 172.21.15.33:0/562494715 >> 172.21.15.33:0/562494715 conn(0x559757533000 msgr2=0x5597575333e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.785+0000 7fa2e21a8740 1 -- 172.21.15.33:0/562494715 shutdown_connections 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.785+0000 7fa2e21a8740 1 -- 172.21.15.33:0/562494715 wait complete. 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.785+0000 7fa2e21a8740 1 -- start start 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.785+0000 7fa2e21a8740 1 Processor -- start 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2e21a8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538600 0x559757545b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2e21a8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x559757542ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2e21a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x559757543430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2e21a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559757548ee0 con 0x5597575461e0 2024-04-02T13:40:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2e21a8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559757457d30 con 0x559757538f20 2024-04-02T13:40:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2e21a8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559757549060 con 0x559757538600 2024-04-02T13:40:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2dd289640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538600 0x559757545b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2dd289640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538600 0x559757545b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:56524/0 (socket says 172.21.15.33:56524) 2024-04-02T13:40:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2dd289640 1 -- 172.21.15.33:0/4103313271 learned_addr learned my addr 172.21.15.33:0/4103313271 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2e21a8740 10 client.0 _ll_register_callbacks cb 0x559757457720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2dda8a640 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x559757543430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2dca88640 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x559757542ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2dd289640 1 -- 172.21.15.33:0/4103313271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 msgr2=0x559757542ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2dd289640 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x559757542ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2dd289640 1 -- 172.21.15.33:0/4103313271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 msgr2=0x559757543430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2dd289640 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x559757543430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2dd289640 1 -- 172.21.15.33:0/4103313271 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597571b03f0 con 0x559757538600 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.786+0000 7fa2dda8a640 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x559757543430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.787+0000 7fa2dd289640 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538600 0x559757545b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4036040 tx=0x7fa2d4009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1518641b944759bd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.787+0000 7fa2c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.787+0000 7fa2c0ff9640 1 -- 172.21.15.33:0/4103313271 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d402cc10 con 0x559757538600 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.787+0000 7fa2c0ff9640 1 -- 172.21.15.33:0/4103313271 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa2d402c9b0 con 0x559757538600 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.787+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4103313271 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d009cab0 con 0x559757538600 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.787+0000 7fa2c0ff9640 1 -- 172.21.15.33:0/4103313271 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa2d4003800 con 0x559757538600 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.787+0000 7fa2e21a8740 10 client.25531 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.787+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4103313271 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5597571700f0 con 0x559757538600 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.787+0000 7fa2c0ff9640 1 -- 172.21.15.33:0/4103313271 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa2d4034440 con 0x559757538600 2024-04-02T13:40:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.787+0000 7fa2c0ff9640 1 -- 172.21.15.33:0/4103313271 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa2d4049410 con 0x559757538600 2024-04-02T13:40:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.788+0000 7fa2e21a8740 10 client.25531 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.788+0000 7fa2e21a8740 10 client.25531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.788+0000 7fa2e21a8740 20 client.25531 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.788+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5597575561c0 0x559757558640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.788+0000 7fa2e21a8740 4 client.25531 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.788+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4103313271 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa2d009cab0 con 0x5597575561c0 2024-04-02T13:40:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.788+0000 7fa2dca88640 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5597575561c0 0x559757558640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.795 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/emev2-clock.txt 2024-04-02T13:40:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.788+0000 7fa2dca88640 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5597575561c0 0x559757558640 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff10ca73529deef server_cookie=e25690c3b91d1a1 in_seq=0 out_seq=0 2024-04-02T13:40:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.788+0000 7fa2c0ff9640 10 client.25531 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.789+0000 7fa2c0ff9640 1 -- 172.21.15.33:0/4103313271 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fa2d009cab0 con 0x5597575561c0 2024-04-02T13:40:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.789+0000 7fa2c0ff9640 10 client.25531 handle_command_reply: tid=0 2024-04-02T13:40:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.789+0000 7fa2e21a8740 1 client.25531 shutdown 2024-04-02T13:40:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.790+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4103313271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538600 msgr2=0x559757545b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.790+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538600 0x559757545b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4036040 tx=0x7fa2d4009560 comp rx=0 tx=0).stop 2024-04-02T13:40:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.790+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4103313271 shutdown_connections 2024-04-02T13:40:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.790+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5597575561c0 0x559757558640 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.790+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x559757543430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.790+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x559757542ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.790+0000 7fa2c0ff9640 0 client.25531 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.790+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4103313271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538600 0x559757545b40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.790+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4103313271 >> 172.21.15.33:0/4103313271 conn(0x559757533000 msgr2=0x559757534d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:32.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.790+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4103313271 shutdown_connections 2024-04-02T13:40:32.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.790+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4103313271 wait complete. 2024-04-02T13:40:32.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.790+0000 7fa2e21a8740 20 client.25531 trim_cache size 0 max 16384 2024-04-02T13:40:32.798 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos3250-clock.txt 2024-04-02T13:40:32.798 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4-clock.txt 2024-04-02T13:40:32.799 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4415-clock.txt 2024-04-02T13:40:32.802 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5250-clock.txt 2024-04-02T13:40:32.805 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5260-clock.txt 2024-04-02T13:40:32.809 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5410-clock.txt 2024-04-02T13:40:32.812 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5420-clock.txt 2024-04-02T13:40:32.816 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5440-clock.txt 2024-04-02T13:40:32.818 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos7-clock.txt 2024-04-02T13:40:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.817+0000 7fa2e21a8740 1 Processor -- start 2024-04-02T13:40:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.817+0000 7fa2e21a8740 1 -- start start 2024-04-02T13:40:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.817+0000 7fa2e21a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559757538600 0x559757534b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.817+0000 7fa2e21a8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x559757545600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.817+0000 7fa2e21a8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597575461e0 0x559757545b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2e21a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81af5d0 con 0x559757538600 2024-04-02T13:40:32.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2e21a8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81acea0 con 0x559757538f20 2024-04-02T13:40:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2e21a8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81aacd0 con 0x5597575461e0 2024-04-02T13:40:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2dda8a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597575461e0 0x559757545b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2dca88640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x559757545600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2dd289640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559757538600 0x559757534b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2dda8a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597575461e0 0x559757545b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:56532/0 (socket says 172.21.15.33:56532) 2024-04-02T13:40:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2dda8a640 1 -- 172.21.15.33:0/4153866306 learned_addr learned my addr 172.21.15.33:0/4153866306 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2dda8a640 1 -- 172.21.15.33:0/4153866306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 msgr2=0x559757545600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2dda8a640 1 --2- 172.21.15.33:0/4153866306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x559757545600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2dda8a640 1 -- 172.21.15.33:0/4153866306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559757538600 msgr2=0x559757534b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2dda8a640 1 --2- 172.21.15.33:0/4153866306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559757538600 0x559757534b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2dda8a640 1 -- 172.21.15.33:0/4153866306 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559757538d50 con 0x5597575461e0 2024-04-02T13:40:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2dca88640 1 --2- 172.21.15.33:0/4153866306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x559757545600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.818+0000 7fa2dda8a640 1 --2- 172.21.15.33:0/4153866306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597575461e0 0x559757545b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0000f80 tx=0x7fa2d00bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb285562fd9a4df server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.819+0000 7fa2c2ffd640 1 -- 172.21.15.33:0/4153866306 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d009d1b0 con 0x5597575461e0 2024-04-02T13:40:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.819+0000 7fa2c2ffd640 1 -- 172.21.15.33:0/4153866306 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa2d00a0e60 con 0x5597575461e0 2024-04-02T13:40:32.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.819+0000 7fa2c2ffd640 1 -- 172.21.15.33:0/4153866306 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0010b90 con 0x5597575461e0 2024-04-02T13:40:32.823 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-clock.txt 2024-04-02T13:40:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.820+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4153866306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597575461e0 msgr2=0x559757545b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.820+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4153866306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597575461e0 0x559757545b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0000f80 tx=0x7fa2d00bd040 comp rx=0 tx=0).stop 2024-04-02T13:40:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.820+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4153866306 shutdown_connections 2024-04-02T13:40:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.820+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4153866306 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5597575461e0 0x559757545b40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.820+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4153866306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x559757545600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.820+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4153866306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559757538600 0x559757534b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.820+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4153866306 >> 172.21.15.33:0/4153866306 conn(0x559757533000 msgr2=0x559757534450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.820+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4153866306 shutdown_connections 2024-04-02T13:40:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.820+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4153866306 wait complete. 2024-04-02T13:40:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.820+0000 7fa2e21a8740 1 -- start start 2024-04-02T13:40:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.820+0000 7fa2e21a8740 1 Processor -- start 2024-04-02T13:40:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.821+0000 7fa2e21a8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538600 0x55975754a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.821+0000 7fa2e21a8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x55975754a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.821+0000 7fa2e21a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x5597575bb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.821+0000 7fa2e21a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81ab360 con 0x5597575461e0 2024-04-02T13:40:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.821+0000 7fa2dda8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x5597575bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.821+0000 7fa2dca88640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x55975754a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.821+0000 7fa2dda8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x5597575bb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47336/0 (socket says 172.21.15.33:47336) 2024-04-02T13:40:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.821+0000 7fa2dca88640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x55975754a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:59910/0 (socket says 172.21.15.33:59910) 2024-04-02T13:40:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.821+0000 7fa2dda8a640 1 -- 172.21.15.33:0/4086310413 learned_addr learned my addr 172.21.15.33:0/4086310413 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:40:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.822+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4086310413 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81ac7d0 con 0x559757538f20 2024-04-02T13:40:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.822+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4086310413 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81aba30 con 0x559757538600 2024-04-02T13:40:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.822+0000 7fa2dd289640 1 --2- 172.21.15.33:0/4086310413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538600 0x55975754a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.822+0000 7fa2e21a8740 10 client.0 _ll_register_callbacks cb 0x7fa2d4004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.822+0000 7fa2dca88640 1 -- 172.21.15.33:0/4086310413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538600 msgr2=0x55975754a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.822+0000 7fa2dca88640 1 --2- 172.21.15.33:0/4086310413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538600 0x55975754a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.822+0000 7fa2dca88640 1 -- 172.21.15.33:0/4086310413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 msgr2=0x5597575bb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.822+0000 7fa2dca88640 1 --2- 172.21.15.33:0/4086310413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x5597575bb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.822+0000 7fa2dca88640 1 -- 172.21.15.33:0/4086310413 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597575bbcc0 con 0x559757538f20 2024-04-02T13:40:32.828 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-factor-clock.txt 2024-04-02T13:40:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.822+0000 7fa2dca88640 1 --2- 172.21.15.33:0/4086310413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x55975754a9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x559757558eb0 tx=0x7fa2c8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d3eeed7e86a548f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.822+0000 7fa2c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T13:40:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.823+0000 7fa2c0ff9640 1 -- 172.21.15.33:0/4086310413 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c80041b0 con 0x559757538f20 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.823+0000 7fa2c0ff9640 1 -- 172.21.15.33:0/4086310413 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa2c8004350 con 0x559757538f20 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.823+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4086310413 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d00a0e60 con 0x559757538f20 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.823+0000 7fa2c0ff9640 1 -- 172.21.15.33:0/4086310413 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa2c802b410 con 0x559757538f20 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.823+0000 7fa2e21a8740 10 client.25676 fetch_fsmap learned FSMap version 25 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.823+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4086310413 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5597575bcfc0 con 0x559757538f20 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.823+0000 7fa2c0ff9640 1 -- 172.21.15.33:0/4086310413 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa2d00a0e60 con 0x559757538f20 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.824+0000 7fa2c0ff9640 1 -- 172.21.15.33:0/4086310413 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa2c8003bc0 con 0x559757538f20 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.824+0000 7fa2e21a8740 10 client.25676 fetch_fsmap finished waiting for FSMap version 25 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.824+0000 7fa2e21a8740 10 client.25676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.824+0000 7fa2e21a8740 20 client.25676 populate_metadata read hostname 'smithi033' 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.824+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4086310413 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5597575561c0 0x5597575bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.824+0000 7fa2e21a8740 4 client.25676 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.824+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4086310413 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5597574a9d70 con 0x5597575561c0 2024-04-02T13:40:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.825+0000 7fa2dd289640 1 --2- 172.21.15.33:0/4086310413 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5597575561c0 0x5597575bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.825+0000 7fa2dd289640 1 --2- 172.21.15.33:0/4086310413 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5597575561c0 0x5597575bf720 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cabceab746d4833 server_cookie=b2ea4cf31d1d5bf4 in_seq=0 out_seq=0 2024-04-02T13:40:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.825+0000 7fa2c0ff9640 10 client.25676 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:32.831 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/gpio-gate-clock.txt 2024-04-02T13:40:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.825+0000 7fa2c0ff9640 1 -- 172.21.15.33:0/4086310413 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5597574a9d70 con 0x5597575561c0 2024-04-02T13:40:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.825+0000 7fa2c0ff9640 10 client.25676 handle_command_reply: tid=0 2024-04-02T13:40:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.825+0000 7fa2e21a8740 1 client.25676 shutdown 2024-04-02T13:40:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.825+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4086310413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 msgr2=0x55975754a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.825+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4086310413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x55975754a9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x559757558eb0 tx=0x7fa2c8005f90 comp rx=0 tx=0).stop 2024-04-02T13:40:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.826+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4086310413 shutdown_connections 2024-04-02T13:40:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.826+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4086310413 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5597575561c0 0x5597575bf720 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.826+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4086310413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597575461e0 0x5597575bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.826+0000 7fa2c0ff9640 0 client.25676 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:40:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.826+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4086310413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559757538f20 0x55975754a9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.826+0000 7fa2e21a8740 1 --2- 172.21.15.33:0/4086310413 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559757538600 0x55975754a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.826+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4086310413 >> 172.21.15.33:0/4086310413 conn(0x559757533000 msgr2=0x55975753e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.826+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4086310413 shutdown_connections 2024-04-02T13:40:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.826+0000 7fa2e21a8740 1 -- 172.21.15.33:0/4086310413 wait complete. 2024-04-02T13:40:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.826+0000 7fa2e21a8740 20 client.25676 trim_cache size 0 max 16384 2024-04-02T13:40:32.833 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:40:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:40:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:40:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: "d3d02c43-90ea-4711-8ee0-b9b22907f1bb": { 2024-04-02T13:40:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:40:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "d3d02c43-90ea-4711-8ee0-b9b22907f1bb", 2024-04-02T13:40:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:40:32.834 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:40:32.834 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:40:32.834 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:40:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.827+0000 7fa2df4ed640 1 -- 172.21.15.33:0/106199807 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa2c40a3d20 msgr2=0x7fa2c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.827+0000 7fa2df4ed640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa2c40a3d20 0x7fa2c40a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8100590 tx=0x7fa2c8009b60 comp rx=0 tx=0).stop 2024-04-02T13:40:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.827+0000 7fa2df4ed640 1 -- 172.21.15.33:0/106199807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d8106980 msgr2=0x7fa2d819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.827+0000 7fa2df4ed640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d8106980 0x7fa2d819cc40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000efd0 tx=0x7fa2d000c6a0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.827+0000 7fa2df4ed640 1 -- 172.21.15.33:0/106199807 shutdown_connections 2024-04-02T13:40:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.827+0000 7fa2df4ed640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa2c40a3d20 0x7fa2c40a61e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.827+0000 7fa2df4ed640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d8106980 0x7fa2d819cc40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.827+0000 7fa2df4ed640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2d8101330 0x7fa2d81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.827+0000 7fa2df4ed640 1 --2- 172.21.15.33:0/106199807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2d8100970 0x7fa2d8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.827+0000 7fa2df4ed640 1 -- 172.21.15.33:0/106199807 >> 172.21.15.33:0/106199807 conn(0x7fa2d8077b00 msgr2=0x7fa2d8068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.827+0000 7fa2df4ed640 1 -- 172.21.15.33:0/106199807 shutdown_connections 2024-04-02T13:40:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:40:32.827+0000 7fa2df4ed640 1 -- 172.21.15.33:0/106199807 wait complete. 2024-04-02T13:40:32.836 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hi3620-clock.txt 2024-04-02T13:40:32.836 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hix5hd2-clock.txt 2024-04-02T13:40:32.836 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx1-clock.txt 2024-04-02T13:40:32.839 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx21-clock.txt 2024-04-02T13:40:32.842 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx23-clock.txt 2024-04-02T13:40:32.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:32 smithi033 ceph-mon[32390]: pgmap v55: 129 pgs: 129 active+clean; 84 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.9 MiB/s wr, 48 op/s 2024-04-02T13:40:32.844 INFO:tasks.cephfs.filesystem:scrub status for tag:d3d02c43-90ea-4711-8ee0-b9b22907f1bb - {'path': '/', 'tag': 'd3d02c43-90ea-4711-8ee0-b9b22907f1bb', 'options': 'recursive,force'} 2024-04-02T13:40:32.845 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx25-clock.txt 2024-04-02T13:40:32.847 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx27-clock.txt 2024-04-02T13:40:32.850 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx28-clock.txt 2024-04-02T13:40:32.852 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx31-clock.txt 2024-04-02T13:40:32.855 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx35-clock.txt 2024-04-02T13:40:32.857 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx5-clock.txt 2024-04-02T13:40:32.860 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6q-clock.txt 2024-04-02T13:40:32.863 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sl-clock.txt 2024-04-02T13:40:32.865 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sx-clock.txt 2024-04-02T13:40:32.868 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-gate.txt 2024-04-02T13:40:32.871 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-pll.txt 2024-04-02T13:40:32.874 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/lsi,axm5516-clks.txt 2024-04-02T13:40:32.876 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,mmp2.txt 2024-04-02T13:40:32.879 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa168.txt 2024-04-02T13:40:32.881 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa910.txt 2024-04-02T13:40:32.883 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77686.txt 2024-04-02T13:40:32.885 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77802.txt 2024-04-02T13:40:32.888 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/moxa,moxart-clock.txt 2024-04-02T13:40:32.891 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt 2024-04-02T13:40:32.894 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-corediv-clock.txt 2024-04-02T13:40:32.896 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt 2024-04-02T13:40:32.899 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-gated-clock.txt 2024-04-02T13:40:32.902 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nspire-clock.txt 2024-04-02T13:40:32.904 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra114-car.txt 2024-04-02T13:40:32.907 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra124-car.txt 2024-04-02T13:40:32.909 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.txt 2024-04-02T13:40:32.912 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra30-car.txt 2024-04-02T13:40:32.915 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/prima2-clock.txt 2024-04-02T13:40:32.917 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/pxa-clock.txt 2024-04-02T13:40:32.920 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,gcc.txt 2024-04-02T13:40:32.922 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,lcc.txt 2024-04-02T13:40:32.924 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,mmcc.txt 2024-04-02T13:40:32.927 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qoriq-clock.txt 2024-04-02T13:40:32.929 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-div6-clocks.txt 2024-04-02T13:40:32.932 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt 2024-04-02T13:40:32.934 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a73a4-cpg-clocks.txt 2024-04-02T13:40:32.937 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7740-cpg-clocks.txt 2024-04-02T13:40:32.939 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7779-cpg-clocks.txt 2024-04-02T13:40:32.942 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rcar-gen2-cpg-clocks.txt 2024-04-02T13:40:32.944 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rz-cpg-clocks.txt 2024-04-02T13:40:32.947 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,sh73a0-cpg-clocks.txt 2024-04-02T13:40:32.949 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt 2024-04-02T13:40:32.951 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt 2024-04-02T13:40:32.954 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip.txt 2024-04-02T13:40:32.956 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2410-clock.txt 2024-04-02T13:40:32.959 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2412-clock.txt 2024-04-02T13:40:32.961 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2443-clock.txt 2024-04-02T13:40:32.964 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c64xx-clock.txt 2024-04-02T13:40:32.967 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt 2024-04-02T13:40:32.969 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si5351.txt 2024-04-02T13:40:32.971 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si570.txt 2024-04-02T13:40:32.973 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st,nomadik.txt 2024-04-02T13:40:32.976 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/ 2024-04-02T13:40:32.978 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-divmux.txt 2024-04-02T13:40:32.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:32 smithi145 ceph-mon[35646]: pgmap v55: 129 pgs: 129 active+clean; 84 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.9 MiB/s wr, 48 op/s 2024-04-02T13:40:32.980 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-mux.txt 2024-04-02T13:40:32.982 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-pll.txt 2024-04-02T13:40:32.984 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-prediv.txt 2024-04-02T13:40:32.987 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-vcc.txt 2024-04-02T13:40:32.989 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen.txt 2024-04-02T13:40:32.991 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,flexgen.txt 2024-04-02T13:40:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:32 smithi050 ceph-mon[35524]: pgmap v55: 129 pgs: 129 active+clean; 84 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.9 MiB/s wr, 48 op/s 2024-04-02T13:40:32.994 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,quadfs.txt 2024-04-02T13:40:32.996 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ste-u300-syscon-clock.txt 2024-04-02T13:40:33.000 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/sunxi.txt 2024-04-02T13:40:33.002 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti,cdce706.txt 2024-04-02T13:40:33.005 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti-keystone-pllctrl.txt 2024-04-02T13:40:33.007 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/ 2024-04-02T13:40:33.009 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/apll.txt 2024-04-02T13:40:33.011 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/autoidle.txt 2024-04-02T13:40:33.014 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/clockdomain.txt 2024-04-02T13:40:33.016 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/composite.txt 2024-04-02T13:40:33.019 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/divider.txt 2024-04-02T13:40:33.022 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dpll.txt 2024-04-02T13:40:33.025 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dra7-atl.txt 2024-04-02T13:40:33.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fapll.txt 2024-04-02T13:40:33.030 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fixed-factor-clock.txt 2024-04-02T13:40:33.033 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/gate.txt 2024-04-02T13:40:33.035 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/interface.txt 2024-04-02T13:40:33.038 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/mux.txt 2024-04-02T13:40:33.041 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vf610-clock.txt 2024-04-02T13:40:33.046 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vt8500.txt 2024-04-02T13:40:33.047 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/xgene.txt 2024-04-02T13:40:33.050 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/zynq-7000.txt 2024-04-02T13:40:33.053 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/ 2024-04-02T13:40:33.058 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/arm_big_little_dt.txt 2024-04-02T13:40:33.061 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-dt.txt 2024-04-02T13:40:33.065 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-exynos5440.txt 2024-04-02T13:40:33.068 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-spear.txt 2024-04-02T13:40:33.071 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/ 2024-04-02T13:40:33.073 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/amd-ccp.txt 2024-04-02T13:40:33.076 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/atmel-crypto.txt 2024-04-02T13:40:33.078 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-dcp.txt 2024-04-02T13:40:33.080 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-imx-sahara.txt 2024-04-02T13:40:33.082 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec2.txt 2024-04-02T13:40:33.085 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec4.txt 2024-04-02T13:40:33.087 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec6.txt 2024-04-02T13:40:33.090 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/mv_cesa.txt 2024-04-02T13:40:33.092 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-aes.txt 2024-04-02T13:40:33.094 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-des.txt 2024-04-02T13:40:33.096 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-sham.txt 2024-04-02T13:40:33.099 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/picochip-spacc.txt 2024-04-02T13:40:33.101 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/qcom-qce.txt 2024-04-02T13:40:33.104 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/samsung-sss.txt 2024-04-02T13:40:33.106 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/ 2024-04-02T13:40:33.109 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/ 2024-04-02T13:40:33.111 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/exynos-ppmu.txt 2024-04-02T13:40:33.113 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ 2024-04-02T13:40:33.116 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/arm-pl330.txt 2024-04-02T13:40:33.118 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-dma.txt 2024-04-02T13:40:33.120 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-xdma.txt 2024-04-02T13:40:33.122 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/bcm2835-dma.txt 2024-04-02T13:40:33.125 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/dma.txt 2024-04-02T13:40:33.127 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-edma.txt 2024-04-02T13:40:33.130 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-dma.txt 2024-04-02T13:40:33.133 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-sdma.txt 2024-04-02T13:40:33.135 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-mxs-dma.txt 2024-04-02T13:40:33.138 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/img-mdc-dma.txt 2024-04-02T13:40:33.140 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/k3dma.txt 2024-04-02T13:40:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mmp-dma.txt 2024-04-02T13:40:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/moxa,moxart-dma.txt 2024-04-02T13:40:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mpc512x-dma.txt 2024-04-02T13:40:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mv-xor.txt 2024-04-02T13:40:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/nbpfaxi.txt 2024-04-02T13:40:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_adm.txt 2024-04-02T13:40:33.157 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt 2024-04-02T13:40:33.160 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/rcar-audmapp.txt 2024-04-02T13:40:33.162 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt 2024-04-02T13:40:33.165 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/shdma.txt 2024-04-02T13:40:33.168 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sirfsoc-dma.txt 2024-04-02T13:40:33.171 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/snps-dma.txt 2024-04-02T13:40:33.173 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-coh901318.txt 2024-04-02T13:40:33.176 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-dma40.txt 2024-04-02T13:40:33.178 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sun6i-dma.txt 2024-04-02T13:40:33.181 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/tegra20-apbdma.txt 2024-04-02T13:40:33.183 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ti-edma.txt 2024-04-02T13:40:33.185 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/ 2024-04-02T13:40:33.187 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt 2024-04-02T13:40:33.189 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt 2024-04-02T13:40:33.192 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/ 2024-04-02T13:40:33.195 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/ 2024-04-02T13:40:33.197 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/marvell,dove-lcd.txt 2024-04-02T13:40:33.199 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/ 2024-04-02T13:40:33.201 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/hlcdc-dc.txt 2024-04-02T13:40:33.204 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/ 2024-04-02T13:40:33.207 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/dw_hdmi.txt 2024-04-02T13:40:33.209 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/ 2024-04-02T13:40:33.212 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/tda998x.txt 2024-04-02T13:40:33.214 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ 2024-04-02T13:40:33.217 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/fsl-imx-drm.txt 2024-04-02T13:40:33.219 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/hdmi.txt 2024-04-02T13:40:33.222 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ldb.txt 2024-04-02T13:40:33.224 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/ 2024-04-02T13:40:33.227 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/gpu.txt 2024-04-02T13:40:33.230 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/hdmi.txt 2024-04-02T13:40:33.233 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/mdp.txt 2024-04-02T13:40:33.235 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/ 2024-04-02T13:40:33.238 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/panel.txt 2024-04-02T13:40:33.240 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/slave.txt 2024-04-02T13:40:33.243 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tfp410.txt 2024-04-02T13:40:33.245 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tilcdc.txt 2024-04-02T13:40:33.250 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/eeprom.txt 2024-04-02T13:40:33.251 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/ 2024-04-02T13:40:33.253 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-palmas.txt 2024-04-02T13:40:33.255 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-rt8973a.txt 2024-04-02T13:40:33.258 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-sm5502.txt 2024-04-02T13:40:33.261 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/ 2024-04-02T13:40:33.263 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/mxsfb.txt 2024-04-02T13:40:33.265 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/sm501fb.txt 2024-04-02T13:40:33.267 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/ 2024-04-02T13:40:33.271 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/altera-socfpga-fpga-mgr.txt 2024-04-02T13:40:33.274 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/ 2024-04-02T13:40:33.275 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.txt 2024-04-02T13:40:33.278 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/ 2024-04-02T13:40:33.281 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/8xxx_gpio.txt 2024-04-02T13:40:33.284 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/abilis,tb10x-gpio.txt 2024-04-02T13:40:33.287 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cavium-octeon-gpio.txt 2024-04-02T13:40:33.289 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cirrus,clps711x-mctrl-gpio.txt 2024-04-02T13:40:33.292 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fsl-imx-gpio.txt 2024-04-02T13:40:33.294 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fujitsu,mb86s70-gpio.txt 2024-04-02T13:40:33.296 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74x164.txt 2024-04-02T13:40:33.300 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74xx-mmio.txt 2024-04-02T13:40:33.302 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-adnp.txt 2024-04-02T13:40:33.304 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-bcm-kona.txt 2024-04-02T13:40:33.307 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-clps711x.txt 2024-04-02T13:40:33.310 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-davinci.txt 2024-04-02T13:40:33.312 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-dsp-keystone.txt 2024-04-02T13:40:33.315 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-fan.txt 2024-04-02T13:40:33.317 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-grgpio.txt 2024-04-02T13:40:33.319 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-lp3943.txt 2024-04-02T13:40:33.322 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-max732x.txt 2024-04-02T13:40:33.324 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mcp23s08.txt 2024-04-02T13:40:33.327 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mm-lantiq.txt 2024-04-02T13:40:33.329 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-msm.txt 2024-04-02T13:40:33.332 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mvebu.txt 2024-04-02T13:40:33.334 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mxs.txt 2024-04-02T13:40:33.337 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-nmk.txt 2024-04-02T13:40:33.339 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-omap.txt 2024-04-02T13:40:33.341 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-palmas.txt 2024-04-02T13:40:33.344 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pca953x.txt 2024-04-02T13:40:33.346 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pcf857x.txt 2024-04-02T13:40:33.348 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-poweroff.txt 2024-04-02T13:40:33.350 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-restart.txt 2024-04-02T13:40:33.353 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-samsung.txt 2024-04-02T13:40:33.355 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stericsson-coh901.txt 2024-04-02T13:40:33.358 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stmpe.txt 2024-04-02T13:40:33.360 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stp-xway.txt 2024-04-02T13:40:33.362 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-sx150x.txt 2024-04-02T13:40:33.365 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-twl4030.txt 2024-04-02T13:40:33.367 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090-pdc.txt 2024-04-02T13:40:33.370 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090.txt 2024-04-02T13:40:33.372 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-vf610.txt 2024-04-02T13:40:33.375 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene-sb.txt 2024-04-02T13:40:33.377 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene.txt 2024-04-02T13:40:33.380 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xilinx.txt 2024-04-02T13:40:33.382 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zevio.txt 2024-04-02T13:40:33.384 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zynq.txt 2024-04-02T13:40:33.387 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio.txt 2024-04-02T13:40:33.390 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_atmel.txt 2024-04-02T13:40:33.392 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_lpc32xx.txt 2024-04-02T13:40:33.395 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/moxa,moxart-gpio.txt 2024-04-02T13:40:33.398 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/mrvl-gpio.txt 2024-04-02T13:40:33.400 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/nvidia,tegra20-gpio.txt 2024-04-02T13:40:33.403 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/pl061-gpio.txt 2024-04-02T13:40:33.405 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt 2024-04-02T13:40:33.407 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/snps-dwapb-gpio.txt 2024-04-02T13:40:33.409 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/sodaville.txt 2024-04-02T13:40:33.412 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/spear_spics.txt 2024-04-02T13:40:33.414 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/ 2024-04-02T13:40:33.417 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,gk20a.txt 2024-04-02T13:40:33.419 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,tegra20-host1x.txt 2024-04-02T13:40:33.421 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-g2d.txt 2024-04-02T13:40:33.425 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-rotator.txt 2024-04-02T13:40:33.427 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/st,stih4xx.txt 2024-04-02T13:40:33.430 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/graph.txt 2024-04-02T13:40:33.434 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/ 2024-04-02T13:40:33.436 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/hid-over-i2c.txt 2024-04-02T13:40:33.439 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/ 2024-04-02T13:40:33.442 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/client-devices.txt 2024-04-02T13:40:33.445 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/nokia-modem.txt 2024-04-02T13:40:33.448 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/omap-ssi.txt 2024-04-02T13:40:33.455 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ 2024-04-02T13:40:33.455 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ads1015.txt 2024-04-02T13:40:33.456 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/g762.txt 2024-04-02T13:40:33.458 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ibmpowernv.txt 2024-04-02T13:40:33.460 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/lm90.txt 2024-04-02T13:40:33.463 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ltc2978.txt 2024-04-02T13:40:33.466 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ntc_thermistor.txt 2024-04-02T13:40:33.468 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/pwm-fan.txt 2024-04-02T13:40:33.471 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/vexpress.txt 2024-04-02T13:40:33.473 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/ 2024-04-02T13:40:33.476 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/atmel-trng.txt 2024-04-02T13:40:33.478 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/omap_rng.txt 2024-04-02T13:40:33.480 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/timeriomem_rng.txt 2024-04-02T13:40:33.482 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ 2024-04-02T13:40:33.485 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,bcm2835-i2c.txt 2024-04-02T13:40:33.487 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,iproc-i2c.txt 2024-04-02T13:40:33.489 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt 2024-04-02T13:40:33.492 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-at91.txt 2024-04-02T13:40:33.494 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-axxia.txt 2024-04-02T13:40:33.497 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-bcm-kona.txt 2024-04-02T13:40:33.499 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cadence.txt 2024-04-02T13:40:33.502 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cbus-gpio.txt 2024-04-02T13:40:33.504 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cros-ec-tunnel.txt 2024-04-02T13:40:33.507 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-davinci.txt 2024-04-02T13:40:33.510 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-designware.txt 2024-04-02T13:40:33.512 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-efm32.txt 2024-04-02T13:40:33.515 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt 2024-04-02T13:40:33.517 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-gpio.txt 2024-04-02T13:40:33.519 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-hix5hd2.txt 2024-04-02T13:40:33.522 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-img-scb.txt 2024-04-02T13:40:33.524 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-imx.txt 2024-04-02T13:40:33.527 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-meson.txt 2024-04-02T13:40:33.529 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mpc.txt 2024-04-02T13:40:33.532 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-gpio.txt 2024-04-02T13:40:33.534 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pca954x.txt 2024-04-02T13:40:33.537 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pinctrl.txt 2024-04-02T13:40:33.539 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux.txt 2024-04-02T13:40:33.542 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt 2024-04-02T13:40:33.545 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mxs.txt 2024-04-02T13:40:33.547 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-nomadik.txt 2024-04-02T13:40:33.550 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-ocores.txt 2024-04-02T13:40:33.552 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-octeon.txt 2024-04-02T13:40:33.555 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-omap.txt 2024-04-02T13:40:33.557 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-opal.txt 2024-04-02T13:40:33.560 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pnx.txt 2024-04-02T13:40:33.562 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa-pci-ce4100.txt 2024-04-02T13:40:33.565 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa.txt 2024-04-02T13:40:33.567 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rcar.txt 2024-04-02T13:40:33.570 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-riic.txt 2024-04-02T13:40:33.572 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rk3x.txt 2024-04-02T13:40:33.575 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-s3c2410.txt 2024-04-02T13:40:33.577 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sh_mobile.txt 2024-04-02T13:40:33.580 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sirf.txt 2024-04-02T13:40:33.582 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st-ddci2c.txt 2024-04-02T13:40:33.585 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st.txt 2024-04-02T13:40:33.588 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sunxi-p2wi.txt 2024-04-02T13:40:33.590 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-versatile.txt 2024-04-02T13:40:33.592 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-vt8500.txt 2024-04-02T13:40:33.595 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-xiic.txt 2024-04-02T13:40:33.597 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina209.txt 2024-04-02T13:40:33.600 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina2xx.txt 2024-04-02T13:40:33.602 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/max6697.txt 2024-04-02T13:40:33.604 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/nvidia,tegra20-i2c.txt 2024-04-02T13:40:33.607 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/qcom,i2c-qup.txt 2024-04-02T13:40:33.609 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ti,bq32k.txt 2024-04-02T13:40:33.612 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/trivial-devices.txt 2024-04-02T13:40:33.614 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/ 2024-04-02T13:40:33.616 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/ 2024-04-02T13:40:33.618 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/bma180.txt 2024-04-02T13:40:33.620 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/ 2024-04-02T13:40:33.622 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/at91_adc.txt 2024-04-02T13:40:33.624 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/cc10001_adc.txt 2024-04-02T13:40:33.627 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/max1027-adc.txt 2024-04-02T13:40:33.630 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/nuvoton-nau7802.txt 2024-04-02T13:40:33.632 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-iadc.txt 2024-04-02T13:40:33.634 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-vadc.txt 2024-04-02T13:40:33.637 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt 2024-04-02T13:40:33.639 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/twl4030-madc.txt 2024-04-02T13:40:33.642 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/vf610-adc.txt 2024-04-02T13:40:33.645 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/xilinx-xadc.txt 2024-04-02T13:40:33.648 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ 2024-04-02T13:40:33.651 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ad7303.txt 2024-04-02T13:40:33.654 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/max5821.txt 2024-04-02T13:40:33.660 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/ 2024-04-02T13:40:33.660 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/adf4350.txt 2024-04-02T13:40:33.663 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/ 2024-04-02T13:40:33.666 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/dht11.txt 2024-04-02T13:40:33.668 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/iio-bindings.txt 2024-04-02T13:40:33.671 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/ 2024-04-02T13:40:33.673 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/apds9300.txt 2024-04-02T13:40:33.675 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/cm36651.txt 2024-04-02T13:40:33.678 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/gp2ap020a00f.txt 2024-04-02T13:40:33.680 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/tsl2563.txt 2024-04-02T13:40:33.683 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ 2024-04-02T13:40:33.685 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt 2024-04-02T13:40:33.687 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/hmc5843.txt 2024-04-02T13:40:33.690 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/ 2024-04-02T13:40:33.692 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/as3935.txt 2024-04-02T13:40:33.695 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/sensorhub.txt 2024-04-02T13:40:33.698 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/st-sensors.txt 2024-04-02T13:40:33.700 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ 2024-04-02T13:40:33.703 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ads7846.txt 2024-04-02T13:40:33.705 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/atmel,maxtouch.txt 2024-04-02T13:40:33.708 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cap11xx.txt 2024-04-02T13:40:33.710 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/clps711x-keypad.txt 2024-04-02T13:40:33.712 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cros-ec-keyb.txt 2024-04-02T13:40:33.715 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/e3x0-button.txt 2024-04-02T13:40:33.717 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elan_i2c.txt 2024-04-02T13:40:33.719 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elants_i2c.txt 2024-04-02T13:40:33.722 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/fsl-mma8450.txt 2024-04-02T13:40:33.724 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-beeper.txt 2024-04-02T13:40:33.726 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys-polled.txt 2024-04-02T13:40:33.729 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys.txt 2024-04-02T13:40:33.731 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-matrix-keypad.txt 2024-04-02T13:40:33.733 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/imx-keypad.txt 2024-04-02T13:40:33.736 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/input-reset.txt 2024-04-02T13:40:33.738 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/lpc32xx-key.txt 2024-04-02T13:40:33.741 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/matrix-keymap.txt 2024-04-02T13:40:33.744 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/nvidia,tegra20-kbc.txt 2024-04-02T13:40:33.746 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/omap-keypad.txt 2024-04-02T13:40:33.748 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ps2keyb-mouse-apbps2.txt 2024-04-02T13:40:33.751 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pwm-beeper.txt 2024-04-02T13:40:33.753 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pxa27x-keypad.txt 2024-04-02T13:40:33.756 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-keypad.txt 2024-04-02T13:40:33.758 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-pwrkey.txt 2024-04-02T13:40:33.761 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-vib.txt 2024-04-02T13:40:33.763 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/regulator-haptic.txt 2024-04-02T13:40:33.765 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/rotary-encoder.txt 2024-04-02T13:40:33.768 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/samsung-keypad.txt 2024-04-02T13:40:33.770 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/spear-keyboard.txt 2024-04-02T13:40:33.773 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/st-keyscan.txt 2024-04-02T13:40:33.776 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/stmpe-keypad.txt 2024-04-02T13:40:33.780 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/sun4i-lradc-keys.txt 2024-04-02T13:40:33.783 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tca8418_keypad.txt 2024-04-02T13:40:33.785 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv260x.txt 2024-04-02T13:40:33.788 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv2667.txt 2024-04-02T13:40:33.790 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,nspire-keypad.txt 2024-04-02T13:40:33.792 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,palmas-pwrbutton.txt 2024-04-02T13:40:33.795 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ 2024-04-02T13:40:33.797 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/auo_pixcir_ts.txt 2024-04-02T13:40:33.799 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/bu21013.txt 2024-04-02T13:40:33.802 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/edt-ft5x06.txt 2024-04-02T13:40:33.804 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/egalax-ts.txt 2024-04-02T13:40:33.806 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/lpc32xx-tsc.txt 2024-04-02T13:40:33.809 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/mms114.txt 2024-04-02T13:40:33.812 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/pixcir_i2c_ts.txt 2024-04-02T13:40:33.815 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sitronix-st1232.txt 2024-04-02T13:40:33.818 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/stmpe.txt 2024-04-02T13:40:33.821 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sun4i.txt 2024-04-02T13:40:33.823 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ti-tsc-adc.txt 2024-04-02T13:40:33.826 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/touchscreen.txt 2024-04-02T13:40:33.828 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2005.txt 2024-04-02T13:40:33.831 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2007.txt 2024-04-02T13:40:33.833 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/zforce_ts.txt 2024-04-02T13:40:33.836 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tps65218-pwrbutton.txt 2024-04-02T13:40:33.839 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-keypad.txt 2024-04-02T13:40:33.842 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-pwrbutton.txt 2024-04-02T13:40:33.844 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ 2024-04-02T13:40:33.847 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/abilis,tb10x-ictl.txt 2024-04-02T13:40:33.849 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun4i-ic.txt 2024-04-02T13:40:33.851 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun67i-sc-nmi.txt 2024-04-02T13:40:33.853 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/atmel,aic.txt 2024-04-02T13:40:33.855 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm2835-armctrl-ic.txt 2024-04-02T13:40:33.863 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm7120-l2-intc.txt 2024-04-02T13:40:33.863 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,l2-intc.txt 2024-04-02T13:40:33.864 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/cirrus,clps711x-intc.txt 2024-04-02T13:40:33.865 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/digicolor-ic.txt 2024-04-02T13:40:33.868 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/interrupts.txt 2024-04-02T13:40:33.870 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/lsi,zevio-intc.txt 2024-04-02T13:40:33.873 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt 2024-04-02T13:40:33.875 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,orion-intc.txt 2024-04-02T13:40:33.878 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/mips-gic.txt 2024-04-02T13:40:33.880 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/opencores,or1k-pic.txt 2024-04-02T13:40:33.883 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,intc-irqpin.txt 2024-04-02T13:40:33.885 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.txt 2024-04-02T13:40:33.887 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/samsung,s3c24xx-irq.txt 2024-04-02T13:40:33.890 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt 2024-04-02T13:40:33.893 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,keystone-irq.txt 2024-04-02T13:40:33.895 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,omap-intc-irq.txt 2024-04-02T13:40:33.897 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ 2024-04-02T13:40:33.900 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/arm,smmu.txt 2024-04-02T13:40:33.902 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/iommu.txt 2024-04-02T13:40:33.904 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra20-gart.txt 2024-04-02T13:40:33.907 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra30-smmu.txt 2024-04-02T13:40:33.909 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/renesas,ipmmu-vmsa.txt 2024-04-02T13:40:33.912 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/rockchip,iommu.txt 2024-04-02T13:40:33.914 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt 2024-04-02T13:40:33.916 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ti,omap-iommu.txt 2024-04-02T13:40:33.919 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/ 2024-04-02T13:40:33.921 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/common.txt 2024-04-02T13:40:33.923 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-gpio.txt 2024-04-02T13:40:33.926 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp55xx.txt 2024-04-02T13:40:33.928 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp8860.txt 2024-04-02T13:40:33.930 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-ns2.txt 2024-04-02T13:40:33.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:33 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:40:33.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:33 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:33.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:33 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:40:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:33 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:40:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:33 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:33.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:33 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:40:34.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:33 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:40:34.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:33 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:34.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:33 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:40:34.142 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-pwm.txt 2024-04-02T13:40:34.144 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/pca963x.txt 2024-04-02T13:40:34.146 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/register-bit-led.txt 2024-04-02T13:40:34.148 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/tca6507.txt 2024-04-02T13:40:34.151 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/ 2024-04-02T13:40:34.155 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2-timings.txt 2024-04-02T13:40:34.157 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2.txt 2024-04-02T13:40:34.160 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/ 2024-04-02T13:40:34.162 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/altera-mailbox.txt 2024-04-02T13:40:34.164 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/mailbox.txt 2024-04-02T13:40:34.167 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt 2024-04-02T13:40:34.170 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/marvell.txt 2024-04-02T13:40:34.173 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ 2024-04-02T13:40:34.175 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/atmel-isi.txt 2024-04-02T13:40:34.177 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/coda.txt 2024-04-02T13:40:34.180 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-fimc-lite.txt 2024-04-02T13:40:34.182 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-jpeg-codec.txt 2024-04-02T13:40:34.185 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos4-fimc-is.txt 2024-04-02T13:40:34.187 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos5-gsc.txt 2024-04-02T13:40:34.190 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/gpio-ir-receiver.txt 2024-04-02T13:40:34.192 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/hix5hd2-ir.txt 2024-04-02T13:40:34.195 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ 2024-04-02T13:40:34.197 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7343.txt 2024-04-02T13:40:34.199 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7604.txt 2024-04-02T13:40:34.202 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9m111.txt 2024-04-02T13:40:34.205 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9p031.txt 2024-04-02T13:40:34.207 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/nokia,smia.txt 2024-04-02T13:40:34.210 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ths8200.txt 2024-04-02T13:40:34.213 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp514x.txt 2024-04-02T13:40:34.215 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp7002.txt 2024-04-02T13:40:34.218 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/img-ir-rev1.txt 2024-04-02T13:40:34.220 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/meson-ir.txt 2024-04-02T13:40:34.223 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/pxa-camera.txt 2024-04-02T13:40:34.225 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/rcar_vin.txt 2024-04-02T13:40:34.228 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/renesas,vsp1.txt 2024-04-02T13:40:34.230 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/s5p-mfc.txt 2024-04-02T13:40:34.232 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-fimc.txt 2024-04-02T13:40:34.236 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-mipi-csis.txt 2024-04-02T13:40:34.237 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5c73m3.txt 2024-04-02T13:40:34.240 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k5baf.txt 2024-04-02T13:40:34.242 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k6a3.txt 2024-04-02T13:40:34.245 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sh_mobile_ceu.txt 2024-04-02T13:40:34.247 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/si4713.txt 2024-04-02T13:40:34.250 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/st-rc.txt 2024-04-02T13:40:34.252 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sunxi-ir.txt 2024-04-02T13:40:34.255 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ti-am437x-vpfe.txt 2024-04-02T13:40:34.257 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/video-interfaces.txt 2024-04-02T13:40:34.260 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ 2024-04-02T13:40:34.262 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ 2024-04-02T13:40:34.263 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt 2024-04-02T13:40:34.266 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-devbus.txt 2024-04-02T13:40:34.269 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-sdram-controller.txt 2024-04-02T13:40:34.271 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra-mc.txt 2024-04-02T13:40:34.274 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/renesas-memory-controllers.txt 2024-04-02T13:40:34.277 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/synopsys.txt 2024-04-02T13:40:34.280 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti-aemif.txt 2024-04-02T13:40:34.282 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/ 2024-04-02T13:40:34.284 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/emif.txt 2024-04-02T13:40:34.286 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/ 2024-04-02T13:40:34.289 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta-intc.txt 2024-04-02T13:40:34.291 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta.txt 2024-04-02T13:40:34.293 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/pdc-intc.txt 2024-04-02T13:40:34.295 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ 2024-04-02T13:40:34.298 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/88pm860x.txt 2024-04-02T13:40:34.300 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ab8500.txt 2024-04-02T13:40:34.302 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/arizona.txt 2024-04-02T13:40:34.304 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3711.txt 2024-04-02T13:40:34.307 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3722.txt 2024-04-02T13:40:34.309 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-gpbr.txt 2024-04-02T13:40:34.311 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-hlcdc.txt 2024-04-02T13:40:34.314 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-matrix.txt 2024-04-02T13:40:34.317 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-smc.txt 2024-04-02T13:40:34.319 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bcm590xx.txt 2024-04-02T13:40:34.322 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bfticu.txt 2024-04-02T13:40:34.324 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/cros-ec.txt 2024-04-02T13:40:34.326 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9052-i2c.txt 2024-04-02T13:40:34.329 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9055.txt 2024-04-02T13:40:34.331 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9063.txt 2024-04-02T13:40:34.334 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/hi6421.txt 2024-04-02T13:40:34.336 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/lp3943.txt 2024-04-02T13:40:34.339 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max14577.txt 2024-04-02T13:40:34.341 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77686.txt 2024-04-02T13:40:34.343 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77693.txt 2024-04-02T13:40:34.346 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8925.txt 2024-04-02T13:40:34.348 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8998.txt 2024-04-02T13:40:34.350 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/mc13xxx.txt 2024-04-02T13:40:34.352 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-host.txt 2024-04-02T13:40:34.355 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-tll.txt 2024-04-02T13:40:34.358 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/palmas.txt 2024-04-02T13:40:34.360 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom,spmi-pmic.txt 2024-04-02T13:40:34.363 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-pm8xxx.txt 2024-04-02T13:40:34.365 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-rpm.txt 2024-04-02T13:40:34.367 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qriox.txt 2024-04-02T13:40:34.370 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rk808.txt 2024-04-02T13:40:34.373 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rn5t618.txt 2024-04-02T13:40:34.375 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mpa01.txt 2024-04-02T13:40:34.377 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mps11.txt 2024-04-02T13:40:34.379 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/stmpe.txt 2024-04-02T13:40:34.382 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/sun6i-prcm.txt 2024-04-02T13:40:34.384 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/syscon.txt 2024-04-02T13:40:34.387 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tc3589x.txt 2024-04-02T13:40:34.389 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ti-keystone-devctrl.txt 2024-04-02T13:40:34.391 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps6507x.txt 2024-04-02T13:40:34.394 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps65910.txt 2024-04-02T13:40:34.396 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl-familly.txt 2024-04-02T13:40:34.399 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-audio.txt 2024-04-02T13:40:34.401 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-power.txt 2024-04-02T13:40:34.404 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl6040.txt 2024-04-02T13:40:34.406 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/ 2024-04-02T13:40:34.409 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/ 2024-04-02T13:40:34.410 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/mipi-dsi-bus.txt 2024-04-02T13:40:34.413 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/nvidia,tegra114-mipi.txt 2024-04-02T13:40:34.416 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ 2024-04-02T13:40:34.418 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/ 2024-04-02T13:40:34.420 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bcm3384-intc.txt 2024-04-02T13:40:34.422 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bmips.txt 2024-04-02T13:40:34.424 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/cm-dsl.txt 2024-04-02T13:40:34.427 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/usb.txt 2024-04-02T13:40:34.429 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ 2024-04-02T13:40:34.432 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/bootbus.txt 2024-04-02T13:40:34.434 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/cib.txt 2024-04-02T13:40:34.436 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu.txt 2024-04-02T13:40:34.438 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu2.txt 2024-04-02T13:40:34.441 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/dma-engine.txt 2024-04-02T13:40:34.444 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/uctl.txt 2024-04-02T13:40:34.446 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cpu_irq.txt 2024-04-02T13:40:34.450 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ralink.txt 2024-04-02T13:40:34.452 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ 2024-04-02T13:40:34.454 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/allwinner,sunxi-sid.txt 2024-04-02T13:40:34.457 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/arm-charlcd.txt 2024-04-02T13:40:34.459 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/at25.txt 2024-04-02T13:40:34.461 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/atmel-ssc.txt 2024-04-02T13:40:34.463 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/bmp085.txt 2024-04-02T13:40:34.466 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt 2024-04-02T13:40:34.469 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ifm-csi.txt 2024-04-02T13:40:34.471 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/lis302.txt 2024-04-02T13:40:34.473 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.txt 2024-04-02T13:40:34.475 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/smc.txt 2024-04-02T13:40:34.477 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/sram.txt 2024-04-02T13:40:34.480 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ti,dac7512.txt 2024-04-02T13:40:34.483 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ 2024-04-02T13:40:34.485 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/arasan,sdhci.txt 2024-04-02T13:40:34.487 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/atmel-hsmci.txt 2024-04-02T13:40:34.490 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/brcm,bcm2835-sdhci.txt 2024-04-02T13:40:34.492 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/davinci_mmc.txt 2024-04-02T13:40:34.495 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/exynos-dw-mshc.txt 2024-04-02T13:40:34.497 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-esdhc.txt 2024-04-02T13:40:34.499 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt 2024-04-02T13:40:34.502 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-mmc.txt 2024-04-02T13:40:34.505 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/img-dw-mshc.txt 2024-04-02T13:40:34.508 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/k3-dw-mshc.txt 2024-04-02T13:40:34.510 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/kona-sdhci.txt 2024-04-02T13:40:34.513 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt 2024-04-02T13:40:34.516 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-simple.txt 2024-04-02T13:40:34.519 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-spi-slot.txt 2024-04-02T13:40:34.521 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc.txt 2024-04-02T13:40:34.524 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmci.txt 2024-04-02T13:40:34.526 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/moxa,moxart-mmc.txt 2024-04-02T13:40:34.528 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mxs-mmc.txt 2024-04-02T13:40:34.531 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.txt 2024-04-02T13:40:34.533 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/orion-sdio.txt 2024-04-02T13:40:34.536 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/pxa-mmc.txt 2024-04-02T13:40:34.538 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/renesas,mmcif.txt 2024-04-02T13:40:34.540 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.txt 2024-04-02T13:40:34.543 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/samsung-sdhci.txt 2024-04-02T13:40:34.545 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-dove.txt 2024-04-02T13:40:34.548 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-fujitsu.txt 2024-04-02T13:40:34.550 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-msm.txt 2024-04-02T13:40:34.553 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-pxa.txt 2024-04-02T13:40:34.556 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-sirf.txt 2024-04-02T13:40:34.558 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-spear.txt 2024-04-02T13:40:34.561 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-st.txt 2024-04-02T13:40:34.563 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/socfpga-dw-mshc.txt 2024-04-02T13:40:34.567 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sunxi-mmc.txt 2024-04-02T13:40:34.569 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt 2024-04-02T13:40:34.572 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap-hsmmc.txt 2024-04-02T13:40:34.575 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap.txt 2024-04-02T13:40:34.577 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/tmio_mmc.txt 2024-04-02T13:40:34.579 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/usdhi6rol0.txt 2024-04-02T13:40:34.582 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/vt8500-sdmmc.txt 2024-04-02T13:40:34.584 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/ 2024-04-02T13:40:34.586 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/arm-versatile.txt 2024-04-02T13:40:34.588 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-dataflash.txt 2024-04-02T13:40:34.591 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-nand.txt 2024-04-02T13:40:34.593 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/davinci-nand.txt 2024-04-02T13:40:34.596 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/denali-nand.txt 2024-04-02T13:40:34.598 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/diskonchip.txt 2024-04-02T13:40:34.601 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/elm.txt 2024-04-02T13:40:34.603 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/flctl-nand.txt 2024-04-02T13:40:34.605 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-quadspi.txt 2024-04-02T13:40:34.608 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-upm-nand.txt 2024-04-02T13:40:34.610 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsmc-nand.txt 2024-04-02T13:40:34.612 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpio-control-nand.txt 2024-04-02T13:40:34.614 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nand.txt 2024-04-02T13:40:34.617 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nor.txt 2024-04-02T13:40:34.619 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-onenand.txt 2024-04-02T13:40:34.622 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmi-nand.txt 2024-04-02T13:40:34.624 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/hisi504-nand.txt 2024-04-02T13:40:34.626 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-mlc.txt 2024-04-02T13:40:34.629 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-slc.txt 2024-04-02T13:40:34.631 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/m25p80.txt 2024-04-02T13:40:34.634 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mtd-physmap.txt 2024-04-02T13:40:34.637 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mxc-nand.txt 2024-04-02T13:40:34.639 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/nand.txt 2024-04-02T13:40:34.641 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/orion-nand.txt 2024-04-02T13:40:34.644 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/partition.txt 2024-04-02T13:40:34.646 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt 2024-04-02T13:40:34.649 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/spear_smi.txt 2024-04-02T13:40:34.651 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/st-fsm.txt 2024-04-02T13:40:34.653 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/sunxi-nand.txt 2024-04-02T13:40:34.656 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ 2024-04-02T13:40:34.658 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-emac.txt 2024-04-02T13:40:34.660 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-mdio.txt 2024-04-02T13:40:34.664 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.txt 2024-04-02T13:40:34.666 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/altera_tse.txt 2024-04-02T13:40:34.668 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe-phy.txt 2024-04-02T13:40:34.671 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe.txt 2024-04-02T13:40:34.673 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/apm-xgene-enet.txt 2024-04-02T13:40:34.677 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/arc_emac.txt 2024-04-02T13:40:34.678 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcm87xx.txt 2024-04-02T13:40:34.680 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcmgenet.txt 2024-04-02T13:40:34.683 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-mdio-unimac.txt 2024-04-02T13:40:34.685 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-sf2.txt 2024-04-02T13:40:34.687 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-systemport.txt 2024-04-02T13:40:34.690 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/calxeda-xgmac.txt 2024-04-02T13:40:34.693 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/ 2024-04-02T13:40:34.695 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/atmel-can.txt 2024-04-02T13:40:34.698 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/c_can.txt 2024-04-02T13:40:34.700 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/cc770.txt 2024-04-02T13:40:34.703 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/fsl-flexcan.txt 2024-04-02T13:40:34.705 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/grcan.txt 2024-04-02T13:40:34.707 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/m_can.txt 2024-04-02T13:40:34.709 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt 2024-04-02T13:40:34.712 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/mpc5xxx-mscan.txt 2024-04-02T13:40:34.714 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/rcar_can.txt 2024-04-02T13:40:34.717 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/sja1000.txt 2024-04-02T13:40:34.719 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/xilinx_can.txt 2024-04-02T13:40:34.722 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mdio.txt 2024-04-02T13:40:34.725 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mix.txt 2024-04-02T13:40:34.727 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-pip.txt 2024-04-02T13:40:34.730 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cdns-emac.txt 2024-04-02T13:40:34.732 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw-phy-sel.txt 2024-04-02T13:40:34.734 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw.txt 2024-04-02T13:40:34.737 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davicom-dm9000.txt 2024-04-02T13:40:34.739 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci-mdio.txt 2024-04-02T13:40:34.741 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci_emac.txt 2024-04-02T13:40:34.744 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/ 2024-04-02T13:40:34.746 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/dsa.txt 2024-04-02T13:40:34.748 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/emac_rockchip.txt 2024-04-02T13:40:34.751 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ethernet.txt 2024-04-02T13:40:34.753 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fixed-link.txt 2024-04-02T13:40:34.756 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-fec.txt 2024-04-02T13:40:34.758 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-tsec-phy.txt 2024-04-02T13:40:34.761 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/gpmc-eth.txt 2024-04-02T13:40:34.763 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hip04-net.txt 2024-04-02T13:40:34.765 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hix5hd2-gmac.txt 2024-04-02T13:40:34.768 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/ 2024-04-02T13:40:34.770 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/at86rf230.txt 2024-04-02T13:40:34.772 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/cc2520.txt 2024-04-02T13:40:34.774 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/keystone-netcp.txt 2024-04-02T13:40:34.781 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/lpc-eth.txt 2024-04-02T13:40:34.784 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/macb.txt 2024-04-02T13:40:34.786 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt 2024-04-02T13:40:34.789 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-mdio.txt 2024-04-02T13:40:34.791 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-net.txt 2024-04-02T13:40:34.793 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pp2.txt 2024-04-02T13:40:34.796 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pxa168.txt 2024-04-02T13:40:34.799 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-gpio.txt 2024-04-02T13:40:34.802 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-gpio.txt 2024-04-02T13:40:34.804 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-mmioreg.txt 2024-04-02T13:40:34.807 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux.txt 2024-04-02T13:40:34.809 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/meson-dwmac.txt 2024-04-02T13:40:34.811 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ks8851.txt 2024-04-02T13:40:34.814 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ksz90x1.txt 2024-04-02T13:40:34.817 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel.txt 2024-04-02T13:40:34.819 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/moxa,moxart-mac.txt 2024-04-02T13:40:34.821 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/ 2024-04-02T13:40:34.823 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/pn544.txt 2024-04-02T13:40:34.825 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfca.txt 2024-04-02T13:40:34.828 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfcb.txt 2024-04-02T13:40:34.831 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/trf7970a.txt 2024-04-02T13:40:34.833 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/opencores-ethoc.txt 2024-04-02T13:40:34.837 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/phy.txt 2024-04-02T13:40:34.838 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/qca-qca7000-spi.txt 2024-04-02T13:40:34.841 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/rockchip-dwmac.txt 2024-04-02T13:40:34.843 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/samsung-sxgbe.txt 2024-04-02T13:40:34.846 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sh_eth.txt 2024-04-02T13:40:34.848 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc-lan91c111.txt 2024-04-02T13:40:34.851 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc911x.txt 2024-04-02T13:40:34.853 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/socfpga-dwmac.txt 2024-04-02T13:40:34.855 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sti-dwmac.txt 2024-04-02T13:40:34.858 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/stmmac.txt 2024-04-02T13:40:34.860 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-rhine.txt 2024-04-02T13:40:34.862 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-velocity.txt 2024-04-02T13:40:34.864 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ 2024-04-02T13:40:34.866 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/brcm,bcm43xx-fmac.txt 2024-04-02T13:40:34.868 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt 2024-04-02T13:40:34.871 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ti,wl1251.txt 2024-04-02T13:40:34.873 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/ 2024-04-02T13:40:34.877 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/nios2.txt 2024-04-02T13:40:34.879 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/timer.txt 2024-04-02T13:40:34.881 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/ 2024-04-02T13:40:34.884 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/nvidia,nvec.txt 2024-04-02T13:40:34.886 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/open-pic.txt 2024-04-02T13:40:34.889 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/ 2024-04-02T13:40:34.892 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101aw03.txt 2024-04-02T13:40:34.894 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101xtn01.txt 2024-04-02T13:40:34.896 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b116xw03.txt 2024-04-02T13:40:34.899 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133htn01.txt 2024-04-02T13:40:34.901 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133xtn01.txt 2024-04-02T13:40:34.904 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/avic,tm070ddh03.txt 2024-04-02T13:40:34.907 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wa01a.txt 2024-04-02T13:40:34.909 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wb03.txt 2024-04-02T13:40:34.912 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et057090dhu.txt 2024-04-02T13:40:34.914 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et070080dh6.txt 2024-04-02T13:40:34.917 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,etm0700g0dh6.txt 2024-04-02T13:40:34.919 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/foxlink,fl500wvr00-a0t.txt 2024-04-02T13:40:34.921 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/giantplus,gpg482739qs5.txt 2024-04-02T13:40:34.924 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hannstar,hsd070pww1.txt 2024-04-02T13:40:34.926 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hit,tx23d38vm0caa.txt 2024-04-02T13:40:34.928 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,g121i1-l01.txt 2024-04-02T13:40:34.931 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n116bge.txt 2024-04-02T13:40:34.933 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n156bge-l21.txt 2024-04-02T13:40:34.935 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,ld070wx3-sl01.txt 2024-04-02T13:40:34.938 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lh500wx1-sd03.txt 2024-04-02T13:40:34.941 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lp129qe.txt 2024-04-02T13:40:34.943 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/panasonic,vvx10f004b00.txt 2024-04-02T13:40:34.945 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ld9040.txt 2024-04-02T13:40:34.948 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ltn101nt05.txt 2024-04-02T13:40:34.950 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,s6e8aa0.txt 2024-04-02T13:40:34.952 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/sharp,lq101r1sx01.txt 2024-04-02T13:40:34.955 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/simple-panel.txt 2024-04-02T13:40:34.957 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ 2024-04-02T13:40:34.960 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/83xx-512x-pci.txt 2024-04-02T13:40:34.962 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/designware-pcie.txt 2024-04-02T13:40:34.965 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt 2024-04-02T13:40:34.967 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,pci.txt 2024-04-02T13:40:34.970 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/host-generic-pci.txt 2024-04-02T13:40:34.972 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/layerscape-pci.txt 2024-04-02T13:40:34.974 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/mvebu-pci.txt 2024-04-02T13:40:34.977 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt 2024-04-02T13:40:34.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:34 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:34.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:34 smithi145 ceph-mon[35646]: pgmap v56: 129 pgs: 129 active+clean; 89 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 7.4 MiB/s wr, 100 op/s 2024-04-02T13:40:34.980 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-keystone.txt 2024-04-02T13:40:34.983 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-rcar-gen2.txt 2024-04-02T13:40:34.985 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci.txt 2024-04-02T13:40:34.988 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ralink,rt3883-pci.txt 2024-04-02T13:40:34.990 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/rcar-pci.txt 2024-04-02T13:40:34.992 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/samsung,exynos5440-pcie.txt 2024-04-02T13:40:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:34 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:34 smithi050 ceph-mon[35524]: pgmap v56: 129 pgs: 129 active+clean; 89 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 7.4 MiB/s wr, 100 op/s 2024-04-02T13:40:34.995 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/spear13xx-pcie.txt 2024-04-02T13:40:34.997 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ti-pci.txt 2024-04-02T13:40:34.999 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/v3-v360epc-pci.txt 2024-04-02T13:40:35.001 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/versatile.txt 2024-04-02T13:40:35.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:34 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:35.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:34 smithi033 ceph-mon[32390]: pgmap v56: 129 pgs: 129 active+clean; 89 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 7.4 MiB/s wr, 100 op/s 2024-04-02T13:40:35.004 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xgene-pci.txt 2024-04-02T13:40:35.006 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xilinx-pcie.txt 2024-04-02T13:40:35.008 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ 2024-04-02T13:40:35.010 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/apm-xgene-phy.txt 2024-04-02T13:40:35.012 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/bcm-phy.txt 2024-04-02T13:40:35.015 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-sata-phy.txt 2024-04-02T13:40:35.017 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-usb-phy.txt 2024-04-02T13:40:35.020 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/hix5hd2-phy.txt 2024-04-02T13:40:35.022 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-bindings.txt 2024-04-02T13:40:35.024 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy28lp.txt 2024-04-02T13:40:35.026 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy365x.txt 2024-04-02T13:40:35.029 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-mvebu.txt 2024-04-02T13:40:35.031 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih407-usb.txt 2024-04-02T13:40:35.034 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt 2024-04-02T13:40:35.037 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-apq8064-sata-phy.txt 2024-04-02T13:40:35.039 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-dwc3-usb-phy.txt 2024-04-02T13:40:35.041 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-ipq806x-sata-phy.txt 2024-04-02T13:40:35.043 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt 2024-04-02T13:40:35.045 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rockchip-usb-phy.txt 2024-04-02T13:40:35.048 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/samsung-phy.txt 2024-04-02T13:40:35.050 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/st-spear-miphy.txt 2024-04-02T13:40:35.053 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/sun4i-usb-phy.txt 2024-04-02T13:40:35.055 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ti-phy.txt 2024-04-02T13:40:35.057 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ 2024-04-02T13:40:35.060 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/abilis,tb10x-iomux.txt 2024-04-02T13:40:35.062 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt 2024-04-02T13:40:35.065 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/atmel,at91-pinctrl.txt 2024-04-02T13:40:35.067 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm11351-pinctrl.txt 2024-04-02T13:40:35.070 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm2835-gpio.txt 2024-04-02T13:40:35.073 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx-pinctrl.txt 2024-04-02T13:40:35.075 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx25-pinctrl.txt 2024-04-02T13:40:35.077 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx27-pinctrl.txt 2024-04-02T13:40:35.080 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx35-pinctrl.txt 2024-04-02T13:40:35.082 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx51-pinctrl.txt 2024-04-02T13:40:35.084 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx53-pinctrl.txt 2024-04-02T13:40:35.087 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6dl-pinctrl.txt 2024-04-02T13:40:35.089 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6q-pinctrl.txt 2024-04-02T13:40:35.092 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sl-pinctrl.txt 2024-04-02T13:40:35.094 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sx-pinctrl.txt 2024-04-02T13:40:35.096 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,mxs-pinctrl.txt 2024-04-02T13:40:35.099 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,vf610-pinctrl.txt 2024-04-02T13:40:35.102 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pdc-pinctrl.txt 2024-04-02T13:40:35.105 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pinctrl.txt 2024-04-02T13:40:35.107 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,falcon-pinumx.txt 2024-04-02T13:40:35.110 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,xway-pinumx.txt 2024-04-02T13:40:35.112 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-370-pinctrl.txt 2024-04-02T13:40:35.115 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-375-pinctrl.txt 2024-04-02T13:40:35.117 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-38x-pinctrl.txt 2024-04-02T13:40:35.120 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-xp-pinctrl.txt 2024-04-02T13:40:35.122 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,dove-pinctrl.txt 2024-04-02T13:40:35.125 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,kirkwood-pinctrl.txt 2024-04-02T13:40:35.127 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,mvebu-pinctrl.txt 2024-04-02T13:40:35.130 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,orion-pinctrl.txt 2024-04-02T13:40:35.133 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/meson,pinctrl.txt 2024-04-02T13:40:35.134 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra114-pinmux.txt 2024-04-02T13:40:35.137 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-pinmux.txt 2024-04-02T13:40:35.139 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt 2024-04-02T13:40:35.141 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra20-pinmux.txt 2024-04-02T13:40:35.144 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra30-pinmux.txt 2024-04-02T13:40:35.146 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt 2024-04-02T13:40:35.149 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt 2024-04-02T13:40:35.151 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-single.txt 2024-04-02T13:40:35.154 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-sirf.txt 2024-04-02T13:40:35.156 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-st.txt 2024-04-02T13:40:35.159 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-vt8500.txt 2024-04-02T13:40:35.161 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl_spear.txt 2024-04-02T13:40:35.164 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8064-pinctrl.txt 2024-04-02T13:40:35.167 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8084-pinctrl.txt 2024-04-02T13:40:35.169 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,ipq8064-pinctrl.txt 2024-04-02T13:40:35.172 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8916-pinctrl.txt 2024-04-02T13:40:35.175 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8960-pinctrl.txt 2024-04-02T13:40:35.178 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8974-pinctrl.txt 2024-04-02T13:40:35.180 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt 2024-04-02T13:40:35.183 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt 2024-04-02T13:40:35.185 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt 2024-04-02T13:40:35.187 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt 2024-04-02T13:40:35.190 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt 2024-04-02T13:40:35.192 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,abx500.txt 2024-04-02T13:40:35.194 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt 2024-04-02T13:40:35.197 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ti,omap-pinctrl.txt 2024-04-02T13:40:35.199 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/xlnx,zynq-pinctrl.txt 2024-04-02T13:40:35.202 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ 2024-04-02T13:40:35.205 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/bq2415x.txt 2024-04-02T13:40:35.208 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/isp1704.txt 2024-04-02T13:40:35.210 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ltc2941.txt 2024-04-02T13:40:35.212 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/opp.txt 2024-04-02T13:40:35.214 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power-controller.txt 2024-04-02T13:40:35.217 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power_domain.txt 2024-04-02T13:40:35.219 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/renesas,sysc-rmobile.txt 2024-04-02T13:40:35.222 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ 2024-04-02T13:40:35.223 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/keystone-reset.txt 2024-04-02T13:40:35.226 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ltc2952-poweroff.txt 2024-04-02T13:40:35.228 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/st-reset.txt 2024-04-02T13:40:35.230 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/syscon-reboot.txt 2024-04-02T13:40:35.232 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rockchip-io-domain.txt 2024-04-02T13:40:35.236 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rx51-battery.txt 2024-04-02T13:40:35.238 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/twl-charger.txt 2024-04-02T13:40:35.240 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ 2024-04-02T13:40:35.243 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/ 2024-04-02T13:40:35.244 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/btemp.txt 2024-04-02T13:40:35.247 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/chargalg.txt 2024-04-02T13:40:35.249 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/charger.txt 2024-04-02T13:40:35.251 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/fg.txt 2024-04-02T13:40:35.254 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/axxia-reset.txt 2024-04-02T13:40:35.257 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/charger-manager.txt 2024-04-02T13:40:35.260 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/gpio-charger.txt 2024-04-02T13:40:35.262 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/imx-snvs-poweroff.txt 2024-04-02T13:40:35.265 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/lp8727_charger.txt 2024-04-02T13:40:35.267 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max17042_battery.txt 2024-04-02T13:40:35.270 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max8925_batter.txt 2024-04-02T13:40:35.272 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/msm-poweroff.txt 2024-04-02T13:40:35.274 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/olpc_battery.txt 2024-04-02T13:40:35.276 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/power_supply.txt 2024-04-02T13:40:35.279 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/qnap-poweroff.txt 2024-04-02T13:40:35.281 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/restart-poweroff.txt 2024-04-02T13:40:35.284 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/sbs_sbs-battery.txt 2024-04-02T13:40:35.286 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ti,bq24735.txt 2024-04-02T13:40:35.289 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/tps65090.txt 2024-04-02T13:40:35.292 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/ 2024-04-02T13:40:35.294 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ 2024-04-02T13:40:35.295 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/akebono.txt 2024-04-02T13:40:35.298 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/cpm.txt 2024-04-02T13:40:35.301 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/emac.txt 2024-04-02T13:40:35.304 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/hsta.txt 2024-04-02T13:40:35.306 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ndfc.txt 2024-04-02T13:40:35.309 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ppc440spe-adma.txt 2024-04-02T13:40:35.312 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/reboot.txt 2024-04-02T13:40:35.314 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ 2024-04-02T13:40:35.317 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/board.txt 2024-04-02T13:40:35.319 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cache_sram.txt 2024-04-02T13:40:35.322 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ccf.txt 2024-04-02T13:40:35.324 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/ 2024-04-02T13:40:35.326 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm.txt 2024-04-02T13:40:35.328 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/ 2024-04-02T13:40:35.330 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/brg.txt 2024-04-02T13:40:35.332 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/i2c.txt 2024-04-02T13:40:35.335 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/pic.txt 2024-04-02T13:40:35.337 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/usb.txt 2024-04-02T13:40:35.340 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/gpio.txt 2024-04-02T13:40:35.343 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/network.txt 2024-04-02T13:40:35.346 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe.txt 2024-04-02T13:40:35.348 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ 2024-04-02T13:40:35.350 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/firmware.txt 2024-04-02T13:40:35.353 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/par_io.txt 2024-04-02T13:40:35.356 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/pincfg.txt 2024-04-02T13:40:35.359 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ucc.txt 2024-04-02T13:40:35.362 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/usb.txt 2024-04-02T13:40:35.364 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/serial.txt 2024-04-02T13:40:35.368 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt 2024-04-02T13:40:35.371 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dcsr.txt 2024-04-02T13:40:35.373 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/diu.txt 2024-04-02T13:40:35.376 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dma.txt 2024-04-02T13:40:35.380 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ecm.txt 2024-04-02T13:40:35.382 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/fman.txt 2024-04-02T13:40:35.385 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/gtm.txt 2024-04-02T13:40:35.388 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/guts.txt 2024-04-02T13:40:35.390 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/interlaken-lac.txt 2024-04-02T13:40:35.393 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/l2cache.txt 2024-04-02T13:40:35.395 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/lbc.txt 2024-04-02T13:40:35.398 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcm.txt 2024-04-02T13:40:35.401 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcu-mpc8349emitx.txt 2024-04-02T13:40:35.403 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mem-ctrlr.txt 2024-04-02T13:40:35.405 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5121-psc.txt 2024-04-02T13:40:35.408 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5200.txt 2024-04-02T13:40:35.411 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt 2024-04-02T13:40:35.413 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-timer.txt 2024-04-02T13:40:35.416 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic.txt 2024-04-02T13:40:35.419 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/msi-pic.txt 2024-04-02T13:40:35.421 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt 2024-04-02T13:40:35.424 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pmc.txt 2024-04-02T13:40:35.426 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/raideng.txt 2024-04-02T13:40:35.428 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio-rmu.txt 2024-04-02T13:40:35.431 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio.txt 2024-04-02T13:40:35.434 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/ 2024-04-02T13:40:35.436 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/gamecube.txt 2024-04-02T13:40:35.438 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/wii.txt 2024-04-02T13:40:35.441 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/ 2024-04-02T13:40:35.444 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/pps-gpio.txt 2024-04-02T13:40:35.446 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ 2024-04-02T13:40:35.448 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-hlcdc-pwm.txt 2024-04-02T13:40:35.451 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-pwm.txt 2024-04-02T13:40:35.453 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-tcb-pwm.txt 2024-04-02T13:40:35.455 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/bcm-kona-pwm.txt 2024-04-02T13:40:35.458 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/cirrus,clps711x-pwm.txt 2024-04-02T13:40:35.461 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/img-pwm.txt 2024-04-02T13:40:35.463 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/imx-pwm.txt 2024-04-02T13:40:35.465 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/lpc32xx-pwm.txt 2024-04-02T13:40:35.467 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/mxs-pwm.txt 2024-04-02T13:40:35.469 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nvidia,tegra20-pwm.txt 2024-04-02T13:40:35.472 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nxp,pca9685-pwm.txt 2024-04-02T13:40:35.474 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-bcm2835.txt 2024-04-02T13:40:35.477 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-fsl-ftm.txt 2024-04-02T13:40:35.479 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-lp3943.txt 2024-04-02T13:40:35.481 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-rockchip.txt 2024-04-02T13:40:35.484 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-samsung.txt 2024-04-02T13:40:35.486 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-st.txt 2024-04-02T13:40:35.488 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-sun4i.txt 2024-04-02T13:40:35.491 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiecap.txt 2024-04-02T13:40:35.493 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiehrpwm.txt 2024-04-02T13:40:35.496 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tipwmss.txt 2024-04-02T13:40:35.498 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm.txt 2024-04-02T13:40:35.500 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pxa-pwm.txt 2024-04-02T13:40:35.503 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/renesas,tpu-pwm.txt 2024-04-02T13:40:35.505 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/spear-pwm.txt 2024-04-02T13:40:35.508 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwm.txt 2024-04-02T13:40:35.511 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwmled.txt 2024-04-02T13:40:35.513 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/vt8500-pwm.txt 2024-04-02T13:40:35.515 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/ 2024-04-02T13:40:35.517 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/regmap.txt 2024-04-02T13:40:35.520 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ 2024-04-02T13:40:35.522 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm800.txt 2024-04-02T13:40:35.524 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm860x.txt 2024-04-02T13:40:35.527 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/act8865-regulator.txt 2024-04-02T13:40:35.529 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/anatop-regulator.txt 2024-04-02T13:40:35.531 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/as3722-regulator.txt 2024-04-02T13:40:35.533 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9210.txt 2024-04-02T13:40:35.536 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9211.txt 2024-04-02T13:40:35.538 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fan53555.txt 2024-04-02T13:40:35.541 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fixed-regulator.txt 2024-04-02T13:40:35.543 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/gpio-regulator.txt 2024-04-02T13:40:35.546 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/isl9305.txt 2024-04-02T13:40:35.549 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/lp872x.txt 2024-04-02T13:40:35.551 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ltc3589.txt 2024-04-02T13:40:35.554 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max1586-regulator.txt 2024-04-02T13:40:35.557 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max77802.txt 2024-04-02T13:40:35.560 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8660.txt 2024-04-02T13:40:35.563 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8907.txt 2024-04-02T13:40:35.565 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8925-regulator.txt 2024-04-02T13:40:35.568 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8952.txt 2024-04-02T13:40:35.570 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8973-regulator.txt 2024-04-02T13:40:35.573 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8997-regulator.txt 2024-04-02T13:40:35.575 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/mt6397-regulator.txt 2024-04-02T13:40:35.578 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/palmas-pmic.txt 2024-04-02T13:40:35.581 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pbias-regulator.txt 2024-04-02T13:40:35.583 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pfuze100.txt 2024-04-02T13:40:35.585 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pwm-regulator.txt 2024-04-02T13:40:35.588 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/regulator.txt 2024-04-02T13:40:35.590 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/s5m8767-regulator.txt 2024-04-02T13:40:35.593 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/sky81452-regulator.txt 2024-04-02T13:40:35.595 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ti-abb-regulator.txt 2024-04-02T13:40:35.598 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps51632-regulator.txt 2024-04-02T13:40:35.600 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps62360-regulator.txt 2024-04-02T13:40:35.602 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65090.txt 2024-04-02T13:40:35.605 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65217.txt 2024-04-02T13:40:35.608 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65218.txt 2024-04-02T13:40:35.611 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps6586x.txt 2024-04-02T13:40:35.613 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/twl-regulator.txt 2024-04-02T13:40:35.616 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/vexpress.txt 2024-04-02T13:40:35.619 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/ 2024-04-02T13:40:35.621 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt 2024-04-02T13:40:35.623 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/ 2024-04-02T13:40:35.626 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/allwinner,sunxi-clock-reset.txt 2024-04-02T13:40:35.628 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/fsl,imx-src.txt 2024-04-02T13:40:35.630 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/reset.txt 2024-04-02T13:40:35.633 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/sirf,rstc.txt 2024-04-02T13:40:35.635 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/socfpga-reset.txt 2024-04-02T13:40:35.637 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-picophyreset.txt 2024-04-02T13:40:35.639 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-powerdown.txt 2024-04-02T13:40:35.642 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-softreset.txt 2024-04-02T13:40:35.645 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/resource-names.txt 2024-04-02T13:40:35.648 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/ 2024-04-02T13:40:35.650 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/apm,rng.txt 2024-04-02T13:40:35.652 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/brcm,bcm2835.txt 2024-04-02T13:40:35.654 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/qcom,prng.txt 2024-04-02T13:40:35.657 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/ 2024-04-02T13:40:35.659 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/armada-380-rtc.txt 2024-04-02T13:40:35.661 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91rm9200-rtc.txt 2024-04-02T13:40:35.663 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91sam9-rtc.txt 2024-04-02T13:40:35.666 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dallas,ds1339.txt 2024-04-02T13:40:35.668 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dw-apb.txt 2024-04-02T13:40:35.671 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt 2024-04-02T13:40:35.675 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/imxdi-rtc.txt 2024-04-02T13:40:35.678 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/isil,isl12057.txt 2024-04-02T13:40:35.681 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/lpc32xx-rtc.txt 2024-04-02T13:40:35.684 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/maxim,ds1742.txt 2024-04-02T13:40:35.687 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/moxa,moxart-rtc.txt 2024-04-02T13:40:35.690 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.txt 2024-04-02T13:40:35.692 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nxp,rtc-2123.txt 2024-04-02T13:40:35.695 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/olpc-xo1-rtc.txt 2024-04-02T13:40:35.697 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/orion-rtc.txt 2024-04-02T13:40:35.701 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/pxa-rtc.txt 2024-04-02T13:40:35.703 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-cmos.txt 2024-04-02T13:40:35.706 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-omap.txt 2024-04-02T13:40:35.708 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-opal.txt 2024-04-02T13:40:35.711 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-palmas.txt 2024-04-02T13:40:35.714 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/s3c-rtc.txt 2024-04-02T13:40:35.716 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt 2024-04-02T13:40:35.718 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/snvs-rtc.txt 2024-04-02T13:40:35.721 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/spear-rtc.txt 2024-04-02T13:40:35.723 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/stmp3xxx-rtc.txt 2024-04-02T13:40:35.726 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sun6i-rtc.txt 2024-04-02T13:40:35.729 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sunxi-rtc.txt 2024-04-02T13:40:35.732 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/twl-rtc.txt 2024-04-02T13:40:35.734 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/via,vt8500-rtc.txt 2024-04-02T13:40:35.737 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/xgene-rtc.txt 2024-04-02T13:40:35.740 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/ 2024-04-02T13:40:35.742 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/ 2024-04-02T13:40:35.745 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/st33zp24-i2c.txt 2024-04-02T13:40:35.748 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/ 2024-04-02T13:40:35.752 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/8250.txt 2024-04-02T13:40:35.755 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_jtaguart.txt 2024-04-02T13:40:35.757 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_uart.txt 2024-04-02T13:40:35.760 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/arc-uart.txt 2024-04-02T13:40:35.762 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/atmel-usart.txt 2024-04-02T13:40:35.765 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/axis,etraxfs-uart.txt 2024-04-02T13:40:35.767 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/bcm63xx-uart.txt 2024-04-02T13:40:35.770 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cavium-uart.txt 2024-04-02T13:40:35.773 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cdns,uart.txt 2024-04-02T13:40:35.775 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cirrus,clps711x-uart.txt 2024-04-02T13:40:35.778 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/digicolor-usart.txt 2024-04-02T13:40:35.786 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/efm32-uart.txt 2024-04-02T13:40:35.788 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt 2024-04-02T13:40:35.791 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-lpuart.txt 2024-04-02T13:40:35.794 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt 2024-04-02T13:40:35.796 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/lantiq_asc.txt 2024-04-02T13:40:35.799 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/maxim,max310x.txt 2024-04-02T13:40:35.802 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl,pxa-ssp.txt 2024-04-02T13:40:35.805 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl-serial.txt 2024-04-02T13:40:35.808 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mtk-uart.txt 2024-04-02T13:40:35.810 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nvidia,tegra20-hsuart.txt 2024-04-02T13:40:35.813 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp,sc16is7xx.txt 2024-04-02T13:40:35.816 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp-lpc32xx-hsuart.txt 2024-04-02T13:40:35.820 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/omap_serial.txt 2024-04-02T13:40:35.822 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/pl011.txt 2024-04-02T13:40:35.824 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qca,ar9330-uart.txt 2024-04-02T13:40:35.827 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uart.txt 2024-04-02T13:40:35.830 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uartdm.txt 2024-04-02T13:40:35.833 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/renesas,sci-serial.txt 2024-04-02T13:40:35.836 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/rs485.txt 2024-04-02T13:40:35.840 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/samsung_uart.txt 2024-04-02T13:40:35.842 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sirf-uart.txt 2024-04-02T13:40:35.845 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.txt 2024-04-02T13:40:35.848 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sprd-uart.txt 2024-04-02T13:40:35.850 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/st-asc.txt 2024-04-02T13:40:35.853 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/vt8500-uart.txt 2024-04-02T13:40:35.856 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/ 2024-04-02T13:40:35.860 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/allwinner,sun4i-ps2.txt 2024-04-02T13:40:35.864 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/altera_ps2.txt 2024-04-02T13:40:35.867 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/olpc,ap-sp.txt 2024-04-02T13:40:35.870 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/snps-arc_ps2.txt 2024-04-02T13:40:35.873 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ 2024-04-02T13:40:35.876 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/ 2024-04-02T13:40:35.879 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman-portals.txt 2024-04-02T13:40:35.882 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman.txt 2024-04-02T13:40:35.886 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman-portals.txt 2024-04-02T13:40:35.888 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman.txt 2024-04-02T13:40:35.891 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/ 2024-04-02T13:40:35.894 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/qcom,gsbi.txt 2024-04-02T13:40:35.898 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/ 2024-04-02T13:40:35.901 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-dma.txt 2024-04-02T13:40:35.904 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt 2024-04-02T13:40:35.908 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ 2024-04-02T13:40:35.911 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,adau1701.txt 2024-04-02T13:40:35.914 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-i2s.txt 2024-04-02T13:40:35.917 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-spdif-tx.txt 2024-04-02T13:40:35.920 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,ssm2602.txt 2024-04-02T13:40:35.923 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4104.txt 2024-04-02T13:40:35.925 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4554.c 2024-04-02T13:40:35.928 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4642.txt 2024-04-02T13:40:35.930 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak5386.txt 2024-04-02T13:40:35.933 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5623.txt 2024-04-02T13:40:35.936 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5632.txt 2024-04-02T13:40:35.939 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/armada-370db-audio.txt 2024-04-02T13:40:35.943 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/arndale.txt 2024-04-02T13:40:35.945 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-at91sam9g20ek-wm8731-audio.txt 2024-04-02T13:40:35.948 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-sam9x5-wm8731-audio.txt 2024-04-02T13:40:35.951 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-wm8904.txt 2024-04-02T13:40:35.953 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel_ac97c.txt 2024-04-02T13:40:35.956 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/bcm2835-i2s.txt 2024-04-02T13:40:35.958 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cdns,xtfpga-i2s.txt 2024-04-02T13:40:35.961 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs35l32.txt 2024-04-02T13:40:35.964 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4265.txt 2024-04-02T13:40:35.968 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4270.txt 2024-04-02T13:40:35.970 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4271.txt 2024-04-02T13:40:35.973 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l52.txt 2024-04-02T13:40:35.976 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l56.txt 2024-04-02T13:40:35.979 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l73.txt 2024-04-02T13:40:35.981 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42xx8.txt 2024-04-02T13:40:35.983 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/da9055.txt 2024-04-02T13:40:35.986 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-evm-audio.txt 2024-04-02T13:40:35.988 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-mcasp-audio.txt 2024-04-02T13:40:35.991 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/designware-i2s.txt 2024-04-02T13:40:35.994 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/es8328.txt 2024-04-02T13:40:35.997 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/eukrea-tlv320.txt 2024-04-02T13:40:36.000 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,asrc.txt 2024-04-02T13:40:36.002 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,esai.txt 2024-04-02T13:40:36.005 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,spdif.txt 2024-04-02T13:40:36.008 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,ssi.txt 2024-04-02T13:40:36.011 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-asoc-card.txt 2024-04-02T13:40:36.014 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-sai.txt 2024-04-02T13:40:36.018 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/hdmi.txt 2024-04-02T13:40:36.020 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-es8328.txt 2024-04-02T13:40:36.022 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-sgtl5000.txt 2024-04-02T13:40:36.026 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-spdif.txt 2024-04-02T13:40:36.029 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-wm8962.txt 2024-04-02T13:40:36.032 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audmux.txt 2024-04-02T13:40:36.036 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ingenic,jz4740-i2s.txt 2024-04-02T13:40:36.039 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98090.txt 2024-04-02T13:40:36.042 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98095.txt 2024-04-02T13:40:36.044 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98357a.txt 2024-04-02T13:40:36.048 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa-ssp.txt 2024-04-02T13:40:36.050 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa2xx-pcm.txt 2024-04-02T13:40:36.054 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mvebu-audio.txt 2024-04-02T13:40:36.057 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-audio-sgtl5000.txt 2024-04-02T13:40:36.059 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-saif.txt 2024-04-02T13:40:36.062 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nokia,rx51.txt 2024-04-02T13:40:36.065 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-alc5632.txt 2024-04-02T13:40:36.068 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-max98090.txt 2024-04-02T13:40:36.070 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5640.txt 2024-04-02T13:40:36.073 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5677.txt 2024-04-02T13:40:36.076 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-trimslice.txt 2024-04-02T13:40:36.078 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8753.txt 2024-04-02T13:40:36.081 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8903.txt 2024-04-02T13:40:36.083 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm9712.txt 2024-04-02T13:40:36.086 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-ac97.txt 2024-04-02T13:40:36.089 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-das.txt 2024-04-02T13:40:36.091 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-i2s.txt 2024-04-02T13:40:36.094 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-ahub.txt 2024-04-02T13:40:36.096 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt 2024-04-02T13:40:36.099 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-i2s.txt 2024-04-02T13:40:36.102 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-abe-twl6040.txt 2024-04-02T13:40:36.105 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-dmic.txt 2024-04-02T13:40:36.107 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcbsp.txt 2024-04-02T13:40:36.110 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcpdm.txt 2024-04-02T13:40:36.113 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-twl4030.txt 2024-04-02T13:40:36.116 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm1792a.txt 2024-04-02T13:40:36.119 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm512x.txt 2024-04-02T13:40:36.122 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,fsi.txt 2024-04-02T13:40:36.125 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,rsnd.txt 2024-04-02T13:40:36.127 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rockchip-i2s.txt 2024-04-02T13:40:36.130 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5631.txt 2024-04-02T13:40:36.132 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5640.txt 2024-04-02T13:40:36.135 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5677.txt 2024-04-02T13:40:36.138 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,odroidx2-max98090.txt 2024-04-02T13:40:36.141 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,smdk-wm8994.txt 2024-04-02T13:40:36.143 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung-i2s.txt 2024-04-02T13:40:36.147 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sgtl5000.txt 2024-04-02T13:40:36.150 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/simple-card.txt 2024-04-02T13:40:36.153 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-codec.txt 2024-04-02T13:40:36.156 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-port.txt 2024-04-02T13:40:36.158 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio.txt 2024-04-02T13:40:36.161 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-usp.txt 2024-04-02T13:40:36.164 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/snow.txt 2024-04-02T13:40:36.166 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/soc-ac97link.txt 2024-04-02T13:40:36.170 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-receiver.txt 2024-04-02T13:40:36.173 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-transmitter.txt 2024-04-02T13:40:36.175 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm2518.txt 2024-04-02T13:40:36.177 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm4567.txt 2024-04-02T13:40:36.181 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta32x.txt 2024-04-02T13:40:36.184 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta350.txt 2024-04-02T13:40:36.187 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tas2552.txt 2024-04-02T13:40:36.190 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tdm-slot.txt 2024-04-02T13:40:36.192 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,pcm1681.txt 2024-04-02T13:40:36.194 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,tas5086.txt 2024-04-02T13:40:36.197 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic31xx.txt 2024-04-02T13:40:36.200 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic32x4.txt 2024-04-02T13:40:36.203 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic3x.txt 2024-04-02T13:40:36.206 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tpa6130a2.txt 2024-04-02T13:40:36.208 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ts3a227e.txt 2024-04-02T13:40:36.211 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-mop500.txt 2024-04-02T13:40:36.214 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-msp.txt 2024-04-02T13:40:36.217 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/widgets.txt 2024-04-02T13:40:36.220 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8510.txt 2024-04-02T13:40:36.223 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8523.txt 2024-04-02T13:40:36.226 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8580.txt 2024-04-02T13:40:36.228 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8711.txt 2024-04-02T13:40:36.231 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8728.txt 2024-04-02T13:40:36.234 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8731.txt 2024-04-02T13:40:36.237 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8737.txt 2024-04-02T13:40:36.239 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8741.txt 2024-04-02T13:40:36.242 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8750.txt 2024-04-02T13:40:36.245 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8753.txt 2024-04-02T13:40:36.247 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8770.txt 2024-04-02T13:40:36.251 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8776.txt 2024-04-02T13:40:36.254 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8804.txt 2024-04-02T13:40:36.256 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8903.txt 2024-04-02T13:40:36.259 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8904.txt 2024-04-02T13:40:36.261 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8960.txt 2024-04-02T13:40:36.264 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8962.txt 2024-04-02T13:40:36.266 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8994.txt 2024-04-02T13:40:36.270 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ 2024-04-02T13:40:36.272 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/brcm,bcm2835-spi.txt 2024-04-02T13:40:36.275 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/efm32-spi.txt 2024-04-02T13:40:36.277 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-imx-cspi.txt 2024-04-02T13:40:36.281 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-spi.txt 2024-04-02T13:40:36.284 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/mxs-spi.txt 2024-04-02T13:40:36.287 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra114-spi.txt 2024-04-02T13:40:36.290 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-sflash.txt 2024-04-02T13:40:36.293 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-slink.txt 2024-04-02T13:40:36.296 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/omap-spi.txt 2024-04-02T13:40:36.299 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/qcom,spi-qup.txt 2024-04-02T13:40:36.301 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-hspi.txt 2024-04-02T13:40:36.303 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-msiof.txt 2024-04-02T13:40:36.306 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/snps,dw-apb-ssi.txt 2024-04-02T13:40:36.309 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-bus.txt 2024-04-02T13:40:36.312 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-cadence.txt 2024-04-02T13:40:36.315 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-davinci.txt 2024-04-02T13:40:36.318 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-dw.txt 2024-04-02T13:40:36.321 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt 2024-04-02T13:40:36.323 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-gpio.txt 2024-04-02T13:40:36.326 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-img-spfi.txt 2024-04-02T13:40:36.328 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-meson.txt 2024-04-02T13:40:36.331 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-octeon.txt 2024-04-02T13:40:36.333 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-orion.txt 2024-04-02T13:40:36.335 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rockchip.txt 2024-04-02T13:40:36.337 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rspi.txt 2024-04-02T13:40:36.340 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-samsung.txt 2024-04-02T13:40:36.342 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sc18is602.txt 2024-04-02T13:40:36.345 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sirf.txt 2024-04-02T13:40:36.348 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-st-ssc.txt 2024-04-02T13:40:36.350 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun4i.txt 2024-04-02T13:40:36.353 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun6i.txt 2024-04-02T13:40:36.355 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-xtensa-xtfpga.txt 2024-04-02T13:40:36.358 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_altera.txt 2024-04-02T13:40:36.360 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_atmel.txt 2024-04-02T13:40:36.362 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_oc_tiny.txt 2024-04-02T13:40:36.365 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_pl022.txt 2024-04-02T13:40:36.367 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ti_qspi.txt 2024-04-02T13:40:36.369 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/ 2024-04-02T13:40:36.372 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/qcom,spmi-pmic-arb.txt 2024-04-02T13:40:36.374 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/spmi.txt 2024-04-02T13:40:36.376 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/ 2024-04-02T13:40:36.379 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/ 2024-04-02T13:40:36.380 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/ 2024-04-02T13:40:36.382 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/lpc32xx-adc.txt 2024-04-02T13:40:36.384 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/mxs-lradc.txt 2024-04-02T13:40:36.387 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/spear-adc.txt 2024-04-02T13:40:36.389 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/submitting-patches.txt 2024-04-02T13:40:36.394 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ 2024-04-02T13:40:36.395 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/armada-thermal.txt 2024-04-02T13:40:36.398 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/db8500-thermal.txt 2024-04-02T13:40:36.400 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/dove-thermal.txt 2024-04-02T13:40:36.402 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/exynos-thermal.txt 2024-04-02T13:40:36.405 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/imx-thermal.txt 2024-04-02T13:40:36.408 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/kirkwood-thermal.txt 2024-04-02T13:40:36.410 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rcar-thermal.txt 2024-04-02T13:40:36.413 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rockchip-thermal.txt 2024-04-02T13:40:36.415 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/spear-thermal.txt 2024-04-02T13:40:36.417 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/st-thermal.txt 2024-04-02T13:40:36.420 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/tegra-soctherm.txt 2024-04-02T13:40:36.422 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/thermal.txt 2024-04-02T13:40:36.425 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ti_soc_thermal.txt 2024-04-02T13:40:36.427 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ 2024-04-02T13:40:36.429 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun4i-timer.txt 2024-04-02T13:40:36.431 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun5i-a13-hstimer.txt 2024-04-02T13:40:36.433 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/amlogic,meson6-timer.txt 2024-04-02T13:40:36.435 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/arm,sp804.txt 2024-04-02T13:40:36.438 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/brcm,bcm2835-system-timer.txt 2024-04-02T13:40:36.440 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cadence,ttc-timer.txt 2024-04-02T13:40:36.443 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cirrus,clps711x-timer.txt 2024-04-02T13:40:36.445 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/digicolor-timer.txt 2024-04-02T13:40:36.447 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/energymicro,efm32-timer.txt 2024-04-02T13:40:36.450 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,ftm-timer.txt 2024-04-02T13:40:36.452 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt 2024-04-02T13:40:36.454 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/lsi,zevio-timer.txt 2024-04-02T13:40:36.457 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,armada-370-xp-timer.txt 2024-04-02T13:40:36.459 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,orion-timer.txt 2024-04-02T13:40:36.462 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/mediatek,mtk-timer.txt 2024-04-02T13:40:36.464 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/moxa,moxart-timer.txt 2024-04-02T13:40:36.466 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra20-timer.txt 2024-04-02T13:40:36.469 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra30-timer.txt 2024-04-02T13:40:36.471 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,cmt.txt 2024-04-02T13:40:36.474 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,mtu2.txt 2024-04-02T13:40:36.476 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,tmu.txt 2024-04-02T13:40:36.479 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/rockchip,rk3288-timer.txt 2024-04-02T13:40:36.481 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/samsung,exynos4210-mct.txt 2024-04-02T13:40:36.483 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/stericsson-u300-apptimer.txt 2024-04-02T13:40:36.485 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ti,keystone-timer.txt 2024-04-02T13:40:36.488 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ 2024-04-02T13:40:36.490 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ufshcd-pltfrm.txt 2024-04-02T13:40:36.493 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/unittest.txt 2024-04-02T13:40:36.495 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ 2024-04-02T13:40:36.498 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/am33xx-usb.txt 2024-04-02T13:40:36.500 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/atmel-usb.txt 2024-04-02T13:40:36.502 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-imx.txt 2024-04-02T13:40:36.504 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-qcom.txt 2024-04-02T13:40:36.506 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-usb2.txt 2024-04-02T13:40:36.509 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-zevio.txt 2024-04-02T13:40:36.511 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc2.txt 2024-04-02T13:40:36.514 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3-st.txt 2024-04-02T13:40:36.516 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3.txt 2024-04-02T13:40:36.519 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-omap.txt 2024-04-02T13:40:36.521 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-orion.txt 2024-04-02T13:40:36.524 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-st.txt 2024-04-02T13:40:36.527 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/exynos-usb.txt 2024-04-02T13:40:36.529 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/fsl-usb.txt 2024-04-02T13:40:36.532 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/generic.txt 2024-04-02T13:40:36.535 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/gr-udc.txt 2024-04-02T13:40:36.537 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/isp1301.txt 2024-04-02T13:40:36.539 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-phy.txt 2024-04-02T13:40:36.542 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-usb.txt 2024-04-02T13:40:36.544 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/lpc32xx-udc.txt 2024-04-02T13:40:36.546 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/msm-hsusb.txt 2024-04-02T13:40:36.549 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/mxs-phy.txt 2024-04-02T13:40:36.554 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-ehci.txt 2024-04-02T13:40:36.580 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-usb-phy.txt 2024-04-02T13:40:36.590 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-nxp.txt 2024-04-02T13:40:36.599 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-omap3.txt 2024-04-02T13:40:36.602 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-st.txt 2024-04-02T13:40:36.604 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/omap-usb.txt 2024-04-02T13:40:36.607 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/pxa-usb.txt 2024-04-02T13:40:36.610 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/qcom,dwc3.txt 2024-04-02T13:40:36.613 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/renesas_usbhs.txt 2024-04-02T13:40:36.616 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-hsotg.txt 2024-04-02T13:40:36.619 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-usbphy.txt 2024-04-02T13:40:36.622 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/spear-usb.txt 2024-04-02T13:40:36.625 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/twlxxxx-usb.txt 2024-04-02T13:40:36.628 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/udc-xilinx.txt 2024-04-02T13:40:36.630 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ehci.txt 2024-04-02T13:40:36.633 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-nop-xceiv.txt 2024-04-02T13:40:36.635 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ohci.txt 2024-04-02T13:40:36.638 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-uhci.txt 2024-04-02T13:40:36.641 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-xhci.txt 2024-04-02T13:40:36.643 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb3503.txt 2024-04-02T13:40:36.646 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usbmisc-imx.txt 2024-04-02T13:40:36.649 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ux500-usb.txt 2024-04-02T13:40:36.651 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/vendor-prefixes.txt 2024-04-02T13:40:36.656 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ 2024-04-02T13:40:36.658 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7123.txt 2024-04-02T13:40:36.661 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7511.txt 2024-04-02T13:40:36.664 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/analog-tv-connector.txt 2024-04-02T13:40:36.667 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/arm,pl11x.txt 2024-04-02T13:40:36.670 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/atmel,lcdc.txt 2024-04-02T13:40:36.673 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/ 2024-04-02T13:40:36.676 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/88pm860x.txt 2024-04-02T13:40:36.679 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/gpio-backlight.txt 2024-04-02T13:40:36.681 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/lp855x.txt 2024-04-02T13:40:36.685 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/max8925-backlight.txt 2024-04-02T13:40:36.689 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/pwm-backlight.txt 2024-04-02T13:40:36.692 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/tps65217-backlight.txt 2024-04-02T13:40:36.697 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ 2024-04-02T13:40:36.700 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ps8622.txt 2024-04-02T13:40:36.703 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ptn3460.txt 2024-04-02T13:40:36.706 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/cirrus,clps711x-fb.txt 2024-04-02T13:40:36.710 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/display-timing.txt 2024-04-02T13:40:36.713 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dvi-connector.txt 2024-04-02T13:40:36.717 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dw_hdmi-rockchip.txt 2024-04-02T13:40:36.719 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos7-decon.txt 2024-04-02T13:40:36.722 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dp.txt 2024-04-02T13:40:36.726 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dsim.txt 2024-04-02T13:40:36.730 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmi.txt 2024-04-02T13:40:36.733 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiddc.txt 2024-04-02T13:40:36.737 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiphy.txt 2024-04-02T13:40:36.740 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_mixer.txt 2024-04-02T13:40:36.742 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/fsl,imx-fb.txt 2024-04-02T13:40:36.744 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/hdmi-connector.txt 2024-04-02T13:40:36.746 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/lgphilips,lb035q02.txt 2024-04-02T13:40:36.749 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dpi.txt 2024-04-02T13:40:36.751 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dsi-cm.txt 2024-04-02T13:40:36.754 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/renesas,du.txt 2024-04-02T13:40:36.756 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-drm.txt 2024-04-02T13:40:36.758 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-vop.txt 2024-04-02T13:40:36.761 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/samsung-fimd.txt 2024-04-02T13:40:36.763 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sharp,ls037v7dw01.txt 2024-04-02T13:40:36.766 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer-sunxi.txt 2024-04-02T13:40:36.768 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer.txt 2024-04-02T13:40:36.771 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sony,acx565akm.txt 2024-04-02T13:40:36.773 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1289fb.txt 2024-04-02T13:40:36.776 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1307fb.txt 2024-04-02T13:40:36.778 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/thine,thc63lvdm83d 2024-04-02T13:40:36.782 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,dra7-dss.txt 2024-04-02T13:40:36.784 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap-dss.txt 2024-04-02T13:40:36.804 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap2-dss.txt 2024-04-02T13:40:36.951 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap3-dss.txt 2024-04-02T13:40:36.954 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap4-dss.txt 2024-04-02T13:40:36.957 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap5-dss.txt 2024-04-02T13:40:36.960 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,opa362.txt 2024-04-02T13:40:36.962 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tfp410.txt 2024-04-02T13:40:36.964 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tpd12s015.txt 2024-04-02T13:40:36.967 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/toppoly,td028ttec1.txt 2024-04-02T13:40:36.970 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/tpo,td043mtea1.txt 2024-04-02T13:40:36.972 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/vga-connector.txt 2024-04-02T13:40:36.975 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/via,vt8500-fb.txt 2024-04-02T13:40:36.977 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,prizm-ge-rops.txt 2024-04-02T13:40:36.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:36 smithi145 ceph-mon[35646]: pgmap v57: 129 pgs: 129 active+clean; 89 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 798 KiB/s wr, 79 op/s 2024-04-02T13:40:36.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:36 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:40:36.980 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,wm8505-fb.txt 2024-04-02T13:40:36.982 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/ 2024-04-02T13:40:36.985 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/mmio.txt 2024-04-02T13:40:36.988 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/ 2024-04-02T13:40:36.991 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/fsl-imx-owire.txt 2024-04-02T13:40:36.993 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/omap-hdq.txt 2024-04-02T13:40:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:36 smithi050 ceph-mon[35524]: pgmap v57: 129 pgs: 129 active+clean; 89 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 798 KiB/s wr, 79 op/s 2024-04-02T13:40:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:36 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:40:36.996 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/w1-gpio.txt 2024-04-02T13:40:36.998 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ 2024-04-02T13:40:37.001 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-at91rm9200-wdt.txt 2024-04-02T13:40:37.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:36 smithi033 ceph-mon[32390]: pgmap v57: 129 pgs: 129 active+clean; 89 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 798 KiB/s wr, 79 op/s 2024-04-02T13:40:37.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:36 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:40:37.004 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-wdt.txt 2024-04-02T13:40:37.007 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/brcm,bcm2835-pm-wdog.txt 2024-04-02T13:40:37.009 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/cadence-wdt.txt 2024-04-02T13:40:37.012 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/davinci-wdt.txt 2024-04-02T13:40:37.015 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/dw_wdt.txt 2024-04-02T13:40:37.017 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/fsl-imx-wdt.txt 2024-04-02T13:40:37.019 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/gpio-wdt.txt 2024-04-02T13:40:37.021 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/imgpdc-wdt.txt 2024-04-02T13:40:37.024 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ingenic,jz4740-wdt.txt 2024-04-02T13:40:37.027 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/marvel.txt 2024-04-02T13:40:37.030 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/men-a021-wdt.txt 2024-04-02T13:40:37.032 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/meson6-wdt.txt 2024-04-02T13:40:37.034 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/moxa,moxart-watchdog.txt 2024-04-02T13:40:37.037 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/mtk-wdt.txt 2024-04-02T13:40:37.039 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/of-xilinx-wdt.txt 2024-04-02T13:40:37.042 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/omap-wdt.txt 2024-04-02T13:40:37.044 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/pnx4008-wdt.txt 2024-04-02T13:40:37.046 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qca-ar7130-wdt.txt 2024-04-02T13:40:37.049 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qcom-wdt.txt 2024-04-02T13:40:37.051 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/rt2880-wdt.txt 2024-04-02T13:40:37.053 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/samsung-wdt.txt 2024-04-02T13:40:37.056 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sirfsoc_wdt.txt 2024-04-02T13:40:37.058 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/stericsson-coh901327.txt 2024-04-02T13:40:37.060 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sunxi-wdt.txt 2024-04-02T13:40:37.063 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/twl4030-wdt.txt 2024-04-02T13:40:37.065 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ 2024-04-02T13:40:37.067 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ce4100.txt 2024-04-02T13:40:37.069 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/interrupt.txt 2024-04-02T13:40:37.072 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/timer.txt 2024-04-02T13:40:37.074 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/xilinx.txt 2024-04-02T13:40:37.078 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/ 2024-04-02T13:40:37.079 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/xillybus.txt 2024-04-02T13:40:37.081 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/booting-without-of.txt 2024-04-02T13:40:37.086 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/changesets.txt 2024-04-02T13:40:37.088 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/dynamic-resolution-notes.txt 2024-04-02T13:40:37.090 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/of_selftest.txt 2024-04-02T13:40:37.093 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/overlay-notes.txt 2024-04-02T13:40:37.095 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/todo.txt 2024-04-02T13:40:37.098 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/devicetree/usage-model.txt 2024-04-02T13:40:37.101 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/digsig.txt 2024-04-02T13:40:37.103 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dma-buf-sharing.txt 2024-04-02T13:40:37.106 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dmaengine/ 2024-04-02T13:40:37.108 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dmaengine/00-INDEX 2024-04-02T13:40:37.109 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dmaengine/client.txt 2024-04-02T13:40:37.112 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dmaengine/dmatest.txt 2024-04-02T13:40:37.113 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dmaengine/provider.txt 2024-04-02T13:40:37.116 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dontdiff 2024-04-02T13:40:37.119 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/driver-model/ 2024-04-02T13:40:37.121 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/driver-model/binding.txt 2024-04-02T13:40:37.123 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/driver-model/bus.txt 2024-04-02T13:40:37.125 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/driver-model/class.txt 2024-04-02T13:40:37.127 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/driver-model/design-patterns.txt 2024-04-02T13:40:37.130 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/driver-model/device.txt 2024-04-02T13:40:37.132 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/driver-model/devres.txt 2024-04-02T13:40:37.134 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/driver-model/driver.txt 2024-04-02T13:40:37.137 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/driver-model/overview.txt 2024-04-02T13:40:37.139 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/driver-model/platform.txt 2024-04-02T13:40:37.142 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/driver-model/porting.txt 2024-04-02T13:40:37.144 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/ 2024-04-02T13:40:37.146 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/README.dvb-usb 2024-04-02T13:40:37.149 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/avermedia.txt 2024-04-02T13:40:37.152 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/bt8xx.txt 2024-04-02T13:40:37.154 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/cards.txt 2024-04-02T13:40:37.157 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/ci.txt 2024-04-02T13:40:37.159 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/contributors.txt 2024-04-02T13:40:37.161 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/faq.txt 2024-04-02T13:40:37.164 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/get_dvb_firmware 2024-04-02T13:40:37.166 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/lmedm04.txt 2024-04-02T13:40:37.169 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/opera-firmware.txt 2024-04-02T13:40:37.171 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/readme.txt 2024-04-02T13:40:37.173 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/technisat.txt 2024-04-02T13:40:37.175 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/ttusb-dec.txt 2024-04-02T13:40:37.178 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dvb/udev.txt 2024-04-02T13:40:37.180 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/dynamic-debug-howto.txt 2024-04-02T13:40:37.183 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/early-userspace/ 2024-04-02T13:40:37.185 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/early-userspace/README 2024-04-02T13:40:37.187 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/early-userspace/buffer-format.txt 2024-04-02T13:40:37.189 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/edac.txt 2024-04-02T13:40:37.192 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/efi-stub.txt 2024-04-02T13:40:37.195 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/eisa.txt 2024-04-02T13:40:37.197 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/email-clients.txt 2024-04-02T13:40:37.199 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/extcon/ 2024-04-02T13:40:37.201 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/extcon/porting-android-switch-class 2024-04-02T13:40:37.203 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fault-injection/ 2024-04-02T13:40:37.204 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fault-injection/fault-injection.txt 2024-04-02T13:40:37.206 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fault-injection/notifier-error-inject.txt 2024-04-02T13:40:37.209 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fault-injection/provoke-crashes.txt 2024-04-02T13:40:37.211 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/ 2024-04-02T13:40:37.213 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/00-INDEX 2024-04-02T13:40:37.215 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/api.txt 2024-04-02T13:40:37.217 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/arkfb.txt 2024-04-02T13:40:37.219 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/aty128fb.txt 2024-04-02T13:40:37.222 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/cirrusfb.txt 2024-04-02T13:40:37.224 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/cmap_xfbdev.txt 2024-04-02T13:40:37.226 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/deferred_io.txt 2024-04-02T13:40:37.228 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/efifb.txt 2024-04-02T13:40:37.231 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/ep93xx-fb.txt 2024-04-02T13:40:37.233 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/fbcon.txt 2024-04-02T13:40:37.235 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/framebuffer.txt 2024-04-02T13:40:37.238 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/gxfb.txt 2024-04-02T13:40:37.240 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/intel810.txt 2024-04-02T13:40:37.243 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/intelfb.txt 2024-04-02T13:40:37.246 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/internals.txt 2024-04-02T13:40:37.247 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/lxfb.txt 2024-04-02T13:40:37.250 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/matroxfb.txt 2024-04-02T13:40:37.252 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/metronomefb.txt 2024-04-02T13:40:37.255 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/modedb.txt 2024-04-02T13:40:37.256 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/pvr2fb.txt 2024-04-02T13:40:37.259 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/pxafb.txt 2024-04-02T13:40:37.261 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/s3fb.txt 2024-04-02T13:40:37.263 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/sa1100fb.txt 2024-04-02T13:40:37.265 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/sh7760fb.txt 2024-04-02T13:40:37.267 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/sisfb.txt 2024-04-02T13:40:37.270 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/sm501.txt 2024-04-02T13:40:37.272 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/sstfb.txt 2024-04-02T13:40:37.274 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/tgafb.txt 2024-04-02T13:40:37.277 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/tridentfb.txt 2024-04-02T13:40:37.279 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/udlfb.txt 2024-04-02T13:40:37.282 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/uvesafb.txt 2024-04-02T13:40:37.284 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/vesafb.txt 2024-04-02T13:40:37.286 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/viafb.modes 2024-04-02T13:40:37.289 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/viafb.txt 2024-04-02T13:40:37.291 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fb/vt8623fb.txt 2024-04-02T13:40:37.294 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/ 2024-04-02T13:40:37.296 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/.gitignore 2024-04-02T13:40:37.298 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/00-INDEX 2024-04-02T13:40:37.300 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/9p.txt 2024-04-02T13:40:37.302 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/Locking 2024-04-02T13:40:37.305 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/Makefile 2024-04-02T13:40:37.307 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/adfs.txt 2024-04-02T13:40:37.309 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/affs.txt 2024-04-02T13:40:37.312 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/afs.txt 2024-04-02T13:40:37.314 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/autofs4-mount-control.txt 2024-04-02T13:40:37.317 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/autofs4.txt 2024-04-02T13:40:37.320 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/automount-support.txt 2024-04-02T13:40:37.322 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/befs.txt 2024-04-02T13:40:37.325 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/bfs.txt 2024-04-02T13:40:37.329 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/btrfs.txt 2024-04-02T13:40:37.330 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/caching/ 2024-04-02T13:40:37.332 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/caching/backend-api.txt 2024-04-02T13:40:37.334 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/caching/cachefiles.txt 2024-04-02T13:40:37.337 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/caching/fscache.txt 2024-04-02T13:40:37.340 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/caching/netfs-api.txt 2024-04-02T13:40:37.343 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/caching/object.txt 2024-04-02T13:40:37.346 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/caching/operations.txt 2024-04-02T13:40:37.349 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/ceph.txt 2024-04-02T13:40:37.351 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/cifs/ 2024-04-02T13:40:37.353 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/cifs/AUTHORS 2024-04-02T13:40:37.355 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/cifs/CHANGES 2024-04-02T13:40:37.358 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/cifs/README 2024-04-02T13:40:37.361 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/cifs/TODO 2024-04-02T13:40:37.364 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/cifs/cifs.txt 2024-04-02T13:40:37.366 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/cifs/winucase_convert.pl 2024-04-02T13:40:37.368 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/coda.txt 2024-04-02T13:40:37.372 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/configfs/ 2024-04-02T13:40:37.374 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/configfs/Makefile 2024-04-02T13:40:37.376 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs.txt 2024-04-02T13:40:37.379 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_explicit.c 2024-04-02T13:40:37.382 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_macros.c 2024-04-02T13:40:37.384 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/cramfs.txt 2024-04-02T13:40:37.387 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/dax.txt 2024-04-02T13:40:37.390 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/debugfs.txt 2024-04-02T13:40:37.392 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/devpts.txt 2024-04-02T13:40:37.394 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/directory-locking 2024-04-02T13:40:37.396 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/dlmfs.txt 2024-04-02T13:40:37.399 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/dnotify.txt 2024-04-02T13:40:37.401 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/dnotify_test.c 2024-04-02T13:40:37.404 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/ecryptfs.txt 2024-04-02T13:40:37.417 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/efivarfs.txt 2024-04-02T13:40:37.419 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/exofs.txt 2024-04-02T13:40:37.422 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/ext2.txt 2024-04-02T13:40:37.424 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/ext3.txt 2024-04-02T13:40:37.426 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/ext4.txt 2024-04-02T13:40:37.429 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/f2fs.txt 2024-04-02T13:40:37.432 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/fiemap.txt 2024-04-02T13:40:37.434 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/files.txt 2024-04-02T13:40:37.437 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/fuse.txt 2024-04-02T13:40:37.439 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/gfs2-glocks.txt 2024-04-02T13:40:37.442 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/gfs2-uevents.txt 2024-04-02T13:40:37.444 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/gfs2.txt 2024-04-02T13:40:37.446 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/hfs.txt 2024-04-02T13:40:37.449 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/hfsplus.txt 2024-04-02T13:40:37.452 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/hpfs.txt 2024-04-02T13:40:37.454 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/inotify.txt 2024-04-02T13:40:37.456 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/isofs.txt 2024-04-02T13:40:37.459 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/jfs.txt 2024-04-02T13:40:37.461 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/locks.txt 2024-04-02T13:40:37.463 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/logfs.txt 2024-04-02T13:40:37.466 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/mandatory-locking.txt 2024-04-02T13:40:37.468 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/ncpfs.txt 2024-04-02T13:40:37.470 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/ 2024-04-02T13:40:37.472 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/00-INDEX 2024-04-02T13:40:37.474 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/Exporting 2024-04-02T13:40:37.477 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/fault_injection.txt 2024-04-02T13:40:37.479 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/idmapper.txt 2024-04-02T13:40:37.482 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/knfsd-stats.txt 2024-04-02T13:40:37.484 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs-rdma.txt 2024-04-02T13:40:37.487 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs.txt 2024-04-02T13:40:37.489 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs41-server.txt 2024-04-02T13:40:37.492 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsd-admin-interfaces.txt 2024-04-02T13:40:37.494 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsroot.txt 2024-04-02T13:40:37.497 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs-block-server.txt 2024-04-02T13:40:37.500 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs.txt 2024-04-02T13:40:37.503 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-cache.txt 2024-04-02T13:40:37.506 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-server-gss.txt 2024-04-02T13:40:37.509 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/nilfs2.txt 2024-04-02T13:40:37.512 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/ntfs.txt 2024-04-02T13:40:37.515 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/ocfs2.txt 2024-04-02T13:40:37.517 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/omfs.txt 2024-04-02T13:40:37.520 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/overlayfs.txt 2024-04-02T13:40:37.522 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/path-lookup.txt 2024-04-02T13:40:37.526 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/ 2024-04-02T13:40:37.526 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/design_notes.txt 2024-04-02T13:40:37.529 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/info.txt 2024-04-02T13:40:37.533 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/network_protocol.txt 2024-04-02T13:40:37.534 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/porting 2024-04-02T13:40:37.537 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/proc.txt 2024-04-02T13:40:37.541 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/qnx6.txt 2024-04-02T13:40:37.543 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/quota.txt 2024-04-02T13:40:37.546 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/ramfs-rootfs-initramfs.txt 2024-04-02T13:40:37.548 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/relay.txt 2024-04-02T13:40:37.551 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/romfs.txt 2024-04-02T13:40:37.554 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/seq_file.txt 2024-04-02T13:40:37.557 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/sharedsubtree.txt 2024-04-02T13:40:37.560 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/spufs.txt 2024-04-02T13:40:37.563 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/squashfs.txt 2024-04-02T13:40:37.566 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/sysfs-pci.txt 2024-04-02T13:40:37.568 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/sysfs-tagging.txt 2024-04-02T13:40:37.570 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/sysfs.txt 2024-04-02T13:40:37.573 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/sysv-fs.txt 2024-04-02T13:40:37.575 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/tmpfs.txt 2024-04-02T13:40:37.577 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/ubifs.txt 2024-04-02T13:40:37.579 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/udf.txt 2024-04-02T13:40:37.582 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/ufs.txt 2024-04-02T13:40:37.585 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/vfat.txt 2024-04-02T13:40:37.587 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/vfs.txt 2024-04-02T13:40:37.591 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/xfs-delayed-logging-design.txt 2024-04-02T13:40:37.594 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/xfs-self-describing-metadata.txt 2024-04-02T13:40:37.596 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/filesystems/xfs.txt 2024-04-02T13:40:37.599 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/firmware_class/ 2024-04-02T13:40:37.603 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/firmware_class/README 2024-04-02T13:40:37.605 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/firmware_class/hotplug-script 2024-04-02T13:40:37.607 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/flexible-arrays.txt 2024-04-02T13:40:37.610 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fmc/ 2024-04-02T13:40:37.612 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fmc/00-INDEX 2024-04-02T13:40:37.615 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fmc/API.txt 2024-04-02T13:40:37.618 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fmc/FMC-and-SDB.txt 2024-04-02T13:40:37.620 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fmc/carrier.txt 2024-04-02T13:40:37.623 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fmc/fmc-chardev.txt 2024-04-02T13:40:37.625 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fmc/fmc-fakedev.txt 2024-04-02T13:40:37.627 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fmc/fmc-trivial.txt 2024-04-02T13:40:37.630 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fmc/fmc-write-eeprom.txt 2024-04-02T13:40:37.632 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fmc/identifiers.txt 2024-04-02T13:40:37.635 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fmc/mezzanine.txt 2024-04-02T13:40:37.638 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/fmc/parameters.txt 2024-04-02T13:40:37.640 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/frv/ 2024-04-02T13:40:37.643 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/frv/README.txt 2024-04-02T13:40:37.645 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/frv/atomic-ops.txt 2024-04-02T13:40:37.647 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/frv/booting.txt 2024-04-02T13:40:37.650 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/frv/clock.txt 2024-04-02T13:40:37.652 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/frv/configuring.txt 2024-04-02T13:40:37.654 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/frv/features.txt 2024-04-02T13:40:37.657 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/frv/gdbinit 2024-04-02T13:40:37.659 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/frv/gdbstub.txt 2024-04-02T13:40:37.662 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/frv/kernel-ABI.txt 2024-04-02T13:40:37.664 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/frv/mmu-layout.txt 2024-04-02T13:40:37.667 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/futex-requeue-pi.txt 2024-04-02T13:40:37.669 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/gcov.txt 2024-04-02T13:40:37.672 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/gdb-kernel-debugging.txt 2024-04-02T13:40:37.674 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/gpio/ 2024-04-02T13:40:37.676 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/gpio/00-INDEX 2024-04-02T13:40:37.678 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/gpio/board.txt 2024-04-02T13:40:37.680 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/gpio/consumer.txt 2024-04-02T13:40:37.683 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/gpio/driver.txt 2024-04-02T13:40:37.685 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/gpio/gpio-legacy.txt 2024-04-02T13:40:37.688 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/gpio/gpio.txt 2024-04-02T13:40:37.690 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/gpio/sysfs.txt 2024-04-02T13:40:37.693 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hid/ 2024-04-02T13:40:37.695 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hid/hid-sensor.txt 2024-04-02T13:40:37.697 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hid/hid-transport.txt 2024-04-02T13:40:37.700 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hid/hiddev.txt 2024-04-02T13:40:37.702 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hid/hidraw.txt 2024-04-02T13:40:37.705 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hid/uhid.txt 2024-04-02T13:40:37.708 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/highuid.txt 2024-04-02T13:40:37.710 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hsi.txt 2024-04-02T13:40:37.713 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hw_random.txt 2024-04-02T13:40:37.716 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ 2024-04-02T13:40:37.717 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ab8500 2024-04-02T13:40:37.720 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/abituguru 2024-04-02T13:40:37.722 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/abituguru-datasheet 2024-04-02T13:40:37.725 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/abituguru3 2024-04-02T13:40:37.727 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/abx500 2024-04-02T13:40:37.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:37 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:40:37.730 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/acpi_power_meter 2024-04-02T13:40:37.732 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ad7314 2024-04-02T13:40:37.734 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/adc128d818 2024-04-02T13:40:37.736 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/adm1021 2024-04-02T13:40:37.739 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/adm1025 2024-04-02T13:40:37.742 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/adm1026 2024-04-02T13:40:37.744 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/adm1031 2024-04-02T13:40:37.746 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/adm1275 2024-04-02T13:40:37.749 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/adm9240 2024-04-02T13:40:37.751 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ads1015 2024-04-02T13:40:37.753 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ads7828 2024-04-02T13:40:37.755 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/adt7410 2024-04-02T13:40:37.758 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/adt7411 2024-04-02T13:40:37.760 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/adt7462 2024-04-02T13:40:37.762 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/adt7470 2024-04-02T13:40:37.764 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/adt7475 2024-04-02T13:40:37.767 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/amc6821 2024-04-02T13:40:37.769 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/asb100 2024-04-02T13:40:37.771 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/asc7621 2024-04-02T13:40:37.774 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/coretemp 2024-04-02T13:40:37.776 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/da9052 2024-04-02T13:40:37.779 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/da9055 2024-04-02T13:40:37.781 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/dme1737 2024-04-02T13:40:37.783 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ds1621 2024-04-02T13:40:37.786 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ds620 2024-04-02T13:40:37.788 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/emc1403 2024-04-02T13:40:37.790 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/emc2103 2024-04-02T13:40:37.792 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/emc6w201 2024-04-02T13:40:37.795 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/f71805f 2024-04-02T13:40:37.797 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/f71882fg 2024-04-02T13:40:37.800 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/fam15h_power 2024-04-02T13:40:37.802 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/g760a 2024-04-02T13:40:37.813 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/g762 2024-04-02T13:40:37.905 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/gl518sm 2024-04-02T13:40:37.908 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/hih6130 2024-04-02T13:40:37.910 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/htu21 2024-04-02T13:40:37.912 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/hwmon-kernel-api.txt 2024-04-02T13:40:37.914 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ibmaem 2024-04-02T13:40:37.916 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ibmpowernv 2024-04-02T13:40:37.919 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ina209 2024-04-02T13:40:37.921 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ina2xx 2024-04-02T13:40:37.923 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/it87 2024-04-02T13:40:37.925 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/jc42 2024-04-02T13:40:37.928 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/k10temp 2024-04-02T13:40:37.930 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/k8temp 2024-04-02T13:40:37.932 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lineage-pem 2024-04-02T13:40:37.934 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm25066 2024-04-02T13:40:37.937 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm63 2024-04-02T13:40:37.939 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm70 2024-04-02T13:40:37.941 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm73 2024-04-02T13:40:37.943 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm75 2024-04-02T13:40:37.945 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm77 2024-04-02T13:40:37.947 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm78 2024-04-02T13:40:37.949 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm80 2024-04-02T13:40:37.952 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm83 2024-04-02T13:40:37.955 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm85 2024-04-02T13:40:37.957 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm87 2024-04-02T13:40:37.960 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm90 2024-04-02T13:40:37.962 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm92 2024-04-02T13:40:37.964 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm93 2024-04-02T13:40:37.967 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm95234 2024-04-02T13:40:37.970 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/lm95245 2024-04-02T13:40:37.972 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ltc2945 2024-04-02T13:40:37.975 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ltc2978 2024-04-02T13:40:37.978 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ltc4151 2024-04-02T13:40:37.980 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ltc4215 2024-04-02T13:40:37.982 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ltc4245 2024-04-02T13:40:37.984 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ltc4260 2024-04-02T13:40:37.987 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ltc4261 2024-04-02T13:40:37.989 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/max16064 2024-04-02T13:40:37.991 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/max16065 2024-04-02T13:40:37.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:37 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:40:37.994 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/max1619 2024-04-02T13:40:37.996 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/max1668 2024-04-02T13:40:37.998 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/max197 2024-04-02T13:40:38.000 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/max34440 2024-04-02T13:40:38.003 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/max6639 2024-04-02T13:40:38.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:37 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:40:38.005 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/max6642 2024-04-02T13:40:38.007 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/max6650 2024-04-02T13:40:38.010 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/max6697 2024-04-02T13:40:38.012 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/max8688 2024-04-02T13:40:38.014 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/mc13783-adc 2024-04-02T13:40:38.016 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/mcp3021 2024-04-02T13:40:38.018 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/menf21bmc 2024-04-02T13:40:38.021 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/nct6683 2024-04-02T13:40:38.023 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/nct6775 2024-04-02T13:40:38.025 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/nct7802 2024-04-02T13:40:38.027 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ntc_thermistor 2024-04-02T13:40:38.030 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/pc87360 2024-04-02T13:40:38.033 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/pc87427 2024-04-02T13:40:38.036 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/pcf8591 2024-04-02T13:40:38.038 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/pmbus 2024-04-02T13:40:38.041 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/pmbus-core 2024-04-02T13:40:38.043 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/powr1220 2024-04-02T13:40:38.045 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/pwm-fan 2024-04-02T13:40:38.048 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/sch5627 2024-04-02T13:40:38.050 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/sch5636 2024-04-02T13:40:38.052 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/sht15 2024-04-02T13:40:38.055 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/sht21 2024-04-02T13:40:38.057 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/shtc1 2024-04-02T13:40:38.059 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/sis5595 2024-04-02T13:40:38.062 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/smm665 2024-04-02T13:40:38.065 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/smsc47b397 2024-04-02T13:40:38.067 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/smsc47m1 2024-04-02T13:40:38.069 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/smsc47m192 2024-04-02T13:40:38.072 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/submitting-patches 2024-04-02T13:40:38.074 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/sysfs-interface 2024-04-02T13:40:38.077 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/thmc50 2024-04-02T13:40:38.079 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/tmp102 2024-04-02T13:40:38.081 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/tmp103 2024-04-02T13:40:38.084 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/tmp401 2024-04-02T13:40:38.086 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/tmp421 2024-04-02T13:40:38.088 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/tps40422 2024-04-02T13:40:38.090 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/twl4030-madc-hwmon 2024-04-02T13:40:38.093 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ucd9000 2024-04-02T13:40:38.095 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/ucd9200 2024-04-02T13:40:38.097 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/userspace-tools 2024-04-02T13:40:38.100 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/vexpress 2024-04-02T13:40:38.107 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/via686a 2024-04-02T13:40:38.107 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/vt1211 2024-04-02T13:40:38.107 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/w83627ehf 2024-04-02T13:40:38.109 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/w83627hf 2024-04-02T13:40:38.112 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/w83781d 2024-04-02T13:40:38.114 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/w83791d 2024-04-02T13:40:38.116 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/w83792d 2024-04-02T13:40:38.119 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/w83793 2024-04-02T13:40:38.121 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/w83795 2024-04-02T13:40:38.124 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/w83l785ts 2024-04-02T13:40:38.126 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/w83l786ng 2024-04-02T13:40:38.128 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/wm831x 2024-04-02T13:40:38.130 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/wm8350 2024-04-02T13:40:38.133 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwmon/zl6100 2024-04-02T13:40:38.135 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/hwspinlock.txt 2024-04-02T13:40:38.139 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/ 2024-04-02T13:40:38.141 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/ 2024-04-02T13:40:38.143 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1535 2024-04-02T13:40:38.145 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1563 2024-04-02T13:40:38.147 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali15x3 2024-04-02T13:40:38.150 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd756 2024-04-02T13:40:38.153 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd8111 2024-04-02T13:40:38.156 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-diolan-u2c 2024-04-02T13:40:38.159 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-i801 2024-04-02T13:40:38.162 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ismt 2024-04-02T13:40:38.164 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-nforce2 2024-04-02T13:40:38.167 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ocores 2024-04-02T13:40:38.169 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport 2024-04-02T13:40:38.172 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport-light 2024-04-02T13:40:38.174 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-pca-isa 2024-04-02T13:40:38.177 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-piix4 2024-04-02T13:40:38.179 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis5595 2024-04-02T13:40:38.182 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis630 2024-04-02T13:40:38.184 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis96x 2024-04-02T13:40:38.187 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-taos-evm 2024-04-02T13:40:38.190 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-via 2024-04-02T13:40:38.192 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-viapro 2024-04-02T13:40:38.195 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/busses/scx200_acb 2024-04-02T13:40:38.197 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/dev-interface 2024-04-02T13:40:38.200 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/fault-codes 2024-04-02T13:40:38.202 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/functionality 2024-04-02T13:40:38.205 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/i2c-protocol 2024-04-02T13:40:38.208 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/i2c-stub 2024-04-02T13:40:38.210 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/instantiating-devices 2024-04-02T13:40:38.213 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/muxes/ 2024-04-02T13:40:38.214 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/muxes/i2c-mux-gpio 2024-04-02T13:40:38.217 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/old-module-parameters 2024-04-02T13:40:38.220 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/smbus-protocol 2024-04-02T13:40:38.223 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/summary 2024-04-02T13:40:38.225 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/ten-bit-addresses 2024-04-02T13:40:38.228 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/upgrading-clients 2024-04-02T13:40:38.232 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2c/writing-clients 2024-04-02T13:40:38.234 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2o/ 2024-04-02T13:40:38.237 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2o/README 2024-04-02T13:40:38.239 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/i2o/ioctl 2024-04-02T13:40:38.242 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ia64/ 2024-04-02T13:40:38.244 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ia64/.gitignore 2024-04-02T13:40:38.247 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ia64/IRQ-redir.txt 2024-04-02T13:40:38.249 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ia64/Makefile 2024-04-02T13:40:38.252 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ia64/README 2024-04-02T13:40:38.254 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ia64/aliasing-test.c 2024-04-02T13:40:38.256 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ia64/aliasing.txt 2024-04-02T13:40:38.259 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ia64/efirtc.txt 2024-04-02T13:40:38.262 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ia64/err_inject.txt 2024-04-02T13:40:38.265 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ia64/fsys.txt 2024-04-02T13:40:38.267 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ia64/mca.txt 2024-04-02T13:40:38.270 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ia64/serial.txt 2024-04-02T13:40:38.272 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ia64/xen.txt 2024-04-02T13:40:38.275 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ide/ 2024-04-02T13:40:38.278 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ide/00-INDEX 2024-04-02T13:40:38.281 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-cd.1994-2004 2024-04-02T13:40:38.284 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-floppy.1996-2002 2024-04-02T13:40:38.287 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-tape.1995-2002 2024-04-02T13:40:38.290 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ide/ide-tape.txt 2024-04-02T13:40:38.292 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ide/ide.txt 2024-04-02T13:40:38.295 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ide/warm-plug-howto.txt 2024-04-02T13:40:38.298 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/infiniband/ 2024-04-02T13:40:38.300 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/infiniband/core_locking.txt 2024-04-02T13:40:38.302 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/infiniband/ipoib.txt 2024-04-02T13:40:38.305 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/infiniband/sysfs.txt 2024-04-02T13:40:38.307 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/infiniband/user_mad.txt 2024-04-02T13:40:38.309 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/infiniband/user_verbs.txt 2024-04-02T13:40:38.312 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/init.txt 2024-04-02T13:40:38.315 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/initrd.txt 2024-04-02T13:40:38.318 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/ 2024-04-02T13:40:38.321 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/alps.txt 2024-04-02T13:40:38.323 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/amijoy.txt 2024-04-02T13:40:38.326 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/appletouch.txt 2024-04-02T13:40:38.329 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/atarikbd.txt 2024-04-02T13:40:38.333 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/bcm5974.txt 2024-04-02T13:40:38.336 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/cd32.txt 2024-04-02T13:40:38.339 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/cma3000_d0x.txt 2024-04-02T13:40:38.342 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/cs461x.txt 2024-04-02T13:40:38.345 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/edt-ft5x06.txt 2024-04-02T13:40:38.349 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/elantech.txt 2024-04-02T13:40:38.350 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/event-codes.txt 2024-04-02T13:40:38.353 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/ff.txt 2024-04-02T13:40:38.356 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/gamepad.txt 2024-04-02T13:40:38.358 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/gameport-programming.txt 2024-04-02T13:40:38.361 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/gpio-tilt.txt 2024-04-02T13:40:38.363 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/iforce-protocol.txt 2024-04-02T13:40:38.366 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/input-programming.txt 2024-04-02T13:40:38.369 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/input.txt 2024-04-02T13:40:38.372 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/interactive.fig 2024-04-02T13:40:38.375 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/joystick-api.txt 2024-04-02T13:40:38.378 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/joystick-parport.txt 2024-04-02T13:40:38.381 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/joystick.txt 2024-04-02T13:40:38.384 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/multi-touch-protocol.txt 2024-04-02T13:40:38.387 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/notifier.txt 2024-04-02T13:40:38.390 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/ntrig.txt 2024-04-02T13:40:38.393 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/rotary-encoder.txt 2024-04-02T13:40:38.396 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/sentelic.txt 2024-04-02T13:40:38.400 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/shape.fig 2024-04-02T13:40:38.403 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/walkera0701.txt 2024-04-02T13:40:38.406 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/xpad.txt 2024-04-02T13:40:38.409 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/input/yealink.txt 2024-04-02T13:40:38.412 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/intel_txt.txt 2024-04-02T13:40:38.415 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/io-mapping.txt 2024-04-02T13:40:38.418 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/io_ordering.txt 2024-04-02T13:40:38.421 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ioctl/ 2024-04-02T13:40:38.423 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ioctl/00-INDEX 2024-04-02T13:40:38.425 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ioctl/botching-up-ioctls.txt 2024-04-02T13:40:38.428 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ioctl/cdrom.txt 2024-04-02T13:40:38.431 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ioctl/hdio.txt 2024-04-02T13:40:38.434 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ioctl/ioctl-decoding.txt 2024-04-02T13:40:38.436 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ioctl/ioctl-number.txt 2024-04-02T13:40:38.439 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/iostats.txt 2024-04-02T13:40:38.442 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/irqflags-tracing.txt 2024-04-02T13:40:38.444 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isapnp.txt 2024-04-02T13:40:38.447 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/ 2024-04-02T13:40:38.449 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/00-INDEX 2024-04-02T13:40:38.452 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/CREDITS 2024-04-02T13:40:38.455 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/HiSax.cert 2024-04-02T13:40:38.457 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/INTERFACE 2024-04-02T13:40:38.461 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.CAPI 2024-04-02T13:40:38.463 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.fax 2024-04-02T13:40:38.466 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README 2024-04-02T13:40:38.469 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.FAQ 2024-04-02T13:40:38.472 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.HiSax 2024-04-02T13:40:38.475 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.act2000 2024-04-02T13:40:38.478 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.audio 2024-04-02T13:40:38.480 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.avmb1 2024-04-02T13:40:38.483 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.concap 2024-04-02T13:40:38.486 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.diversion 2024-04-02T13:40:38.488 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.fax 2024-04-02T13:40:38.491 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.gigaset 2024-04-02T13:40:38.494 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.hfc-pci 2024-04-02T13:40:38.496 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.hysdn 2024-04-02T13:40:38.499 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.icn 2024-04-02T13:40:38.501 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.mISDN 2024-04-02T13:40:38.504 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.pcbit 2024-04-02T13:40:38.506 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.sc 2024-04-02T13:40:38.509 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.syncppp 2024-04-02T13:40:38.512 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/README.x25 2024-04-02T13:40:38.515 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/isdn/syncPPP.FAQ 2024-04-02T13:40:38.518 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ja_JP/ 2024-04-02T13:40:38.521 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ja_JP/HOWTO 2024-04-02T13:40:38.524 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ja_JP/SubmitChecklist 2024-04-02T13:40:38.527 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ja_JP/SubmittingPatches 2024-04-02T13:40:38.530 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ja_JP/stable_api_nonsense.txt 2024-04-02T13:40:38.533 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ja_JP/stable_kernel_rules.txt 2024-04-02T13:40:38.536 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/java.txt 2024-04-02T13:40:38.539 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kasan.txt 2024-04-02T13:40:38.542 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kbuild/ 2024-04-02T13:40:38.544 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kbuild/00-INDEX 2024-04-02T13:40:38.547 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kbuild/headers_install.txt 2024-04-02T13:40:38.549 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kbuild/kbuild.txt 2024-04-02T13:40:38.553 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kbuild/kconfig-language.txt 2024-04-02T13:40:38.555 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kbuild/kconfig.txt 2024-04-02T13:40:38.558 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kbuild/makefiles.txt 2024-04-02T13:40:38.562 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kbuild/modules.txt 2024-04-02T13:40:38.565 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kdump/ 2024-04-02T13:40:38.568 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kdump/gdbmacros.txt 2024-04-02T13:40:38.570 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kdump/kdump.txt 2024-04-02T13:40:38.573 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kernel-doc-nano-HOWTO.txt 2024-04-02T13:40:38.577 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kernel-docs.txt 2024-04-02T13:40:38.580 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kernel-parameters.txt 2024-04-02T13:40:38.586 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kernel-per-CPU-kthreads.txt 2024-04-02T13:40:38.589 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kmemcheck.txt 2024-04-02T13:40:38.593 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kmemleak.txt 2024-04-02T13:40:38.596 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ko_KR/ 2024-04-02T13:40:38.597 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ko_KR/HOWTO 2024-04-02T13:40:38.601 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ko_KR/stable_api_nonsense.txt 2024-04-02T13:40:38.603 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kobject.txt 2024-04-02T13:40:38.607 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kprobes.txt 2024-04-02T13:40:38.611 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kref.txt 2024-04-02T13:40:38.614 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/kselftest.txt 2024-04-02T13:40:38.617 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/laptops/ 2024-04-02T13:40:38.619 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/laptops/.gitignore 2024-04-02T13:40:38.622 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/laptops/00-INDEX 2024-04-02T13:40:38.625 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/laptops/Makefile 2024-04-02T13:40:38.628 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/laptops/asus-laptop.txt 2024-04-02T13:40:38.630 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/laptops/disk-shock-protection.txt 2024-04-02T13:40:38.633 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/laptops/dslm.c 2024-04-02T13:40:38.636 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/laptops/freefall.c 2024-04-02T13:40:38.638 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/laptops/laptop-mode.txt 2024-04-02T13:40:38.642 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/laptops/sony-laptop.txt 2024-04-02T13:40:38.644 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/laptops/sonypi.txt 2024-04-02T13:40:38.647 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/laptops/thinkpad-acpi.txt 2024-04-02T13:40:38.651 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/laptops/toshiba_haps.txt 2024-04-02T13:40:38.654 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ldm.txt 2024-04-02T13:40:38.657 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/leds/ 2024-04-02T13:40:38.659 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/leds/00-INDEX 2024-04-02T13:40:38.661 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/leds/leds-blinkm.txt 2024-04-02T13:40:38.664 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/leds/leds-class.txt 2024-04-02T13:40:38.666 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/leds/leds-lm3556.txt 2024-04-02T13:40:38.669 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/leds/leds-lp3944.txt 2024-04-02T13:40:38.671 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/leds/leds-lp5521.txt 2024-04-02T13:40:38.674 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/leds/leds-lp5523.txt 2024-04-02T13:40:38.677 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/leds/leds-lp5562.txt 2024-04-02T13:40:38.680 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/leds/leds-lp55xx.txt 2024-04-02T13:40:38.682 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/leds/ledtrig-oneshot.txt 2024-04-02T13:40:38.684 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/leds/ledtrig-transient.txt 2024-04-02T13:40:38.687 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/local_ops.txt 2024-04-02T13:40:38.691 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/locking/ 2024-04-02T13:40:38.692 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/locking/00-INDEX 2024-04-02T13:40:38.695 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/locking/lglock.txt 2024-04-02T13:40:38.698 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/locking/lockdep-design.txt 2024-04-02T13:40:38.700 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/locking/lockstat.txt 2024-04-02T13:40:38.703 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/locking/locktorture.txt 2024-04-02T13:40:38.705 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/locking/mutex-design.txt 2024-04-02T13:40:38.708 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/locking/rt-mutex-design.txt 2024-04-02T13:40:38.712 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/locking/rt-mutex.txt 2024-04-02T13:40:38.714 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/locking/spinlocks.txt 2024-04-02T13:40:38.716 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/locking/ww-mutex-design.txt 2024-04-02T13:40:38.720 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/lockup-watchdogs.txt 2024-04-02T13:40:38.722 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/logo.gif 2024-04-02T13:40:38.725 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/logo.txt 2024-04-02T13:40:38.728 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/lzo.txt 2024-04-02T13:40:38.731 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/m68k/ 2024-04-02T13:40:38.734 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/m68k/00-INDEX 2024-04-02T13:40:38.737 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/m68k/README.buddha 2024-04-02T13:40:38.739 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/m68k/kernel-options.txt 2024-04-02T13:40:38.743 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/magic-number.txt 2024-04-02T13:40:38.746 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mailbox.txt 2024-04-02T13:40:38.749 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/md.txt 2024-04-02T13:40:38.752 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/media-framework.txt 2024-04-02T13:40:38.755 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/memory-barriers.txt 2024-04-02T13:40:38.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:38 smithi145 ceph-mon[35646]: pgmap v58: 129 pgs: 129 active+clean; 89 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 798 KiB/s wr, 79 op/s 2024-04-02T13:40:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:38 smithi050 ceph-mon[35524]: pgmap v58: 129 pgs: 129 active+clean; 89 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 798 KiB/s wr, 79 op/s 2024-04-02T13:40:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:38 smithi033 ceph-mon[32390]: pgmap v58: 129 pgs: 129 active+clean; 89 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 798 KiB/s wr, 79 op/s 2024-04-02T13:40:39.038 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/memory-devices/ 2024-04-02T13:40:39.072 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/memory-devices/ti-emif.txt 2024-04-02T13:40:39.074 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/memory-hotplug.txt 2024-04-02T13:40:39.077 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/metag/ 2024-04-02T13:40:39.079 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/metag/00-INDEX 2024-04-02T13:40:39.081 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/metag/kernel-ABI.txt 2024-04-02T13:40:39.084 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mic/ 2024-04-02T13:40:39.086 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mic/Makefile 2024-04-02T13:40:39.088 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mic/mic_overview.txt 2024-04-02T13:40:39.090 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mic/mpssd/ 2024-04-02T13:40:39.093 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mic/mpssd/.gitignore 2024-04-02T13:40:39.095 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mic/mpssd/Makefile 2024-04-02T13:40:39.098 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mic/mpssd/micctrl 2024-04-02T13:40:39.101 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mic/mpssd/mpss 2024-04-02T13:40:39.103 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.c 2024-04-02T13:40:39.106 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.h 2024-04-02T13:40:39.108 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mic/mpssd/sysfs.c 2024-04-02T13:40:39.110 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mips/ 2024-04-02T13:40:39.113 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mips/00-INDEX 2024-04-02T13:40:39.116 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mips/AU1xxx_IDE.README 2024-04-02T13:40:39.118 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/ 2024-04-02T13:40:39.120 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/Makefile 2024-04-02T13:40:39.122 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/ad525x_dpot.txt 2024-04-02T13:40:39.124 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/apds990x.txt 2024-04-02T13:40:39.127 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/bh1770glc.txt 2024-04-02T13:40:39.129 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/c2port.txt 2024-04-02T13:40:39.131 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/eeprom 2024-04-02T13:40:39.134 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/ics932s401 2024-04-02T13:40:39.136 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/isl29003 2024-04-02T13:40:39.139 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/lis3lv02d 2024-04-02T13:40:39.141 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/max6875 2024-04-02T13:40:39.143 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/mei/ 2024-04-02T13:40:39.145 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/mei/.gitignore 2024-04-02T13:40:39.147 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/mei/Makefile 2024-04-02T13:40:39.149 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/mei/TODO 2024-04-02T13:40:39.151 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-amt-version.c 2024-04-02T13:40:39.154 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-client-bus.txt 2024-04-02T13:40:39.157 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei.txt 2024-04-02T13:40:39.159 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/misc-devices/spear-pcie-gadget.txt 2024-04-02T13:40:39.161 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mmc/ 2024-04-02T13:40:39.164 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mmc/00-INDEX 2024-04-02T13:40:39.165 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mmc/mmc-async-req.txt 2024-04-02T13:40:39.168 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-attrs.txt 2024-04-02T13:40:39.170 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-parts.txt 2024-04-02T13:40:39.173 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mn10300/ 2024-04-02T13:40:39.175 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mn10300/ABI.txt 2024-04-02T13:40:39.176 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mn10300/compartmentalisation.txt 2024-04-02T13:40:39.179 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/module-signing.txt 2024-04-02T13:40:39.182 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mono.txt 2024-04-02T13:40:39.184 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mtd/ 2024-04-02T13:40:39.186 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mtd/nand/ 2024-04-02T13:40:39.188 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt 2024-04-02T13:40:39.190 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mtd/nand_ecc.txt 2024-04-02T13:40:39.193 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/mtd/spi-nor.txt 2024-04-02T13:40:39.195 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/namespaces/ 2024-04-02T13:40:39.197 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/namespaces/compatibility-list.txt 2024-04-02T13:40:39.199 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/namespaces/resource-control.txt 2024-04-02T13:40:39.201 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/netlabel/ 2024-04-02T13:40:39.203 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/netlabel/00-INDEX 2024-04-02T13:40:39.205 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/netlabel/cipso_ipv4.txt 2024-04-02T13:40:39.207 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/netlabel/draft-ietf-cipso-ipsecurity-01.txt 2024-04-02T13:40:39.210 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/netlabel/introduction.txt 2024-04-02T13:40:39.212 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/netlabel/lsm_interface.txt 2024-04-02T13:40:39.216 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ 2024-04-02T13:40:39.217 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/00-INDEX 2024-04-02T13:40:39.219 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/3c509.txt 2024-04-02T13:40:39.221 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/6pack.txt 2024-04-02T13:40:39.223 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/LICENSE.qla3xxx 2024-04-02T13:40:39.226 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlcnic 2024-04-02T13:40:39.230 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlge 2024-04-02T13:40:39.232 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/Makefile 2024-04-02T13:40:39.234 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/PLIP.txt 2024-04-02T13:40:39.237 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/README.ipw2100 2024-04-02T13:40:39.239 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/README.ipw2200 2024-04-02T13:40:39.241 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/README.sb1000 2024-04-02T13:40:39.244 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/alias.txt 2024-04-02T13:40:39.246 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/altera_tse.txt 2024-04-02T13:40:39.249 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/arcnet-hardware.txt 2024-04-02T13:40:39.253 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/arcnet.txt 2024-04-02T13:40:39.256 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/atm.txt 2024-04-02T13:40:39.258 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ax25.txt 2024-04-02T13:40:39.260 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/batman-adv.txt 2024-04-02T13:40:39.263 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/baycom.txt 2024-04-02T13:40:39.265 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/bonding.txt 2024-04-02T13:40:39.269 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/bridge.txt 2024-04-02T13:40:39.272 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/caif/ 2024-04-02T13:40:39.273 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/caif/Linux-CAIF.txt 2024-04-02T13:40:39.276 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/caif/README 2024-04-02T13:40:39.278 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/caif/spi_porting.txt 2024-04-02T13:40:39.281 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/can.txt 2024-04-02T13:40:39.285 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/cdc_mbim.txt 2024-04-02T13:40:39.288 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/cops.txt 2024-04-02T13:40:39.290 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/cs89x0.txt 2024-04-02T13:40:39.293 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/cxacru-cf.py 2024-04-02T13:40:39.296 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/cxacru.txt 2024-04-02T13:40:39.298 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/cxgb.txt 2024-04-02T13:40:39.300 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/dccp.txt 2024-04-02T13:40:39.303 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/dctcp.txt 2024-04-02T13:40:39.305 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/de4x5.txt 2024-04-02T13:40:39.308 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/decnet.txt 2024-04-02T13:40:39.310 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/dl2k.txt 2024-04-02T13:40:39.312 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/dm9000.txt 2024-04-02T13:40:39.314 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/dmfe.txt 2024-04-02T13:40:39.317 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/dns_resolver.txt 2024-04-02T13:40:39.319 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/driver.txt 2024-04-02T13:40:39.322 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/e100.txt 2024-04-02T13:40:39.324 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/e1000.txt 2024-04-02T13:40:39.327 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/e1000e.txt 2024-04-02T13:40:39.329 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/eql.txt 2024-04-02T13:40:39.332 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/fib_trie.txt 2024-04-02T13:40:39.334 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/filter.txt 2024-04-02T13:40:39.337 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/fore200e.txt 2024-04-02T13:40:39.340 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/framerelay.txt 2024-04-02T13:40:39.342 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/gen_stats.txt 2024-04-02T13:40:39.344 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/generic-hdlc.txt 2024-04-02T13:40:39.347 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/generic_netlink.txt 2024-04-02T13:40:39.349 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/gianfar.txt 2024-04-02T13:40:39.352 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/i40e.txt 2024-04-02T13:40:39.354 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/i40evf.txt 2024-04-02T13:40:39.356 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ieee802154.txt 2024-04-02T13:40:39.358 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/igb.txt 2024-04-02T13:40:39.361 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/igbvf.txt 2024-04-02T13:40:39.363 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ip-sysctl.txt 2024-04-02T13:40:39.367 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ip_dynaddr.txt 2024-04-02T13:40:39.369 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ipddp.txt 2024-04-02T13:40:39.371 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/iphase.txt 2024-04-02T13:40:39.374 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ipsec.txt 2024-04-02T13:40:39.377 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ipv6.txt 2024-04-02T13:40:39.379 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ipvlan.txt 2024-04-02T13:40:39.382 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ipvs-sysctl.txt 2024-04-02T13:40:39.384 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/irda.txt 2024-04-02T13:40:39.387 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ixgb.txt 2024-04-02T13:40:39.389 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ixgbe.txt 2024-04-02T13:40:39.392 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ixgbevf.txt 2024-04-02T13:40:39.394 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/l2tp.txt 2024-04-02T13:40:39.397 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/lapb-module.txt 2024-04-02T13:40:39.400 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ltpc.txt 2024-04-02T13:40:39.401 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/mac80211-auth-assoc-deauth.txt 2024-04-02T13:40:39.403 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/mac80211-injection.txt 2024-04-02T13:40:39.406 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/ 2024-04-02T13:40:39.407 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/README 2024-04-02T13:40:39.410 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/hostapd.conf 2024-04-02T13:40:39.411 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/wpa_supplicant.conf 2024-04-02T13:40:39.414 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/multiqueue.txt 2024-04-02T13:40:39.417 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/netconsole.txt 2024-04-02T13:40:39.420 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/netdev-FAQ.txt 2024-04-02T13:40:39.423 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/netdev-features.txt 2024-04-02T13:40:39.425 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/netdevices.txt 2024-04-02T13:40:39.428 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/netif-msg.txt 2024-04-02T13:40:39.430 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/netlink_mmap.txt 2024-04-02T13:40:39.433 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/nf_conntrack-sysctl.txt 2024-04-02T13:40:39.435 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/nfc.txt 2024-04-02T13:40:39.438 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/openvswitch.txt 2024-04-02T13:40:39.440 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/operstates.txt 2024-04-02T13:40:39.443 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/packet_mmap.txt 2024-04-02T13:40:39.446 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/phonet.txt 2024-04-02T13:40:39.448 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/phy.txt 2024-04-02T13:40:39.451 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/pktgen.txt 2024-04-02T13:40:39.453 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/policy-routing.txt 2024-04-02T13:40:39.455 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ppp_generic.txt 2024-04-02T13:40:39.460 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/proc_net_tcp.txt 2024-04-02T13:40:39.461 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/radiotap-headers.txt 2024-04-02T13:40:39.463 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/ray_cs.txt 2024-04-02T13:40:39.465 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/rds.txt 2024-04-02T13:40:39.468 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/regulatory.txt 2024-04-02T13:40:39.470 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/rxrpc.txt 2024-04-02T13:40:39.473 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/s2io.txt 2024-04-02T13:40:39.475 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/scaling.txt 2024-04-02T13:40:39.478 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/sctp.txt 2024-04-02T13:40:39.480 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/secid.txt 2024-04-02T13:40:39.482 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/skfp.txt 2024-04-02T13:40:39.485 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/smc9.txt 2024-04-02T13:40:39.487 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/spider_net.txt 2024-04-02T13:40:39.490 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/stmmac.txt 2024-04-02T13:40:39.492 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/switchdev.txt 2024-04-02T13:40:39.494 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/tc-actions-env-rules.txt 2024-04-02T13:40:39.496 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/tcp-thin.txt 2024-04-02T13:40:39.498 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/tcp.txt 2024-04-02T13:40:39.501 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/team.txt 2024-04-02T13:40:39.503 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/timestamping.txt 2024-04-02T13:40:39.506 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/timestamping/ 2024-04-02T13:40:39.508 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/timestamping/.gitignore 2024-04-02T13:40:39.510 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/timestamping/Makefile 2024-04-02T13:40:39.512 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c 2024-04-02T13:40:39.514 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/timestamping/timestamping.c 2024-04-02T13:40:39.517 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c 2024-04-02T13:40:39.520 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/tlan.txt 2024-04-02T13:40:39.523 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/tproxy.txt 2024-04-02T13:40:39.526 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/tuntap.txt 2024-04-02T13:40:39.529 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/udplite.txt 2024-04-02T13:40:39.531 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/vortex.txt 2024-04-02T13:40:39.534 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/vxge.txt 2024-04-02T13:40:39.536 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/vxlan.txt 2024-04-02T13:40:39.538 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/x25-iface.txt 2024-04-02T13:40:39.541 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/x25.txt 2024-04-02T13:40:39.543 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/xfrm_proc.txt 2024-04-02T13:40:39.545 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/xfrm_sync.txt 2024-04-02T13:40:39.549 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/xfrm_sysctl.txt 2024-04-02T13:40:39.549 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/networking/z8530drv.txt 2024-04-02T13:40:39.552 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/nfc/ 2024-04-02T13:40:39.554 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/nfc/nfc-hci.txt 2024-04-02T13:40:39.556 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/nfc/nfc-pn544.txt 2024-04-02T13:40:39.559 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/nios2/ 2024-04-02T13:40:39.561 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/nios2/README 2024-04-02T13:40:39.563 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/nommu-mmap.txt 2024-04-02T13:40:39.566 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/numastat.txt 2024-04-02T13:40:39.568 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/oops-tracing.txt 2024-04-02T13:40:39.570 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/padata.txt 2024-04-02T13:40:39.573 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/parisc/ 2024-04-02T13:40:39.575 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/parisc/00-INDEX 2024-04-02T13:40:39.577 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/parisc/debugging 2024-04-02T13:40:39.580 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/parisc/registers 2024-04-02T13:40:39.582 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/parport-lowlevel.txt 2024-04-02T13:40:39.585 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/parport.txt 2024-04-02T13:40:39.587 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pcmcia/ 2024-04-02T13:40:39.589 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pcmcia/.gitignore 2024-04-02T13:40:39.591 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pcmcia/Makefile 2024-04-02T13:40:39.593 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pcmcia/crc32hash.c 2024-04-02T13:40:39.595 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pcmcia/devicetable.txt 2024-04-02T13:40:39.598 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pcmcia/driver-changes.txt 2024-04-02T13:40:39.600 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pcmcia/driver.txt 2024-04-02T13:40:39.603 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pcmcia/locking.txt 2024-04-02T13:40:39.605 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/percpu-rw-semaphore.txt 2024-04-02T13:40:39.608 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/phy.txt 2024-04-02T13:40:39.610 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/phy/ 2024-04-02T13:40:39.612 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/phy/samsung-usb2.txt 2024-04-02T13:40:39.614 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pi-futex.txt 2024-04-02T13:40:39.617 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pinctrl.txt 2024-04-02T13:40:39.620 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/platform/ 2024-04-02T13:40:39.622 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/platform/x86-laptop-drivers.txt 2024-04-02T13:40:39.624 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pnp.txt 2024-04-02T13:40:39.627 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/ 2024-04-02T13:40:39.629 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/00-INDEX 2024-04-02T13:40:39.631 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/apm-acpi.txt 2024-04-02T13:40:39.634 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/basic-pm-debugging.txt 2024-04-02T13:40:39.636 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/charger-manager.txt 2024-04-02T13:40:39.638 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/devices.txt 2024-04-02T13:40:39.641 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/drivers-testing.txt 2024-04-02T13:40:39.644 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/freezing-of-tasks.txt 2024-04-02T13:40:39.646 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/interface.txt 2024-04-02T13:40:39.648 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/notifiers.txt 2024-04-02T13:40:39.650 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/opp.txt 2024-04-02T13:40:39.652 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/pci.txt 2024-04-02T13:40:39.656 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/pm_qos_interface.txt 2024-04-02T13:40:39.659 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/power_supply_class.txt 2024-04-02T13:40:39.661 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/powercap/ 2024-04-02T13:40:39.663 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/powercap/powercap.txt 2024-04-02T13:40:39.665 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/regulator/ 2024-04-02T13:40:39.668 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/regulator/consumer.txt 2024-04-02T13:40:39.670 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/regulator/design.txt 2024-04-02T13:40:39.672 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/regulator/machine.txt 2024-04-02T13:40:39.675 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/regulator/overview.txt 2024-04-02T13:40:39.677 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/regulator/regulator.txt 2024-04-02T13:40:39.679 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/runtime_pm.txt 2024-04-02T13:40:39.683 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/s2ram.txt 2024-04-02T13:40:39.686 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/states.txt 2024-04-02T13:40:39.688 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/suspend-and-cpuhotplug.txt 2024-04-02T13:40:39.690 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/suspend-and-interrupts.txt 2024-04-02T13:40:39.693 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/swsusp-and-swap-files.txt 2024-04-02T13:40:39.695 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/swsusp-dmcrypt.txt 2024-04-02T13:40:39.698 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/swsusp.txt 2024-04-02T13:40:39.702 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/tricks.txt 2024-04-02T13:40:39.702 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/userland-swsusp.txt 2024-04-02T13:40:39.705 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/power/video.txt 2024-04-02T13:40:39.707 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/ 2024-04-02T13:40:39.709 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/00-INDEX 2024-04-02T13:40:39.714 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/bootwrapper.txt 2024-04-02T13:40:39.714 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/cpu_families.txt 2024-04-02T13:40:39.716 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/cpu_features.txt 2024-04-02T13:40:39.718 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/cxl.txt 2024-04-02T13:40:39.721 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/eeh-pci-error-recovery.txt 2024-04-02T13:40:39.724 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/firmware-assisted-dump.txt 2024-04-02T13:40:39.726 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/hvcs.txt 2024-04-02T13:40:39.729 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/mpc52xx.txt 2024-04-02T13:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/pmu-ebb.txt 2024-04-02T13:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/ptrace.txt 2024-04-02T13:40:39.736 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/qe_firmware.txt 2024-04-02T13:40:39.739 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/powerpc/transactional_memory.txt 2024-04-02T13:40:39.742 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pps/ 2024-04-02T13:40:39.744 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pps/pps.txt 2024-04-02T13:40:39.747 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/prctl/ 2024-04-02T13:40:39.749 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/prctl/.gitignore 2024-04-02T13:40:39.751 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/prctl/Makefile 2024-04-02T13:40:39.753 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-ctxt-sw-stress-test.c 2024-04-02T13:40:39.755 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-on-off-stress-test.c 2024-04-02T13:40:39.758 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-test.c 2024-04-02T13:40:39.760 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/prctl/no_new_privs.txt 2024-04-02T13:40:39.762 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/prctl/seccomp_filter.txt 2024-04-02T13:40:39.764 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/preempt-locking.txt 2024-04-02T13:40:39.767 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/printk-formats.txt 2024-04-02T13:40:39.769 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pti/ 2024-04-02T13:40:39.771 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pti/pti_intel_mid.txt 2024-04-02T13:40:39.773 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ptp/ 2024-04-02T13:40:39.775 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ptp/.gitignore 2024-04-02T13:40:39.777 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ptp/Makefile 2024-04-02T13:40:39.779 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ptp/ptp.txt 2024-04-02T13:40:39.781 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ptp/testptp.c 2024-04-02T13:40:39.783 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ptp/testptp.mk 2024-04-02T13:40:39.786 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/pwm.txt 2024-04-02T13:40:39.788 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/ramoops.txt 2024-04-02T13:40:39.791 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/rapidio/ 2024-04-02T13:40:39.792 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/rapidio/rapidio.txt 2024-04-02T13:40:39.795 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/rapidio/sysfs.txt 2024-04-02T13:40:39.798 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/rapidio/tsi721.txt 2024-04-02T13:40:39.800 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/rbtree.txt 2024-04-02T13:40:39.803 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/remoteproc.txt 2024-04-02T13:40:39.806 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/rfkill.txt 2024-04-02T13:40:39.808 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/robust-futex-ABI.txt 2024-04-02T13:40:39.810 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/robust-futexes.txt 2024-04-02T13:40:39.813 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/rpmsg.txt 2024-04-02T13:40:39.816 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/rtc.txt 2024-04-02T13:40:39.818 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/ 2024-04-02T13:40:39.820 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/00-INDEX 2024-04-02T13:40:39.822 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/3270.ChangeLog 2024-04-02T13:40:39.824 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/3270.txt 2024-04-02T13:40:39.826 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/CommonIO 2024-04-02T13:40:39.829 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/DASD 2024-04-02T13:40:39.831 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/Debugging390.txt 2024-04-02T13:40:39.853 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/cds.txt 2024-04-02T13:40:40.027 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/config3270.sh 2024-04-02T13:40:40.027 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/driver-model.txt 2024-04-02T13:40:40.027 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/kvm.txt 2024-04-02T13:40:40.027 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/monreader.txt 2024-04-02T13:40:40.027 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/qeth.txt 2024-04-02T13:40:40.027 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/s390dbf.txt 2024-04-02T13:40:40.027 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/s390/zfcpdump.txt 2024-04-02T13:40:40.027 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scheduler/ 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scheduler/00-INDEX 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scheduler/completion.txt 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scheduler/sched-arch.txt 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scheduler/sched-bwc.txt 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scheduler/sched-deadline.txt 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scheduler/sched-design-CFS.txt 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scheduler/sched-domains.txt 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scheduler/sched-nice-design.txt 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scheduler/sched-rt-group.txt 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scheduler/sched-stats.txt 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/ 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/00-INDEX 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/53c700.txt 2024-04-02T13:40:40.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/BusLogic.txt 2024-04-02T13:40:40.029 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.1992-1997 2024-04-02T13:40:40.029 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.arcmsr 2024-04-02T13:40:40.029 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ips 2024-04-02T13:40:40.029 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.lpfc 2024-04-02T13:40:40.030 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid 2024-04-02T13:40:40.033 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid_sas 2024-04-02T13:40:40.036 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ncr53c8xx 2024-04-02T13:40:40.039 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx 2024-04-02T13:40:40.042 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx_2 2024-04-02T13:40:40.044 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/FlashPoint.txt 2024-04-02T13:40:40.046 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/LICENSE.FlashPoint 2024-04-02T13:40:40.048 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla2xxx 2024-04-02T13:40:40.051 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla4xxx 2024-04-02T13:40:40.053 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/Mylex.txt 2024-04-02T13:40:40.055 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/NinjaSCSI.txt 2024-04-02T13:40:40.058 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/aacraid.txt 2024-04-02T13:40:40.060 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/advansys.txt 2024-04-02T13:40:40.062 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/aha152x.txt 2024-04-02T13:40:40.064 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/aic79xx.txt 2024-04-02T13:40:40.067 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/aic7xxx.txt 2024-04-02T13:40:40.070 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/arcmsr_spec.txt 2024-04-02T13:40:40.074 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/bfa.txt 2024-04-02T13:40:40.075 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/bnx2fc.txt 2024-04-02T13:40:40.077 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/cxgb3i.txt 2024-04-02T13:40:40.080 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/dc395x.txt 2024-04-02T13:40:40.082 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/dpti.txt 2024-04-02T13:40:40.084 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/dtc3x80.txt 2024-04-02T13:40:40.087 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/g_NCR5380.txt 2024-04-02T13:40:40.089 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/hpsa.txt 2024-04-02T13:40:40.091 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/hptiop.txt 2024-04-02T13:40:40.094 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/in2000.txt 2024-04-02T13:40:40.096 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/libsas.txt 2024-04-02T13:40:40.099 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/link_power_management_policy.txt 2024-04-02T13:40:40.101 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/lpfc.txt 2024-04-02T13:40:40.103 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/megaraid.txt 2024-04-02T13:40:40.106 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/ncr53c8xx.txt 2024-04-02T13:40:40.110 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/osd.txt 2024-04-02T13:40:40.112 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/osst.txt 2024-04-02T13:40:40.115 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/ppa.txt 2024-04-02T13:40:40.118 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/qlogicfas.txt 2024-04-02T13:40:40.119 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/scsi-changer.txt 2024-04-02T13:40:40.122 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/scsi-generic.txt 2024-04-02T13:40:40.124 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/scsi-parameters.txt 2024-04-02T13:40:40.126 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/scsi.txt 2024-04-02T13:40:40.129 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/scsi_eh.txt 2024-04-02T13:40:40.131 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/scsi_fc_transport.txt 2024-04-02T13:40:40.134 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/scsi_mid_low_api.txt 2024-04-02T13:40:40.137 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/ 2024-04-02T13:40:40.139 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/Makefile 2024-04-02T13:40:40.141 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/rport_state_diagram.dot 2024-04-02T13:40:40.144 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/st.txt 2024-04-02T13:40:40.147 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/sym53c500_cs.txt 2024-04-02T13:40:40.149 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/sym53c8xx_2.txt 2024-04-02T13:40:40.152 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/tmscsim.txt 2024-04-02T13:40:40.155 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/ufs.txt 2024-04-02T13:40:40.157 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/scsi/wd719x.txt 2024-04-02T13:40:40.159 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/ 2024-04-02T13:40:40.161 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/00-INDEX 2024-04-02T13:40:40.163 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/IMA-templates.txt 2024-04-02T13:40:40.166 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/LSM.txt 2024-04-02T13:40:40.168 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/SELinux.txt 2024-04-02T13:40:40.170 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/Smack.txt 2024-04-02T13:40:40.173 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/Yama.txt 2024-04-02T13:40:40.176 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/apparmor.txt 2024-04-02T13:40:40.178 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/credentials.txt 2024-04-02T13:40:40.181 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/keys-ecryptfs.txt 2024-04-02T13:40:40.183 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/keys-request-key.txt 2024-04-02T13:40:40.185 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/keys-trusted-encrypted.txt 2024-04-02T13:40:40.188 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/keys.txt 2024-04-02T13:40:40.191 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/security/tomoyo.txt 2024-04-02T13:40:40.193 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/serial-console.txt 2024-04-02T13:40:40.196 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/serial/ 2024-04-02T13:40:40.198 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/serial/00-INDEX 2024-04-02T13:40:40.200 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/serial/README.cycladesZ 2024-04-02T13:40:40.202 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/serial/driver 2024-04-02T13:40:40.205 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/serial/moxa-smartio 2024-04-02T13:40:40.207 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/serial/n_gsm.txt 2024-04-02T13:40:40.209 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/serial/rocket.txt 2024-04-02T13:40:40.211 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/serial/serial-rs485.txt 2024-04-02T13:40:40.214 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/serial/tty.txt 2024-04-02T13:40:40.216 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sgi-ioc4.txt 2024-04-02T13:40:40.219 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sh/ 2024-04-02T13:40:40.220 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sh/new-machine.txt 2024-04-02T13:40:40.223 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sh/register-banks.txt 2024-04-02T13:40:40.225 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/smsc_ece1099.txt 2024-04-02T13:40:40.227 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/ 2024-04-02T13:40:40.229 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/ 2024-04-02T13:40:40.231 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/ALSA-Configuration.txt 2024-04-02T13:40:40.234 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/Audigy-mixer.txt 2024-04-02T13:40:40.237 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/Audiophile-Usb.txt 2024-04-02T13:40:40.240 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/Bt87x.txt 2024-04-02T13:40:40.242 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/CMIPCI.txt 2024-04-02T13:40:40.244 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/Channel-Mapping-API.txt 2024-04-02T13:40:40.247 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/ControlNames.txt 2024-04-02T13:40:40.249 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Controls.txt 2024-04-02T13:40:40.252 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Models.txt 2024-04-02T13:40:40.254 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio.txt 2024-04-02T13:40:40.257 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/Joystick.txt 2024-04-02T13:40:40.260 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/MIXART.txt 2024-04-02T13:40:40.262 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/OSS-Emulation.txt 2024-04-02T13:40:40.265 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/Procfile.txt 2024-04-02T13:40:40.267 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/README.maya44 2024-04-02T13:40:40.270 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/SB-Live-mixer.txt 2024-04-02T13:40:40.273 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/VIA82xx-mixer.txt 2024-04-02T13:40:40.275 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/alsa-parameters.txt 2024-04-02T13:40:40.277 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/compress_offload.txt 2024-04-02T13:40:40.280 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/emu10k1-jack.txt 2024-04-02T13:40:40.282 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/hda_codec.txt 2024-04-02T13:40:40.285 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/hdspm.txt 2024-04-02T13:40:40.287 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/powersave.txt 2024-04-02T13:40:40.291 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/seq_oss.html 2024-04-02T13:40:40.293 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/serial-u16550.txt 2024-04-02T13:40:40.296 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/soc/ 2024-04-02T13:40:40.298 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt 2024-04-02T13:40:40.300 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt 2024-04-02T13:40:40.303 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt 2024-04-02T13:40:40.305 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/soc/codec.txt 2024-04-02T13:40:40.307 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt 2024-04-02T13:40:40.310 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/soc/jack.txt 2024-04-02T13:40:40.314 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/soc/machine.txt 2024-04-02T13:40:40.315 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/soc/overview.txt 2024-04-02T13:40:40.318 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/soc/platform.txt 2024-04-02T13:40:40.320 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt 2024-04-02T13:40:40.322 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/ 2024-04-02T13:40:40.325 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/ALS 2024-04-02T13:40:40.327 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16 2024-04-02T13:40:40.329 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/CMI8330 2024-04-02T13:40:40.331 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/ESS 2024-04-02T13:40:40.334 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/ESS1868 2024-04-02T13:40:40.336 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/Introduction 2024-04-02T13:40:40.339 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/MultiSound 2024-04-02T13:40:40.342 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/OPL3 2024-04-02T13:40:40.344 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/Opti 2024-04-02T13:40:40.346 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/PAS16 2024-04-02T13:40:40.349 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/PSS 2024-04-02T13:40:40.351 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/PSS-updates 2024-04-02T13:40:40.354 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/README.OSS 2024-04-02T13:40:40.358 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/README.modules 2024-04-02T13:40:40.360 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/README.ymfsb 2024-04-02T13:40:40.362 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/SoundPro 2024-04-02T13:40:40.365 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/Soundblaster 2024-04-02T13:40:40.367 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/Tropez+ 2024-04-02T13:40:40.369 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/VIBRA16 2024-04-02T13:40:40.371 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/WaveArtist 2024-04-02T13:40:40.374 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/btaudio 2024-04-02T13:40:40.376 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/mwave 2024-04-02T13:40:40.379 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/oss-parameters.txt 2024-04-02T13:40:40.382 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sound/oss/ultrasound 2024-04-02T13:40:40.385 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sparse.txt 2024-04-02T13:40:40.388 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/spi/ 2024-04-02T13:40:40.390 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/spi/.gitignore 2024-04-02T13:40:40.392 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/spi/00-INDEX 2024-04-02T13:40:40.395 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/spi/Makefile 2024-04-02T13:40:40.398 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/spi/butterfly 2024-04-02T13:40:40.400 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/spi/ep93xx_spi 2024-04-02T13:40:40.403 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/spi/pxa2xx 2024-04-02T13:40:40.405 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/spi/spi-lm70llp 2024-04-02T13:40:40.407 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/spi/spi-sc18is602 2024-04-02T13:40:40.410 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/spi/spi-summary 2024-04-02T13:40:40.414 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/spi/spidev 2024-04-02T13:40:40.415 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/spi/spidev_fdx.c 2024-04-02T13:40:40.418 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/spi/spidev_test.c 2024-04-02T13:40:40.420 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/stable_api_nonsense.txt 2024-04-02T13:40:40.423 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/stable_kernel_rules.txt 2024-04-02T13:40:40.425 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/static-keys.txt 2024-04-02T13:40:40.427 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/svga.txt 2024-04-02T13:40:40.430 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sysctl/ 2024-04-02T13:40:40.431 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sysctl/00-INDEX 2024-04-02T13:40:40.434 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sysctl/README 2024-04-02T13:40:40.436 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sysctl/abi.txt 2024-04-02T13:40:40.438 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sysctl/fs.txt 2024-04-02T13:40:40.441 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sysctl/kernel.txt 2024-04-02T13:40:40.444 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sysctl/net.txt 2024-04-02T13:40:40.447 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sysctl/sunrpc.txt 2024-04-02T13:40:40.448 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sysctl/vm.txt 2024-04-02T13:40:40.452 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sysfs-rules.txt 2024-04-02T13:40:40.454 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/sysrq.txt 2024-04-02T13:40:40.457 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/target/ 2024-04-02T13:40:40.459 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.py 2024-04-02T13:40:40.461 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.txt 2024-04-02T13:40:40.464 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/target/tcmu-design.txt 2024-04-02T13:40:40.467 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/thermal/ 2024-04-02T13:40:40.469 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/thermal/cpu-cooling-api.txt 2024-04-02T13:40:40.471 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal 2024-04-02T13:40:40.474 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal_emulation 2024-04-02T13:40:40.475 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/thermal/intel_powerclamp.txt 2024-04-02T13:40:40.478 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/thermal/nouveau_thermal 2024-04-02T13:40:40.481 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/thermal/sysfs-api.txt 2024-04-02T13:40:40.483 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/thermal/x86_pkg_temperature_thermal 2024-04-02T13:40:40.485 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/this_cpu_ops.txt 2024-04-02T13:40:40.488 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/timers/ 2024-04-02T13:40:40.490 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/timers/.gitignore 2024-04-02T13:40:40.492 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/timers/00-INDEX 2024-04-02T13:40:40.494 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/timers/Makefile 2024-04-02T13:40:40.496 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/timers/NO_HZ.txt 2024-04-02T13:40:40.499 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/timers/highres.txt 2024-04-02T13:40:40.502 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/timers/hpet.txt 2024-04-02T13:40:40.504 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/timers/hpet_example.c 2024-04-02T13:40:40.506 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/timers/hrtimers.txt 2024-04-02T13:40:40.509 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/timers/timekeeping.txt 2024-04-02T13:40:40.512 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/timers/timer_stats.txt 2024-04-02T13:40:40.514 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/timers/timers-howto.txt 2024-04-02T13:40:40.517 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/tpm/ 2024-04-02T13:40:40.518 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/tpm/xen-tpmfront.txt 2024-04-02T13:40:40.520 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/ 2024-04-02T13:40:40.522 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/coresight.txt 2024-04-02T13:40:40.525 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/events-kmem.txt 2024-04-02T13:40:40.527 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/events-nmi.txt 2024-04-02T13:40:40.530 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/events-power.txt 2024-04-02T13:40:40.532 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/events.txt 2024-04-02T13:40:40.535 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/ftrace-design.txt 2024-04-02T13:40:40.537 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/ftrace.txt 2024-04-02T13:40:40.542 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/function-graph-fold.vim 2024-04-02T13:40:40.545 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/kprobetrace.txt 2024-04-02T13:40:40.547 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/mmiotrace.txt 2024-04-02T13:40:40.550 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/postprocess/ 2024-04-02T13:40:40.551 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-pagealloc-postprocess.pl 2024-04-02T13:40:40.554 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-vmscan-postprocess.pl 2024-04-02T13:40:40.557 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/ring-buffer-design.txt 2024-04-02T13:40:40.560 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/tracepoint-analysis.txt 2024-04-02T13:40:40.562 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/tracepoints.txt 2024-04-02T13:40:40.565 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/trace/uprobetracer.txt 2024-04-02T13:40:40.568 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/unaligned-memory-access.txt 2024-04-02T13:40:40.571 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/unicode.txt 2024-04-02T13:40:40.574 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/unshare.txt 2024-04-02T13:40:40.577 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/ 2024-04-02T13:40:40.579 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/CREDITS 2024-04-02T13:40:40.581 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/URB.txt 2024-04-02T13:40:40.584 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/WUSB-Design-overview.txt 2024-04-02T13:40:40.587 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/acm.txt 2024-04-02T13:40:40.589 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/anchors.txt 2024-04-02T13:40:40.591 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/authorization.txt 2024-04-02T13:40:40.594 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/bulk-streams.txt 2024-04-02T13:40:40.596 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/callbacks.txt 2024-04-02T13:40:40.598 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/chipidea.txt 2024-04-02T13:40:40.601 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/dma.txt 2024-04-02T13:40:40.603 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/dwc3.txt 2024-04-02T13:40:40.606 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/ehci.txt 2024-04-02T13:40:40.608 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/error-codes.txt 2024-04-02T13:40:40.610 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/functionfs.txt 2024-04-02T13:40:40.613 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/gadget-testing.txt 2024-04-02T13:40:40.617 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/gadget_configfs.txt 2024-04-02T13:40:40.619 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/gadget_hid.txt 2024-04-02T13:40:40.621 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/gadget_multi.txt 2024-04-02T13:40:40.623 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/gadget_printer.txt 2024-04-02T13:40:40.625 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/gadget_serial.txt 2024-04-02T13:40:40.628 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/hotplug.txt 2024-04-02T13:40:40.630 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/iuu_phoenix.txt 2024-04-02T13:40:40.632 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/linux-cdc-acm.inf 2024-04-02T13:40:40.635 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/linux.inf 2024-04-02T13:40:40.637 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/mass-storage.txt 2024-04-02T13:40:40.639 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/misc_usbsevseg.txt 2024-04-02T13:40:40.642 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/mtouchusb.txt 2024-04-02T13:40:40.644 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/ohci.txt 2024-04-02T13:40:40.646 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/persist.txt 2024-04-02T13:40:40.649 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/power-management.txt 2024-04-02T13:40:40.652 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/proc_usb_info.txt 2024-04-02T13:40:40.655 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/rio.txt 2024-04-02T13:40:40.657 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/usb-help.txt 2024-04-02T13:40:40.659 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/usb-serial.txt 2024-04-02T13:40:40.662 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/usbmon.txt 2024-04-02T13:40:40.664 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/usb/wusb-cbaf 2024-04-02T13:40:40.666 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vDSO/ 2024-04-02T13:40:40.669 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vDSO/.gitignore 2024-04-02T13:40:40.670 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vDSO/Makefile 2024-04-02T13:40:40.685 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vDSO/parse_vdso.c 2024-04-02T13:40:40.711 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vDSO/vdso_standalone_test_x86.c 2024-04-02T13:40:40.715 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vDSO/vdso_test.c 2024-04-02T13:40:40.720 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vfio.txt 2024-04-02T13:40:40.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:40 smithi145 ceph-mon[35646]: pgmap v59: 129 pgs: 129 active+clean; 97 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.5 MiB/s wr, 223 op/s 2024-04-02T13:40:40.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:40 smithi050 ceph-mon[35524]: pgmap v59: 129 pgs: 129 active+clean; 97 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.5 MiB/s wr, 223 op/s 2024-04-02T13:40:41.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:40 smithi033 ceph-mon[32390]: pgmap v59: 129 pgs: 129 active+clean; 97 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.5 MiB/s wr, 223 op/s 2024-04-02T13:40:41.387 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vgaarbiter.txt 2024-04-02T13:40:41.390 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video-output.txt 2024-04-02T13:40:41.392 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/ 2024-04-02T13:40:41.395 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/4CCs.txt 2024-04-02T13:40:41.398 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/API.html 2024-04-02T13:40:41.399 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.au0828 2024-04-02T13:40:41.402 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.bttv 2024-04-02T13:40:41.405 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx23885 2024-04-02T13:40:41.407 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx88 2024-04-02T13:40:41.411 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.em28xx 2024-04-02T13:40:41.414 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.ivtv 2024-04-02T13:40:41.416 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7134 2024-04-02T13:40:41.419 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7164 2024-04-02T13:40:41.422 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tm6000 2024-04-02T13:40:41.424 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tuner 2024-04-02T13:40:41.427 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.usbvision 2024-04-02T13:40:41.429 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/Makefile 2024-04-02T13:40:41.432 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/README.cpia2 2024-04-02T13:40:41.434 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/README.cx88 2024-04-02T13:40:41.437 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/README.davinci-vpbe 2024-04-02T13:40:41.440 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/README.ir 2024-04-02T13:40:41.442 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/README.ivtv 2024-04-02T13:40:41.445 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/README.pvrusb2 2024-04-02T13:40:41.448 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/README.saa7134 2024-04-02T13:40:41.450 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/Zoran 2024-04-02T13:40:41.453 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/ 2024-04-02T13:40:41.455 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/CONTRIBUTORS 2024-04-02T13:40:41.458 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/Cards 2024-04-02T13:40:41.462 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/ICs 2024-04-02T13:40:41.464 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/Insmod-options 2024-04-02T13:40:41.467 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/MAKEDEV 2024-04-02T13:40:41.470 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modprobe.conf 2024-04-02T13:40:41.473 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modules.conf 2024-04-02T13:40:41.475 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/PROBLEMS 2024-04-02T13:40:41.477 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/README 2024-04-02T13:40:41.480 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.WINVIEW 2024-04-02T13:40:41.482 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.freeze 2024-04-02T13:40:41.485 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.quirks 2024-04-02T13:40:41.487 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/Sound-FAQ 2024-04-02T13:40:41.490 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/Specs 2024-04-02T13:40:41.493 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/THANKS 2024-04-02T13:40:41.496 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/bttv/Tuners 2024-04-02T13:40:41.500 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cafe_ccic 2024-04-02T13:40:41.503 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cpia2_overview.txt 2024-04-02T13:40:41.506 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx18.txt 2024-04-02T13:40:41.509 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/ 2024-04-02T13:40:41.511 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.hm12 2024-04-02T13:40:41.515 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.vbi 2024-04-02T13:40:41.517 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-calling.txt 2024-04-02T13:40:41.521 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-api.txt 2024-04-02T13:40:41.523 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-regs.txt 2024-04-02T13:40:41.527 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-dma.txt 2024-04-02T13:40:41.529 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-encoder-api.txt 2024-04-02T13:40:41.532 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-memory.txt 2024-04-02T13:40:41.534 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-osd-api.txt 2024-04-02T13:40:41.537 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-upload.txt 2024-04-02T13:40:41.541 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx88/ 2024-04-02T13:40:41.543 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/cx88/hauppauge-wintv-cx88-ir.txt 2024-04-02T13:40:41.549 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/extract_xc3028.pl 2024-04-02T13:40:41.556 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/fimc.txt 2024-04-02T13:40:41.562 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/gspca.txt 2024-04-02T13:40:41.566 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/hauppauge-wintv-cx88-ir.txt 2024-04-02T13:40:41.569 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/lifeview.txt 2024-04-02T13:40:41.571 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/meye.txt 2024-04-02T13:40:41.574 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/not-in-cx2388x-datasheet.txt 2024-04-02T13:40:41.577 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/omap3isp.txt 2024-04-02T13:40:41.580 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/omap4_camera.txt 2024-04-02T13:40:41.583 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/pxa_camera.txt 2024-04-02T13:40:41.585 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/radiotrack.txt 2024-04-02T13:40:41.588 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/sh_mobile_ceu_camera.txt 2024-04-02T13:40:41.591 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/si470x.txt 2024-04-02T13:40:41.594 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/si4713.txt 2024-04-02T13:40:41.597 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/si476x.txt 2024-04-02T13:40:41.600 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/soc-camera.txt 2024-04-02T13:40:41.602 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/uvcvideo.txt 2024-04-02T13:40:41.605 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/v4l2-controls.txt 2024-04-02T13:40:41.609 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/v4l2-framework.txt 2024-04-02T13:40:41.615 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/v4l2-pci-skeleton.c 2024-04-02T13:40:41.619 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/videobuf 2024-04-02T13:40:41.623 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/vivid.txt 2024-04-02T13:40:41.631 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/video4linux/zr364xx.txt 2024-04-02T13:40:41.634 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/ 2024-04-02T13:40:41.636 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/00-INDEX 2024-04-02T13:40:41.638 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/ 2024-04-02T13:40:41.639 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/00-INDEX 2024-04-02T13:40:41.642 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/api.txt 2024-04-02T13:40:41.646 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/cpuid.txt 2024-04-02T13:40:41.649 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/ 2024-04-02T13:40:41.651 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/README 2024-04-02T13:40:41.653 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt 2024-04-02T13:40:41.656 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/mpic.txt 2024-04-02T13:40:41.659 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/s390_flic.txt 2024-04-02T13:40:41.661 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vfio.txt 2024-04-02T13:40:41.663 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vm.txt 2024-04-02T13:40:41.666 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/xics.txt 2024-04-02T13:40:41.668 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/hypercalls.txt 2024-04-02T13:40:41.671 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/locking.txt 2024-04-02T13:40:41.673 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/mmu.txt 2024-04-02T13:40:41.676 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/msr.txt 2024-04-02T13:40:41.679 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/nested-vmx.txt 2024-04-02T13:40:41.681 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/ppc-pv.txt 2024-04-02T13:40:41.684 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/review-checklist.txt 2024-04-02T13:40:41.687 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/s390-diag.txt 2024-04-02T13:40:41.689 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/kvm/timekeeping.txt 2024-04-02T13:40:41.691 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/paravirt_ops.txt 2024-04-02T13:40:41.694 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/uml/ 2024-04-02T13:40:41.695 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt 2024-04-02T13:40:41.700 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/ 2024-04-02T13:40:41.702 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/.gitignore 2024-04-02T13:40:41.704 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/00-INDEX 2024-04-02T13:40:41.707 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/active_mm.txt 2024-04-02T13:40:41.709 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/balance 2024-04-02T13:40:41.711 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/cleancache.txt 2024-04-02T13:40:41.714 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/frontswap.txt 2024-04-02T13:40:41.716 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/highmem.txt 2024-04-02T13:40:41.719 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/hugetlbpage.txt 2024-04-02T13:40:41.722 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/hwpoison.txt 2024-04-02T13:40:41.723 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/ksm.txt 2024-04-02T13:40:41.727 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/numa 2024-04-02T13:40:41.729 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/numa_memory_policy.txt 2024-04-02T13:40:41.731 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/overcommit-accounting 2024-04-02T13:40:41.733 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/page_migration 2024-04-02T13:40:41.736 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/page_owner.txt 2024-04-02T13:40:41.738 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/pagemap.txt 2024-04-02T13:40:41.741 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/remap_file_pages.txt 2024-04-02T13:40:41.743 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/slub.txt 2024-04-02T13:40:41.745 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/soft-dirty.txt 2024-04-02T13:40:41.749 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/split_page_table_lock 2024-04-02T13:40:41.750 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/transhuge.txt 2024-04-02T13:40:41.752 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/unevictable-lru.txt 2024-04-02T13:40:41.755 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vm/zswap.txt 2024-04-02T13:40:41.758 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/vme_api.txt 2024-04-02T13:40:41.760 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/volatile-considered-harmful.txt 2024-04-02T13:40:41.762 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/ 2024-04-02T13:40:41.764 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/00-INDEX 2024-04-02T13:40:41.766 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/masters/ 2024-04-02T13:40:41.767 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/masters/00-INDEX 2024-04-02T13:40:41.769 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/masters/ds2482 2024-04-02T13:40:41.772 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/masters/ds2490 2024-04-02T13:40:41.774 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/masters/mxc-w1 2024-04-02T13:40:41.776 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/masters/omap-hdq 2024-04-02T13:40:41.778 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/masters/w1-gpio 2024-04-02T13:40:41.781 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/slaves/ 2024-04-02T13:40:41.783 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/slaves/00-INDEX 2024-04-02T13:40:41.785 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2406 2024-04-02T13:40:41.787 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2423 2024-04-02T13:40:41.789 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds28e04 2024-04-02T13:40:41.792 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/slaves/w1_therm 2024-04-02T13:40:41.794 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/w1.generic 2024-04-02T13:40:41.796 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/w1/w1.netlink 2024-04-02T13:40:41.799 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/ 2024-04-02T13:40:41.801 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/Makefile 2024-04-02T13:40:41.803 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/convert_drivers_to_kernel_api.txt 2024-04-02T13:40:41.805 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/hpwdt.txt 2024-04-02T13:40:41.808 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/pcwd-watchdog.txt 2024-04-02T13:40:41.810 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/src/ 2024-04-02T13:40:41.813 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/src/.gitignore 2024-04-02T13:40:41.815 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/src/Makefile 2024-04-02T13:40:41.818 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-simple.c 2024-04-02T13:40:41.821 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-test.c 2024-04-02T13:40:41.823 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/watchdog-api.txt 2024-04-02T13:40:41.826 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/watchdog-kernel-api.txt 2024-04-02T13:40:41.829 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/watchdog-parameters.txt 2024-04-02T13:40:41.832 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/watchdog/wdt.txt 2024-04-02T13:40:41.834 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/wimax/ 2024-04-02T13:40:41.837 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/wimax/README.i2400m 2024-04-02T13:40:41.840 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/wimax/README.wimax 2024-04-02T13:40:41.843 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/workqueue.txt 2024-04-02T13:40:41.846 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/ 2024-04-02T13:40:41.848 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/00-INDEX 2024-04-02T13:40:41.854 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/boot.txt 2024-04-02T13:40:41.854 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/early-microcode.txt 2024-04-02T13:40:41.857 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/earlyprintk.txt 2024-04-02T13:40:41.860 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/entry_64.txt 2024-04-02T13:40:41.862 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/exception-tables.txt 2024-04-02T13:40:41.865 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/i386/ 2024-04-02T13:40:41.867 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/i386/IO-APIC.txt 2024-04-02T13:40:41.891 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/intel_mpx.txt 2024-04-02T13:40:42.011 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/mtrr.txt 2024-04-02T13:40:42.014 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/pat.txt 2024-04-02T13:40:42.016 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/tlb.txt 2024-04-02T13:40:42.019 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/usb-legacy-support.txt 2024-04-02T13:40:42.021 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/x86_64/ 2024-04-02T13:40:42.023 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/x86_64/00-INDEX 2024-04-02T13:40:42.025 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/x86_64/boot-options.txt 2024-04-02T13:40:42.028 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/x86_64/cpu-hotplug-spec 2024-04-02T13:40:42.030 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/x86_64/fake-numa-for-cpusets 2024-04-02T13:40:42.033 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/x86_64/kernel-stacks 2024-04-02T13:40:42.035 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/x86_64/machinecheck 2024-04-02T13:40:42.038 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/x86_64/mm.txt 2024-04-02T13:40:42.040 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/x86_64/uefi.txt 2024-04-02T13:40:42.042 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/x86/zero-page.txt 2024-04-02T13:40:42.044 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/xillybus.txt 2024-04-02T13:40:42.048 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/xtensa/ 2024-04-02T13:40:42.049 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/xtensa/atomctl.txt 2024-04-02T13:40:42.051 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/xtensa/mmu.txt 2024-04-02T13:40:42.053 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/xz.txt 2024-04-02T13:40:42.057 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/ 2024-04-02T13:40:42.058 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/CodingStyle 2024-04-02T13:40:42.061 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/HOWTO 2024-04-02T13:40:42.064 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/IRQ.txt 2024-04-02T13:40:42.066 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/SecurityBugs 2024-04-02T13:40:42.069 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingDrivers 2024-04-02T13:40:42.071 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingPatches 2024-04-02T13:40:42.074 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/arm/ 2024-04-02T13:40:42.076 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/arm/Booting 2024-04-02T13:40:42.079 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/arm/kernel_user_helpers.txt 2024-04-02T13:40:42.082 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/arm64/ 2024-04-02T13:40:42.084 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/arm64/booting.txt 2024-04-02T13:40:42.086 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/arm64/memory.txt 2024-04-02T13:40:42.088 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/arm64/tagged-pointers.txt 2024-04-02T13:40:42.091 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/basic_profiling.txt 2024-04-02T13:40:42.093 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/email-clients.txt 2024-04-02T13:40:42.095 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/ 2024-04-02T13:40:42.098 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/sysfs.txt 2024-04-02T13:40:42.101 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/gpio.txt 2024-04-02T13:40:42.104 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/io_ordering.txt 2024-04-02T13:40:42.106 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/magic-number.txt 2024-04-02T13:40:42.109 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/oops-tracing.txt 2024-04-02T13:40:42.112 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/sparse.txt 2024-04-02T13:40:42.114 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/stable_api_nonsense.txt 2024-04-02T13:40:42.116 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/stable_kernel_rules.txt 2024-04-02T13:40:42.118 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/ 2024-04-02T13:40:42.120 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/omap3isp.txt 2024-04-02T13:40:42.122 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/v4l2-framework.txt 2024-04-02T13:40:42.126 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zh_CN/volatile-considered-harmful.txt 2024-04-02T13:40:42.129 INFO:tasks.workunit.client.0.smithi033.stdout:linux-4.0.5/Documentation/zorro.txt 2024-04-02T13:40:42.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:42 smithi145 ceph-mon[35646]: pgmap v60: 129 pgs: 129 active+clean; 97 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.2 MiB/s wr, 196 op/s 2024-04-02T13:40:42.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:42 smithi050 ceph-mon[35524]: pgmap v60: 129 pgs: 129 active+clean; 97 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.2 MiB/s wr, 196 op/s 2024-04-02T13:40:43.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:42 smithi033 ceph-mon[32390]: pgmap v60: 129 pgs: 129 active+clean; 97 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.2 MiB/s wr, 196 op/s 2024-04-02T13:40:44.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:44 smithi145 ceph-mon[35646]: pgmap v61: 129 pgs: 129 active+clean; 112 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 2.5 MiB/s wr, 301 op/s 2024-04-02T13:40:44.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:44 smithi050 ceph-mon[35524]: pgmap v61: 129 pgs: 129 active+clean; 112 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 2.5 MiB/s wr, 301 op/s 2024-04-02T13:40:45.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:44 smithi033 ceph-mon[32390]: pgmap v61: 129 pgs: 129 active+clean; 112 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 2.5 MiB/s wr, 301 op/s 2024-04-02T13:40:46.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:46 smithi145 ceph-mon[35646]: pgmap v62: 129 pgs: 129 active+clean; 112 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-04-02T13:40:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:46 smithi050 ceph-mon[35524]: pgmap v62: 129 pgs: 129 active+clean; 112 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-04-02T13:40:47.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:46 smithi033 ceph-mon[32390]: pgmap v62: 129 pgs: 129 active+clean; 112 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-04-02T13:40:47.765 INFO:tasks.workunit.client.0.smithi033.stderr:+ tar cf doc.tar linux-4.0.5 2024-04-02T13:40:48.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:48 smithi145 ceph-mon[35646]: pgmap v63: 129 pgs: 129 active+clean; 112 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-04-02T13:40:48.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:48 smithi050 ceph-mon[35524]: pgmap v63: 129 pgs: 129 active+clean; 112 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-04-02T13:40:49.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:48 smithi033 ceph-mon[32390]: pgmap v63: 129 pgs: 129 active+clean; 112 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-04-02T13:40:50.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:50 smithi145 ceph-mon[35646]: pgmap v64: 129 pgs: 129 active+clean; 115 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 262 op/s 2024-04-02T13:40:50.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:50 smithi050 ceph-mon[35524]: pgmap v64: 129 pgs: 129 active+clean; 115 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 262 op/s 2024-04-02T13:40:51.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:50 smithi033 ceph-mon[32390]: pgmap v64: 129 pgs: 129 active+clean; 115 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 262 op/s 2024-04-02T13:40:51.534 INFO:tasks.workunit.client.0.smithi033.stderr:+ tar xf doc.tar 2024-04-02T13:40:52.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:52 smithi145 ceph-mon[35646]: pgmap v65: 129 pgs: 129 active+clean; 115 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-04-02T13:40:52.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:52 smithi050 ceph-mon[35524]: pgmap v65: 129 pgs: 129 active+clean; 115 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-04-02T13:40:53.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:52 smithi033 ceph-mon[32390]: pgmap v65: 129 pgs: 129 active+clean; 115 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-04-02T13:40:53.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:53 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:40:53.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:53 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:40:54.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:53 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:40:54.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:54 smithi145 ceph-mon[35646]: pgmap v66: 129 pgs: 129 active+clean; 142 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 149 op/s 2024-04-02T13:40:54.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:54 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:40:54.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:54 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:40:54.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:54 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:40:54.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:54 smithi050 ceph-mon[35524]: pgmap v66: 129 pgs: 129 active+clean; 142 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 149 op/s 2024-04-02T13:40:54.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:54 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:40:54.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:54 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:40:54.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:54 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:40:55.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:54 smithi033 ceph-mon[32390]: pgmap v66: 129 pgs: 129 active+clean; 142 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 149 op/s 2024-04-02T13:40:55.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:54 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:40:55.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:54 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:40:55.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:54 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:40:56.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:56 smithi145 ceph-mon[35646]: pgmap v67: 129 pgs: 129 active+clean; 142 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 2.6 MiB/s wr, 45 op/s 2024-04-02T13:40:56.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:56 smithi050 ceph-mon[35524]: pgmap v67: 129 pgs: 129 active+clean; 142 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 2.6 MiB/s wr, 45 op/s 2024-04-02T13:40:57.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:56 smithi033 ceph-mon[32390]: pgmap v67: 129 pgs: 129 active+clean; 142 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 2.6 MiB/s wr, 45 op/s 2024-04-02T13:40:57.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:57 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:40:57.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:57 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:40:58.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:57 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:40:58.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:40:58 smithi145 ceph-mon[35646]: pgmap v68: 129 pgs: 129 active+clean; 142 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 2.6 MiB/s wr, 45 op/s 2024-04-02T13:40:58.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:40:58 smithi050 ceph-mon[35524]: pgmap v68: 129 pgs: 129 active+clean; 142 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 2.6 MiB/s wr, 45 op/s 2024-04-02T13:40:59.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:40:58 smithi033 ceph-mon[32390]: pgmap v68: 129 pgs: 129 active+clean; 142 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 2.6 MiB/s wr, 45 op/s 2024-04-02T13:41:00.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:00 smithi145 ceph-mon[35646]: pgmap v69: 129 pgs: 129 active+clean; 147 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 152 op/s 2024-04-02T13:41:00.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:00 smithi050 ceph-mon[35524]: pgmap v69: 129 pgs: 129 active+clean; 147 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 152 op/s 2024-04-02T13:41:01.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:00 smithi033 ceph-mon[32390]: pgmap v69: 129 pgs: 129 active+clean; 147 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 152 op/s 2024-04-02T13:41:02.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:41:02.845 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:41:02.873 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:02 smithi033 ceph-mon[32390]: pgmap v70: 129 pgs: 129 active+clean; 147 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 2.8 MiB/s wr, 139 op/s 2024-04-02T13:41:02.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.963+0000 7f750e80d640 1 -- 172.21.15.33:0/2869115907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f750806a9f0 msgr2=0x7f750810e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:02.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.963+0000 7f750e80d640 1 --2- 172.21.15.33:0/2869115907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f750806a9f0 0x7f750810e980 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7500009cd0 tx=0x7f750002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.964+0000 7f750e80d640 1 -- 172.21.15.33:0/2869115907 shutdown_connections 2024-04-02T13:41:02.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.964+0000 7f750e80d640 1 --2- 172.21.15.33:0/2869115907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f750806a9f0 0x7f750810e980 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.964+0000 7f750e80d640 1 --2- 172.21.15.33:0/2869115907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f750806a070 0x7f750806a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.964+0000 7f750e80d640 1 --2- 172.21.15.33:0/2869115907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7508107ac0 0x7f7508107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.964+0000 7f750e80d640 1 -- 172.21.15.33:0/2869115907 >> 172.21.15.33:0/2869115907 conn(0x7f75080779e0 msgr2=0x7f7508077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:02.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.964+0000 7f750e80d640 1 -- 172.21.15.33:0/2869115907 shutdown_connections 2024-04-02T13:41:02.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.964+0000 7f750e80d640 1 -- 172.21.15.33:0/2869115907 wait complete. 2024-04-02T13:41:02.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.964+0000 7f750e80d640 1 Processor -- start 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.964+0000 7f750e80d640 1 -- start start 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f750e80d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f750806a070 0x7f7508198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f750e80d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f750806a9f0 0x7f75081988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f750e80d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7508107ac0 0x7f750819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f750e80d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7508111e90 con 0x7f750806a070 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f750e80d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7508111d10 con 0x7f750806a9f0 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f750e80d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7508112010 con 0x7f7508107ac0 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f7507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f750806a070 0x7f7508198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f75077fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f750806a9f0 0x7f75081988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f7507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f750806a070 0x7f7508198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36178/0 (socket says 172.21.15.33:36178) 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f7507fff640 1 -- 172.21.15.33:0/2125042792 learned_addr learned my addr 172.21.15.33:0/2125042792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f750cdaa640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7508107ac0 0x7f750819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f75077fe640 1 -- 172.21.15.33:0/2125042792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7508107ac0 msgr2=0x7f750819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f75077fe640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7508107ac0 0x7f750819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f75077fe640 1 -- 172.21.15.33:0/2125042792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f750806a070 msgr2=0x7f7508198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:02.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f75077fe640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f750806a070 0x7f7508198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f75077fe640 1 -- 172.21.15.33:0/2125042792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750819d380 con 0x7f750806a9f0 2024-04-02T13:41:02.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.965+0000 7f7507fff640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f750806a070 0x7f7508198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.966+0000 7f75077fe640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f750806a9f0 0x7f75081988d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009ce0 tx=0x7f74fc00b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.966+0000 7f75057fa640 1 -- 172.21.15.33:0/2125042792 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc004020 con 0x7f750806a9f0 2024-04-02T13:41:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.966+0000 7f750e80d640 1 -- 172.21.15.33:0/2125042792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750819d670 con 0x7f750806a9f0 2024-04-02T13:41:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.966+0000 7f750e80d640 1 -- 172.21.15.33:0/2125042792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7508100ce0 con 0x7f750806a9f0 2024-04-02T13:41:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.966+0000 7f75057fa640 1 -- 172.21.15.33:0/2125042792 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f74fc0041c0 con 0x7f750806a9f0 2024-04-02T13:41:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.966+0000 7f75057fa640 1 -- 172.21.15.33:0/2125042792 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc027060 con 0x7f750806a9f0 2024-04-02T13:41:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.969+0000 7f75057fa640 1 -- 172.21.15.33:0/2125042792 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f74fc0035f0 con 0x7f750806a9f0 2024-04-02T13:41:02.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.969+0000 7f75113f7740 1 Processor -- start 2024-04-02T13:41:02.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.969+0000 7f75113f7740 1 -- start start 2024-04-02T13:41:02.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75057fa640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f74f00a3d20 0x7f74f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:02.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75113f7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675d7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:02.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75113f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 0x55df675e0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:02.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75113f7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675e0dd0 0x55df675e31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75113f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55df6753b830 con 0x55df675d81f0 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75113f7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55df674e3820 con 0x55df675d78d0 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75113f7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55df675377e0 con 0x55df675e0dd0 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 0x55df675e0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f750cdaa640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f74f00a3d20 0x7f74f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f7507fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675d7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 0x55df675e0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36194/0 (socket says 172.21.15.33:36194) 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75077fe640 1 -- 172.21.15.33:0/1644947939 learned_addr learned my addr 172.21.15.33:0/1644947939 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75057fa640 1 -- 172.21.15.33:0/2125042792 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f74fc0d4260 con 0x7f750806a9f0 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f750cdaa640 1 --2- 172.21.15.33:0/1644947939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675e0dd0 0x55df675e31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75077fe640 1 -- 172.21.15.33:0/1644947939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675e0dd0 msgr2=0x55df675e31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75077fe640 1 --2- 172.21.15.33:0/1644947939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675e0dd0 0x55df675e31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75077fe640 1 -- 172.21.15.33:0/1644947939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 msgr2=0x55df675d7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75077fe640 1 --2- 172.21.15.33:0/1644947939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675d7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75077fe640 1 -- 172.21.15.33:0/1644947939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df673197d0 con 0x55df675d81f0 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f7507fff640 1 --2- 172.21.15.33:0/1644947939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675d7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f75077fe640 1 --2- 172.21.15.33:0/1644947939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 0x55df675e0890 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00bb90 tx=0x7f74fc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4beefa083d07b2cc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.970+0000 7f750cdaa640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f74f00a3d20 0x7f74f00a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7500005cc0 tx=0x7f75000323c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.971+0000 7f74ee7fc640 1 -- 172.21.15.33:0/1644947939 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc09ff40 con 0x55df675d81f0 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.971+0000 7f74ee7fc640 1 -- 172.21.15.33:0/1644947939 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f74fc0cde50 con 0x55df675d81f0 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.971+0000 7f74ee7fc640 1 -- 172.21.15.33:0/1644947939 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0038e0 con 0x55df675d81f0 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.971+0000 7f75113f7740 1 -- 172.21.15.33:0/1644947939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 msgr2=0x55df675e0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.971+0000 7f75113f7740 1 --2- 172.21.15.33:0/1644947939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 0x55df675e0890 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00bb90 tx=0x7f74fc009f60 comp rx=0 tx=0).stop 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.971+0000 7f75113f7740 1 -- 172.21.15.33:0/1644947939 shutdown_connections 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.971+0000 7f75113f7740 1 --2- 172.21.15.33:0/1644947939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675e0dd0 0x55df675e31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.971+0000 7f75113f7740 1 --2- 172.21.15.33:0/1644947939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 0x55df675e0890 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.971+0000 7f75113f7740 1 --2- 172.21.15.33:0/1644947939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675d7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.971+0000 7f75113f7740 1 -- 172.21.15.33:0/1644947939 >> 172.21.15.33:0/1644947939 conn(0x55df675d2000 msgr2=0x55df675d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.971+0000 7f75113f7740 1 -- 172.21.15.33:0/1644947939 shutdown_connections 2024-04-02T13:41:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.971+0000 7f75113f7740 1 -- 172.21.15.33:0/1644947939 wait complete. 2024-04-02T13:41:02.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.972+0000 7f75113f7740 1 -- start start 2024-04-02T13:41:02.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.972+0000 7f75113f7740 1 Processor -- start 2024-04-02T13:41:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.972+0000 7f75113f7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675e05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.972+0000 7f75113f7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675d81f0 0x55df675dd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.972+0000 7f75113f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675e0dd0 0x55df675ddea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.972+0000 7f75113f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55df675e3bc0 con 0x55df675e0dd0 2024-04-02T13:41:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.972+0000 7f75113f7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55df674f6d30 con 0x55df675d78d0 2024-04-02T13:41:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.972+0000 7f75113f7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55df675e3d40 con 0x55df675d81f0 2024-04-02T13:41:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f75113f7740 10 client.0 _ll_register_callbacks cb 0x55df674f6720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f750cdaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675e0dd0 0x55df675ddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f7507fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675e05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f750cdaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675e0dd0 0x55df675ddea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36200/0 (socket says 172.21.15.33:36200) 2024-04-02T13:41:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f7507fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675e05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:53126/0 (socket says 172.21.15.33:53126) 2024-04-02T13:41:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f750cdaa640 1 -- 172.21.15.33:0/1999178911 learned_addr learned my addr 172.21.15.33:0/1999178911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f75077fe640 1 --2- 172.21.15.33:0/1999178911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675d81f0 0x55df675dd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f750cdaa640 1 -- 172.21.15.33:0/1999178911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675d81f0 msgr2=0x55df675dd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f750cdaa640 1 --2- 172.21.15.33:0/1999178911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675d81f0 0x55df675dd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f750cdaa640 1 -- 172.21.15.33:0/1999178911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 msgr2=0x55df675e05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f750cdaa640 1 --2- 172.21.15.33:0/1999178911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675e05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f750cdaa640 1 -- 172.21.15.33:0/1999178911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6724f3f0 con 0x55df675e0dd0 2024-04-02T13:41:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f750cdaa640 1 --2- 172.21.15.33:0/1999178911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675e0dd0 0x55df675ddea0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7500004af0 tx=0x7f7500005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b9c514458d7cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f74e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f74e77fe640 1 -- 172.21.15.33:0/1999178911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500053060 con 0x55df675e0dd0 2024-04-02T13:41:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f74e77fe640 1 -- 172.21.15.33:0/1999178911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7500004790 con 0x55df675e0dd0 2024-04-02T13:41:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f75113f7740 1 -- 172.21.15.33:0/1999178911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74fc0cde50 con 0x55df675e0dd0 2024-04-02T13:41:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.973+0000 7f74e77fe640 1 -- 172.21.15.33:0/1999178911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7500004b40 con 0x55df675e0dd0 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.974+0000 7f75113f7740 10 client.16086 fetch_fsmap learned FSMap version 27 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.974+0000 7f75113f7740 1 -- 172.21.15.33:0/1999178911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df6720f0f0 con 0x55df675e0dd0 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.974+0000 7f74e77fe640 1 -- 172.21.15.33:0/1999178911 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f74fc0cde50 con 0x55df675e0dd0 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.974+0000 7f74e77fe640 1 -- 172.21.15.33:0/1999178911 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f750003f040 con 0x55df675e0dd0 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.974+0000 7f75113f7740 10 client.16086 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.974+0000 7f75113f7740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.974+0000 7f75113f7740 20 client.16086 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.974+0000 7f75113f7740 1 --2- 172.21.15.33:0/1999178911 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55df675f0da0 0x55df675f3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.974+0000 7f75113f7740 4 client.16086 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.974+0000 7f75113f7740 1 -- 172.21.15.33:0/1999178911 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df67548d70 con 0x55df675f0da0 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.974+0000 7f7507fff640 1 --2- 172.21.15.33:0/1999178911 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55df675f0da0 0x55df675f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.975+0000 7f74e77fe640 10 client.16086 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.975+0000 7f7507fff640 1 --2- 172.21.15.33:0/1999178911 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55df675f0da0 0x55df675f3220 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967f657eb1c5b9a1 server_cookie=8012a7a9113240c4 in_seq=0 out_seq=0 2024-04-02T13:41:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f74e77fe640 1 -- 172.21.15.33:0/1999178911 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df67548d70 con 0x55df675f0da0 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f74e77fe640 10 client.16086 handle_command_reply: tid=0 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f75113f7740 1 client.16086 shutdown 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f75113f7740 1 -- 172.21.15.33:0/1999178911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675e0dd0 msgr2=0x55df675ddea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f75113f7740 1 --2- 172.21.15.33:0/1999178911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675e0dd0 0x55df675ddea0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7500004af0 tx=0x7f7500005c50 comp rx=0 tx=0).stop 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f75113f7740 1 -- 172.21.15.33:0/1999178911 shutdown_connections 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f75113f7740 1 --2- 172.21.15.33:0/1999178911 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55df675f0da0 0x55df675f3220 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f75113f7740 1 --2- 172.21.15.33:0/1999178911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675e0dd0 0x55df675ddea0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f75113f7740 1 --2- 172.21.15.33:0/1999178911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675d81f0 0x55df675dd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f75113f7740 1 --2- 172.21.15.33:0/1999178911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675e05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f74e77fe640 0 client.16086 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f75113f7740 1 -- 172.21.15.33:0/1999178911 >> 172.21.15.33:0/1999178911 conn(0x55df675d2000 msgr2=0x55df675d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f75113f7740 1 -- 172.21.15.33:0/1999178911 shutdown_connections 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f75113f7740 1 -- 172.21.15.33:0/1999178911 wait complete. 2024-04-02T13:41:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.976+0000 7f75113f7740 20 client.16086 trim_cache size 0 max 16384 2024-04-02T13:41:02.985 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:02 smithi145 ceph-mon[35646]: pgmap v70: 129 pgs: 129 active+clean; 147 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 2.8 MiB/s wr, 139 op/s 2024-04-02T13:41:02.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:02 smithi050 ceph-mon[35524]: pgmap v70: 129 pgs: 129 active+clean; 147 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 2.8 MiB/s wr, 139 op/s 2024-04-02T13:41:03.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.998+0000 7f75113f7740 1 Processor -- start 2024-04-02T13:41:03.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.998+0000 7f75113f7740 1 -- start start 2024-04-02T13:41:03.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75113f7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75113f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 0x55df675d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75113f7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675e0dd0 0x55df675e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75113f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74f4012780 con 0x55df675d81f0 2024-04-02T13:41:03.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75113f7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74f4010020 con 0x55df675d78d0 2024-04-02T13:41:03.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75113f7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f74f400de50 con 0x55df675e0dd0 2024-04-02T13:41:03.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 0x55df675d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 0x55df675d3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36212/0 (socket says 172.21.15.33:36212) 2024-04-02T13:41:03.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75077fe640 1 -- 172.21.15.33:0/3946196255 learned_addr learned my addr 172.21.15.33:0/3946196255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f7507fff640 1 --2- 172.21.15.33:0/3946196255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f750cdaa640 1 --2- 172.21.15.33:0/3946196255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675e0dd0 0x55df675e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75077fe640 1 -- 172.21.15.33:0/3946196255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675e0dd0 msgr2=0x55df675e0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75077fe640 1 --2- 172.21.15.33:0/3946196255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675e0dd0 0x55df675e0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75077fe640 1 -- 172.21.15.33:0/3946196255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 msgr2=0x55df675ecae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75077fe640 1 --2- 172.21.15.33:0/3946196255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675ecae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75077fe640 1 -- 172.21.15.33:0/3946196255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df675d8020 con 0x55df675d81f0 2024-04-02T13:41:03.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f75077fe640 1 --2- 172.21.15.33:0/3946196255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 0x55df675d3b40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009fd0 tx=0x7f74fc005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe55493929236e24 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:02.999+0000 7f74edffb640 1 -- 172.21.15.33:0/3946196255 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0a4240 con 0x55df675d81f0 2024-04-02T13:41:03.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.000+0000 7f74edffb640 1 -- 172.21.15.33:0/3946196255 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f74fc0cde50 con 0x55df675d81f0 2024-04-02T13:41:03.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.000+0000 7f74edffb640 1 -- 172.21.15.33:0/3946196255 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0d18c0 con 0x55df675d81f0 2024-04-02T13:41:03.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.000+0000 7f75113f7740 1 -- 172.21.15.33:0/3946196255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 msgr2=0x55df675d3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.000+0000 7f75113f7740 1 --2- 172.21.15.33:0/3946196255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 0x55df675d3b40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009fd0 tx=0x7f74fc005bb0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.000+0000 7f75113f7740 1 -- 172.21.15.33:0/3946196255 shutdown_connections 2024-04-02T13:41:03.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.000+0000 7f75113f7740 1 --2- 172.21.15.33:0/3946196255 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675e0dd0 0x55df675e0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.000+0000 7f75113f7740 1 --2- 172.21.15.33:0/3946196255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d81f0 0x55df675d3b40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.000+0000 7f75113f7740 1 --2- 172.21.15.33:0/3946196255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675d78d0 0x55df675ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.000+0000 7f75113f7740 1 -- 172.21.15.33:0/3946196255 >> 172.21.15.33:0/3946196255 conn(0x55df675d2000 msgr2=0x55df675d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.000+0000 7f75113f7740 1 -- 172.21.15.33:0/3946196255 shutdown_connections 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.000+0000 7f75113f7740 1 -- 172.21.15.33:0/3946196255 wait complete. 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f75113f7740 1 -- start start 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f75113f7740 1 Processor -- start 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f75113f7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d78d0 0x55df675da6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f75113f7740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675d81f0 0x55df675dac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f75113f7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675e0dd0 0x55df675db150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f75113f7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74f400e4e0 con 0x55df675d78d0 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f75113f7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f74f400f950 con 0x55df675e0dd0 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f75113f7740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f74f400ebb0 con 0x55df675d81f0 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f75113f7740 10 client.0 _ll_register_callbacks cb 0x7f7500003ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f7507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d78d0 0x55df675da6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f750cdaa640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675e0dd0 0x55df675db150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f7507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d78d0 0x55df675da6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36222/0 (socket says 172.21.15.33:36222) 2024-04-02T13:41:03.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f7507fff640 1 -- 172.21.15.33:0/842779874 learned_addr learned my addr 172.21.15.33:0/842779874 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.001+0000 7f75077fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675d81f0 0x55df675dac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f75077fe640 1 -- 172.21.15.33:0/842779874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675e0dd0 msgr2=0x55df675db150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f75077fe640 1 --2- 172.21.15.33:0/842779874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675e0dd0 0x55df675db150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f75077fe640 1 -- 172.21.15.33:0/842779874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d78d0 msgr2=0x55df675da6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f75077fe640 1 --2- 172.21.15.33:0/842779874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d78d0 0x55df675da6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f75077fe640 1 -- 172.21.15.33:0/842779874 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df676563b0 con 0x55df675d81f0 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f7507fff640 1 --2- 172.21.15.33:0/842779874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d78d0 0x55df675da6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f75077fe640 1 --2- 172.21.15.33:0/842779874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675d81f0 0x55df675dac10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009fd0 tx=0x7f74fc0a8b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e62b25ed1ce4fbbd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f74e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f74e77fe640 1 -- 172.21.15.33:0/842779874 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0d1ac0 con 0x55df675d81f0 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f74e77fe640 1 -- 172.21.15.33:0/842779874 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f74fc0c02d0 con 0x55df675d81f0 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f75113f7740 1 -- 172.21.15.33:0/842779874 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74fc0cde50 con 0x55df675d81f0 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f74e77fe640 1 -- 172.21.15.33:0/842779874 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f74fc0c0510 con 0x55df675d81f0 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f75113f7740 10 client.25555 fetch_fsmap learned FSMap version 27 2024-04-02T13:41:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f75113f7740 1 -- 172.21.15.33:0/842779874 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df676576c0 con 0x55df675d81f0 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.002+0000 7f74e77fe640 1 -- 172.21.15.33:0/842779874 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f74fc0cde50 con 0x55df675d81f0 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.003+0000 7f74e77fe640 1 -- 172.21.15.33:0/842779874 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f74fc0b90f0 con 0x55df675d81f0 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.003+0000 7f75113f7740 10 client.25555 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.003+0000 7f75113f7740 10 client.25555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.003+0000 7f75113f7740 20 client.25555 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.003+0000 7f75113f7740 1 --2- 172.21.15.33:0/842779874 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55df67657710 0x55df67659b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.003+0000 7f75113f7740 4 client.25555 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.003+0000 7f75113f7740 1 -- 172.21.15.33:0/842779874 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df675430c0 con 0x55df67657710 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.003+0000 7f7507fff640 1 --2- 172.21.15.33:0/842779874 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55df67657710 0x55df67659b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.003+0000 7f7507fff640 1 --2- 172.21.15.33:0/842779874 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55df67657710 0x55df67659b00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f158c45830384b8 server_cookie=458d4c96690a334c in_seq=0 out_seq=0 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.003+0000 7f74e77fe640 10 client.25555 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.003+0000 7f74e77fe640 1 -- 172.21.15.33:0/842779874 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55df675430c0 con 0x55df67657710 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.003+0000 7f74e77fe640 10 client.25555 handle_command_reply: tid=0 2024-04-02T13:41:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.003+0000 7f75113f7740 1 client.25555 shutdown 2024-04-02T13:41:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f75113f7740 1 -- 172.21.15.33:0/842779874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675d81f0 msgr2=0x55df675dac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f75113f7740 1 --2- 172.21.15.33:0/842779874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675d81f0 0x55df675dac10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009fd0 tx=0x7f74fc0a8b90 comp rx=0 tx=0).stop 2024-04-02T13:41:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f75113f7740 1 -- 172.21.15.33:0/842779874 shutdown_connections 2024-04-02T13:41:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f75113f7740 1 --2- 172.21.15.33:0/842779874 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55df67657710 0x55df67659b00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f75113f7740 1 --2- 172.21.15.33:0/842779874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55df675e0dd0 0x55df675db150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f74e77fe640 0 client.25555 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f75113f7740 1 --2- 172.21.15.33:0/842779874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55df675d81f0 0x55df675dac10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f75113f7740 1 --2- 172.21.15.33:0/842779874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55df675d78d0 0x55df675da6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f75113f7740 1 -- 172.21.15.33:0/842779874 >> 172.21.15.33:0/842779874 conn(0x55df675d2000 msgr2=0x55df675d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f75113f7740 1 -- 172.21.15.33:0/842779874 shutdown_connections 2024-04-02T13:41:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f75113f7740 1 -- 172.21.15.33:0/842779874 wait complete. 2024-04-02T13:41:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f75113f7740 20 client.25555 trim_cache size 0 max 16384 2024-04-02T13:41:03.013 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:41:03.013 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:41:03.013 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:41:03.013 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:41:03.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f750e80d640 1 -- 172.21.15.33:0/2125042792 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f74f00a3d20 msgr2=0x7f74f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f750e80d640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f74f00a3d20 0x7f74f00a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7500005cc0 tx=0x7f75000323c0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f750e80d640 1 -- 172.21.15.33:0/2125042792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f750806a9f0 msgr2=0x7f75081988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.004+0000 7f750e80d640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f750806a9f0 0x7f75081988d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009ce0 tx=0x7f74fc00b690 comp rx=0 tx=0).stop 2024-04-02T13:41:03.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.005+0000 7f750e80d640 1 -- 172.21.15.33:0/2125042792 shutdown_connections 2024-04-02T13:41:03.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.005+0000 7f750e80d640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f74f00a3d20 0x7f74f00a61e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.005+0000 7f750e80d640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7508107ac0 0x7f750819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.005+0000 7f750e80d640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f750806a9f0 0x7f75081988d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.005+0000 7f750e80d640 1 --2- 172.21.15.33:0/2125042792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f750806a070 0x7f7508198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.005+0000 7f750e80d640 1 -- 172.21.15.33:0/2125042792 >> 172.21.15.33:0/2125042792 conn(0x7f75080779e0 msgr2=0x7f75080fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.005+0000 7f750e80d640 1 -- 172.21.15.33:0/2125042792 shutdown_connections 2024-04-02T13:41:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.005+0000 7f750e80d640 1 -- 172.21.15.33:0/2125042792 wait complete. 2024-04-02T13:41:03.019 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:41:03.019 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:41:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.198+0000 7f48a1160640 1 -- 172.21.15.33:0/4068583178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f489c112aa0 msgr2=0x7f489c114e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.198+0000 7f48a1160640 1 --2- 172.21.15.33:0/4068583178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f489c112aa0 0x7f489c114e90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4894009ca0 tx=0x7f489402c180 comp rx=0 tx=0).stop 2024-04-02T13:41:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.198+0000 7f48a1160640 1 -- 172.21.15.33:0/4068583178 shutdown_connections 2024-04-02T13:41:03.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.198+0000 7f48a1160640 1 --2- 172.21.15.33:0/4068583178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f489c112aa0 0x7f489c114e90 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.198+0000 7f48a1160640 1 --2- 172.21.15.33:0/4068583178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f489c0747c0 0x7f489c112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.198+0000 7f48a1160640 1 --2- 172.21.15.33:0/4068583178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489c073ea0 0x7f489c074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.198+0000 7f48a1160640 1 -- 172.21.15.33:0/4068583178 >> 172.21.15.33:0/4068583178 conn(0x7f489c0fd5a0 msgr2=0x7f489c0ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.198+0000 7f48a1160640 1 -- 172.21.15.33:0/4068583178 shutdown_connections 2024-04-02T13:41:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.198+0000 7f48a1160640 1 -- 172.21.15.33:0/4068583178 wait complete. 2024-04-02T13:41:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.199+0000 7f48a1160640 1 Processor -- start 2024-04-02T13:41:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.199+0000 7f48a1160640 1 -- start start 2024-04-02T13:41:03.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.199+0000 7f48a1160640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489c073ea0 0x7f489c1a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.199+0000 7f48a1160640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f489c0747c0 0x7f489c1a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.199+0000 7f48a1160640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f489c112aa0 0x7f489c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.199+0000 7f48a1160640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f489c118470 con 0x7f489c0747c0 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.199+0000 7f48a1160640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1182f0 con 0x7f489c073ea0 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.199+0000 7f48a1160640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1185f0 con 0x7f489c112aa0 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.199+0000 7f489a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f489c0747c0 0x7f489c1a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.199+0000 7f489a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f489c0747c0 0x7f489c1a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36234/0 (socket says 172.21.15.33:36234) 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.199+0000 7f489a59c640 1 -- 172.21.15.33:0/3915219729 learned_addr learned my addr 172.21.15.33:0/3915219729 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.199+0000 7f489ad9d640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489c073ea0 0x7f489c1a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.199+0000 7f489b59e640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f489c112aa0 0x7f489c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.200+0000 7f489a59c640 1 -- 172.21.15.33:0/3915219729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f489c112aa0 msgr2=0x7f489c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.200+0000 7f489a59c640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f489c112aa0 0x7f489c19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.200+0000 7f489a59c640 1 -- 172.21.15.33:0/3915219729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489c073ea0 msgr2=0x7f489c1a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.200+0000 7f489a59c640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489c073ea0 0x7f489c1a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.200+0000 7f489a59c640 1 -- 172.21.15.33:0/3915219729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c19f000 con 0x7f489c0747c0 2024-04-02T13:41:03.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.200+0000 7f489ad9d640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489c073ea0 0x7f489c1a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:03.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.200+0000 7f489a59c640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f489c0747c0 0x7f489c1a4bb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f489000e9f0 tx=0x7f489000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.200+0000 7f4883fff640 1 -- 172.21.15.33:0/3915219729 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489000cd80 con 0x7f489c0747c0 2024-04-02T13:41:03.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.200+0000 7f4883fff640 1 -- 172.21.15.33:0/3915219729 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f48900054f0 con 0x7f489c0747c0 2024-04-02T13:41:03.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.200+0000 7f48a1160640 1 -- 172.21.15.33:0/3915219729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c19f2f0 con 0x7f489c0747c0 2024-04-02T13:41:03.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.200+0000 7f4883fff640 1 -- 172.21.15.33:0/3915219729 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4890010690 con 0x7f489c0747c0 2024-04-02T13:41:03.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.200+0000 7f48a1160640 1 -- 172.21.15.33:0/3915219729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c1ab4c0 con 0x7f489c0747c0 2024-04-02T13:41:03.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.201+0000 7f48a1160640 1 -- 172.21.15.33:0/3915219729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f489c0755e0 con 0x7f489c0747c0 2024-04-02T13:41:03.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.202+0000 7f4883fff640 1 -- 172.21.15.33:0/3915219729 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4890004f20 con 0x7f489c0747c0 2024-04-02T13:41:03.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.203+0000 7f4883fff640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f48840a3d20 0x7f48840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.203+0000 7f489ad9d640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f48840a3d20 0x7f48840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.203+0000 7f4883fff640 1 -- 172.21.15.33:0/3915219729 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f48900d0f60 con 0x7f489c0747c0 2024-04-02T13:41:03.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.203+0000 7f489ad9d640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f48840a3d20 0x7f48840a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f488c009fc0 tx=0x7f488c005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.208+0000 7f4883fff640 1 -- 172.21.15.33:0/3915219729 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f489009d250 con 0x7f489c0747c0 2024-04-02T13:41:03.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.471+0000 7f48a1160640 1 -- 172.21.15.33:0/3915219729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f489c02d370 con 0x7f489c0747c0 2024-04-02T13:41:03.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.471+0000 7f4883fff640 1 -- 172.21.15.33:0/3915219729 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11877 (secure 0 0 0) 0x7f489009cb50 con 0x7f489c0747c0 2024-04-02T13:41:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:41:03.474 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:40:55.698235+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:41:03.475 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-02T13:41:03.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.473+0000 7f48a1160640 1 -- 172.21.15.33:0/3915219729 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f48840a3d20 msgr2=0x7f48840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.473+0000 7f48a1160640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f48840a3d20 0x7f48840a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f488c009fc0 tx=0x7f488c005f90 comp rx=0 tx=0).stop 2024-04-02T13:41:03.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.473+0000 7f48a1160640 1 -- 172.21.15.33:0/3915219729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f489c0747c0 msgr2=0x7f489c1a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.473+0000 7f48a1160640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f489c0747c0 0x7f489c1a4bb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f489000e9f0 tx=0x7f489000edf0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.474+0000 7f48a1160640 1 -- 172.21.15.33:0/3915219729 shutdown_connections 2024-04-02T13:41:03.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.474+0000 7f48a1160640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f48840a3d20 0x7f48840a61e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.474+0000 7f48a1160640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f489c112aa0 0x7f489c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.474+0000 7f48a1160640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f489c0747c0 0x7f489c1a4bb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.474+0000 7f48a1160640 1 --2- 172.21.15.33:0/3915219729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f489c073ea0 0x7f489c1a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.474+0000 7f48a1160640 1 -- 172.21.15.33:0/3915219729 >> 172.21.15.33:0/3915219729 conn(0x7f489c0fd5a0 msgr2=0x7f489c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.474+0000 7f48a1160640 1 -- 172.21.15.33:0/3915219729 shutdown_connections 2024-04-02T13:41:03.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.474+0000 7f48a1160640 1 -- 172.21.15.33:0/3915219729 wait complete. 2024-04-02T13:41:03.488 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:41:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.604+0000 7f7752c2c640 1 -- 172.21.15.33:0/1519962938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f774c100bc0 msgr2=0x7f774c1066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.604+0000 7f7752c2c640 1 --2- 172.21.15.33:0/1519962938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f774c100bc0 0x7f774c1066f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7738009900 tx=0x7f773802bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.604+0000 7f7752c2c640 1 -- 172.21.15.33:0/1519962938 shutdown_connections 2024-04-02T13:41:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.604+0000 7f7752c2c640 1 --2- 172.21.15.33:0/1519962938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c106dd0 0x7f774c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.604+0000 7f7752c2c640 1 --2- 172.21.15.33:0/1519962938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f774c100bc0 0x7f774c1066f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.604+0000 7f7752c2c640 1 --2- 172.21.15.33:0/1519962938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f774c100210 0x7f774c1005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.604+0000 7f7752c2c640 1 -- 172.21.15.33:0/1519962938 >> 172.21.15.33:0/1519962938 conn(0x7f774c076f20 msgr2=0x7f774c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.604+0000 7f7752c2c640 1 -- 172.21.15.33:0/1519962938 shutdown_connections 2024-04-02T13:41:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.604+0000 7f7752c2c640 1 -- 172.21.15.33:0/1519962938 wait complete. 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f7752c2c640 1 Processor -- start 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f7752c2c640 1 -- start start 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f7752c2c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c100210 0x7f774c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f7752c2c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f774c106dd0 0x7f774c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f7752c2c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f774c19ccf0 0x7f774c19d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f7752c2c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f774c111eb0 con 0x7f774c106dd0 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f7752c2c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f774c111d30 con 0x7f774c100210 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f7752c2c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f774c112030 con 0x7f774c19ccf0 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f774bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f774c106dd0 0x7f774c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f77509c8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c100210 0x7f774c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f774bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f774c106dd0 0x7f774c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40370/0 (socket says 172.21.15.33:40370) 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f77509c8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c100210 0x7f774c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:49598/0 (socket says 172.21.15.33:49598) 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f774bfff640 1 -- 172.21.15.33:0/2058078291 learned_addr learned my addr 172.21.15.33:0/2058078291 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.605+0000 7f77511c9640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f774c19ccf0 0x7f774c19d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.606+0000 7f774bfff640 1 -- 172.21.15.33:0/2058078291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f774c19ccf0 msgr2=0x7f774c19d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.606+0000 7f774bfff640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f774c19ccf0 0x7f774c19d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.606+0000 7f774bfff640 1 -- 172.21.15.33:0/2058078291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c100210 msgr2=0x7f774c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.606+0000 7f774bfff640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c100210 0x7f774c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.606+0000 7f774bfff640 1 -- 172.21.15.33:0/2058078291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c19d7e0 con 0x7f774c106dd0 2024-04-02T13:41:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.606+0000 7f77509c8640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c100210 0x7f774c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.606+0000 7f77511c9640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f774c19ccf0 0x7f774c19d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.606+0000 7f774bfff640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f774c106dd0 0x7f774c198970 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7738005ab0 tx=0x7f7738004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.606+0000 7f7749ffb640 1 -- 172.21.15.33:0/2058078291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738043020 con 0x7f774c106dd0 2024-04-02T13:41:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.606+0000 7f7749ffb640 1 -- 172.21.15.33:0/2058078291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f773802c5d0 con 0x7f774c106dd0 2024-04-02T13:41:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.606+0000 7f7752c2c640 1 -- 172.21.15.33:0/2058078291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774c19da70 con 0x7f774c106dd0 2024-04-02T13:41:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.606+0000 7f7749ffb640 1 -- 172.21.15.33:0/2058078291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77380039f0 con 0x7f774c106dd0 2024-04-02T13:41:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.606+0000 7f7752c2c640 1 -- 172.21.15.33:0/2058078291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f774c1a5230 con 0x7f774c106dd0 2024-04-02T13:41:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.609+0000 7f7749ffb640 1 -- 172.21.15.33:0/2058078291 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7738046020 con 0x7f774c106dd0 2024-04-02T13:41:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.609+0000 7f775580f740 1 Processor -- start 2024-04-02T13:41:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.609+0000 7f775580f740 1 -- start start 2024-04-02T13:41:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.609+0000 7f775580f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa387a0 0x55696fa38b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.609+0000 7f775580f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa41760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.609+0000 7f775580f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa41ca0 0x55696fa44090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.609+0000 7f7749ffb640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f773c0a3d20 0x7f773c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.609+0000 7f775580f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55696f99c7b0 con 0x55696fa41ca0 2024-04-02T13:41:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.609+0000 7f775580f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55696f93b570 con 0x55696fa387a0 2024-04-02T13:41:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.609+0000 7f775580f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55696f998760 con 0x55696fa390c0 2024-04-02T13:41:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77511c9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa387a0 0x55696fa38b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77509c8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa41760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77511c9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa387a0 0x55696fa38b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:49610/0 (socket says 172.21.15.33:49610) 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77509c8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa41760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:38660/0 (socket says 172.21.15.33:38660) 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77511c9640 1 -- 172.21.15.33:0/789345360 learned_addr learned my addr 172.21.15.33:0/789345360 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f774bfff640 1 --2- 172.21.15.33:0/789345360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa41ca0 0x55696fa44090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77511c9640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f773c0a3d20 0x7f773c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f7749ffb640 1 -- 172.21.15.33:0/2058078291 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f77380f2400 con 0x7f774c106dd0 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77511c9640 1 -- 172.21.15.33:0/789345360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 msgr2=0x55696fa41760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77511c9640 1 --2- 172.21.15.33:0/789345360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa41760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77511c9640 1 -- 172.21.15.33:0/789345360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa41ca0 msgr2=0x55696fa44090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77511c9640 1 --2- 172.21.15.33:0/789345360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa41ca0 0x55696fa44090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77511c9640 1 -- 172.21.15.33:0/789345360 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55696f6b03f0 con 0x55696fa387a0 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77509c8640 1 --2- 172.21.15.33:0/789345360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa41760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77511c9640 1 --2- 172.21.15.33:0/789345360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa387a0 0x55696fa38b80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7744008690 tx=0x7f7744008b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=421ec2895a71a105 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f77511c9640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f773c0a3d20 0x7f773c0a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7744017640 tx=0x7f774401b000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f7736ffd640 1 -- 172.21.15.33:0/789345360 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744027070 con 0x55696fa387a0 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f7736ffd640 1 -- 172.21.15.33:0/789345360 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f774400b620 con 0x55696fa387a0 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.610+0000 7f7736ffd640 1 -- 172.21.15.33:0/789345360 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774401e070 con 0x55696fa387a0 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.611+0000 7f775580f740 1 -- 172.21.15.33:0/789345360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa387a0 msgr2=0x55696fa38b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.611+0000 7f775580f740 1 --2- 172.21.15.33:0/789345360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa387a0 0x55696fa38b80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7744008690 tx=0x7f7744008b50 comp rx=0 tx=0).stop 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.611+0000 7f775580f740 1 -- 172.21.15.33:0/789345360 shutdown_connections 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.611+0000 7f775580f740 1 --2- 172.21.15.33:0/789345360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa41ca0 0x55696fa44090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.611+0000 7f775580f740 1 --2- 172.21.15.33:0/789345360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa41760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.611+0000 7f775580f740 1 --2- 172.21.15.33:0/789345360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa387a0 0x55696fa38b80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.611+0000 7f775580f740 1 -- 172.21.15.33:0/789345360 >> 172.21.15.33:0/789345360 conn(0x55696fa32ed0 msgr2=0x55696fa332b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.611+0000 7f775580f740 1 -- 172.21.15.33:0/789345360 shutdown_connections 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.611+0000 7f775580f740 1 -- 172.21.15.33:0/789345360 wait complete. 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.611+0000 7f775580f740 1 -- start start 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.612+0000 7f775580f740 1 Processor -- start 2024-04-02T13:41:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.612+0000 7f775580f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 0x55696fa3eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.612+0000 7f775580f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa3f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.612+0000 7f775580f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa41ca0 0x55696fa3f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.612+0000 7f775580f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55696fa44a30 con 0x55696fa387a0 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.612+0000 7f775580f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55696f957e80 con 0x55696fa41ca0 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.612+0000 7f775580f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55696fa44bb0 con 0x55696fa390c0 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.612+0000 7f775580f740 10 client.0 _ll_register_callbacks cb 0x55696f993aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.612+0000 7f77509c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 0x55696fa3eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.612+0000 7f77509c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 0x55696fa3eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40382/0 (socket says 172.21.15.33:40382) 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.612+0000 7f77511c9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa41ca0 0x55696fa3f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.612+0000 7f77509c8640 1 -- 172.21.15.33:0/1191303348 learned_addr learned my addr 172.21.15.33:0/1191303348 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.612+0000 7f774bfff640 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa3f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f77511c9640 1 -- 172.21.15.33:0/1191303348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 msgr2=0x55696fa3f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f77511c9640 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa3f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f77511c9640 1 -- 172.21.15.33:0/1191303348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 msgr2=0x55696fa3eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f77511c9640 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 0x55696fa3eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f77511c9640 1 -- 172.21.15.33:0/1191303348 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55696f77a7d0 con 0x55696fa41ca0 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f774bfff640 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa3f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f77509c8640 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 0x55696fa3eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f77511c9640 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa41ca0 0x55696fa3f5c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7744009690 tx=0x7f7744008320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab6e9f9629a6a065 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f772bfff640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f772bfff640 1 -- 172.21.15.33:0/1191303348 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774401e070 con 0x55696fa41ca0 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f775580f740 1 -- 172.21.15.33:0/1191303348 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55696f9a9cf0 con 0x55696fa41ca0 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f772bfff640 1 -- 172.21.15.33:0/1191303348 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f774401abd0 con 0x55696fa41ca0 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f772bfff640 1 -- 172.21.15.33:0/1191303348 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f774403a050 con 0x55696fa41ca0 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f775580f740 10 client.25709 fetch_fsmap learned FSMap version 27 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f775580f740 1 -- 172.21.15.33:0/1191303348 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55696fa50b40 con 0x55696fa41ca0 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f772bfff640 1 -- 172.21.15.33:0/1191303348 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55696f9a9cf0 con 0x55696fa41ca0 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f772bfff640 1 -- 172.21.15.33:0/1191303348 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f774402b070 con 0x55696fa41ca0 2024-04-02T13:41:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f775580f740 10 client.25709 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.613+0000 7f775580f740 10 client.25709 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.614+0000 7f775580f740 20 client.25709 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.614+0000 7f775580f740 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55696fa51fa0 0x55696fa54390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.614+0000 7f775580f740 4 client.25709 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.614+0000 7f775580f740 1 -- 172.21.15.33:0/1191303348 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55696f9a4040 con 0x55696fa51fa0 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.614+0000 7f77509c8640 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55696fa51fa0 0x55696fa54390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.614+0000 7f77509c8640 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55696fa51fa0 0x55696fa54390 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e4964014f65143f server_cookie=593b5fecad4163a3 in_seq=0 out_seq=0 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.614+0000 7f772bfff640 10 client.25709 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f772bfff640 1 -- 172.21.15.33:0/1191303348 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55696f9a4040 con 0x55696fa51fa0 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f772bfff640 10 client.25709 handle_command_reply: tid=0 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f775580f740 1 client.25709 shutdown 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f775580f740 1 -- 172.21.15.33:0/1191303348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa41ca0 msgr2=0x55696fa3f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f775580f740 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa41ca0 0x55696fa3f5c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7744009690 tx=0x7f7744008320 comp rx=0 tx=0).stop 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f775580f740 1 -- 172.21.15.33:0/1191303348 shutdown_connections 2024-04-02T13:41:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f775580f740 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55696fa51fa0 0x55696fa54390 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f775580f740 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa41ca0 0x55696fa3f5c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f775580f740 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa3f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f772bfff640 0 client.25709 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:41:03.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f775580f740 1 --2- 172.21.15.33:0/1191303348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 0x55696fa3eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f775580f740 1 -- 172.21.15.33:0/1191303348 >> 172.21.15.33:0/1191303348 conn(0x55696fa32ed0 msgr2=0x55696fa34c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f775580f740 1 -- 172.21.15.33:0/1191303348 shutdown_connections 2024-04-02T13:41:03.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f775580f740 1 -- 172.21.15.33:0/1191303348 wait complete. 2024-04-02T13:41:03.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.615+0000 7f775580f740 20 client.25709 trim_cache size 0 max 16384 2024-04-02T13:41:03.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f775580f740 1 Processor -- start 2024-04-02T13:41:03.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f775580f740 1 -- start start 2024-04-02T13:41:03.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f775580f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 0x55696fa41270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f775580f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa4d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f775580f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa41ca0 0x55696fa59bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f775580f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1b18a0 con 0x55696fa387a0 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f775580f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1af140 con 0x55696fa41ca0 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f775580f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1acf70 con 0x55696fa390c0 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f77509c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 0x55696fa41270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f77509c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 0x55696fa41270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40396/0 (socket says 172.21.15.33:40396) 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f77509c8640 1 -- 172.21.15.33:0/2833680227 learned_addr learned my addr 172.21.15.33:0/2833680227 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f77511c9640 1 --2- 172.21.15.33:0/2833680227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa41ca0 0x55696fa59bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f774bfff640 1 --2- 172.21.15.33:0/2833680227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa4d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f77509c8640 1 -- 172.21.15.33:0/2833680227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 msgr2=0x55696fa4d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f77509c8640 1 --2- 172.21.15.33:0/2833680227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa4d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f77509c8640 1 -- 172.21.15.33:0/2833680227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa41ca0 msgr2=0x55696fa59bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f77509c8640 1 --2- 172.21.15.33:0/2833680227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa41ca0 0x55696fa59bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f77509c8640 1 -- 172.21.15.33:0/2833680227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55696fa38ef0 con 0x55696fa387a0 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.631+0000 7f77509c8640 1 --2- 172.21.15.33:0/2833680227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 0x55696fa41270 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55696fa54b60 tx=0x7f7740005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73b383044e1d8a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.632+0000 7f77367fc640 1 -- 172.21.15.33:0/2833680227 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77400220f0 con 0x55696fa387a0 2024-04-02T13:41:03.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.632+0000 7f77367fc640 1 -- 172.21.15.33:0/2833680227 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7740027070 con 0x55696fa387a0 2024-04-02T13:41:03.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.632+0000 7f77367fc640 1 -- 172.21.15.33:0/2833680227 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7740003d70 con 0x55696fa387a0 2024-04-02T13:41:03.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.632+0000 7f775580f740 1 -- 172.21.15.33:0/2833680227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 msgr2=0x55696fa41270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.632+0000 7f775580f740 1 --2- 172.21.15.33:0/2833680227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 0x55696fa41270 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55696fa54b60 tx=0x7f7740005f90 comp rx=0 tx=0).stop 2024-04-02T13:41:03.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.632+0000 7f775580f740 1 -- 172.21.15.33:0/2833680227 shutdown_connections 2024-04-02T13:41:03.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.632+0000 7f775580f740 1 --2- 172.21.15.33:0/2833680227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa41ca0 0x55696fa59bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.632+0000 7f775580f740 1 --2- 172.21.15.33:0/2833680227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa4d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.632+0000 7f775580f740 1 --2- 172.21.15.33:0/2833680227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa387a0 0x55696fa41270 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.632+0000 7f775580f740 1 -- 172.21.15.33:0/2833680227 >> 172.21.15.33:0/2833680227 conn(0x55696fa32ed0 msgr2=0x55696fa34b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.633+0000 7f775580f740 1 -- 172.21.15.33:0/2833680227 shutdown_connections 2024-04-02T13:41:03.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.633+0000 7f775580f740 1 -- 172.21.15.33:0/2833680227 wait complete. 2024-04-02T13:41:03.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.633+0000 7f775580f740 1 -- start start 2024-04-02T13:41:03.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.633+0000 7f775580f740 1 Processor -- start 2024-04-02T13:41:03.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.633+0000 7f775580f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa387a0 0x55696fa49e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.633+0000 7f775580f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa4a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.633+0000 7f775580f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa41ca0 0x55696fa3b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.633+0000 7f775580f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1ad600 con 0x55696fa41ca0 2024-04-02T13:41:03.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.633+0000 7f775580f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1aea70 con 0x55696fa387a0 2024-04-02T13:41:03.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.633+0000 7f775580f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1adcd0 con 0x55696fa390c0 2024-04-02T13:41:03.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.633+0000 7f775580f740 10 client.0 _ll_register_callbacks cb 0x7f7744025070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:03.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f77511c9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa41ca0 0x55696fa3b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f774bfff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa4a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f77509c8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa387a0 0x55696fa49e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f77511c9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa41ca0 0x55696fa3b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40408/0 (socket says 172.21.15.33:40408) 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f77511c9640 1 -- 172.21.15.33:0/3550249880 learned_addr learned my addr 172.21.15.33:0/3550249880 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f774bfff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa4a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:38668/0 (socket says 172.21.15.33:38668) 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f77511c9640 1 -- 172.21.15.33:0/3550249880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 msgr2=0x55696fa4a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f77511c9640 1 --2- 172.21.15.33:0/3550249880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa4a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f77511c9640 1 -- 172.21.15.33:0/3550249880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa387a0 msgr2=0x55696fa49e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f77511c9640 1 --2- 172.21.15.33:0/3550249880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa387a0 0x55696fa49e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f77511c9640 1 -- 172.21.15.33:0/3550249880 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55696fa3bab0 con 0x55696fa41ca0 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f774bfff640 1 --2- 172.21.15.33:0/3550249880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa4a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f77511c9640 1 --2- 172.21.15.33:0/3550249880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa41ca0 0x55696fa3b2b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7744002230 tx=0x7f774400bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce311d30bc5930e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f772bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f772bfff640 1 -- 172.21.15.33:0/3550249880 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774401e070 con 0x55696fa41ca0 2024-04-02T13:41:03.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f772bfff640 1 -- 172.21.15.33:0/3550249880 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f774401ade0 con 0x55696fa41ca0 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f775580f740 1 -- 172.21.15.33:0/3550249880 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7740027070 con 0x55696fa41ca0 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f772bfff640 1 -- 172.21.15.33:0/3550249880 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f774404d050 con 0x55696fa41ca0 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f775580f740 10 client.16116 fetch_fsmap learned FSMap version 27 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.634+0000 7f775580f740 1 -- 172.21.15.33:0/3550249880 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55696fab7fb0 con 0x55696fa41ca0 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.635+0000 7f772bfff640 1 -- 172.21.15.33:0/3550249880 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7740027070 con 0x55696fa41ca0 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.635+0000 7f772bfff640 1 -- 172.21.15.33:0/3550249880 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f774402b040 con 0x55696fa41ca0 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.635+0000 7f775580f740 10 client.16116 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.635+0000 7f775580f740 10 client.16116 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.635+0000 7f775580f740 20 client.16116 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.635+0000 7f775580f740 1 --2- 172.21.15.33:0/3550249880 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55696fab86c0 0x55696fabaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.635+0000 7f775580f740 4 client.16116 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.635+0000 7f775580f740 1 -- 172.21.15.33:0/3550249880 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55696f9216b0 con 0x55696fab86c0 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.635+0000 7f77509c8640 1 --2- 172.21.15.33:0/3550249880 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55696fab86c0 0x55696fabaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f77509c8640 1 --2- 172.21.15.33:0/3550249880 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55696fab86c0 0x55696fabaae0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4815c498f9e66fc4 server_cookie=bf0e78e722103a96 in_seq=0 out_seq=0 2024-04-02T13:41:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f772bfff640 10 client.16116 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f772bfff640 1 -- 172.21.15.33:0/3550249880 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55696f9216b0 con 0x55696fab86c0 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f772bfff640 10 client.16116 handle_command_reply: tid=0 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f775580f740 1 client.16116 shutdown 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f775580f740 1 -- 172.21.15.33:0/3550249880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa41ca0 msgr2=0x55696fa3b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f775580f740 1 --2- 172.21.15.33:0/3550249880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa41ca0 0x55696fa3b2b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7744002230 tx=0x7f774400bb50 comp rx=0 tx=0).stop 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f775580f740 1 -- 172.21.15.33:0/3550249880 shutdown_connections 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f775580f740 1 --2- 172.21.15.33:0/3550249880 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55696fab86c0 0x55696fabaae0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f775580f740 1 --2- 172.21.15.33:0/3550249880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55696fa41ca0 0x55696fa3b2b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f772bfff640 0 client.16116 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f775580f740 1 --2- 172.21.15.33:0/3550249880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55696fa390c0 0x55696fa4a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f775580f740 1 --2- 172.21.15.33:0/3550249880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55696fa387a0 0x55696fa49e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f775580f740 1 -- 172.21.15.33:0/3550249880 >> 172.21.15.33:0/3550249880 conn(0x55696fa32ed0 msgr2=0x55696fa3eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f775580f740 1 -- 172.21.15.33:0/3550249880 shutdown_connections 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f775580f740 1 -- 172.21.15.33:0/3550249880 wait complete. 2024-04-02T13:41:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.636+0000 7f775580f740 20 client.16116 trim_cache size 0 max 16384 2024-04-02T13:41:03.645 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:41:03.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.637+0000 7f7752c2c640 1 -- 172.21.15.33:0/2058078291 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f773c0a3d20 msgr2=0x7f773c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.637+0000 7f7752c2c640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f773c0a3d20 0x7f773c0a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7744017640 tx=0x7f774401b000 comp rx=0 tx=0).stop 2024-04-02T13:41:03.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.637+0000 7f7752c2c640 1 -- 172.21.15.33:0/2058078291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f774c106dd0 msgr2=0x7f774c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.637+0000 7f7752c2c640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f774c106dd0 0x7f774c198970 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7738005ab0 tx=0x7f7738004600 comp rx=0 tx=0).stop 2024-04-02T13:41:03.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.637+0000 7f7752c2c640 1 -- 172.21.15.33:0/2058078291 shutdown_connections 2024-04-02T13:41:03.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.637+0000 7f7752c2c640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f773c0a3d20 0x7f773c0a61e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.637+0000 7f7752c2c640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f774c19ccf0 0x7f774c19d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.637+0000 7f7752c2c640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f774c106dd0 0x7f774c198970 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.637+0000 7f7752c2c640 1 --2- 172.21.15.33:0/2058078291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c100210 0x7f774c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.637+0000 7f7752c2c640 1 -- 172.21.15.33:0/2058078291 >> 172.21.15.33:0/2058078291 conn(0x7f774c076f20 msgr2=0x7f774c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.637+0000 7f7752c2c640 1 -- 172.21.15.33:0/2058078291 shutdown_connections 2024-04-02T13:41:03.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.637+0000 7f7752c2c640 1 -- 172.21.15.33:0/2058078291 wait complete. 2024-04-02T13:41:03.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:41:03.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:03 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3915219729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:41:03.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.778+0000 7f13066ba640 1 -- 172.21.15.33:0/2804794613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1300106a30 msgr2=0x7f1300106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.778+0000 7f13066ba640 1 --2- 172.21.15.33:0/2804794613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1300106a30 0x7f1300106e10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009900 tx=0x7f12f002bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:03.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.778+0000 7f13066ba640 1 -- 172.21.15.33:0/2804794613 shutdown_connections 2024-04-02T13:41:03.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.778+0000 7f13066ba640 1 --2- 172.21.15.33:0/2804794613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13001016a0 0x7f130010d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.778+0000 7f13066ba640 1 --2- 172.21.15.33:0/2804794613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1300100ce0 0x7f1300101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.778+0000 7f13066ba640 1 --2- 172.21.15.33:0/2804794613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1300106a30 0x7f1300106e10 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.778+0000 7f13066ba640 1 -- 172.21.15.33:0/2804794613 >> 172.21.15.33:0/2804794613 conn(0x7f13000779e0 msgr2=0x7f1300077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.778+0000 7f13066ba640 1 -- 172.21.15.33:0/2804794613 shutdown_connections 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.778+0000 7f13066ba640 1 -- 172.21.15.33:0/2804794613 wait complete. 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.779+0000 7f13066ba640 1 Processor -- start 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.779+0000 7f13066ba640 1 -- start start 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.779+0000 7f13066ba640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1300100ce0 0x7f130019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.779+0000 7f13066ba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13001016a0 0x7f130019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.779+0000 7f13066ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1300106a30 0x7f130019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.779+0000 7f13066ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1300110d50 con 0x7f1300106a30 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.779+0000 7f13066ba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1300110bd0 con 0x7f13001016a0 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.779+0000 7f13066ba640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1300110ed0 con 0x7f1300100ce0 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f1304c57640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1300106a30 0x7f130019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f1304c57640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1300106a30 0x7f130019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40418/0 (socket says 172.21.15.33:40418) 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f12fffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1300100ce0 0x7f130019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f1304c57640 1 -- 172.21.15.33:0/3582612659 learned_addr learned my addr 172.21.15.33:0/3582612659 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f12ff7fe640 1 --2- 172.21.15.33:0/3582612659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13001016a0 0x7f130019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f1304c57640 1 -- 172.21.15.33:0/3582612659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1300100ce0 msgr2=0x7f130019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f1304c57640 1 --2- 172.21.15.33:0/3582612659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1300100ce0 0x7f130019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f1304c57640 1 -- 172.21.15.33:0/3582612659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13001016a0 msgr2=0x7f130019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f1304c57640 1 --2- 172.21.15.33:0/3582612659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13001016a0 0x7f130019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f1304c57640 1 -- 172.21.15.33:0/3582612659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130019f4a0 con 0x7f1300106a30 2024-04-02T13:41:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f1304c57640 1 --2- 172.21.15.33:0/3582612659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1300106a30 0x7f130019ed70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f12f800cad0 tx=0x7f12f800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f12fd7fa640 1 -- 172.21.15.33:0/3582612659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f800ae20 con 0x7f1300106a30 2024-04-02T13:41:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f12fd7fa640 1 -- 172.21.15.33:0/3582612659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f12f8005020 con 0x7f1300106a30 2024-04-02T13:41:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f13066ba640 1 -- 172.21.15.33:0/3582612659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130019f790 con 0x7f1300106a30 2024-04-02T13:41:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f12fd7fa640 1 -- 172.21.15.33:0/3582612659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f800f690 con 0x7f1300106a30 2024-04-02T13:41:03.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.780+0000 7f13066ba640 1 -- 172.21.15.33:0/3582612659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13001a7020 con 0x7f1300106a30 2024-04-02T13:41:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.782+0000 7f12fd7fa640 1 -- 172.21.15.33:0/3582612659 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f12f80051d0 con 0x7f1300106a30 2024-04-02T13:41:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f1309380740 1 Processor -- start 2024-04-02T13:41:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f1309380740 1 -- start start 2024-04-02T13:41:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f1309380740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb067a0 0x56200fb06b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f12fd7fa640 1 --2- 172.21.15.33:0/3582612659 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12e80a3d20 0x7f12e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f1309380740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb070c0 0x56200fb0f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f1309380740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb0fca0 0x56200fb12090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f1309380740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56200fa6a7b0 con 0x56200fb0fca0 2024-04-02T13:41:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f1309380740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56200fa09570 con 0x56200fb067a0 2024-04-02T13:41:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f1309380740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56200fa66760 con 0x56200fb070c0 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f1304c57640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb0fca0 0x56200fb12090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f12fffff640 1 --2- 172.21.15.33:0/3582612659 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12e80a3d20 0x7f12e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f12ff7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb067a0 0x56200fb06b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f1304c57640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb0fca0 0x56200fb12090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40420/0 (socket says 172.21.15.33:40420) 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f1304c57640 1 -- 172.21.15.33:0/3699599332 learned_addr learned my addr 172.21.15.33:0/3699599332 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f12ff7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb067a0 0x56200fb06b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:49668/0 (socket says 172.21.15.33:49668) 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f12fd7fa640 1 -- 172.21.15.33:0/3582612659 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f12f80d0a90 con 0x7f1300106a30 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f12ff7fe640 1 --2- 172.21.15.33:0/3699599332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb070c0 0x56200fb0f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f12ff7fe640 1 -- 172.21.15.33:0/3699599332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb070c0 msgr2=0x56200fb0f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.783+0000 7f12ff7fe640 1 --2- 172.21.15.33:0/3699599332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb070c0 0x56200fb0f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.784+0000 7f12ff7fe640 1 -- 172.21.15.33:0/3699599332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb0fca0 msgr2=0x56200fb12090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.784+0000 7f12ff7fe640 1 --2- 172.21.15.33:0/3699599332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb0fca0 0x56200fb12090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.784+0000 7f12ff7fe640 1 -- 172.21.15.33:0/3699599332 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56200f77e3f0 con 0x56200fb067a0 2024-04-02T13:41:03.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.784+0000 7f12ff7fe640 1 --2- 172.21.15.33:0/3699599332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb067a0 0x56200fb06b80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f12f400b6c0 tx=0x7f12f400b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d2807514952ecbf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.784+0000 7f12fffff640 1 --2- 172.21.15.33:0/3582612659 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12e80a3d20 0x7f12e80a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005ab0 tx=0x7f12f0005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.784+0000 7f12e67fc640 1 -- 172.21.15.33:0/3699599332 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4004020 con 0x56200fb067a0 2024-04-02T13:41:03.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.784+0000 7f12e67fc640 1 -- 172.21.15.33:0/3699599332 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f12f40054f0 con 0x56200fb067a0 2024-04-02T13:41:03.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.784+0000 7f12e67fc640 1 -- 172.21.15.33:0/3699599332 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4004b20 con 0x56200fb067a0 2024-04-02T13:41:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.785+0000 7f1309380740 1 -- 172.21.15.33:0/3699599332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb067a0 msgr2=0x56200fb06b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.785+0000 7f1309380740 1 --2- 172.21.15.33:0/3699599332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb067a0 0x56200fb06b80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f12f400b6c0 tx=0x7f12f400b650 comp rx=0 tx=0).stop 2024-04-02T13:41:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.785+0000 7f1309380740 1 -- 172.21.15.33:0/3699599332 shutdown_connections 2024-04-02T13:41:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.785+0000 7f1309380740 1 --2- 172.21.15.33:0/3699599332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb0fca0 0x56200fb12090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.785+0000 7f1309380740 1 --2- 172.21.15.33:0/3699599332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb070c0 0x56200fb0f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.785+0000 7f1309380740 1 --2- 172.21.15.33:0/3699599332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb067a0 0x56200fb06b80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.785+0000 7f1309380740 1 -- 172.21.15.33:0/3699599332 >> 172.21.15.33:0/3699599332 conn(0x56200fb00ed0 msgr2=0x56200fb012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.785+0000 7f1309380740 1 -- 172.21.15.33:0/3699599332 shutdown_connections 2024-04-02T13:41:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.785+0000 7f1309380740 1 -- 172.21.15.33:0/3699599332 wait complete. 2024-04-02T13:41:03.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.786+0000 7f1309380740 1 -- start start 2024-04-02T13:41:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.786+0000 7f1309380740 1 Processor -- start 2024-04-02T13:41:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.786+0000 7f1309380740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb067a0 0x56200fb0cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.786+0000 7f1309380740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb070c0 0x56200fb0d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.786+0000 7f1309380740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb0fca0 0x56200fb0d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f1309380740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56200fb12a90 con 0x56200fb067a0 2024-04-02T13:41:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f12fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb067a0 0x56200fb0cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f12ff7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb070c0 0x56200fb0d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f1309380740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56200fa25e80 con 0x56200fb070c0 2024-04-02T13:41:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f1309380740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56200fb12c10 con 0x56200fb0fca0 2024-04-02T13:41:03.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f12ff7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb070c0 0x56200fb0d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:49684/0 (socket says 172.21.15.33:49684) 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f12ff7fe640 1 -- 172.21.15.33:0/2584597768 learned_addr learned my addr 172.21.15.33:0/2584597768 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f1309380740 10 client.0 _ll_register_callbacks cb 0x56200fa61aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f1304c57640 1 --2- 172.21.15.33:0/2584597768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb0fca0 0x56200fb0d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f12ff7fe640 1 -- 172.21.15.33:0/2584597768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb0fca0 msgr2=0x56200fb0d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f12ff7fe640 1 --2- 172.21.15.33:0/2584597768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb0fca0 0x56200fb0d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f12ff7fe640 1 -- 172.21.15.33:0/2584597768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb067a0 msgr2=0x56200fb0cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f12ff7fe640 1 --2- 172.21.15.33:0/2584597768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb067a0 0x56200fb0cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f12ff7fe640 1 -- 172.21.15.33:0/2584597768 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56200f8487d0 con 0x56200fb070c0 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f1304c57640 1 --2- 172.21.15.33:0/2584597768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb0fca0 0x56200fb0d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f12ff7fe640 1 --2- 172.21.15.33:0/2584597768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb070c0 0x56200fb0d0a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f12f40057c0 tx=0x7f12f4012300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e666ffbccea72075 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f12df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f12df7fe640 1 -- 172.21.15.33:0/2584597768 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f402c060 con 0x56200fb070c0 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f1309380740 1 -- 172.21.15.33:0/2584597768 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12f40054f0 con 0x56200fb070c0 2024-04-02T13:41:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.787+0000 7f12df7fe640 1 -- 172.21.15.33:0/2584597768 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f12f4010670 con 0x56200fb070c0 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.788+0000 7f12df7fe640 1 -- 172.21.15.33:0/2584597768 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f12f4033050 con 0x56200fb070c0 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.788+0000 7f1309380740 10 client.25727 fetch_fsmap learned FSMap version 27 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.788+0000 7f1309380740 1 -- 172.21.15.33:0/2584597768 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56200fb1eba0 con 0x56200fb070c0 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.788+0000 7f12df7fe640 1 -- 172.21.15.33:0/2584597768 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f12f40054f0 con 0x56200fb070c0 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.788+0000 7f12df7fe640 1 -- 172.21.15.33:0/2584597768 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f12f400ea70 con 0x56200fb070c0 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.788+0000 7f1309380740 10 client.25727 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.788+0000 7f1309380740 10 client.25727 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.788+0000 7f1309380740 20 client.25727 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.788+0000 7f1309380740 1 --2- 172.21.15.33:0/2584597768 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56200fb1ff50 0x56200fb22340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.788+0000 7f1309380740 4 client.25727 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.788+0000 7f1309380740 1 -- 172.21.15.33:0/2584597768 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56200fa77cf0 con 0x56200fb1ff50 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.788+0000 7f12fffff640 1 --2- 172.21.15.33:0/2584597768 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56200fb1ff50 0x56200fb22340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.789+0000 7f12fffff640 1 --2- 172.21.15.33:0/2584597768 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56200fb1ff50 0x56200fb22340 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a431c2e6ed24d88 server_cookie=170453625debdb3c in_seq=0 out_seq=0 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.789+0000 7f12df7fe640 10 client.25727 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:41:03.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f12df7fe640 1 -- 172.21.15.33:0/2584597768 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56200fa77cf0 con 0x56200fb1ff50 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f12df7fe640 10 client.25727 handle_command_reply: tid=0 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f1309380740 1 client.25727 shutdown 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f1309380740 1 -- 172.21.15.33:0/2584597768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb070c0 msgr2=0x56200fb0d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f1309380740 1 --2- 172.21.15.33:0/2584597768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb070c0 0x56200fb0d0a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f12f40057c0 tx=0x7f12f4012300 comp rx=0 tx=0).stop 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f1309380740 1 -- 172.21.15.33:0/2584597768 shutdown_connections 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f1309380740 1 --2- 172.21.15.33:0/2584597768 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56200fb1ff50 0x56200fb22340 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f1309380740 1 --2- 172.21.15.33:0/2584597768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb0fca0 0x56200fb0d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f12df7fe640 0 client.25727 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f1309380740 1 --2- 172.21.15.33:0/2584597768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb070c0 0x56200fb0d0a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f1309380740 1 --2- 172.21.15.33:0/2584597768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb067a0 0x56200fb0cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f1309380740 1 -- 172.21.15.33:0/2584597768 >> 172.21.15.33:0/2584597768 conn(0x56200fb00ed0 msgr2=0x56200fb02c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f1309380740 1 -- 172.21.15.33:0/2584597768 shutdown_connections 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f1309380740 1 -- 172.21.15.33:0/2584597768 wait complete. 2024-04-02T13:41:03.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.790+0000 7f1309380740 20 client.25727 trim_cache size 0 max 16384 2024-04-02T13:41:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.807+0000 7f1309380740 1 Processor -- start 2024-04-02T13:41:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f1309380740 1 -- start start 2024-04-02T13:41:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f1309380740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb067a0 0x56200fb0f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f1309380740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb070c0 0x56200fb1b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f1309380740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb0fca0 0x56200fb0d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f1309380740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13001b3a20 con 0x56200fb070c0 2024-04-02T13:41:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f1309380740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f13001b12c0 con 0x56200fb0fca0 2024-04-02T13:41:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f1309380740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f13001af0f0 con 0x56200fb067a0 2024-04-02T13:41:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f12ff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb070c0 0x56200fb1b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f12fffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb067a0 0x56200fb0f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f1304c57640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb0fca0 0x56200fb0d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f12ff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb070c0 0x56200fb1b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40438/0 (socket says 172.21.15.33:40438) 2024-04-02T13:41:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f12fffff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb067a0 0x56200fb0f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:38698/0 (socket says 172.21.15.33:38698) 2024-04-02T13:41:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f12ff7fe640 1 -- 172.21.15.33:0/3464930918 learned_addr learned my addr 172.21.15.33:0/3464930918 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f12fffff640 1 -- 172.21.15.33:0/3464930918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb0fca0 msgr2=0x56200fb0d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f12fffff640 1 --2- 172.21.15.33:0/3464930918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb0fca0 0x56200fb0d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f12fffff640 1 -- 172.21.15.33:0/3464930918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb070c0 msgr2=0x56200fb1b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f12fffff640 1 --2- 172.21.15.33:0/3464930918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb070c0 0x56200fb1b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f12fffff640 1 -- 172.21.15.33:0/3464930918 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56200fb06ef0 con 0x56200fb067a0 2024-04-02T13:41:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f12ff7fe640 1 --2- 172.21.15.33:0/3464930918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb070c0 0x56200fb1b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.808+0000 7f12fffff640 1 --2- 172.21.15.33:0/3464930918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb067a0 0x56200fb0f290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009230 tx=0x7f12f003e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c72b2ad2a972ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.809+0000 7f12e5ffb640 1 -- 172.21.15.33:0/3464930918 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f002c750 con 0x56200fb067a0 2024-04-02T13:41:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.809+0000 7f12e5ffb640 1 -- 172.21.15.33:0/3464930918 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f12f00046c0 con 0x56200fb067a0 2024-04-02T13:41:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.809+0000 7f12e5ffb640 1 -- 172.21.15.33:0/3464930918 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f00035f0 con 0x56200fb067a0 2024-04-02T13:41:03.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.809+0000 7f1309380740 1 -- 172.21.15.33:0/3464930918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb067a0 msgr2=0x56200fb0f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.809+0000 7f1309380740 1 --2- 172.21.15.33:0/3464930918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb067a0 0x56200fb0f290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009230 tx=0x7f12f003e000 comp rx=0 tx=0).stop 2024-04-02T13:41:03.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.809+0000 7f1309380740 1 -- 172.21.15.33:0/3464930918 shutdown_connections 2024-04-02T13:41:03.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.809+0000 7f1309380740 1 --2- 172.21.15.33:0/3464930918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb0fca0 0x56200fb0d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.809+0000 7f1309380740 1 --2- 172.21.15.33:0/3464930918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb070c0 0x56200fb1b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.809+0000 7f1309380740 1 --2- 172.21.15.33:0/3464930918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb067a0 0x56200fb0f290 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.809+0000 7f1309380740 1 -- 172.21.15.33:0/3464930918 >> 172.21.15.33:0/3464930918 conn(0x56200fb00ed0 msgr2=0x56200fb0cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.809+0000 7f1309380740 1 -- 172.21.15.33:0/3464930918 shutdown_connections 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.809+0000 7f1309380740 1 -- 172.21.15.33:0/3464930918 wait complete. 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.810+0000 7f1309380740 1 -- start start 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.810+0000 7f1309380740 1 Processor -- start 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.810+0000 7f1309380740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb067a0 0x56200fb09930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.810+0000 7f1309380740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb070c0 0x56200fb09e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.810+0000 7f1309380740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb0fca0 0x56200fb0a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.810+0000 7f1309380740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13001af780 con 0x56200fb0fca0 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.810+0000 7f1309380740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f13001b0bf0 con 0x56200fb070c0 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.810+0000 7f1309380740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f13001afe50 con 0x56200fb067a0 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.810+0000 7f1309380740 10 client.0 _ll_register_callbacks cb 0x7f12f80058d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.810+0000 7f1304c57640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb0fca0 0x56200fb0a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.810+0000 7f1304c57640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb0fca0 0x56200fb0a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40452/0 (socket says 172.21.15.33:40452) 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.810+0000 7f1304c57640 1 -- 172.21.15.33:0/1361675747 learned_addr learned my addr 172.21.15.33:0/1361675747 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.810+0000 7f12ff7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb070c0 0x56200fb09e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f12fffff640 1 --2- 172.21.15.33:0/1361675747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb067a0 0x56200fb09930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f1304c57640 1 -- 172.21.15.33:0/1361675747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb067a0 msgr2=0x56200fb09930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f1304c57640 1 --2- 172.21.15.33:0/1361675747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb067a0 0x56200fb09930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f1304c57640 1 -- 172.21.15.33:0/1361675747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb070c0 msgr2=0x56200fb09e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f1304c57640 1 --2- 172.21.15.33:0/1361675747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb070c0 0x56200fb09e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f1304c57640 1 -- 172.21.15.33:0/1361675747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56200fb853f0 con 0x56200fb0fca0 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f12ff7fe640 1 --2- 172.21.15.33:0/1361675747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb070c0 0x56200fb09e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f1304c57640 1 --2- 172.21.15.33:0/1361675747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb0fca0 0x56200fb0a3b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f12f80a4e00 tx=0x7f12f801c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd08a6a091c75ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f12df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f12df7fe640 1 -- 172.21.15.33:0/1361675747 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80055d0 con 0x56200fb0fca0 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f12df7fe640 1 -- 172.21.15.33:0/1361675747 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f12f809aa70 con 0x56200fb0fca0 2024-04-02T13:41:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f1309380740 1 -- 172.21.15.33:0/1361675747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12f00046c0 con 0x56200fb0fca0 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f12df7fe640 1 -- 172.21.15.33:0/1361675747 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f12f80afb40 con 0x56200fb0fca0 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f1309380740 10 client.16134 fetch_fsmap learned FSMap version 27 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f1309380740 1 -- 172.21.15.33:0/1361675747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56200fb866f0 con 0x56200fb0fca0 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.811+0000 7f12df7fe640 1 -- 172.21.15.33:0/1361675747 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f12f00046c0 con 0x56200fb0fca0 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.812+0000 7f12df7fe640 1 -- 172.21.15.33:0/1361675747 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f12f80ce0f0 con 0x56200fb0fca0 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.812+0000 7f1309380740 10 client.16134 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.812+0000 7f1309380740 10 client.16134 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.812+0000 7f1309380740 20 client.16134 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.812+0000 7f1309380740 1 --2- 172.21.15.33:0/1361675747 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56200fb86e10 0x56200fb89230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.812+0000 7f1309380740 4 client.16134 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.812+0000 7f1309380740 1 -- 172.21.15.33:0/1361675747 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56200fa72040 con 0x56200fb86e10 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.812+0000 7f12fffff640 1 --2- 172.21.15.33:0/1361675747 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56200fb86e10 0x56200fb89230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.812+0000 7f12df7fe640 10 client.16134 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.812+0000 7f12fffff640 1 --2- 172.21.15.33:0/1361675747 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56200fb86e10 0x56200fb89230 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5b0f78273d310f7 server_cookie=77da78b5f0edc9cb in_seq=0 out_seq=0 2024-04-02T13:41:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f12df7fe640 1 -- 172.21.15.33:0/1361675747 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56200fa72040 con 0x56200fb86e10 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f12df7fe640 10 client.16134 handle_command_reply: tid=0 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f1309380740 1 client.16134 shutdown 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f1309380740 1 -- 172.21.15.33:0/1361675747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb0fca0 msgr2=0x56200fb0a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f1309380740 1 --2- 172.21.15.33:0/1361675747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb0fca0 0x56200fb0a3b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f12f80a4e00 tx=0x7f12f801c7f0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f1309380740 1 -- 172.21.15.33:0/1361675747 shutdown_connections 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f1309380740 1 --2- 172.21.15.33:0/1361675747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56200fb0fca0 0x56200fb0a3b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f1309380740 1 --2- 172.21.15.33:0/1361675747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56200fb070c0 0x56200fb09e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f1309380740 1 --2- 172.21.15.33:0/1361675747 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56200fb86e10 0x56200fb89230 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f1309380740 1 --2- 172.21.15.33:0/1361675747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56200fb067a0 0x56200fb09930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f12df7fe640 0 client.16134 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f1309380740 1 -- 172.21.15.33:0/1361675747 >> 172.21.15.33:0/1361675747 conn(0x56200fb00ed0 msgr2=0x56200fafff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f1309380740 1 -- 172.21.15.33:0/1361675747 shutdown_connections 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f1309380740 1 -- 172.21.15.33:0/1361675747 wait complete. 2024-04-02T13:41:03.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.813+0000 7f1309380740 20 client.16134 trim_cache size 0 max 16384 2024-04-02T13:41:03.822 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:41:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.814+0000 7f13066ba640 1 -- 172.21.15.33:0/3582612659 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12e80a3d20 msgr2=0x7f12e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.814+0000 7f13066ba640 1 --2- 172.21.15.33:0/3582612659 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12e80a3d20 0x7f12e80a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005ab0 tx=0x7f12f0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.814+0000 7f13066ba640 1 -- 172.21.15.33:0/3582612659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1300106a30 msgr2=0x7f130019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.814+0000 7f13066ba640 1 --2- 172.21.15.33:0/3582612659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1300106a30 0x7f130019ed70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f12f800cad0 tx=0x7f12f800cf90 comp rx=0 tx=0).stop 2024-04-02T13:41:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.814+0000 7f13066ba640 1 -- 172.21.15.33:0/3582612659 shutdown_connections 2024-04-02T13:41:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.814+0000 7f13066ba640 1 --2- 172.21.15.33:0/3582612659 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12e80a3d20 0x7f12e80a61e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.814+0000 7f13066ba640 1 --2- 172.21.15.33:0/3582612659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1300106a30 0x7f130019ed70 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.814+0000 7f13066ba640 1 --2- 172.21.15.33:0/3582612659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13001016a0 0x7f130019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.814+0000 7f13066ba640 1 --2- 172.21.15.33:0/3582612659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1300100ce0 0x7f130019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.814+0000 7f13066ba640 1 -- 172.21.15.33:0/3582612659 >> 172.21.15.33:0/3582612659 conn(0x7f13000779e0 msgr2=0x7f130006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.814+0000 7f13066ba640 1 -- 172.21.15.33:0/3582612659 shutdown_connections 2024-04-02T13:41:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.814+0000 7f13066ba640 1 -- 172.21.15.33:0/3582612659 wait complete. 2024-04-02T13:41:03.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:41:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.949+0000 7fc01920d640 1 -- 172.21.15.33:0/2770859511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc014106980 msgr2=0x7fc014106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.949+0000 7fc01920d640 1 --2- 172.21.15.33:0/2770859511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc014106980 0x7fc014106d60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc004009930 tx=0x7fc00402bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.949+0000 7fc01920d640 1 -- 172.21.15.33:0/2770859511 shutdown_connections 2024-04-02T13:41:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.949+0000 7fc01920d640 1 --2- 172.21.15.33:0/2770859511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc014101330 0x7fc01410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.949+0000 7fc01920d640 1 --2- 172.21.15.33:0/2770859511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc014100970 0x7fc014100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.949+0000 7fc01920d640 1 --2- 172.21.15.33:0/2770859511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc014106980 0x7fc014106d60 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.949+0000 7fc01920d640 1 -- 172.21.15.33:0/2770859511 >> 172.21.15.33:0/2770859511 conn(0x7fc014077b00 msgr2=0x7fc014077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.949+0000 7fc01920d640 1 -- 172.21.15.33:0/2770859511 shutdown_connections 2024-04-02T13:41:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.949+0000 7fc01920d640 1 -- 172.21.15.33:0/2770859511 wait complete. 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.950+0000 7fc01920d640 1 Processor -- start 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.950+0000 7fc01920d640 1 -- start start 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.950+0000 7fc01920d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc014100970 0x7fc0141991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.950+0000 7fc01920d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc014101330 0x7fc014199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.950+0000 7fc01920d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc014106980 0x7fc0141a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.950+0000 7fc01920d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc014110db0 con 0x7fc014101330 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.950+0000 7fc01920d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc014110c30 con 0x7fc014106980 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.950+0000 7fc01920d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc014110f30 con 0x7fc014100970 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.951+0000 7fc012d9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc014100970 0x7fc0141991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.951+0000 7fc012d9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc014100970 0x7fc0141991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:38724/0 (socket says 172.21.15.33:38724) 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.951+0000 7fc01359e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc014106980 0x7fc0141a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.951+0000 7fc012d9d640 1 -- 172.21.15.33:0/3267363380 learned_addr learned my addr 172.21.15.33:0/3267363380 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.951+0000 7fc00a59c640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc014101330 0x7fc014199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.951+0000 7fc012d9d640 1 -- 172.21.15.33:0/3267363380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc014106980 msgr2=0x7fc0141a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.951+0000 7fc012d9d640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc014106980 0x7fc0141a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.951+0000 7fc012d9d640 1 -- 172.21.15.33:0/3267363380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc014101330 msgr2=0x7fc014199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.951+0000 7fc012d9d640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc014101330 0x7fc014199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.951+0000 7fc012d9d640 1 -- 172.21.15.33:0/3267363380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0141a4790 con 0x7fc014100970 2024-04-02T13:41:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.951+0000 7fc01359e640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc014106980 0x7fc0141a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.951+0000 7fc012d9d640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc014100970 0x7fc0141991d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc004000c00 tx=0x7fc004004e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.952+0000 7fc010d99640 1 -- 172.21.15.33:0/3267363380 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004043020 con 0x7fc014100970 2024-04-02T13:41:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.952+0000 7fc010d99640 1 -- 172.21.15.33:0/3267363380 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc00402c5d0 con 0x7fc014100970 2024-04-02T13:41:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.952+0000 7fc01920d640 1 -- 172.21.15.33:0/3267363380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0141a4960 con 0x7fc014100970 2024-04-02T13:41:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.952+0000 7fc010d99640 1 -- 172.21.15.33:0/3267363380 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004003a80 con 0x7fc014100970 2024-04-02T13:41:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.952+0000 7fc01920d640 1 -- 172.21.15.33:0/3267363380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0141a4e70 con 0x7fc014100970 2024-04-02T13:41:03.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.954+0000 7fc010d99640 1 -- 172.21.15.33:0/3267363380 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc004046020 con 0x7fc014100970 2024-04-02T13:41:03.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.954+0000 7fc01bdf4740 1 Processor -- start 2024-04-02T13:41:03.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.954+0000 7fc01bdf4740 1 -- start start 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc010d99640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc0000a3da0 0x7fc0000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc01bdf4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa37a0 0x56165caa3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc01bdf4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165caac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc01bdf4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 0x56165caaf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc01bdf4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56165ca077b0 con 0x56165caa40c0 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc01bdf4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56165c9a6570 con 0x56165caa37a0 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc01bdf4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56165ca03760 con 0x56165caacca0 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc01359e640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc0000a3da0 0x7fc0000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc010d99640 1 -- 172.21.15.33:0/3267363380 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc0040f2990 con 0x7fc014100970 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc012d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa37a0 0x56165caa3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc00a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165caac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc012d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa37a0 0x56165caa3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:49738/0 (socket says 172.21.15.33:49738) 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc012d9d640 1 -- 172.21.15.33:0/686068227 learned_addr learned my addr 172.21.15.33:0/686068227 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc01359e640 1 --2- 172.21.15.33:0/686068227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 0x56165caaf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc012d9d640 1 -- 172.21.15.33:0/686068227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 msgr2=0x56165caaf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc012d9d640 1 --2- 172.21.15.33:0/686068227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 0x56165caaf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc012d9d640 1 -- 172.21.15.33:0/686068227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 msgr2=0x56165caac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc012d9d640 1 --2- 172.21.15.33:0/686068227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165caac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc012d9d640 1 -- 172.21.15.33:0/686068227 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56165c71b3f0 con 0x56165caa37a0 2024-04-02T13:41:03.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc01359e640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc0000a3da0 0x7fc0000a6260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0099e0 tx=0x7fc00c0135e0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.955+0000 7fc012d9d640 1 --2- 172.21.15.33:0/686068227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa37a0 0x56165caa3b80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc004006010 tx=0x7fc004009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c432003e4c9725a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.956+0000 7fc00959a640 1 -- 172.21.15.33:0/686068227 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0040d4070 con 0x56165caa37a0 2024-04-02T13:41:03.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.956+0000 7fc00959a640 1 -- 172.21.15.33:0/686068227 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc0040bedd0 con 0x56165caa37a0 2024-04-02T13:41:03.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.956+0000 7fc00959a640 1 -- 172.21.15.33:0/686068227 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0040c2a80 con 0x56165caa37a0 2024-04-02T13:41:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.956+0000 7fc01bdf4740 1 -- 172.21.15.33:0/686068227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa37a0 msgr2=0x56165caa3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.956+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/686068227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa37a0 0x56165caa3b80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc004006010 tx=0x7fc004009560 comp rx=0 tx=0).stop 2024-04-02T13:41:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 -- 172.21.15.33:0/686068227 shutdown_connections 2024-04-02T13:41:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/686068227 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 0x56165caaf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/686068227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165caac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/686068227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa37a0 0x56165caa3b80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 -- 172.21.15.33:0/686068227 >> 172.21.15.33:0/686068227 conn(0x56165ca9ded0 msgr2=0x56165ca9e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 -- 172.21.15.33:0/686068227 shutdown_connections 2024-04-02T13:41:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 -- 172.21.15.33:0/686068227 wait complete. 2024-04-02T13:41:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 -- start start 2024-04-02T13:41:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 Processor -- start 2024-04-02T13:41:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa37a0 0x56165caa9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa40c0 0x56165caaa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 0x56165caaa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56165caaf850 con 0x56165caa37a0 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56165c9c2e80 con 0x56165caa40c0 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56165caaf9d0 con 0x56165caacca0 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01bdf4740 10 client.0 _ll_register_callbacks cb 0x56165c9feaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc012d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa37a0 0x56165caa9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc012d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa37a0 0x56165caa9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40478/0 (socket says 172.21.15.33:40478) 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc012d9d640 1 -- 172.21.15.33:0/3819195695 learned_addr learned my addr 172.21.15.33:0/3819195695 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc00a59c640 1 --2- 172.21.15.33:0/3819195695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa40c0 0x56165caaa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.957+0000 7fc01359e640 1 --2- 172.21.15.33:0/3819195695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 0x56165caaa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fc012d9d640 1 -- 172.21.15.33:0/3819195695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 msgr2=0x56165caaa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fc012d9d640 1 --2- 172.21.15.33:0/3819195695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 0x56165caaa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fc012d9d640 1 -- 172.21.15.33:0/3819195695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa40c0 msgr2=0x56165caaa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fc012d9d640 1 --2- 172.21.15.33:0/3819195695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa40c0 0x56165caaa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fc012d9d640 1 -- 172.21.15.33:0/3819195695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56165c7e57d0 con 0x56165caa37a0 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fc012d9d640 1 --2- 172.21.15.33:0/3819195695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa37a0 0x56165caa9b00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc00402c170 tx=0x7fc0040efe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd0b411d67bae93 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fbffa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fbffa7fc640 1 -- 172.21.15.33:0/3819195695 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0040d4030 con 0x56165caa37a0 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fbffa7fc640 1 -- 172.21.15.33:0/3819195695 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc0040bc770 con 0x56165caa37a0 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fc01bdf4740 1 -- 172.21.15.33:0/3819195695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0040bedd0 con 0x56165caa37a0 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fbffa7fc640 1 -- 172.21.15.33:0/3819195695 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0040f0200 con 0x56165caa37a0 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fc01bdf4740 10 client.16146 fetch_fsmap learned FSMap version 27 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fc01bdf4740 1 -- 172.21.15.33:0/3819195695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56165caaf850 con 0x56165caa37a0 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.958+0000 7fbffa7fc640 1 -- 172.21.15.33:0/3819195695 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc0040bedd0 con 0x56165caa37a0 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.959+0000 7fbffa7fc640 1 -- 172.21.15.33:0/3819195695 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fc0040c6eb0 con 0x56165caa37a0 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.959+0000 7fc01bdf4740 10 client.16146 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.959+0000 7fc01bdf4740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.959+0000 7fc01bdf4740 20 client.16146 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.959+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/3819195695 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56165cabcb20 0x56165cabef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.959+0000 7fc01bdf4740 4 client.16146 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.959+0000 7fc01bdf4740 1 -- 172.21.15.33:0/3819195695 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56165ca14cf0 con 0x56165cabcb20 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.959+0000 7fc00a59c640 1 --2- 172.21.15.33:0/3819195695 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56165cabcb20 0x56165cabef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.959+0000 7fc00a59c640 1 --2- 172.21.15.33:0/3819195695 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56165cabcb20 0x56165cabef90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=328f6c2401cc6591 server_cookie=5971d4369a80df3f in_seq=0 out_seq=0 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.959+0000 7fbffa7fc640 10 client.16146 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.961+0000 7fbffa7fc640 1 -- 172.21.15.33:0/3819195695 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56165ca14cf0 con 0x56165cabcb20 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.961+0000 7fbffa7fc640 10 client.16146 handle_command_reply: tid=0 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.961+0000 7fc01bdf4740 1 client.16146 shutdown 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.961+0000 7fc01bdf4740 1 -- 172.21.15.33:0/3819195695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa37a0 msgr2=0x56165caa9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.961+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/3819195695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa37a0 0x56165caa9b00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc00402c170 tx=0x7fc0040efe20 comp rx=0 tx=0).stop 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.961+0000 7fc01bdf4740 1 -- 172.21.15.33:0/3819195695 shutdown_connections 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.961+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/3819195695 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56165cabcb20 0x56165cabef90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.962+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/3819195695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 0x56165caaa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.962+0000 7fbffa7fc640 0 client.16146 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.962+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/3819195695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa40c0 0x56165caaa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.962+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/3819195695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa37a0 0x56165caa9b00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.962+0000 7fc01bdf4740 1 -- 172.21.15.33:0/3819195695 >> 172.21.15.33:0/3819195695 conn(0x56165ca9ded0 msgr2=0x56165ca9fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.962+0000 7fc01bdf4740 1 -- 172.21.15.33:0/3819195695 shutdown_connections 2024-04-02T13:41:03.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.962+0000 7fc01bdf4740 1 -- 172.21.15.33:0/3819195695 wait complete. 2024-04-02T13:41:03.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.962+0000 7fc01bdf4740 20 client.16146 trim_cache size 0 max 16384 2024-04-02T13:41:03.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:03 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3915219729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:41:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.976+0000 7fc01bdf4740 1 Processor -- start 2024-04-02T13:41:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.976+0000 7fc01bdf4740 1 -- start start 2024-04-02T13:41:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.976+0000 7fc01bdf4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa37a0 0x56165caac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.976+0000 7fc01bdf4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165cab85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.976+0000 7fc01bdf4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 0x56165cac4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.976+0000 7fc01bdf4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141b17c0 con 0x56165caa40c0 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.976+0000 7fc01bdf4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141af060 con 0x56165caa37a0 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.976+0000 7fc01bdf4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141ace90 con 0x56165caacca0 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.976+0000 7fc00a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165cab85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.976+0000 7fc00a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165cab85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40494/0 (socket says 172.21.15.33:40494) 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.976+0000 7fc00a59c640 1 -- 172.21.15.33:0/1291746368 learned_addr learned my addr 172.21.15.33:0/1291746368 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.976+0000 7fc012d9d640 1 --2- 172.21.15.33:0/1291746368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa37a0 0x56165caac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.977+0000 7fc01359e640 1 --2- 172.21.15.33:0/1291746368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 0x56165cac4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.977+0000 7fc00a59c640 1 -- 172.21.15.33:0/1291746368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 msgr2=0x56165cac4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.977+0000 7fc00a59c640 1 --2- 172.21.15.33:0/1291746368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 0x56165cac4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.977+0000 7fc00a59c640 1 -- 172.21.15.33:0/1291746368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa37a0 msgr2=0x56165caac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.977+0000 7fc00a59c640 1 --2- 172.21.15.33:0/1291746368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa37a0 0x56165caac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.977+0000 7fc00a59c640 1 -- 172.21.15.33:0/1291746368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56165caa35d0 con 0x56165caa40c0 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.977+0000 7fc00a59c640 1 --2- 172.21.15.33:0/1291746368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165cab85f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x56165caa9360 tx=0x7fbffc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421413b25fe311d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.977+0000 7fc008d99640 1 -- 172.21.15.33:0/1291746368 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc01f630 con 0x56165caa40c0 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.977+0000 7fc008d99640 1 -- 172.21.15.33:0/1291746368 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbffc0054f0 con 0x56165caa40c0 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.977+0000 7fc008d99640 1 -- 172.21.15.33:0/1291746368 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc01e390 con 0x56165caa40c0 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.978+0000 7fc01bdf4740 1 -- 172.21.15.33:0/1291746368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 msgr2=0x56165cab85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.978+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/1291746368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165cab85f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x56165caa9360 tx=0x7fbffc005f90 comp rx=0 tx=0).stop 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.978+0000 7fc01bdf4740 1 -- 172.21.15.33:0/1291746368 shutdown_connections 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.978+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/1291746368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caacca0 0x56165cac4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.978+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/1291746368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165cab85f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.978+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/1291746368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caa37a0 0x56165caac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.978+0000 7fc01bdf4740 1 -- 172.21.15.33:0/1291746368 >> 172.21.15.33:0/1291746368 conn(0x56165ca9ded0 msgr2=0x56165ca9cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.978+0000 7fc01bdf4740 1 -- 172.21.15.33:0/1291746368 shutdown_connections 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.978+0000 7fc01bdf4740 1 -- 172.21.15.33:0/1291746368 wait complete. 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.978+0000 7fc01bdf4740 1 -- start start 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.978+0000 7fc01bdf4740 1 Processor -- start 2024-04-02T13:41:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc01bdf4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caa37a0 0x56165cab05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc01bdf4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165cab0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc01bdf4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caacca0 0x56165cab1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc01bdf4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141ad520 con 0x56165caa40c0 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc01bdf4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141ae990 con 0x56165caacca0 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc01bdf4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141adbf0 con 0x56165caa37a0 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc01bdf4740 10 client.0 _ll_register_callbacks cb 0x7fbffc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc00a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165cab0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc00a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165cab0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40502/0 (socket says 172.21.15.33:40502) 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc00a59c640 1 -- 172.21.15.33:0/114400814 learned_addr learned my addr 172.21.15.33:0/114400814 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc012d9d640 1 --2- 172.21.15.33:0/114400814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caa37a0 0x56165cab05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc01359e640 1 --2- 172.21.15.33:0/114400814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caacca0 0x56165cab1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc00a59c640 1 -- 172.21.15.33:0/114400814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caa37a0 msgr2=0x56165cab05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc00a59c640 1 --2- 172.21.15.33:0/114400814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caa37a0 0x56165cab05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc00a59c640 1 -- 172.21.15.33:0/114400814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caacca0 msgr2=0x56165cab1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc00a59c640 1 --2- 172.21.15.33:0/114400814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caacca0 0x56165cab1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc00a59c640 1 -- 172.21.15.33:0/114400814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56165cb22120 con 0x56165caa40c0 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fc00a59c640 1 --2- 172.21.15.33:0/114400814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165cab0b10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009b90 tx=0x7fbffc009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad269acc4e62236a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.979+0000 7fbffa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fbffa7fc640 1 -- 172.21.15.33:0/114400814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc01f630 con 0x56165caa40c0 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fbffa7fc640 1 -- 172.21.15.33:0/114400814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbffc01f7d0 con 0x56165caa40c0 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fc01bdf4740 1 -- 172.21.15.33:0/114400814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbffc0054f0 con 0x56165caa40c0 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fbffa7fc640 1 -- 172.21.15.33:0/114400814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbffc039050 con 0x56165caa40c0 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fc01bdf4740 10 client.16152 fetch_fsmap learned FSMap version 27 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fc01bdf4740 1 -- 172.21.15.33:0/114400814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56165cb23450 con 0x56165caa40c0 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fbffa7fc640 1 -- 172.21.15.33:0/114400814 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbffc0054f0 con 0x56165caa40c0 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fbffa7fc640 1 -- 172.21.15.33:0/114400814 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fbffc027850 con 0x56165caa40c0 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fc01bdf4740 10 client.16152 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fc01bdf4740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fc01bdf4740 20 client.16152 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/114400814 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56165cabcb20 0x56165cb25c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fc01bdf4740 4 client.16152 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.980+0000 7fc01bdf4740 1 -- 172.21.15.33:0/114400814 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56165ca0f040 con 0x56165cabcb20 2024-04-02T13:41:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.981+0000 7fc012d9d640 1 --2- 172.21.15.33:0/114400814 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56165cabcb20 0x56165cb25c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.981+0000 7fc012d9d640 1 --2- 172.21.15.33:0/114400814 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56165cabcb20 0x56165cb25c40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebefd4d66d410da7 server_cookie=2467e901324585ee in_seq=0 out_seq=0 2024-04-02T13:41:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.981+0000 7fbffa7fc640 10 client.16152 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fbffa7fc640 1 -- 172.21.15.33:0/114400814 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56165ca0f040 con 0x56165cabcb20 2024-04-02T13:41:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fbffa7fc640 10 client.16152 handle_command_reply: tid=0 2024-04-02T13:41:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fc01bdf4740 1 client.16152 shutdown 2024-04-02T13:41:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fc01bdf4740 1 -- 172.21.15.33:0/114400814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 msgr2=0x56165cab0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/114400814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165cab0b10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009b90 tx=0x7fbffc009bc0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fc01bdf4740 1 -- 172.21.15.33:0/114400814 shutdown_connections 2024-04-02T13:41:03.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/114400814 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56165cabcb20 0x56165cb25c40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/114400814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165caacca0 0x56165cab1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fbffa7fc640 0 client.16152 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:03.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/114400814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56165caa40c0 0x56165cab0b10 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fc01bdf4740 1 --2- 172.21.15.33:0/114400814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56165caa37a0 0x56165cab05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fc01bdf4740 1 -- 172.21.15.33:0/114400814 >> 172.21.15.33:0/114400814 conn(0x56165ca9ded0 msgr2=0x56165caa82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fc01bdf4740 1 -- 172.21.15.33:0/114400814 shutdown_connections 2024-04-02T13:41:03.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fc01bdf4740 1 -- 172.21.15.33:0/114400814 wait complete. 2024-04-02T13:41:03.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.985+0000 7fc01bdf4740 20 client.16152 trim_cache size 0 max 16384 2024-04-02T13:41:03.990 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:41:03.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.986+0000 7fc01920d640 1 -- 172.21.15.33:0/3267363380 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc0000a3da0 msgr2=0x7fc0000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.986+0000 7fc01920d640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc0000a3da0 0x7fc0000a6260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0099e0 tx=0x7fc00c0135e0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.986+0000 7fc01920d640 1 -- 172.21.15.33:0/3267363380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc014100970 msgr2=0x7fc0141991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:03.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.986+0000 7fc01920d640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc014100970 0x7fc0141991d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc004000c00 tx=0x7fc004004e40 comp rx=0 tx=0).stop 2024-04-02T13:41:03.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.986+0000 7fc01920d640 1 -- 172.21.15.33:0/3267363380 shutdown_connections 2024-04-02T13:41:03.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.986+0000 7fc01920d640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc0000a3da0 0x7fc0000a6260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.986+0000 7fc01920d640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc014106980 0x7fc0141a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.986+0000 7fc01920d640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc014101330 0x7fc014199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.986+0000 7fc01920d640 1 --2- 172.21.15.33:0/3267363380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc014100970 0x7fc0141991d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:03.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.986+0000 7fc01920d640 1 -- 172.21.15.33:0/3267363380 >> 172.21.15.33:0/3267363380 conn(0x7fc014077b00 msgr2=0x7fc0141030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:03.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.986+0000 7fc01920d640 1 -- 172.21.15.33:0/3267363380 shutdown_connections 2024-04-02T13:41:03.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:03.986+0000 7fc01920d640 1 -- 172.21.15.33:0/3267363380 wait complete. 2024-04-02T13:41:03.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:03 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3915219729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:41:04.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:04 smithi145 ceph-mon[35646]: pgmap v71: 129 pgs: 129 active+clean; 154 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 276 op/s 2024-04-02T13:41:04.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:04 smithi050 ceph-mon[35524]: pgmap v71: 129 pgs: 129 active+clean; 154 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 276 op/s 2024-04-02T13:41:05.000 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:41:05.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:41:05.000 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:41:05.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:04 smithi033 ceph-mon[32390]: pgmap v71: 129 pgs: 129 active+clean; 154 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 276 op/s 2024-04-02T13:41:05.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.129+0000 7faf870f4640 1 -- 172.21.15.33:0/840527461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf80101370 msgr2=0x7faf8010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.129+0000 7faf870f4640 1 --2- 172.21.15.33:0/840527461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf80101370 0x7faf8010d880 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7faf78009cd0 tx=0x7faf7802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.129+0000 7faf870f4640 1 -- 172.21.15.33:0/840527461 shutdown_connections 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.129+0000 7faf870f4640 1 --2- 172.21.15.33:0/840527461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf80101370 0x7faf8010d880 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.129+0000 7faf870f4640 1 --2- 172.21.15.33:0/840527461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf801009b0 0x7faf80100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.129+0000 7faf870f4640 1 --2- 172.21.15.33:0/840527461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf801069c0 0x7faf80106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.129+0000 7faf870f4640 1 -- 172.21.15.33:0/840527461 >> 172.21.15.33:0/840527461 conn(0x7faf80076d80 msgr2=0x7faf80077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.129+0000 7faf870f4640 1 -- 172.21.15.33:0/840527461 shutdown_connections 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.129+0000 7faf870f4640 1 -- 172.21.15.33:0/840527461 wait complete. 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.129+0000 7faf870f4640 1 Processor -- start 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.129+0000 7faf870f4640 1 -- start start 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf870f4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf801009b0 0x7faf80198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf870f4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf80101370 0x7faf801988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf870f4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf801069c0 0x7faf8019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf870f4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faf80110da0 con 0x7faf80101370 2024-04-02T13:41:05.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf870f4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faf80110c20 con 0x7faf801069c0 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf870f4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf80110f20 con 0x7faf801009b0 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf7ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf80101370 0x7faf801988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf7ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf80101370 0x7faf801988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40516/0 (socket says 172.21.15.33:40516) 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf7ffff640 1 -- 172.21.15.33:0/1648694273 learned_addr learned my addr 172.21.15.33:0/1648694273 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf84e90640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf801009b0 0x7faf80198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf85691640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf801069c0 0x7faf8019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf7ffff640 1 -- 172.21.15.33:0/1648694273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf801009b0 msgr2=0x7faf80198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf7ffff640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf801009b0 0x7faf80198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf7ffff640 1 -- 172.21.15.33:0/1648694273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf801069c0 msgr2=0x7faf8019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf7ffff640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf801069c0 0x7faf8019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf7ffff640 1 -- 172.21.15.33:0/1648694273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf8019d370 con 0x7faf80101370 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf7ffff640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf80101370 0x7faf801988c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00e9f0 tx=0x7faf6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf7dffb640 1 -- 172.21.15.33:0/1648694273 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c00cd80 con 0x7faf80101370 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf7dffb640 1 -- 172.21.15.33:0/1648694273 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7faf6c010900 con 0x7faf80101370 2024-04-02T13:41:05.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf7dffb640 1 -- 172.21.15.33:0/1648694273 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c0189e0 con 0x7faf80101370 2024-04-02T13:41:05.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.130+0000 7faf870f4640 1 -- 172.21.15.33:0/1648694273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf8019d660 con 0x7faf80101370 2024-04-02T13:41:05.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.131+0000 7faf870f4640 1 -- 172.21.15.33:0/1648694273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf801a4ef0 con 0x7faf80101370 2024-04-02T13:41:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.133+0000 7faf7dffb640 1 -- 172.21.15.33:0/1648694273 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faf6c010ab0 con 0x7faf80101370 2024-04-02T13:41:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.133+0000 7faf89db0740 1 Processor -- start 2024-04-02T13:41:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.133+0000 7faf89db0740 1 -- start start 2024-04-02T13:41:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf89db0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 0x5575a0ac36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf7dffb640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7faf700a3d20 0x7faf700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf89db0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0ac3be0 0x5575a0acc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf89db0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0acc7c0 0x5575a0acebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf89db0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5575a0a27770 con 0x5575a0ac32c0 2024-04-02T13:41:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf89db0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5575a09cf960 con 0x5575a0acc7c0 2024-04-02T13:41:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf89db0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5575a0a238c0 con 0x5575a0ac3be0 2024-04-02T13:41:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf85691640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 0x5575a0ac36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf84e90640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7faf700a3d20 0x7faf700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf85691640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 0x5575a0ac36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40520/0 (socket says 172.21.15.33:40520) 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf85691640 1 -- 172.21.15.33:0/701357569 learned_addr learned my addr 172.21.15.33:0/701357569 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf7dffb640 1 -- 172.21.15.33:0/1648694273 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faf6c0d0e80 con 0x7faf80101370 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf85691640 1 --2- 172.21.15.33:0/701357569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0acc7c0 0x5575a0acebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf7ffff640 1 --2- 172.21.15.33:0/701357569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0ac3be0 0x5575a0acc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf85691640 1 -- 172.21.15.33:0/701357569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0ac3be0 msgr2=0x5575a0acc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf85691640 1 --2- 172.21.15.33:0/701357569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0ac3be0 0x5575a0acc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf85691640 1 -- 172.21.15.33:0/701357569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0acc7c0 msgr2=0x5575a0acebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf85691640 1 --2- 172.21.15.33:0/701357569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0acc7c0 0x5575a0acebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf85691640 1 -- 172.21.15.33:0/701357569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575a08037e0 con 0x5575a0ac32c0 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf84e90640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7faf700a3d20 0x7faf700a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf74005fd0 tx=0x7faf7400b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf85691640 1 --2- 172.21.15.33:0/701357569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 0x5575a0ac36a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7faf78032570 tx=0x7faf78009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8f0068988ceddc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf6affd640 1 -- 172.21.15.33:0/701357569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7804a070 con 0x5575a0ac32c0 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.134+0000 7faf6affd640 1 -- 172.21.15.33:0/701357569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7faf7803a040 con 0x5575a0ac32c0 2024-04-02T13:41:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.135+0000 7faf6affd640 1 -- 172.21.15.33:0/701357569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78039040 con 0x5575a0ac32c0 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.135+0000 7faf89db0740 1 -- 172.21.15.33:0/701357569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 msgr2=0x5575a0ac36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.135+0000 7faf89db0740 1 --2- 172.21.15.33:0/701357569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 0x5575a0ac36a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7faf78032570 tx=0x7faf78009820 comp rx=0 tx=0).stop 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.135+0000 7faf89db0740 1 -- 172.21.15.33:0/701357569 shutdown_connections 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.135+0000 7faf89db0740 1 --2- 172.21.15.33:0/701357569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0acc7c0 0x5575a0acebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.135+0000 7faf89db0740 1 --2- 172.21.15.33:0/701357569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0ac3be0 0x5575a0acc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.135+0000 7faf89db0740 1 --2- 172.21.15.33:0/701357569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 0x5575a0ac36a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.135+0000 7faf89db0740 1 -- 172.21.15.33:0/701357569 >> 172.21.15.33:0/701357569 conn(0x5575a0abd9f0 msgr2=0x5575a0abddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.135+0000 7faf89db0740 1 -- 172.21.15.33:0/701357569 shutdown_connections 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.135+0000 7faf89db0740 1 -- 172.21.15.33:0/701357569 wait complete. 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.135+0000 7faf89db0740 1 -- start start 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf89db0740 1 Processor -- start 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf89db0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac32c0 0x5575a0ac9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf89db0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 0x5575a0ac9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf89db0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 0x5575a0aca290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf89db0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5575a0acf550 con 0x5575a0ac3be0 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf89db0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5575a09e2d20 con 0x5575a0ac32c0 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf89db0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5575a0acf6d0 con 0x5575a0acc7c0 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf89db0740 10 client.0 _ll_register_callbacks cb 0x5575a09afa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf7ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 0x5575a0ac9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf7ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 0x5575a0ac9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40524/0 (socket says 172.21.15.33:40524) 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf7ffff640 1 -- 172.21.15.33:0/686541373 learned_addr learned my addr 172.21.15.33:0/686541373 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf84e90640 1 --2- 172.21.15.33:0/686541373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac32c0 0x5575a0ac9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf85691640 1 --2- 172.21.15.33:0/686541373 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 0x5575a0aca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf7ffff640 1 -- 172.21.15.33:0/686541373 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 msgr2=0x5575a0aca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf7ffff640 1 --2- 172.21.15.33:0/686541373 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 0x5575a0aca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf7ffff640 1 -- 172.21.15.33:0/686541373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac32c0 msgr2=0x5575a0ac9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf7ffff640 1 --2- 172.21.15.33:0/686541373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac32c0 0x5575a0ac9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.136+0000 7faf7ffff640 1 -- 172.21.15.33:0/686541373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575a0863f70 con 0x5575a0ac3be0 2024-04-02T13:41:05.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf7ffff640 1 --2- 172.21.15.33:0/686541373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 0x5575a0ac9d50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7faf6c09c860 tx=0x7faf6c09c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e4ce9f81e00f15 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf5ffff640 1 -- 172.21.15.33:0/686541373 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c09b120 con 0x5575a0ac3be0 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf5ffff640 1 -- 172.21.15.33:0/686541373 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faf6c09a5b0 con 0x5575a0ac3be0 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf89db0740 1 -- 172.21.15.33:0/686541373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7803a040 con 0x5575a0ac3be0 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf5ffff640 1 -- 172.21.15.33:0/686541373 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faf6c09aa40 con 0x5575a0ac3be0 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf89db0740 10 client.16164 fetch_fsmap learned FSMap version 27 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf89db0740 1 -- 172.21.15.33:0/686541373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575a06fb170 con 0x5575a0ac3be0 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf5ffff640 1 -- 172.21.15.33:0/686541373 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7faf7803a040 con 0x5575a0ac3be0 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf5ffff640 1 -- 172.21.15.33:0/686541373 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7faf6c018b80 con 0x5575a0ac3be0 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf89db0740 10 client.16164 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf89db0740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf89db0740 20 client.16164 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf89db0740 1 --2- 172.21.15.33:0/686541373 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5575a0adcb40 0x5575a0adefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf89db0740 4 client.16164 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf89db0740 1 -- 172.21.15.33:0/686541373 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575a0a34cb0 con 0x5575a0adcb40 2024-04-02T13:41:05.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.137+0000 7faf84e90640 1 --2- 172.21.15.33:0/686541373 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5575a0adcb40 0x5575a0adefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.138+0000 7faf84e90640 1 --2- 172.21.15.33:0/686541373 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5575a0adcb40 0x5575a0adefb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfeda1da54870ed8 server_cookie=b91c18c062cc7436 in_seq=0 out_seq=0 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.138+0000 7faf5ffff640 10 client.16164 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf5ffff640 1 -- 172.21.15.33:0/686541373 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5575a0a34cb0 con 0x5575a0adcb40 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf5ffff640 10 client.16164 handle_command_reply: tid=0 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf89db0740 1 client.16164 shutdown 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf89db0740 1 -- 172.21.15.33:0/686541373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 msgr2=0x5575a0ac9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf89db0740 1 --2- 172.21.15.33:0/686541373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 0x5575a0ac9d50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7faf6c09c860 tx=0x7faf6c09c6a0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf89db0740 1 -- 172.21.15.33:0/686541373 shutdown_connections 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf89db0740 1 --2- 172.21.15.33:0/686541373 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5575a0adcb40 0x5575a0adefb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf89db0740 1 --2- 172.21.15.33:0/686541373 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 0x5575a0aca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf5ffff640 0 client.16164 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf89db0740 1 --2- 172.21.15.33:0/686541373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 0x5575a0ac9d50 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf89db0740 1 --2- 172.21.15.33:0/686541373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac32c0 0x5575a0ac9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf89db0740 1 -- 172.21.15.33:0/686541373 >> 172.21.15.33:0/686541373 conn(0x5575a0abd9f0 msgr2=0x5575a0abf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:05.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf89db0740 1 -- 172.21.15.33:0/686541373 shutdown_connections 2024-04-02T13:41:05.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.139+0000 7faf89db0740 1 -- 172.21.15.33:0/686541373 wait complete. 2024-04-02T13:41:05.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.140+0000 7faf89db0740 20 client.16164 trim_cache size 0 max 16384 2024-04-02T13:41:05.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.157+0000 7faf89db0740 1 Processor -- start 2024-04-02T13:41:05.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.157+0000 7faf89db0740 1 -- start start 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.157+0000 7faf89db0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 0x5575a0acbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.157+0000 7faf89db0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac3be0 0x5575a0ad8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.157+0000 7faf89db0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 0x5575a0ad4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.157+0000 7faf89db0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faf801b1800 con 0x5575a0ac32c0 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.157+0000 7faf89db0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faf801af0a0 con 0x5575a0ac3be0 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.157+0000 7faf89db0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf801aced0 con 0x5575a0acc7c0 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.157+0000 7faf84e90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 0x5575a0acbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.157+0000 7faf84e90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 0x5575a0acbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40530/0 (socket says 172.21.15.33:40530) 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.157+0000 7faf84e90640 1 -- 172.21.15.33:0/3366951301 learned_addr learned my addr 172.21.15.33:0/3366951301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.157+0000 7faf7ffff640 1 --2- 172.21.15.33:0/3366951301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac3be0 0x5575a0ad8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.157+0000 7faf85691640 1 --2- 172.21.15.33:0/3366951301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 0x5575a0ad4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.158+0000 7faf84e90640 1 -- 172.21.15.33:0/3366951301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 msgr2=0x5575a0ad4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.158+0000 7faf84e90640 1 --2- 172.21.15.33:0/3366951301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 0x5575a0ad4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.158+0000 7faf84e90640 1 -- 172.21.15.33:0/3366951301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac3be0 msgr2=0x5575a0ad8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.158+0000 7faf84e90640 1 --2- 172.21.15.33:0/3366951301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac3be0 0x5575a0ad8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.158+0000 7faf84e90640 1 -- 172.21.15.33:0/3366951301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575a0ac3a10 con 0x5575a0ac32c0 2024-04-02T13:41:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.158+0000 7faf84e90640 1 --2- 172.21.15.33:0/3366951301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 0x5575a0acbf40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7faf74005f00 tx=0x7faf74013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc3c2dff352c922 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.158+0000 7faf6a7fc640 1 -- 172.21.15.33:0/3366951301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7402d070 con 0x5575a0ac32c0 2024-04-02T13:41:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.158+0000 7faf6a7fc640 1 -- 172.21.15.33:0/3366951301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7faf74020ce0 con 0x5575a0ac32c0 2024-04-02T13:41:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.158+0000 7faf6a7fc640 1 -- 172.21.15.33:0/3366951301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74027830 con 0x5575a0ac32c0 2024-04-02T13:41:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.158+0000 7faf89db0740 1 -- 172.21.15.33:0/3366951301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 msgr2=0x5575a0acbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.158+0000 7faf89db0740 1 --2- 172.21.15.33:0/3366951301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 0x5575a0acbf40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7faf74005f00 tx=0x7faf74013000 comp rx=0 tx=0).stop 2024-04-02T13:41:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 -- 172.21.15.33:0/3366951301 shutdown_connections 2024-04-02T13:41:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 --2- 172.21.15.33:0/3366951301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 0x5575a0ad4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 --2- 172.21.15.33:0/3366951301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac3be0 0x5575a0ad8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 --2- 172.21.15.33:0/3366951301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac32c0 0x5575a0acbf40 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 -- 172.21.15.33:0/3366951301 >> 172.21.15.33:0/3366951301 conn(0x5575a0abd9f0 msgr2=0x5575a0abf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:05.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 -- 172.21.15.33:0/3366951301 shutdown_connections 2024-04-02T13:41:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 -- 172.21.15.33:0/3366951301 wait complete. 2024-04-02T13:41:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 -- start start 2024-04-02T13:41:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 Processor -- start 2024-04-02T13:41:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac32c0 0x5575a0ac7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 0x5575a0ac5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 0x5575a0ac5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faf801ad560 con 0x5575a0ac3be0 2024-04-02T13:41:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faf801ae9d0 con 0x5575a0ac32c0 2024-04-02T13:41:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf801adc30 con 0x5575a0acc7c0 2024-04-02T13:41:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf89db0740 10 client.0 _ll_register_callbacks cb 0x7faf6c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf7ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 0x5575a0ac5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf7ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 0x5575a0ac5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40540/0 (socket says 172.21.15.33:40540) 2024-04-02T13:41:05.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf7ffff640 1 -- 172.21.15.33:0/871252101 learned_addr learned my addr 172.21.15.33:0/871252101 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf84e90640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac32c0 0x5575a0ac7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.159+0000 7faf85691640 1 --2- 172.21.15.33:0/871252101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 0x5575a0ac5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf7ffff640 1 -- 172.21.15.33:0/871252101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 msgr2=0x5575a0ac5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf7ffff640 1 --2- 172.21.15.33:0/871252101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 0x5575a0ac5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf7ffff640 1 -- 172.21.15.33:0/871252101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac32c0 msgr2=0x5575a0ac7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf7ffff640 1 --2- 172.21.15.33:0/871252101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac32c0 0x5575a0ac7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf7ffff640 1 -- 172.21.15.33:0/871252101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575a0ac6580 con 0x5575a0ac3be0 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf84e90640 1 --2- 172.21.15.33:0/871252101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac32c0 0x5575a0ac7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf7ffff640 1 --2- 172.21.15.33:0/871252101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 0x5575a0ac5a80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00cf60 tx=0x7faf6c0ce710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00f911bfbc459f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf5ffff640 1 -- 172.21.15.33:0/871252101 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c09aca0 con 0x5575a0ac3be0 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf5ffff640 1 -- 172.21.15.33:0/871252101 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faf6c010430 con 0x5575a0ac3be0 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf89db0740 1 -- 172.21.15.33:0/871252101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf74020ce0 con 0x5575a0ac3be0 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf5ffff640 1 -- 172.21.15.33:0/871252101 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faf6c09b120 con 0x5575a0ac3be0 2024-04-02T13:41:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf89db0740 10 client.16170 fetch_fsmap learned FSMap version 27 2024-04-02T13:41:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf89db0740 1 -- 172.21.15.33:0/871252101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575a0b47f50 con 0x5575a0ac3be0 2024-04-02T13:41:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf5ffff640 1 -- 172.21.15.33:0/871252101 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7faf74020ce0 con 0x5575a0ac3be0 2024-04-02T13:41:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf5ffff640 1 -- 172.21.15.33:0/871252101 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7faf6c0988b0 con 0x5575a0ac3be0 2024-04-02T13:41:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf89db0740 10 client.16170 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:41:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf89db0740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf89db0740 20 client.16170 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf89db0740 1 --2- 172.21.15.33:0/871252101 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5575a0b485c0 0x5575a0b4a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.160+0000 7faf89db0740 4 client.16170 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:41:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.161+0000 7faf89db0740 1 -- 172.21.15.33:0/871252101 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5575a0a2f000 con 0x5575a0b485c0 2024-04-02T13:41:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.161+0000 7faf84e90640 1 --2- 172.21.15.33:0/871252101 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5575a0b485c0 0x5575a0b4a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.161+0000 7faf84e90640 1 --2- 172.21.15.33:0/871252101 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5575a0b485c0 0x5575a0b4a9e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4769006c5f92b310 server_cookie=28345e946b8a00f7 in_seq=0 out_seq=0 2024-04-02T13:41:05.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.161+0000 7faf5ffff640 10 client.16170 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf5ffff640 1 -- 172.21.15.33:0/871252101 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5575a0a2f000 con 0x5575a0b485c0 2024-04-02T13:41:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf5ffff640 10 client.16170 handle_command_reply: tid=0 2024-04-02T13:41:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf89db0740 1 client.16170 shutdown 2024-04-02T13:41:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf89db0740 1 -- 172.21.15.33:0/871252101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 msgr2=0x5575a0ac5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf89db0740 1 --2- 172.21.15.33:0/871252101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 0x5575a0ac5a80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00cf60 tx=0x7faf6c0ce710 comp rx=0 tx=0).stop 2024-04-02T13:41:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf89db0740 1 -- 172.21.15.33:0/871252101 shutdown_connections 2024-04-02T13:41:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf89db0740 1 --2- 172.21.15.33:0/871252101 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5575a0b485c0 0x5575a0b4a9e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf89db0740 1 --2- 172.21.15.33:0/871252101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5575a0acc7c0 0x5575a0ac5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf89db0740 1 --2- 172.21.15.33:0/871252101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575a0ac3be0 0x5575a0ac5a80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf5ffff640 0 client.16170 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf89db0740 1 --2- 172.21.15.33:0/871252101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a0ac32c0 0x5575a0ac7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf89db0740 1 -- 172.21.15.33:0/871252101 >> 172.21.15.33:0/871252101 conn(0x5575a0abd9f0 msgr2=0x5575a0ac97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:05.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf89db0740 1 -- 172.21.15.33:0/871252101 shutdown_connections 2024-04-02T13:41:05.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf89db0740 1 -- 172.21.15.33:0/871252101 wait complete. 2024-04-02T13:41:05.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.163+0000 7faf89db0740 20 client.16170 trim_cache size 0 max 16384 2024-04-02T13:41:05.172 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:41:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:41:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "76cd3e6e-c6cf-430c-9bae-8730a23e5c72", 2024-04-02T13:41:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:41:05.172 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:41:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.164+0000 7faf870f4640 1 -- 172.21.15.33:0/1648694273 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7faf700a3d20 msgr2=0x7faf700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.164+0000 7faf870f4640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7faf700a3d20 0x7faf700a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf74005fd0 tx=0x7faf7400b390 comp rx=0 tx=0).stop 2024-04-02T13:41:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.164+0000 7faf870f4640 1 -- 172.21.15.33:0/1648694273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf80101370 msgr2=0x7faf801988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.164+0000 7faf870f4640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf80101370 0x7faf801988c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00e9f0 tx=0x7faf6c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.164+0000 7faf870f4640 1 -- 172.21.15.33:0/1648694273 shutdown_connections 2024-04-02T13:41:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.164+0000 7faf870f4640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7faf700a3d20 0x7faf700a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.164+0000 7faf870f4640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf801069c0 0x7faf8019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.164+0000 7faf870f4640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf80101370 0x7faf801988c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.164+0000 7faf870f4640 1 --2- 172.21.15.33:0/1648694273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf801009b0 0x7faf80198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.164+0000 7faf870f4640 1 -- 172.21.15.33:0/1648694273 >> 172.21.15.33:0/1648694273 conn(0x7faf80076d80 msgr2=0x7faf8006a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:05.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.164+0000 7faf870f4640 1 -- 172.21.15.33:0/1648694273 shutdown_connections 2024-04-02T13:41:05.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.164+0000 7faf870f4640 1 -- 172.21.15.33:0/1648694273 wait complete. 2024-04-02T13:41:05.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:41:05.177 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:41:05.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.282+0000 7fa33902a640 1 -- 172.21.15.33:0/3309227806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa334107b50 msgr2=0x7fa334107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.282+0000 7fa33902a640 1 --2- 172.21.15.33:0/3309227806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa334107b50 0x7fa334107f30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa324009930 tx=0x7fa32402bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:05.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.282+0000 7fa33902a640 1 -- 172.21.15.33:0/3309227806 shutdown_connections 2024-04-02T13:41:05.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.282+0000 7fa33902a640 1 --2- 172.21.15.33:0/3309227806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3340696c0 0x7fa33410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.282+0000 7fa33902a640 1 --2- 172.21.15.33:0/3309227806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa334068d00 0x7fa334069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.282+0000 7fa33902a640 1 --2- 172.21.15.33:0/3309227806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa334107b50 0x7fa334107f30 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.282+0000 7fa33902a640 1 -- 172.21.15.33:0/3309227806 >> 172.21.15.33:0/3309227806 conn(0x7fa334077b00 msgr2=0x7fa334077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:05.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.282+0000 7fa33902a640 1 -- 172.21.15.33:0/3309227806 shutdown_connections 2024-04-02T13:41:05.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.282+0000 7fa33902a640 1 -- 172.21.15.33:0/3309227806 wait complete. 2024-04-02T13:41:05.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.283+0000 7fa33902a640 1 Processor -- start 2024-04-02T13:41:05.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.283+0000 7fa33902a640 1 -- start start 2024-04-02T13:41:05.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa33902a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa334068d00 0x7fa334198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa33902a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3340696c0 0x7fa334198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa33902a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa334107b50 0x7fa33419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa33902a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa334111e90 con 0x7fa334068d00 2024-04-02T13:41:05.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa33902a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa334111d10 con 0x7fa3340696c0 2024-04-02T13:41:05.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa33902a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa334112010 con 0x7fa334107b50 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa332d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa334068d00 0x7fa334198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa332d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa334068d00 0x7fa334198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40560/0 (socket says 172.21.15.33:40560) 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa332d9d640 1 -- 172.21.15.33:0/1174612623 learned_addr learned my addr 172.21.15.33:0/1174612623 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa33359e640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa334107b50 0x7fa33419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa33259c640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3340696c0 0x7fa334198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa332d9d640 1 -- 172.21.15.33:0/1174612623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa334107b50 msgr2=0x7fa33419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa332d9d640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa334107b50 0x7fa33419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa332d9d640 1 -- 172.21.15.33:0/1174612623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3340696c0 msgr2=0x7fa334198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa332d9d640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3340696c0 0x7fa334198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa332d9d640 1 -- 172.21.15.33:0/1174612623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33419d340 con 0x7fa334068d00 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa33359e640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa334107b50 0x7fa33419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa332d9d640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa334068d00 0x7fa334198350 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa324000c00 tx=0x7fa3240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.284+0000 7fa31bfff640 1 -- 172.21.15.33:0/1174612623 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324043020 con 0x7fa334068d00 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.285+0000 7fa33902a640 1 -- 172.21.15.33:0/1174612623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33419d5d0 con 0x7fa334068d00 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.285+0000 7fa33902a640 1 -- 172.21.15.33:0/1174612623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3341a4ec0 con 0x7fa334068d00 2024-04-02T13:41:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.285+0000 7fa31bfff640 1 -- 172.21.15.33:0/1174612623 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa324005470 con 0x7fa334068d00 2024-04-02T13:41:05.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.285+0000 7fa31bfff640 1 -- 172.21.15.33:0/1174612623 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324003a90 con 0x7fa334068d00 2024-04-02T13:41:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.287+0000 7fa31bfff640 1 -- 172.21.15.33:0/1174612623 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa324046020 con 0x7fa334068d00 2024-04-02T13:41:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.287+0000 7fa33bc14740 1 Processor -- start 2024-04-02T13:41:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.287+0000 7fa33bc14740 1 -- start start 2024-04-02T13:41:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.287+0000 7fa33bc14740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8688d0 0x55977e868cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.287+0000 7fa33bc14740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8691f0 0x55977e871890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.287+0000 7fa33bc14740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e8741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.287+0000 7fa33bc14740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55977e7cc830 con 0x55977e8691f0 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.287+0000 7fa33bc14740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55977e774820 con 0x55977e8688d0 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.287+0000 7fa33bc14740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55977e7c87e0 con 0x55977e871dd0 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.287+0000 7fa31bfff640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa31c0a3d20 0x7fa31c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa33259c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e8741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa33359e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8688d0 0x55977e868cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa33259c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e8741c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:38854/0 (socket says 172.21.15.33:38854) 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa33359e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8688d0 0x55977e868cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:49832/0 (socket says 172.21.15.33:49832) 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa33259c640 1 -- 172.21.15.33:0/789054320 learned_addr learned my addr 172.21.15.33:0/789054320 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa332d9d640 1 --2- 172.21.15.33:0/789054320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8691f0 0x55977e871890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa31bfff640 1 -- 172.21.15.33:0/1174612623 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa3240f24a0 con 0x7fa334068d00 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa33359e640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa31c0a3d20 0x7fa31c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa33259c640 1 -- 172.21.15.33:0/789054320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8688d0 msgr2=0x55977e868cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa33259c640 1 --2- 172.21.15.33:0/789054320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8688d0 0x55977e868cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa33259c640 1 -- 172.21.15.33:0/789054320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8691f0 msgr2=0x55977e871890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa33259c640 1 --2- 172.21.15.33:0/789054320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8691f0 0x55977e871890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa33259c640 1 -- 172.21.15.33:0/789054320 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55977e5aa7d0 con 0x55977e871dd0 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa332d9d640 1 --2- 172.21.15.33:0/789054320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8691f0 0x55977e871890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa33259c640 1 --2- 172.21.15.33:0/789054320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e8741c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa328009c90 tx=0x7fa328011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54bdc6461b527fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa33359e640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa31c0a3d20 0x7fa31c0a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0049d0 tx=0x7fa32c004ab0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.288+0000 7fa3197fa640 1 -- 172.21.15.33:0/789054320 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32801e070 con 0x55977e871dd0 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.289+0000 7fa3197fa640 1 -- 172.21.15.33:0/789054320 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa328019b00 con 0x55977e871dd0 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.289+0000 7fa3197fa640 1 -- 172.21.15.33:0/789054320 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32800ea20 con 0x55977e871dd0 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.289+0000 7fa33bc14740 1 -- 172.21.15.33:0/789054320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 msgr2=0x55977e8741c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.289+0000 7fa33bc14740 1 --2- 172.21.15.33:0/789054320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e8741c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa328009c90 tx=0x7fa328011810 comp rx=0 tx=0).stop 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.289+0000 7fa33bc14740 1 -- 172.21.15.33:0/789054320 shutdown_connections 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.289+0000 7fa33bc14740 1 --2- 172.21.15.33:0/789054320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e8741c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.289+0000 7fa33bc14740 1 --2- 172.21.15.33:0/789054320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8691f0 0x55977e871890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.289+0000 7fa33bc14740 1 --2- 172.21.15.33:0/789054320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8688d0 0x55977e868cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.289+0000 7fa33bc14740 1 -- 172.21.15.33:0/789054320 >> 172.21.15.33:0/789054320 conn(0x55977e863000 msgr2=0x55977e8633e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.289+0000 7fa33bc14740 1 -- 172.21.15.33:0/789054320 shutdown_connections 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.289+0000 7fa33bc14740 1 -- 172.21.15.33:0/789054320 wait complete. 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa33bc14740 1 -- start start 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa33bc14740 1 Processor -- start 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa33bc14740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8688d0 0x55977e8715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa33bc14740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8691f0 0x55977e86e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa33bc14740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e86ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa33bc14740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55977e874bc0 con 0x55977e8688d0 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa33bc14740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55977e787d30 con 0x55977e8691f0 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa33bc14740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55977e874d40 con 0x55977e871dd0 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa33bc14740 10 client.0 _ll_register_callbacks cb 0x55977e787720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa332d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8688d0 0x55977e8715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa332d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8688d0 0x55977e8715c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40584/0 (socket says 172.21.15.33:40584) 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa33259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8691f0 0x55977e86e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa332d9d640 1 -- 172.21.15.33:0/245904793 learned_addr learned my addr 172.21.15.33:0/245904793 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa33359e640 1 --2- 172.21.15.33:0/245904793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e86ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa332d9d640 1 -- 172.21.15.33:0/245904793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 msgr2=0x55977e86ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa332d9d640 1 --2- 172.21.15.33:0/245904793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e86ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa332d9d640 1 -- 172.21.15.33:0/245904793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8691f0 msgr2=0x55977e86e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa332d9d640 1 --2- 172.21.15.33:0/245904793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8691f0 0x55977e86e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.290+0000 7fa332d9d640 1 -- 172.21.15.33:0/245904793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55977e4e03f0 con 0x55977e8688d0 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.291+0000 7fa332d9d640 1 --2- 172.21.15.33:0/245904793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8688d0 0x55977e8715c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa3240babb0 tx=0x7fa324039c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94afb443640b0a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.291+0000 7fa316ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.291+0000 7fa316ff5640 1 -- 172.21.15.33:0/245904793 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3240bdb40 con 0x55977e8688d0 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.291+0000 7fa316ff5640 1 -- 172.21.15.33:0/245904793 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa3240efb00 con 0x55977e8688d0 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.291+0000 7fa33bc14740 1 -- 172.21.15.33:0/245904793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa328019b00 con 0x55977e8688d0 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.291+0000 7fa316ff5640 1 -- 172.21.15.33:0/245904793 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa3240044e0 con 0x55977e8688d0 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.291+0000 7fa33bc14740 10 client.16182 fetch_fsmap learned FSMap version 27 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.291+0000 7fa33bc14740 1 -- 172.21.15.33:0/245904793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55977e4a00f0 con 0x55977e8688d0 2024-04-02T13:41:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.291+0000 7fa316ff5640 1 -- 172.21.15.33:0/245904793 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa328019b00 con 0x55977e8688d0 2024-04-02T13:41:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.291+0000 7fa316ff5640 1 -- 172.21.15.33:0/245904793 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fa324039c80 con 0x55977e8688d0 2024-04-02T13:41:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.292+0000 7fa33bc14740 10 client.16182 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:41:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.292+0000 7fa33bc14740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.292+0000 7fa33bc14740 20 client.16182 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.292+0000 7fa33bc14740 1 --2- 172.21.15.33:0/245904793 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55977e881c70 0x55977e884130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.292+0000 7fa33bc14740 4 client.16182 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.292+0000 7fa33bc14740 1 -- 172.21.15.33:0/245904793 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55977e7d9d70 con 0x55977e881c70 2024-04-02T13:41:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.292+0000 7fa33259c640 1 --2- 172.21.15.33:0/245904793 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55977e881c70 0x55977e884130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.292+0000 7fa33259c640 1 --2- 172.21.15.33:0/245904793 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55977e881c70 0x55977e884130 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e75a455591968aa9 server_cookie=8f3016ceff94e341 in_seq=0 out_seq=0 2024-04-02T13:41:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.292+0000 7fa316ff5640 10 client.16182 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.295+0000 7fa316ff5640 1 -- 172.21.15.33:0/245904793 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55977e7d9d70 con 0x55977e881c70 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.295+0000 7fa316ff5640 10 client.16182 handle_command_reply: tid=0 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.295+0000 7fa33bc14740 1 client.16182 shutdown 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.296+0000 7fa33bc14740 1 -- 172.21.15.33:0/245904793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8688d0 msgr2=0x55977e8715c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.296+0000 7fa33bc14740 1 --2- 172.21.15.33:0/245904793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8688d0 0x55977e8715c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa3240babb0 tx=0x7fa324039c10 comp rx=0 tx=0).stop 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.296+0000 7fa33bc14740 1 -- 172.21.15.33:0/245904793 shutdown_connections 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.296+0000 7fa33bc14740 1 --2- 172.21.15.33:0/245904793 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55977e881c70 0x55977e884130 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.296+0000 7fa33bc14740 1 --2- 172.21.15.33:0/245904793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e86ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.296+0000 7fa33bc14740 1 --2- 172.21.15.33:0/245904793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8691f0 0x55977e86e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.296+0000 7fa316ff5640 0 client.16182 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.296+0000 7fa33bc14740 1 --2- 172.21.15.33:0/245904793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8688d0 0x55977e8715c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.296+0000 7fa33bc14740 1 -- 172.21.15.33:0/245904793 >> 172.21.15.33:0/245904793 conn(0x55977e863000 msgr2=0x55977e864d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.296+0000 7fa33bc14740 1 -- 172.21.15.33:0/245904793 shutdown_connections 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.296+0000 7fa33bc14740 1 -- 172.21.15.33:0/245904793 wait complete. 2024-04-02T13:41:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.296+0000 7fa33bc14740 20 client.16182 trim_cache size 0 max 16384 2024-04-02T13:41:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.315+0000 7fa33bc14740 1 Processor -- start 2024-04-02T13:41:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.315+0000 7fa33bc14740 1 -- start start 2024-04-02T13:41:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33bc14740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8688d0 0x55977e86d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33bc14740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8691f0 0x55977e86a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33bc14740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e86e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33bc14740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341b17e0 con 0x55977e8691f0 2024-04-02T13:41:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33bc14740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341af080 con 0x55977e8688d0 2024-04-02T13:41:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33bc14740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341aceb0 con 0x55977e871dd0 2024-04-02T13:41:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8691f0 0x55977e86a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8691f0 0x55977e86a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40598/0 (socket says 172.21.15.33:40598) 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33259c640 1 -- 172.21.15.33:0/3887881750 learned_addr learned my addr 172.21.15.33:0/3887881750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa332d9d640 1 --2- 172.21.15.33:0/3887881750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8688d0 0x55977e86d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33359e640 1 --2- 172.21.15.33:0/3887881750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e86e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33259c640 1 -- 172.21.15.33:0/3887881750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 msgr2=0x55977e86e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33259c640 1 --2- 172.21.15.33:0/3887881750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e86e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33259c640 1 -- 172.21.15.33:0/3887881750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8688d0 msgr2=0x55977e86d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33259c640 1 --2- 172.21.15.33:0/3887881750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8688d0 0x55977e86d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33259c640 1 -- 172.21.15.33:0/3887881750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55977e862e30 con 0x55977e8691f0 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.316+0000 7fa33259c640 1 --2- 172.21.15.33:0/3887881750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8691f0 0x55977e86a0c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa3280058c0 tx=0x7fa328009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45fcfe78b194bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.317+0000 7fa318ff9640 1 -- 172.21.15.33:0/3887881750 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32802d020 con 0x55977e8691f0 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.317+0000 7fa318ff9640 1 -- 172.21.15.33:0/3887881750 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa328029170 con 0x55977e8691f0 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.317+0000 7fa318ff9640 1 -- 172.21.15.33:0/3887881750 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328029e60 con 0x55977e8691f0 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.317+0000 7fa33bc14740 1 -- 172.21.15.33:0/3887881750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8691f0 msgr2=0x55977e86a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.317+0000 7fa33bc14740 1 --2- 172.21.15.33:0/3887881750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8691f0 0x55977e86a0c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa3280058c0 tx=0x7fa328009f00 comp rx=0 tx=0).stop 2024-04-02T13:41:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.317+0000 7fa33bc14740 1 -- 172.21.15.33:0/3887881750 shutdown_connections 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.317+0000 7fa33bc14740 1 --2- 172.21.15.33:0/3887881750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e871dd0 0x55977e86e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.317+0000 7fa33bc14740 1 --2- 172.21.15.33:0/3887881750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e8691f0 0x55977e86a0c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.317+0000 7fa33bc14740 1 --2- 172.21.15.33:0/3887881750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8688d0 0x55977e86d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.317+0000 7fa33bc14740 1 -- 172.21.15.33:0/3887881750 >> 172.21.15.33:0/3887881750 conn(0x55977e863000 msgr2=0x55977e8619b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.317+0000 7fa33bc14740 1 -- 172.21.15.33:0/3887881750 shutdown_connections 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.317+0000 7fa33bc14740 1 -- 172.21.15.33:0/3887881750 wait complete. 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33bc14740 1 -- start start 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33bc14740 1 Processor -- start 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33bc14740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e8688d0 0x55977e87a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33bc14740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8691f0 0x55977e875890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33bc14740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e871dd0 0x55977e875f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33bc14740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341ad540 con 0x55977e871dd0 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33bc14740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341ae9b0 con 0x55977e8691f0 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33bc14740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341adc10 con 0x55977e8688d0 2024-04-02T13:41:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33bc14740 10 client.0 _ll_register_callbacks cb 0x7fa3240ba480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e871dd0 0x55977e875f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e871dd0 0x55977e875f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40614/0 (socket says 172.21.15.33:40614) 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8691f0 0x55977e875890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33359e640 1 -- 172.21.15.33:0/162569610 learned_addr learned my addr 172.21.15.33:0/162569610 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa332d9d640 1 --2- 172.21.15.33:0/162569610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e8688d0 0x55977e87a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33359e640 1 -- 172.21.15.33:0/162569610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e8688d0 msgr2=0x55977e87a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33359e640 1 --2- 172.21.15.33:0/162569610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e8688d0 0x55977e87a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33359e640 1 -- 172.21.15.33:0/162569610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8691f0 msgr2=0x55977e875890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33359e640 1 --2- 172.21.15.33:0/162569610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8691f0 0x55977e875890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.318+0000 7fa33359e640 1 -- 172.21.15.33:0/162569610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55977e8eb680 con 0x55977e871dd0 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa33359e640 1 --2- 172.21.15.33:0/162569610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e871dd0 0x55977e875f80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa32c005fd0 tx=0x7fa32c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b82983227b07fe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa316ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa316ff5640 1 -- 172.21.15.33:0/162569610 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c004b20 con 0x55977e871dd0 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa33bc14740 1 -- 172.21.15.33:0/162569610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa328029170 con 0x55977e871dd0 2024-04-02T13:41:05.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa316ff5640 1 -- 172.21.15.33:0/162569610 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa32c004cc0 con 0x55977e871dd0 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa316ff5640 1 -- 172.21.15.33:0/162569610 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa32c037050 con 0x55977e871dd0 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa33bc14740 10 client.16188 fetch_fsmap learned FSMap version 27 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa33bc14740 1 -- 172.21.15.33:0/162569610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55977e8ec880 con 0x55977e871dd0 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa316ff5640 1 -- 172.21.15.33:0/162569610 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa328029170 con 0x55977e871dd0 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa316ff5640 1 -- 172.21.15.33:0/162569610 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fa32c01f040 con 0x55977e871dd0 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa33bc14740 10 client.16188 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa33bc14740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa33bc14740 20 client.16188 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa33bc14740 1 --2- 172.21.15.33:0/162569610 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55977e8ed060 0x55977e8ef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa33bc14740 4 client.16188 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.319+0000 7fa33bc14740 1 -- 172.21.15.33:0/162569610 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55977e7d40c0 con 0x55977e8ed060 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.320+0000 7fa332d9d640 1 --2- 172.21.15.33:0/162569610 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55977e8ed060 0x55977e8ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.320+0000 7fa332d9d640 1 --2- 172.21.15.33:0/162569610 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55977e8ed060 0x55977e8ef480 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297cb49ece79ff2b server_cookie=1bc3cfa839dd9e57 in_seq=0 out_seq=0 2024-04-02T13:41:05.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.320+0000 7fa316ff5640 10 client.16188 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa316ff5640 1 -- 172.21.15.33:0/162569610 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55977e7d40c0 con 0x55977e8ed060 2024-04-02T13:41:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa316ff5640 10 client.16188 handle_command_reply: tid=0 2024-04-02T13:41:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa33bc14740 1 client.16188 shutdown 2024-04-02T13:41:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa33bc14740 1 -- 172.21.15.33:0/162569610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e871dd0 msgr2=0x55977e875f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa33bc14740 1 --2- 172.21.15.33:0/162569610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e871dd0 0x55977e875f80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa32c005fd0 tx=0x7fa32c004a40 comp rx=0 tx=0).stop 2024-04-02T13:41:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa33bc14740 1 -- 172.21.15.33:0/162569610 shutdown_connections 2024-04-02T13:41:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa33bc14740 1 --2- 172.21.15.33:0/162569610 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55977e8ed060 0x55977e8ef480 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa33bc14740 1 --2- 172.21.15.33:0/162569610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977e871dd0 0x55977e875f80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa33bc14740 1 --2- 172.21.15.33:0/162569610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55977e8691f0 0x55977e875890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa316ff5640 0 client.16188 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa33bc14740 1 --2- 172.21.15.33:0/162569610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55977e8688d0 0x55977e87a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa33bc14740 1 -- 172.21.15.33:0/162569610 >> 172.21.15.33:0/162569610 conn(0x55977e863000 msgr2=0x55977e862240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:05.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa33bc14740 1 -- 172.21.15.33:0/162569610 shutdown_connections 2024-04-02T13:41:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa33bc14740 1 -- 172.21.15.33:0/162569610 wait complete. 2024-04-02T13:41:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.323+0000 7fa33bc14740 20 client.16188 trim_cache size 0 max 16384 2024-04-02T13:41:05.330 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:41:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "scrub active (1172 inodes in the stack)", 2024-04-02T13:41:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:41:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "76cd3e6e-c6cf-430c-9bae-8730a23e5c72": { 2024-04-02T13:41:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:41:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "76cd3e6e-c6cf-430c-9bae-8730a23e5c72", 2024-04-02T13:41:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:41:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:41:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:41:05.331 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:41:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.324+0000 7fa33902a640 1 -- 172.21.15.33:0/1174612623 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa31c0a3d20 msgr2=0x7fa31c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.324+0000 7fa33902a640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa31c0a3d20 0x7fa31c0a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0049d0 tx=0x7fa32c004ab0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.324+0000 7fa33902a640 1 -- 172.21.15.33:0/1174612623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa334068d00 msgr2=0x7fa334198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.324+0000 7fa33902a640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa334068d00 0x7fa334198350 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa324000c00 tx=0x7fa3240042f0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.324+0000 7fa33902a640 1 -- 172.21.15.33:0/1174612623 shutdown_connections 2024-04-02T13:41:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.324+0000 7fa33902a640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa31c0a3d20 0x7fa31c0a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.324+0000 7fa33902a640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa334107b50 0x7fa33419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.324+0000 7fa33902a640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3340696c0 0x7fa334198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.324+0000 7fa33902a640 1 --2- 172.21.15.33:0/1174612623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa334068d00 0x7fa334198350 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.324+0000 7fa33902a640 1 -- 172.21.15.33:0/1174612623 >> 172.21.15.33:0/1174612623 conn(0x7fa334077b00 msgr2=0x7fa3340fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.324+0000 7fa33902a640 1 -- 172.21.15.33:0/1174612623 shutdown_connections 2024-04-02T13:41:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:05.324+0000 7fa33902a640 1 -- 172.21.15.33:0/1174612623 wait complete. 2024-04-02T13:41:05.338 INFO:tasks.cephfs.filesystem:scrub status for tag:76cd3e6e-c6cf-430c-9bae-8730a23e5c72 - {'path': '/', 'tag': '76cd3e6e-c6cf-430c-9bae-8730a23e5c72', 'options': 'recursive,force'} 2024-04-02T13:41:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: pgmap v72: 129 pgs: 129 active+clean; 154 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 1.2 MiB/s wr, 244 op/s 2024-04-02T13:41:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:41:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: pgmap v72: 129 pgs: 129 active+clean; 154 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 1.2 MiB/s wr, 244 op/s 2024-04-02T13:41:06.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:41:06.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:06.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:06.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: pgmap v72: 129 pgs: 129 active+clean; 154 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 1.2 MiB/s wr, 244 op/s 2024-04-02T13:41:07.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:41:07.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.009 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.009 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.009 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.009 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:07.009 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:07 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:07.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:07 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:07 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:07.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:07 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:07 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:07 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:07.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:07 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:07 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:07.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:07 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:07.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:07 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:08.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:07 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:08.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:07 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:08.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:07 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:08.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:07 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:08.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:07 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:08.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:08 smithi145 ceph-mon[35646]: pgmap v73: 129 pgs: 129 active+clean; 154 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 1.2 MiB/s wr, 244 op/s 2024-04-02T13:41:08.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:08 smithi050 ceph-mon[35524]: pgmap v73: 129 pgs: 129 active+clean; 154 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 1.2 MiB/s wr, 244 op/s 2024-04-02T13:41:09.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:08 smithi033 ceph-mon[32390]: pgmap v73: 129 pgs: 129 active+clean; 154 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 1.2 MiB/s wr, 244 op/s 2024-04-02T13:41:10.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:10 smithi145 ceph-mon[35646]: pgmap v74: 129 pgs: 129 active+clean; 161 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 376 op/s 2024-04-02T13:41:10.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:10 smithi050 ceph-mon[35524]: pgmap v74: 129 pgs: 129 active+clean; 161 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 376 op/s 2024-04-02T13:41:11.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:10 smithi033 ceph-mon[32390]: pgmap v74: 129 pgs: 129 active+clean; 161 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 376 op/s 2024-04-02T13:41:12.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:12 smithi050 ceph-mon[35524]: pgmap v75: 129 pgs: 129 active+clean; 161 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.4 MiB/s wr, 269 op/s 2024-04-02T13:41:12.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:12 smithi145 ceph-mon[35646]: pgmap v75: 129 pgs: 129 active+clean; 161 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.4 MiB/s wr, 269 op/s 2024-04-02T13:41:13.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:12 smithi033 ceph-mon[32390]: pgmap v75: 129 pgs: 129 active+clean; 161 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.4 MiB/s wr, 269 op/s 2024-04-02T13:41:14.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:14 smithi145 ceph-mon[35646]: pgmap v76: 129 pgs: 129 active+clean; 172 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 404 op/s 2024-04-02T13:41:14.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:14 smithi050 ceph-mon[35524]: pgmap v76: 129 pgs: 129 active+clean; 172 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 404 op/s 2024-04-02T13:41:15.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:14 smithi033 ceph-mon[32390]: pgmap v76: 129 pgs: 129 active+clean; 172 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 404 op/s 2024-04-02T13:41:16.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:16 smithi145 ceph-mon[35646]: pgmap v77: 129 pgs: 129 active+clean; 172 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-04-02T13:41:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:16 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:41:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:16 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:41:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:16 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:41:16.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:16 smithi050 ceph-mon[35524]: pgmap v77: 129 pgs: 129 active+clean; 172 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-04-02T13:41:16.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:16 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:41:16.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:16 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:41:16.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:16 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:41:17.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:16 smithi033 ceph-mon[32390]: pgmap v77: 129 pgs: 129 active+clean; 172 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-04-02T13:41:17.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:16 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:41:17.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:16 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:41:17.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:16 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:41:18.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:18 smithi145 ceph-mon[35646]: pgmap v78: 129 pgs: 129 active+clean; 172 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-04-02T13:41:18.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:18 smithi050 ceph-mon[35524]: pgmap v78: 129 pgs: 129 active+clean; 172 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-04-02T13:41:19.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:18 smithi033 ceph-mon[32390]: pgmap v78: 129 pgs: 129 active+clean; 172 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-04-02T13:41:20.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:20 smithi145 ceph-mon[35646]: pgmap v79: 129 pgs: 129 active+clean; 185 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 386 op/s 2024-04-02T13:41:20.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:20 smithi050 ceph-mon[35524]: pgmap v79: 129 pgs: 129 active+clean; 185 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 386 op/s 2024-04-02T13:41:21.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:20 smithi033 ceph-mon[32390]: pgmap v79: 129 pgs: 129 active+clean; 185 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 386 op/s 2024-04-02T13:41:22.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:22 smithi145 ceph-mon[35646]: pgmap v80: 129 pgs: 129 active+clean; 185 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 254 op/s 2024-04-02T13:41:22.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:22 smithi050 ceph-mon[35524]: pgmap v80: 129 pgs: 129 active+clean; 185 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 254 op/s 2024-04-02T13:41:23.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:22 smithi033 ceph-mon[32390]: pgmap v80: 129 pgs: 129 active+clean; 185 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 254 op/s 2024-04-02T13:41:23.691 INFO:tasks.workunit.client.0.smithi033.stderr:+ sync 2024-04-02T13:41:24.815 INFO:tasks.workunit.client.0.smithi033.stderr:+ tar c linux-4.0.5 2024-04-02T13:41:24.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:24 smithi145 ceph-mon[35646]: pgmap v81: 129 pgs: 129 active+clean; 195 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 385 op/s 2024-04-02T13:41:24.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:24 smithi050 ceph-mon[35524]: pgmap v81: 129 pgs: 129 active+clean; 195 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 385 op/s 2024-04-02T13:41:25.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:24 smithi033 ceph-mon[32390]: pgmap v81: 129 pgs: 129 active+clean; 195 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 385 op/s 2024-04-02T13:41:26.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:26 smithi145 ceph-mon[35646]: pgmap v82: 129 pgs: 129 active+clean; 195 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-04-02T13:41:26.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:26 smithi050 ceph-mon[35524]: pgmap v82: 129 pgs: 129 active+clean; 195 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-04-02T13:41:27.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:26 smithi033 ceph-mon[32390]: pgmap v82: 129 pgs: 129 active+clean; 195 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-04-02T13:41:27.142 INFO:tasks.workunit.client.0.smithi033.stderr:+ rm -rf linux-4.0.5 2024-04-02T13:41:28.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:28 smithi145 ceph-mon[35646]: pgmap v83: 129 pgs: 129 active+clean; 195 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-04-02T13:41:28.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:28 smithi050 ceph-mon[35524]: pgmap v83: 129 pgs: 129 active+clean; 195 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-04-02T13:41:29.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:28 smithi033 ceph-mon[32390]: pgmap v83: 129 pgs: 129 active+clean; 195 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-04-02T13:41:30.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:30 smithi145 ceph-mon[35646]: pgmap v84: 129 pgs: 129 active+clean; 199 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 312 op/s 2024-04-02T13:41:30.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:30 smithi050 ceph-mon[35524]: pgmap v84: 129 pgs: 129 active+clean; 199 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 312 op/s 2024-04-02T13:41:31.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:30 smithi033 ceph-mon[32390]: pgmap v84: 129 pgs: 129 active+clean; 199 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 312 op/s 2024-04-02T13:41:32.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:32 smithi145 ceph-mon[35646]: pgmap v85: 129 pgs: 129 active+clean; 199 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.2 MiB/s wr, 193 op/s 2024-04-02T13:41:32.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:32 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:41:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:32 smithi050 ceph-mon[35524]: pgmap v85: 129 pgs: 129 active+clean; 199 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.2 MiB/s wr, 193 op/s 2024-04-02T13:41:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:32 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:41:33.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:32 smithi033 ceph-mon[32390]: pgmap v85: 129 pgs: 129 active+clean; 199 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.2 MiB/s wr, 193 op/s 2024-04-02T13:41:33.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:32 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:41:34.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:34 smithi145 ceph-mon[35646]: pgmap v86: 129 pgs: 129 active+clean; 207 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-04-02T13:41:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:34 smithi050 ceph-mon[35524]: pgmap v86: 129 pgs: 129 active+clean; 207 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-04-02T13:41:35.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:34 smithi033 ceph-mon[32390]: pgmap v86: 129 pgs: 129 active+clean; 207 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-04-02T13:41:35.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:41:35.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:41:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.448+0000 7f823f63a640 1 -- 172.21.15.33:0/2105593137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82380696c0 msgr2=0x7f823810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.448+0000 7f823f63a640 1 --2- 172.21.15.33:0/2105593137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82380696c0 0x7f823810e960 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8230009cd0 tx=0x7f823002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.448+0000 7f823f63a640 1 -- 172.21.15.33:0/2105593137 shutdown_connections 2024-04-02T13:41:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.448+0000 7f823f63a640 1 --2- 172.21.15.33:0/2105593137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82380696c0 0x7f823810e960 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.448+0000 7f823f63a640 1 --2- 172.21.15.33:0/2105593137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8238068d00 0x7f8238069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.448+0000 7f823f63a640 1 --2- 172.21.15.33:0/2105593137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8238107b50 0x7f8238107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.448+0000 7f823f63a640 1 -- 172.21.15.33:0/2105593137 >> 172.21.15.33:0/2105593137 conn(0x7f8238077b00 msgr2=0x7f8238077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.448+0000 7f823f63a640 1 -- 172.21.15.33:0/2105593137 shutdown_connections 2024-04-02T13:41:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.448+0000 7f823f63a640 1 -- 172.21.15.33:0/2105593137 wait complete. 2024-04-02T13:41:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.449+0000 7f823f63a640 1 Processor -- start 2024-04-02T13:41:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.449+0000 7f823f63a640 1 -- start start 2024-04-02T13:41:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.449+0000 7f823f63a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238068d00 0x7f82381982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.449+0000 7f823f63a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82380696c0 0x7f8238198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.449+0000 7f823f63a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8238107b50 0x7f823819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.449+0000 7f823f63a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8238111e70 con 0x7f8238068d00 2024-04-02T13:41:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.449+0000 7f823f63a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8238111cf0 con 0x7f8238107b50 2024-04-02T13:41:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.449+0000 7f823f63a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8238111ff0 con 0x7f82380696c0 2024-04-02T13:41:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.449+0000 7f823d3d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238068d00 0x7f82381982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.449+0000 7f823d3d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238068d00 0x7f82381982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35258/0 (socket says 172.21.15.33:35258) 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.449+0000 7f823d3d6640 1 -- 172.21.15.33:0/2969706172 learned_addr learned my addr 172.21.15.33:0/2969706172 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.449+0000 7f823dbd7640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8238107b50 0x7f823819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.449+0000 7f823cbd5640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82380696c0 0x7f8238198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.450+0000 7f823d3d6640 1 -- 172.21.15.33:0/2969706172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82380696c0 msgr2=0x7f8238198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.450+0000 7f823d3d6640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82380696c0 0x7f8238198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.450+0000 7f823d3d6640 1 -- 172.21.15.33:0/2969706172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8238107b50 msgr2=0x7f823819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.450+0000 7f823d3d6640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8238107b50 0x7f823819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.450+0000 7f823d3d6640 1 -- 172.21.15.33:0/2969706172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823819d2c0 con 0x7f8238068d00 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.450+0000 7f823cbd5640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82380696c0 0x7f8238198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.450+0000 7f823d3d6640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238068d00 0x7f82381982d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f823400e9f0 tx=0x7f823400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.450+0000 7f822e7fc640 1 -- 172.21.15.33:0/2969706172 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823400cd80 con 0x7f8238068d00 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.450+0000 7f823f63a640 1 -- 172.21.15.33:0/2969706172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823819d5b0 con 0x7f8238068d00 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.450+0000 7f822e7fc640 1 -- 172.21.15.33:0/2969706172 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8234010900 con 0x7f8238068d00 2024-04-02T13:41:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.450+0000 7f822e7fc640 1 -- 172.21.15.33:0/2969706172 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82340189e0 con 0x7f8238068d00 2024-04-02T13:41:35.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.450+0000 7f823f63a640 1 -- 172.21.15.33:0/2969706172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82381a4e40 con 0x7f8238068d00 2024-04-02T13:41:35.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.452+0000 7f822e7fc640 1 -- 172.21.15.33:0/2969706172 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8234010ab0 con 0x7f8238068d00 2024-04-02T13:41:35.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f824221c740 1 Processor -- start 2024-04-02T13:41:35.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f824221c740 1 -- start start 2024-04-02T13:41:35.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f822e7fc640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f82240a3d20 0x7f82240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f824221c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecd6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f824221c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f824221c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ece21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f824221c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fc6ec3a830 con 0x55fc6ecd71f0 2024-04-02T13:41:35.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f824221c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fc6ebe2820 con 0x55fc6ecd68d0 2024-04-02T13:41:35.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f824221c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fc6ec367e0 con 0x55fc6ecdfdd0 2024-04-02T13:41:35.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f823cbd5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f823d3d6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecd6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f823dbd7640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f82240a3d20 0x7f82240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f823cbd5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35262/0 (socket says 172.21.15.33:35262) 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f823cbd5640 1 -- 172.21.15.33:0/2376112015 learned_addr learned my addr 172.21.15.33:0/2376112015 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.453+0000 7f822e7fc640 1 -- 172.21.15.33:0/2969706172 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f82340d0f30 con 0x7f8238068d00 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.454+0000 7f823dbd7640 1 --2- 172.21.15.33:0/2376112015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ece21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.454+0000 7f823cbd5640 1 -- 172.21.15.33:0/2376112015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 msgr2=0x55fc6ece21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.454+0000 7f823cbd5640 1 --2- 172.21.15.33:0/2376112015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ece21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.454+0000 7f823cbd5640 1 -- 172.21.15.33:0/2376112015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd68d0 msgr2=0x55fc6ecd6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.454+0000 7f823cbd5640 1 --2- 172.21.15.33:0/2376112015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecd6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.454+0000 7f823cbd5640 1 -- 172.21.15.33:0/2376112015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc6ea187d0 con 0x55fc6ecd71f0 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.454+0000 7f823d3d6640 1 --2- 172.21.15.33:0/2376112015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecd6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.454+0000 7f823cbd5640 1 --2- 172.21.15.33:0/2376112015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdf890 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f822800bd30 tx=0x7f822800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bc414287d361f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.454+0000 7f823dbd7640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f82240a3d20 0x7f82240a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8230005cc0 tx=0x7f8230032c40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.454+0000 7f82237fe640 1 -- 172.21.15.33:0/2376112015 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228015070 con 0x55fc6ecd71f0 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.454+0000 7f82237fe640 1 -- 172.21.15.33:0/2376112015 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f822800be20 con 0x55fc6ecd71f0 2024-04-02T13:41:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.454+0000 7f82237fe640 1 -- 172.21.15.33:0/2376112015 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228011930 con 0x55fc6ecd71f0 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.455+0000 7f824221c740 1 -- 172.21.15.33:0/2376112015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd71f0 msgr2=0x55fc6ecdf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.455+0000 7f824221c740 1 --2- 172.21.15.33:0/2376112015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdf890 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f822800bd30 tx=0x7f822800f730 comp rx=0 tx=0).stop 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.455+0000 7f824221c740 1 -- 172.21.15.33:0/2376112015 shutdown_connections 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.455+0000 7f824221c740 1 --2- 172.21.15.33:0/2376112015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ece21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.455+0000 7f824221c740 1 --2- 172.21.15.33:0/2376112015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdf890 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.455+0000 7f824221c740 1 --2- 172.21.15.33:0/2376112015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecd6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.455+0000 7f824221c740 1 -- 172.21.15.33:0/2376112015 >> 172.21.15.33:0/2376112015 conn(0x55fc6ecd1000 msgr2=0x55fc6ecd13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.455+0000 7f824221c740 1 -- 172.21.15.33:0/2376112015 shutdown_connections 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.455+0000 7f824221c740 1 -- 172.21.15.33:0/2376112015 wait complete. 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.455+0000 7f824221c740 1 -- start start 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.455+0000 7f824221c740 1 Processor -- start 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f824221c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecdf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f824221c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f824221c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ecdd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f824221c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fc6ece2bc0 con 0x55fc6ecd71f0 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f824221c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fc6ebf5d30 con 0x55fc6ecd68d0 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f824221c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fc6ece2d40 con 0x55fc6ecdfdd0 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f824221c740 10 client.0 _ll_register_callbacks cb 0x55fc6ebf5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f823cbd5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f823d3d6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecdf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f823d3d6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecdf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:57610/0 (socket says 172.21.15.33:57610) 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f823d3d6640 1 -- 172.21.15.33:0/856306224 learned_addr learned my addr 172.21.15.33:0/856306224 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f823cbd5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35278/0 (socket says 172.21.15.33:35278) 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f823dbd7640 1 --2- 172.21.15.33:0/856306224 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ecdd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f823d3d6640 1 -- 172.21.15.33:0/856306224 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 msgr2=0x55fc6ecdd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f823d3d6640 1 --2- 172.21.15.33:0/856306224 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ecdd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f823d3d6640 1 -- 172.21.15.33:0/856306224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd71f0 msgr2=0x55fc6ecdcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f823d3d6640 1 --2- 172.21.15.33:0/856306224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f823d3d6640 1 -- 172.21.15.33:0/856306224 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc6e94e3f0 con 0x55fc6ecd68d0 2024-04-02T13:41:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f823d3d6640 1 --2- 172.21.15.33:0/856306224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecdf740 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8234098fc0 tx=0x7f823409c5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270397a29da2a7ca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.456+0000 7f8220ff9640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f8220ff9640 1 -- 172.21.15.33:0/856306224 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234026040 con 0x55fc6ecd68d0 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f8220ff9640 1 -- 172.21.15.33:0/856306224 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f82340b1600 con 0x55fc6ecd68d0 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f824221c740 1 -- 172.21.15.33:0/856306224 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f822800be20 con 0x55fc6ecd68d0 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f8220ff9640 1 -- 172.21.15.33:0/856306224 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f82340b4710 con 0x55fc6ecd68d0 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f824221c740 10 client.25799 fetch_fsmap learned FSMap version 29 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f824221c740 1 -- 172.21.15.33:0/856306224 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc6e90e0f0 con 0x55fc6ecd68d0 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f8220ff9640 1 -- 172.21.15.33:0/856306224 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f822800be20 con 0x55fc6ecd68d0 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f8220ff9640 1 -- 172.21.15.33:0/856306224 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f82340ce590 con 0x55fc6ecd68d0 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f824221c740 10 client.25799 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f824221c740 10 client.25799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f824221c740 20 client.25799 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f824221c740 1 --2- 172.21.15.33:0/856306224 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55fc6ecefd60 0x55fc6ecf2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f824221c740 4 client.25799 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f824221c740 1 -- 172.21.15.33:0/856306224 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc6ec47d70 con 0x55fc6ecefd60 2024-04-02T13:41:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.457+0000 7f823cbd5640 1 --2- 172.21.15.33:0/856306224 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55fc6ecefd60 0x55fc6ecf2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.458+0000 7f8220ff9640 10 client.25799 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.458+0000 7f823cbd5640 1 --2- 172.21.15.33:0/856306224 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55fc6ecefd60 0x55fc6ecf2190 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41edc6c33b728797 server_cookie=ff4a45694d00e4ff in_seq=0 out_seq=0 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.458+0000 7f8220ff9640 1 -- 172.21.15.33:0/856306224 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc6ec47d70 con 0x55fc6ecefd60 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.458+0000 7f8220ff9640 10 client.25799 handle_command_reply: tid=0 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.458+0000 7f824221c740 1 client.25799 shutdown 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.459+0000 7f824221c740 1 -- 172.21.15.33:0/856306224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd68d0 msgr2=0x55fc6ecdf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.459+0000 7f824221c740 1 --2- 172.21.15.33:0/856306224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecdf740 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8234098fc0 tx=0x7f823409c5b0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.459+0000 7f824221c740 1 -- 172.21.15.33:0/856306224 shutdown_connections 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.459+0000 7f824221c740 1 --2- 172.21.15.33:0/856306224 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55fc6ecefd60 0x55fc6ecf2190 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.459+0000 7f824221c740 1 --2- 172.21.15.33:0/856306224 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ecdd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.459+0000 7f824221c740 1 --2- 172.21.15.33:0/856306224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.459+0000 7f8220ff9640 0 client.25799 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.459+0000 7f824221c740 1 --2- 172.21.15.33:0/856306224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecdf740 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.459+0000 7f824221c740 1 -- 172.21.15.33:0/856306224 >> 172.21.15.33:0/856306224 conn(0x55fc6ecd1000 msgr2=0x55fc6ecd2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:35.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.459+0000 7f824221c740 1 -- 172.21.15.33:0/856306224 shutdown_connections 2024-04-02T13:41:35.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.459+0000 7f824221c740 1 -- 172.21.15.33:0/856306224 wait complete. 2024-04-02T13:41:35.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.459+0000 7f824221c740 20 client.25799 trim_cache size 0 max 16384 2024-04-02T13:41:35.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f824221c740 1 Processor -- start 2024-04-02T13:41:35.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f824221c740 1 -- start start 2024-04-02T13:41:35.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f824221c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecd7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f824221c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f824221c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ecdf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f824221c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82381af680 con 0x55fc6ecdfdd0 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f824221c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f82381acec0 con 0x55fc6ecd71f0 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f824221c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f82381aacf0 con 0x55fc6ecd68d0 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f823dbd7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ecdf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f823cbd5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f823dbd7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ecdf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35280/0 (socket says 172.21.15.33:35280) 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f823dbd7640 1 -- 172.21.15.33:0/3805172778 learned_addr learned my addr 172.21.15.33:0/3805172778 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f823cbd5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:57614/0 (socket says 172.21.15.33:57614) 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f823d3d6640 1 --2- 172.21.15.33:0/3805172778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecd7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f823dbd7640 1 -- 172.21.15.33:0/3805172778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecd68d0 msgr2=0x55fc6ecd7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.477+0000 7f823dbd7640 1 --2- 172.21.15.33:0/3805172778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecd7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.478+0000 7f823dbd7640 1 -- 172.21.15.33:0/3805172778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd71f0 msgr2=0x55fc6ecdf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.478+0000 7f823dbd7640 1 --2- 172.21.15.33:0/3805172778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.478+0000 7f823dbd7640 1 -- 172.21.15.33:0/3805172778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc6ecccc50 con 0x55fc6ecdfdd0 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.478+0000 7f823cbd5640 1 --2- 172.21.15.33:0/3805172778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.478+0000 7f823dbd7640 1 --2- 172.21.15.33:0/3805172778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ecdf740 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8230036000 tx=0x7f8230046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5755947d09ba4350 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.478+0000 7f8222ffd640 1 -- 172.21.15.33:0/3805172778 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230039040 con 0x55fc6ecdfdd0 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.478+0000 7f8222ffd640 1 -- 172.21.15.33:0/3805172778 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8230054070 con 0x55fc6ecdfdd0 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.478+0000 7f8222ffd640 1 -- 172.21.15.33:0/3805172778 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823004f480 con 0x55fc6ecdfdd0 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 -- 172.21.15.33:0/3805172778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecdfdd0 msgr2=0x55fc6ecdf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 --2- 172.21.15.33:0/3805172778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ecdf740 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8230036000 tx=0x7f8230046000 comp rx=0 tx=0).stop 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 -- 172.21.15.33:0/3805172778 shutdown_connections 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 --2- 172.21.15.33:0/3805172778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ecdf740 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 --2- 172.21.15.33:0/3805172778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd71f0 0x55fc6ecdf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 --2- 172.21.15.33:0/3805172778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecd68d0 0x55fc6ecd7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 -- 172.21.15.33:0/3805172778 >> 172.21.15.33:0/3805172778 conn(0x55fc6ecd1000 msgr2=0x55fc6ecd7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:35.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 -- 172.21.15.33:0/3805172778 shutdown_connections 2024-04-02T13:41:35.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 -- 172.21.15.33:0/3805172778 wait complete. 2024-04-02T13:41:35.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 -- start start 2024-04-02T13:41:35.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 Processor -- start 2024-04-02T13:41:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd68d0 0x55fc6ece3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd71f0 0x55fc6ece4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ece4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82381ab380 con 0x55fc6ecd68d0 2024-04-02T13:41:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f82381ac7f0 con 0x55fc6ecd71f0 2024-04-02T13:41:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f82381aba50 con 0x55fc6ecdfdd0 2024-04-02T13:41:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.479+0000 7f824221c740 10 client.0 _ll_register_callbacks cb 0x7f823409c790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f823d3d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd68d0 0x55fc6ece3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f823d3d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd68d0 0x55fc6ece3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35288/0 (socket says 172.21.15.33:35288) 2024-04-02T13:41:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f823d3d6640 1 -- 172.21.15.33:0/1324570278 learned_addr learned my addr 172.21.15.33:0/1324570278 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f823cbd5640 1 --2- 172.21.15.33:0/1324570278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd71f0 0x55fc6ece4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f823dbd7640 1 --2- 172.21.15.33:0/1324570278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ece4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f823d3d6640 1 -- 172.21.15.33:0/1324570278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 msgr2=0x55fc6ece4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f823d3d6640 1 --2- 172.21.15.33:0/1324570278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ece4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f823d3d6640 1 -- 172.21.15.33:0/1324570278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd71f0 msgr2=0x55fc6ece4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f823d3d6640 1 --2- 172.21.15.33:0/1324570278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd71f0 0x55fc6ece4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f823d3d6640 1 -- 172.21.15.33:0/1324570278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc6ed55520 con 0x55fc6ecd68d0 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f823d3d6640 1 --2- 172.21.15.33:0/1324570278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd68d0 0x55fc6ece3bc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f82340b5e80 tx=0x7f82340bd2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24f92bbe319b480 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f8220ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f8220ff9640 1 -- 172.21.15.33:0/1324570278 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82340a0f10 con 0x55fc6ecd68d0 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f8220ff9640 1 -- 172.21.15.33:0/1324570278 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8234010430 con 0x55fc6ecd68d0 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f824221c740 1 -- 172.21.15.33:0/1324570278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8230054070 con 0x55fc6ecd68d0 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f8220ff9640 1 -- 172.21.15.33:0/1324570278 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f82340a57d0 con 0x55fc6ecd68d0 2024-04-02T13:41:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f824221c740 10 client.16206 fetch_fsmap learned FSMap version 29 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.480+0000 7f824221c740 1 -- 172.21.15.33:0/1324570278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc6ed567f0 con 0x55fc6ecd68d0 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.481+0000 7f8220ff9640 1 -- 172.21.15.33:0/1324570278 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8230054070 con 0x55fc6ecd68d0 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.481+0000 7f8220ff9640 1 -- 172.21.15.33:0/1324570278 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f823409ba20 con 0x55fc6ecd68d0 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.481+0000 7f824221c740 10 client.16206 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.481+0000 7f824221c740 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.481+0000 7f824221c740 20 client.16206 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.481+0000 7f824221c740 1 --2- 172.21.15.33:0/1324570278 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55fc6ed56f00 0x55fc6ed592f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.481+0000 7f824221c740 4 client.16206 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.481+0000 7f824221c740 1 -- 172.21.15.33:0/1324570278 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fc6ec420c0 con 0x55fc6ed56f00 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.481+0000 7f823cbd5640 1 --2- 172.21.15.33:0/1324570278 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55fc6ed56f00 0x55fc6ed592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.481+0000 7f823cbd5640 1 --2- 172.21.15.33:0/1324570278 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55fc6ed56f00 0x55fc6ed592f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f669a67ecac0ec9b server_cookie=fda7816e09acd90 in_seq=0 out_seq=0 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.481+0000 7f8220ff9640 10 client.16206 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.482+0000 7f8220ff9640 1 -- 172.21.15.33:0/1324570278 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fc6ec420c0 con 0x55fc6ed56f00 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.482+0000 7f8220ff9640 10 client.16206 handle_command_reply: tid=0 2024-04-02T13:41:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.482+0000 7f824221c740 1 client.16206 shutdown 2024-04-02T13:41:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f824221c740 1 -- 172.21.15.33:0/1324570278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd68d0 msgr2=0x55fc6ece3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f824221c740 1 --2- 172.21.15.33:0/1324570278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd68d0 0x55fc6ece3bc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f82340b5e80 tx=0x7f82340bd2c0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f824221c740 1 -- 172.21.15.33:0/1324570278 shutdown_connections 2024-04-02T13:41:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f824221c740 1 --2- 172.21.15.33:0/1324570278 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55fc6ed56f00 0x55fc6ed592f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f824221c740 1 --2- 172.21.15.33:0/1324570278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fc6ecdfdd0 0x55fc6ece4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f824221c740 1 --2- 172.21.15.33:0/1324570278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc6ecd71f0 0x55fc6ece4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f8220ff9640 0 client.16206 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f824221c740 1 --2- 172.21.15.33:0/1324570278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fc6ecd68d0 0x55fc6ece3bc0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f824221c740 1 -- 172.21.15.33:0/1324570278 >> 172.21.15.33:0/1324570278 conn(0x55fc6ecd1000 msgr2=0x55fc6ecd7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f824221c740 1 -- 172.21.15.33:0/1324570278 shutdown_connections 2024-04-02T13:41:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f824221c740 1 -- 172.21.15.33:0/1324570278 wait complete. 2024-04-02T13:41:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f824221c740 20 client.16206 trim_cache size 0 max 16384 2024-04-02T13:41:35.492 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:41:35.492 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:41:35.492 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:41:35.492 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:41:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f823f63a640 1 -- 172.21.15.33:0/2969706172 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f82240a3d20 msgr2=0x7f82240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f823f63a640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f82240a3d20 0x7f82240a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8230005cc0 tx=0x7f8230032c40 comp rx=0 tx=0).stop 2024-04-02T13:41:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f823f63a640 1 -- 172.21.15.33:0/2969706172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238068d00 msgr2=0x7f82381982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f823f63a640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238068d00 0x7f82381982d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f823400e9f0 tx=0x7f823400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f823f63a640 1 -- 172.21.15.33:0/2969706172 shutdown_connections 2024-04-02T13:41:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f823f63a640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f82240a3d20 0x7f82240a61e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f823f63a640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8238107b50 0x7f823819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f823f63a640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82380696c0 0x7f8238198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f823f63a640 1 --2- 172.21.15.33:0/2969706172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8238068d00 0x7f82381982d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.483+0000 7f823f63a640 1 -- 172.21.15.33:0/2969706172 >> 172.21.15.33:0/2969706172 conn(0x7f8238077b00 msgr2=0x7f82380fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.484+0000 7f823f63a640 1 -- 172.21.15.33:0/2969706172 shutdown_connections 2024-04-02T13:41:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.484+0000 7f823f63a640 1 -- 172.21.15.33:0/2969706172 wait complete. 2024-04-02T13:41:35.497 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:41:35.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:41:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.603+0000 7f5fedeb8640 1 -- 172.21.15.33:0/2275354915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe8100d90 msgr2=0x7f5fe810e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.603+0000 7f5fedeb8640 1 --2- 172.21.15.33:0/2275354915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe8100d90 0x7f5fe810e990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009cd0 tx=0x7f5fe002c180 comp rx=0 tx=0).stop 2024-04-02T13:41:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.603+0000 7f5fedeb8640 1 -- 172.21.15.33:0/2275354915 shutdown_connections 2024-04-02T13:41:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.603+0000 7f5fedeb8640 1 --2- 172.21.15.33:0/2275354915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe8100d90 0x7f5fe810e990 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.603+0000 7f5fedeb8640 1 --2- 172.21.15.33:0/2275354915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fe81003d0 0x7f5fe8100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.603+0000 7f5fedeb8640 1 --2- 172.21.15.33:0/2275354915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fe8107d10 0x7f5fe81080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.603+0000 7f5fedeb8640 1 -- 172.21.15.33:0/2275354915 >> 172.21.15.33:0/2275354915 conn(0x7f5fe8075d50 msgr2=0x7f5fe8076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.603+0000 7f5fedeb8640 1 -- 172.21.15.33:0/2275354915 shutdown_connections 2024-04-02T13:41:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.603+0000 7f5fedeb8640 1 -- 172.21.15.33:0/2275354915 wait complete. 2024-04-02T13:41:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.603+0000 7f5fedeb8640 1 Processor -- start 2024-04-02T13:41:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fedeb8640 1 -- start start 2024-04-02T13:41:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fedeb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81003d0 0x7f5fe81015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fedeb8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fe8100d90 0x7f5fe8101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fedeb8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fe8107d10 0x7f5fe8102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fedeb8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe8111f40 con 0x7f5fe81003d0 2024-04-02T13:41:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fedeb8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe8111dc0 con 0x7f5fe8100d90 2024-04-02T13:41:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fedeb8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81120c0 con 0x7f5fe8107d10 2024-04-02T13:41:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fe77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81003d0 0x7f5fe81015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fe77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81003d0 0x7f5fe81015b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35310/0 (socket says 172.21.15.33:35310) 2024-04-02T13:41:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fe77fe640 1 -- 172.21.15.33:0/264130858 learned_addr learned my addr 172.21.15.33:0/264130858 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fe6ffd640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fe8100d90 0x7f5fe8101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fe7fff640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fe8107d10 0x7f5fe8102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fe77fe640 1 -- 172.21.15.33:0/264130858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fe8107d10 msgr2=0x7f5fe8102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fe77fe640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fe8107d10 0x7f5fe8102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fe77fe640 1 -- 172.21.15.33:0/264130858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fe8100d90 msgr2=0x7f5fe8101af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fe77fe640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fe8100d90 0x7f5fe8101af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fe77fe640 1 -- 172.21.15.33:0/264130858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe81a68a0 con 0x7f5fe81003d0 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fe7fff640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fe8107d10 0x7f5fe8102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.604+0000 7f5fe77fe640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81003d0 0x7f5fe81015b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800d770 tx=0x7f5fd800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.605+0000 7f5fe4ff9640 1 -- 172.21.15.33:0/264130858 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8017070 con 0x7f5fe81003d0 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.605+0000 7f5fe4ff9640 1 -- 172.21.15.33:0/264130858 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5fd80054f0 con 0x7f5fe81003d0 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.605+0000 7f5fedeb8640 1 -- 172.21.15.33:0/264130858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe81a6b90 con 0x7f5fe81003d0 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.605+0000 7f5fe4ff9640 1 -- 172.21.15.33:0/264130858 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8004d60 con 0x7f5fe81003d0 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.605+0000 7f5fedeb8640 1 -- 172.21.15.33:0/264130858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe81a7090 con 0x7f5fe81003d0 2024-04-02T13:41:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.606+0000 7f5fedeb8640 1 -- 172.21.15.33:0/264130858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe8068d40 con 0x7f5fe81003d0 2024-04-02T13:41:35.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.607+0000 7f5fe4ff9640 1 -- 172.21.15.33:0/264130858 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5fd8004f00 con 0x7f5fe81003d0 2024-04-02T13:41:35.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.607+0000 7f5fe4ff9640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5fd00a3d20 0x7f5fd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.608+0000 7f5fe6ffd640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5fd00a3d20 0x7f5fd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.608+0000 7f5fe4ff9640 1 -- 172.21.15.33:0/264130858 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5fd80d0560 con 0x7f5fe81003d0 2024-04-02T13:41:35.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.608+0000 7f5fe6ffd640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5fd00a3d20 0x7f5fd00a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0095f0 tx=0x7f5fdc005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:35.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.612+0000 7f5fe4ff9640 1 -- 172.21.15.33:0/264130858 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5fd809c890 con 0x7f5fe81003d0 2024-04-02T13:41:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.848+0000 7f5fedeb8640 1 -- 172.21.15.33:0/264130858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fe8109ac0 con 0x7f5fe81003d0 2024-04-02T13:41:35.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.848+0000 7f5fe4ff9640 1 -- 172.21.15.33:0/264130858 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11880 (secure 0 0 0) 0x7f5fd809c190 con 0x7f5fe81003d0 2024-04-02T13:41:35.851 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:41:35.851 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:41:34.223111+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24512,24551,24473]},"id":1}]} 2024-04-02T13:41:35.852 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-04-02T13:41:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.850+0000 7f5fedeb8640 1 -- 172.21.15.33:0/264130858 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5fd00a3d20 msgr2=0x7f5fd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.850+0000 7f5fedeb8640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5fd00a3d20 0x7f5fd00a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0095f0 tx=0x7f5fdc005f90 comp rx=0 tx=0).stop 2024-04-02T13:41:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.850+0000 7f5fedeb8640 1 -- 172.21.15.33:0/264130858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81003d0 msgr2=0x7f5fe81015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.850+0000 7f5fedeb8640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81003d0 0x7f5fe81015b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800d770 tx=0x7f5fd800dc30 comp rx=0 tx=0).stop 2024-04-02T13:41:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.850+0000 7f5fedeb8640 1 -- 172.21.15.33:0/264130858 shutdown_connections 2024-04-02T13:41:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.850+0000 7f5fedeb8640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5fd00a3d20 0x7f5fd00a61e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.850+0000 7f5fedeb8640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5fe8107d10 0x7f5fe8102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.850+0000 7f5fedeb8640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fe8100d90 0x7f5fe8101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.850+0000 7f5fedeb8640 1 --2- 172.21.15.33:0/264130858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe81003d0 0x7f5fe81015b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.850+0000 7f5fedeb8640 1 -- 172.21.15.33:0/264130858 >> 172.21.15.33:0/264130858 conn(0x7f5fe8075d50 msgr2=0x7f5fe80feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.851+0000 7f5fedeb8640 1 -- 172.21.15.33:0/264130858 shutdown_connections 2024-04-02T13:41:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.851+0000 7f5fedeb8640 1 -- 172.21.15.33:0/264130858 wait complete. 2024-04-02T13:41:35.864 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:41:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.973+0000 7f4c82760640 1 -- 172.21.15.33:0/506102089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c7c100970 msgr2=0x7f4c7c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.973+0000 7f4c82760640 1 --2- 172.21.15.33:0/506102089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c7c100970 0x7f4c7c100df0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009930 tx=0x7f4c7002bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.973+0000 7f4c82760640 1 -- 172.21.15.33:0/506102089 shutdown_connections 2024-04-02T13:41:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.973+0000 7f4c82760640 1 --2- 172.21.15.33:0/506102089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7c101330 0x7f4c7c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.973+0000 7f4c82760640 1 --2- 172.21.15.33:0/506102089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c7c100970 0x7f4c7c100df0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.973+0000 7f4c82760640 1 --2- 172.21.15.33:0/506102089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c7c106980 0x7f4c7c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.973+0000 7f4c82760640 1 -- 172.21.15.33:0/506102089 >> 172.21.15.33:0/506102089 conn(0x7f4c7c077b00 msgr2=0x7f4c7c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.973+0000 7f4c82760640 1 -- 172.21.15.33:0/506102089 shutdown_connections 2024-04-02T13:41:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.973+0000 7f4c82760640 1 -- 172.21.15.33:0/506102089 wait complete. 2024-04-02T13:41:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.974+0000 7f4c82760640 1 Processor -- start 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.974+0000 7f4c82760640 1 -- start start 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.974+0000 7f4c82760640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c7c100970 0x7f4c7c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.974+0000 7f4c82760640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7c101330 0x7f4c7c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.974+0000 7f4c82760640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c7c106980 0x7f4c7c19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.974+0000 7f4c82760640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c110d80 con 0x7f4c7c106980 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.974+0000 7f4c82760640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c110c00 con 0x7f4c7c100970 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.974+0000 7f4c82760640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c110f00 con 0x7f4c7c101330 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.974+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c7c100970 0x7f4c7c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.974+0000 7f4c80cfd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c7c106980 0x7f4c7c19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.974+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c7c100970 0x7f4c7c197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:57672/0 (socket says 172.21.15.33:57672) 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.974+0000 7f4c80cfd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c7c106980 0x7f4c7c19bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35334/0 (socket says 172.21.15.33:35334) 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.974+0000 7f4c7bfff640 1 -- 172.21.15.33:0/572989581 learned_addr learned my addr 172.21.15.33:0/572989581 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c7b7fe640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7c101330 0x7f4c7c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c7bfff640 1 -- 172.21.15.33:0/572989581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7c101330 msgr2=0x7f4c7c197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c7bfff640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7c101330 0x7f4c7c197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c7bfff640 1 -- 172.21.15.33:0/572989581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c7c106980 msgr2=0x7f4c7c19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c7bfff640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c7c106980 0x7f4c7c19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c7bfff640 1 -- 172.21.15.33:0/572989581 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c19c2f0 con 0x7f4c7c100970 2024-04-02T13:41:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c7b7fe640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7c101330 0x7f4c7c197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c80cfd640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c7c106980 0x7f4c7c19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c7bfff640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c7c100970 0x7f4c7c197300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00e9f0 tx=0x7f4c6c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c797fa640 1 -- 172.21.15.33:0/572989581 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00cd80 con 0x7f4c7c100970 2024-04-02T13:41:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c82760640 1 -- 172.21.15.33:0/572989581 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7c19c5e0 con 0x7f4c7c100970 2024-04-02T13:41:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c797fa640 1 -- 172.21.15.33:0/572989581 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4c6c010900 con 0x7f4c7c100970 2024-04-02T13:41:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c797fa640 1 -- 172.21.15.33:0/572989581 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c018980 con 0x7f4c7c100970 2024-04-02T13:41:35.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.975+0000 7f4c82760640 1 -- 172.21.15.33:0/572989581 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c7c1a3e70 con 0x7f4c7c100970 2024-04-02T13:41:35.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:35 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:41:35.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:35 smithi145 ceph-mon[35646]: pgmap v87: 129 pgs: 129 active+clean; 207 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1022 KiB/s wr, 153 op/s 2024-04-02T13:41:35.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.977+0000 7f4c797fa640 1 -- 172.21.15.33:0/572989581 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4c6c010ab0 con 0x7f4c7c100970 2024-04-02T13:41:35.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c85428740 1 Processor -- start 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c85428740 1 -- start start 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c85428740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b17974d0 0x5648b17978b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c85428740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b1797df0 0x5648b17a2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c85428740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17a2eb0 0x5648b17a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c85428740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5648b16fb7b0 con 0x5648b17974d0 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c85428740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5648b169a570 con 0x5648b1797df0 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c85428740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5648b16f7760 con 0x5648b17a2eb0 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c80cfd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b17974d0 0x5648b17978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c80cfd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b17974d0 0x5648b17978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35336/0 (socket says 172.21.15.33:35336) 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b1797df0 0x5648b17a2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c80cfd640 1 -- 172.21.15.33:0/869034446 learned_addr learned my addr 172.21.15.33:0/869034446 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c7b7fe640 1 --2- 172.21.15.33:0/869034446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17a2eb0 0x5648b17a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c797fa640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4c640a3d20 0x7f4c640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c80cfd640 1 -- 172.21.15.33:0/869034446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17a2eb0 msgr2=0x5648b17a52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c80cfd640 1 --2- 172.21.15.33:0/869034446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17a2eb0 0x5648b17a52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c80cfd640 1 -- 172.21.15.33:0/869034446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b1797df0 msgr2=0x5648b17a2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c80cfd640 1 --2- 172.21.15.33:0/869034446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b1797df0 0x5648b17a2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c80cfd640 1 -- 172.21.15.33:0/869034446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648b140f3f0 con 0x5648b17974d0 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c7bfff640 1 --2- 172.21.15.33:0/869034446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b1797df0 0x5648b17a2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c797fa640 1 -- 172.21.15.33:0/572989581 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4c6c0d0ee0 con 0x7f4c7c100970 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.978+0000 7f4c80cfd640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4c640a3d20 0x7f4c640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.979+0000 7f4c80cfd640 1 --2- 172.21.15.33:0/869034446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b17974d0 0x5648b17978b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400a980 tx=0x7f4c7400ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8755e782144ec7f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.979+0000 7f4c627fc640 1 -- 172.21.15.33:0/869034446 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7401a040 con 0x5648b17974d0 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.979+0000 7f4c627fc640 1 -- 172.21.15.33:0/869034446 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4c74016ac0 con 0x5648b17974d0 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.979+0000 7f4c627fc640 1 -- 172.21.15.33:0/869034446 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74020840 con 0x5648b17974d0 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.979+0000 7f4c85428740 1 -- 172.21.15.33:0/869034446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b17974d0 msgr2=0x5648b17978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.979+0000 7f4c85428740 1 --2- 172.21.15.33:0/869034446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b17974d0 0x5648b17978b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400a980 tx=0x7f4c7400ae40 comp rx=0 tx=0).stop 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.979+0000 7f4c80cfd640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4c640a3d20 0x7f4c640a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4c74020000 tx=0x7f4c74021000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.979+0000 7f4c85428740 1 -- 172.21.15.33:0/869034446 shutdown_connections 2024-04-02T13:41:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.979+0000 7f4c85428740 1 --2- 172.21.15.33:0/869034446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17a2eb0 0x5648b17a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.979+0000 7f4c85428740 1 --2- 172.21.15.33:0/869034446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b1797df0 0x5648b17a2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.979+0000 7f4c85428740 1 --2- 172.21.15.33:0/869034446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b17974d0 0x5648b17978b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.979+0000 7f4c85428740 1 -- 172.21.15.33:0/869034446 >> 172.21.15.33:0/869034446 conn(0x5648b1791ed0 msgr2=0x5648b17922b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c85428740 1 -- 172.21.15.33:0/869034446 shutdown_connections 2024-04-02T13:41:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c85428740 1 -- 172.21.15.33:0/869034446 wait complete. 2024-04-02T13:41:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c85428740 1 -- start start 2024-04-02T13:41:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c85428740 1 Processor -- start 2024-04-02T13:41:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c85428740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17974d0 0x5648b17a2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c85428740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c85428740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17a2eb0 0x5648b17a0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c85428740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5648b17a5c70 con 0x5648b1797df0 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c85428740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5648b16b6e80 con 0x5648b17974d0 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c85428740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5648b17a5df0 con 0x5648b17a2eb0 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c85428740 10 client.0 _ll_register_callbacks cb 0x5648b16f2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17974d0 0x5648b17a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35346/0 (socket says 172.21.15.33:35346) 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.980+0000 7f4c7b7fe640 1 -- 172.21.15.33:0/26540723 learned_addr learned my addr 172.21.15.33:0/26540723 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c80cfd640 1 --2- 172.21.15.33:0/26540723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17a2eb0 0x5648b17a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c7bfff640 1 -- 172.21.15.33:0/26540723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17a2eb0 msgr2=0x5648b17a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c7bfff640 1 --2- 172.21.15.33:0/26540723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17a2eb0 0x5648b17a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c7bfff640 1 -- 172.21.15.33:0/26540723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 msgr2=0x5648b179fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c7bfff640 1 --2- 172.21.15.33:0/26540723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c7bfff640 1 -- 172.21.15.33:0/26540723 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648b14d97d0 con 0x5648b17974d0 2024-04-02T13:41:35.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c7b7fe640 1 --2- 172.21.15.33:0/26540723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:35.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c7bfff640 1 --2- 172.21.15.33:0/26540723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17974d0 0x5648b17a2930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0ce2c0 tx=0x7f4c6c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ce2b068d9f8b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:35.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T13:41:35.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c5b7fe640 1 -- 172.21.15.33:0/26540723 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c09c470 con 0x5648b17974d0 2024-04-02T13:41:35.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c5b7fe640 1 -- 172.21.15.33:0/26540723 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4c6c010430 con 0x5648b17974d0 2024-04-02T13:41:35.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c85428740 1 -- 172.21.15.33:0/26540723 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c74016ac0 con 0x5648b17974d0 2024-04-02T13:41:35.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c5b7fe640 1 -- 172.21.15.33:0/26540723 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c6c0ce740 con 0x5648b17974d0 2024-04-02T13:41:35.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c85428740 10 client.25823 fetch_fsmap learned FSMap version 29 2024-04-02T13:41:35.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.981+0000 7f4c85428740 1 -- 172.21.15.33:0/26540723 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648b17b1d90 con 0x5648b17974d0 2024-04-02T13:41:35.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.982+0000 7f4c5b7fe640 1 -- 172.21.15.33:0/26540723 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4c74016ac0 con 0x5648b17974d0 2024-04-02T13:41:35.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.982+0000 7f4c5b7fe640 1 -- 172.21.15.33:0/26540723 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f4c6c018b20 con 0x5648b17974d0 2024-04-02T13:41:35.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.982+0000 7f4c85428740 10 client.25823 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:41:35.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.982+0000 7f4c85428740 10 client.25823 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:41:35.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.982+0000 7f4c85428740 20 client.25823 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:35.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.982+0000 7f4c85428740 1 --2- 172.21.15.33:0/26540723 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5648b17b32a0 0x5648b17b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:35.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.982+0000 7f4c85428740 4 client.25823 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:35.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.982+0000 7f4c85428740 1 -- 172.21.15.33:0/26540723 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648b1708cf0 con 0x5648b17b32a0 2024-04-02T13:41:35.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.982+0000 7f4c7b7fe640 1 --2- 172.21.15.33:0/26540723 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5648b17b32a0 0x5648b17b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:35.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.982+0000 7f4c5b7fe640 10 client.25823 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:41:35.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.982+0000 7f4c7b7fe640 1 --2- 172.21.15.33:0/26540723 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5648b17b32a0 0x5648b17b5740 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e9317bbbbaa81c5 server_cookie=3d383cfff9796418 in_seq=0 out_seq=0 2024-04-02T13:41:35.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c5b7fe640 1 -- 172.21.15.33:0/26540723 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648b1708cf0 con 0x5648b17b32a0 2024-04-02T13:41:35.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c5b7fe640 10 client.25823 handle_command_reply: tid=0 2024-04-02T13:41:35.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c85428740 1 client.25823 shutdown 2024-04-02T13:41:35.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c85428740 1 -- 172.21.15.33:0/26540723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17974d0 msgr2=0x5648b17a2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:35.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c85428740 1 --2- 172.21.15.33:0/26540723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17974d0 0x5648b17a2930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0ce2c0 tx=0x7f4c6c000bc0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c85428740 1 -- 172.21.15.33:0/26540723 shutdown_connections 2024-04-02T13:41:35.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c85428740 1 --2- 172.21.15.33:0/26540723 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5648b17b32a0 0x5648b17b5740 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c85428740 1 --2- 172.21.15.33:0/26540723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17a2eb0 0x5648b17a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c85428740 1 --2- 172.21.15.33:0/26540723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c85428740 1 --2- 172.21.15.33:0/26540723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17974d0 0x5648b17a2930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:35.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c5b7fe640 0 client.25823 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:41:35.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c85428740 1 -- 172.21.15.33:0/26540723 >> 172.21.15.33:0/26540723 conn(0x5648b1791ed0 msgr2=0x5648b1793c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:35.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c85428740 1 -- 172.21.15.33:0/26540723 shutdown_connections 2024-04-02T13:41:35.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c85428740 1 -- 172.21.15.33:0/26540723 wait complete. 2024-04-02T13:41:35.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.983+0000 7f4c85428740 20 client.25823 trim_cache size 0 max 16384 2024-04-02T13:41:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:35 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:41:35.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:35 smithi050 ceph-mon[35524]: pgmap v87: 129 pgs: 129 active+clean; 207 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1022 KiB/s wr, 153 op/s 2024-04-02T13:41:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c85428740 1 Processor -- start 2024-04-02T13:41:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c85428740 1 -- start start 2024-04-02T13:41:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c85428740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17974d0 0x5648b17aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c85428740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c85428740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17a2eb0 0x5648b17a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c85428740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1af740 con 0x5648b1797df0 2024-04-02T13:41:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c85428740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1acf80 con 0x5648b17a2eb0 2024-04-02T13:41:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c85428740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1aadb0 con 0x5648b17974d0 2024-04-02T13:41:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35362/0 (socket says 172.21.15.33:35362) 2024-04-02T13:41:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c7b7fe640 1 -- 172.21.15.33:0/4289922670 learned_addr learned my addr 172.21.15.33:0/4289922670 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c80cfd640 1 --2- 172.21.15.33:0/4289922670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17a2eb0 0x5648b17a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c7bfff640 1 --2- 172.21.15.33:0/4289922670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17974d0 0x5648b17aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c7b7fe640 1 -- 172.21.15.33:0/4289922670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17974d0 msgr2=0x5648b17aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c7b7fe640 1 --2- 172.21.15.33:0/4289922670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17974d0 0x5648b17aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c7b7fe640 1 -- 172.21.15.33:0/4289922670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17a2eb0 msgr2=0x5648b17a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c7b7fe640 1 --2- 172.21.15.33:0/4289922670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17a2eb0 0x5648b17a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.998+0000 7f4c7b7fe640 1 -- 172.21.15.33:0/4289922670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648b178d7a0 con 0x5648b1797df0 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.999+0000 7f4c7b7fe640 1 --2- 172.21.15.33:0/4289922670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179eca0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4c70036040 tx=0x7f4c70009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbebbf49f5111ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.999+0000 7f4c61ffb640 1 -- 172.21.15.33:0/4289922670 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70047bc0 con 0x5648b1797df0 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.999+0000 7f4c61ffb640 1 -- 172.21.15.33:0/4289922670 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4c70047d60 con 0x5648b1797df0 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.999+0000 7f4c61ffb640 1 -- 172.21.15.33:0/4289922670 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7002c830 con 0x5648b1797df0 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.999+0000 7f4c85428740 1 -- 172.21.15.33:0/4289922670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 msgr2=0x5648b179eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.999+0000 7f4c85428740 1 --2- 172.21.15.33:0/4289922670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179eca0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4c70036040 tx=0x7f4c70009260 comp rx=0 tx=0).stop 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.999+0000 7f4c85428740 1 -- 172.21.15.33:0/4289922670 shutdown_connections 2024-04-02T13:41:36.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.999+0000 7f4c85428740 1 --2- 172.21.15.33:0/4289922670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17a2eb0 0x5648b17a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.999+0000 7f4c85428740 1 --2- 172.21.15.33:0/4289922670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179eca0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.999+0000 7f4c85428740 1 --2- 172.21.15.33:0/4289922670 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17974d0 0x5648b17aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:35.999+0000 7f4c85428740 1 -- 172.21.15.33:0/4289922670 >> 172.21.15.33:0/4289922670 conn(0x5648b1791ed0 msgr2=0x5648b179e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c85428740 1 -- 172.21.15.33:0/4289922670 shutdown_connections 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c85428740 1 -- 172.21.15.33:0/4289922670 wait complete. 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c85428740 1 -- start start 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c85428740 1 Processor -- start 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c85428740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17974d0 0x5648b179c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c85428740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c85428740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17a2eb0 0x5648b179cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c85428740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1ab440 con 0x5648b1797df0 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c85428740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1ac8b0 con 0x5648b17a2eb0 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c85428740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1abb10 con 0x5648b17974d0 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c85428740 10 client.0 _ll_register_callbacks cb 0x7f4c6c010da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35376/0 (socket says 172.21.15.33:35376) 2024-04-02T13:41:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c7b7fe640 1 -- 172.21.15.33:0/1982229134 learned_addr learned my addr 172.21.15.33:0/1982229134 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.000+0000 7f4c80cfd640 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17a2eb0 0x5648b179cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c7bfff640 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17974d0 0x5648b179c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c7b7fe640 1 -- 172.21.15.33:0/1982229134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17974d0 msgr2=0x5648b179c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c7b7fe640 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17974d0 0x5648b179c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c7b7fe640 1 -- 172.21.15.33:0/1982229134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17a2eb0 msgr2=0x5648b179cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c7b7fe640 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17a2eb0 0x5648b179cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c7b7fe640 1 -- 172.21.15.33:0/1982229134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648b17aa2f0 con 0x5648b1797df0 2024-04-02T13:41:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c7bfff640 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17974d0 0x5648b179c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:36.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:35 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:41:36.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:35 smithi033 ceph-mon[32390]: pgmap v87: 129 pgs: 129 active+clean; 207 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1022 KiB/s wr, 153 op/s 2024-04-02T13:41:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c7b7fe640 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179c5d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4c7002c170 tx=0x7f4c700478d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e61c3fc2fc61ae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c5b7fe640 1 -- 172.21.15.33:0/1982229134 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70054070 con 0x5648b1797df0 2024-04-02T13:41:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c5b7fe640 1 -- 172.21.15.33:0/1982229134 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4c70047260 con 0x5648b1797df0 2024-04-02T13:41:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c85428740 1 -- 172.21.15.33:0/1982229134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c70047d60 con 0x5648b1797df0 2024-04-02T13:41:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c5b7fe640 1 -- 172.21.15.33:0/1982229134 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c70060050 con 0x5648b1797df0 2024-04-02T13:41:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c85428740 10 client.16230 fetch_fsmap learned FSMap version 29 2024-04-02T13:41:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.001+0000 7f4c85428740 1 -- 172.21.15.33:0/1982229134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648b17ab5b0 con 0x5648b1797df0 2024-04-02T13:41:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c5b7fe640 1 -- 172.21.15.33:0/1982229134 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4c70047d60 con 0x5648b1797df0 2024-04-02T13:41:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c5b7fe640 1 -- 172.21.15.33:0/1982229134 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f4c7002c800 con 0x5648b1797df0 2024-04-02T13:41:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c85428740 10 client.16230 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c85428740 10 client.16230 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c85428740 20 client.16230 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c85428740 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5648b1819a60 0x5648b181be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c85428740 4 client.16230 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c85428740 1 -- 172.21.15.33:0/1982229134 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648b1703040 con 0x5648b1819a60 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c7bfff640 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5648b1819a60 0x5648b181be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c7bfff640 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5648b1819a60 0x5648b181be80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e8e8c6f47130e52 server_cookie=9bf95931aebac9fd in_seq=0 out_seq=0 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c5b7fe640 10 client.16230 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c5b7fe640 1 -- 172.21.15.33:0/1982229134 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648b1703040 con 0x5648b1819a60 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c5b7fe640 10 client.16230 handle_command_reply: tid=0 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.002+0000 7f4c85428740 1 client.16230 shutdown 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c85428740 1 -- 172.21.15.33:0/1982229134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 msgr2=0x5648b179c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c85428740 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179c5d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4c7002c170 tx=0x7f4c700478d0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c85428740 1 -- 172.21.15.33:0/1982229134 shutdown_connections 2024-04-02T13:41:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c85428740 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5648b1819a60 0x5648b181be80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c5b7fe640 0 client.16230 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:41:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c85428740 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5648b17a2eb0 0x5648b179cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c85428740 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b1797df0 0x5648b179c5d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c85428740 1 --2- 172.21.15.33:0/1982229134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5648b17974d0 0x5648b179c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c85428740 1 -- 172.21.15.33:0/1982229134 >> 172.21.15.33:0/1982229134 conn(0x5648b1791ed0 msgr2=0x5648b1790f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c85428740 1 -- 172.21.15.33:0/1982229134 shutdown_connections 2024-04-02T13:41:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c85428740 1 -- 172.21.15.33:0/1982229134 wait complete. 2024-04-02T13:41:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c85428740 20 client.16230 trim_cache size 0 max 16384 2024-04-02T13:41:36.012 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:41:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c82760640 1 -- 172.21.15.33:0/572989581 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4c640a3d20 msgr2=0x7f4c640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c82760640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4c640a3d20 0x7f4c640a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4c74020000 tx=0x7f4c74021000 comp rx=0 tx=0).stop 2024-04-02T13:41:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c82760640 1 -- 172.21.15.33:0/572989581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c7c100970 msgr2=0x7f4c7c197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.003+0000 7f4c82760640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c7c100970 0x7f4c7c197300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00e9f0 tx=0x7f4c6c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.004+0000 7f4c82760640 1 -- 172.21.15.33:0/572989581 shutdown_connections 2024-04-02T13:41:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.004+0000 7f4c82760640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4c640a3d20 0x7f4c640a61e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.004+0000 7f4c82760640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c7c106980 0x7f4c7c19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.004+0000 7f4c82760640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c7c101330 0x7f4c7c197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.004+0000 7f4c82760640 1 --2- 172.21.15.33:0/572989581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4c7c100970 0x7f4c7c197300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.004+0000 7f4c82760640 1 -- 172.21.15.33:0/572989581 >> 172.21.15.33:0/572989581 conn(0x7f4c7c077b00 msgr2=0x7f4c7c1031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.004+0000 7f4c82760640 1 -- 172.21.15.33:0/572989581 shutdown_connections 2024-04-02T13:41:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.004+0000 7f4c82760640 1 -- 172.21.15.33:0/572989581 wait complete. 2024-04-02T13:41:36.017 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:41:36.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.123+0000 7fe2c8a7b640 1 -- 172.21.15.33:0/631089650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c4068d00 msgr2=0x7fe2c4069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.123+0000 7fe2c8a7b640 1 --2- 172.21.15.33:0/631089650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c4068d00 0x7fe2c4069180 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009930 tx=0x7fe2b802bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:36.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.123+0000 7fe2c8a7b640 1 -- 172.21.15.33:0/631089650 shutdown_connections 2024-04-02T13:41:36.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.123+0000 7fe2c8a7b640 1 --2- 172.21.15.33:0/631089650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2c40696c0 0x7fe2c410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.123+0000 7fe2c8a7b640 1 --2- 172.21.15.33:0/631089650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c4068d00 0x7fe2c4069180 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.123+0000 7fe2c8a7b640 1 --2- 172.21.15.33:0/631089650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c4107b50 0x7fe2c4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.123+0000 7fe2c8a7b640 1 -- 172.21.15.33:0/631089650 >> 172.21.15.33:0/631089650 conn(0x7fe2c4077b00 msgr2=0x7fe2c4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.123+0000 7fe2c8a7b640 1 -- 172.21.15.33:0/631089650 shutdown_connections 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.123+0000 7fe2c8a7b640 1 -- 172.21.15.33:0/631089650 wait complete. 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.123+0000 7fe2c8a7b640 1 Processor -- start 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c8a7b640 1 -- start start 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c8a7b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c4068d00 0x7fe2c4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c8a7b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c40696c0 0x7fe2c4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c8a7b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2c4107b50 0x7fe2c419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c8a7b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c4111e70 con 0x7fe2c40696c0 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c8a7b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c4111cf0 con 0x7fe2c4107b50 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c8a7b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c4111ff0 con 0x7fe2c4068d00 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c40696c0 0x7fe2c4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c40696c0 0x7fe2c4198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35380/0 (socket says 172.21.15.33:35380) 2024-04-02T13:41:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c1d9b640 1 -- 172.21.15.33:0/1216754421 learned_addr learned my addr 172.21.15.33:0/1216754421 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c2d9d640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2c4107b50 0x7fe2c419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c259c640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c4068d00 0x7fe2c4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c1d9b640 1 -- 172.21.15.33:0/1216754421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c4068d00 msgr2=0x7fe2c4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c1d9b640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c4068d00 0x7fe2c4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c1d9b640 1 -- 172.21.15.33:0/1216754421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2c4107b50 msgr2=0x7fe2c419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c1d9b640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2c4107b50 0x7fe2c419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c1d9b640 1 -- 172.21.15.33:0/1216754421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c419d400 con 0x7fe2c40696c0 2024-04-02T13:41:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2c1d9b640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c40696c0 0x7fe2c4198950 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009900 tx=0x7fe2b802ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.124+0000 7fe2b37fe640 1 -- 172.21.15.33:0/1216754421 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8043020 con 0x7fe2c40696c0 2024-04-02T13:41:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.125+0000 7fe2c8a7b640 1 -- 172.21.15.33:0/1216754421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c419d690 con 0x7fe2c40696c0 2024-04-02T13:41:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.125+0000 7fe2b37fe640 1 -- 172.21.15.33:0/1216754421 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe2b802cc30 con 0x7fe2c40696c0 2024-04-02T13:41:36.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.125+0000 7fe2b37fe640 1 -- 172.21.15.33:0/1216754421 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8003970 con 0x7fe2c40696c0 2024-04-02T13:41:36.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.125+0000 7fe2c8a7b640 1 -- 172.21.15.33:0/1216754421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c41a4f80 con 0x7fe2c40696c0 2024-04-02T13:41:36.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.127+0000 7fe2b37fe640 1 -- 172.21.15.33:0/1216754421 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe2b8046020 con 0x7fe2c40696c0 2024-04-02T13:41:36.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.127+0000 7fe2cb666740 1 Processor -- start 2024-04-02T13:41:36.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.127+0000 7fe2cb666740 1 -- start start 2024-04-02T13:41:36.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2cb666740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x558955027b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2cb666740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589550280c0 0x558955030760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2b37fe640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe2ac0a3d20 0x7fe2ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2cb666740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558955030ca0 0x558955033090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2cb666740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558954f8b7b0 con 0x5589550277a0 2024-04-02T13:41:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2cb666740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558954f2a570 con 0x5589550280c0 2024-04-02T13:41:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2cb666740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558954f87760 con 0x558955030ca0 2024-04-02T13:41:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x558955027b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589550280c0 0x558955030760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x558955027b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35392/0 (socket says 172.21.15.33:35392) 2024-04-02T13:41:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c2d9d640 1 -- 172.21.15.33:0/3500795915 learned_addr learned my addr 172.21.15.33:0/3500795915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c2d9d640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe2ac0a3d20 0x7fe2ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2b37fe640 1 -- 172.21.15.33:0/1216754421 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe2b80f23e0 con 0x7fe2c40696c0 2024-04-02T13:41:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c1d9b640 1 --2- 172.21.15.33:0/3500795915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558955030ca0 0x558955033090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c259c640 1 -- 172.21.15.33:0/3500795915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558955030ca0 msgr2=0x558955033090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c259c640 1 --2- 172.21.15.33:0/3500795915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558955030ca0 0x558955033090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c259c640 1 -- 172.21.15.33:0/3500795915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 msgr2=0x558955027b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c259c640 1 --2- 172.21.15.33:0/3500795915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x558955027b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c259c640 1 -- 172.21.15.33:0/3500795915 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558954c9f3f0 con 0x5589550280c0 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c2d9d640 1 --2- 172.21.15.33:0/3500795915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x558955027b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c2d9d640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe2ac0a3d20 0x7fe2ac0a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc005c40 tx=0x7fe2bc009500 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.128+0000 7fe2c259c640 1 --2- 172.21.15.33:0/3500795915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589550280c0 0x558955030760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009c00 tx=0x7fe2b4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88c4083c58e36d2c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.129+0000 7fe2b0ff9640 1 -- 172.21.15.33:0/3500795915 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4004050 con 0x5589550280c0 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.129+0000 7fe2b0ff9640 1 -- 172.21.15.33:0/3500795915 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe2b40054f0 con 0x5589550280c0 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.129+0000 7fe2b0ff9640 1 -- 172.21.15.33:0/3500795915 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4004cc0 con 0x5589550280c0 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.129+0000 7fe2cb666740 1 -- 172.21.15.33:0/3500795915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589550280c0 msgr2=0x558955030760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.129+0000 7fe2cb666740 1 --2- 172.21.15.33:0/3500795915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589550280c0 0x558955030760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009c00 tx=0x7fe2b4011810 comp rx=0 tx=0).stop 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.129+0000 7fe2cb666740 1 -- 172.21.15.33:0/3500795915 shutdown_connections 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.129+0000 7fe2cb666740 1 --2- 172.21.15.33:0/3500795915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558955030ca0 0x558955033090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.129+0000 7fe2cb666740 1 --2- 172.21.15.33:0/3500795915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589550280c0 0x558955030760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.129+0000 7fe2cb666740 1 --2- 172.21.15.33:0/3500795915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x558955027b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.129+0000 7fe2cb666740 1 -- 172.21.15.33:0/3500795915 >> 172.21.15.33:0/3500795915 conn(0x558955021ed0 msgr2=0x5589550222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.129+0000 7fe2cb666740 1 -- 172.21.15.33:0/3500795915 shutdown_connections 2024-04-02T13:41:36.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2cb666740 1 -- 172.21.15.33:0/3500795915 wait complete. 2024-04-02T13:41:36.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2cb666740 1 -- start start 2024-04-02T13:41:36.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2cb666740 1 Processor -- start 2024-04-02T13:41:36.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2cb666740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x558955030670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2cb666740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5589550280c0 0x55895502da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2cb666740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558955030ca0 0x55895502e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2cb666740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558955033a90 con 0x5589550277a0 2024-04-02T13:41:36.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2cb666740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558954f46e80 con 0x558955030ca0 2024-04-02T13:41:36.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2cb666740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558955033c10 con 0x5589550280c0 2024-04-02T13:41:36.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2cb666740 10 client.0 _ll_register_callbacks cb 0x558954f82aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:36.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2c259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x558955030670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2c259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x558955030670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35406/0 (socket says 172.21.15.33:35406) 2024-04-02T13:41:36.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2c259c640 1 -- 172.21.15.33:0/1629938423 learned_addr learned my addr 172.21.15.33:0/1629938423 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:36.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2c2d9d640 1 --2- 172.21.15.33:0/1629938423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558955030ca0 0x55895502e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2c1d9b640 1 --2- 172.21.15.33:0/1629938423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5589550280c0 0x55895502da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2c259c640 1 -- 172.21.15.33:0/1629938423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5589550280c0 msgr2=0x55895502da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2c259c640 1 --2- 172.21.15.33:0/1629938423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5589550280c0 0x55895502da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2c259c640 1 -- 172.21.15.33:0/1629938423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558955030ca0 msgr2=0x55895502e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2c259c640 1 --2- 172.21.15.33:0/1629938423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558955030ca0 0x55895502e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.130+0000 7fe2c259c640 1 -- 172.21.15.33:0/1629938423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558954d697d0 con 0x5589550277a0 2024-04-02T13:41:36.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.131+0000 7fe2c259c640 1 --2- 172.21.15.33:0/1629938423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x558955030670 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009710 tx=0x7fe2b4005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499bb95b76deb90d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.131+0000 7fe2a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.131+0000 7fe2a9ffb640 1 -- 172.21.15.33:0/1629938423 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4029070 con 0x5589550277a0 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.131+0000 7fe2a9ffb640 1 -- 172.21.15.33:0/1629938423 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe2b4004250 con 0x5589550277a0 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.131+0000 7fe2cb666740 1 -- 172.21.15.33:0/1629938423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b40054f0 con 0x5589550277a0 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.131+0000 7fe2a9ffb640 1 -- 172.21.15.33:0/1629938423 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2b4035050 con 0x5589550277a0 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.131+0000 7fe2cb666740 10 client.16242 fetch_fsmap learned FSMap version 29 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.131+0000 7fe2cb666740 1 -- 172.21.15.33:0/1629938423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55895503fa60 con 0x5589550277a0 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.131+0000 7fe2a9ffb640 1 -- 172.21.15.33:0/1629938423 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe2b40054f0 con 0x5589550277a0 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.131+0000 7fe2a9ffb640 1 -- 172.21.15.33:0/1629938423 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe2b4003b70 con 0x5589550277a0 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.132+0000 7fe2cb666740 10 client.16242 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.132+0000 7fe2cb666740 10 client.16242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.132+0000 7fe2cb666740 20 client.16242 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.132+0000 7fe2cb666740 1 --2- 172.21.15.33:0/1629938423 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558955040e10 0x558955043200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.132+0000 7fe2cb666740 4 client.16242 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.132+0000 7fe2cb666740 1 -- 172.21.15.33:0/1629938423 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558954f98cf0 con 0x558955040e10 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.132+0000 7fe2c1d9b640 1 --2- 172.21.15.33:0/1629938423 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558955040e10 0x558955043200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.132+0000 7fe2c1d9b640 1 --2- 172.21.15.33:0/1629938423 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558955040e10 0x558955043200 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db8aebe50ce1d0e1 server_cookie=77bfa11d2cd86e1b in_seq=0 out_seq=0 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.132+0000 7fe2a9ffb640 10 client.16242 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2a9ffb640 1 -- 172.21.15.33:0/1629938423 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558954f98cf0 con 0x558955040e10 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2a9ffb640 10 client.16242 handle_command_reply: tid=0 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2cb666740 1 client.16242 shutdown 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2cb666740 1 -- 172.21.15.33:0/1629938423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 msgr2=0x558955030670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2cb666740 1 --2- 172.21.15.33:0/1629938423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x558955030670 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009710 tx=0x7fe2b4005700 comp rx=0 tx=0).stop 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2cb666740 1 -- 172.21.15.33:0/1629938423 shutdown_connections 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2cb666740 1 --2- 172.21.15.33:0/1629938423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558955030ca0 0x55895502e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2cb666740 1 --2- 172.21.15.33:0/1629938423 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558955040e10 0x558955043200 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2cb666740 1 --2- 172.21.15.33:0/1629938423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5589550280c0 0x55895502da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2cb666740 1 --2- 172.21.15.33:0/1629938423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x558955030670 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2cb666740 1 -- 172.21.15.33:0/1629938423 >> 172.21.15.33:0/1629938423 conn(0x558955021ed0 msgr2=0x558955023c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2a9ffb640 0 client.16242 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:41:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2cb666740 1 -- 172.21.15.33:0/1629938423 shutdown_connections 2024-04-02T13:41:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2cb666740 1 -- 172.21.15.33:0/1629938423 wait complete. 2024-04-02T13:41:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.133+0000 7fe2cb666740 20 client.16242 trim_cache size 0 max 16384 2024-04-02T13:41:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2cb666740 1 Processor -- start 2024-04-02T13:41:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2cb666740 1 -- start start 2024-04-02T13:41:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2cb666740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5589550277a0 0x55895503c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2cb666740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550280c0 0x558955030130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2cb666740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558955030ca0 0x558955030670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2cb666740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c41b1870 con 0x5589550280c0 2024-04-02T13:41:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2cb666740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c41af0b0 con 0x558955030ca0 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2cb666740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c41acee0 con 0x5589550277a0 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2c1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550280c0 0x558955030130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2c1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550280c0 0x558955030130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35412/0 (socket says 172.21.15.33:35412) 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2c1d9b640 1 -- 172.21.15.33:0/1239214630 learned_addr learned my addr 172.21.15.33:0/1239214630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2c2d9d640 1 --2- 172.21.15.33:0/1239214630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558955030ca0 0x558955030670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2c1d9b640 1 -- 172.21.15.33:0/1239214630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5589550277a0 msgr2=0x55895503c950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2c1d9b640 1 --2- 172.21.15.33:0/1239214630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5589550277a0 0x55895503c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2c1d9b640 1 -- 172.21.15.33:0/1239214630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558955030ca0 msgr2=0x558955030670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2c1d9b640 1 --2- 172.21.15.33:0/1239214630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558955030ca0 0x558955030670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2c1d9b640 1 -- 172.21.15.33:0/1239214630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895502e610 con 0x5589550280c0 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2c1d9b640 1 --2- 172.21.15.33:0/1239214630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550280c0 0x558955030130 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80be0b0 tx=0x7fe2b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fbb82b5802e927 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2abfff640 1 -- 172.21.15.33:0/1239214630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80be120 con 0x5589550280c0 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2abfff640 1 -- 172.21.15.33:0/1239214630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe2b80c24d0 con 0x5589550280c0 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.147+0000 7fe2abfff640 1 -- 172.21.15.33:0/1239214630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80bc420 con 0x5589550280c0 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.148+0000 7fe2cb666740 1 -- 172.21.15.33:0/1239214630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550280c0 msgr2=0x558955030130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.148+0000 7fe2cb666740 1 --2- 172.21.15.33:0/1239214630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550280c0 0x558955030130 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80be0b0 tx=0x7fe2b8009260 comp rx=0 tx=0).stop 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.148+0000 7fe2cb666740 1 -- 172.21.15.33:0/1239214630 shutdown_connections 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.148+0000 7fe2cb666740 1 --2- 172.21.15.33:0/1239214630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558955030ca0 0x558955030670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.148+0000 7fe2cb666740 1 --2- 172.21.15.33:0/1239214630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550280c0 0x558955030130 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.148+0000 7fe2cb666740 1 --2- 172.21.15.33:0/1239214630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5589550277a0 0x55895503c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.148+0000 7fe2cb666740 1 -- 172.21.15.33:0/1239214630 >> 172.21.15.33:0/1239214630 conn(0x558955021ed0 msgr2=0x558955028ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.148+0000 7fe2cb666740 1 -- 172.21.15.33:0/1239214630 shutdown_connections 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.148+0000 7fe2cb666740 1 -- 172.21.15.33:0/1239214630 wait complete. 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.148+0000 7fe2cb666740 1 -- start start 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.148+0000 7fe2cb666740 1 Processor -- start 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2cb666740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x5589550390c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2cb666740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589550280c0 0x55895502ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2cb666740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558955030ca0 0x55895502b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2cb666740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c41ad570 con 0x5589550277a0 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2cb666740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c41ae9e0 con 0x5589550280c0 2024-04-02T13:41:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2cb666740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c41adc40 con 0x558955030ca0 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2cb666740 10 client.0 _ll_register_callbacks cb 0x7fe2bc011460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2c259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x5589550390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2c259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x5589550390c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35428/0 (socket says 172.21.15.33:35428) 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2c259c640 1 -- 172.21.15.33:0/2081987529 learned_addr learned my addr 172.21.15.33:0/2081987529 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2c1d9b640 1 --2- 172.21.15.33:0/2081987529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589550280c0 0x55895502ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2c259c640 1 -- 172.21.15.33:0/2081987529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558955030ca0 msgr2=0x55895502b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2c259c640 1 --2- 172.21.15.33:0/2081987529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558955030ca0 0x55895502b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2c259c640 1 -- 172.21.15.33:0/2081987529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589550280c0 msgr2=0x55895502ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2c259c640 1 --2- 172.21.15.33:0/2081987529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589550280c0 0x55895502ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2c259c640 1 -- 172.21.15.33:0/2081987529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895502b860 con 0x5589550277a0 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2c259c640 1 --2- 172.21.15.33:0/2081987529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x5589550390c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005330 tx=0x7fe2b4005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd171ad9fbabed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2a9ffb640 1 -- 172.21.15.33:0/2081987529 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4029070 con 0x5589550277a0 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2a9ffb640 1 -- 172.21.15.33:0/2081987529 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe2b4003f90 con 0x5589550277a0 2024-04-02T13:41:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2cb666740 1 -- 172.21.15.33:0/2081987529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b80c24d0 con 0x5589550277a0 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.149+0000 7fe2a9ffb640 1 -- 172.21.15.33:0/2081987529 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2b4042050 con 0x5589550277a0 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.150+0000 7fe2cb666740 10 client.16248 fetch_fsmap learned FSMap version 29 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.150+0000 7fe2cb666740 1 -- 172.21.15.33:0/2081987529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589550ab6e0 con 0x5589550277a0 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.150+0000 7fe2a9ffb640 1 -- 172.21.15.33:0/2081987529 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe2b80c24d0 con 0x5589550277a0 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.150+0000 7fe2a9ffb640 1 -- 172.21.15.33:0/2081987529 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe2b4023890 con 0x5589550277a0 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.150+0000 7fe2cb666740 10 client.16248 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.150+0000 7fe2cb666740 10 client.16248 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.150+0000 7fe2cb666740 20 client.16248 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.150+0000 7fe2cb666740 1 --2- 172.21.15.33:0/2081987529 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x5589550abd70 0x5589550ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.150+0000 7fe2cb666740 4 client.16248 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.150+0000 7fe2cb666740 1 -- 172.21.15.33:0/2081987529 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558954f93040 con 0x5589550abd70 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.150+0000 7fe2c1d9b640 1 --2- 172.21.15.33:0/2081987529 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x5589550abd70 0x5589550ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.150+0000 7fe2c1d9b640 1 --2- 172.21.15.33:0/2081987529 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x5589550abd70 0x5589550ae1b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ec48048f274de80 server_cookie=ca0ca29edee1beb4 in_seq=0 out_seq=0 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.150+0000 7fe2a9ffb640 10 client.16248 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:41:36.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2a9ffb640 1 -- 172.21.15.33:0/2081987529 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558954f93040 con 0x5589550abd70 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2a9ffb640 10 client.16248 handle_command_reply: tid=0 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2cb666740 1 client.16248 shutdown 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2cb666740 1 -- 172.21.15.33:0/2081987529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 msgr2=0x5589550390c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2cb666740 1 --2- 172.21.15.33:0/2081987529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x5589550390c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005330 tx=0x7fe2b4005360 comp rx=0 tx=0).stop 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2cb666740 1 -- 172.21.15.33:0/2081987529 shutdown_connections 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2cb666740 1 --2- 172.21.15.33:0/2081987529 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x5589550abd70 0x5589550ae1b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2cb666740 1 --2- 172.21.15.33:0/2081987529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558955030ca0 0x55895502b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2a9ffb640 0 client.16248 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2cb666740 1 --2- 172.21.15.33:0/2081987529 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589550280c0 0x55895502ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2cb666740 1 --2- 172.21.15.33:0/2081987529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589550277a0 0x5589550390c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2cb666740 1 -- 172.21.15.33:0/2081987529 >> 172.21.15.33:0/2081987529 conn(0x558955021ed0 msgr2=0x55895502c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2cb666740 1 -- 172.21.15.33:0/2081987529 shutdown_connections 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2cb666740 1 -- 172.21.15.33:0/2081987529 wait complete. 2024-04-02T13:41:36.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2cb666740 20 client.16248 trim_cache size 0 max 16384 2024-04-02T13:41:36.160 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:41:36.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2c8a7b640 1 -- 172.21.15.33:0/1216754421 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe2ac0a3d20 msgr2=0x7fe2ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2c8a7b640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe2ac0a3d20 0x7fe2ac0a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc005c40 tx=0x7fe2bc009500 comp rx=0 tx=0).stop 2024-04-02T13:41:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2c8a7b640 1 -- 172.21.15.33:0/1216754421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c40696c0 msgr2=0x7fe2c4198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.151+0000 7fe2c8a7b640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c40696c0 0x7fe2c4198950 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009900 tx=0x7fe2b802ca40 comp rx=0 tx=0).stop 2024-04-02T13:41:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.152+0000 7fe2c8a7b640 1 -- 172.21.15.33:0/1216754421 shutdown_connections 2024-04-02T13:41:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.152+0000 7fe2c8a7b640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe2ac0a3d20 0x7fe2ac0a61e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.152+0000 7fe2c8a7b640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2c4107b50 0x7fe2c419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.152+0000 7fe2c8a7b640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c40696c0 0x7fe2c4198950 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.152+0000 7fe2c8a7b640 1 --2- 172.21.15.33:0/1216754421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2c4068d00 0x7fe2c4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.152+0000 7fe2c8a7b640 1 -- 172.21.15.33:0/1216754421 >> 172.21.15.33:0/1216754421 conn(0x7fe2c4077b00 msgr2=0x7fe2c40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.152+0000 7fe2c8a7b640 1 -- 172.21.15.33:0/1216754421 shutdown_connections 2024-04-02T13:41:36.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.152+0000 7fe2c8a7b640 1 -- 172.21.15.33:0/1216754421 wait complete. 2024-04-02T13:41:36.166 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:41:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.270+0000 7f122b25e640 1 -- 172.21.15.33:0/967106340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1224102460 msgr2=0x7f122410e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.270+0000 7f122b25e640 1 --2- 172.21.15.33:0/967106340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1224102460 0x7f122410e970 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1214009930 tx=0x7f121402bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.270+0000 7f122b25e640 1 -- 172.21.15.33:0/967106340 shutdown_connections 2024-04-02T13:41:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.270+0000 7f122b25e640 1 --2- 172.21.15.33:0/967106340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1224102460 0x7f122410e970 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.270+0000 7f122b25e640 1 --2- 172.21.15.33:0/967106340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1224101aa0 0x7f1224101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.270+0000 7f122b25e640 1 --2- 172.21.15.33:0/967106340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1224107ab0 0x7f1224107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.270+0000 7f122b25e640 1 -- 172.21.15.33:0/967106340 >> 172.21.15.33:0/967106340 conn(0x7f12240fd530 msgr2=0x7f12240ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.270+0000 7f122b25e640 1 -- 172.21.15.33:0/967106340 shutdown_connections 2024-04-02T13:41:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.270+0000 7f122b25e640 1 -- 172.21.15.33:0/967106340 wait complete. 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.270+0000 7f122b25e640 1 Processor -- start 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.270+0000 7f122b25e640 1 -- start start 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f122b25e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1224101aa0 0x7f1224198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f122b25e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1224102460 0x7f12241988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f122b25e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1224107ab0 0x7f122419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f122b25e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1224111e90 con 0x7f1224101aa0 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f122b25e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1224111d10 con 0x7f1224102460 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f122b25e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1224112010 con 0x7f1224107ab0 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1228ffa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1224101aa0 0x7f1224198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1228ffa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1224101aa0 0x7f1224198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35444/0 (socket says 172.21.15.33:35444) 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1228ffa640 1 -- 172.21.15.33:0/4030728098 learned_addr learned my addr 172.21.15.33:0/4030728098 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1223fff640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1224102460 0x7f12241988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f12297fb640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1224107ab0 0x7f122419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1228ffa640 1 -- 172.21.15.33:0/4030728098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1224107ab0 msgr2=0x7f122419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1228ffa640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1224107ab0 0x7f122419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1228ffa640 1 -- 172.21.15.33:0/4030728098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1224102460 msgr2=0x7f12241988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1228ffa640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1224102460 0x7f12241988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1228ffa640 1 -- 172.21.15.33:0/4030728098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122419d380 con 0x7f1224101aa0 2024-04-02T13:41:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1223fff640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1224102460 0x7f12241988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1228ffa640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1224101aa0 0x7f1224198390 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f121c00eb90 tx=0x7f121c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1221ffb640 1 -- 172.21.15.33:0/4030728098 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c004d20 con 0x7f1224101aa0 2024-04-02T13:41:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1221ffb640 1 -- 172.21.15.33:0/4030728098 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f121c00ee90 con 0x7f1224101aa0 2024-04-02T13:41:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f122b25e640 1 -- 172.21.15.33:0/4030728098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122419d670 con 0x7f1224101aa0 2024-04-02T13:41:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.271+0000 7f1221ffb640 1 -- 172.21.15.33:0/4030728098 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c010830 con 0x7f1224101aa0 2024-04-02T13:41:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.272+0000 7f122b25e640 1 -- 172.21.15.33:0/4030728098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12241a4f00 con 0x7f1224101aa0 2024-04-02T13:41:36.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.274+0000 7f1221ffb640 1 -- 172.21.15.33:0/4030728098 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f121c004fc0 con 0x7f1224101aa0 2024-04-02T13:41:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.274+0000 7f122de3d740 1 Processor -- start 2024-04-02T13:41:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.274+0000 7f122de3d740 1 -- start start 2024-04-02T13:41:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f122de3d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd62c7a0 0x55bacd62cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f1221ffb640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12100a3d20 0x7f12100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f122de3d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 0x55bacd635760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f122de3d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd635ca0 0x55bacd638090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f122de3d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bacd5907b0 con 0x55bacd62d0c0 2024-04-02T13:41:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f122de3d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bacd52f570 con 0x55bacd635ca0 2024-04-02T13:41:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f122de3d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bacd58c760 con 0x55bacd62c7a0 2024-04-02T13:41:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f1223fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 0x55bacd635760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f12297fb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd635ca0 0x55bacd638090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f1228ffa640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12100a3d20 0x7f12100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f1223fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 0x55bacd635760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35452/0 (socket says 172.21.15.33:35452) 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f1223fff640 1 -- 172.21.15.33:0/2843120622 learned_addr learned my addr 172.21.15.33:0/2843120622 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f12297fb640 1 --2- 172.21.15.33:0/2843120622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd635ca0 0x55bacd638090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:57776/0 (socket says 172.21.15.33:57776) 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f12297fb640 1 --2- 172.21.15.33:0/2843120622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd62c7a0 0x55bacd62cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f1221ffb640 1 -- 172.21.15.33:0/4030728098 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f121c0d1160 con 0x7f1224101aa0 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f1223fff640 1 -- 172.21.15.33:0/2843120622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd62c7a0 msgr2=0x55bacd62cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f1223fff640 1 --2- 172.21.15.33:0/2843120622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd62c7a0 0x55bacd62cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f1223fff640 1 -- 172.21.15.33:0/2843120622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd635ca0 msgr2=0x55bacd638090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f1223fff640 1 --2- 172.21.15.33:0/2843120622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd635ca0 0x55bacd638090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f1223fff640 1 -- 172.21.15.33:0/2843120622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacd2a43f0 con 0x55bacd62d0c0 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.275+0000 7f12297fb640 1 --2- 172.21.15.33:0/2843120622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd635ca0 0x55bacd638090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.276+0000 7f1228ffa640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12100a3d20 0x7f12100a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f121c09ca50 tx=0x7f121c01bb30 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.276+0000 7f1223fff640 1 --2- 172.21.15.33:0/2843120622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 0x55bacd635760 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f120c0114f0 tx=0x7f120c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cc2553b01fee60 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.276+0000 7f121affd640 1 -- 172.21.15.33:0/2843120622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c011250 con 0x55bacd62d0c0 2024-04-02T13:41:36.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.276+0000 7f121affd640 1 -- 172.21.15.33:0/2843120622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f120c0035f0 con 0x55bacd62d0c0 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.276+0000 7f121affd640 1 -- 172.21.15.33:0/2843120622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c00f820 con 0x55bacd62d0c0 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.276+0000 7f122de3d740 1 -- 172.21.15.33:0/2843120622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 msgr2=0x55bacd635760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.276+0000 7f122de3d740 1 --2- 172.21.15.33:0/2843120622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 0x55bacd635760 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f120c0114f0 tx=0x7f120c0119b0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.276+0000 7f122de3d740 1 -- 172.21.15.33:0/2843120622 shutdown_connections 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.276+0000 7f122de3d740 1 --2- 172.21.15.33:0/2843120622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd635ca0 0x55bacd638090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.276+0000 7f122de3d740 1 --2- 172.21.15.33:0/2843120622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 0x55bacd635760 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.276+0000 7f122de3d740 1 --2- 172.21.15.33:0/2843120622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd62c7a0 0x55bacd62cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.276+0000 7f122de3d740 1 -- 172.21.15.33:0/2843120622 >> 172.21.15.33:0/2843120622 conn(0x55bacd626ed0 msgr2=0x55bacd6272b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.277+0000 7f122de3d740 1 -- 172.21.15.33:0/2843120622 shutdown_connections 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.277+0000 7f122de3d740 1 -- 172.21.15.33:0/2843120622 wait complete. 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.277+0000 7f122de3d740 1 -- start start 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.277+0000 7f122de3d740 1 Processor -- start 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.277+0000 7f122de3d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62c7a0 0x55bacd632b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.277+0000 7f122de3d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62d0c0 0x55bacd633070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.277+0000 7f122de3d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd635ca0 0x55bacd6335b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.277+0000 7f122de3d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bacd638a90 con 0x55bacd62c7a0 2024-04-02T13:41:36.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.277+0000 7f122de3d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bacd54be80 con 0x55bacd62d0c0 2024-04-02T13:41:36.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.277+0000 7f122de3d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bacd638c10 con 0x55bacd635ca0 2024-04-02T13:41:36.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f122de3d740 10 client.0 _ll_register_callbacks cb 0x55bacd587aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:36.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f1223fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62d0c0 0x55bacd633070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f12297fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62c7a0 0x55bacd632b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f1223fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62d0c0 0x55bacd633070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:57788/0 (socket says 172.21.15.33:57788) 2024-04-02T13:41:36.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f1223fff640 1 -- 172.21.15.33:0/245763682 learned_addr learned my addr 172.21.15.33:0/245763682 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:36.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f12297fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62c7a0 0x55bacd632b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35466/0 (socket says 172.21.15.33:35466) 2024-04-02T13:41:36.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f12297fb640 1 --2- 172.21.15.33:0/245763682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd635ca0 0x55bacd6335b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f1223fff640 1 -- 172.21.15.33:0/245763682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd635ca0 msgr2=0x55bacd6335b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f1223fff640 1 --2- 172.21.15.33:0/245763682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd635ca0 0x55bacd6335b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f1223fff640 1 -- 172.21.15.33:0/245763682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62c7a0 msgr2=0x55bacd632b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f1223fff640 1 --2- 172.21.15.33:0/245763682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62c7a0 0x55bacd632b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f1223fff640 1 -- 172.21.15.33:0/245763682 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacd36e7d0 con 0x55bacd62d0c0 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f12297fb640 1 --2- 172.21.15.33:0/245763682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62c7a0 0x55bacd632b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f1223fff640 1 --2- 172.21.15.33:0/245763682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62d0c0 0x55bacd633070 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f120c01b040 tx=0x7f120c011480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cfcf2abb14e85f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f1203fff640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f1203fff640 1 -- 172.21.15.33:0/245763682 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c028070 con 0x55bacd62d0c0 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f122de3d740 1 -- 172.21.15.33:0/245763682 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120c0035f0 con 0x55bacd62d0c0 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.278+0000 7f1203fff640 1 -- 172.21.15.33:0/245763682 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f120c004210 con 0x55bacd62d0c0 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.279+0000 7f1203fff640 1 -- 172.21.15.33:0/245763682 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f120c034050 con 0x55bacd62d0c0 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.279+0000 7f122de3d740 10 client.25859 fetch_fsmap learned FSMap version 29 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.279+0000 7f122de3d740 1 -- 172.21.15.33:0/245763682 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bacd644be0 con 0x55bacd62d0c0 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.279+0000 7f1203fff640 1 -- 172.21.15.33:0/245763682 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f120c0035f0 con 0x55bacd62d0c0 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.279+0000 7f1203fff640 1 -- 172.21.15.33:0/245763682 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f120c0046c0 con 0x55bacd62d0c0 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.279+0000 7f122de3d740 10 client.25859 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:41:36.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.279+0000 7f122de3d740 10 client.25859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:36.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.279+0000 7f122de3d740 20 client.25859 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:36.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.279+0000 7f122de3d740 1 --2- 172.21.15.33:0/245763682 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bacd645ff0 0x55bacd6483e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.279+0000 7f122de3d740 4 client.25859 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:36.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.279+0000 7f122de3d740 1 -- 172.21.15.33:0/245763682 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bacd59dcf0 con 0x55bacd645ff0 2024-04-02T13:41:36.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.279+0000 7f12297fb640 1 --2- 172.21.15.33:0/245763682 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bacd645ff0 0x55bacd6483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.280+0000 7f12297fb640 1 --2- 172.21.15.33:0/245763682 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bacd645ff0 0x55bacd6483e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aee15ba3df22bb1 server_cookie=2da6afbbfe1cb278 in_seq=0 out_seq=0 2024-04-02T13:41:36.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.280+0000 7f1203fff640 10 client.25859 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:36.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.281+0000 7f1203fff640 1 -- 172.21.15.33:0/245763682 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bacd59dcf0 con 0x55bacd645ff0 2024-04-02T13:41:36.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.281+0000 7f1203fff640 10 client.25859 handle_command_reply: tid=0 2024-04-02T13:41:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.281+0000 7f122de3d740 1 client.25859 shutdown 2024-04-02T13:41:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.281+0000 7f122de3d740 1 -- 172.21.15.33:0/245763682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62d0c0 msgr2=0x55bacd633070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.281+0000 7f122de3d740 1 --2- 172.21.15.33:0/245763682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62d0c0 0x55bacd633070 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f120c01b040 tx=0x7f120c011480 comp rx=0 tx=0).stop 2024-04-02T13:41:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.282+0000 7f122de3d740 1 -- 172.21.15.33:0/245763682 shutdown_connections 2024-04-02T13:41:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.282+0000 7f122de3d740 1 --2- 172.21.15.33:0/245763682 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bacd645ff0 0x55bacd6483e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.282+0000 7f122de3d740 1 --2- 172.21.15.33:0/245763682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd635ca0 0x55bacd6335b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.282+0000 7f122de3d740 1 --2- 172.21.15.33:0/245763682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62d0c0 0x55bacd633070 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.282+0000 7f1203fff640 0 client.25859 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.282+0000 7f122de3d740 1 --2- 172.21.15.33:0/245763682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62c7a0 0x55bacd632b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.282+0000 7f122de3d740 1 -- 172.21.15.33:0/245763682 >> 172.21.15.33:0/245763682 conn(0x55bacd626ed0 msgr2=0x55bacd628c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.282+0000 7f122de3d740 1 -- 172.21.15.33:0/245763682 shutdown_connections 2024-04-02T13:41:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.282+0000 7f122de3d740 1 -- 172.21.15.33:0/245763682 wait complete. 2024-04-02T13:41:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.282+0000 7f122de3d740 20 client.25859 trim_cache size 0 max 16384 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.295+0000 7f122de3d740 1 Processor -- start 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.295+0000 7f122de3d740 1 -- start start 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.295+0000 7f122de3d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62c7a0 0x55bacd635260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.295+0000 7f122de3d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 0x55bacd641950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.295+0000 7f122de3d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd635ca0 0x55bacd631c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.295+0000 7f122de3d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f12241b18d0 con 0x55bacd62d0c0 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.295+0000 7f122de3d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f12241af110 con 0x55bacd62c7a0 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.295+0000 7f122de3d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f12241acf40 con 0x55bacd635ca0 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f1223fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 0x55bacd641950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f1223fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 0x55bacd641950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35468/0 (socket says 172.21.15.33:35468) 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f1223fff640 1 -- 172.21.15.33:0/3409808379 learned_addr learned my addr 172.21.15.33:0/3409808379 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f12297fb640 1 --2- 172.21.15.33:0/3409808379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62c7a0 0x55bacd635260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f12297fb640 1 --2- 172.21.15.33:0/3409808379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd635ca0 0x55bacd631c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f1223fff640 1 -- 172.21.15.33:0/3409808379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd635ca0 msgr2=0x55bacd631c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f1223fff640 1 --2- 172.21.15.33:0/3409808379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd635ca0 0x55bacd631c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f1223fff640 1 -- 172.21.15.33:0/3409808379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62c7a0 msgr2=0x55bacd635260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f1223fff640 1 --2- 172.21.15.33:0/3409808379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62c7a0 0x55bacd635260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f1223fff640 1 -- 172.21.15.33:0/3409808379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacd638970 con 0x55bacd62d0c0 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f12297fb640 1 --2- 172.21.15.33:0/3409808379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd635ca0 0x55bacd631c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f1223fff640 1 --2- 172.21.15.33:0/3409808379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 0x55bacd641950 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f120c0119f0 tx=0x7f120c00f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b334afe9c369574 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f121a7fc640 1 -- 172.21.15.33:0/3409808379 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c03d070 con 0x55bacd62d0c0 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f121a7fc640 1 -- 172.21.15.33:0/3409808379 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f120c031030 con 0x55bacd62d0c0 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.296+0000 7f121a7fc640 1 -- 172.21.15.33:0/3409808379 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c028040 con 0x55bacd62d0c0 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.297+0000 7f122de3d740 1 -- 172.21.15.33:0/3409808379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 msgr2=0x55bacd641950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.297+0000 7f122de3d740 1 --2- 172.21.15.33:0/3409808379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 0x55bacd641950 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f120c0119f0 tx=0x7f120c00f5c0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.297+0000 7f122de3d740 1 -- 172.21.15.33:0/3409808379 shutdown_connections 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.297+0000 7f122de3d740 1 --2- 172.21.15.33:0/3409808379 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd635ca0 0x55bacd631c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.297+0000 7f122de3d740 1 --2- 172.21.15.33:0/3409808379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd62d0c0 0x55bacd641950 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.297+0000 7f122de3d740 1 --2- 172.21.15.33:0/3409808379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62c7a0 0x55bacd635260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.297+0000 7f122de3d740 1 -- 172.21.15.33:0/3409808379 >> 172.21.15.33:0/3409808379 conn(0x55bacd626ed0 msgr2=0x55bacd632af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.297+0000 7f122de3d740 1 -- 172.21.15.33:0/3409808379 shutdown_connections 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.297+0000 7f122de3d740 1 -- 172.21.15.33:0/3409808379 wait complete. 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.297+0000 7f122de3d740 1 -- start start 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.297+0000 7f122de3d740 1 Processor -- start 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f122de3d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62c7a0 0x55bacd63e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f122de3d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd62d0c0 0x55bacd6393e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f122de3d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd635ca0 0x55bacd639920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f122de3d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f12241ad5d0 con 0x55bacd635ca0 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f122de3d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f12241aea40 con 0x55bacd62c7a0 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f122de3d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f12241adca0 con 0x55bacd62d0c0 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f122de3d740 10 client.0 _ll_register_callbacks cb 0x7f121c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f12297fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd635ca0 0x55bacd639920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f12297fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd635ca0 0x55bacd639920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35480/0 (socket says 172.21.15.33:35480) 2024-04-02T13:41:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f12297fb640 1 -- 172.21.15.33:0/3276975075 learned_addr learned my addr 172.21.15.33:0/3276975075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f12297fb640 1 --2- 172.21.15.33:0/3276975075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62c7a0 0x55bacd63e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f1223fff640 1 --2- 172.21.15.33:0/3276975075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd62d0c0 0x55bacd6393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f12297fb640 1 -- 172.21.15.33:0/3276975075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd62d0c0 msgr2=0x55bacd6393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f12297fb640 1 --2- 172.21.15.33:0/3276975075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd62d0c0 0x55bacd6393e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f12297fb640 1 -- 172.21.15.33:0/3276975075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62c7a0 msgr2=0x55bacd63e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f12297fb640 1 --2- 172.21.15.33:0/3276975075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62c7a0 0x55bacd63e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f12297fb640 1 -- 172.21.15.33:0/3276975075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacd639ee0 con 0x55bacd635ca0 2024-04-02T13:41:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f12297fb640 1 --2- 172.21.15.33:0/3276975075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd635ca0 0x55bacd639920 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1214005940 tx=0x7f1214045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ffd327f1013a33 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f1203fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f1203fff640 1 -- 172.21.15.33:0/3276975075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121404d2a0 con 0x55bacd635ca0 2024-04-02T13:41:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f1203fff640 1 -- 172.21.15.33:0/3276975075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f121403a040 con 0x55bacd635ca0 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.298+0000 7f122de3d740 1 -- 172.21.15.33:0/3276975075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120c031030 con 0x55bacd635ca0 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.299+0000 7f1203fff640 1 -- 172.21.15.33:0/3276975075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1214062050 con 0x55bacd635ca0 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.299+0000 7f122de3d740 10 client.16266 fetch_fsmap learned FSMap version 29 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.299+0000 7f122de3d740 1 -- 172.21.15.33:0/3276975075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bacd6ac1c0 con 0x55bacd635ca0 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.299+0000 7f1203fff640 1 -- 172.21.15.33:0/3276975075 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f120c031030 con 0x55bacd635ca0 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.299+0000 7f1203fff640 1 -- 172.21.15.33:0/3276975075 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f12140046c0 con 0x55bacd635ca0 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.299+0000 7f122de3d740 10 client.16266 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.299+0000 7f122de3d740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.299+0000 7f122de3d740 20 client.16266 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.299+0000 7f122de3d740 1 --2- 172.21.15.33:0/3276975075 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bacd6ac8f0 0x55bacd6aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.299+0000 7f122de3d740 4 client.16266 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.299+0000 7f122de3d740 1 -- 172.21.15.33:0/3276975075 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bacd598040 con 0x55bacd6ac8f0 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.299+0000 7f1223fff640 1 --2- 172.21.15.33:0/3276975075 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bacd6ac8f0 0x55bacd6aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.300+0000 7f1203fff640 10 client.16266 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.300+0000 7f1223fff640 1 --2- 172.21.15.33:0/3276975075 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bacd6ac8f0 0x55bacd6aed10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57fd430788839785 server_cookie=d8f5e9230d65bc27 in_seq=0 out_seq=0 2024-04-02T13:41:36.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f1203fff640 1 -- 172.21.15.33:0/3276975075 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bacd598040 con 0x55bacd6ac8f0 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f1203fff640 10 client.16266 handle_command_reply: tid=0 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f122de3d740 1 client.16266 shutdown 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f122de3d740 1 -- 172.21.15.33:0/3276975075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd635ca0 msgr2=0x55bacd639920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f122de3d740 1 --2- 172.21.15.33:0/3276975075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd635ca0 0x55bacd639920 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1214005940 tx=0x7f1214045040 comp rx=0 tx=0).stop 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f122de3d740 1 -- 172.21.15.33:0/3276975075 shutdown_connections 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f122de3d740 1 --2- 172.21.15.33:0/3276975075 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bacd6ac8f0 0x55bacd6aed10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f122de3d740 1 --2- 172.21.15.33:0/3276975075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bacd635ca0 0x55bacd639920 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f122de3d740 1 --2- 172.21.15.33:0/3276975075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bacd62d0c0 0x55bacd6393e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f122de3d740 1 --2- 172.21.15.33:0/3276975075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacd62c7a0 0x55bacd63e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f1203fff640 0 client.16266 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f122de3d740 1 -- 172.21.15.33:0/3276975075 >> 172.21.15.33:0/3276975075 conn(0x55bacd626ed0 msgr2=0x55bacd6256a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f122de3d740 1 -- 172.21.15.33:0/3276975075 shutdown_connections 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f122de3d740 1 -- 172.21.15.33:0/3276975075 wait complete. 2024-04-02T13:41:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.301+0000 7f122de3d740 20 client.16266 trim_cache size 0 max 16384 2024-04-02T13:41:36.310 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:41:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.302+0000 7f122b25e640 1 -- 172.21.15.33:0/4030728098 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12100a3d20 msgr2=0x7f12100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.302+0000 7f122b25e640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12100a3d20 0x7f12100a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f121c09ca50 tx=0x7f121c01bb30 comp rx=0 tx=0).stop 2024-04-02T13:41:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.302+0000 7f122b25e640 1 -- 172.21.15.33:0/4030728098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1224101aa0 msgr2=0x7f1224198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.302+0000 7f122b25e640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1224101aa0 0x7f1224198390 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f121c00eb90 tx=0x7f121c00c6a0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.302+0000 7f122b25e640 1 -- 172.21.15.33:0/4030728098 shutdown_connections 2024-04-02T13:41:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.302+0000 7f122b25e640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12100a3d20 0x7f12100a61e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.302+0000 7f122b25e640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1224107ab0 0x7f122419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.302+0000 7f122b25e640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1224102460 0x7f12241988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.302+0000 7f122b25e640 1 --2- 172.21.15.33:0/4030728098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1224101aa0 0x7f1224198390 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.302+0000 7f122b25e640 1 -- 172.21.15.33:0/4030728098 >> 172.21.15.33:0/4030728098 conn(0x7f12240fd530 msgr2=0x7f12240fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.302+0000 7f122b25e640 1 -- 172.21.15.33:0/4030728098 shutdown_connections 2024-04-02T13:41:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:36.302+0000 7f122b25e640 1 -- 172.21.15.33:0/4030728098 wait complete. 2024-04-02T13:41:36.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:36 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/264130858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:41:36.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:36 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/264130858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:41:37.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:36 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/264130858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:41:37.316 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:41:37.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:41:37.317 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:41:37.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.492+0000 7fce5759e640 1 -- 172.21.15.33:0/3726562745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce581009f0 msgr2=0x7fce581048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.492+0000 7fce5759e640 1 --2- 172.21.15.33:0/3726562745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce581009f0 0x7fce581048f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fce40009860 tx=0x7fce4002bb90 comp rx=0 tx=0).stop 2024-04-02T13:41:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.493+0000 7fce5759e640 1 -- 172.21.15.33:0/3726562745 shutdown_connections 2024-04-02T13:41:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.493+0000 7fce5759e640 1 --2- 172.21.15.33:0/3726562745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce58104e30 0x7fce581052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.493+0000 7fce5759e640 1 --2- 172.21.15.33:0/3726562745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce581009f0 0x7fce581048f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.493+0000 7fce5759e640 1 --2- 172.21.15.33:0/3726562745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce581000d0 0x7fce581004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.493+0000 7fce5759e640 1 -- 172.21.15.33:0/3726562745 >> 172.21.15.33:0/3726562745 conn(0x7fce580756c0 msgr2=0x7fce58075ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:37.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.493+0000 7fce5759e640 1 -- 172.21.15.33:0/3726562745 shutdown_connections 2024-04-02T13:41:37.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.493+0000 7fce5759e640 1 -- 172.21.15.33:0/3726562745 wait complete. 2024-04-02T13:41:37.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.494+0000 7fce5759e640 1 Processor -- start 2024-04-02T13:41:37.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.495+0000 7fce5759e640 1 -- start start 2024-04-02T13:41:37.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.495+0000 7fce5759e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce581000d0 0x7fce58116060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.495+0000 7fce5759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce581009f0 0x7fce581165a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.495+0000 7fce5759e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce58104e30 0x7fce5811bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.495+0000 7fce5759e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce5810bb90 con 0x7fce581009f0 2024-04-02T13:41:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.495+0000 7fce5759e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fce5810ba10 con 0x7fce58104e30 2024-04-02T13:41:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.495+0000 7fce5759e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fce5810fe80 con 0x7fce581000d0 2024-04-02T13:41:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.496+0000 7fce56d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce58104e30 0x7fce5811bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.496+0000 7fce56d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce58104e30 0x7fce5811bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:57840/0 (socket says 172.21.15.33:57840) 2024-04-02T13:41:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.496+0000 7fce56d9d640 1 -- 172.21.15.33:0/701143094 learned_addr learned my addr 172.21.15.33:0/701143094 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.496+0000 7fce5659c640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce581000d0 0x7fce58116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.496+0000 7fce4dd9b640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce581009f0 0x7fce581165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.496+0000 7fce5659c640 1 -- 172.21.15.33:0/701143094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce58104e30 msgr2=0x7fce5811bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.496+0000 7fce5659c640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce58104e30 0x7fce5811bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.496+0000 7fce5659c640 1 -- 172.21.15.33:0/701143094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce581009f0 msgr2=0x7fce581165a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.496+0000 7fce5659c640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce581009f0 0x7fce581165a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.496+0000 7fce5659c640 1 -- 172.21.15.33:0/701143094 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5811c630 con 0x7fce581000d0 2024-04-02T13:41:37.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.496+0000 7fce4dd9b640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce581009f0 0x7fce581165a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:37.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.497+0000 7fce5659c640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce581000d0 0x7fce58116060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fce4800d6d0 tx=0x7fce4800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:37.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.497+0000 7fce4ffff640 1 -- 172.21.15.33:0/701143094 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48017070 con 0x7fce581000d0 2024-04-02T13:41:37.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.497+0000 7fce4ffff640 1 -- 172.21.15.33:0/701143094 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fce48009820 con 0x7fce581000d0 2024-04-02T13:41:37.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.497+0000 7fce4ffff640 1 -- 172.21.15.33:0/701143094 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48017070 con 0x7fce581000d0 2024-04-02T13:41:37.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.498+0000 7fce5759e640 1 -- 172.21.15.33:0/701143094 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce58111170 con 0x7fce581000d0 2024-04-02T13:41:37.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.498+0000 7fce5759e640 1 -- 172.21.15.33:0/701143094 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce58111640 con 0x7fce581000d0 2024-04-02T13:41:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.501+0000 7fce5f9ff740 1 Processor -- start 2024-04-02T13:41:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.502+0000 7fce5f9ff740 1 -- start start 2024-04-02T13:41:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.502+0000 7fce5f9ff740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7490 0x5624464a7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.502+0000 7fce5f9ff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7db0 0x5624464b0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.502+0000 7fce5f9ff740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464b0990 0x5624464b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.502+0000 7fce5f9ff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56244640b770 con 0x5624464a7db0 2024-04-02T13:41:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.502+0000 7fce5f9ff740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5624463b3960 con 0x5624464b0990 2024-04-02T13:41:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.502+0000 7fce5f9ff740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5624464078c0 con 0x5624464a7490 2024-04-02T13:41:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.502+0000 7fce56d9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7490 0x5624464a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.502+0000 7fce56d9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7490 0x5624464a7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:51054/0 (socket says 172.21.15.33:51054) 2024-04-02T13:41:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.502+0000 7fce56d9d640 1 -- 172.21.15.33:0/467543430 learned_addr learned my addr 172.21.15.33:0/467543430 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.503+0000 7fce56d9d640 1 -- 172.21.15.33:0/467543430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464b0990 msgr2=0x5624464b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.503+0000 7fce56d9d640 1 --2- 172.21.15.33:0/467543430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464b0990 0x5624464b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.503+0000 7fce56d9d640 1 -- 172.21.15.33:0/467543430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7db0 msgr2=0x5624464b0450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:41:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.503+0000 7fce56d9d640 1 --2- 172.21.15.33:0/467543430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7db0 0x5624464b0450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.503+0000 7fce56d9d640 1 -- 172.21.15.33:0/467543430 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624461e77e0 con 0x5624464a7490 2024-04-02T13:41:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.503+0000 7fce4ffff640 1 -- 172.21.15.33:0/701143094 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fce480099d0 con 0x7fce581000d0 2024-04-02T13:41:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.503+0000 7fce56d9d640 1 --2- 172.21.15.33:0/467543430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7490 0x5624464a7870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e500 tx=0x7fce500095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bda321ef4b91aa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.504+0000 7fce4ffff640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fce440a41a0 0x7fce440a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.504+0000 7fce4cd99640 1 -- 172.21.15.33:0/467543430 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50003970 con 0x5624464a7490 2024-04-02T13:41:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.504+0000 7fce4cd99640 1 -- 172.21.15.33:0/467543430 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fce5000d770 con 0x5624464a7490 2024-04-02T13:41:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.504+0000 7fce4ffff640 1 -- 172.21.15.33:0/701143094 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fce48005020 con 0x7fce581000d0 2024-04-02T13:41:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.504+0000 7fce4cd99640 1 -- 172.21.15.33:0/467543430 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50004560 con 0x5624464a7490 2024-04-02T13:41:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.504+0000 7fce5659c640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fce440a41a0 0x7fce440a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.505+0000 7fce5f9ff740 1 -- 172.21.15.33:0/467543430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7490 msgr2=0x5624464a7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.505+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/467543430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7490 0x5624464a7870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e500 tx=0x7fce500095c0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.505+0000 7fce5f9ff740 1 -- 172.21.15.33:0/467543430 shutdown_connections 2024-04-02T13:41:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.505+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/467543430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464b0990 0x5624464b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.505+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/467543430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7db0 0x5624464b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.505+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/467543430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7490 0x5624464a7870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.505+0000 7fce5f9ff740 1 -- 172.21.15.33:0/467543430 >> 172.21.15.33:0/467543430 conn(0x5624464a1bc0 msgr2=0x5624464a1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.506+0000 7fce5659c640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fce440a41a0 0x7fce440a6660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5624464a6f60 tx=0x7fce480bc040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.506+0000 7fce5f9ff740 1 -- 172.21.15.33:0/467543430 shutdown_connections 2024-04-02T13:41:37.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.507+0000 7fce5f9ff740 1 -- 172.21.15.33:0/467543430 wait complete. 2024-04-02T13:41:37.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.508+0000 7fce5f9ff740 1 -- start start 2024-04-02T13:41:37.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.508+0000 7fce5f9ff740 1 Processor -- start 2024-04-02T13:41:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.509+0000 7fce5f9ff740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464a7490 0x5624464ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.509+0000 7fce5f9ff740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7db0 0x5624464adf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.509+0000 7fce5f9ff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464b0990 0x5624464ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.509+0000 7fce5f9ff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5624464b37d0 con 0x5624464b0990 2024-04-02T13:41:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.509+0000 7fce5f9ff740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5624463c6d20 con 0x5624464a7490 2024-04-02T13:41:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.509+0000 7fce5f9ff740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5624464b3950 con 0x5624464a7db0 2024-04-02T13:41:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.509+0000 7fce5f9ff740 10 client.0 _ll_register_callbacks cb 0x562446393a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.509+0000 7fce4dd9b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7db0 0x5624464adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.509+0000 7fce4dd9b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7db0 0x5624464adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:51058/0 (socket says 172.21.15.33:51058) 2024-04-02T13:41:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.509+0000 7fce4dd9b640 1 -- 172.21.15.33:0/3765961377 learned_addr learned my addr 172.21.15.33:0/3765961377 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.509+0000 7fce56d9d640 1 --2- 172.21.15.33:0/3765961377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464a7490 0x5624464ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.509+0000 7fce56d9d640 1 --2- 172.21.15.33:0/3765961377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464b0990 0x5624464ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.510+0000 7fce4dd9b640 1 -- 172.21.15.33:0/3765961377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464a7490 msgr2=0x5624464ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.510+0000 7fce4dd9b640 1 --2- 172.21.15.33:0/3765961377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464a7490 0x5624464ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.510+0000 7fce4dd9b640 1 -- 172.21.15.33:0/3765961377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464b0990 msgr2=0x5624464ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.510+0000 7fce4dd9b640 1 --2- 172.21.15.33:0/3765961377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464b0990 0x5624464ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.510+0000 7fce4dd9b640 1 -- 172.21.15.33:0/3765961377 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562446247f70 con 0x5624464a7db0 2024-04-02T13:41:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.510+0000 7fce4dd9b640 1 --2- 172.21.15.33:0/3765961377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7db0 0x5624464adf70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fce40005d40 tx=0x7fce40005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb4e3d8bce1f8dda server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.510+0000 7fce3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-04-02T13:41:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.511+0000 7fce3dffb640 1 -- 172.21.15.33:0/3765961377 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40043030 con 0x5624464a7db0 2024-04-02T13:41:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.511+0000 7fce3dffb640 1 -- 172.21.15.33:0/3765961377 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fce40004250 con 0x5624464a7db0 2024-04-02T13:41:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.511+0000 7fce5f9ff740 1 -- 172.21.15.33:0/3765961377 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5000d770 con 0x5624464a7db0 2024-04-02T13:41:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.511+0000 7fce3dffb640 1 -- 172.21.15.33:0/3765961377 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce40039070 con 0x5624464a7db0 2024-04-02T13:41:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.512+0000 7fce5f9ff740 10 client.25705 fetch_fsmap learned FSMap version 29 2024-04-02T13:41:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.512+0000 7fce5f9ff740 1 -- 172.21.15.33:0/3765961377 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5624460df170 con 0x5624464a7db0 2024-04-02T13:41:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.512+0000 7fce3dffb640 1 -- 172.21.15.33:0/3765961377 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fce5000d770 con 0x5624464a7db0 2024-04-02T13:41:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.513+0000 7fce3dffb640 1 -- 172.21.15.33:0/3765961377 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fce400035f0 con 0x5624464a7db0 2024-04-02T13:41:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.513+0000 7fce5f9ff740 10 client.25705 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:41:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.513+0000 7fce5f9ff740 10 client.25705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.513+0000 7fce5f9ff740 20 client.25705 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.513+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/3765961377 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5624464c0e60 0x5624464c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.513+0000 7fce5f9ff740 4 client.25705 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.513+0000 7fce5f9ff740 1 -- 172.21.15.33:0/3765961377 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562446418cb0 con 0x5624464c0e60 2024-04-02T13:41:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.513+0000 7fce56d9d640 1 --2- 172.21.15.33:0/3765961377 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5624464c0e60 0x5624464c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.516+0000 7fce56d9d640 1 --2- 172.21.15.33:0/3765961377 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5624464c0e60 0x5624464c3250 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d04db065f7e0ffc server_cookie=eacae756330ba3d5 in_seq=0 out_seq=0 2024-04-02T13:41:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.516+0000 7fce3dffb640 10 client.25705 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.518+0000 7fce3dffb640 1 -- 172.21.15.33:0/3765961377 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562446418cb0 con 0x5624464c0e60 2024-04-02T13:41:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.518+0000 7fce3dffb640 10 client.25705 handle_command_reply: tid=0 2024-04-02T13:41:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.518+0000 7fce5f9ff740 1 client.25705 shutdown 2024-04-02T13:41:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.519+0000 7fce5f9ff740 1 -- 172.21.15.33:0/3765961377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7db0 msgr2=0x5624464adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.519+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/3765961377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7db0 0x5624464adf70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fce40005d40 tx=0x7fce40005ab0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.519+0000 7fce5f9ff740 1 -- 172.21.15.33:0/3765961377 shutdown_connections 2024-04-02T13:41:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.519+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/3765961377 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5624464c0e60 0x5624464c3250 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.519+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/3765961377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464b0990 0x5624464ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.519+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/3765961377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7db0 0x5624464adf70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.519+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/3765961377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464a7490 0x5624464ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.519+0000 7fce5f9ff740 1 -- 172.21.15.33:0/3765961377 >> 172.21.15.33:0/3765961377 conn(0x5624464a1bc0 msgr2=0x5624464a3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:37.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.519+0000 7fce3dffb640 0 client.25705 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:37.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.520+0000 7fce5f9ff740 1 -- 172.21.15.33:0/3765961377 shutdown_connections 2024-04-02T13:41:37.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.520+0000 7fce5f9ff740 1 -- 172.21.15.33:0/3765961377 wait complete. 2024-04-02T13:41:37.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.520+0000 7fce5f9ff740 20 client.25705 trim_cache size 0 max 16384 2024-04-02T13:41:37.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.557+0000 7fce5f9ff740 1 Processor -- start 2024-04-02T13:41:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.557+0000 7fce5f9ff740 1 -- start start 2024-04-02T13:41:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce5f9ff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 0x5624464b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce5f9ff740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7db0 0x5624464bc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce5f9ff740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464b0990 0x5624464c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce5f9ff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce581c0530 con 0x5624464a7490 2024-04-02T13:41:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce5f9ff740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fce581bdd70 con 0x5624464b0990 2024-04-02T13:41:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce5f9ff740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fce581bbba0 con 0x5624464a7db0 2024-04-02T13:41:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce56d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 0x5624464b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce56d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 0x5624464b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35518/0 (socket says 172.21.15.33:35518) 2024-04-02T13:41:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce56d9d640 1 -- 172.21.15.33:0/3180912011 learned_addr learned my addr 172.21.15.33:0/3180912011 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce4dd9b640 1 --2- 172.21.15.33:0/3180912011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7db0 0x5624464bc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce56d9d640 1 --2- 172.21.15.33:0/3180912011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464b0990 0x5624464c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce56d9d640 1 -- 172.21.15.33:0/3180912011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7db0 msgr2=0x5624464bc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce56d9d640 1 --2- 172.21.15.33:0/3180912011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7db0 0x5624464bc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce56d9d640 1 -- 172.21.15.33:0/3180912011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464b0990 msgr2=0x5624464c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce56d9d640 1 --2- 172.21.15.33:0/3180912011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464b0990 0x5624464c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.558+0000 7fce56d9d640 1 -- 172.21.15.33:0/3180912011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624464ac9e0 con 0x5624464a7490 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.559+0000 7fce56d9d640 1 --2- 172.21.15.33:0/3180912011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 0x5624464b0160 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fce5000a010 tx=0x7fce50010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90dc11ec7d01f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.559+0000 7fce3ffff640 1 -- 172.21.15.33:0/3180912011 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50004a00 con 0x5624464a7490 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.559+0000 7fce3ffff640 1 -- 172.21.15.33:0/3180912011 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fce50003e40 con 0x5624464a7490 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.559+0000 7fce3ffff640 1 -- 172.21.15.33:0/3180912011 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce500315c0 con 0x5624464a7490 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 -- 172.21.15.33:0/3180912011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 msgr2=0x5624464b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/3180912011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 0x5624464b0160 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fce5000a010 tx=0x7fce50010040 comp rx=0 tx=0).stop 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 -- 172.21.15.33:0/3180912011 shutdown_connections 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/3180912011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464b0990 0x5624464c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/3180912011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464a7db0 0x5624464bc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/3180912011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 0x5624464b0160 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 -- 172.21.15.33:0/3180912011 >> 172.21.15.33:0/3180912011 conn(0x5624464a1bc0 msgr2=0x5624464a3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 -- 172.21.15.33:0/3180912011 shutdown_connections 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 -- 172.21.15.33:0/3180912011 wait complete. 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 -- start start 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 Processor -- start 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 0x5624464b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464a7db0 0x5624464b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464b0990 0x5624464b5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce581bc230 con 0x5624464a7490 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fce581bd6a0 con 0x5624464a7db0 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fce581bc900 con 0x5624464b0990 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce5f9ff740 10 client.0 _ll_register_callbacks cb 0x7fce50009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce56d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 0x5624464b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce56d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 0x5624464b8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35528/0 (socket says 172.21.15.33:35528) 2024-04-02T13:41:37.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce56d9d640 1 -- 172.21.15.33:0/1361606719 learned_addr learned my addr 172.21.15.33:0/1361606719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:37.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.560+0000 7fce4dd9b640 1 --2- 172.21.15.33:0/1361606719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464a7db0 0x5624464b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce56d9d640 1 --2- 172.21.15.33:0/1361606719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464b0990 0x5624464b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce56d9d640 1 -- 172.21.15.33:0/1361606719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464b0990 msgr2=0x5624464b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce56d9d640 1 --2- 172.21.15.33:0/1361606719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464b0990 0x5624464b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce56d9d640 1 -- 172.21.15.33:0/1361606719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464a7db0 msgr2=0x5624464b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce56d9d640 1 --2- 172.21.15.33:0/1361606719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464a7db0 0x5624464b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce56d9d640 1 -- 172.21.15.33:0/1361606719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624464a7be0 con 0x5624464a7490 2024-04-02T13:41:37.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce56d9d640 1 --2- 172.21.15.33:0/1361606719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 0x5624464b8da0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fce500097c0 tx=0x7fce50009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f962e029af54855 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:37.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce3dffb640 1 -- 172.21.15.33:0/1361606719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50035020 con 0x5624464a7490 2024-04-02T13:41:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce3dffb640 1 -- 172.21.15.33:0/1361606719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fce50004a00 con 0x5624464a7490 2024-04-02T13:41:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce5f9ff740 1 -- 172.21.15.33:0/1361606719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce50003e40 con 0x5624464a7490 2024-04-02T13:41:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce3dffb640 1 -- 172.21.15.33:0/1361606719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce50003c80 con 0x5624464a7490 2024-04-02T13:41:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce5f9ff740 10 client.16281 fetch_fsmap learned FSMap version 29 2024-04-02T13:41:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce5f9ff740 1 -- 172.21.15.33:0/1361606719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562446528040 con 0x5624464a7490 2024-04-02T13:41:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.561+0000 7fce3dffb640 1 -- 172.21.15.33:0/1361606719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fce50003e40 con 0x5624464a7490 2024-04-02T13:41:37.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.562+0000 7fce3dffb640 1 -- 172.21.15.33:0/1361606719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fce5000b040 con 0x5624464a7490 2024-04-02T13:41:37.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.562+0000 7fce5f9ff740 10 client.16281 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:41:37.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.562+0000 7fce5f9ff740 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:37.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.562+0000 7fce5f9ff740 20 client.16281 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:37.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.562+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/1361606719 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562446528560 0x56244652a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.562+0000 7fce5f9ff740 4 client.16281 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:41:37.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.562+0000 7fce5f9ff740 1 -- 172.21.15.33:0/1361606719 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562446413000 con 0x562446528560 2024-04-02T13:41:37.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.616+0000 7fce4dd9b640 1 --2- 172.21.15.33:0/1361606719 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562446528560 0x56244652a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.617+0000 7fce4dd9b640 1 --2- 172.21.15.33:0/1361606719 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562446528560 0x56244652a980 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8cf34ead40d342 server_cookie=1a066228ca13b733 in_seq=0 out_seq=0 2024-04-02T13:41:37.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.618+0000 7fce3dffb640 10 client.16281 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:37.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.620+0000 7fce3dffb640 1 -- 172.21.15.33:0/1361606719 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562446413000 con 0x562446528560 2024-04-02T13:41:37.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.620+0000 7fce3dffb640 10 client.16281 handle_command_reply: tid=0 2024-04-02T13:41:37.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.620+0000 7fce5f9ff740 1 client.16281 shutdown 2024-04-02T13:41:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.621+0000 7fce5f9ff740 1 -- 172.21.15.33:0/1361606719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 msgr2=0x5624464b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.621+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/1361606719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 0x5624464b8da0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fce500097c0 tx=0x7fce50009be0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.621+0000 7fce5f9ff740 1 -- 172.21.15.33:0/1361606719 shutdown_connections 2024-04-02T13:41:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.621+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/1361606719 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562446528560 0x56244652a980 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.621+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/1361606719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5624464b0990 0x5624464b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.621+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/1361606719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624464a7db0 0x5624464b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.621+0000 7fce5f9ff740 1 --2- 172.21.15.33:0/1361606719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624464a7490 0x5624464b8da0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.621+0000 7fce5f9ff740 1 -- 172.21.15.33:0/1361606719 >> 172.21.15.33:0/1361606719 conn(0x5624464a1bc0 msgr2=0x5624464a0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:37.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.621+0000 7fce3dffb640 0 client.16281 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:37.627 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:41:37.627 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:41:37.627 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "0975e94a-c470-4832-b39c-2ff36d1785b4", 2024-04-02T13:41:37.627 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:41:37.627 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:41:37.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.622+0000 7fce5f9ff740 1 -- 172.21.15.33:0/1361606719 shutdown_connections 2024-04-02T13:41:37.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.622+0000 7fce5f9ff740 1 -- 172.21.15.33:0/1361606719 wait complete. 2024-04-02T13:41:37.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.623+0000 7fce5f9ff740 20 client.16281 trim_cache size 0 max 16384 2024-04-02T13:41:37.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.624+0000 7fce5759e640 1 -- 172.21.15.33:0/701143094 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fce440a41a0 msgr2=0x7fce440a6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.624+0000 7fce5759e640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fce440a41a0 0x7fce440a6660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5624464a6f60 tx=0x7fce480bc040 comp rx=0 tx=0).stop 2024-04-02T13:41:37.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.624+0000 7fce5759e640 1 -- 172.21.15.33:0/701143094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce581000d0 msgr2=0x7fce58116060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.624+0000 7fce5759e640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce581000d0 0x7fce58116060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fce4800d6d0 tx=0x7fce4800db90 comp rx=0 tx=0).stop 2024-04-02T13:41:37.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.624+0000 7fce5759e640 1 -- 172.21.15.33:0/701143094 shutdown_connections 2024-04-02T13:41:37.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.624+0000 7fce5759e640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fce440a41a0 0x7fce440a6660 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.624+0000 7fce5759e640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce58104e30 0x7fce5811bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.624+0000 7fce5759e640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce581009f0 0x7fce581165a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.624+0000 7fce5759e640 1 --2- 172.21.15.33:0/701143094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fce581000d0 0x7fce58116060 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.624+0000 7fce5759e640 1 -- 172.21.15.33:0/701143094 >> 172.21.15.33:0/701143094 conn(0x7fce580756c0 msgr2=0x7fce580feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.624+0000 7fce5759e640 1 -- 172.21.15.33:0/701143094 shutdown_connections 2024-04-02T13:41:37.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.624+0000 7fce5759e640 1 -- 172.21.15.33:0/701143094 wait complete. 2024-04-02T13:41:37.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:41:37.648 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:41:37.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.766+0000 7f962bdaa640 1 -- 172.21.15.33:0/2440705797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9624101370 msgr2=0x7f962410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.766+0000 7f962bdaa640 1 --2- 172.21.15.33:0/2440705797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9624101370 0x7f962410d880 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f961c009cd0 tx=0x7f961c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.766+0000 7f962bdaa640 1 -- 172.21.15.33:0/2440705797 shutdown_connections 2024-04-02T13:41:37.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.766+0000 7f962bdaa640 1 --2- 172.21.15.33:0/2440705797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9624101370 0x7f962410d880 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.766+0000 7f962bdaa640 1 --2- 172.21.15.33:0/2440705797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96241009b0 0x7f9624100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.766+0000 7f962bdaa640 1 --2- 172.21.15.33:0/2440705797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96241069c0 0x7f9624106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.766+0000 7f962bdaa640 1 -- 172.21.15.33:0/2440705797 >> 172.21.15.33:0/2440705797 conn(0x7f9624076d80 msgr2=0x7f9624077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.766+0000 7f962bdaa640 1 -- 172.21.15.33:0/2440705797 shutdown_connections 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.766+0000 7f962bdaa640 1 -- 172.21.15.33:0/2440705797 wait complete. 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.767+0000 7f962bdaa640 1 Processor -- start 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.767+0000 7f962bdaa640 1 -- start start 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.767+0000 7f962bdaa640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96241009b0 0x7f962419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.767+0000 7f962bdaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9624101370 0x7f962419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.767+0000 7f962bdaa640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96241069c0 0x7f962419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.767+0000 7f962bdaa640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9624111ea0 con 0x7f9624101370 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.767+0000 7f962bdaa640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9624111d20 con 0x7f96241069c0 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.767+0000 7f962bdaa640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9624112020 con 0x7f96241009b0 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.767+0000 7f9629345640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9624101370 0x7f962419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.767+0000 7f9629345640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9624101370 0x7f962419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35542/0 (socket says 172.21.15.33:35542) 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.767+0000 7f9629345640 1 -- 172.21.15.33:0/1795796936 learned_addr learned my addr 172.21.15.33:0/1795796936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.767+0000 7f962a347640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96241069c0 0x7f962419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.767+0000 7f9629b46640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96241009b0 0x7f962419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.768+0000 7f9629345640 1 -- 172.21.15.33:0/1795796936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96241009b0 msgr2=0x7f962419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.768+0000 7f9629345640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96241009b0 0x7f962419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.768+0000 7f9629345640 1 -- 172.21.15.33:0/1795796936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96241069c0 msgr2=0x7f962419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.768+0000 7f9629345640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96241069c0 0x7f962419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.768+0000 7f9629345640 1 -- 172.21.15.33:0/1795796936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962419f510 con 0x7f9624101370 2024-04-02T13:41:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.768+0000 7f962a347640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96241069c0 0x7f962419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.768+0000 7f9629345640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9624101370 0x7f962419aa60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9f0 tx=0x7f961400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.768+0000 7f961affd640 1 -- 172.21.15.33:0/1795796936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961400cd80 con 0x7f9624101370 2024-04-02T13:41:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.768+0000 7f961affd640 1 -- 172.21.15.33:0/1795796936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f96140054f0 con 0x7f9624101370 2024-04-02T13:41:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.768+0000 7f962bdaa640 1 -- 172.21.15.33:0/1795796936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962419f800 con 0x7f9624101370 2024-04-02T13:41:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.768+0000 7f961affd640 1 -- 172.21.15.33:0/1795796936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614010690 con 0x7f9624101370 2024-04-02T13:41:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.768+0000 7f962bdaa640 1 -- 172.21.15.33:0/1795796936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96241a7090 con 0x7f9624101370 2024-04-02T13:41:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.770+0000 7f961affd640 1 -- 172.21.15.33:0/1795796936 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9614004f20 con 0x7f9624101370 2024-04-02T13:41:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f962e98a740 1 Processor -- start 2024-04-02T13:41:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f962e98a740 1 -- start start 2024-04-02T13:41:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f962e98a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f962e98a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f85f7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f962e98a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85fa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f961affd640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f96100a3d20 0x7f96100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f962e98a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5564f8552830 con 0x5564f85ef1f0 2024-04-02T13:41:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f962e98a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5564f84fa820 con 0x5564f85f7dd0 2024-04-02T13:41:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f962e98a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5564f854e7e0 con 0x5564f85ee8d0 2024-04-02T13:41:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f9629b46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f85f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f962a347640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f96100a3d20 0x7f96100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f9629345640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85fa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f9629b46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f85f7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35556/0 (socket says 172.21.15.33:35556) 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f9629345640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85fa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:57880/0 (socket says 172.21.15.33:57880) 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f9629b46640 1 -- 172.21.15.33:0/4254058688 learned_addr learned my addr 172.21.15.33:0/4254058688 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f961affd640 1 -- 172.21.15.33:0/1795796936 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f96140d1010 con 0x7f9624101370 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.771+0000 7f962a347640 1 --2- 172.21.15.33:0/4254058688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.772+0000 7f9629b46640 1 -- 172.21.15.33:0/4254058688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 msgr2=0x5564f85eecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.772+0000 7f9629b46640 1 --2- 172.21.15.33:0/4254058688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85eecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.772+0000 7f9629b46640 1 -- 172.21.15.33:0/4254058688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 msgr2=0x5564f85fa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.772+0000 7f9629b46640 1 --2- 172.21.15.33:0/4254058688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85fa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.772+0000 7f9629b46640 1 -- 172.21.15.33:0/4254058688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564f83307d0 con 0x5564f85ef1f0 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.772+0000 7f9629345640 1 --2- 172.21.15.33:0/4254058688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85fa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.772+0000 7f962a347640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f96100a3d20 0x7f96100a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f961c005cc0 tx=0x7f961c032e90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.772+0000 7f9629b46640 1 --2- 172.21.15.33:0/4254058688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f85f7890 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9620009bf0 tx=0x7f96200113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d938f7ccf153398 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.772+0000 7f960ffff640 1 -- 172.21.15.33:0/4254058688 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620011e10 con 0x5564f85ef1f0 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.772+0000 7f960ffff640 1 -- 172.21.15.33:0/4254058688 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9620012920 con 0x5564f85ef1f0 2024-04-02T13:41:37.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.772+0000 7f960ffff640 1 -- 172.21.15.33:0/4254058688 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962000f9e0 con 0x5564f85ef1f0 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.773+0000 7f962e98a740 1 -- 172.21.15.33:0/4254058688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 msgr2=0x5564f85f7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.773+0000 7f962e98a740 1 --2- 172.21.15.33:0/4254058688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f85f7890 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9620009bf0 tx=0x7f96200113e0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.773+0000 7f962e98a740 1 -- 172.21.15.33:0/4254058688 shutdown_connections 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.773+0000 7f962e98a740 1 --2- 172.21.15.33:0/4254058688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85fa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.773+0000 7f962e98a740 1 --2- 172.21.15.33:0/4254058688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f85f7890 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.773+0000 7f962e98a740 1 --2- 172.21.15.33:0/4254058688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.773+0000 7f962e98a740 1 -- 172.21.15.33:0/4254058688 >> 172.21.15.33:0/4254058688 conn(0x5564f85e9000 msgr2=0x5564f85e93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.773+0000 7f962e98a740 1 -- 172.21.15.33:0/4254058688 shutdown_connections 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.773+0000 7f962e98a740 1 -- 172.21.15.33:0/4254058688 wait complete. 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.773+0000 7f962e98a740 1 -- start start 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.773+0000 7f962e98a740 1 Processor -- start 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.773+0000 7f962e98a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85f7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f962e98a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f85f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f962e98a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f962e98a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5564f85fabc0 con 0x5564f85ef1f0 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f962e98a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5564f850dd30 con 0x5564f85f7dd0 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f962e98a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5564f85fad40 con 0x5564f85ee8d0 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f962e98a740 10 client.0 _ll_register_callbacks cb 0x5564f850d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f9629b46640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f962a347640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f9629b46640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85f7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:51110/0 (socket says 172.21.15.33:51110) 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f9629b46640 1 -- 172.21.15.33:0/4002755250 learned_addr learned my addr 172.21.15.33:0/4002755250 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f9629345640 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f85f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f9629b46640 1 -- 172.21.15.33:0/4002755250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 msgr2=0x5564f85f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f9629b46640 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f9629b46640 1 -- 172.21.15.33:0/4002755250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 msgr2=0x5564f85f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f9629b46640 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f85f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f9629b46640 1 -- 172.21.15.33:0/4002755250 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564f82663f0 con 0x5564f85ee8d0 2024-04-02T13:41:37.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f962a347640 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:37.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f9629345640 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f85f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:37.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f9629b46640 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85f7660 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f962000f600 tx=0x7f9620012700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75e8f9fa4d45331a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:37.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f960d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-04-02T13:41:37.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.774+0000 7f960d7fa640 1 -- 172.21.15.33:0/4002755250 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962001f040 con 0x5564f85ee8d0 2024-04-02T13:41:37.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f960d7fa640 1 -- 172.21.15.33:0/4002755250 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9620012cd0 con 0x5564f85ee8d0 2024-04-02T13:41:37.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f962e98a740 1 -- 172.21.15.33:0/4002755250 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9620012920 con 0x5564f85ee8d0 2024-04-02T13:41:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f960d7fa640 1 -- 172.21.15.33:0/4002755250 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9620031050 con 0x5564f85ee8d0 2024-04-02T13:41:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f962e98a740 10 client.25723 fetch_fsmap learned FSMap version 29 2024-04-02T13:41:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f962e98a740 1 -- 172.21.15.33:0/4002755250 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5564f82260f0 con 0x5564f85ee8d0 2024-04-02T13:41:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f960d7fa640 1 -- 172.21.15.33:0/4002755250 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9620012920 con 0x5564f85ee8d0 2024-04-02T13:41:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f960d7fa640 1 -- 172.21.15.33:0/4002755250 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f9620022070 con 0x5564f85ee8d0 2024-04-02T13:41:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f962e98a740 10 client.25723 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:41:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f962e98a740 10 client.25723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f962e98a740 20 client.25723 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f962e98a740 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5564f8607cd0 0x5564f860a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f962e98a740 4 client.25723 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f962e98a740 1 -- 172.21.15.33:0/4002755250 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564f855fd70 con 0x5564f8607cd0 2024-04-02T13:41:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.775+0000 7f9629345640 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5564f8607cd0 0x5564f860a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.776+0000 7f9629345640 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5564f8607cd0 0x5564f860a0c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4effb77bb32d02 server_cookie=a8b1b90cb5139d57 in_seq=0 out_seq=0 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.776+0000 7f960d7fa640 10 client.25723 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.776+0000 7f960d7fa640 1 -- 172.21.15.33:0/4002755250 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5564f855fd70 con 0x5564f8607cd0 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f960d7fa640 10 client.25723 handle_command_reply: tid=0 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f962e98a740 1 client.25723 shutdown 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f962e98a740 1 -- 172.21.15.33:0/4002755250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 msgr2=0x5564f85f7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f962e98a740 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85f7660 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f962000f600 tx=0x7f9620012700 comp rx=0 tx=0).stop 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f962e98a740 1 -- 172.21.15.33:0/4002755250 shutdown_connections 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f962e98a740 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5564f8607cd0 0x5564f860a0c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f962e98a740 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f962e98a740 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f85f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f962e98a740 1 --2- 172.21.15.33:0/4002755250 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85f7660 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f960d7fa640 0 client.25723 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f962e98a740 1 -- 172.21.15.33:0/4002755250 >> 172.21.15.33:0/4002755250 conn(0x5564f85e9000 msgr2=0x5564f85ead50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f962e98a740 1 -- 172.21.15.33:0/4002755250 shutdown_connections 2024-04-02T13:41:37.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f962e98a740 1 -- 172.21.15.33:0/4002755250 wait complete. 2024-04-02T13:41:37.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.777+0000 7f962e98a740 20 client.25723 trim_cache size 0 max 16384 2024-04-02T13:41:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f962e98a740 1 Processor -- start 2024-04-02T13:41:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f962e98a740 1 -- start start 2024-04-02T13:41:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f962e98a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f8603a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f962e98a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f860f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f962e98a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f860fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f962e98a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96241b3ae0 con 0x5564f85ef1f0 2024-04-02T13:41:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f962e98a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96241b1320 con 0x5564f85f7dd0 2024-04-02T13:41:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f962e98a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f96241af150 con 0x5564f85ee8d0 2024-04-02T13:41:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f9629345640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f860f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f9629345640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f860f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35570/0 (socket says 172.21.15.33:35570) 2024-04-02T13:41:37.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f9629345640 1 -- 172.21.15.33:0/117839990 learned_addr learned my addr 172.21.15.33:0/117839990 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f9629b46640 1 --2- 172.21.15.33:0/117839990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f8603a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f9629345640 1 -- 172.21.15.33:0/117839990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 msgr2=0x5564f8603a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f9629345640 1 --2- 172.21.15.33:0/117839990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f8603a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f9629345640 1 -- 172.21.15.33:0/117839990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 msgr2=0x5564f860fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f9629345640 1 --2- 172.21.15.33:0/117839990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f860fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f9629345640 1 -- 172.21.15.33:0/117839990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564f85f5530 con 0x5564f85ef1f0 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f9629b46640 1 --2- 172.21.15.33:0/117839990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f8603a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.795+0000 7f9629345640 1 --2- 172.21.15.33:0/117839990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f860f780 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f961409cbd0 tx=0x7f96140d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b497d389652548ae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.796+0000 7f960f7fe640 1 -- 172.21.15.33:0/117839990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961409af40 con 0x5564f85ef1f0 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.796+0000 7f960f7fe640 1 -- 172.21.15.33:0/117839990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9614091070 con 0x5564f85ef1f0 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.796+0000 7f960f7fe640 1 -- 172.21.15.33:0/117839990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96140ce670 con 0x5564f85ef1f0 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.796+0000 7f962e98a740 1 -- 172.21.15.33:0/117839990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 msgr2=0x5564f860f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.796+0000 7f962e98a740 1 --2- 172.21.15.33:0/117839990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f860f780 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f961409cbd0 tx=0x7f96140d0040 comp rx=0 tx=0).stop 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.796+0000 7f962e98a740 1 -- 172.21.15.33:0/117839990 shutdown_connections 2024-04-02T13:41:37.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.796+0000 7f962e98a740 1 --2- 172.21.15.33:0/117839990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f860fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.796+0000 7f962e98a740 1 --2- 172.21.15.33:0/117839990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f860f780 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.796+0000 7f962e98a740 1 --2- 172.21.15.33:0/117839990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f8603a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.796+0000 7f962e98a740 1 -- 172.21.15.33:0/117839990 >> 172.21.15.33:0/117839990 conn(0x5564f85e9000 msgr2=0x5564f85f00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.796+0000 7f962e98a740 1 -- 172.21.15.33:0/117839990 shutdown_connections 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.796+0000 7f962e98a740 1 -- 172.21.15.33:0/117839990 wait complete. 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f962e98a740 1 -- start start 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f962e98a740 1 Processor -- start 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f962e98a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85fff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f962e98a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f86004d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f962e98a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f962e98a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96241af7e0 con 0x5564f85ef1f0 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f962e98a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96241b0c50 con 0x5564f85f7dd0 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f962e98a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f96241afeb0 con 0x5564f85ee8d0 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f962e98a740 10 client.0 _ll_register_callbacks cb 0x7f962000f440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:37.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f9629345640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f86004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f9629345640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f86004d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35574/0 (socket says 172.21.15.33:35574) 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f9629345640 1 -- 172.21.15.33:0/3720245324 learned_addr learned my addr 172.21.15.33:0/3720245324 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f9629b46640 1 --2- 172.21.15.33:0/3720245324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f962a347640 1 --2- 172.21.15.33:0/3720245324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f9629345640 1 -- 172.21.15.33:0/3720245324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 msgr2=0x5564f85fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f9629345640 1 --2- 172.21.15.33:0/3720245324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f9629345640 1 -- 172.21.15.33:0/3720245324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 msgr2=0x5564f85fcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f9629345640 1 --2- 172.21.15.33:0/3720245324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85fcf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.797+0000 7f9629345640 1 -- 172.21.15.33:0/3720245324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564f85ee2d0 con 0x5564f85ef1f0 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f9629345640 1 --2- 172.21.15.33:0/3720245324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f86004d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f96140097d0 tx=0x7f9614004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79fa79646ec4524 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f960d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f960d7fa640 1 -- 172.21.15.33:0/3720245324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96140b6070 con 0x5564f85ef1f0 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f960d7fa640 1 -- 172.21.15.33:0/3720245324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f96140992b0 con 0x5564f85ef1f0 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f962e98a740 1 -- 172.21.15.33:0/3720245324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9614091070 con 0x5564f85ef1f0 2024-04-02T13:41:37.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f960d7fa640 1 -- 172.21.15.33:0/3720245324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96140ce670 con 0x5564f85ef1f0 2024-04-02T13:41:37.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f962e98a740 10 client.16299 fetch_fsmap learned FSMap version 29 2024-04-02T13:41:37.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f962e98a740 1 -- 172.21.15.33:0/3720245324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5564f85e8e30 con 0x5564f85ef1f0 2024-04-02T13:41:37.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f960d7fa640 1 -- 172.21.15.33:0/3720245324 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9614091070 con 0x5564f85ef1f0 2024-04-02T13:41:37.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f960d7fa640 1 -- 172.21.15.33:0/3720245324 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f96140ceb40 con 0x5564f85ef1f0 2024-04-02T13:41:37.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f962e98a740 10 client.16299 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:41:37.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f962e98a740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:37.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f962e98a740 20 client.16299 populate_metadata read hostname 'smithi033' 2024-04-02T13:41:37.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f962e98a740 1 --2- 172.21.15.33:0/3720245324 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5564f866eb80 0x5564f8670fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:37.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f962e98a740 4 client.16299 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:41:37.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f962e98a740 1 -- 172.21.15.33:0/3720245324 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5564f855a0c0 con 0x5564f866eb80 2024-04-02T13:41:37.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.798+0000 7f9629b46640 1 --2- 172.21.15.33:0/3720245324 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5564f866eb80 0x5564f8670fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:37.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f960d7fa640 10 client.16299 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:37.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f9629b46640 1 --2- 172.21.15.33:0/3720245324 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5564f866eb80 0x5564f8670fa0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a764d13c7a3929e2 server_cookie=a13fd1a844be4527 in_seq=0 out_seq=0 2024-04-02T13:41:37.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f960d7fa640 1 -- 172.21.15.33:0/3720245324 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5564f855a0c0 con 0x5564f866eb80 2024-04-02T13:41:37.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f960d7fa640 10 client.16299 handle_command_reply: tid=0 2024-04-02T13:41:37.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f962e98a740 1 client.16299 shutdown 2024-04-02T13:41:37.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f962e98a740 1 -- 172.21.15.33:0/3720245324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 msgr2=0x5564f86004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f962e98a740 1 --2- 172.21.15.33:0/3720245324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f86004d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f96140097d0 tx=0x7f9614004990 comp rx=0 tx=0).stop 2024-04-02T13:41:37.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f962e98a740 1 -- 172.21.15.33:0/3720245324 shutdown_connections 2024-04-02T13:41:37.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f962e98a740 1 --2- 172.21.15.33:0/3720245324 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5564f866eb80 0x5564f8670fa0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f962e98a740 1 --2- 172.21.15.33:0/3720245324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5564f85f7dd0 0x5564f85fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f962e98a740 1 --2- 172.21.15.33:0/3720245324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564f85ef1f0 0x5564f86004d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f960d7fa640 0 client.16299 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:41:37.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f962e98a740 1 --2- 172.21.15.33:0/3720245324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5564f85ee8d0 0x5564f85fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f962e98a740 1 -- 172.21.15.33:0/3720245324 >> 172.21.15.33:0/3720245324 conn(0x5564f85e9000 msgr2=0x5564f85e79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:37.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f962e98a740 1 -- 172.21.15.33:0/3720245324 shutdown_connections 2024-04-02T13:41:37.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f962e98a740 1 -- 172.21.15.33:0/3720245324 wait complete. 2024-04-02T13:41:37.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.799+0000 7f962e98a740 20 client.16299 trim_cache size 0 max 16384 2024-04-02T13:41:37.811 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:41:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "scrub active (595 inodes in the stack)", 2024-04-02T13:41:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:41:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: "0975e94a-c470-4832-b39c-2ff36d1785b4": { 2024-04-02T13:41:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:41:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "0975e94a-c470-4832-b39c-2ff36d1785b4", 2024-04-02T13:41:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:41:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:41:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:41:37.812 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:41:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.800+0000 7f962bdaa640 1 -- 172.21.15.33:0/1795796936 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f96100a3d20 msgr2=0x7f96100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.800+0000 7f962bdaa640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f96100a3d20 0x7f96100a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f961c005cc0 tx=0x7f961c032e90 comp rx=0 tx=0).stop 2024-04-02T13:41:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.800+0000 7f962bdaa640 1 -- 172.21.15.33:0/1795796936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9624101370 msgr2=0x7f962419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.800+0000 7f962bdaa640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9624101370 0x7f962419aa60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9f0 tx=0x7f961400edf0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.800+0000 7f962bdaa640 1 -- 172.21.15.33:0/1795796936 shutdown_connections 2024-04-02T13:41:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.800+0000 7f962bdaa640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f96100a3d20 0x7f96100a61e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.800+0000 7f962bdaa640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96241069c0 0x7f962419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.800+0000 7f962bdaa640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9624101370 0x7f962419aa60 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.800+0000 7f962bdaa640 1 --2- 172.21.15.33:0/1795796936 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96241009b0 0x7f962419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.800+0000 7f962bdaa640 1 -- 172.21.15.33:0/1795796936 >> 172.21.15.33:0/1795796936 conn(0x7f9624076d80 msgr2=0x7f962410d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.800+0000 7f962bdaa640 1 -- 172.21.15.33:0/1795796936 shutdown_connections 2024-04-02T13:41:37.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:41:37.800+0000 7f962bdaa640 1 -- 172.21.15.33:0/1795796936 wait complete. 2024-04-02T13:41:37.814 INFO:tasks.cephfs.filesystem:scrub status for tag:0975e94a-c470-4832-b39c-2ff36d1785b4 - {'path': '/', 'tag': '0975e94a-c470-4832-b39c-2ff36d1785b4', 'options': 'recursive,force'} 2024-04-02T13:41:37.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:37 smithi145 ceph-mon[35646]: pgmap v88: 129 pgs: 129 active+clean; 207 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1022 KiB/s wr, 153 op/s 2024-04-02T13:41:37.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:37 smithi050 ceph-mon[35524]: pgmap v88: 129 pgs: 129 active+clean; 207 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1022 KiB/s wr, 153 op/s 2024-04-02T13:41:38.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:37 smithi033 ceph-mon[32390]: pgmap v88: 129 pgs: 129 active+clean; 207 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1022 KiB/s wr, 153 op/s 2024-04-02T13:41:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:38 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:41:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:38 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:38 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:38 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:38 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:38 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:38 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:38 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:38 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:38 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:38.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:38 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:38.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:38 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:41:38.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:38 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:38 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:41:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:38 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:38 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:38 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:38 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:38 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:38.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:38 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:38.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:38 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:38.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:38 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:38.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:38 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:38.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:38 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:38.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:38 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:41:38.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:38 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:39.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:38 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:41:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:38 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:38 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:38 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:38 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:38 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:38 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:38 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:38 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:38 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:38 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:38 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:41:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:38 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:39.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:39 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:39.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:39 smithi145 ceph-mon[35646]: pgmap v89: 129 pgs: 129 active+clean; 208 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 300 op/s 2024-04-02T13:41:39.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:39 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:39.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:39 smithi050 ceph-mon[35524]: pgmap v89: 129 pgs: 129 active+clean; 208 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 300 op/s 2024-04-02T13:41:40.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:39 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:40.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:39 smithi033 ceph-mon[32390]: pgmap v89: 129 pgs: 129 active+clean; 208 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 300 op/s 2024-04-02T13:41:40.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:40 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:41:40.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:40 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:41:41.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:40 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:41:41.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:41 smithi145 ceph-mon[35646]: pgmap v90: 129 pgs: 129 active+clean; 208 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 238 op/s 2024-04-02T13:41:41.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:41 smithi050 ceph-mon[35524]: pgmap v90: 129 pgs: 129 active+clean; 208 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 238 op/s 2024-04-02T13:41:42.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:41 smithi033 ceph-mon[32390]: pgmap v90: 129 pgs: 129 active+clean; 208 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 238 op/s 2024-04-02T13:41:42.312 INFO:tasks.workunit.client.0.smithi033.stderr:+ tar xf doc.tar 2024-04-02T13:41:44.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:43 smithi050 ceph-mon[35524]: pgmap v91: 129 pgs: 129 active+clean; 207 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 355 op/s 2024-04-02T13:41:44.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:43 smithi033 ceph-mon[32390]: pgmap v91: 129 pgs: 129 active+clean; 207 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 355 op/s 2024-04-02T13:41:44.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:43 smithi145 ceph-mon[35646]: pgmap v91: 129 pgs: 129 active+clean; 207 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 355 op/s 2024-04-02T13:41:46.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:45 smithi050 ceph-mon[35524]: pgmap v92: 129 pgs: 129 active+clean; 207 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 264 op/s 2024-04-02T13:41:46.245 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:45 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:41:46.246 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:45 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:41:46.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:45 smithi033 ceph-mon[32390]: pgmap v92: 129 pgs: 129 active+clean; 207 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 264 op/s 2024-04-02T13:41:46.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:45 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:41:46.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:45 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:41:46.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:45 smithi145 ceph-mon[35646]: pgmap v92: 129 pgs: 129 active+clean; 207 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 264 op/s 2024-04-02T13:41:46.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:45 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:41:46.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:45 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:41:47.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:46 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:41:47.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:46 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:41:47.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:46 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:41:48.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:47 smithi050 ceph-mon[35524]: pgmap v93: 129 pgs: 129 active+clean; 207 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 264 op/s 2024-04-02T13:41:48.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:47 smithi033 ceph-mon[32390]: pgmap v93: 129 pgs: 129 active+clean; 207 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 264 op/s 2024-04-02T13:41:48.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:47 smithi145 ceph-mon[35646]: pgmap v93: 129 pgs: 129 active+clean; 207 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 264 op/s 2024-04-02T13:41:50.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:49 smithi050 ceph-mon[35524]: pgmap v94: 129 pgs: 129 active+clean; 210 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 450 op/s 2024-04-02T13:41:50.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:49 smithi033 ceph-mon[32390]: pgmap v94: 129 pgs: 129 active+clean; 210 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 450 op/s 2024-04-02T13:41:50.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:49 smithi145 ceph-mon[35646]: pgmap v94: 129 pgs: 129 active+clean; 210 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 450 op/s 2024-04-02T13:41:52.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:51 smithi033 ceph-mon[32390]: pgmap v95: 129 pgs: 129 active+clean; 210 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.3 MiB/s wr, 302 op/s 2024-04-02T13:41:52.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:51 smithi145 ceph-mon[35646]: pgmap v95: 129 pgs: 129 active+clean; 210 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.3 MiB/s wr, 302 op/s 2024-04-02T13:41:52.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:51 smithi050 ceph-mon[35524]: pgmap v95: 129 pgs: 129 active+clean; 210 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.3 MiB/s wr, 302 op/s 2024-04-02T13:41:54.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:53 smithi033 ceph-mon[32390]: pgmap v96: 129 pgs: 129 active+clean; 225 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 419 op/s 2024-04-02T13:41:54.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:53 smithi145 ceph-mon[35646]: pgmap v96: 129 pgs: 129 active+clean; 225 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 419 op/s 2024-04-02T13:41:54.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:53 smithi050 ceph-mon[35524]: pgmap v96: 129 pgs: 129 active+clean; 225 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 419 op/s 2024-04-02T13:41:55.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:54 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:41:55.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:54 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:41:55.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:54 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:41:55.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:54 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:41:55.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:54 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:41:55.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:54 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:41:55.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:54 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:41:55.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:54 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:41:55.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:54 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:41:55.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:54 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:41:55.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:54 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:41:55.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:54 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:41:56.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:55 smithi033 ceph-mon[32390]: pgmap v97: 129 pgs: 129 active+clean; 225 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-04-02T13:41:56.378 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:55 smithi145 ceph-mon[35646]: pgmap v97: 129 pgs: 129 active+clean; 225 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-04-02T13:41:56.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:55 smithi050 ceph-mon[35524]: pgmap v97: 129 pgs: 129 active+clean; 225 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-04-02T13:41:57.250 INFO:tasks.workunit.client.0.smithi033.stderr:+ sync 2024-04-02T13:41:58.082 INFO:tasks.workunit.client.0.smithi033.stderr:+ tar c linux-4.0.5 2024-04-02T13:41:58.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:41:58 smithi033 ceph-mon[32390]: pgmap v98: 129 pgs: 129 active+clean; 225 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-04-02T13:41:58.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:41:58 smithi145 ceph-mon[35646]: pgmap v98: 129 pgs: 129 active+clean; 225 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-04-02T13:41:58.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:41:58 smithi050 ceph-mon[35524]: pgmap v98: 129 pgs: 129 active+clean; 225 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-04-02T13:42:00.252 INFO:tasks.workunit.client.0.smithi033.stdout:Ok 2024-04-02T13:42:00.253 INFO:tasks.workunit.client.0.smithi033.stderr:+ echo Ok 2024-04-02T13:42:00.254 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-02T13:42:00.254 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-02T13:42:00.279 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:00 smithi033 ceph-mon[32390]: pgmap v99: 129 pgs: 129 active+clean; 236 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 381 op/s 2024-04-02T13:42:00.295 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:00 smithi050 ceph-mon[35524]: pgmap v99: 129 pgs: 129 active+clean; 236 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 381 op/s 2024-04-02T13:42:00.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:00 smithi145 ceph-mon[35646]: pgmap v99: 129 pgs: 129 active+clean; 236 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 381 op/s 2024-04-02T13:42:02.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:02 smithi145 ceph-mon[35646]: pgmap v100: 129 pgs: 129 active+clean; 236 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-04-02T13:42:02.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:02 smithi050 ceph-mon[35524]: pgmap v100: 129 pgs: 129 active+clean; 236 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-04-02T13:42:02.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:02 smithi033 ceph-mon[32390]: pgmap v100: 129 pgs: 129 active+clean; 236 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-04-02T13:42:04.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:04 smithi145 ceph-mon[35646]: pgmap v101: 129 pgs: 129 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 239 op/s 2024-04-02T13:42:04.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:04 smithi050 ceph-mon[35524]: pgmap v101: 129 pgs: 129 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 239 op/s 2024-04-02T13:42:04.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:04 smithi033 ceph-mon[32390]: pgmap v101: 129 pgs: 129 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 239 op/s 2024-04-02T13:42:06.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:06 smithi145 ceph-mon[35646]: pgmap v102: 129 pgs: 129 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-04-02T13:42:06.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:06 smithi050 ceph-mon[35524]: pgmap v102: 129 pgs: 129 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-04-02T13:42:06.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:06 smithi033 ceph-mon[32390]: pgmap v102: 129 pgs: 129 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-04-02T13:42:07.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:42:07.817 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:42:07.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.924+0000 7f285247b640 1 -- 172.21.15.33:0/628326175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284c101aa0 msgr2=0x7f284c101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:07.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.924+0000 7f285247b640 1 --2- 172.21.15.33:0/628326175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284c101aa0 0x7f284c101f20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2840009930 tx=0x7f284002bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:07.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.924+0000 7f285247b640 1 -- 172.21.15.33:0/628326175 shutdown_connections 2024-04-02T13:42:07.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.924+0000 7f285247b640 1 --2- 172.21.15.33:0/628326175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f284c102460 0x7f284c10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.924+0000 7f285247b640 1 --2- 172.21.15.33:0/628326175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284c101aa0 0x7f284c101f20 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.924+0000 7f285247b640 1 --2- 172.21.15.33:0/628326175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284c107ab0 0x7f284c107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.924+0000 7f285247b640 1 -- 172.21.15.33:0/628326175 >> 172.21.15.33:0/628326175 conn(0x7f284c0fd530 msgr2=0x7f284c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:07.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.924+0000 7f285247b640 1 -- 172.21.15.33:0/628326175 shutdown_connections 2024-04-02T13:42:07.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.924+0000 7f285247b640 1 -- 172.21.15.33:0/628326175 wait complete. 2024-04-02T13:42:07.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.925+0000 7f285247b640 1 Processor -- start 2024-04-02T13:42:07.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.925+0000 7f285247b640 1 -- start start 2024-04-02T13:42:07.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.925+0000 7f285247b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f284c101aa0 0x7f284c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.925+0000 7f285247b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284c102460 0x7f284c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.925+0000 7f285247b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284c107ab0 0x7f284c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.925+0000 7f285247b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f284c111e90 con 0x7f284c107ab0 2024-04-02T13:42:07.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.925+0000 7f285247b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f284c111d10 con 0x7f284c102460 2024-04-02T13:42:07.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.925+0000 7f285247b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f284c112010 con 0x7f284c101aa0 2024-04-02T13:42:07.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.925+0000 7f2850a18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284c107ab0 0x7f284c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.925+0000 7f2850a18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284c107ab0 0x7f284c19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37340/0 (socket says 172.21.15.33:37340) 2024-04-02T13:42:07.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.925+0000 7f284b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284c102460 0x7f284c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.925+0000 7f2850a18640 1 -- 172.21.15.33:0/2467120512 learned_addr learned my addr 172.21.15.33:0/2467120512 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:07.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.926+0000 7f284bfff640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f284c101aa0 0x7f284c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.926+0000 7f2850a18640 1 -- 172.21.15.33:0/2467120512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f284c101aa0 msgr2=0x7f284c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:07.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.926+0000 7f2850a18640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f284c101aa0 0x7f284c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.926+0000 7f2850a18640 1 -- 172.21.15.33:0/2467120512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284c102460 msgr2=0x7f284c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:07.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.926+0000 7f2850a18640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284c102460 0x7f284c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.926+0000 7f2850a18640 1 -- 172.21.15.33:0/2467120512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c19d390 con 0x7f284c107ab0 2024-04-02T13:42:07.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.926+0000 7f284b7fe640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284c102460 0x7f284c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:07.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.926+0000 7f2850a18640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284c107ab0 0x7f284c19cc60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f284400cce0 tx=0x7f284400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:07.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.926+0000 7f28497fa640 1 -- 172.21.15.33:0/2467120512 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284400f450 con 0x7f284c107ab0 2024-04-02T13:42:07.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.926+0000 7f28497fa640 1 -- 172.21.15.33:0/2467120512 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f284400fac0 con 0x7f284c107ab0 2024-04-02T13:42:07.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.926+0000 7f285247b640 1 -- 172.21.15.33:0/2467120512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c19d680 con 0x7f284c107ab0 2024-04-02T13:42:07.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.926+0000 7f28497fa640 1 -- 172.21.15.33:0/2467120512 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844018be0 con 0x7f284c107ab0 2024-04-02T13:42:07.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.926+0000 7f285247b640 1 -- 172.21.15.33:0/2467120512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c1a4f10 con 0x7f284c107ab0 2024-04-02T13:42:07.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.928+0000 7f28497fa640 1 -- 172.21.15.33:0/2467120512 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f284400fc70 con 0x7f284c107ab0 2024-04-02T13:42:07.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f2855068740 1 Processor -- start 2024-04-02T13:42:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f2855068740 1 -- start start 2024-04-02T13:42:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f28497fa640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f28340a3d20 0x7f28340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f2855068740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9158d0 0x55caee915cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f2855068740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9161f0 0x55caee91e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f2855068740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee91edd0 0x55caee9211c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f2855068740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55caee879830 con 0x55caee9158d0 2024-04-02T13:42:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f2855068740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55caee821820 con 0x55caee91edd0 2024-04-02T13:42:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f2855068740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55caee8757e0 con 0x55caee9161f0 2024-04-02T13:42:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f284b7fe640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f28340a3d20 0x7f28340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f28497fa640 1 -- 172.21.15.33:0/2467120512 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f28440d1250 con 0x7f284c107ab0 2024-04-02T13:42:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f284bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9158d0 0x55caee915cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f284bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9158d0 0x55caee915cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37350/0 (socket says 172.21.15.33:37350) 2024-04-02T13:42:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f284bfff640 1 -- 172.21.15.33:0/3443071277 learned_addr learned my addr 172.21.15.33:0/3443071277 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f284b7fe640 1 --2- 172.21.15.33:0/3443071277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9161f0 0x55caee91e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f2850a18640 1 --2- 172.21.15.33:0/3443071277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee91edd0 0x55caee9211c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f284bfff640 1 -- 172.21.15.33:0/3443071277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9161f0 msgr2=0x55caee91e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f284bfff640 1 --2- 172.21.15.33:0/3443071277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9161f0 0x55caee91e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f284bfff640 1 -- 172.21.15.33:0/3443071277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee91edd0 msgr2=0x55caee9211c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f284bfff640 1 --2- 172.21.15.33:0/3443071277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee91edd0 0x55caee9211c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f284bfff640 1 -- 172.21.15.33:0/3443071277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caee6577d0 con 0x55caee9158d0 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f284b7fe640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f28340a3d20 0x7f28340a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2840000c00 tx=0x7f2840004b60 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.929+0000 7f284bfff640 1 --2- 172.21.15.33:0/3443071277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9158d0 0x55caee915cb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f283c009bf0 tx=0x7f283c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d6ddddd1a71c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.930+0000 7f28327fc640 1 -- 172.21.15.33:0/3443071277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c012450 con 0x55caee9158d0 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.930+0000 7f28327fc640 1 -- 172.21.15.33:0/3443071277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f283c012ac0 con 0x55caee9158d0 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.930+0000 7f28327fc640 1 -- 172.21.15.33:0/3443071277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c00faf0 con 0x55caee9158d0 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.930+0000 7f2855068740 1 -- 172.21.15.33:0/3443071277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9158d0 msgr2=0x55caee915cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.930+0000 7f2855068740 1 --2- 172.21.15.33:0/3443071277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9158d0 0x55caee915cb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f283c009bf0 tx=0x7f283c0117b0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.930+0000 7f2855068740 1 -- 172.21.15.33:0/3443071277 shutdown_connections 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.930+0000 7f2855068740 1 --2- 172.21.15.33:0/3443071277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee91edd0 0x55caee9211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.930+0000 7f2855068740 1 --2- 172.21.15.33:0/3443071277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9161f0 0x55caee91e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.930+0000 7f2855068740 1 --2- 172.21.15.33:0/3443071277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9158d0 0x55caee915cb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.930+0000 7f2855068740 1 -- 172.21.15.33:0/3443071277 >> 172.21.15.33:0/3443071277 conn(0x55caee910000 msgr2=0x55caee9103e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.930+0000 7f2855068740 1 -- 172.21.15.33:0/3443071277 shutdown_connections 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.930+0000 7f2855068740 1 -- 172.21.15.33:0/3443071277 wait complete. 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f2855068740 1 -- start start 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f2855068740 1 Processor -- start 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f2855068740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee9158d0 0x55caee91e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f2855068740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9161f0 0x55caee91b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f2855068740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee91edd0 0x55caee91bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f2855068740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55caee921bc0 con 0x55caee91edd0 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f2855068740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55caee834d30 con 0x55caee9158d0 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f2855068740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55caee921d40 con 0x55caee9161f0 2024-04-02T13:42:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f2855068740 10 client.0 _ll_register_callbacks cb 0x55caee834720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f284bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee9158d0 0x55caee91e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f2850a18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee91edd0 0x55caee91bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f284bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee9158d0 0x55caee91e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:46930/0 (socket says 172.21.15.33:46930) 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f284bfff640 1 -- 172.21.15.33:0/615019892 learned_addr learned my addr 172.21.15.33:0/615019892 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f2850a18640 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee91edd0 0x55caee91bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37364/0 (socket says 172.21.15.33:37364) 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.931+0000 7f284b7fe640 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9161f0 0x55caee91b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f2850a18640 1 -- 172.21.15.33:0/615019892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9161f0 msgr2=0x55caee91b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f2850a18640 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9161f0 0x55caee91b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f2850a18640 1 -- 172.21.15.33:0/615019892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee9158d0 msgr2=0x55caee91e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f2850a18640 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee9158d0 0x55caee91e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f2850a18640 1 -- 172.21.15.33:0/615019892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caee58d3f0 con 0x55caee91edd0 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f284bfff640 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee9158d0 0x55caee91e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f2850a18640 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee91edd0 0x55caee91bf20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f28440ce630 tx=0x7f284409d580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131d4bff845c0b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f282b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f282b7fe640 1 -- 172.21.15.33:0/615019892 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28440a1230 con 0x55caee91edd0 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f282b7fe640 1 -- 172.21.15.33:0/615019892 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f28440a5660 con 0x55caee91edd0 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f2855068740 1 -- 172.21.15.33:0/615019892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f283c012ac0 con 0x55caee91edd0 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f282b7fe640 1 -- 172.21.15.33:0/615019892 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f28440a5af0 con 0x55caee91edd0 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f2855068740 10 client.16311 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f2855068740 1 -- 172.21.15.33:0/615019892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55caee54d0f0 con 0x55caee91edd0 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.932+0000 7f282b7fe640 1 -- 172.21.15.33:0/615019892 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f283c012ac0 con 0x55caee91edd0 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.933+0000 7f282b7fe640 1 -- 172.21.15.33:0/615019892 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2844098c60 con 0x55caee91edd0 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.933+0000 7f2855068740 10 client.16311 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.933+0000 7f2855068740 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.933+0000 7f2855068740 20 client.16311 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.933+0000 7f2855068740 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55caee92edf0 0x55caee931200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.933+0000 7f2855068740 4 client.16311 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.933+0000 7f2855068740 1 -- 172.21.15.33:0/615019892 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caee886d70 con 0x55caee92edf0 2024-04-02T13:42:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.933+0000 7f284bfff640 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55caee92edf0 0x55caee931200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.933+0000 7f282b7fe640 10 client.16311 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.933+0000 7f284bfff640 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55caee92edf0 0x55caee931200 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dcf21551befca0a server_cookie=abc38b05b6acbd77 in_seq=0 out_seq=0 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f282b7fe640 1 -- 172.21.15.33:0/615019892 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55caee886d70 con 0x55caee92edf0 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f282b7fe640 10 client.16311 handle_command_reply: tid=0 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f2855068740 1 client.16311 shutdown 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f2855068740 1 -- 172.21.15.33:0/615019892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee91edd0 msgr2=0x55caee91bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f2855068740 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee91edd0 0x55caee91bf20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f28440ce630 tx=0x7f284409d580 comp rx=0 tx=0).stop 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f2855068740 1 -- 172.21.15.33:0/615019892 shutdown_connections 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f2855068740 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55caee92edf0 0x55caee931200 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f2855068740 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee91edd0 0x55caee91bf20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f2855068740 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9161f0 0x55caee91b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f2855068740 1 --2- 172.21.15.33:0/615019892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee9158d0 0x55caee91e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f282b7fe640 0 client.16311 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f2855068740 1 -- 172.21.15.33:0/615019892 >> 172.21.15.33:0/615019892 conn(0x55caee910000 msgr2=0x55caee911d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f2855068740 1 -- 172.21.15.33:0/615019892 shutdown_connections 2024-04-02T13:42:07.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f2855068740 1 -- 172.21.15.33:0/615019892 wait complete. 2024-04-02T13:42:07.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.934+0000 7f2855068740 20 client.16311 trim_cache size 0 max 16384 2024-04-02T13:42:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f2855068740 1 Processor -- start 2024-04-02T13:42:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f2855068740 1 -- start start 2024-04-02T13:42:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f2855068740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9158d0 0x55caee92aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f2855068740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 0x55caee911ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f2855068740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee91edd0 0x55caee91e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f2855068740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b1840 con 0x55caee9161f0 2024-04-02T13:42:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f2855068740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1af050 con 0x55caee91edd0 2024-04-02T13:42:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f2855068740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1ace80 con 0x55caee9158d0 2024-04-02T13:42:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f284b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 0x55caee911ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f284b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 0x55caee911ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37378/0 (socket says 172.21.15.33:37378) 2024-04-02T13:42:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f284b7fe640 1 -- 172.21.15.33:0/1874727309 learned_addr learned my addr 172.21.15.33:0/1874727309 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f2850a18640 1 --2- 172.21.15.33:0/1874727309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee91edd0 0x55caee91e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f284bfff640 1 --2- 172.21.15.33:0/1874727309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9158d0 0x55caee92aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f284b7fe640 1 -- 172.21.15.33:0/1874727309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9158d0 msgr2=0x55caee92aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f284b7fe640 1 --2- 172.21.15.33:0/1874727309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9158d0 0x55caee92aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f284b7fe640 1 -- 172.21.15.33:0/1874727309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee91edd0 msgr2=0x55caee91e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f284b7fe640 1 --2- 172.21.15.33:0/1874727309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee91edd0 0x55caee91e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f284b7fe640 1 -- 172.21.15.33:0/1874727309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caee90fe30 con 0x55caee9161f0 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.953+0000 7f284bfff640 1 --2- 172.21.15.33:0/1874727309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9158d0 0x55caee92aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.954+0000 7f284b7fe640 1 --2- 172.21.15.33:0/1874727309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 0x55caee911ce0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2840009230 tx=0x7f284004b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2993090046832 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.954+0000 7f2831ffb640 1 -- 172.21.15.33:0/1874727309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284002c5d0 con 0x55caee9161f0 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.954+0000 7f2831ffb640 1 -- 172.21.15.33:0/1874727309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f284002cc40 con 0x55caee9161f0 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.954+0000 7f2831ffb640 1 -- 172.21.15.33:0/1874727309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840003ce0 con 0x55caee9161f0 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.954+0000 7f2855068740 1 -- 172.21.15.33:0/1874727309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 msgr2=0x55caee911ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.954+0000 7f2855068740 1 --2- 172.21.15.33:0/1874727309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 0x55caee911ce0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2840009230 tx=0x7f284004b000 comp rx=0 tx=0).stop 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.954+0000 7f2855068740 1 -- 172.21.15.33:0/1874727309 shutdown_connections 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.954+0000 7f2855068740 1 --2- 172.21.15.33:0/1874727309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee91edd0 0x55caee91e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.954+0000 7f2855068740 1 --2- 172.21.15.33:0/1874727309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 0x55caee911ce0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.954+0000 7f2855068740 1 --2- 172.21.15.33:0/1874727309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee9158d0 0x55caee92aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.954+0000 7f2855068740 1 -- 172.21.15.33:0/1874727309 >> 172.21.15.33:0/1874727309 conn(0x55caee910000 msgr2=0x55caee91a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f2855068740 1 -- 172.21.15.33:0/1874727309 shutdown_connections 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f2855068740 1 -- 172.21.15.33:0/1874727309 wait complete. 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f2855068740 1 -- start start 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f2855068740 1 Processor -- start 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f2855068740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee9158d0 0x55caee9185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f2855068740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 0x55caee918af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f2855068740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee91edd0 0x55caee919030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f2855068740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1ad510 con 0x55caee9161f0 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f2855068740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1ae980 con 0x55caee9158d0 2024-04-02T13:42:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f2855068740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1adbe0 con 0x55caee91edd0 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f2855068740 10 client.0 _ll_register_callbacks cb 0x7f284409ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f284b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 0x55caee918af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f284bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee9158d0 0x55caee9185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f284b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 0x55caee918af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37384/0 (socket says 172.21.15.33:37384) 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.955+0000 7f284b7fe640 1 -- 172.21.15.33:0/2218595575 learned_addr learned my addr 172.21.15.33:0/2218595575 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f2850a18640 1 --2- 172.21.15.33:0/2218595575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee91edd0 0x55caee919030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f284b7fe640 1 -- 172.21.15.33:0/2218595575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee91edd0 msgr2=0x55caee919030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f284b7fe640 1 --2- 172.21.15.33:0/2218595575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee91edd0 0x55caee919030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f284b7fe640 1 -- 172.21.15.33:0/2218595575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee9158d0 msgr2=0x55caee9185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f284b7fe640 1 --2- 172.21.15.33:0/2218595575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee9158d0 0x55caee9185b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f284b7fe640 1 -- 172.21.15.33:0/2218595575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caee9211c0 con 0x55caee9161f0 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f284bfff640 1 --2- 172.21.15.33:0/2218595575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee9158d0 0x55caee9185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f284b7fe640 1 --2- 172.21.15.33:0/2218595575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 0x55caee918af0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f284002c170 tx=0x7f2840009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80a6f0769d8aed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f282b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f282b7fe640 1 -- 172.21.15.33:0/2218595575 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840003e70 con 0x55caee9161f0 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f282b7fe640 1 -- 172.21.15.33:0/2218595575 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f28400035f0 con 0x55caee9161f0 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f2855068740 1 -- 172.21.15.33:0/2218595575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f284002cc40 con 0x55caee9161f0 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f282b7fe640 1 -- 172.21.15.33:0/2218595575 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f28400037f0 con 0x55caee9161f0 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f2855068740 10 client.16317 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f2855068740 1 -- 172.21.15.33:0/2218595575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55caee9952f0 con 0x55caee9161f0 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.956+0000 7f282b7fe640 1 -- 172.21.15.33:0/2218595575 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f284002cc40 con 0x55caee9161f0 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.957+0000 7f282b7fe640 1 -- 172.21.15.33:0/2218595575 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f284004d040 con 0x55caee9161f0 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.957+0000 7f2855068740 10 client.16317 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.957+0000 7f2855068740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.957+0000 7f2855068740 20 client.16317 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.957+0000 7f2855068740 1 --2- 172.21.15.33:0/2218595575 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55caee995b20 0x55caee997f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.957+0000 7f2855068740 4 client.16317 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.957+0000 7f2855068740 1 -- 172.21.15.33:0/2218595575 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55caee8810c0 con 0x55caee995b20 2024-04-02T13:42:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.957+0000 7f284bfff640 1 --2- 172.21.15.33:0/2218595575 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55caee995b20 0x55caee997f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.957+0000 7f284bfff640 1 --2- 172.21.15.33:0/2218595575 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55caee995b20 0x55caee997f40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1acde4b31b8c5389 server_cookie=efbeded0f9352378 in_seq=0 out_seq=0 2024-04-02T13:42:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.957+0000 7f282b7fe640 10 client.16317 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.957+0000 7f282b7fe640 1 -- 172.21.15.33:0/2218595575 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55caee8810c0 con 0x55caee995b20 2024-04-02T13:42:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.957+0000 7f282b7fe640 10 client.16317 handle_command_reply: tid=0 2024-04-02T13:42:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.957+0000 7f2855068740 1 client.16317 shutdown 2024-04-02T13:42:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f2855068740 1 -- 172.21.15.33:0/2218595575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 msgr2=0x55caee918af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f2855068740 1 --2- 172.21.15.33:0/2218595575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 0x55caee918af0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f284002c170 tx=0x7f2840009260 comp rx=0 tx=0).stop 2024-04-02T13:42:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f2855068740 1 -- 172.21.15.33:0/2218595575 shutdown_connections 2024-04-02T13:42:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f2855068740 1 --2- 172.21.15.33:0/2218595575 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55caee995b20 0x55caee997f40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f2855068740 1 --2- 172.21.15.33:0/2218595575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55caee91edd0 0x55caee919030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f2855068740 1 --2- 172.21.15.33:0/2218595575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55caee9161f0 0x55caee918af0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f2855068740 1 --2- 172.21.15.33:0/2218595575 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55caee9158d0 0x55caee9185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f282b7fe640 0 client.16317 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f2855068740 1 -- 172.21.15.33:0/2218595575 >> 172.21.15.33:0/2218595575 conn(0x55caee910000 msgr2=0x55caee90f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f2855068740 1 -- 172.21.15.33:0/2218595575 shutdown_connections 2024-04-02T13:42:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f2855068740 1 -- 172.21.15.33:0/2218595575 wait complete. 2024-04-02T13:42:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f2855068740 20 client.16317 trim_cache size 0 max 16384 2024-04-02T13:42:07.967 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:42:07.967 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:42:07.967 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:42:07.967 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:42:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f285247b640 1 -- 172.21.15.33:0/2467120512 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f28340a3d20 msgr2=0x7f28340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f285247b640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f28340a3d20 0x7f28340a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2840000c00 tx=0x7f2840004b60 comp rx=0 tx=0).stop 2024-04-02T13:42:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f285247b640 1 -- 172.21.15.33:0/2467120512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284c107ab0 msgr2=0x7f284c19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f285247b640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284c107ab0 0x7f284c19cc60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f284400cce0 tx=0x7f284400a620 comp rx=0 tx=0).stop 2024-04-02T13:42:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f285247b640 1 -- 172.21.15.33:0/2467120512 shutdown_connections 2024-04-02T13:42:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f285247b640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f28340a3d20 0x7f28340a61e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f285247b640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f284c107ab0 0x7f284c19cc60 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f285247b640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f284c102460 0x7f284c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f285247b640 1 --2- 172.21.15.33:0/2467120512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f284c101aa0 0x7f284c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f285247b640 1 -- 172.21.15.33:0/2467120512 >> 172.21.15.33:0/2467120512 conn(0x7f284c0fd530 msgr2=0x7f284c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f285247b640 1 -- 172.21.15.33:0/2467120512 shutdown_connections 2024-04-02T13:42:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:07.958+0000 7f285247b640 1 -- 172.21.15.33:0/2467120512 wait complete. 2024-04-02T13:42:07.972 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:42:07.972 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:42:08.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.075+0000 7eff0fd11640 1 -- 172.21.15.33:0/2789845899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff081069b0 msgr2=0x7eff08106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.075+0000 7eff0fd11640 1 --2- 172.21.15.33:0/2789845899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff081069b0 0x7eff08106d90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7eff04009930 tx=0x7eff0402bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:08.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.075+0000 7eff0fd11640 1 -- 172.21.15.33:0/2789845899 shutdown_connections 2024-04-02T13:42:08.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.075+0000 7eff0fd11640 1 --2- 172.21.15.33:0/2789845899 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff08101360 0x7eff0810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.075+0000 7eff0fd11640 1 --2- 172.21.15.33:0/2789845899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff081009a0 0x7eff08100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.075+0000 7eff0fd11640 1 --2- 172.21.15.33:0/2789845899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff081069b0 0x7eff08106d90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.075+0000 7eff0fd11640 1 -- 172.21.15.33:0/2789845899 >> 172.21.15.33:0/2789845899 conn(0x7eff08076d80 msgr2=0x7eff08077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.075+0000 7eff0fd11640 1 -- 172.21.15.33:0/2789845899 shutdown_connections 2024-04-02T13:42:08.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.075+0000 7eff0fd11640 1 -- 172.21.15.33:0/2789845899 wait complete. 2024-04-02T13:42:08.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0fd11640 1 Processor -- start 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0fd11640 1 -- start start 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0fd11640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff081009a0 0x7eff08197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0fd11640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff08101360 0x7eff08197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0fd11640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff081069b0 0x7eff08197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0fd11640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff08110d60 con 0x7eff08101360 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0fd11640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff08110be0 con 0x7eff081069b0 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0fd11640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7eff08110ee0 con 0x7eff081009a0 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0d2ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff08101360 0x7eff08197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0d2ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff08101360 0x7eff08197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37406/0 (socket says 172.21.15.33:37406) 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0d2ac640 1 -- 172.21.15.33:0/212922738 learned_addr learned my addr 172.21.15.33:0/212922738 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0e2ae640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff081069b0 0x7eff08197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0daad640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff081009a0 0x7eff08197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0d2ac640 1 -- 172.21.15.33:0/212922738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff081009a0 msgr2=0x7eff08197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0d2ac640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff081009a0 0x7eff08197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0d2ac640 1 -- 172.21.15.33:0/212922738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff081069b0 msgr2=0x7eff08197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0d2ac640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff081069b0 0x7eff08197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.076+0000 7eff0d2ac640 1 -- 172.21.15.33:0/212922738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff081a68a0 con 0x7eff08101360 2024-04-02T13:42:08.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.077+0000 7eff0d2ac640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff08101360 0x7eff08197570 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7efef800e9f0 tx=0x7efef800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.077+0000 7efefeffd640 1 -- 172.21.15.33:0/212922738 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef800cd80 con 0x7eff08101360 2024-04-02T13:42:08.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.077+0000 7eff0fd11640 1 -- 172.21.15.33:0/212922738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff081a6b90 con 0x7eff08101360 2024-04-02T13:42:08.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.077+0000 7efefeffd640 1 -- 172.21.15.33:0/212922738 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efef8010900 con 0x7eff08101360 2024-04-02T13:42:08.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.077+0000 7efefeffd640 1 -- 172.21.15.33:0/212922738 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef80189e0 con 0x7eff08101360 2024-04-02T13:42:08.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.077+0000 7eff0fd11640 1 -- 172.21.15.33:0/212922738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff081a70e0 con 0x7eff08101360 2024-04-02T13:42:08.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.078+0000 7eff0fd11640 1 -- 172.21.15.33:0/212922738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff081020e0 con 0x7eff08101360 2024-04-02T13:42:08.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.079+0000 7efefeffd640 1 -- 172.21.15.33:0/212922738 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efef8010ab0 con 0x7eff08101360 2024-04-02T13:42:08.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.079+0000 7efefeffd640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efef40a3d20 0x7efef40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.079+0000 7eff0daad640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efef40a3d20 0x7efef40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.079+0000 7efefeffd640 1 -- 172.21.15.33:0/212922738 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7efef80d1390 con 0x7eff08101360 2024-04-02T13:42:08.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.080+0000 7eff0daad640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efef40a3d20 0x7efef40a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff040096c0 tx=0x7eff04005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.083+0000 7efefeffd640 1 -- 172.21.15.33:0/212922738 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efef809d6c0 con 0x7eff08101360 2024-04-02T13:42:08.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:08 smithi033 ceph-mon[32390]: pgmap v103: 129 pgs: 129 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-04-02T13:42:08.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.318+0000 7eff0fd11640 1 -- 172.21.15.33:0/212922738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff08068d40 con 0x7eff08101360 2024-04-02T13:42:08.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.318+0000 7efefeffd640 1 -- 172.21.15.33:0/212922738 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11882 (secure 0 0 0) 0x7efef809cfc0 con 0x7eff08101360 2024-04-02T13:42:08.321 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:42:08.321 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:41:39.222860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:42:08.322 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-02T13:42:08.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.321+0000 7eff0fd11640 1 -- 172.21.15.33:0/212922738 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efef40a3d20 msgr2=0x7efef40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.321+0000 7eff0fd11640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efef40a3d20 0x7efef40a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff040096c0 tx=0x7eff04005fd0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.321+0000 7eff0fd11640 1 -- 172.21.15.33:0/212922738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff08101360 msgr2=0x7eff08197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.321+0000 7eff0fd11640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff08101360 0x7eff08197570 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7efef800e9f0 tx=0x7efef800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.321+0000 7eff0fd11640 1 -- 172.21.15.33:0/212922738 shutdown_connections 2024-04-02T13:42:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.321+0000 7eff0fd11640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efef40a3d20 0x7efef40a61e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.321+0000 7eff0fd11640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff081069b0 0x7eff08197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.321+0000 7eff0fd11640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff08101360 0x7eff08197570 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.321+0000 7eff0fd11640 1 --2- 172.21.15.33:0/212922738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff081009a0 0x7eff08197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.321+0000 7eff0fd11640 1 -- 172.21.15.33:0/212922738 >> 172.21.15.33:0/212922738 conn(0x7eff08076d80 msgr2=0x7eff080fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.321+0000 7eff0fd11640 1 -- 172.21.15.33:0/212922738 shutdown_connections 2024-04-02T13:42:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.321+0000 7eff0fd11640 1 -- 172.21.15.33:0/212922738 wait complete. 2024-04-02T13:42:08.335 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:42:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.442+0000 7f4f7b060640 1 -- 172.21.15.33:0/4174143471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f74101a90 msgr2=0x7f4f74101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.442+0000 7f4f7b060640 1 --2- 172.21.15.33:0/4174143471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f74101a90 0x7f4f74101f10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009930 tx=0x7f4f5c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.442+0000 7f4f7b060640 1 -- 172.21.15.33:0/4174143471 shutdown_connections 2024-04-02T13:42:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.442+0000 7f4f7b060640 1 --2- 172.21.15.33:0/4174143471 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f74102450 0x7f4f7410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.442+0000 7f4f7b060640 1 --2- 172.21.15.33:0/4174143471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f74101a90 0x7f4f74101f10 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.442+0000 7f4f7b060640 1 --2- 172.21.15.33:0/4174143471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f74107aa0 0x7f4f74107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.442+0000 7f4f7b060640 1 -- 172.21.15.33:0/4174143471 >> 172.21.15.33:0/4174143471 conn(0x7f4f74078f00 msgr2=0x7f4f74079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.442+0000 7f4f7b060640 1 -- 172.21.15.33:0/4174143471 shutdown_connections 2024-04-02T13:42:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.442+0000 7f4f7b060640 1 -- 172.21.15.33:0/4174143471 wait complete. 2024-04-02T13:42:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f7b060640 1 Processor -- start 2024-04-02T13:42:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f7b060640 1 -- start start 2024-04-02T13:42:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f7b060640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f74101a90 0x7f4f74073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f7b060640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f74102450 0x7f4f74074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f7b060640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f74107aa0 0x7f4f74074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f7b060640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74111ea0 con 0x7f4f74107aa0 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f7b060640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74111d20 con 0x7f4f74102450 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f7b060640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74112020 con 0x7f4f74101a90 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f795fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f74107aa0 0x7f4f74074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f74102450 0x7f4f74074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f74102450 0x7f4f74074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:46974/0 (socket says 172.21.15.33:46974) 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f73fff640 1 -- 172.21.15.33:0/18063294 learned_addr learned my addr 172.21.15.33:0/18063294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f795fd640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f74107aa0 0x7f4f74074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37422/0 (socket says 172.21.15.33:37422) 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.443+0000 7f4f78dfc640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f74101a90 0x7f4f74073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.444+0000 7f4f795fd640 1 -- 172.21.15.33:0/18063294 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f74101a90 msgr2=0x7f4f74073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.444+0000 7f4f795fd640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f74101a90 0x7f4f74073e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.444+0000 7f4f795fd640 1 -- 172.21.15.33:0/18063294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f74102450 msgr2=0x7f4f74074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.444+0000 7f4f795fd640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f74102450 0x7f4f74074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.444+0000 7f4f795fd640 1 -- 172.21.15.33:0/18063294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f741a68e0 con 0x7f4f74107aa0 2024-04-02T13:42:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.444+0000 7f4f73fff640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f74102450 0x7f4f74074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.444+0000 7f4f795fd640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f74107aa0 0x7f4f74074880 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00be20 tx=0x7f4f6c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.444+0000 7f4f71ffb640 1 -- 172.21.15.33:0/18063294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c019070 con 0x7f4f74107aa0 2024-04-02T13:42:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.444+0000 7f4f71ffb640 1 -- 172.21.15.33:0/18063294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4f6c014910 con 0x7f4f74107aa0 2024-04-02T13:42:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.444+0000 7f4f7b060640 1 -- 172.21.15.33:0/18063294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f741a6bd0 con 0x7f4f74107aa0 2024-04-02T13:42:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.444+0000 7f4f71ffb640 1 -- 172.21.15.33:0/18063294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c012890 con 0x7f4f74107aa0 2024-04-02T13:42:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.444+0000 7f4f7b060640 1 -- 172.21.15.33:0/18063294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f741a70d0 con 0x7f4f74107aa0 2024-04-02T13:42:08.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f71ffb640 1 -- 172.21.15.33:0/18063294 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4f6c01c070 con 0x7f4f74107aa0 2024-04-02T13:42:08.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f7dc4a740 1 Processor -- start 2024-04-02T13:42:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f7dc4a740 1 -- start start 2024-04-02T13:42:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f7dc4a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 0x5618b6a12b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f7dc4a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 0x5618b6a1b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f7dc4a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a1bca0 0x5618b6a1e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f7dc4a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5618b69767b0 con 0x5618b6a127a0 2024-04-02T13:42:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f7dc4a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5618b6915570 con 0x5618b6a1bca0 2024-04-02T13:42:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f7dc4a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5618b6972760 con 0x5618b6a130c0 2024-04-02T13:42:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 0x5618b6a12b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 0x5618b6a12b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37428/0 (socket says 172.21.15.33:37428) 2024-04-02T13:42:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f73fff640 1 -- 172.21.15.33:0/3146744660 learned_addr learned my addr 172.21.15.33:0/3146744660 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f71ffb640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4f600a3d50 0x7f4f600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f73fff640 1 --2- 172.21.15.33:0/3146744660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a1bca0 0x5618b6a1e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f795fd640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4f600a3d50 0x7f4f600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f78dfc640 1 --2- 172.21.15.33:0/3146744660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 0x5618b6a1b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.447+0000 7f4f71ffb640 1 -- 172.21.15.33:0/18063294 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4f6c0d02d0 con 0x7f4f74107aa0 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.448+0000 7f4f73fff640 1 -- 172.21.15.33:0/3146744660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 msgr2=0x5618b6a1b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.448+0000 7f4f73fff640 1 --2- 172.21.15.33:0/3146744660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 0x5618b6a1b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.448+0000 7f4f73fff640 1 -- 172.21.15.33:0/3146744660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a1bca0 msgr2=0x5618b6a1e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.448+0000 7f4f73fff640 1 --2- 172.21.15.33:0/3146744660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a1bca0 0x5618b6a1e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.448+0000 7f4f73fff640 1 -- 172.21.15.33:0/3146744660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618b668a3f0 con 0x5618b6a127a0 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.448+0000 7f4f73fff640 1 --2- 172.21.15.33:0/3146744660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 0x5618b6a12b80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009900 tx=0x7f4f5c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e371bf0df3a91f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.448+0000 7f4f795fd640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4f600a3d50 0x7f4f600a6210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c098520 tx=0x7f4f6c04b040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.448+0000 7f4f6affd640 1 -- 172.21.15.33:0/3146744660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c004320 con 0x5618b6a127a0 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.448+0000 7f4f6affd640 1 -- 172.21.15.33:0/3146744660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4f5c0044c0 con 0x5618b6a127a0 2024-04-02T13:42:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.448+0000 7f4f6affd640 1 -- 172.21.15.33:0/3146744660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c02cc10 con 0x5618b6a127a0 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.449+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/3146744660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 msgr2=0x5618b6a12b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.449+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/3146744660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 0x5618b6a12b80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009900 tx=0x7f4f5c005b50 comp rx=0 tx=0).stop 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.449+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/3146744660 shutdown_connections 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.449+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/3146744660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a1bca0 0x5618b6a1e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.449+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/3146744660 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 0x5618b6a1b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.449+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/3146744660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 0x5618b6a12b80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.449+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/3146744660 >> 172.21.15.33:0/3146744660 conn(0x5618b6a0ced0 msgr2=0x5618b6a0d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.449+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/3146744660 shutdown_connections 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.449+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/3146744660 wait complete. 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.449+0000 7f4f7dc4a740 1 -- start start 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.449+0000 7f4f7dc4a740 1 Processor -- start 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f7dc4a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 0x5618b6a1b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f7dc4a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a130c0 0x5618b6a18aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f7dc4a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a1bca0 0x5618b6a191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f7dc4a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5618b6a1ea30 con 0x5618b6a127a0 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f7dc4a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5618b6931e80 con 0x5618b6a130c0 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f7dc4a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5618b6a1ebb0 con 0x5618b6a1bca0 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f7dc4a740 10 client.0 _ll_register_callbacks cb 0x5618b696daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 0x5618b6a1b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 0x5618b6a1b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37434/0 (socket says 172.21.15.33:37434) 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f73fff640 1 -- 172.21.15.33:0/2972627344 learned_addr learned my addr 172.21.15.33:0/2972627344 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f78dfc640 1 --2- 172.21.15.33:0/2972627344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a130c0 0x5618b6a18aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f73fff640 1 --2- 172.21.15.33:0/2972627344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a1bca0 0x5618b6a191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f73fff640 1 -- 172.21.15.33:0/2972627344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a1bca0 msgr2=0x5618b6a191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f73fff640 1 --2- 172.21.15.33:0/2972627344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a1bca0 0x5618b6a191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f73fff640 1 -- 172.21.15.33:0/2972627344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a130c0 msgr2=0x5618b6a18aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f73fff640 1 --2- 172.21.15.33:0/2972627344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a130c0 0x5618b6a18aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f73fff640 1 -- 172.21.15.33:0/2972627344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618b67547d0 con 0x5618b6a127a0 2024-04-02T13:42:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f73fff640 1 --2- 172.21.15.33:0/2972627344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 0x5618b6a1b710 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0059a0 tx=0x7f4f5c02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc7f7dcafa79356 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f53fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f53fff640 1 -- 172.21.15.33:0/2972627344 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c005280 con 0x5618b6a127a0 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f53fff640 1 -- 172.21.15.33:0/2972627344 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4f5c005420 con 0x5618b6a127a0 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.450+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/2972627344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5618b6983cf0 con 0x5618b6a127a0 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f53fff640 1 -- 172.21.15.33:0/2972627344 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f5c005620 con 0x5618b6a127a0 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f7dc4a740 10 client.16335 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/2972627344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5618b6a2ab20 con 0x5618b6a127a0 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f53fff640 1 -- 172.21.15.33:0/2972627344 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x5618b6983cf0 con 0x5618b6a127a0 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f53fff640 1 -- 172.21.15.33:0/2972627344 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4f5c042780 con 0x5618b6a127a0 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f7dc4a740 10 client.16335 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f7dc4a740 10 client.16335 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f7dc4a740 20 client.16335 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/2972627344 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5618b6a2be70 0x5618b6a2e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f7dc4a740 4 client.16335 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:08.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/2972627344 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618b697e040 con 0x5618b6a2be70 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f78dfc640 1 --2- 172.21.15.33:0/2972627344 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5618b6a2be70 0x5618b6a2e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f78dfc640 1 --2- 172.21.15.33:0/2972627344 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5618b6a2be70 0x5618b6a2e260 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af388d0216838980 server_cookie=4965e45cdb50ebc5 in_seq=0 out_seq=0 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.451+0000 7f4f53fff640 10 client.16335 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.452+0000 7f4f53fff640 1 -- 172.21.15.33:0/2972627344 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618b697e040 con 0x5618b6a2be70 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.452+0000 7f4f53fff640 10 client.16335 handle_command_reply: tid=0 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.452+0000 7f4f7dc4a740 1 client.16335 shutdown 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.452+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/2972627344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 msgr2=0x5618b6a1b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.452+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/2972627344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 0x5618b6a1b710 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0059a0 tx=0x7f4f5c02c850 comp rx=0 tx=0).stop 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.452+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/2972627344 shutdown_connections 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.452+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/2972627344 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5618b6a2be70 0x5618b6a2e260 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.452+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/2972627344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a1bca0 0x5618b6a191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.452+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/2972627344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a130c0 0x5618b6a18aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.452+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/2972627344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a127a0 0x5618b6a1b710 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.452+0000 7f4f53fff640 0 client.16335 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.452+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/2972627344 >> 172.21.15.33:0/2972627344 conn(0x5618b6a0ced0 msgr2=0x5618b6a0ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.453+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/2972627344 shutdown_connections 2024-04-02T13:42:08.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.453+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/2972627344 wait complete. 2024-04-02T13:42:08.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.453+0000 7f4f7dc4a740 20 client.16335 trim_cache size 0 max 16384 2024-04-02T13:42:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.467+0000 7f4f7dc4a740 1 Processor -- start 2024-04-02T13:42:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.467+0000 7f4f7dc4a740 1 -- start start 2024-04-02T13:42:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f7dc4a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a127a0 0x5618b6a17c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f7dc4a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 0x5618b6a33860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f7dc4a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 0x5618b6a33da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f7dc4a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4f741b3a80 con 0x5618b6a1bca0 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f7dc4a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f741b1290 con 0x5618b6a127a0 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f7dc4a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4f741af0c0 con 0x5618b6a130c0 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 0x5618b6a33da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 0x5618b6a33da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37442/0 (socket says 172.21.15.33:37442) 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f73fff640 1 -- 172.21.15.33:0/1492084618 learned_addr learned my addr 172.21.15.33:0/1492084618 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f73fff640 1 --2- 172.21.15.33:0/1492084618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a127a0 0x5618b6a17c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f73fff640 1 -- 172.21.15.33:0/1492084618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 msgr2=0x5618b6a33860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f73fff640 1 --2- 172.21.15.33:0/1492084618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 0x5618b6a33860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f73fff640 1 -- 172.21.15.33:0/1492084618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a127a0 msgr2=0x5618b6a17c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f73fff640 1 --2- 172.21.15.33:0/1492084618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a127a0 0x5618b6a17c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f73fff640 1 -- 172.21.15.33:0/1492084618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618b6a19550 con 0x5618b6a1bca0 2024-04-02T13:42:08.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f73fff640 1 --2- 172.21.15.33:0/1492084618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 0x5618b6a33da0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c005970 tx=0x7f4f5c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f41e327c06e2f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f6a7fc640 1 -- 172.21.15.33:0/1492084618 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0041b0 con 0x5618b6a1bca0 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f6a7fc640 1 -- 172.21.15.33:0/1492084618 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4f5c042470 con 0x5618b6a1bca0 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.468+0000 7f4f6a7fc640 1 -- 172.21.15.33:0/1492084618 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c003a60 con 0x5618b6a1bca0 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.469+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/1492084618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 msgr2=0x5618b6a33da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.469+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/1492084618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 0x5618b6a33da0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c005970 tx=0x7f4f5c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.469+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/1492084618 shutdown_connections 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.469+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/1492084618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 0x5618b6a33da0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.469+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/1492084618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 0x5618b6a33860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.469+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/1492084618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a127a0 0x5618b6a17c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.469+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/1492084618 >> 172.21.15.33:0/1492084618 conn(0x5618b6a0ced0 msgr2=0x5618b6a0b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.469+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/1492084618 shutdown_connections 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.469+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/1492084618 wait complete. 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.469+0000 7f4f7dc4a740 1 -- start start 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.469+0000 7f4f7dc4a740 1 Processor -- start 2024-04-02T13:42:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f7dc4a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a127a0 0x5618b6a1c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f7dc4a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 0x5618b6a1cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f7dc4a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 0x5618b6a22bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f7dc4a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4f741af750 con 0x5618b6a1bca0 2024-04-02T13:42:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f7dc4a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f741b0bc0 con 0x5618b6a127a0 2024-04-02T13:42:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f7dc4a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4f741afe20 con 0x5618b6a130c0 2024-04-02T13:42:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f7dc4a740 10 client.0 _ll_register_callbacks cb 0x7f4f5c004d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 0x5618b6a22bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 0x5618b6a22bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37444/0 (socket says 172.21.15.33:37444) 2024-04-02T13:42:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f73fff640 1 -- 172.21.15.33:0/121573798 learned_addr learned my addr 172.21.15.33:0/121573798 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f73fff640 1 --2- 172.21.15.33:0/121573798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a127a0 0x5618b6a1c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f78dfc640 1 --2- 172.21.15.33:0/121573798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 0x5618b6a1cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f73fff640 1 -- 172.21.15.33:0/121573798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 msgr2=0x5618b6a1cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f73fff640 1 --2- 172.21.15.33:0/121573798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 0x5618b6a1cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f73fff640 1 -- 172.21.15.33:0/121573798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a127a0 msgr2=0x5618b6a1c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f73fff640 1 --2- 172.21.15.33:0/121573798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a127a0 0x5618b6a1c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f73fff640 1 -- 172.21.15.33:0/121573798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618b6a2e5d0 con 0x5618b6a1bca0 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f73fff640 1 --2- 172.21.15.33:0/121573798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 0x5618b6a22bc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x5618b6a1d7f0 tx=0x7f4f5c004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df8bbda3207df87 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f53fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f53fff640 1 -- 172.21.15.33:0/121573798 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c042620 con 0x5618b6a1bca0 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f53fff640 1 -- 172.21.15.33:0/121573798 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4f5c0427c0 con 0x5618b6a1bca0 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.470+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/121573798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f5c042470 con 0x5618b6a1bca0 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.471+0000 7f4f53fff640 1 -- 172.21.15.33:0/121573798 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f5c0429c0 con 0x5618b6a1bca0 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.471+0000 7f4f7dc4a740 10 client.16341 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.471+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/121573798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5618b6a24230 con 0x5618b6a1bca0 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.471+0000 7f4f53fff640 1 -- 172.21.15.33:0/121573798 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4f5c042470 con 0x5618b6a1bca0 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.471+0000 7f4f53fff640 1 -- 172.21.15.33:0/121573798 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4f5c034850 con 0x5618b6a1bca0 2024-04-02T13:42:08.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.471+0000 7f4f7dc4a740 10 client.16341 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.471+0000 7f4f7dc4a740 10 client.16341 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.471+0000 7f4f7dc4a740 20 client.16341 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.471+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/121573798 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5618b6a920d0 0x5618b6a944c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.471+0000 7f4f7dc4a740 4 client.16341 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.471+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/121573798 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618b68fb6b0 con 0x5618b6a920d0 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.471+0000 7f4f78dfc640 1 --2- 172.21.15.33:0/121573798 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5618b6a920d0 0x5618b6a944c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f78dfc640 1 --2- 172.21.15.33:0/121573798 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5618b6a920d0 0x5618b6a944c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8c709a9237d002a server_cookie=442d26964fcd35e3 in_seq=0 out_seq=0 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f53fff640 10 client.16341 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f53fff640 1 -- 172.21.15.33:0/121573798 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618b68fb6b0 con 0x5618b6a920d0 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f53fff640 10 client.16341 handle_command_reply: tid=0 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f7dc4a740 1 client.16341 shutdown 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/121573798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 msgr2=0x5618b6a22bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/121573798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 0x5618b6a22bc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x5618b6a1d7f0 tx=0x7f4f5c004c30 comp rx=0 tx=0).stop 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/121573798 shutdown_connections 2024-04-02T13:42:08.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/121573798 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5618b6a920d0 0x5618b6a944c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/121573798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618b6a1bca0 0x5618b6a22bc0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f53fff640 0 client.16341 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:42:08.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/121573798 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5618b6a130c0 0x5618b6a1cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f7dc4a740 1 --2- 172.21.15.33:0/121573798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5618b6a127a0 0x5618b6a1c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/121573798 >> 172.21.15.33:0/121573798 conn(0x5618b6a0ced0 msgr2=0x5618b6a17a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/121573798 shutdown_connections 2024-04-02T13:42:08.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f7dc4a740 1 -- 172.21.15.33:0/121573798 wait complete. 2024-04-02T13:42:08.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.472+0000 7f4f7dc4a740 20 client.16341 trim_cache size 0 max 16384 2024-04-02T13:42:08.481 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:42:08.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.473+0000 7f4f7b060640 1 -- 172.21.15.33:0/18063294 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4f600a3d50 msgr2=0x7f4f600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.473+0000 7f4f7b060640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4f600a3d50 0x7f4f600a6210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c098520 tx=0x7f4f6c04b040 comp rx=0 tx=0).stop 2024-04-02T13:42:08.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.473+0000 7f4f7b060640 1 -- 172.21.15.33:0/18063294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f74107aa0 msgr2=0x7f4f74074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.473+0000 7f4f7b060640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f74107aa0 0x7f4f74074880 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00be20 tx=0x7f4f6c00bf10 comp rx=0 tx=0).stop 2024-04-02T13:42:08.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.473+0000 7f4f7b060640 1 -- 172.21.15.33:0/18063294 shutdown_connections 2024-04-02T13:42:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.473+0000 7f4f7b060640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4f600a3d50 0x7f4f600a6210 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.473+0000 7f4f7b060640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f74107aa0 0x7f4f74074880 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.473+0000 7f4f7b060640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f74102450 0x7f4f74074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.473+0000 7f4f7b060640 1 --2- 172.21.15.33:0/18063294 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f74101a90 0x7f4f74073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.473+0000 7f4f7b060640 1 -- 172.21.15.33:0/18063294 >> 172.21.15.33:0/18063294 conn(0x7f4f74078f00 msgr2=0x7f4f741044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.473+0000 7f4f7b060640 1 -- 172.21.15.33:0/18063294 shutdown_connections 2024-04-02T13:42:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.473+0000 7f4f7b060640 1 -- 172.21.15.33:0/18063294 wait complete. 2024-04-02T13:42:08.483 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:08 smithi145 ceph-mon[35646]: pgmap v103: 129 pgs: 129 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-04-02T13:42:08.486 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:42:08.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:08 smithi050 ceph-mon[35524]: pgmap v103: 129 pgs: 129 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-04-02T13:42:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.594+0000 7f62fba06640 1 -- 172.21.15.33:0/3168603880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f4107b50 msgr2=0x7f62f4107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.594+0000 7f62fba06640 1 --2- 172.21.15.33:0/3168603880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f4107b50 0x7f62f4107f30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009930 tx=0x7f62e402bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.594+0000 7f62fba06640 1 -- 172.21.15.33:0/3168603880 shutdown_connections 2024-04-02T13:42:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.594+0000 7f62fba06640 1 --2- 172.21.15.33:0/3168603880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62f40696c0 0x7f62f410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.594+0000 7f62fba06640 1 --2- 172.21.15.33:0/3168603880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62f4068d00 0x7f62f4069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.594+0000 7f62fba06640 1 --2- 172.21.15.33:0/3168603880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f4107b50 0x7f62f4107f30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.594+0000 7f62fba06640 1 -- 172.21.15.33:0/3168603880 >> 172.21.15.33:0/3168603880 conn(0x7f62f4077b00 msgr2=0x7f62f4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.594+0000 7f62fba06640 1 -- 172.21.15.33:0/3168603880 shutdown_connections 2024-04-02T13:42:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.594+0000 7f62fba06640 1 -- 172.21.15.33:0/3168603880 wait complete. 2024-04-02T13:42:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62fba06640 1 Processor -- start 2024-04-02T13:42:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62fba06640 1 -- start start 2024-04-02T13:42:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62fba06640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f4068d00 0x7f62f4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62fba06640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62f40696c0 0x7f62f4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62fba06640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62f4107b50 0x7f62f419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62fba06640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f62f4111e70 con 0x7f62f4068d00 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62fba06640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f62f4111cf0 con 0x7f62f4107b50 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62fba06640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f62f4111ff0 con 0x7f62f40696c0 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62f97a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f4068d00 0x7f62f4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62f97a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f4068d00 0x7f62f4198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37466/0 (socket says 172.21.15.33:37466) 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62f97a2640 1 -- 172.21.15.33:0/1360240011 learned_addr learned my addr 172.21.15.33:0/1360240011 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62f9fa3640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62f4107b50 0x7f62f419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62f8fa1640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62f40696c0 0x7f62f4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62f97a2640 1 -- 172.21.15.33:0/1360240011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62f40696c0 msgr2=0x7f62f4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62f97a2640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62f40696c0 0x7f62f4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62f97a2640 1 -- 172.21.15.33:0/1360240011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62f4107b50 msgr2=0x7f62f419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62f97a2640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62f4107b50 0x7f62f419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.595+0000 7f62f97a2640 1 -- 172.21.15.33:0/1360240011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f419d400 con 0x7f62f4068d00 2024-04-02T13:42:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.596+0000 7f62f97a2640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f4068d00 0x7f62f4198410 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009900 tx=0x7f62e402ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.596+0000 7f62ea7fc640 1 -- 172.21.15.33:0/1360240011 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4043020 con 0x7f62f4068d00 2024-04-02T13:42:08.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.596+0000 7f62fba06640 1 -- 172.21.15.33:0/1360240011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f419d690 con 0x7f62f4068d00 2024-04-02T13:42:08.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.596+0000 7f62ea7fc640 1 -- 172.21.15.33:0/1360240011 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f62e40054d0 con 0x7f62f4068d00 2024-04-02T13:42:08.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.596+0000 7f62ea7fc640 1 -- 172.21.15.33:0/1360240011 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4003a80 con 0x7f62f4068d00 2024-04-02T13:42:08.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.596+0000 7f62fba06640 1 -- 172.21.15.33:0/1360240011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f41a4f80 con 0x7f62f4068d00 2024-04-02T13:42:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.598+0000 7f62ea7fc640 1 -- 172.21.15.33:0/1360240011 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62e4046020 con 0x7f62f4068d00 2024-04-02T13:42:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.598+0000 7f62fe5ee740 1 Processor -- start 2024-04-02T13:42:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.598+0000 7f62fe5ee740 1 -- start start 2024-04-02T13:42:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62ea7fc640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f62e00a3d20 0x7f62e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62fe5ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b4785b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62fe5ee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47860c0 0x55b7b478e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62fe5ee740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b478eca0 0x55b7b4791090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62fe5ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b7b46e97b0 con 0x55b7b47857a0 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62fe5ee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b7b4688570 con 0x55b7b47860c0 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62fe5ee740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b7b46e5760 con 0x55b7b478eca0 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62f9fa3640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f62e00a3d20 0x7f62e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62f97a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b4785b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62f8fa1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47860c0 0x55b7b478e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62f97a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b4785b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37470/0 (socket says 172.21.15.33:37470) 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62f97a2640 1 -- 172.21.15.33:0/2780525222 learned_addr learned my addr 172.21.15.33:0/2780525222 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62ea7fc640 1 -- 172.21.15.33:0/1360240011 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f62e40f2600 con 0x7f62f4068d00 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62f9fa3640 1 --2- 172.21.15.33:0/2780525222 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b478eca0 0x55b7b4791090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62f8fa1640 1 -- 172.21.15.33:0/2780525222 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b478eca0 msgr2=0x55b7b4791090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62f8fa1640 1 --2- 172.21.15.33:0/2780525222 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b478eca0 0x55b7b4791090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62f8fa1640 1 -- 172.21.15.33:0/2780525222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 msgr2=0x55b7b4785b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62f8fa1640 1 --2- 172.21.15.33:0/2780525222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b4785b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62f8fa1640 1 -- 172.21.15.33:0/2780525222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b43fd3f0 con 0x55b7b47860c0 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62f9fa3640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f62e00a3d20 0x7f62e00a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f62ec005fd0 tx=0x7f62ec012ad0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.599+0000 7f62f8fa1640 1 --2- 172.21.15.33:0/2780525222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47860c0 0x55b7b478e760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62f00058c0 tx=0x7f62f000af10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e18abaa92fae36d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.600+0000 7f62df7fe640 1 -- 172.21.15.33:0/2780525222 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0004020 con 0x55b7b47860c0 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.600+0000 7f62df7fe640 1 -- 172.21.15.33:0/2780525222 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f62f00041c0 con 0x55b7b47860c0 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.600+0000 7f62df7fe640 1 -- 172.21.15.33:0/2780525222 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f00186a0 con 0x55b7b47860c0 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.600+0000 7f62fe5ee740 1 -- 172.21.15.33:0/2780525222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47860c0 msgr2=0x55b7b478e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.600+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/2780525222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47860c0 0x55b7b478e760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62f00058c0 tx=0x7f62f000af10 comp rx=0 tx=0).stop 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.600+0000 7f62fe5ee740 1 -- 172.21.15.33:0/2780525222 shutdown_connections 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.600+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/2780525222 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b478eca0 0x55b7b4791090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.600+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/2780525222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47860c0 0x55b7b478e760 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.600+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/2780525222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b4785b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.600+0000 7f62fe5ee740 1 -- 172.21.15.33:0/2780525222 >> 172.21.15.33:0/2780525222 conn(0x55b7b477fed0 msgr2=0x55b7b47802b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.600+0000 7f62fe5ee740 1 -- 172.21.15.33:0/2780525222 shutdown_connections 2024-04-02T13:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.600+0000 7f62fe5ee740 1 -- 172.21.15.33:0/2780525222 wait complete. 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.601+0000 7f62fe5ee740 1 -- start start 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.601+0000 7f62fe5ee740 1 Processor -- start 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.601+0000 7f62fe5ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b478e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.601+0000 7f62fe5ee740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b47860c0 0x55b7b478ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.601+0000 7f62fe5ee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b478eca0 0x55b7b478c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.601+0000 7f62fe5ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b7b4791a90 con 0x55b7b47857a0 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.601+0000 7f62fe5ee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b7b46a4e80 con 0x55b7b478eca0 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.601+0000 7f62fe5ee740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b7b4791c10 con 0x55b7b47860c0 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.601+0000 7f62fe5ee740 10 client.0 _ll_register_callbacks cb 0x55b7b46e0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.601+0000 7f62f97a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b478e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.601+0000 7f62f97a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b478e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37482/0 (socket says 172.21.15.33:37482) 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.601+0000 7f62f97a2640 1 -- 172.21.15.33:0/4190790804 learned_addr learned my addr 172.21.15.33:0/4190790804 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.601+0000 7f62f9fa3640 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b478eca0 0x55b7b478c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62f8fa1640 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b47860c0 0x55b7b478ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62f97a2640 1 -- 172.21.15.33:0/4190790804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b47860c0 msgr2=0x55b7b478ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62f97a2640 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b47860c0 0x55b7b478ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62f97a2640 1 -- 172.21.15.33:0/4190790804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b478eca0 msgr2=0x55b7b478c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62f97a2640 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b478eca0 0x55b7b478c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62f97a2640 1 -- 172.21.15.33:0/4190790804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7b44c77d0 con 0x55b7b47857a0 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62f9fa3640 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b478eca0 0x55b7b478c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62f97a2640 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b478e6b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f62e4006010 tx=0x7f62e4004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c232dc34296b3f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62dcff9640 1 -- 172.21.15.33:0/4190790804 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e40bea40 con 0x55b7b47857a0 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62dcff9640 1 -- 172.21.15.33:0/4190790804 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f62e40df000 con 0x55b7b47857a0 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4190790804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62f00041c0 con 0x55b7b47857a0 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62dcff9640 1 -- 172.21.15.33:0/4190790804 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f62e40d7ac0 con 0x55b7b47857a0 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62fe5ee740 10 client.16353 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4190790804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b7b4791a90 con 0x55b7b47857a0 2024-04-02T13:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62dcff9640 1 -- 172.21.15.33:0/4190790804 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f62f00041c0 con 0x55b7b47857a0 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62dcff9640 1 -- 172.21.15.33:0/4190790804 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f62e40efc60 con 0x55b7b47857a0 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62fe5ee740 10 client.16353 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62fe5ee740 10 client.16353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.602+0000 7f62fe5ee740 20 client.16353 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.603+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b7b479ede0 0x55b7b47a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.603+0000 7f62fe5ee740 4 client.16353 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.603+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4190790804 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7b46f6cf0 con 0x55b7b479ede0 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.603+0000 7f62f8fa1640 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b7b479ede0 0x55b7b47a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.603+0000 7f62dcff9640 10 client.16353 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.603+0000 7f62f8fa1640 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b7b479ede0 0x55b7b47a1240 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d20b0aa25dbb6e9 server_cookie=9de375e30915ebc3 in_seq=0 out_seq=0 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62dcff9640 1 -- 172.21.15.33:0/4190790804 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7b46f6cf0 con 0x55b7b479ede0 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62dcff9640 10 client.16353 handle_command_reply: tid=0 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62fe5ee740 1 client.16353 shutdown 2024-04-02T13:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4190790804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 msgr2=0x55b7b478e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b478e6b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f62e4006010 tx=0x7f62e4004590 comp rx=0 tx=0).stop 2024-04-02T13:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4190790804 shutdown_connections 2024-04-02T13:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b478eca0 0x55b7b478c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b7b479ede0 0x55b7b47a1240 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b47860c0 0x55b7b478ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62dcff9640 0 client.16353 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/4190790804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b478e6b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4190790804 >> 172.21.15.33:0/4190790804 conn(0x55b7b477fed0 msgr2=0x55b7b4781c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4190790804 shutdown_connections 2024-04-02T13:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4190790804 wait complete. 2024-04-02T13:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.604+0000 7f62fe5ee740 20 client.16353 trim_cache size 0 max 16384 2024-04-02T13:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.618+0000 7f62fe5ee740 1 Processor -- start 2024-04-02T13:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.618+0000 7f62fe5ee740 1 -- start start 2024-04-02T13:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.618+0000 7f62fe5ee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47857a0 0x55b7b479a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.618+0000 7f62fe5ee740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b47860c0 0x55b7b478ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.618+0000 7f62fe5ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b478eca0 0x55b7b478e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.618+0000 7f62fe5ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41b18a0 con 0x55b7b478eca0 2024-04-02T13:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.618+0000 7f62fe5ee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41af0b0 con 0x55b7b47857a0 2024-04-02T13:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.618+0000 7f62fe5ee740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41acee0 con 0x55b7b47860c0 2024-04-02T13:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62f97a2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47857a0 0x55b7b479a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62f9fa3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b478eca0 0x55b7b478e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62f97a2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47857a0 0x55b7b479a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:47044/0 (socket says 172.21.15.33:47044) 2024-04-02T13:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62f97a2640 1 -- 172.21.15.33:0/3524318044 learned_addr learned my addr 172.21.15.33:0/3524318044 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62f97a2640 1 -- 172.21.15.33:0/3524318044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b47860c0 msgr2=0x55b7b478ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62f8fa1640 1 --2- 172.21.15.33:0/3524318044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b47860c0 0x55b7b478ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62f97a2640 1 --2- 172.21.15.33:0/3524318044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b47860c0 0x55b7b478ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62f97a2640 1 -- 172.21.15.33:0/3524318044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b478eca0 msgr2=0x55b7b478e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62f97a2640 1 --2- 172.21.15.33:0/3524318044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b478eca0 0x55b7b478e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62f97a2640 1 -- 172.21.15.33:0/3524318044 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7b478c3c0 con 0x55b7b47857a0 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62f8fa1640 1 --2- 172.21.15.33:0/3524318044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b47860c0 0x55b7b478ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62f97a2640 1 --2- 172.21.15.33:0/3524318044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47857a0 0x55b7b479a980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f62e4005820 tx=0x7f62e40dee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea1d17078f3540f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62deffd640 1 -- 172.21.15.33:0/3524318044 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e40bea40 con 0x55b7b47857a0 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62deffd640 1 -- 172.21.15.33:0/3524318044 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f62e40be340 con 0x55b7b47857a0 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.619+0000 7f62deffd640 1 -- 172.21.15.33:0/3524318044 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e40a73e0 con 0x55b7b47857a0 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.620+0000 7f62fe5ee740 1 -- 172.21.15.33:0/3524318044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47857a0 msgr2=0x55b7b479a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.620+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/3524318044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47857a0 0x55b7b479a980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f62e4005820 tx=0x7f62e40dee90 comp rx=0 tx=0).stop 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.620+0000 7f62fe5ee740 1 -- 172.21.15.33:0/3524318044 shutdown_connections 2024-04-02T13:42:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.620+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/3524318044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b478eca0 0x55b7b478e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.620+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/3524318044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b47860c0 0x55b7b478ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.620+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/3524318044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47857a0 0x55b7b479a980 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.620+0000 7f62fe5ee740 1 -- 172.21.15.33:0/3524318044 >> 172.21.15.33:0/3524318044 conn(0x55b7b477fed0 msgr2=0x55b7b478a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.620+0000 7f62fe5ee740 1 -- 172.21.15.33:0/3524318044 shutdown_connections 2024-04-02T13:42:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.620+0000 7f62fe5ee740 1 -- 172.21.15.33:0/3524318044 wait complete. 2024-04-02T13:42:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.620+0000 7f62fe5ee740 1 -- start start 2024-04-02T13:42:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.620+0000 7f62fe5ee740 1 Processor -- start 2024-04-02T13:42:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62fe5ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b4790af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62fe5ee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47860c0 0x55b7b4796e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62fe5ee740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b478eca0 0x55b7b4788cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62fe5ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41ad570 con 0x55b7b47857a0 2024-04-02T13:42:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62fe5ee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41ae9e0 con 0x55b7b47860c0 2024-04-02T13:42:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62fe5ee740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41adc40 con 0x55b7b478eca0 2024-04-02T13:42:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62fe5ee740 10 client.0 _ll_register_callbacks cb 0x7f62e40bde80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62f97a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b4790af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62f97a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b4790af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37506/0 (socket says 172.21.15.33:37506) 2024-04-02T13:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62f97a2640 1 -- 172.21.15.33:0/4053532851 learned_addr learned my addr 172.21.15.33:0/4053532851 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62f8fa1640 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47860c0 0x55b7b4796e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62f9fa3640 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b478eca0 0x55b7b4788cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62f97a2640 1 -- 172.21.15.33:0/4053532851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b478eca0 msgr2=0x55b7b4788cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62f97a2640 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b478eca0 0x55b7b4788cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62f97a2640 1 -- 172.21.15.33:0/4053532851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47860c0 msgr2=0x55b7b4796e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62f97a2640 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47860c0 0x55b7b4796e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62f97a2640 1 -- 172.21.15.33:0/4053532851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7b4789200 con 0x55b7b47857a0 2024-04-02T13:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62f8fa1640 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47860c0 0x55b7b4796e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62f97a2640 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b4790af0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f62e40dede0 tx=0x7f62e402cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ebfab717827609 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62dcff9640 1 -- 172.21.15.33:0/4053532851 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e40dd2e0 con 0x55b7b47857a0 2024-04-02T13:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62dcff9640 1 -- 172.21.15.33:0/4053532851 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f62e40a73e0 con 0x55b7b47857a0 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.621+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4053532851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e40be340 con 0x55b7b47857a0 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62dcff9640 1 -- 172.21.15.33:0/4053532851 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f62e40f2440 con 0x55b7b47857a0 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62fe5ee740 10 client.16359 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4053532851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b7b4789200 con 0x55b7b47857a0 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62dcff9640 1 -- 172.21.15.33:0/4053532851 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f62e40be340 con 0x55b7b47857a0 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62dcff9640 1 -- 172.21.15.33:0/4053532851 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f62e4005000 con 0x55b7b47857a0 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62fe5ee740 10 client.16359 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62fe5ee740 10 client.16359 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62fe5ee740 20 client.16359 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b7b478fa20 0x55b7b4807380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62fe5ee740 4 client.16359 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4053532851 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7b46f1040 con 0x55b7b478fa20 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62f8fa1640 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b7b478fa20 0x55b7b4807380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62dcff9640 10 client.16359 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.622+0000 7f62f8fa1640 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b7b478fa20 0x55b7b4807380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3aa58f252c1c50cc server_cookie=81f2a5fcbb16816 in_seq=0 out_seq=0 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62dcff9640 1 -- 172.21.15.33:0/4053532851 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7b46f1040 con 0x55b7b478fa20 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62dcff9640 10 client.16359 handle_command_reply: tid=0 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62fe5ee740 1 client.16359 shutdown 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4053532851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 msgr2=0x55b7b4790af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b4790af0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f62e40dede0 tx=0x7f62e402cd40 comp rx=0 tx=0).stop 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4053532851 shutdown_connections 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b7b478fa20 0x55b7b4807380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b7b478eca0 0x55b7b4788cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b7b47860c0 0x55b7b4796e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62dcff9640 0 client.16359 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62fe5ee740 1 --2- 172.21.15.33:0/4053532851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b7b47857a0 0x55b7b4790af0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4053532851 >> 172.21.15.33:0/4053532851 conn(0x55b7b477fed0 msgr2=0x55b7b477e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4053532851 shutdown_connections 2024-04-02T13:42:08.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62fe5ee740 1 -- 172.21.15.33:0/4053532851 wait complete. 2024-04-02T13:42:08.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.623+0000 7f62fe5ee740 20 client.16359 trim_cache size 0 max 16384 2024-04-02T13:42:08.633 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:42:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.624+0000 7f62fba06640 1 -- 172.21.15.33:0/1360240011 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f62e00a3d20 msgr2=0x7f62e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.624+0000 7f62fba06640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f62e00a3d20 0x7f62e00a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f62ec005fd0 tx=0x7f62ec012ad0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.624+0000 7f62fba06640 1 -- 172.21.15.33:0/1360240011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f4068d00 msgr2=0x7f62f4198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.624+0000 7f62fba06640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f4068d00 0x7f62f4198410 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009900 tx=0x7f62e402ca90 comp rx=0 tx=0).stop 2024-04-02T13:42:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.624+0000 7f62fba06640 1 -- 172.21.15.33:0/1360240011 shutdown_connections 2024-04-02T13:42:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.624+0000 7f62fba06640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f62e00a3d20 0x7f62e00a61e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.624+0000 7f62fba06640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f62f4107b50 0x7f62f419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.624+0000 7f62fba06640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62f40696c0 0x7f62f4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.624+0000 7f62fba06640 1 --2- 172.21.15.33:0/1360240011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f4068d00 0x7f62f4198410 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.624+0000 7f62fba06640 1 -- 172.21.15.33:0/1360240011 >> 172.21.15.33:0/1360240011 conn(0x7f62f4077b00 msgr2=0x7f62f40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.624+0000 7f62fba06640 1 -- 172.21.15.33:0/1360240011 shutdown_connections 2024-04-02T13:42:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.624+0000 7f62fba06640 1 -- 172.21.15.33:0/1360240011 wait complete. 2024-04-02T13:42:08.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:42:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.752+0000 7f8c29a3c640 1 -- 172.21.15.33:0/4022098517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c24101330 msgr2=0x7f8c2410d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.752+0000 7f8c29a3c640 1 --2- 172.21.15.33:0/4022098517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c24101330 0x7f8c2410d840 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009930 tx=0x7f8c1c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.752+0000 7f8c29a3c640 1 -- 172.21.15.33:0/4022098517 shutdown_connections 2024-04-02T13:42:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.752+0000 7f8c29a3c640 1 --2- 172.21.15.33:0/4022098517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c24101330 0x7f8c2410d840 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.752+0000 7f8c29a3c640 1 --2- 172.21.15.33:0/4022098517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c24100970 0x7f8c24100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.752+0000 7f8c29a3c640 1 --2- 172.21.15.33:0/4022098517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c24106980 0x7f8c24106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.752+0000 7f8c29a3c640 1 -- 172.21.15.33:0/4022098517 >> 172.21.15.33:0/4022098517 conn(0x7f8c24077b00 msgr2=0x7f8c24077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.752+0000 7f8c29a3c640 1 -- 172.21.15.33:0/4022098517 shutdown_connections 2024-04-02T13:42:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.752+0000 7f8c29a3c640 1 -- 172.21.15.33:0/4022098517 wait complete. 2024-04-02T13:42:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.752+0000 7f8c29a3c640 1 Processor -- start 2024-04-02T13:42:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c29a3c640 1 -- start start 2024-04-02T13:42:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c29a3c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c24100970 0x7f8c24198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c29a3c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c24101330 0x7f8c24198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c29a3c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c24106980 0x7f8c2419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c29a3c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c24110d60 con 0x7f8c24101330 2024-04-02T13:42:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c29a3c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c24110be0 con 0x7f8c24100970 2024-04-02T13:42:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c29a3c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8c24110ee0 con 0x7f8c24106980 2024-04-02T13:42:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c227fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c24101330 0x7f8c24198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c24100970 0x7f8c24198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c227fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c24101330 0x7f8c24198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37526/0 (socket says 172.21.15.33:37526) 2024-04-02T13:42:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c227fc640 1 -- 172.21.15.33:0/1606887225 learned_addr learned my addr 172.21.15.33:0/1606887225 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c237fe640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c24106980 0x7f8c2419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c227fc640 1 -- 172.21.15.33:0/1606887225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c24106980 msgr2=0x7f8c2419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c227fc640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c24106980 0x7f8c2419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c227fc640 1 -- 172.21.15.33:0/1606887225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c24100970 msgr2=0x7f8c24198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c227fc640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c24100970 0x7f8c24198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c227fc640 1 -- 172.21.15.33:0/1606887225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2419d3d0 con 0x7f8c24101330 2024-04-02T13:42:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c24100970 0x7f8c24198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.753+0000 7f8c227fc640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c24101330 0x7f8c24198970 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800e9f0 tx=0x7f8c1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.754+0000 7f8c28a3a640 1 -- 172.21.15.33:0/1606887225 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1800cd80 con 0x7f8c24101330 2024-04-02T13:42:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.754+0000 7f8c29a3c640 1 -- 172.21.15.33:0/1606887225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c2419d6c0 con 0x7f8c24101330 2024-04-02T13:42:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.754+0000 7f8c28a3a640 1 -- 172.21.15.33:0/1606887225 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8c18010900 con 0x7f8c24101330 2024-04-02T13:42:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.754+0000 7f8c28a3a640 1 -- 172.21.15.33:0/1606887225 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c180189e0 con 0x7f8c24101330 2024-04-02T13:42:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.754+0000 7f8c29a3c640 1 -- 172.21.15.33:0/1606887225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c241a4f50 con 0x7f8c24101330 2024-04-02T13:42:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.756+0000 7f8c28a3a640 1 -- 172.21.15.33:0/1606887225 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8c18010ab0 con 0x7f8c24101330 2024-04-02T13:42:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.756+0000 7f8c2c628740 1 Processor -- start 2024-04-02T13:42:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.756+0000 7f8c2c628740 1 -- start start 2024-04-02T13:42:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.756+0000 7f8c28a3a640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8c0c0a3d20 0x7f8c0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.756+0000 7f8c2c628740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732cde30 0x55f4732ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.756+0000 7f8c2c628740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732bb980 0x55f4732bbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.756+0000 7f8c2c628740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bc340 0x55f4732d5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.756+0000 7f8c2c628740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f4732257b0 con 0x55f4732cde30 2024-04-02T13:42:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.756+0000 7f8c2c628740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f4731c4570 con 0x55f4732bb980 2024-04-02T13:42:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.756+0000 7f8c2c628740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f473221760 con 0x55f4732bc340 2024-04-02T13:42:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c28a3a640 1 -- 172.21.15.33:0/1606887225 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8c180d0f30 con 0x7f8c24101330 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c237fe640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8c0c0a3d20 0x7f8c0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732cde30 0x55f4732ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c227fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732bb980 0x55f4732bbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732cde30 0x55f4732ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37534/0 (socket says 172.21.15.33:37534) 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c22ffd640 1 -- 172.21.15.33:0/1903302882 learned_addr learned my addr 172.21.15.33:0/1903302882 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c237fe640 1 --2- 172.21.15.33:0/1903302882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bc340 0x55f4732d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c227fc640 1 -- 172.21.15.33:0/1903302882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bc340 msgr2=0x55f4732d5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c227fc640 1 --2- 172.21.15.33:0/1903302882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bc340 0x55f4732d5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c227fc640 1 -- 172.21.15.33:0/1903302882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732cde30 msgr2=0x55f4732ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c227fc640 1 --2- 172.21.15.33:0/1903302882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732cde30 0x55f4732ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c227fc640 1 -- 172.21.15.33:0/1903302882 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f472f393f0 con 0x55f4732bb980 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/1903302882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732cde30 0x55f4732ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c237fe640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8c0c0a3d20 0x7f8c0c0a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c006010 tx=0x7f8c1c004260 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c227fc640 1 --2- 172.21.15.33:0/1903302882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732bb980 0x55f4732bbe00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c180ce310 tx=0x7f8c1809c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=764a08301398af74 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c09ffb640 1 -- 172.21.15.33:0/1903302882 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18004d30 con 0x55f4732bb980 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c09ffb640 1 -- 172.21.15.33:0/1903302882 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8c1809d260 con 0x55f4732bb980 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.757+0000 7f8c09ffb640 1 -- 172.21.15.33:0/1903302882 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c180046c0 con 0x55f4732bb980 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.758+0000 7f8c2c628740 1 -- 172.21.15.33:0/1903302882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732bb980 msgr2=0x55f4732bbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.758+0000 7f8c2c628740 1 --2- 172.21.15.33:0/1903302882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732bb980 0x55f4732bbe00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c180ce310 tx=0x7f8c1809c4c0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.758+0000 7f8c2c628740 1 -- 172.21.15.33:0/1903302882 shutdown_connections 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.758+0000 7f8c2c628740 1 --2- 172.21.15.33:0/1903302882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bc340 0x55f4732d5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.758+0000 7f8c2c628740 1 --2- 172.21.15.33:0/1903302882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732bb980 0x55f4732bbe00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.758+0000 7f8c2c628740 1 --2- 172.21.15.33:0/1903302882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732cde30 0x55f4732ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.758+0000 7f8c2c628740 1 -- 172.21.15.33:0/1903302882 >> 172.21.15.33:0/1903302882 conn(0x55f4732be440 msgr2=0x55f4732be820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.758+0000 7f8c2c628740 1 -- 172.21.15.33:0/1903302882 shutdown_connections 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.758+0000 7f8c2c628740 1 -- 172.21.15.33:0/1903302882 wait complete. 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.758+0000 7f8c2c628740 1 -- start start 2024-04-02T13:42:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c2c628740 1 Processor -- start 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c2c628740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bb980 0x55f4732d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c2c628740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bc340 0x55f4732d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c2c628740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732cde30 0x55f4732d3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c2c628740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f4732d62a0 con 0x55f4732bc340 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c2c628740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f4731e0e80 con 0x55f4732cde30 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c2c628740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f4732d6420 con 0x55f4732bb980 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c2c628740 10 client.0 _ll_register_callbacks cb 0x55f47321caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c227fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bc340 0x55f4732d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c227fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bc340 0x55f4732d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37548/0 (socket says 172.21.15.33:37548) 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c227fc640 1 -- 172.21.15.33:0/1883632359 learned_addr learned my addr 172.21.15.33:0/1883632359 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c237fe640 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732cde30 0x55f4732d3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bb980 0x55f4732d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c227fc640 1 -- 172.21.15.33:0/1883632359 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bb980 msgr2=0x55f4732d5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c227fc640 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bb980 0x55f4732d5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c227fc640 1 -- 172.21.15.33:0/1883632359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732cde30 msgr2=0x55f4732d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c227fc640 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732cde30 0x55f4732d3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c227fc640 1 -- 172.21.15.33:0/1883632359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4730037d0 con 0x55f4732bc340 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c237fe640 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732cde30 0x55f4732d3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c227fc640 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bc340 0x55f4732d2fc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8c180ce370 tx=0x7f8c18009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cfa79f66cf8734 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.759+0000 7f8c02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c02ffd640 1 -- 172.21.15.33:0/1883632359 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c180b5020 con 0x55f4732bc340 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c02ffd640 1 -- 172.21.15.33:0/1883632359 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8c180213f0 con 0x55f4732bc340 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c2c628740 1 -- 172.21.15.33:0/1883632359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c1809d260 con 0x55f4732bc340 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c02ffd640 1 -- 172.21.15.33:0/1883632359 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8c180ce7a0 con 0x55f4732bc340 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c2c628740 10 client.16371 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c2c628740 1 -- 172.21.15.33:0/1883632359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f4732e2290 con 0x55f4732bc340 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c02ffd640 1 -- 172.21.15.33:0/1883632359 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8c1809d260 con 0x55f4732bc340 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c02ffd640 1 -- 172.21.15.33:0/1883632359 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8c18004b30 con 0x55f4732bc340 2024-04-02T13:42:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c2c628740 10 client.16371 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:08.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c2c628740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:08.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c2c628740 20 client.16371 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:08.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c2c628740 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f4732e35e0 0x55f4732e59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c2c628740 4 client.16371 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:08.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c2c628740 1 -- 172.21.15.33:0/1883632359 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f473232cf0 con 0x55f4732e35e0 2024-04-02T13:42:08.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.760+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f4732e35e0 0x55f4732e59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.761+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f4732e35e0 0x55f4732e59d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f07e51b02bfa28 server_cookie=26ffaf951cc2ab5f in_seq=0 out_seq=0 2024-04-02T13:42:08.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.761+0000 7f8c02ffd640 10 client.16371 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c02ffd640 1 -- 172.21.15.33:0/1883632359 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f473232cf0 con 0x55f4732e35e0 2024-04-02T13:42:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c02ffd640 10 client.16371 handle_command_reply: tid=0 2024-04-02T13:42:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c2c628740 1 client.16371 shutdown 2024-04-02T13:42:08.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c2c628740 1 -- 172.21.15.33:0/1883632359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bc340 msgr2=0x55f4732d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c2c628740 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bc340 0x55f4732d2fc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8c180ce370 tx=0x7f8c18009310 comp rx=0 tx=0).stop 2024-04-02T13:42:08.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c2c628740 1 -- 172.21.15.33:0/1883632359 shutdown_connections 2024-04-02T13:42:08.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c2c628740 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f4732e35e0 0x55f4732e59d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c2c628740 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732cde30 0x55f4732d3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c02ffd640 0 client.16371 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:08.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c2c628740 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bc340 0x55f4732d2fc0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c2c628740 1 --2- 172.21.15.33:0/1883632359 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bb980 0x55f4732d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c2c628740 1 -- 172.21.15.33:0/1883632359 >> 172.21.15.33:0/1883632359 conn(0x55f4732be440 msgr2=0x55f4732cf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c2c628740 1 -- 172.21.15.33:0/1883632359 shutdown_connections 2024-04-02T13:42:08.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c2c628740 1 -- 172.21.15.33:0/1883632359 wait complete. 2024-04-02T13:42:08.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.763+0000 7f8c2c628740 20 client.16371 trim_cache size 0 max 16384 2024-04-02T13:42:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.777+0000 7f8c2c628740 1 Processor -- start 2024-04-02T13:42:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.777+0000 7f8c2c628740 1 -- start start 2024-04-02T13:42:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.777+0000 7f8c2c628740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 0x55f4732cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.777+0000 7f8c2c628740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732bc340 0x55f4732bd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.777+0000 7f8c2c628740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732cde30 0x55f4732df160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.777+0000 7f8c2c628740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241af730 con 0x55f4732bb980 2024-04-02T13:42:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.777+0000 7f8c2c628740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241acf40 con 0x55f4732bc340 2024-04-02T13:42:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.777+0000 7f8c2c628740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241aad70 con 0x55f4732cde30 2024-04-02T13:42:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.777+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 0x55f4732cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.777+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 0x55f4732cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37550/0 (socket says 172.21.15.33:37550) 2024-04-02T13:42:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.777+0000 7f8c227fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732bc340 0x55f4732bd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.777+0000 7f8c22ffd640 1 -- 172.21.15.33:0/3749197303 learned_addr learned my addr 172.21.15.33:0/3749197303 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.778+0000 7f8c237fe640 1 --2- 172.21.15.33:0/3749197303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732cde30 0x55f4732df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.778+0000 7f8c22ffd640 1 -- 172.21.15.33:0/3749197303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732cde30 msgr2=0x55f4732df160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.778+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/3749197303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732cde30 0x55f4732df160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.778+0000 7f8c22ffd640 1 -- 172.21.15.33:0/3749197303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732bc340 msgr2=0x55f4732bd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.778+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/3749197303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732bc340 0x55f4732bd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.778+0000 7f8c22ffd640 1 -- 172.21.15.33:0/3749197303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4732d1510 con 0x55f4732bb980 2024-04-02T13:42:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.778+0000 7f8c237fe640 1 --2- 172.21.15.33:0/3749197303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732cde30 0x55f4732df160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.778+0000 7f8c227fc640 1 --2- 172.21.15.33:0/3749197303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732bc340 0x55f4732bd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.778+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/3749197303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 0x55f4732cb8a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8c140106e0 tx=0x7f8c1400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db198b7967dc44e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.778+0000 7f8c097fa640 1 -- 172.21.15.33:0/3749197303 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14004b20 con 0x55f4732bb980 2024-04-02T13:42:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.778+0000 7f8c097fa640 1 -- 172.21.15.33:0/3749197303 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8c14004cc0 con 0x55f4732bb980 2024-04-02T13:42:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.778+0000 7f8c097fa640 1 -- 172.21.15.33:0/3749197303 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c140054b0 con 0x55f4732bb980 2024-04-02T13:42:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.779+0000 7f8c2c628740 1 -- 172.21.15.33:0/3749197303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 msgr2=0x55f4732cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.779+0000 7f8c2c628740 1 --2- 172.21.15.33:0/3749197303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 0x55f4732cb8a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8c140106e0 tx=0x7f8c1400a570 comp rx=0 tx=0).stop 2024-04-02T13:42:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.779+0000 7f8c2c628740 1 -- 172.21.15.33:0/3749197303 shutdown_connections 2024-04-02T13:42:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.779+0000 7f8c2c628740 1 --2- 172.21.15.33:0/3749197303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732cde30 0x55f4732df160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.779+0000 7f8c2c628740 1 --2- 172.21.15.33:0/3749197303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732bc340 0x55f4732bd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.779+0000 7f8c2c628740 1 --2- 172.21.15.33:0/3749197303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 0x55f4732cb8a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.779+0000 7f8c2c628740 1 -- 172.21.15.33:0/3749197303 >> 172.21.15.33:0/3749197303 conn(0x55f4732be440 msgr2=0x55f4732bce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.779+0000 7f8c2c628740 1 -- 172.21.15.33:0/3749197303 shutdown_connections 2024-04-02T13:42:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.779+0000 7f8c2c628740 1 -- 172.21.15.33:0/3749197303 wait complete. 2024-04-02T13:42:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.779+0000 7f8c2c628740 1 -- start start 2024-04-02T13:42:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.779+0000 7f8c2c628740 1 Processor -- start 2024-04-02T13:42:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c2c628740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 0x55f4732d4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c2c628740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bc340 0x55f4732d4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c2c628740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732cde30 0x55f4732d4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c2c628740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241ab400 con 0x55f4732bb980 2024-04-02T13:42:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c2c628740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241ac870 con 0x55f4732cde30 2024-04-02T13:42:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c2c628740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241abad0 con 0x55f4732bc340 2024-04-02T13:42:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c2c628740 10 client.0 _ll_register_callbacks cb 0x7f8c1400ad20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 0x55f4732d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 0x55f4732d4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37556/0 (socket says 172.21.15.33:37556) 2024-04-02T13:42:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c22ffd640 1 -- 172.21.15.33:0/3376773478 learned_addr learned my addr 172.21.15.33:0/3376773478 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:08.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c237fe640 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732cde30 0x55f4732d4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c227fc640 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bc340 0x55f4732d4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c22ffd640 1 -- 172.21.15.33:0/3376773478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bc340 msgr2=0x55f4732d4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bc340 0x55f4732d4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c22ffd640 1 -- 172.21.15.33:0/3376773478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732cde30 msgr2=0x55f4732d4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:08.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732cde30 0x55f4732d4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c22ffd640 1 -- 172.21.15.33:0/3376773478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4732da910 con 0x55f4732bb980 2024-04-02T13:42:08.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c227fc640 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bc340 0x55f4732d4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:08.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c22ffd640 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 0x55f4732d4570 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8c14010820 tx=0x7f8c1400ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332e6e888dfc84c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:08.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c02ffd640 1 -- 172.21.15.33:0/3376773478 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14020310 con 0x55f4732bb980 2024-04-02T13:42:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c02ffd640 1 -- 172.21.15.33:0/3376773478 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8c140204b0 con 0x55f4732bb980 2024-04-02T13:42:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.780+0000 7f8c2c628740 1 -- 172.21.15.33:0/3376773478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c14004cc0 con 0x55f4732bb980 2024-04-02T13:42:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c02ffd640 1 -- 172.21.15.33:0/3376773478 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8c1403b050 con 0x55f4732bb980 2024-04-02T13:42:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c2c628740 10 client.16377 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c2c628740 1 -- 172.21.15.33:0/3376773478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f4732dbc10 con 0x55f4732bb980 2024-04-02T13:42:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c02ffd640 1 -- 172.21.15.33:0/3376773478 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8c14004cc0 con 0x55f4732bb980 2024-04-02T13:42:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c02ffd640 1 -- 172.21.15.33:0/3376773478 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8c1402a430 con 0x55f4732bb980 2024-04-02T13:42:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c2c628740 10 client.16377 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c2c628740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c2c628740 20 client.16377 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c2c628740 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f473349940 0x55f47334bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c2c628740 4 client.16377 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c2c628740 1 -- 172.21.15.33:0/3376773478 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f47322d040 con 0x55f473349940 2024-04-02T13:42:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c227fc640 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f473349940 0x55f47334bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c02ffd640 10 client.16377 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.781+0000 7f8c227fc640 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f473349940 0x55f47334bd80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce4056d48b36919 server_cookie=7a665809c0a456b0 in_seq=0 out_seq=0 2024-04-02T13:42:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c02ffd640 1 -- 172.21.15.33:0/3376773478 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f47322d040 con 0x55f473349940 2024-04-02T13:42:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c02ffd640 10 client.16377 handle_command_reply: tid=0 2024-04-02T13:42:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c2c628740 1 client.16377 shutdown 2024-04-02T13:42:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c2c628740 1 -- 172.21.15.33:0/3376773478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 msgr2=0x55f4732d4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c2c628740 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 0x55f4732d4570 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8c14010820 tx=0x7f8c1400ffa0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c2c628740 1 -- 172.21.15.33:0/3376773478 shutdown_connections 2024-04-02T13:42:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c2c628740 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f473349940 0x55f47334bd80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c2c628740 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4732cde30 0x55f4732d4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c2c628740 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f4732bc340 0x55f4732d4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c2c628740 1 --2- 172.21.15.33:0/3376773478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4732bb980 0x55f4732d4570 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c02ffd640 0 client.16377 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c2c628740 1 -- 172.21.15.33:0/3376773478 >> 172.21.15.33:0/3376773478 conn(0x55f4732be440 msgr2=0x55f4732d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c2c628740 1 -- 172.21.15.33:0/3376773478 shutdown_connections 2024-04-02T13:42:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c2c628740 1 -- 172.21.15.33:0/3376773478 wait complete. 2024-04-02T13:42:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.783+0000 7f8c2c628740 20 client.16377 trim_cache size 0 max 16384 2024-04-02T13:42:08.799 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:42:08.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.784+0000 7f8c29a3c640 1 -- 172.21.15.33:0/1606887225 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8c0c0a3d20 msgr2=0x7f8c0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.784+0000 7f8c29a3c640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8c0c0a3d20 0x7f8c0c0a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c006010 tx=0x7f8c1c004260 comp rx=0 tx=0).stop 2024-04-02T13:42:08.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.784+0000 7f8c29a3c640 1 -- 172.21.15.33:0/1606887225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c24101330 msgr2=0x7f8c24198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:08.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.784+0000 7f8c29a3c640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c24101330 0x7f8c24198970 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800e9f0 tx=0x7f8c1800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.784+0000 7f8c29a3c640 1 -- 172.21.15.33:0/1606887225 shutdown_connections 2024-04-02T13:42:08.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.784+0000 7f8c29a3c640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8c0c0a3d20 0x7f8c0c0a61e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.784+0000 7f8c29a3c640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c24106980 0x7f8c2419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.784+0000 7f8c29a3c640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c24101330 0x7f8c24198970 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.784+0000 7f8c29a3c640 1 --2- 172.21.15.33:0/1606887225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c24100970 0x7f8c24198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:08.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.784+0000 7f8c29a3c640 1 -- 172.21.15.33:0/1606887225 >> 172.21.15.33:0/1606887225 conn(0x7f8c24077b00 msgr2=0x7f8c24069040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:08.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.784+0000 7f8c29a3c640 1 -- 172.21.15.33:0/1606887225 shutdown_connections 2024-04-02T13:42:08.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:08.784+0000 7f8c29a3c640 1 -- 172.21.15.33:0/1606887225 wait complete. 2024-04-02T13:42:09.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:09 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/212922738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:42:09.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:09 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/212922738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:42:09.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:09 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/212922738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:42:09.802 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:42:09.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:42:09.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:42:09.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.912+0000 7f63c2fdb640 1 -- 172.21.15.33:0/1055150404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc101330 msgr2=0x7f63bc10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:09.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.912+0000 7f63c2fdb640 1 --2- 172.21.15.33:0/1055150404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc101330 0x7f63bc10d840 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009cd0 tx=0x7f63b402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.912+0000 7f63c2fdb640 1 -- 172.21.15.33:0/1055150404 shutdown_connections 2024-04-02T13:42:09.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.912+0000 7f63c2fdb640 1 --2- 172.21.15.33:0/1055150404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc101330 0x7f63bc10d840 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.912+0000 7f63c2fdb640 1 --2- 172.21.15.33:0/1055150404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63bc100970 0x7f63bc100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.912+0000 7f63c2fdb640 1 --2- 172.21.15.33:0/1055150404 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63bc106980 0x7f63bc106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.912+0000 7f63c2fdb640 1 -- 172.21.15.33:0/1055150404 >> 172.21.15.33:0/1055150404 conn(0x7f63bc077b00 msgr2=0x7f63bc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.913+0000 7f63c2fdb640 1 -- 172.21.15.33:0/1055150404 shutdown_connections 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.913+0000 7f63c2fdb640 1 -- 172.21.15.33:0/1055150404 wait complete. 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.913+0000 7f63c2fdb640 1 Processor -- start 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.913+0000 7f63c2fdb640 1 -- start start 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.913+0000 7f63c2fdb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63bc100970 0x7f63bc1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.913+0000 7f63c2fdb640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63bc101330 0x7f63bc197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.913+0000 7f63c2fdb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc106980 0x7f63bc19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.913+0000 7f63c2fdb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc110d80 con 0x7f63bc106980 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.913+0000 7f63c2fdb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc110c00 con 0x7f63bc100970 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.913+0000 7f63c2fdb640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc110f00 con 0x7f63bc101330 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.913+0000 7f63c1578640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc106980 0x7f63bc19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.913+0000 7f63c0d77640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63bc100970 0x7f63bc1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63c1578640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc106980 0x7f63bc19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37572/0 (socket says 172.21.15.33:37572) 2024-04-02T13:42:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63c1578640 1 -- 172.21.15.33:0/430812863 learned_addr learned my addr 172.21.15.33:0/430812863 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63bbfff640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63bc101330 0x7f63bc197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63c1578640 1 -- 172.21.15.33:0/430812863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63bc101330 msgr2=0x7f63bc197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63c1578640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63bc101330 0x7f63bc197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63c1578640 1 -- 172.21.15.33:0/430812863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63bc100970 msgr2=0x7f63bc1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63c1578640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63bc100970 0x7f63bc1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63c1578640 1 -- 172.21.15.33:0/430812863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63bc19c2b0 con 0x7f63bc106980 2024-04-02T13:42:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63c0d77640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63bc100970 0x7f63bc1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63c1578640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc106980 0x7f63bc19bb80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f63b4005cc0 tx=0x7f63b4034b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63b9ffb640 1 -- 172.21.15.33:0/430812863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4039050 con 0x7f63bc106980 2024-04-02T13:42:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63b9ffb640 1 -- 172.21.15.33:0/430812863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f63b4045070 con 0x7f63bc106980 2024-04-02T13:42:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63c2fdb640 1 -- 172.21.15.33:0/430812863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63bc19c540 con 0x7f63bc106980 2024-04-02T13:42:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63b9ffb640 1 -- 172.21.15.33:0/430812863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b40044d0 con 0x7f63bc106980 2024-04-02T13:42:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.914+0000 7f63c2fdb640 1 -- 172.21.15.33:0/430812863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63bc1a3e30 con 0x7f63bc106980 2024-04-02T13:42:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63b9ffb640 1 -- 172.21.15.33:0/430812863 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f63b4032860 con 0x7f63bc106980 2024-04-02T13:42:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63c5bc4740 1 Processor -- start 2024-04-02T13:42:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63c5bc4740 1 -- start start 2024-04-02T13:42:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63b9ffb640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f63ac0a3da0 0x7f63ac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63c5bc4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bd9e20 0x55ad30bda200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63c5bc4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bda740 0x55ad30be5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63c5bc4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30be81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63c5bc4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ad30b3c770 con 0x55ad30be5dc0 2024-04-02T13:42:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63c5bc4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ad30ae4960 con 0x55ad30bd9e20 2024-04-02T13:42:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63c5bc4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ad30b388c0 con 0x55ad30bda740 2024-04-02T13:42:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63bbfff640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f63ac0a3da0 0x7f63ac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63c1578640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30be81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63c0d77640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bd9e20 0x55ad30bda200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63c0d77640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bd9e20 0x55ad30bda200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:47148/0 (socket says 172.21.15.33:47148) 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63c0d77640 1 -- 172.21.15.33:0/3886523082 learned_addr learned my addr 172.21.15.33:0/3886523082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.917+0000 7f63b9ffb640 1 -- 172.21.15.33:0/430812863 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f63b40f3460 con 0x7f63bc106980 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.918+0000 7f63c0d77640 1 -- 172.21.15.33:0/3886523082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bda740 msgr2=0x55ad30be5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.918+0000 7f63bbfff640 1 --2- 172.21.15.33:0/3886523082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bda740 0x55ad30be5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.918+0000 7f63c0d77640 1 --2- 172.21.15.33:0/3886523082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bda740 0x55ad30be5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.918+0000 7f63c0d77640 1 -- 172.21.15.33:0/3886523082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 msgr2=0x55ad30be81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.918+0000 7f63c0d77640 1 --2- 172.21.15.33:0/3886523082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30be81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.918+0000 7f63c0d77640 1 -- 172.21.15.33:0/3886523082 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad309187e0 con 0x55ad30bd9e20 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.918+0000 7f63c1578640 1 --2- 172.21.15.33:0/3886523082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30be81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.918+0000 7f63bbfff640 1 --2- 172.21.15.33:0/3886523082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bda740 0x55ad30be5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.918+0000 7f63bbfff640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f63ac0a3da0 0x7f63ac0a6260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f63bc105020 tx=0x7f63a80097c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.918+0000 7f63c0d77640 1 --2- 172.21.15.33:0/3886523082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bd9e20 0x55ad30bda200 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009e70 tx=0x7f63b000e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffcb5897ff213c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.918+0000 7f63a6ffd640 1 -- 172.21.15.33:0/3886523082 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0017070 con 0x55ad30bd9e20 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.918+0000 7f63a6ffd640 1 -- 172.21.15.33:0/3886523082 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f63b0013940 con 0x55ad30bd9e20 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.918+0000 7f63a6ffd640 1 -- 172.21.15.33:0/3886523082 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0012860 con 0x55ad30bd9e20 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.919+0000 7f63c5bc4740 1 -- 172.21.15.33:0/3886523082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bd9e20 msgr2=0x55ad30bda200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.919+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/3886523082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bd9e20 0x55ad30bda200 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009e70 tx=0x7f63b000e5a0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.919+0000 7f63c5bc4740 1 -- 172.21.15.33:0/3886523082 shutdown_connections 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.919+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/3886523082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30be81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.919+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/3886523082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bda740 0x55ad30be5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.919+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/3886523082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bd9e20 0x55ad30bda200 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.919+0000 7f63c5bc4740 1 -- 172.21.15.33:0/3886523082 >> 172.21.15.33:0/3886523082 conn(0x55ad30bd4c00 msgr2=0x55ad30bd4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.919+0000 7f63c5bc4740 1 -- 172.21.15.33:0/3886523082 shutdown_connections 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.919+0000 7f63c5bc4740 1 -- 172.21.15.33:0/3886523082 wait complete. 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.919+0000 7f63c5bc4740 1 -- start start 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.919+0000 7f63c5bc4740 1 Processor -- start 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c5bc4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bd9e20 0x55ad30be2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c5bc4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bda740 0x55ad30be3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c5bc4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30be3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c5bc4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ad30be8990 con 0x55ad30be5dc0 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c5bc4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ad30af7d20 con 0x55ad30bd9e20 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c5bc4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ad30be8b10 con 0x55ad30bda740 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c5bc4740 10 client.0 _ll_register_callbacks cb 0x55ad30ac4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c1578640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30be3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c0d77640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bd9e20 0x55ad30be2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c0d77640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bd9e20 0x55ad30be2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:47154/0 (socket says 172.21.15.33:47154) 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c1578640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30be3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37588/0 (socket says 172.21.15.33:37588) 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c0d77640 1 -- 172.21.15.33:0/2703603991 learned_addr learned my addr 172.21.15.33:0/2703603991 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63bbfff640 1 --2- 172.21.15.33:0/2703603991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bda740 0x55ad30be3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c1578640 1 -- 172.21.15.33:0/2703603991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bda740 msgr2=0x55ad30be3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c1578640 1 --2- 172.21.15.33:0/2703603991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bda740 0x55ad30be3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c1578640 1 -- 172.21.15.33:0/2703603991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bd9e20 msgr2=0x55ad30be2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:09.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c1578640 1 --2- 172.21.15.33:0/2703603991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bd9e20 0x55ad30be2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63c1578640 1 -- 172.21.15.33:0/2703603991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad30978f70 con 0x55ad30be5dc0 2024-04-02T13:42:09.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.920+0000 7f63bbfff640 1 --2- 172.21.15.33:0/2703603991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bda740 0x55ad30be3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:09.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f63c1578640 1 --2- 172.21.15.33:0/2703603991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30be3840 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f63b40f0950 tx=0x7f63b404a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6996a2341a8d3395 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:09.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f639bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:09.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f639bfff640 1 -- 172.21.15.33:0/2703603991 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b40333a0 con 0x55ad30be5dc0 2024-04-02T13:42:09.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f639bfff640 1 -- 172.21.15.33:0/2703603991 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f63b40bb810 con 0x55ad30be5dc0 2024-04-02T13:42:09.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f63c5bc4740 1 -- 172.21.15.33:0/2703603991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b0013940 con 0x55ad30be5dc0 2024-04-02T13:42:09.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f639bfff640 1 -- 172.21.15.33:0/2703603991 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f63b40bf8a0 con 0x55ad30be5dc0 2024-04-02T13:42:09.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f63c5bc4740 10 client.16389 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f63c5bc4740 1 -- 172.21.15.33:0/2703603991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ad30810170 con 0x55ad30be5dc0 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f639bfff640 1 -- 172.21.15.33:0/2703603991 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f63b0013940 con 0x55ad30be5dc0 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f639bfff640 1 -- 172.21.15.33:0/2703603991 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f63b40f0ac0 con 0x55ad30be5dc0 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f63c5bc4740 10 client.16389 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f63c5bc4740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f63c5bc4740 20 client.16389 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/2703603991 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad30bf5f90 0x55ad30bf83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f63c5bc4740 4 client.16389 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.921+0000 7f63c5bc4740 1 -- 172.21.15.33:0/2703603991 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad30b49cb0 con 0x55ad30bf5f90 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.922+0000 7f63c0d77640 1 --2- 172.21.15.33:0/2703603991 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad30bf5f90 0x55ad30bf83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.922+0000 7f639bfff640 10 client.16389 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.922+0000 7f63c0d77640 1 --2- 172.21.15.33:0/2703603991 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad30bf5f90 0x55ad30bf83d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4606882a95926a36 server_cookie=8154ee39587657ae in_seq=0 out_seq=0 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f639bfff640 1 -- 172.21.15.33:0/2703603991 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad30b49cb0 con 0x55ad30bf5f90 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f639bfff640 10 client.16389 handle_command_reply: tid=0 2024-04-02T13:42:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f63c5bc4740 1 client.16389 shutdown 2024-04-02T13:42:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f63c5bc4740 1 -- 172.21.15.33:0/2703603991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 msgr2=0x55ad30be3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/2703603991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30be3840 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f63b40f0950 tx=0x7f63b404a000 comp rx=0 tx=0).stop 2024-04-02T13:42:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f63c5bc4740 1 -- 172.21.15.33:0/2703603991 shutdown_connections 2024-04-02T13:42:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/2703603991 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad30bf5f90 0x55ad30bf83d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/2703603991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30be3840 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/2703603991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bda740 0x55ad30be3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/2703603991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bd9e20 0x55ad30be2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f639bfff640 0 client.16389 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f63c5bc4740 1 -- 172.21.15.33:0/2703603991 >> 172.21.15.33:0/2703603991 conn(0x55ad30bd4c00 msgr2=0x55ad30bd6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f63c5bc4740 1 -- 172.21.15.33:0/2703603991 shutdown_connections 2024-04-02T13:42:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f63c5bc4740 1 -- 172.21.15.33:0/2703603991 wait complete. 2024-04-02T13:42:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.923+0000 7f63c5bc4740 20 client.16389 trim_cache size 0 max 16384 2024-04-02T13:42:09.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.940+0000 7f63c5bc4740 1 Processor -- start 2024-04-02T13:42:09.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c5bc4740 1 -- start start 2024-04-02T13:42:09.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c5bc4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30bd9e20 0x55ad30be54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c5bc4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bda740 0x55ad30bd1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c5bc4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30be5dc0 0x55ad30bf18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c5bc4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc1af650 con 0x55ad30bd9e20 2024-04-02T13:42:09.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c5bc4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc1ace60 con 0x55ad30bda740 2024-04-02T13:42:09.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c5bc4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc1aac90 con 0x55ad30be5dc0 2024-04-02T13:42:09.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c0d77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30bd9e20 0x55ad30be54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c0d77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30bd9e20 0x55ad30be54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37600/0 (socket says 172.21.15.33:37600) 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c0d77640 1 -- 172.21.15.33:0/849703811 learned_addr learned my addr 172.21.15.33:0/849703811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63bbfff640 1 --2- 172.21.15.33:0/849703811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bda740 0x55ad30bd1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c1578640 1 --2- 172.21.15.33:0/849703811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30be5dc0 0x55ad30bf18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c0d77640 1 -- 172.21.15.33:0/849703811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30be5dc0 msgr2=0x55ad30bf18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c0d77640 1 --2- 172.21.15.33:0/849703811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30be5dc0 0x55ad30bf18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c0d77640 1 -- 172.21.15.33:0/849703811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bda740 msgr2=0x55ad30bd1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c0d77640 1 --2- 172.21.15.33:0/849703811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bda740 0x55ad30bd1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c0d77640 1 -- 172.21.15.33:0/849703811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad30be1d70 con 0x55ad30bd9e20 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63c0d77640 1 --2- 172.21.15.33:0/849703811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30bd9e20 0x55ad30be54f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009e70 tx=0x7f63b001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2370758bcaa90479 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63a67fc640 1 -- 172.21.15.33:0/849703811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0017040 con 0x55ad30bd9e20 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63a67fc640 1 -- 172.21.15.33:0/849703811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f63b0013cf0 con 0x55ad30bd9e20 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.941+0000 7f63a67fc640 1 -- 172.21.15.33:0/849703811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0028680 con 0x55ad30bd9e20 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.942+0000 7f63c5bc4740 1 -- 172.21.15.33:0/849703811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30bd9e20 msgr2=0x55ad30be54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:09.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.942+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/849703811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30bd9e20 0x55ad30be54f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009e70 tx=0x7f63b001e040 comp rx=0 tx=0).stop 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.942+0000 7f63c5bc4740 1 -- 172.21.15.33:0/849703811 shutdown_connections 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.942+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/849703811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30be5dc0 0x55ad30bf18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.942+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/849703811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bda740 0x55ad30bd1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.942+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/849703811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30bd9e20 0x55ad30be54f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.942+0000 7f63c5bc4740 1 -- 172.21.15.33:0/849703811 >> 172.21.15.33:0/849703811 conn(0x55ad30bd4c00 msgr2=0x55ad30bd1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.942+0000 7f63c5bc4740 1 -- 172.21.15.33:0/849703811 shutdown_connections 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.942+0000 7f63c5bc4740 1 -- 172.21.15.33:0/849703811 wait complete. 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.942+0000 7f63c5bc4740 1 -- start start 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c5bc4740 1 Processor -- start 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c5bc4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bd9e20 0x55ad30bede00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c5bc4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bda740 0x55ad30bee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c5bc4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30beae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c5bc4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc1ab320 con 0x55ad30be5dc0 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c5bc4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc1ac790 con 0x55ad30bda740 2024-04-02T13:42:09.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c5bc4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc1ab9f0 con 0x55ad30bd9e20 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c5bc4740 10 client.0 _ll_register_callbacks cb 0x7f63b40bef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c1578640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30beae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63bbfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bda740 0x55ad30bee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c1578640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30beae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37612/0 (socket says 172.21.15.33:37612) 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c1578640 1 -- 172.21.15.33:0/3466439454 learned_addr learned my addr 172.21.15.33:0/3466439454 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c0d77640 1 --2- 172.21.15.33:0/3466439454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bd9e20 0x55ad30bede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c1578640 1 -- 172.21.15.33:0/3466439454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bd9e20 msgr2=0x55ad30bede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c1578640 1 --2- 172.21.15.33:0/3466439454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bd9e20 0x55ad30bede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c1578640 1 -- 172.21.15.33:0/3466439454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bda740 msgr2=0x55ad30bee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c1578640 1 --2- 172.21.15.33:0/3466439454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bda740 0x55ad30bee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.943+0000 7f63c1578640 1 -- 172.21.15.33:0/3466439454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad30beb680 con 0x55ad30be5dc0 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f63c1578640 1 --2- 172.21.15.33:0/3466439454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30beae50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f63b4004af0 tx=0x7f63b40bb510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e36be983f5b823a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f639bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:09.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f639bfff640 1 -- 172.21.15.33:0/3466439454 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b40beb00 con 0x55ad30be5dc0 2024-04-02T13:42:09.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f639bfff640 1 -- 172.21.15.33:0/3466439454 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f63b40c7980 con 0x55ad30be5dc0 2024-04-02T13:42:09.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f63c5bc4740 1 -- 172.21.15.33:0/3466439454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b0013cf0 con 0x55ad30be5dc0 2024-04-02T13:42:09.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f639bfff640 1 -- 172.21.15.33:0/3466439454 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f63b40333a0 con 0x55ad30be5dc0 2024-04-02T13:42:09.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f63c5bc4740 10 client.16395 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:09.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f63c5bc4740 1 -- 172.21.15.33:0/3466439454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ad30c5d140 con 0x55ad30be5dc0 2024-04-02T13:42:09.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f639bfff640 1 -- 172.21.15.33:0/3466439454 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f63b0013cf0 con 0x55ad30be5dc0 2024-04-02T13:42:09.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f639bfff640 1 -- 172.21.15.33:0/3466439454 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f63b4004b40 con 0x55ad30be5dc0 2024-04-02T13:42:09.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f63c5bc4740 10 client.16395 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:09.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f63c5bc4740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:09.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f63c5bc4740 20 client.16395 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:09.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/3466439454 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad30c5d700 0x55ad30c5fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:09.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f63c5bc4740 4 client.16395 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:42:09.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f63c5bc4740 1 -- 172.21.15.33:0/3466439454 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ad30b44000 con 0x55ad30c5d700 2024-04-02T13:42:09.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.944+0000 7f63c0d77640 1 --2- 172.21.15.33:0/3466439454 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad30c5d700 0x55ad30c5fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:09.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.945+0000 7f63c0d77640 1 --2- 172.21.15.33:0/3466439454 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad30c5d700 0x55ad30c5fb20 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b02fa5254f4f0b5 server_cookie=53ea14b50a43f7fe in_seq=0 out_seq=0 2024-04-02T13:42:09.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.945+0000 7f639bfff640 10 client.16395 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f639bfff640 1 -- 172.21.15.33:0/3466439454 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ad30b44000 con 0x55ad30c5d700 2024-04-02T13:42:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f639bfff640 10 client.16395 handle_command_reply: tid=0 2024-04-02T13:42:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f63c5bc4740 1 client.16395 shutdown 2024-04-02T13:42:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f63c5bc4740 1 -- 172.21.15.33:0/3466439454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 msgr2=0x55ad30beae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/3466439454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30beae50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f63b4004af0 tx=0x7f63b40bb510 comp rx=0 tx=0).stop 2024-04-02T13:42:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f63c5bc4740 1 -- 172.21.15.33:0/3466439454 shutdown_connections 2024-04-02T13:42:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/3466439454 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad30c5d700 0x55ad30c5fb20 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/3466439454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad30be5dc0 0x55ad30beae50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/3466439454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad30bda740 0x55ad30bee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f639bfff640 0 client.16395 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f63c5bc4740 1 --2- 172.21.15.33:0/3466439454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad30bd9e20 0x55ad30bede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f63c5bc4740 1 -- 172.21.15.33:0/3466439454 >> 172.21.15.33:0/3466439454 conn(0x55ad30bd4c00 msgr2=0x55ad30bd6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f63c5bc4740 1 -- 172.21.15.33:0/3466439454 shutdown_connections 2024-04-02T13:42:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f63c5bc4740 1 -- 172.21.15.33:0/3466439454 wait complete. 2024-04-02T13:42:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.946+0000 7f63c5bc4740 20 client.16395 trim_cache size 0 max 16384 2024-04-02T13:42:09.955 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:42:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:42:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "3e930239-6f3e-48f4-9282-b4dd3d774589", 2024-04-02T13:42:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:42:09.956 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:42:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.947+0000 7f63c2fdb640 1 -- 172.21.15.33:0/430812863 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f63ac0a3da0 msgr2=0x7f63ac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.947+0000 7f63c2fdb640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f63ac0a3da0 0x7f63ac0a6260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f63bc105020 tx=0x7f63a80097c0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.947+0000 7f63c2fdb640 1 -- 172.21.15.33:0/430812863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc106980 msgr2=0x7f63bc19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.947+0000 7f63c2fdb640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc106980 0x7f63bc19bb80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f63b4005cc0 tx=0x7f63b4034b40 comp rx=0 tx=0).stop 2024-04-02T13:42:09.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.947+0000 7f63c2fdb640 1 -- 172.21.15.33:0/430812863 shutdown_connections 2024-04-02T13:42:09.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.947+0000 7f63c2fdb640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f63ac0a3da0 0x7f63ac0a6260 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.947+0000 7f63c2fdb640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc106980 0x7f63bc19bb80 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.947+0000 7f63c2fdb640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63bc101330 0x7f63bc197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.947+0000 7f63c2fdb640 1 --2- 172.21.15.33:0/430812863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63bc100970 0x7f63bc1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:09.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.947+0000 7f63c2fdb640 1 -- 172.21.15.33:0/430812863 >> 172.21.15.33:0/430812863 conn(0x7f63bc077b00 msgr2=0x7f63bc103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:09.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.947+0000 7f63c2fdb640 1 -- 172.21.15.33:0/430812863 shutdown_connections 2024-04-02T13:42:09.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:09.947+0000 7f63c2fdb640 1 -- 172.21.15.33:0/430812863 wait complete. 2024-04-02T13:42:09.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:42:09.960 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:42:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.069+0000 7f10578b7640 1 -- 172.21.15.33:0/1449638386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1050068ce0 msgr2=0x7f1050069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.069+0000 7f10578b7640 1 --2- 172.21.15.33:0/1449638386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1050068ce0 0x7f1050069160 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f1044009930 tx=0x7f104402bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.069+0000 7f10578b7640 1 -- 172.21.15.33:0/1449638386 shutdown_connections 2024-04-02T13:42:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.069+0000 7f10578b7640 1 --2- 172.21.15.33:0/1449638386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10500696a0 0x7f105010e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.069+0000 7f10578b7640 1 --2- 172.21.15.33:0/1449638386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1050068ce0 0x7f1050069160 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.069+0000 7f10578b7640 1 --2- 172.21.15.33:0/1449638386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1050107a80 0x7f1050107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.069+0000 7f10578b7640 1 -- 172.21.15.33:0/1449638386 >> 172.21.15.33:0/1449638386 conn(0x7f1050077b00 msgr2=0x7f1050077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.069+0000 7f10578b7640 1 -- 172.21.15.33:0/1449638386 shutdown_connections 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.069+0000 7f10578b7640 1 -- 172.21.15.33:0/1449638386 wait complete. 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f10578b7640 1 Processor -- start 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f10578b7640 1 -- start start 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f10578b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1050068ce0 0x7f10501983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f10578b7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10500696a0 0x7f1050198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f10578b7640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1050107a80 0x7f105019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f10578b7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1050111e60 con 0x7f1050068ce0 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f10578b7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1050111ce0 con 0x7f10500696a0 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f10578b7640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1050111fe0 con 0x7f1050107a80 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f1055653640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1050068ce0 0x7f10501983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f1055653640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1050068ce0 0x7f10501983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37640/0 (socket says 172.21.15.33:37640) 2024-04-02T13:42:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f1055653640 1 -- 172.21.15.33:0/2499165727 learned_addr learned my addr 172.21.15.33:0/2499165727 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f1054e52640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10500696a0 0x7f1050198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f1055e54640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1050107a80 0x7f105019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f1055653640 1 -- 172.21.15.33:0/2499165727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1050107a80 msgr2=0x7f105019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f1055653640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1050107a80 0x7f105019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f1055653640 1 -- 172.21.15.33:0/2499165727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10500696a0 msgr2=0x7f1050198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f1055653640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10500696a0 0x7f1050198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.070+0000 7f1055653640 1 -- 172.21.15.33:0/2499165727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105019d3b0 con 0x7f1050068ce0 2024-04-02T13:42:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.071+0000 7f1055653640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1050068ce0 0x7f10501983c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f104000ebc0 tx=0x7f104000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.071+0000 7f104e7fc640 1 -- 172.21.15.33:0/2499165727 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040010630 con 0x7f1050068ce0 2024-04-02T13:42:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.071+0000 7f104e7fc640 1 -- 172.21.15.33:0/2499165727 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1040010ca0 con 0x7f1050068ce0 2024-04-02T13:42:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.071+0000 7f10578b7640 1 -- 172.21.15.33:0/2499165727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105019d6a0 con 0x7f1050068ce0 2024-04-02T13:42:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.071+0000 7f104e7fc640 1 -- 172.21.15.33:0/2499165727 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040018c00 con 0x7f1050068ce0 2024-04-02T13:42:10.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.071+0000 7f10578b7640 1 -- 172.21.15.33:0/2499165727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10501a4f30 con 0x7f1050068ce0 2024-04-02T13:42:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.073+0000 7f104e7fc640 1 -- 172.21.15.33:0/2499165727 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f10400107d0 con 0x7f1050068ce0 2024-04-02T13:42:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.073+0000 7f105a57b740 1 Processor -- start 2024-04-02T13:42:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.073+0000 7f105a57b740 1 -- start start 2024-04-02T13:42:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f105a57b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 0x557221eeecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f105a57b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221eef1f0 0x557221ef7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f105a57b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221ef7dd0 0x557221efa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f105a57b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557221e52830 con 0x557221eee8d0 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f105a57b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557221dfa820 con 0x557221ef7dd0 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f104e7fc640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f10380a3d20 0x7f10380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f105a57b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557221e4e7e0 con 0x557221eef1f0 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1055e54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 0x557221eeecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1055653640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221eef1f0 0x557221ef7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1054e52640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221ef7dd0 0x557221efa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1055e54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 0x557221eeecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37646/0 (socket says 172.21.15.33:37646) 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1055e54640 1 -- 172.21.15.33:0/15369546 learned_addr learned my addr 172.21.15.33:0/15369546 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1055e54640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f10380a3d20 0x7f10380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f104e7fc640 1 -- 172.21.15.33:0/2499165727 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f10400d1190 con 0x7f1050068ce0 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1055653640 1 -- 172.21.15.33:0/15369546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221ef7dd0 msgr2=0x557221efa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1055653640 1 --2- 172.21.15.33:0/15369546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221ef7dd0 0x557221efa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1055653640 1 -- 172.21.15.33:0/15369546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 msgr2=0x557221eeecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1055653640 1 --2- 172.21.15.33:0/15369546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 0x557221eeecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1055653640 1 -- 172.21.15.33:0/15369546 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557221c307d0 con 0x557221eef1f0 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1055e54640 1 --2- 172.21.15.33:0/15369546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 0x557221eeecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1055653640 1 --2- 172.21.15.33:0/15369546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221eef1f0 0x557221ef7890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10400ce530 tx=0x7f1040000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb95e917efc87e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.074+0000 7f1055e54640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f10380a3d20 0x7f10380a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1048004650 tx=0x7f1048005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.075+0000 7f10377fe640 1 -- 172.21.15.33:0/15369546 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104009c790 con 0x557221eef1f0 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.075+0000 7f10377fe640 1 -- 172.21.15.33:0/15369546 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1040010ac0 con 0x557221eef1f0 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.075+0000 7f10377fe640 1 -- 172.21.15.33:0/15369546 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040099610 con 0x557221eef1f0 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.075+0000 7f105a57b740 1 -- 172.21.15.33:0/15369546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221eef1f0 msgr2=0x557221ef7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.075+0000 7f105a57b740 1 --2- 172.21.15.33:0/15369546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221eef1f0 0x557221ef7890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10400ce530 tx=0x7f1040000f40 comp rx=0 tx=0).stop 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.075+0000 7f105a57b740 1 -- 172.21.15.33:0/15369546 shutdown_connections 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.075+0000 7f105a57b740 1 --2- 172.21.15.33:0/15369546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221ef7dd0 0x557221efa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.075+0000 7f105a57b740 1 --2- 172.21.15.33:0/15369546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221eef1f0 0x557221ef7890 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.075+0000 7f105a57b740 1 --2- 172.21.15.33:0/15369546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 0x557221eeecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.075+0000 7f105a57b740 1 -- 172.21.15.33:0/15369546 >> 172.21.15.33:0/15369546 conn(0x557221ee9000 msgr2=0x557221ee93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.075+0000 7f105a57b740 1 -- 172.21.15.33:0/15369546 shutdown_connections 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.075+0000 7f105a57b740 1 -- 172.21.15.33:0/15369546 wait complete. 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f105a57b740 1 -- start start 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f105a57b740 1 Processor -- start 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f105a57b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221eee8d0 0x557221ef7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f105a57b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 0x557221ef4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f105a57b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221ef7dd0 0x557221ef4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f105a57b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557221efab00 con 0x557221eef1f0 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f105a57b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557221e0dd30 con 0x557221eee8d0 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f105a57b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557221efac80 con 0x557221ef7dd0 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f105a57b740 10 client.0 _ll_register_callbacks cb 0x557221e0d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f1054e52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 0x557221ef4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f1055653640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221eee8d0 0x557221ef7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f1054e52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 0x557221ef4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37658/0 (socket says 172.21.15.33:37658) 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f1054e52640 1 -- 172.21.15.33:0/528518802 learned_addr learned my addr 172.21.15.33:0/528518802 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.076+0000 7f1055e54640 1 --2- 172.21.15.33:0/528518802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221ef7dd0 0x557221ef4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f1054e52640 1 -- 172.21.15.33:0/528518802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221ef7dd0 msgr2=0x557221ef4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f1054e52640 1 --2- 172.21.15.33:0/528518802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221ef7dd0 0x557221ef4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f1054e52640 1 -- 172.21.15.33:0/528518802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221eee8d0 msgr2=0x557221ef7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f1054e52640 1 --2- 172.21.15.33:0/528518802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221eee8d0 0x557221ef7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f1054e52640 1 -- 172.21.15.33:0/528518802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557221b663f0 con 0x557221eef1f0 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f1055e54640 1 --2- 172.21.15.33:0/528518802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221ef7dd0 0x557221ef4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f1054e52640 1 --2- 172.21.15.33:0/528518802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 0x557221ef4a10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1044009230 tx=0x7f1044005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e45f90a4eab1fb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f1034ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f1034ff9640 1 -- 172.21.15.33:0/528518802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044046070 con 0x557221eef1f0 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f105a57b740 1 -- 172.21.15.33:0/528518802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1040010ac0 con 0x557221eef1f0 2024-04-02T13:42:10.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f1034ff9640 1 -- 172.21.15.33:0/528518802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f104402c5d0 con 0x557221eef1f0 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f1034ff9640 1 -- 172.21.15.33:0/528518802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f10440055d0 con 0x557221eef1f0 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f105a57b740 10 client.16407 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f105a57b740 1 -- 172.21.15.33:0/528518802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557221b260f0 con 0x557221eef1f0 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.077+0000 7f1034ff9640 1 -- 172.21.15.33:0/528518802 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1040010ac0 con 0x557221eef1f0 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.078+0000 7f1034ff9640 1 -- 172.21.15.33:0/528518802 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1044004be0 con 0x557221eef1f0 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.078+0000 7f105a57b740 10 client.16407 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.078+0000 7f105a57b740 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.078+0000 7f105a57b740 20 client.16407 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.078+0000 7f105a57b740 1 --2- 172.21.15.33:0/528518802 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x557221f07c40 0x557221f0a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.078+0000 7f105a57b740 4 client.16407 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.078+0000 7f105a57b740 1 -- 172.21.15.33:0/528518802 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557221e5fd70 con 0x557221f07c40 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.078+0000 7f1055653640 1 --2- 172.21.15.33:0/528518802 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x557221f07c40 0x557221f0a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.078+0000 7f1055653640 1 --2- 172.21.15.33:0/528518802 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x557221f07c40 0x557221f0a0c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba77f2abc7dcf62e server_cookie=558f7ca0a8418641 in_seq=0 out_seq=0 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.078+0000 7f1034ff9640 10 client.16407 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f1034ff9640 1 -- 172.21.15.33:0/528518802 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557221e5fd70 con 0x557221f07c40 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f1034ff9640 10 client.16407 handle_command_reply: tid=0 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f105a57b740 1 client.16407 shutdown 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f105a57b740 1 -- 172.21.15.33:0/528518802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 msgr2=0x557221ef4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f105a57b740 1 --2- 172.21.15.33:0/528518802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 0x557221ef4a10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1044009230 tx=0x7f1044005250 comp rx=0 tx=0).stop 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f105a57b740 1 -- 172.21.15.33:0/528518802 shutdown_connections 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f105a57b740 1 --2- 172.21.15.33:0/528518802 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x557221f07c40 0x557221f0a0c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f105a57b740 1 --2- 172.21.15.33:0/528518802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221ef7dd0 0x557221ef4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f105a57b740 1 --2- 172.21.15.33:0/528518802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 0x557221ef4a10 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f1034ff9640 0 client.16407 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f105a57b740 1 --2- 172.21.15.33:0/528518802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221eee8d0 0x557221ef7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f105a57b740 1 -- 172.21.15.33:0/528518802 >> 172.21.15.33:0/528518802 conn(0x557221ee9000 msgr2=0x557221eead50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f105a57b740 1 -- 172.21.15.33:0/528518802 shutdown_connections 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f105a57b740 1 -- 172.21.15.33:0/528518802 wait complete. 2024-04-02T13:42:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.080+0000 7f105a57b740 20 client.16407 trim_cache size 0 max 16384 2024-04-02T13:42:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f105a57b740 1 Processor -- start 2024-04-02T13:42:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f105a57b740 1 -- start start 2024-04-02T13:42:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f105a57b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221eee8d0 0x557221ee8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f105a57b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 0x557221eeaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f105a57b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221ef7dd0 0x557221ef7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f105a57b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f10501b1920 con 0x557221eef1f0 2024-04-02T13:42:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f105a57b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f10501af130 con 0x557221ef7dd0 2024-04-02T13:42:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f105a57b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10501acf60 con 0x557221eee8d0 2024-04-02T13:42:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f1054e52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 0x557221eeaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f1054e52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 0x557221eeaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37674/0 (socket says 172.21.15.33:37674) 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f1054e52640 1 -- 172.21.15.33:0/1822083577 learned_addr learned my addr 172.21.15.33:0/1822083577 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f1055e54640 1 --2- 172.21.15.33:0/1822083577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221ef7dd0 0x557221ef7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f1055653640 1 --2- 172.21.15.33:0/1822083577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221eee8d0 0x557221ee8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f1054e52640 1 -- 172.21.15.33:0/1822083577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221eee8d0 msgr2=0x557221ee8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f1054e52640 1 --2- 172.21.15.33:0/1822083577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221eee8d0 0x557221ee8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f1054e52640 1 -- 172.21.15.33:0/1822083577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221ef7dd0 msgr2=0x557221ef7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f1054e52640 1 --2- 172.21.15.33:0/1822083577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221ef7dd0 0x557221ef7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f1054e52640 1 -- 172.21.15.33:0/1822083577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557221ee8e30 con 0x557221eef1f0 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f1055653640 1 --2- 172.21.15.33:0/1822083577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221eee8d0 0x557221ee8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.100+0000 7f1054e52640 1 --2- 172.21.15.33:0/1822083577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 0x557221eeaaf0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1044009850 tx=0x7f104402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe72486d887235 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.101+0000 7f1036ffd640 1 -- 172.21.15.33:0/1822083577 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044046040 con 0x557221eef1f0 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.101+0000 7f1036ffd640 1 -- 172.21.15.33:0/1822083577 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1044009520 con 0x557221eef1f0 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.101+0000 7f1036ffd640 1 -- 172.21.15.33:0/1822083577 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104402c5d0 con 0x557221eef1f0 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.101+0000 7f105a57b740 1 -- 172.21.15.33:0/1822083577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 msgr2=0x557221eeaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.101+0000 7f105a57b740 1 --2- 172.21.15.33:0/1822083577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 0x557221eeaaf0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1044009850 tx=0x7f104402bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.101+0000 7f105a57b740 1 -- 172.21.15.33:0/1822083577 shutdown_connections 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.101+0000 7f105a57b740 1 --2- 172.21.15.33:0/1822083577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221ef7dd0 0x557221ef7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.101+0000 7f105a57b740 1 --2- 172.21.15.33:0/1822083577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eef1f0 0x557221eeaaf0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.101+0000 7f105a57b740 1 --2- 172.21.15.33:0/1822083577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221eee8d0 0x557221ee8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.101+0000 7f105a57b740 1 -- 172.21.15.33:0/1822083577 >> 172.21.15.33:0/1822083577 conn(0x557221ee9000 msgr2=0x557221eea3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.101+0000 7f105a57b740 1 -- 172.21.15.33:0/1822083577 shutdown_connections 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.101+0000 7f105a57b740 1 -- 172.21.15.33:0/1822083577 wait complete. 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f105a57b740 1 -- start start 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f105a57b740 1 Processor -- start 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f105a57b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 0x557221ef1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f105a57b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221eef1f0 0x557221ef1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f105a57b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221ef7dd0 0x557221ef22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f105a57b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f10501ad5f0 con 0x557221eee8d0 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f105a57b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f10501aea60 con 0x557221eef1f0 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f105a57b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10501adcc0 con 0x557221ef7dd0 2024-04-02T13:42:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f105a57b740 10 client.0 _ll_register_callbacks cb 0x7f1044004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f1055653640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 0x557221ef1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f1055653640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 0x557221ef1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37688/0 (socket says 172.21.15.33:37688) 2024-04-02T13:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f1055653640 1 -- 172.21.15.33:0/2144364794 learned_addr learned my addr 172.21.15.33:0/2144364794 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f1054e52640 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221eef1f0 0x557221ef1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.102+0000 7f1055e54640 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221ef7dd0 0x557221ef22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f1055653640 1 -- 172.21.15.33:0/2144364794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221ef7dd0 msgr2=0x557221ef22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f1055653640 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221ef7dd0 0x557221ef22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f1055653640 1 -- 172.21.15.33:0/2144364794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221eef1f0 msgr2=0x557221ef1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f1055653640 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221eef1f0 0x557221ef1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f1055653640 1 -- 172.21.15.33:0/2144364794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557221eee2d0 con 0x557221eee8d0 2024-04-02T13:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f1055e54640 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221ef7dd0 0x557221ef22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f1055653640 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 0x557221ef1820 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f10400d1140 tx=0x7f104000ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b53b2ec5ac16d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f1034ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f1034ff9640 1 -- 172.21.15.33:0/2144364794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10400ba070 con 0x557221eee8d0 2024-04-02T13:42:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f1034ff9640 1 -- 172.21.15.33:0/2144364794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f104009b130 con 0x557221eee8d0 2024-04-02T13:42:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f105a57b740 1 -- 172.21.15.33:0/2144364794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1044009520 con 0x557221eee8d0 2024-04-02T13:42:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f1034ff9640 1 -- 172.21.15.33:0/2144364794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1040010ac0 con 0x557221eee8d0 2024-04-02T13:42:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f105a57b740 10 client.16413 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f105a57b740 1 -- 172.21.15.33:0/2144364794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557221f6e450 con 0x557221eee8d0 2024-04-02T13:42:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f1034ff9640 1 -- 172.21.15.33:0/2144364794 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1044009520 con 0x557221eee8d0 2024-04-02T13:42:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f1034ff9640 1 -- 172.21.15.33:0/2144364794 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f10400ce7b0 con 0x557221eee8d0 2024-04-02T13:42:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f105a57b740 10 client.16413 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f105a57b740 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:10.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f105a57b740 20 client.16413 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:10.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f105a57b740 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x557221f6e850 0x557221f70c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:10.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f105a57b740 4 client.16413 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:42:10.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.103+0000 7f105a57b740 1 -- 172.21.15.33:0/2144364794 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557221e5a0c0 con 0x557221f6e850 2024-04-02T13:42:10.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.104+0000 7f1054e52640 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x557221f6e850 0x557221f70c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:10.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.104+0000 7f1054e52640 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x557221f6e850 0x557221f70c40 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6d3d86c80ff81b6 server_cookie=f72db7df585254c9 in_seq=0 out_seq=0 2024-04-02T13:42:10.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.104+0000 7f1034ff9640 10 client.16413 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f1034ff9640 1 -- 172.21.15.33:0/2144364794 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x557221e5a0c0 con 0x557221f6e850 2024-04-02T13:42:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f1034ff9640 10 client.16413 handle_command_reply: tid=0 2024-04-02T13:42:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f105a57b740 1 client.16413 shutdown 2024-04-02T13:42:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f105a57b740 1 -- 172.21.15.33:0/2144364794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 msgr2=0x557221ef1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f105a57b740 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 0x557221ef1820 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f10400d1140 tx=0x7f104000ef50 comp rx=0 tx=0).stop 2024-04-02T13:42:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f105a57b740 1 -- 172.21.15.33:0/2144364794 shutdown_connections 2024-04-02T13:42:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f105a57b740 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x557221f6e850 0x557221f70c40 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f105a57b740 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557221ef7dd0 0x557221ef22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f1034ff9640 0 client.16413 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f105a57b740 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557221eef1f0 0x557221ef1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f105a57b740 1 --2- 172.21.15.33:0/2144364794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557221eee8d0 0x557221ef1820 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f105a57b740 1 -- 172.21.15.33:0/2144364794 >> 172.21.15.33:0/2144364794 conn(0x557221ee9000 msgr2=0x557221eeac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:10.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f105a57b740 1 -- 172.21.15.33:0/2144364794 shutdown_connections 2024-04-02T13:42:10.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f105a57b740 1 -- 172.21.15.33:0/2144364794 wait complete. 2024-04-02T13:42:10.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.106+0000 7f105a57b740 20 client.16413 trim_cache size 0 max 16384 2024-04-02T13:42:10.115 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:42:10.115 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "scrub active (953 inodes in the stack)", 2024-04-02T13:42:10.115 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:42:10.115 INFO:teuthology.orchestra.run.smithi033.stdout: "3e930239-6f3e-48f4-9282-b4dd3d774589": { 2024-04-02T13:42:10.115 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:42:10.115 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "3e930239-6f3e-48f4-9282-b4dd3d774589", 2024-04-02T13:42:10.116 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:42:10.116 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:42:10.116 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:42:10.116 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:42:10.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.107+0000 7f10578b7640 1 -- 172.21.15.33:0/2499165727 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f10380a3d20 msgr2=0x7f10380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:10.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.107+0000 7f10578b7640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f10380a3d20 0x7f10380a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1048004650 tx=0x7f1048005f90 comp rx=0 tx=0).stop 2024-04-02T13:42:10.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.107+0000 7f10578b7640 1 -- 172.21.15.33:0/2499165727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1050068ce0 msgr2=0x7f10501983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:10.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.107+0000 7f10578b7640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1050068ce0 0x7f10501983c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f104000ebc0 tx=0x7f104000c6a0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.107+0000 7f10578b7640 1 -- 172.21.15.33:0/2499165727 shutdown_connections 2024-04-02T13:42:10.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.107+0000 7f10578b7640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f10380a3d20 0x7f10380a61e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.107+0000 7f10578b7640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1050107a80 0x7f105019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.107+0000 7f10578b7640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10500696a0 0x7f1050198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.107+0000 7f10578b7640 1 --2- 172.21.15.33:0/2499165727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1050068ce0 0x7f10501983c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:10.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.107+0000 7f10578b7640 1 -- 172.21.15.33:0/2499165727 >> 172.21.15.33:0/2499165727 conn(0x7f1050077b00 msgr2=0x7f10500fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:10.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.107+0000 7f10578b7640 1 -- 172.21.15.33:0/2499165727 shutdown_connections 2024-04-02T13:42:10.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:10.107+0000 7f10578b7640 1 -- 172.21.15.33:0/2499165727 wait complete. 2024-04-02T13:42:10.119 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:10 smithi033 ceph-mon[32390]: pgmap v104: 129 pgs: 129 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 251 op/s 2024-04-02T13:42:10.120 INFO:tasks.cephfs.filesystem:scrub status for tag:3e930239-6f3e-48f4-9282-b4dd3d774589 - {'path': '/', 'tag': '3e930239-6f3e-48f4-9282-b4dd3d774589', 'options': 'recursive,force'} 2024-04-02T13:42:10.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:10 smithi145 ceph-mon[35646]: pgmap v104: 129 pgs: 129 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 251 op/s 2024-04-02T13:42:10.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:10 smithi050 ceph-mon[35524]: pgmap v104: 129 pgs: 129 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 251 op/s 2024-04-02T13:42:11.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:42:11.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:11.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:11.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:11.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:11.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:11.480 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:11 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:11.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:11.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:11.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:11.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:11.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:11 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:42:11.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:11.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:11.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:11.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:11.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:11.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:11 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:12.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:12 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:12.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:12 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:12.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:12 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:12.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:12 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:12.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:12 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:12.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:12 smithi145 ceph-mon[35646]: pgmap v105: 129 pgs: 129 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-04-02T13:42:12.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:12 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:12.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:12 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:12.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:12 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:12.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:12 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:12.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:12 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:12.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:12 smithi050 ceph-mon[35524]: pgmap v105: 129 pgs: 129 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-04-02T13:42:12.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:12 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:12.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:12 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:12.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:12 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:12.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:12 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:12.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:12 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:12.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:12 smithi033 ceph-mon[32390]: pgmap v105: 129 pgs: 129 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-04-02T13:42:14.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:14 smithi145 ceph-mon[35646]: pgmap v106: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-04-02T13:42:14.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:14 smithi050 ceph-mon[35524]: pgmap v106: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-04-02T13:42:14.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:14 smithi033 ceph-mon[32390]: pgmap v106: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-04-02T13:42:16.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:16 smithi145 ceph-mon[35646]: pgmap v107: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-04-02T13:42:16.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:16 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:42:16.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:16 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:42:16.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:16 smithi050 ceph-mon[35524]: pgmap v107: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-04-02T13:42:16.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:16 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:42:16.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:16 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:42:16.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:16 smithi033 ceph-mon[32390]: pgmap v107: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-04-02T13:42:16.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:16 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:42:16.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:16 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:42:16.891 INFO:tasks.workunit:Stopping ['fs/norstats'] on client.0... 2024-04-02T13:42:16.891 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-02T13:42:17.171 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:17 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:42:17.172 DEBUG:teuthology.parallel:result is None 2024-04-02T13:42:17.173 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-02T13:42:17.211 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-02T13:42:17.211 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-02T13:42:17.222 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-02T13:42:17.222 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T13:42:17.222 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-02T13:42:17.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-02T13:42:17.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-02T13:42:17.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:17 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:42:17.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:17 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:42:18.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:18 smithi145 ceph-mon[35646]: pgmap v108: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-04-02T13:42:18.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:18 smithi033 ceph-mon[32390]: pgmap v108: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-04-02T13:42:18.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:18 smithi050 ceph-mon[35524]: pgmap v108: 129 pgs: 129 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-04-02T13:42:20.306 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:20 smithi033 ceph-mon[32390]: pgmap v109: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 354 op/s 2024-04-02T13:42:20.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:20 smithi145 ceph-mon[35646]: pgmap v109: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 354 op/s 2024-04-02T13:42:20.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:20 smithi050 ceph-mon[35524]: pgmap v109: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 354 op/s 2024-04-02T13:42:22.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:22 smithi145 ceph-mon[35646]: pgmap v110: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 226 op/s 2024-04-02T13:42:22.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:22 smithi050 ceph-mon[35524]: pgmap v110: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 226 op/s 2024-04-02T13:42:22.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:22 smithi033 ceph-mon[32390]: pgmap v110: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 226 op/s 2024-04-02T13:42:24.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:24 smithi145 ceph-mon[35646]: pgmap v111: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 258 op/s 2024-04-02T13:42:24.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:24 smithi050 ceph-mon[35524]: pgmap v111: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 258 op/s 2024-04-02T13:42:24.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:24 smithi033 ceph-mon[32390]: pgmap v111: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 258 op/s 2024-04-02T13:42:26.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:26 smithi145 ceph-mon[35646]: pgmap v112: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 606 KiB/s wr, 156 op/s 2024-04-02T13:42:26.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:26 smithi050 ceph-mon[35524]: pgmap v112: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 606 KiB/s wr, 156 op/s 2024-04-02T13:42:26.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:26 smithi033 ceph-mon[32390]: pgmap v112: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 606 KiB/s wr, 156 op/s 2024-04-02T13:42:28.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:28 smithi145 ceph-mon[35646]: pgmap v113: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 606 KiB/s wr, 156 op/s 2024-04-02T13:42:28.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:28 smithi050 ceph-mon[35524]: pgmap v113: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 606 KiB/s wr, 156 op/s 2024-04-02T13:42:28.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:28 smithi033 ceph-mon[32390]: pgmap v113: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 606 KiB/s wr, 156 op/s 2024-04-02T13:42:30.481 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:30 smithi145 ceph-mon[35646]: pgmap v114: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 610 KiB/s wr, 159 op/s 2024-04-02T13:42:30.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:30 smithi050 ceph-mon[35524]: pgmap v114: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 610 KiB/s wr, 159 op/s 2024-04-02T13:42:30.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:30 smithi033 ceph-mon[32390]: pgmap v114: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 610 KiB/s wr, 159 op/s 2024-04-02T13:42:32.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:32 smithi145 ceph-mon[35646]: pgmap v115: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 35 KiB/s wr, 36 op/s 2024-04-02T13:42:32.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:32 smithi050 ceph-mon[35524]: pgmap v115: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 35 KiB/s wr, 36 op/s 2024-04-02T13:42:32.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:32 smithi033 ceph-mon[32390]: pgmap v115: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 35 KiB/s wr, 36 op/s 2024-04-02T13:42:34.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:34 smithi145 ceph-mon[35646]: pgmap v116: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 35 KiB/s wr, 39 op/s 2024-04-02T13:42:34.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:34 smithi050 ceph-mon[35524]: pgmap v116: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 35 KiB/s wr, 39 op/s 2024-04-02T13:42:34.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:34 smithi033 ceph-mon[32390]: pgmap v116: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 35 KiB/s wr, 39 op/s 2024-04-02T13:42:36.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:36 smithi145 ceph-mon[35646]: pgmap v117: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-02T13:42:36.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:36 smithi050 ceph-mon[35524]: pgmap v117: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-02T13:42:36.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:36 smithi033 ceph-mon[32390]: pgmap v117: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-02T13:42:38.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:38 smithi145 ceph-mon[35646]: pgmap v118: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-02T13:42:38.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:38 smithi050 ceph-mon[35524]: pgmap v118: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-02T13:42:38.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:38 smithi033 ceph-mon[32390]: pgmap v118: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-02T13:42:40.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:42:40.121 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:42:40.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.235+0000 7f71c8782640 1 -- 172.21.15.33:0/4036962016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71c0102d70 msgr2=0x7f71c0103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.235+0000 7f71c8782640 1 --2- 172.21.15.33:0/4036962016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71c0102d70 0x7f71c0103150 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009900 tx=0x7f71bc02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.235+0000 7f71c8782640 1 -- 172.21.15.33:0/4036962016 shutdown_connections 2024-04-02T13:42:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.235+0000 7f71c8782640 1 --2- 172.21.15.33:0/4036962016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71c010c140 0x7f71c010e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.235+0000 7f71c8782640 1 --2- 172.21.15.33:0/4036962016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71c0103690 0x7f71c010bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.235+0000 7f71c8782640 1 --2- 172.21.15.33:0/4036962016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71c0102d70 0x7f71c0103150 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.235+0000 7f71c8782640 1 -- 172.21.15.33:0/4036962016 >> 172.21.15.33:0/4036962016 conn(0x7f71c0077b60 msgr2=0x7f71c0077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.236+0000 7f71c8782640 1 -- 172.21.15.33:0/4036962016 shutdown_connections 2024-04-02T13:42:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.236+0000 7f71c8782640 1 -- 172.21.15.33:0/4036962016 wait complete. 2024-04-02T13:42:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.236+0000 7f71c8782640 1 Processor -- start 2024-04-02T13:42:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.236+0000 7f71c8782640 1 -- start start 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.236+0000 7f71c8782640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71c0102d70 0x7f71c01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.236+0000 7f71c8782640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71c0103690 0x7f71c01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.236+0000 7f71c8782640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71c010c140 0x7f71c019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.236+0000 7f71c8782640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0111ee0 con 0x7f71c0103690 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.236+0000 7f71c8782640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0111d60 con 0x7f71c010c140 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.236+0000 7f71c8782640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0112060 con 0x7f71c0102d70 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.236+0000 7f71c5d1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71c0103690 0x7f71c01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.236+0000 7f71c5d1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71c0103690 0x7f71c01988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39286/0 (socket says 172.21.15.33:39286) 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.236+0000 7f71c5d1d640 1 -- 172.21.15.33:0/3123968192 learned_addr learned my addr 172.21.15.33:0/3123968192 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.237+0000 7f71c651e640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71c0102d70 0x7f71c01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.237+0000 7f71c6d1f640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71c010c140 0x7f71c019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.237+0000 7f71c5d1d640 1 -- 172.21.15.33:0/3123968192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71c0102d70 msgr2=0x7f71c01983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.237+0000 7f71c5d1d640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71c0102d70 0x7f71c01983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.237+0000 7f71c5d1d640 1 -- 172.21.15.33:0/3123968192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71c010c140 msgr2=0x7f71c019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.237+0000 7f71c5d1d640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71c010c140 0x7f71c019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.237+0000 7f71c5d1d640 1 -- 172.21.15.33:0/3123968192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c019d340 con 0x7f71c0103690 2024-04-02T13:42:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.237+0000 7f71c5d1d640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71c0103690 0x7f71c01988e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f71b000c920 tx=0x7f71b000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.237+0000 7f71b77fe640 1 -- 172.21.15.33:0/3123968192 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0007c70 con 0x7f71c0103690 2024-04-02T13:42:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.237+0000 7f71b77fe640 1 -- 172.21.15.33:0/3123968192 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f71b0007e10 con 0x7f71c0103690 2024-04-02T13:42:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.237+0000 7f71c8782640 1 -- 172.21.15.33:0/3123968192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c019d630 con 0x7f71c0103690 2024-04-02T13:42:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.237+0000 7f71b77fe640 1 -- 172.21.15.33:0/3123968192 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0018850 con 0x7f71c0103690 2024-04-02T13:42:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.237+0000 7f71c8782640 1 -- 172.21.15.33:0/3123968192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c01a4ec0 con 0x7f71c0103690 2024-04-02T13:42:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.240+0000 7f71b77fe640 1 -- 172.21.15.33:0/3123968192 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f71b0018a30 con 0x7f71c0103690 2024-04-02T13:42:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.240+0000 7f71cb36a740 1 Processor -- start 2024-04-02T13:42:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.240+0000 7f71cb36a740 1 -- start start 2024-04-02T13:42:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71cb36a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504238d0 0x560f50423cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71cb36a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f504241f0 0x560f5042c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71cb36a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 0x560f5042f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71cb36a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560f50387830 con 0x560f5042cdd0 2024-04-02T13:42:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71cb36a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560f5032f820 con 0x560f504238d0 2024-04-02T13:42:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71cb36a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560f503837e0 con 0x560f504241f0 2024-04-02T13:42:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71c5d1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 0x560f5042f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71b77fe640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f71ac0a3d20 0x7f71ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71c5d1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 0x560f5042f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39296/0 (socket says 172.21.15.33:39296) 2024-04-02T13:42:40.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71c5d1d640 1 -- 172.21.15.33:0/1821498303 learned_addr learned my addr 172.21.15.33:0/1821498303 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71c6d1f640 1 --2- 172.21.15.33:0/1821498303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504238d0 0x560f50423cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71c651e640 1 --2- 172.21.15.33:0/1821498303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f504241f0 0x560f5042c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71c6d1f640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f71ac0a3d20 0x7f71ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71b77fe640 1 -- 172.21.15.33:0/3123968192 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f71b00d0c30 con 0x7f71c0103690 2024-04-02T13:42:40.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71c5d1d640 1 -- 172.21.15.33:0/1821498303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f504241f0 msgr2=0x560f5042c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71c5d1d640 1 --2- 172.21.15.33:0/1821498303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f504241f0 0x560f5042c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71c5d1d640 1 -- 172.21.15.33:0/1821498303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504238d0 msgr2=0x560f50423cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71c5d1d640 1 --2- 172.21.15.33:0/1821498303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504238d0 0x560f50423cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71c5d1d640 1 -- 172.21.15.33:0/1821498303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f501657d0 con 0x560f5042cdd0 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.241+0000 7f71c6d1f640 1 --2- 172.21.15.33:0/1821498303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504238d0 0x560f50423cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.242+0000 7f71c5d1d640 1 --2- 172.21.15.33:0/1821498303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 0x560f5042f1c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f71b001b000 tx=0x7f71b000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83fd05c6a156104 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.242+0000 7f71c6d1f640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f71ac0a3d20 0x7f71ac0a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71b8005fd0 tx=0x7f71b8009500 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.242+0000 7f71b4ff9640 1 -- 172.21.15.33:0/1821498303 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b009cf20 con 0x560f5042cdd0 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.242+0000 7f71b4ff9640 1 -- 172.21.15.33:0/1821498303 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f71b009c820 con 0x560f5042cdd0 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.242+0000 7f71b4ff9640 1 -- 172.21.15.33:0/1821498303 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b00ce250 con 0x560f5042cdd0 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.242+0000 7f71cb36a740 1 -- 172.21.15.33:0/1821498303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 msgr2=0x560f5042f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.242+0000 7f71cb36a740 1 --2- 172.21.15.33:0/1821498303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 0x560f5042f1c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f71b001b000 tx=0x7f71b000cf90 comp rx=0 tx=0).stop 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.242+0000 7f71cb36a740 1 -- 172.21.15.33:0/1821498303 shutdown_connections 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.242+0000 7f71cb36a740 1 --2- 172.21.15.33:0/1821498303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 0x560f5042f1c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.242+0000 7f71cb36a740 1 --2- 172.21.15.33:0/1821498303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f504241f0 0x560f5042c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71cb36a740 1 --2- 172.21.15.33:0/1821498303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504238d0 0x560f50423cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71cb36a740 1 -- 172.21.15.33:0/1821498303 >> 172.21.15.33:0/1821498303 conn(0x560f5041e000 msgr2=0x560f5041e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71cb36a740 1 -- 172.21.15.33:0/1821498303 shutdown_connections 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71cb36a740 1 -- 172.21.15.33:0/1821498303 wait complete. 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71cb36a740 1 -- start start 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71cb36a740 1 Processor -- start 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71cb36a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f504238d0 0x560f5042c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71cb36a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50429ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71cb36a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 0x560f5042a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71cb36a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560f5042fb00 con 0x560f5042cdd0 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71cb36a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560f50342d30 con 0x560f504241f0 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71cb36a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560f5042fc80 con 0x560f504238d0 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71cb36a740 10 client.0 _ll_register_callbacks cb 0x560f50342720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71c6d1f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 0x560f5042a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71c6d1f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 0x560f5042a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39304/0 (socket says 172.21.15.33:39304) 2024-04-02T13:42:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.243+0000 7f71c6d1f640 1 -- 172.21.15.33:0/1250671561 learned_addr learned my addr 172.21.15.33:0/1250671561 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71c5d1d640 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50429ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71c651e640 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f504238d0 0x560f5042c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71c6d1f640 1 -- 172.21.15.33:0/1250671561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f504238d0 msgr2=0x560f5042c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71c6d1f640 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f504238d0 0x560f5042c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71c6d1f640 1 -- 172.21.15.33:0/1250671561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 msgr2=0x560f50429ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71c6d1f640 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50429ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71c6d1f640 1 -- 172.21.15.33:0/1250671561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f5009b3f0 con 0x560f5042cdd0 2024-04-02T13:42:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71c5d1d640 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50429ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71c6d1f640 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 0x560f5042a010 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f71b8011fd0 tx=0x7f71b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9991d48783eb2ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:40.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71a9ffb640 1 -- 172.21.15.33:0/1250671561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b802e060 con 0x560f5042cdd0 2024-04-02T13:42:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71a9ffb640 1 -- 172.21.15.33:0/1250671561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f71b8021b90 con 0x560f5042cdd0 2024-04-02T13:42:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71cb36a740 1 -- 172.21.15.33:0/1250671561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b009c820 con 0x560f5042cdd0 2024-04-02T13:42:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71a9ffb640 1 -- 172.21.15.33:0/1250671561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f71b8035050 con 0x560f5042cdd0 2024-04-02T13:42:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71cb36a740 10 client.16425 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.244+0000 7f71cb36a740 1 -- 172.21.15.33:0/1250671561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f5005b0f0 con 0x560f5042cdd0 2024-04-02T13:42:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.245+0000 7f71a9ffb640 1 -- 172.21.15.33:0/1250671561 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f71b009c820 con 0x560f5042cdd0 2024-04-02T13:42:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.245+0000 7f71a9ffb640 1 -- 172.21.15.33:0/1250671561 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f71b801d070 con 0x560f5042cdd0 2024-04-02T13:42:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.245+0000 7f71cb36a740 10 client.16425 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.245+0000 7f71cb36a740 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.245+0000 7f71cb36a740 20 client.16425 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.245+0000 7f71cb36a740 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x560f5043ce10 0x560f5043f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.245+0000 7f71cb36a740 4 client.16425 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.245+0000 7f71cb36a740 1 -- 172.21.15.33:0/1250671561 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f50394d70 con 0x560f5043ce10 2024-04-02T13:42:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.245+0000 7f71c651e640 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x560f5043ce10 0x560f5043f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.246+0000 7f71a9ffb640 10 client.16425 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.246+0000 7f71c651e640 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x560f5043ce10 0x560f5043f230 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13f4c97807dafacf server_cookie=3e3160d9bed6501c in_seq=0 out_seq=0 2024-04-02T13:42:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.246+0000 7f71a9ffb640 1 -- 172.21.15.33:0/1250671561 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f50394d70 con 0x560f5043ce10 2024-04-02T13:42:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.246+0000 7f71a9ffb640 10 client.16425 handle_command_reply: tid=0 2024-04-02T13:42:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.246+0000 7f71cb36a740 1 client.16425 shutdown 2024-04-02T13:42:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.246+0000 7f71cb36a740 1 -- 172.21.15.33:0/1250671561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 msgr2=0x560f5042a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.246+0000 7f71cb36a740 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 0x560f5042a010 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f71b8011fd0 tx=0x7f71b8004a40 comp rx=0 tx=0).stop 2024-04-02T13:42:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.246+0000 7f71cb36a740 1 -- 172.21.15.33:0/1250671561 shutdown_connections 2024-04-02T13:42:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.246+0000 7f71cb36a740 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x560f5043ce10 0x560f5043f230 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.246+0000 7f71cb36a740 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f5042cdd0 0x560f5042a010 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.246+0000 7f71cb36a740 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50429ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.247+0000 7f71cb36a740 1 --2- 172.21.15.33:0/1250671561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f504238d0 0x560f5042c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.246+0000 7f71a9ffb640 0 client.16425 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.247+0000 7f71cb36a740 1 -- 172.21.15.33:0/1250671561 >> 172.21.15.33:0/1250671561 conn(0x560f5041e000 msgr2=0x560f5041fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.247+0000 7f71cb36a740 1 -- 172.21.15.33:0/1250671561 shutdown_connections 2024-04-02T13:42:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.247+0000 7f71cb36a740 1 -- 172.21.15.33:0/1250671561 wait complete. 2024-04-02T13:42:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.247+0000 7f71cb36a740 20 client.16425 trim_cache size 0 max 16384 2024-04-02T13:42:40.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.264+0000 7f71cb36a740 1 Processor -- start 2024-04-02T13:42:40.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.264+0000 7f71cb36a740 1 -- start start 2024-04-02T13:42:40.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.264+0000 7f71cb36a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 0x560f504389c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.264+0000 7f71cb36a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50428380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.264+0000 7f71cb36a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f5042cdd0 0x560f5042c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.264+0000 7f71cb36a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01b18c0 con 0x560f504238d0 2024-04-02T13:42:40.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.264+0000 7f71cb36a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01af0d0 con 0x560f504241f0 2024-04-02T13:42:40.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.264+0000 7f71cb36a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01acf00 con 0x560f5042cdd0 2024-04-02T13:42:40.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.264+0000 7f71c651e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 0x560f504389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.265+0000 7f71c651e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 0x560f504389c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39316/0 (socket says 172.21.15.33:39316) 2024-04-02T13:42:40.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.265+0000 7f71c651e640 1 -- 172.21.15.33:0/2167073546 learned_addr learned my addr 172.21.15.33:0/2167073546 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.265+0000 7f71c5d1d640 1 --2- 172.21.15.33:0/2167073546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50428380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.265+0000 7f71c651e640 1 -- 172.21.15.33:0/2167073546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f5042cdd0 msgr2=0x560f5042c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.265+0000 7f71c651e640 1 --2- 172.21.15.33:0/2167073546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f5042cdd0 0x560f5042c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.265+0000 7f71c651e640 1 -- 172.21.15.33:0/2167073546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 msgr2=0x560f50428380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.265+0000 7f71c651e640 1 --2- 172.21.15.33:0/2167073546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50428380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.265+0000 7f71c651e640 1 -- 172.21.15.33:0/2167073546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f5041de30 con 0x560f504238d0 2024-04-02T13:42:40.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.265+0000 7f71c5d1d640 1 --2- 172.21.15.33:0/2167073546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50428380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:40.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.265+0000 7f71c651e640 1 --2- 172.21.15.33:0/2167073546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 0x560f504389c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f71bc036040 tx=0x7f71bc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94bcee63acc3b54 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.265+0000 7f71abfff640 1 -- 172.21.15.33:0/2167073546 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc004290 con 0x560f504238d0 2024-04-02T13:42:40.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.265+0000 7f71abfff640 1 -- 172.21.15.33:0/2167073546 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f71bc04b8e0 con 0x560f504238d0 2024-04-02T13:42:40.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.265+0000 7f71abfff640 1 -- 172.21.15.33:0/2167073546 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0054f0 con 0x560f504238d0 2024-04-02T13:42:40.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 -- 172.21.15.33:0/2167073546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 msgr2=0x560f504389c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 --2- 172.21.15.33:0/2167073546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 0x560f504389c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f71bc036040 tx=0x7f71bc005fd0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 -- 172.21.15.33:0/2167073546 shutdown_connections 2024-04-02T13:42:40.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 --2- 172.21.15.33:0/2167073546 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f5042cdd0 0x560f5042c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 --2- 172.21.15.33:0/2167073546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50428380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 --2- 172.21.15.33:0/2167073546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 0x560f504389c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 -- 172.21.15.33:0/2167073546 >> 172.21.15.33:0/2167073546 conn(0x560f5041e000 msgr2=0x560f504250c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 -- 172.21.15.33:0/2167073546 shutdown_connections 2024-04-02T13:42:40.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 -- 172.21.15.33:0/2167073546 wait complete. 2024-04-02T13:42:40.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 -- start start 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 Processor -- start 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 0x560f50435300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50430670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f5042cdd0 0x560f50430bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01ad590 con 0x560f504238d0 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01aea00 con 0x560f504241f0 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01adc60 con 0x560f5042cdd0 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71cb36a740 10 client.0 _ll_register_callbacks cb 0x7f71b8019040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71c651e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 0x560f50435300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71c651e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 0x560f50435300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39318/0 (socket says 172.21.15.33:39318) 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71c651e640 1 -- 172.21.15.33:0/2984164622 learned_addr learned my addr 172.21.15.33:0/2984164622 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.266+0000 7f71c5d1d640 1 --2- 172.21.15.33:0/2984164622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50430670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71c6d1f640 1 --2- 172.21.15.33:0/2984164622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f5042cdd0 0x560f50430bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71c651e640 1 -- 172.21.15.33:0/2984164622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f5042cdd0 msgr2=0x560f50430bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71c651e640 1 --2- 172.21.15.33:0/2984164622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f5042cdd0 0x560f50430bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71c651e640 1 -- 172.21.15.33:0/2984164622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 msgr2=0x560f50430670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71c651e640 1 --2- 172.21.15.33:0/2984164622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50430670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71c651e640 1 -- 172.21.15.33:0/2984164622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f50431400 con 0x560f504238d0 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71c651e640 1 --2- 172.21.15.33:0/2984164622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 0x560f50435300 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f71bc04d040 tx=0x7f71bc005490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2e4d8b81142fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71a9ffb640 1 -- 172.21.15.33:0/2984164622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc059020 con 0x560f504238d0 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71a9ffb640 1 -- 172.21.15.33:0/2984164622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f71bc04bbd0 con 0x560f504238d0 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71cb36a740 1 -- 172.21.15.33:0/2984164622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71bc04b8e0 con 0x560f504238d0 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71a9ffb640 1 -- 172.21.15.33:0/2984164622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f71bc061050 con 0x560f504238d0 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71cb36a740 10 client.16431 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71cb36a740 1 -- 172.21.15.33:0/2984164622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f504a7820 con 0x560f504238d0 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.267+0000 7f71a9ffb640 1 -- 172.21.15.33:0/2984164622 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f71bc04b8e0 con 0x560f504238d0 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.268+0000 7f71a9ffb640 1 -- 172.21.15.33:0/2984164622 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f71bc054070 con 0x560f504238d0 2024-04-02T13:42:40.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.268+0000 7f71cb36a740 10 client.16431 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.268+0000 7f71cb36a740 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.268+0000 7f71cb36a740 20 client.16431 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.268+0000 7f71cb36a740 1 --2- 172.21.15.33:0/2984164622 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x560f504a7f10 0x560f504aa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.268+0000 7f71cb36a740 4 client.16431 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.268+0000 7f71cb36a740 1 -- 172.21.15.33:0/2984164622 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f5038f0c0 con 0x560f504a7f10 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.268+0000 7f71c5d1d640 1 --2- 172.21.15.33:0/2984164622 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x560f504a7f10 0x560f504aa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.268+0000 7f71a9ffb640 10 client.16431 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.268+0000 7f71c5d1d640 1 --2- 172.21.15.33:0/2984164622 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x560f504a7f10 0x560f504aa330 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99f5816f2235e233 server_cookie=f07984d4b21cebba in_seq=0 out_seq=0 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.268+0000 7f71a9ffb640 1 -- 172.21.15.33:0/2984164622 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560f5038f0c0 con 0x560f504a7f10 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.268+0000 7f71a9ffb640 10 client.16431 handle_command_reply: tid=0 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.268+0000 7f71cb36a740 1 client.16431 shutdown 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71cb36a740 1 -- 172.21.15.33:0/2984164622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 msgr2=0x560f50435300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71cb36a740 1 --2- 172.21.15.33:0/2984164622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 0x560f50435300 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f71bc04d040 tx=0x7f71bc005490 comp rx=0 tx=0).stop 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71cb36a740 1 -- 172.21.15.33:0/2984164622 shutdown_connections 2024-04-02T13:42:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71cb36a740 1 --2- 172.21.15.33:0/2984164622 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x560f504a7f10 0x560f504aa330 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71cb36a740 1 --2- 172.21.15.33:0/2984164622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560f5042cdd0 0x560f50430bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71a9ffb640 0 client.16431 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:40.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71cb36a740 1 --2- 172.21.15.33:0/2984164622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f504241f0 0x560f50430670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71cb36a740 1 --2- 172.21.15.33:0/2984164622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f504238d0 0x560f50435300 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71cb36a740 1 -- 172.21.15.33:0/2984164622 >> 172.21.15.33:0/2984164622 conn(0x560f5041e000 msgr2=0x560f50424710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71cb36a740 1 -- 172.21.15.33:0/2984164622 shutdown_connections 2024-04-02T13:42:40.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71cb36a740 1 -- 172.21.15.33:0/2984164622 wait complete. 2024-04-02T13:42:40.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71cb36a740 20 client.16431 trim_cache size 0 max 16384 2024-04-02T13:42:40.280 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:42:40.280 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:42:40.281 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:42:40.281 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:42:40.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71c8782640 1 -- 172.21.15.33:0/3123968192 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f71ac0a3d20 msgr2=0x7f71ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71c8782640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f71ac0a3d20 0x7f71ac0a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71b8005fd0 tx=0x7f71b8009500 comp rx=0 tx=0).stop 2024-04-02T13:42:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71c8782640 1 -- 172.21.15.33:0/3123968192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71c0103690 msgr2=0x7f71c01988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71c8782640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71c0103690 0x7f71c01988e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f71b000c920 tx=0x7f71b000cde0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71c8782640 1 -- 172.21.15.33:0/3123968192 shutdown_connections 2024-04-02T13:42:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71c8782640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f71ac0a3d20 0x7f71ac0a61e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71c8782640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71c010c140 0x7f71c019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71c8782640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71c0103690 0x7f71c01988e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71c8782640 1 --2- 172.21.15.33:0/3123968192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f71c0102d70 0x7f71c01983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71c8782640 1 -- 172.21.15.33:0/3123968192 >> 172.21.15.33:0/3123968192 conn(0x7f71c0077b60 msgr2=0x7f71c00fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71c8782640 1 -- 172.21.15.33:0/3123968192 shutdown_connections 2024-04-02T13:42:40.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.269+0000 7f71c8782640 1 -- 172.21.15.33:0/3123968192 wait complete. 2024-04-02T13:42:40.284 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:42:40.284 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:42:40.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.387+0000 7fab25375640 1 -- 172.21.15.33:0/195776189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab20101150 msgr2=0x7fab201015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.387+0000 7fab25375640 1 --2- 172.21.15.33:0/195776189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab20101150 0x7fab201015b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fab10009900 tx=0x7fab1002bc80 comp rx=0 tx=0).stop 2024-04-02T13:42:40.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.387+0000 7fab25375640 1 -- 172.21.15.33:0/195776189 shutdown_connections 2024-04-02T13:42:40.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.387+0000 7fab25375640 1 --2- 172.21.15.33:0/195776189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab20101af0 0x7fab2010d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.387+0000 7fab25375640 1 --2- 172.21.15.33:0/195776189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab20101150 0x7fab201015b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.387+0000 7fab25375640 1 --2- 172.21.15.33:0/195776189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab201059b0 0x7fab20105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.387+0000 7fab25375640 1 -- 172.21.15.33:0/195776189 >> 172.21.15.33:0/195776189 conn(0x7fab20076f90 msgr2=0x7fab200fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.387+0000 7fab25375640 1 -- 172.21.15.33:0/195776189 shutdown_connections 2024-04-02T13:42:40.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.387+0000 7fab25375640 1 -- 172.21.15.33:0/195776189 wait complete. 2024-04-02T13:42:40.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.387+0000 7fab25375640 1 Processor -- start 2024-04-02T13:42:40.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab25375640 1 -- start start 2024-04-02T13:42:40.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab25375640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab20101150 0x7fab201983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab25375640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab20101af0 0x7fab20198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab25375640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab201059b0 0x7fab2019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab25375640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab20110e80 con 0x7fab201059b0 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab25375640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fab20110d00 con 0x7fab20101af0 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab25375640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fab20111000 con 0x7fab20101150 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab1f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab201059b0 0x7fab2019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab1f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab201059b0 0x7fab2019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39334/0 (socket says 172.21.15.33:39334) 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab1f7fe640 1 -- 172.21.15.33:0/3413283768 learned_addr learned my addr 172.21.15.33:0/3413283768 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab1e7fc640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab20101af0 0x7fab20198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab1effd640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab20101150 0x7fab201983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab1f7fe640 1 -- 172.21.15.33:0/3413283768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab20101150 msgr2=0x7fab201983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab1f7fe640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab20101150 0x7fab201983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab1f7fe640 1 -- 172.21.15.33:0/3413283768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab20101af0 msgr2=0x7fab20198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab1f7fe640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab20101af0 0x7fab20198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab1f7fe640 1 -- 172.21.15.33:0/3413283768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2019d380 con 0x7fab201059b0 2024-04-02T13:42:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab1f7fe640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab201059b0 0x7fab2019cc50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fab1800cce0 tx=0x7fab1800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab07fff640 1 -- 172.21.15.33:0/3413283768 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1800f450 con 0x7fab201059b0 2024-04-02T13:42:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab07fff640 1 -- 172.21.15.33:0/3413283768 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fab1800fac0 con 0x7fab201059b0 2024-04-02T13:42:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab25375640 1 -- 172.21.15.33:0/3413283768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2019d670 con 0x7fab201059b0 2024-04-02T13:42:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab07fff640 1 -- 172.21.15.33:0/3413283768 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18018be0 con 0x7fab201059b0 2024-04-02T13:42:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.388+0000 7fab25375640 1 -- 172.21.15.33:0/3413283768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab201a4f00 con 0x7fab201059b0 2024-04-02T13:42:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.390+0000 7fab25375640 1 -- 172.21.15.33:0/3413283768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab20068d40 con 0x7fab201059b0 2024-04-02T13:42:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.391+0000 7fab07fff640 1 -- 172.21.15.33:0/3413283768 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab1800fc70 con 0x7fab201059b0 2024-04-02T13:42:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.391+0000 7fab07fff640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fab080a3d20 0x7fab080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.392+0000 7fab1effd640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fab080a3d20 0x7fab080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.392+0000 7fab07fff640 1 -- 172.21.15.33:0/3413283768 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fab180d1250 con 0x7fab201059b0 2024-04-02T13:42:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.392+0000 7fab1effd640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fab080a3d20 0x7fab080a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fab14005fd0 tx=0x7fab14005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.396+0000 7fab07fff640 1 -- 172.21.15.33:0/3413283768 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fab18014030 con 0x7fab201059b0 2024-04-02T13:42:40.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:40 smithi145 ceph-mon[35646]: pgmap v119: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-02T13:42:40.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:40 smithi050 ceph-mon[35524]: pgmap v119: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-02T13:42:40.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:40 smithi033 ceph-mon[32390]: pgmap v119: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-02T13:42:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.624+0000 7fab25375640 1 -- 172.21.15.33:0/3413283768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab20109ed0 con 0x7fab201059b0 2024-04-02T13:42:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.624+0000 7fab07fff640 1 -- 172.21.15.33:0/3413283768 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11882 (secure 0 0 0) 0x7fab1809d580 con 0x7fab201059b0 2024-04-02T13:42:40.628 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:42:40.628 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:41:39.222860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:42:40.629 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-02T13:42:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.626+0000 7fab25375640 1 -- 172.21.15.33:0/3413283768 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fab080a3d20 msgr2=0x7fab080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.627+0000 7fab25375640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fab080a3d20 0x7fab080a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fab14005fd0 tx=0x7fab14005850 comp rx=0 tx=0).stop 2024-04-02T13:42:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.627+0000 7fab25375640 1 -- 172.21.15.33:0/3413283768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab201059b0 msgr2=0x7fab2019cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.627+0000 7fab25375640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab201059b0 0x7fab2019cc50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fab1800cce0 tx=0x7fab1800a620 comp rx=0 tx=0).stop 2024-04-02T13:42:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.627+0000 7fab25375640 1 -- 172.21.15.33:0/3413283768 shutdown_connections 2024-04-02T13:42:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.627+0000 7fab25375640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fab080a3d20 0x7fab080a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.627+0000 7fab25375640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab201059b0 0x7fab2019cc50 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.627+0000 7fab25375640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab20101af0 0x7fab20198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.627+0000 7fab25375640 1 --2- 172.21.15.33:0/3413283768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab20101150 0x7fab201983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.627+0000 7fab25375640 1 -- 172.21.15.33:0/3413283768 >> 172.21.15.33:0/3413283768 conn(0x7fab20076f90 msgr2=0x7fab20100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.627+0000 7fab25375640 1 -- 172.21.15.33:0/3413283768 shutdown_connections 2024-04-02T13:42:40.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.627+0000 7fab25375640 1 -- 172.21.15.33:0/3413283768 wait complete. 2024-04-02T13:42:40.642 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:42:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.743+0000 7efc8b160640 1 -- 172.21.15.33:0/128795249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc84101aa0 msgr2=0x7efc84101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.743+0000 7efc8b160640 1 --2- 172.21.15.33:0/128795249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc84101aa0 0x7efc84101f20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7efc70009930 tx=0x7efc7002bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.743+0000 7efc8b160640 1 -- 172.21.15.33:0/128795249 shutdown_connections 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.743+0000 7efc8b160640 1 --2- 172.21.15.33:0/128795249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc84102460 0x7efc8410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.743+0000 7efc8b160640 1 --2- 172.21.15.33:0/128795249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc84101aa0 0x7efc84101f20 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.743+0000 7efc8b160640 1 --2- 172.21.15.33:0/128795249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc84107ab0 0x7efc84107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.743+0000 7efc8b160640 1 -- 172.21.15.33:0/128795249 >> 172.21.15.33:0/128795249 conn(0x7efc840fd530 msgr2=0x7efc840ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.743+0000 7efc8b160640 1 -- 172.21.15.33:0/128795249 shutdown_connections 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.743+0000 7efc8b160640 1 -- 172.21.15.33:0/128795249 wait complete. 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc8b160640 1 Processor -- start 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc8b160640 1 -- start start 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc8b160640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc84101aa0 0x7efc84198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc8b160640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc84102460 0x7efc841989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc8b160640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc84107ab0 0x7efc8419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc8b160640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc84111e90 con 0x7efc84101aa0 2024-04-02T13:42:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc8b160640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efc84111d10 con 0x7efc84107ab0 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc8b160640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efc84112010 con 0x7efc84102460 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc88efc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc84101aa0 0x7efc84198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc88efc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc84101aa0 0x7efc84198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39350/0 (socket says 172.21.15.33:39350) 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc88efc640 1 -- 172.21.15.33:0/1803473785 learned_addr learned my addr 172.21.15.33:0/1803473785 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc896fd640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc84107ab0 0x7efc8419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc83fff640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc84102460 0x7efc841989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc88efc640 1 -- 172.21.15.33:0/1803473785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc84102460 msgr2=0x7efc841989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc88efc640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc84102460 0x7efc841989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc88efc640 1 -- 172.21.15.33:0/1803473785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc84107ab0 msgr2=0x7efc8419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc88efc640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc84107ab0 0x7efc8419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.744+0000 7efc88efc640 1 -- 172.21.15.33:0/1803473785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8419d480 con 0x7efc84101aa0 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.745+0000 7efc88efc640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc84101aa0 0x7efc84198490 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efc7800c970 tx=0x7efc7800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.745+0000 7efc81ffb640 1 -- 172.21.15.33:0/1803473785 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78007bb0 con 0x7efc84101aa0 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.745+0000 7efc81ffb640 1 -- 172.21.15.33:0/1803473785 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efc78007d50 con 0x7efc84101aa0 2024-04-02T13:42:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.745+0000 7efc8b160640 1 -- 172.21.15.33:0/1803473785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8419d770 con 0x7efc84101aa0 2024-04-02T13:42:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.745+0000 7efc81ffb640 1 -- 172.21.15.33:0/1803473785 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7800f450 con 0x7efc84101aa0 2024-04-02T13:42:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.745+0000 7efc8b160640 1 -- 172.21.15.33:0/1803473785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc841a5000 con 0x7efc84101aa0 2024-04-02T13:42:40.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc81ffb640 1 -- 172.21.15.33:0/1803473785 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efc78004fb0 con 0x7efc84101aa0 2024-04-02T13:42:40.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc8de25740 1 Processor -- start 2024-04-02T13:42:40.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc8de25740 1 -- start start 2024-04-02T13:42:40.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc8de25740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 0x55d7a11c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc8de25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c20c0 0x55d7a11ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc8de25740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11caca0 0x55d7a11cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc8de25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d7a11257b0 con 0x55d7a11c20c0 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc8de25740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d7a10c4570 con 0x55d7a11c17a0 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc8de25740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d7a1121760 con 0x55d7a11caca0 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc81ffb640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efc740a3d20 0x7efc740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc88efc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c20c0 0x55d7a11ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc88efc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c20c0 0x55d7a11ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39356/0 (socket says 172.21.15.33:39356) 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc88efc640 1 -- 172.21.15.33:0/796572841 learned_addr learned my addr 172.21.15.33:0/796572841 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc896fd640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efc740a3d20 0x7efc740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.748+0000 7efc896fd640 1 --2- 172.21.15.33:0/796572841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 0x55d7a11c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.749+0000 7efc83fff640 1 --2- 172.21.15.33:0/796572841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11caca0 0x55d7a11cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.749+0000 7efc81ffb640 1 -- 172.21.15.33:0/1803473785 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7efc780d08e0 con 0x7efc84101aa0 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.749+0000 7efc88efc640 1 -- 172.21.15.33:0/796572841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11caca0 msgr2=0x55d7a11cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.749+0000 7efc88efc640 1 --2- 172.21.15.33:0/796572841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11caca0 0x55d7a11cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.749+0000 7efc88efc640 1 -- 172.21.15.33:0/796572841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 msgr2=0x55d7a11c1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.749+0000 7efc88efc640 1 --2- 172.21.15.33:0/796572841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 0x55d7a11c1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.749+0000 7efc88efc640 1 -- 172.21.15.33:0/796572841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a0e393f0 con 0x55d7a11c20c0 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.749+0000 7efc83fff640 1 --2- 172.21.15.33:0/796572841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11caca0 0x55d7a11cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.749+0000 7efc896fd640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efc740a3d20 0x7efc740a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efc7c005fd0 tx=0x7efc7c0094f0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.749+0000 7efc88efc640 1 --2- 172.21.15.33:0/796572841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c20c0 0x55d7a11ca760 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efc7800cfd0 tx=0x7efc78098cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26624a20645877eb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.749+0000 7efc6effd640 1 -- 172.21.15.33:0/796572841 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7801d070 con 0x55d7a11c20c0 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.749+0000 7efc6effd640 1 -- 172.21.15.33:0/796572841 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efc780bce50 con 0x55d7a11c20c0 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.749+0000 7efc6effd640 1 -- 172.21.15.33:0/796572841 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc780046c0 con 0x55d7a11c20c0 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.750+0000 7efc8de25740 1 -- 172.21.15.33:0/796572841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c20c0 msgr2=0x55d7a11ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.750+0000 7efc8de25740 1 --2- 172.21.15.33:0/796572841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c20c0 0x55d7a11ca760 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efc7800cfd0 tx=0x7efc78098cb0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.750+0000 7efc8de25740 1 -- 172.21.15.33:0/796572841 shutdown_connections 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.750+0000 7efc8de25740 1 --2- 172.21.15.33:0/796572841 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11caca0 0x55d7a11cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.750+0000 7efc8de25740 1 --2- 172.21.15.33:0/796572841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c20c0 0x55d7a11ca760 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.750+0000 7efc8de25740 1 --2- 172.21.15.33:0/796572841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 0x55d7a11c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.750+0000 7efc8de25740 1 -- 172.21.15.33:0/796572841 >> 172.21.15.33:0/796572841 conn(0x55d7a11bbed0 msgr2=0x55d7a11bc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.750+0000 7efc8de25740 1 -- 172.21.15.33:0/796572841 shutdown_connections 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.750+0000 7efc8de25740 1 -- 172.21.15.33:0/796572841 wait complete. 2024-04-02T13:42:40.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.750+0000 7efc8de25740 1 -- start start 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.750+0000 7efc8de25740 1 Processor -- start 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc8de25740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 0x55d7a11ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc8de25740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11c20c0 0x55d7a11c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc8de25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 0x55d7a11c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc8de25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d7a11cda90 con 0x55d7a11caca0 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc8de25740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d7a10e0e80 con 0x55d7a11c17a0 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc8de25740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d7a11cdc10 con 0x55d7a11c20c0 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc8de25740 10 client.0 _ll_register_callbacks cb 0x55d7a111caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc896fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 0x55d7a11c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc896fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 0x55d7a11c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39370/0 (socket says 172.21.15.33:39370) 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc896fd640 1 -- 172.21.15.33:0/1330505183 learned_addr learned my addr 172.21.15.33:0/1330505183 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc88efc640 1 --2- 172.21.15.33:0/1330505183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 0x55d7a11ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc83fff640 1 --2- 172.21.15.33:0/1330505183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11c20c0 0x55d7a11c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc896fd640 1 -- 172.21.15.33:0/1330505183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11c20c0 msgr2=0x55d7a11c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc896fd640 1 --2- 172.21.15.33:0/1330505183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11c20c0 0x55d7a11c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc896fd640 1 -- 172.21.15.33:0/1330505183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 msgr2=0x55d7a11ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc896fd640 1 --2- 172.21.15.33:0/1330505183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 0x55d7a11ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc896fd640 1 -- 172.21.15.33:0/1330505183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a0f037d0 con 0x55d7a11caca0 2024-04-02T13:42:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc896fd640 1 --2- 172.21.15.33:0/1330505183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 0x55d7a11c81b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0123c0 tx=0x7efc7c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f054350c8f89bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc63fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc63fff640 1 -- 172.21.15.33:0/1330505183 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c025030 con 0x55d7a11caca0 2024-04-02T13:42:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc63fff640 1 -- 172.21.15.33:0/1330505183 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7efc7c01b070 con 0x55d7a11caca0 2024-04-02T13:42:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.751+0000 7efc8de25740 1 -- 172.21.15.33:0/1330505183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc780bce50 con 0x55d7a11caca0 2024-04-02T13:42:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.752+0000 7efc63fff640 1 -- 172.21.15.33:0/1330505183 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc7c0119f0 con 0x55d7a11caca0 2024-04-02T13:42:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.752+0000 7efc8de25740 10 client.16449 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.752+0000 7efc8de25740 1 -- 172.21.15.33:0/1330505183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d7a11d9ac0 con 0x55d7a11caca0 2024-04-02T13:42:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.752+0000 7efc63fff640 1 -- 172.21.15.33:0/1330505183 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efc780bce50 con 0x55d7a11caca0 2024-04-02T13:42:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.752+0000 7efc63fff640 1 -- 172.21.15.33:0/1330505183 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efc7c01e050 con 0x55d7a11caca0 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.752+0000 7efc8de25740 10 client.16449 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.752+0000 7efc8de25740 10 client.16449 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.752+0000 7efc8de25740 20 client.16449 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.752+0000 7efc8de25740 1 --2- 172.21.15.33:0/1330505183 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d7a11daf20 0x55d7a11dd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.752+0000 7efc8de25740 4 client.16449 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.752+0000 7efc8de25740 1 -- 172.21.15.33:0/1330505183 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7a1132cf0 con 0x55d7a11daf20 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.752+0000 7efc88efc640 1 --2- 172.21.15.33:0/1330505183 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d7a11daf20 0x55d7a11dd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.753+0000 7efc88efc640 1 --2- 172.21.15.33:0/1330505183 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d7a11daf20 0x55d7a11dd380 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89b46e5fa2a51ee6 server_cookie=8ceeb8b2e58758d9 in_seq=0 out_seq=0 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.753+0000 7efc63fff640 10 client.16449 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.753+0000 7efc63fff640 1 -- 172.21.15.33:0/1330505183 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7a1132cf0 con 0x55d7a11daf20 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.753+0000 7efc63fff640 10 client.16449 handle_command_reply: tid=0 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.753+0000 7efc8de25740 1 client.16449 shutdown 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.753+0000 7efc8de25740 1 -- 172.21.15.33:0/1330505183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 msgr2=0x55d7a11c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.753+0000 7efc8de25740 1 --2- 172.21.15.33:0/1330505183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 0x55d7a11c81b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0123c0 tx=0x7efc7c004ab0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.753+0000 7efc8de25740 1 -- 172.21.15.33:0/1330505183 shutdown_connections 2024-04-02T13:42:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.753+0000 7efc8de25740 1 --2- 172.21.15.33:0/1330505183 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d7a11daf20 0x55d7a11dd380 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.754+0000 7efc8de25740 1 --2- 172.21.15.33:0/1330505183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 0x55d7a11c81b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.754+0000 7efc8de25740 1 --2- 172.21.15.33:0/1330505183 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11c20c0 0x55d7a11c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.754+0000 7efc8de25740 1 --2- 172.21.15.33:0/1330505183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 0x55d7a11ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.754+0000 7efc63fff640 0 client.16449 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:42:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.754+0000 7efc8de25740 1 -- 172.21.15.33:0/1330505183 >> 172.21.15.33:0/1330505183 conn(0x55d7a11bbed0 msgr2=0x55d7a11bdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.754+0000 7efc8de25740 1 -- 172.21.15.33:0/1330505183 shutdown_connections 2024-04-02T13:42:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.754+0000 7efc8de25740 1 -- 172.21.15.33:0/1330505183 wait complete. 2024-04-02T13:42:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.754+0000 7efc8de25740 20 client.16449 trim_cache size 0 max 16384 2024-04-02T13:42:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc8de25740 1 Processor -- start 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc8de25740 1 -- start start 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc8de25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c17a0 0x55d7a11d6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc8de25740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c20c0 0x55d7a11ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc8de25740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11caca0 0x55d7a11ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc8de25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc841b19c0 con 0x55d7a11c17a0 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc8de25740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efc841af1d0 con 0x55d7a11c20c0 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc8de25740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efc841ad000 con 0x55d7a11caca0 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc88efc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c17a0 0x55d7a11d6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc88efc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c17a0 0x55d7a11d6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39386/0 (socket says 172.21.15.33:39386) 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc88efc640 1 -- 172.21.15.33:0/4137821800 learned_addr learned my addr 172.21.15.33:0/4137821800 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc896fd640 1 --2- 172.21.15.33:0/4137821800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11caca0 0x55d7a11ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc88efc640 1 -- 172.21.15.33:0/4137821800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11caca0 msgr2=0x55d7a11ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc83fff640 1 --2- 172.21.15.33:0/4137821800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c20c0 0x55d7a11ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc88efc640 1 --2- 172.21.15.33:0/4137821800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11caca0 0x55d7a11ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc88efc640 1 -- 172.21.15.33:0/4137821800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c20c0 msgr2=0x55d7a11ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc88efc640 1 --2- 172.21.15.33:0/4137821800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c20c0 0x55d7a11ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc88efc640 1 -- 172.21.15.33:0/4137821800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a11c0ac0 con 0x55d7a11c17a0 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.767+0000 7efc83fff640 1 --2- 172.21.15.33:0/4137821800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c20c0 0x55d7a11ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.768+0000 7efc88efc640 1 --2- 172.21.15.33:0/4137821800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c17a0 0x55d7a11d6980 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efc7809c210 tx=0x7efc780980f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836f634cfe056d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.768+0000 7efc6e7fc640 1 -- 172.21.15.33:0/4137821800 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7801d040 con 0x55d7a11c17a0 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.768+0000 7efc6e7fc640 1 -- 172.21.15.33:0/4137821800 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efc780bce50 con 0x55d7a11c17a0 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.768+0000 7efc6e7fc640 1 -- 172.21.15.33:0/4137821800 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc780046c0 con 0x55d7a11c17a0 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.768+0000 7efc8de25740 1 -- 172.21.15.33:0/4137821800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c17a0 msgr2=0x55d7a11d6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.768+0000 7efc8de25740 1 --2- 172.21.15.33:0/4137821800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c17a0 0x55d7a11d6980 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efc7809c210 tx=0x7efc780980f0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.768+0000 7efc8de25740 1 -- 172.21.15.33:0/4137821800 shutdown_connections 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.768+0000 7efc8de25740 1 --2- 172.21.15.33:0/4137821800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11caca0 0x55d7a11ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.768+0000 7efc8de25740 1 --2- 172.21.15.33:0/4137821800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c20c0 0x55d7a11ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.768+0000 7efc8de25740 1 --2- 172.21.15.33:0/4137821800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11c17a0 0x55d7a11d6980 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.768+0000 7efc8de25740 1 -- 172.21.15.33:0/4137821800 >> 172.21.15.33:0/4137821800 conn(0x55d7a11bbed0 msgr2=0x55d7a11c2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.768+0000 7efc8de25740 1 -- 172.21.15.33:0/4137821800 shutdown_connections 2024-04-02T13:42:40.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.768+0000 7efc8de25740 1 -- 172.21.15.33:0/4137821800 wait complete. 2024-04-02T13:42:40.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc8de25740 1 -- start start 2024-04-02T13:42:40.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc8de25740 1 Processor -- start 2024-04-02T13:42:40.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc8de25740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 0x55d7a11d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc8de25740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11c20c0 0x55d7a11cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc8de25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 0x55d7a11cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc8de25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc841ad690 con 0x55d7a11caca0 2024-04-02T13:42:40.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc8de25740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efc841aeb00 con 0x55d7a11c17a0 2024-04-02T13:42:40.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc8de25740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efc841add60 con 0x55d7a11c20c0 2024-04-02T13:42:40.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc8de25740 10 client.0 _ll_register_callbacks cb 0x7efc7c012040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc896fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 0x55d7a11cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc896fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 0x55d7a11cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39396/0 (socket says 172.21.15.33:39396) 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc896fd640 1 -- 172.21.15.33:0/2330868203 learned_addr learned my addr 172.21.15.33:0/2330868203 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc88efc640 1 --2- 172.21.15.33:0/2330868203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 0x55d7a11d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc896fd640 1 -- 172.21.15.33:0/2330868203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11c20c0 msgr2=0x55d7a11cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc896fd640 1 --2- 172.21.15.33:0/2330868203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11c20c0 0x55d7a11cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc896fd640 1 -- 172.21.15.33:0/2330868203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 msgr2=0x55d7a11d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc896fd640 1 --2- 172.21.15.33:0/2330868203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 0x55d7a11d3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.769+0000 7efc896fd640 1 -- 172.21.15.33:0/2330868203 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a11d0ef0 con 0x55d7a11caca0 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc896fd640 1 --2- 172.21.15.33:0/2330868203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 0x55d7a11cc080 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00d770 tx=0x7efc7c011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda81760f690c87b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc63fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc63fff640 1 -- 172.21.15.33:0/2330868203 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c025070 con 0x55d7a11caca0 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc63fff640 1 -- 172.21.15.33:0/2330868203 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7efc7c012910 con 0x55d7a11caca0 2024-04-02T13:42:40.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc8de25740 1 -- 172.21.15.33:0/2330868203 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc780bce50 con 0x55d7a11caca0 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc63fff640 1 -- 172.21.15.33:0/2330868203 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc7c004b20 con 0x55d7a11caca0 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc8de25740 10 client.16455 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc8de25740 1 -- 172.21.15.33:0/2330868203 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d7a12459c0 con 0x55d7a11caca0 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc63fff640 1 -- 172.21.15.33:0/2330868203 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efc780bce50 con 0x55d7a11caca0 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc63fff640 1 -- 172.21.15.33:0/2330868203 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efc7c01e040 con 0x55d7a11caca0 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc8de25740 10 client.16455 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc8de25740 10 client.16455 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc8de25740 20 client.16455 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc8de25740 1 --2- 172.21.15.33:0/2330868203 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d7a1246110 0x55d7a1248530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc8de25740 4 client.16455 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc8de25740 1 -- 172.21.15.33:0/2330868203 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7a112d040 con 0x55d7a1246110 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.770+0000 7efc88efc640 1 --2- 172.21.15.33:0/2330868203 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d7a1246110 0x55d7a1248530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc88efc640 1 --2- 172.21.15.33:0/2330868203 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d7a1246110 0x55d7a1248530 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5200ea73ac3e3a99 server_cookie=bf67f6d2075de1e0 in_seq=0 out_seq=0 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc63fff640 10 client.16455 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc63fff640 1 -- 172.21.15.33:0/2330868203 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7a112d040 con 0x55d7a1246110 2024-04-02T13:42:40.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc63fff640 10 client.16455 handle_command_reply: tid=0 2024-04-02T13:42:40.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc8de25740 1 client.16455 shutdown 2024-04-02T13:42:40.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc8de25740 1 -- 172.21.15.33:0/2330868203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 msgr2=0x55d7a11cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc8de25740 1 --2- 172.21.15.33:0/2330868203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 0x55d7a11cc080 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00d770 tx=0x7efc7c011970 comp rx=0 tx=0).stop 2024-04-02T13:42:40.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc8de25740 1 -- 172.21.15.33:0/2330868203 shutdown_connections 2024-04-02T13:42:40.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc8de25740 1 --2- 172.21.15.33:0/2330868203 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d7a1246110 0x55d7a1248530 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc8de25740 1 --2- 172.21.15.33:0/2330868203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7a11caca0 0x55d7a11cc080 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc8de25740 1 --2- 172.21.15.33:0/2330868203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7a11c20c0 0x55d7a11cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc8de25740 1 --2- 172.21.15.33:0/2330868203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7a11c17a0 0x55d7a11d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc63fff640 0 client.16455 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:42:40.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc8de25740 1 -- 172.21.15.33:0/2330868203 >> 172.21.15.33:0/2330868203 conn(0x55d7a11bbed0 msgr2=0x55d7a11bd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc8de25740 1 -- 172.21.15.33:0/2330868203 shutdown_connections 2024-04-02T13:42:40.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc8de25740 1 -- 172.21.15.33:0/2330868203 wait complete. 2024-04-02T13:42:40.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.771+0000 7efc8de25740 20 client.16455 trim_cache size 0 max 16384 2024-04-02T13:42:40.782 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:42:40.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.772+0000 7efc8b160640 1 -- 172.21.15.33:0/1803473785 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efc740a3d20 msgr2=0x7efc740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.772+0000 7efc8b160640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efc740a3d20 0x7efc740a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efc7c005fd0 tx=0x7efc7c0094f0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.772+0000 7efc8b160640 1 -- 172.21.15.33:0/1803473785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc84101aa0 msgr2=0x7efc84198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.772+0000 7efc8b160640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc84101aa0 0x7efc84198490 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efc7800c970 tx=0x7efc7800ce30 comp rx=0 tx=0).stop 2024-04-02T13:42:40.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.772+0000 7efc8b160640 1 -- 172.21.15.33:0/1803473785 shutdown_connections 2024-04-02T13:42:40.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.772+0000 7efc8b160640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efc740a3d20 0x7efc740a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.772+0000 7efc8b160640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc84107ab0 0x7efc8419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.772+0000 7efc8b160640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc84102460 0x7efc841989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.772+0000 7efc8b160640 1 --2- 172.21.15.33:0/1803473785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc84101aa0 0x7efc84198490 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.772+0000 7efc8b160640 1 -- 172.21.15.33:0/1803473785 >> 172.21.15.33:0/1803473785 conn(0x7efc840fd530 msgr2=0x7efc840fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.772+0000 7efc8b160640 1 -- 172.21.15.33:0/1803473785 shutdown_connections 2024-04-02T13:42:40.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.772+0000 7efc8b160640 1 -- 172.21.15.33:0/1803473785 wait complete. 2024-04-02T13:42:40.786 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:42:40.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.888+0000 7f1d227cd640 1 -- 172.21.15.33:0/345547166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d1c105130 msgr2=0x7f1d1c105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.888+0000 7f1d227cd640 1 --2- 172.21.15.33:0/345547166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d1c105130 0x7f1d1c105510 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009900 tx=0x7f1d0c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:40.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.888+0000 7f1d227cd640 1 -- 172.21.15.33:0/345547166 shutdown_connections 2024-04-02T13:42:40.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.888+0000 7f1d227cd640 1 --2- 172.21.15.33:0/345547166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d1c100d60 0x7f1d1c10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.888+0000 7f1d227cd640 1 --2- 172.21.15.33:0/345547166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d1c1003c0 0x7f1d1c100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.888+0000 7f1d227cd640 1 --2- 172.21.15.33:0/345547166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d1c105130 0x7f1d1c105510 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.888+0000 7f1d227cd640 1 -- 172.21.15.33:0/345547166 >> 172.21.15.33:0/345547166 conn(0x7f1d1c075d40 msgr2=0x7f1d1c076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.888+0000 7f1d227cd640 1 -- 172.21.15.33:0/345547166 shutdown_connections 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.888+0000 7f1d227cd640 1 -- 172.21.15.33:0/345547166 wait complete. 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d227cd640 1 Processor -- start 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d227cd640 1 -- start start 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d227cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d1c1003c0 0x7f1d1c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d227cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d1c100d60 0x7f1d1c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d227cd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d1c105130 0x7f1d1c1a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d227cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c110db0 con 0x7f1d1c1003c0 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d227cd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c110c30 con 0x7f1d1c105130 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d227cd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c110f30 con 0x7f1d1c100d60 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d1c1003c0 0x7f1d1c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d1c1003c0 0x7f1d1c199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39416/0 (socket says 172.21.15.33:39416) 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d1bfff640 1 -- 172.21.15.33:0/4125281780 learned_addr learned my addr 172.21.15.33:0/4125281780 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d20d6a640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d1c105130 0x7f1d1c1a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d1b7fe640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d1c100d60 0x7f1d1c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d1bfff640 1 -- 172.21.15.33:0/4125281780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d1c100d60 msgr2=0x7f1d1c1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.889+0000 7f1d1bfff640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d1c100d60 0x7f1d1c1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.890+0000 7f1d1bfff640 1 -- 172.21.15.33:0/4125281780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d1c105130 msgr2=0x7f1d1c1a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.890+0000 7f1d1bfff640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d1c105130 0x7f1d1c1a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.890+0000 7f1d1bfff640 1 -- 172.21.15.33:0/4125281780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c1a4730 con 0x7f1d1c1003c0 2024-04-02T13:42:40.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.890+0000 7f1d20d6a640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d1c105130 0x7f1d1c1a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:40.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.890+0000 7f1d1bfff640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d1c1003c0 0x7f1d1c199170 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009230 tx=0x7f1d0c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.890+0000 7f1d197fa640 1 -- 172.21.15.33:0/4125281780 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c043020 con 0x7f1d1c1003c0 2024-04-02T13:42:40.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.890+0000 7f1d227cd640 1 -- 172.21.15.33:0/4125281780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1c1a4900 con 0x7f1d1c1003c0 2024-04-02T13:42:40.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.890+0000 7f1d197fa640 1 -- 172.21.15.33:0/4125281780 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1d0c0054f0 con 0x7f1d1c1003c0 2024-04-02T13:42:40.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.890+0000 7f1d197fa640 1 -- 172.21.15.33:0/4125281780 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c003b60 con 0x7f1d1c1003c0 2024-04-02T13:42:40.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.890+0000 7f1d227cd640 1 -- 172.21.15.33:0/4125281780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d1c1a4df0 con 0x7f1d1c1003c0 2024-04-02T13:42:40.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.892+0000 7f1d197fa640 1 -- 172.21.15.33:0/4125281780 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1d0c046020 con 0x7f1d1c1003c0 2024-04-02T13:42:40.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.892+0000 7f1d253bc740 1 Processor -- start 2024-04-02T13:42:40.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d253bc740 1 -- start start 2024-04-02T13:42:40.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d197fa640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1d040a3d20 0x7f1d040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d253bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d253bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d253bc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 0x560fb42f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d253bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560fb424f7b0 con 0x560fb42eb7a0 2024-04-02T13:42:40.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d253bc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560fb41ee570 con 0x560fb42f4ca0 2024-04-02T13:42:40.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d253bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560fb424b760 con 0x560fb42ec0c0 2024-04-02T13:42:40.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d20d6a640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1d040a3d20 0x7f1d040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d197fa640 1 -- 172.21.15.33:0/4125281780 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1d0c0f2630 con 0x7f1d1c1003c0 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42ebb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39422/0 (socket says 172.21.15.33:39422) 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d1bfff640 1 -- 172.21.15.33:0/1013530500 learned_addr learned my addr 172.21.15.33:0/1013530500 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d20d6a640 1 --2- 172.21.15.33:0/1013530500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 0x560fb42f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d1b7fe640 1 --2- 172.21.15.33:0/1013530500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d1bfff640 1 -- 172.21.15.33:0/1013530500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 msgr2=0x560fb42f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d1bfff640 1 --2- 172.21.15.33:0/1013530500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d1bfff640 1 -- 172.21.15.33:0/1013530500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 msgr2=0x560fb42f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d1bfff640 1 --2- 172.21.15.33:0/1013530500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 0x560fb42f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d1bfff640 1 -- 172.21.15.33:0/1013530500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb3f633f0 con 0x560fb42eb7a0 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d20d6a640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1d040a3d20 0x7f1d040a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1d14005fd0 tx=0x7f1d14014af0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.893+0000 7f1d1bfff640 1 --2- 172.21.15.33:0/1013530500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42ebb80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c006010 tx=0x7f1d0c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753a6287c34dd3ce server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.894+0000 7f1d027fc640 1 -- 172.21.15.33:0/1013530500 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c09b990 con 0x560fb42eb7a0 2024-04-02T13:42:40.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.894+0000 7f1d027fc640 1 -- 172.21.15.33:0/1013530500 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1d0c004c70 con 0x560fb42eb7a0 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.894+0000 7f1d027fc640 1 -- 172.21.15.33:0/1013530500 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c09b7b0 con 0x560fb42eb7a0 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.894+0000 7f1d253bc740 1 -- 172.21.15.33:0/1013530500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 msgr2=0x560fb42ebb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.894+0000 7f1d253bc740 1 --2- 172.21.15.33:0/1013530500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42ebb80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c006010 tx=0x7f1d0c005bd0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.894+0000 7f1d253bc740 1 -- 172.21.15.33:0/1013530500 shutdown_connections 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.894+0000 7f1d253bc740 1 --2- 172.21.15.33:0/1013530500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 0x560fb42f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.894+0000 7f1d253bc740 1 --2- 172.21.15.33:0/1013530500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.894+0000 7f1d253bc740 1 --2- 172.21.15.33:0/1013530500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42ebb80 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.894+0000 7f1d253bc740 1 -- 172.21.15.33:0/1013530500 >> 172.21.15.33:0/1013530500 conn(0x560fb42e5ed0 msgr2=0x560fb42e62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.894+0000 7f1d253bc740 1 -- 172.21.15.33:0/1013530500 shutdown_connections 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.894+0000 7f1d253bc740 1 -- 172.21.15.33:0/1013530500 wait complete. 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d253bc740 1 -- start start 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d253bc740 1 Processor -- start 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d253bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42f4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d253bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d253bc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 0x560fb42f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d253bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560fb424f7b0 con 0x560fb42eb7a0 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d253bc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560fb420ae80 con 0x560fb42f4ca0 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d253bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560fb42f7850 con 0x560fb42ec0c0 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d253bc740 10 client.0 _ll_register_callbacks cb 0x560fb4246aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39436/0 (socket says 172.21.15.33:39436) 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d1bfff640 1 -- 172.21.15.33:0/1445070296 learned_addr learned my addr 172.21.15.33:0/1445070296 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d1b7fe640 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d20d6a640 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 0x560fb42f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d1bfff640 1 -- 172.21.15.33:0/1445070296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 msgr2=0x560fb42f19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d1bfff640 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d1bfff640 1 -- 172.21.15.33:0/1445070296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 msgr2=0x560fb42f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d1bfff640 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 0x560fb42f1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d1bfff640 1 -- 172.21.15.33:0/1445070296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb402d7d0 con 0x560fb42eb7a0 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.895+0000 7f1d1b7fe640 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1d1bfff640 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42f4640 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c09d010 tx=0x7f1d0c036fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc70f9d70884afc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1cfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1cfb7fe640 1 -- 172.21.15.33:0/1445070296 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c0d6070 con 0x560fb42eb7a0 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1cfb7fe640 1 -- 172.21.15.33:0/1445070296 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1d0c003d00 con 0x560fb42eb7a0 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1d253bc740 1 -- 172.21.15.33:0/1445070296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d0c004c70 con 0x560fb42eb7a0 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1cfb7fe640 1 -- 172.21.15.33:0/1445070296 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d0c0dc050 con 0x560fb42eb7a0 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1d253bc740 10 client.16467 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1d253bc740 1 -- 172.21.15.33:0/1445070296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560fb43035b0 con 0x560fb42eb7a0 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1cfb7fe640 1 -- 172.21.15.33:0/1445070296 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1d0c004c70 con 0x560fb42eb7a0 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1cfb7fe640 1 -- 172.21.15.33:0/1445070296 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1d0c084080 con 0x560fb42eb7a0 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1d253bc740 10 client.16467 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1d253bc740 10 client.16467 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:42:40.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1d253bc740 20 client.16467 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1d253bc740 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x560fb43049c0 0x560fb4306db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1d253bc740 4 client.16467 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.896+0000 7f1d253bc740 1 -- 172.21.15.33:0/1445070296 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fb425ccf0 con 0x560fb43049c0 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.897+0000 7f1d1b7fe640 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x560fb43049c0 0x560fb4306db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.897+0000 7f1d1b7fe640 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x560fb43049c0 0x560fb4306db0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35c0bff7abf42020 server_cookie=a3a672a8facc54e8 in_seq=0 out_seq=0 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.897+0000 7f1cfb7fe640 10 client.16467 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1cfb7fe640 1 -- 172.21.15.33:0/1445070296 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560fb425ccf0 con 0x560fb43049c0 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1cfb7fe640 10 client.16467 handle_command_reply: tid=0 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1d253bc740 1 client.16467 shutdown 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1d253bc740 1 -- 172.21.15.33:0/1445070296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 msgr2=0x560fb42f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1d253bc740 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42f4640 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c09d010 tx=0x7f1d0c036fb0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1d253bc740 1 -- 172.21.15.33:0/1445070296 shutdown_connections 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1d253bc740 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 0x560fb42f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1d253bc740 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x560fb43049c0 0x560fb4306db0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1d253bc740 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1d253bc740 1 --2- 172.21.15.33:0/1445070296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42f4640 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1cfb7fe640 0 client.16467 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:42:40.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1d253bc740 1 -- 172.21.15.33:0/1445070296 >> 172.21.15.33:0/1445070296 conn(0x560fb42e5ed0 msgr2=0x560fb42e7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1d253bc740 1 -- 172.21.15.33:0/1445070296 shutdown_connections 2024-04-02T13:42:40.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1d253bc740 1 -- 172.21.15.33:0/1445070296 wait complete. 2024-04-02T13:42:40.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.898+0000 7f1d253bc740 20 client.16467 trim_cache size 0 max 16384 2024-04-02T13:42:40.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.911+0000 7f1d253bc740 1 Processor -- start 2024-04-02T13:42:40.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.911+0000 7f1d253bc740 1 -- start start 2024-04-02T13:42:40.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d253bc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42eb7a0 0x560fb4300500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d253bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d253bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42f4ca0 0x560fb42f4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d253bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1b1870 con 0x560fb42f4ca0 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d253bc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1af080 con 0x560fb42eb7a0 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d253bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1aceb0 con 0x560fb42ec0c0 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d20d6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42f4ca0 0x560fb42f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d20d6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42f4ca0 0x560fb42f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39444/0 (socket says 172.21.15.33:39444) 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d20d6a640 1 -- 172.21.15.33:0/2822469508 learned_addr learned my addr 172.21.15.33:0/2822469508 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d1b7fe640 1 --2- 172.21.15.33:0/2822469508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d1bfff640 1 --2- 172.21.15.33:0/2822469508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42eb7a0 0x560fb4300500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d20d6a640 1 -- 172.21.15.33:0/2822469508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 msgr2=0x560fb42f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d20d6a640 1 --2- 172.21.15.33:0/2822469508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d20d6a640 1 -- 172.21.15.33:0/2822469508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42eb7a0 msgr2=0x560fb4300500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d20d6a640 1 --2- 172.21.15.33:0/2822469508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42eb7a0 0x560fb4300500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d20d6a640 1 -- 172.21.15.33:0/2822469508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb42f22e0 con 0x560fb42f4ca0 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d1b7fe640 1 --2- 172.21.15.33:0/2822469508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d20d6a640 1 --2- 172.21.15.33:0/2822469508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42f4ca0 0x560fb42f4640 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1d14014320 tx=0x7f1d1401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9305caa5f661e856 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d01ffb640 1 -- 172.21.15.33:0/2822469508 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14013600 con 0x560fb42f4ca0 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d01ffb640 1 -- 172.21.15.33:0/2822469508 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1d14013c70 con 0x560fb42f4ca0 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.912+0000 7f1d01ffb640 1 -- 172.21.15.33:0/2822469508 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14021d90 con 0x560fb42f4ca0 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.913+0000 7f1d253bc740 1 -- 172.21.15.33:0/2822469508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42f4ca0 msgr2=0x560fb42f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.913+0000 7f1d253bc740 1 --2- 172.21.15.33:0/2822469508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42f4ca0 0x560fb42f4640 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1d14014320 tx=0x7f1d1401d000 comp rx=0 tx=0).stop 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.913+0000 7f1d253bc740 1 -- 172.21.15.33:0/2822469508 shutdown_connections 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.913+0000 7f1d253bc740 1 --2- 172.21.15.33:0/2822469508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42f4ca0 0x560fb42f4640 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.913+0000 7f1d253bc740 1 --2- 172.21.15.33:0/2822469508 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.913+0000 7f1d253bc740 1 --2- 172.21.15.33:0/2822469508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42eb7a0 0x560fb4300500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.913+0000 7f1d253bc740 1 -- 172.21.15.33:0/2822469508 >> 172.21.15.33:0/2822469508 conn(0x560fb42e5ed0 msgr2=0x560fb42f02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.913+0000 7f1d253bc740 1 -- 172.21.15.33:0/2822469508 shutdown_connections 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.913+0000 7f1d253bc740 1 -- 172.21.15.33:0/2822469508 wait complete. 2024-04-02T13:42:40.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.913+0000 7f1d253bc740 1 -- start start 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.913+0000 7f1d253bc740 1 Processor -- start 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d253bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42fcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d253bc740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42ee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d253bc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 0x560fb42ee880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d253bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1ad540 con 0x560fb42eb7a0 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d253bc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1ae9b0 con 0x560fb42f4ca0 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d253bc740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1adc10 con 0x560fb42ec0c0 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d253bc740 10 client.0 _ll_register_callbacks cb 0x7f1d0c0ba610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42fcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42fcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39454/0 (socket says 172.21.15.33:39454) 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d1bfff640 1 -- 172.21.15.33:0/3578434409 learned_addr learned my addr 172.21.15.33:0/3578434409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d1b7fe640 1 --2- 172.21.15.33:0/3578434409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d20d6a640 1 --2- 172.21.15.33:0/3578434409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 0x560fb42ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d1bfff640 1 -- 172.21.15.33:0/3578434409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 msgr2=0x560fb42ee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d1bfff640 1 --2- 172.21.15.33:0/3578434409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42ee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d1bfff640 1 -- 172.21.15.33:0/3578434409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 msgr2=0x560fb42ee880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d1bfff640 1 --2- 172.21.15.33:0/3578434409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 0x560fb42ee880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d1bfff640 1 -- 172.21.15.33:0/3578434409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb4307120 con 0x560fb42eb7a0 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d1bfff640 1 --2- 172.21.15.33:0/3578434409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42fcd50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000c00 tx=0x7f1d0c0ba6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8751096d9572d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1cfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1cfb7fe640 1 -- 172.21.15.33:0/3578434409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c0d6070 con 0x560fb42eb7a0 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1cfb7fe640 1 -- 172.21.15.33:0/3578434409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1d0c0c6b50 con 0x560fb42eb7a0 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.914+0000 7f1d253bc740 1 -- 172.21.15.33:0/3578434409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d14013c70 con 0x560fb42eb7a0 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1cfb7fe640 1 -- 172.21.15.33:0/3578434409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d0c004c70 con 0x560fb42eb7a0 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1d253bc740 10 client.16473 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1d253bc740 1 -- 172.21.15.33:0/3578434409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560fb4307120 con 0x560fb42eb7a0 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1cfb7fe640 1 -- 172.21.15.33:0/3578434409 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1d14013c70 con 0x560fb42eb7a0 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1cfb7fe640 1 -- 172.21.15.33:0/3578434409 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1d0c0bd7c0 con 0x560fb42eb7a0 2024-04-02T13:42:40.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1d253bc740 10 client.16473 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1d253bc740 10 client.16473 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1d253bc740 20 client.16473 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1d253bc740 1 --2- 172.21.15.33:0/3578434409 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x560fb436b070 0x560fb436d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1d253bc740 4 client.16473 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1d253bc740 1 -- 172.21.15.33:0/3578434409 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fb4257040 con 0x560fb436b070 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1d1b7fe640 1 --2- 172.21.15.33:0/3578434409 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x560fb436b070 0x560fb436d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1cfb7fe640 10 client.16473 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.915+0000 7f1d1b7fe640 1 --2- 172.21.15.33:0/3578434409 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x560fb436b070 0x560fb436d490 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69ab0f6c9458af7f server_cookie=22761b9be6453418 in_seq=0 out_seq=0 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1cfb7fe640 1 -- 172.21.15.33:0/3578434409 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fb4257040 con 0x560fb436b070 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1cfb7fe640 10 client.16473 handle_command_reply: tid=0 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d253bc740 1 client.16473 shutdown 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d253bc740 1 -- 172.21.15.33:0/3578434409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 msgr2=0x560fb42fcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d253bc740 1 --2- 172.21.15.33:0/3578434409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42fcd50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000c00 tx=0x7f1d0c0ba6c0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d253bc740 1 -- 172.21.15.33:0/3578434409 shutdown_connections 2024-04-02T13:42:40.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d253bc740 1 --2- 172.21.15.33:0/3578434409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560fb42f4ca0 0x560fb42ee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d253bc740 1 --2- 172.21.15.33:0/3578434409 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x560fb436b070 0x560fb436d490 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d253bc740 1 --2- 172.21.15.33:0/3578434409 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560fb42ec0c0 0x560fb42ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d253bc740 1 --2- 172.21.15.33:0/3578434409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fb42eb7a0 0x560fb42fcd50 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1cfb7fe640 0 client.16473 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:42:40.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d253bc740 1 -- 172.21.15.33:0/3578434409 >> 172.21.15.33:0/3578434409 conn(0x560fb42e5ed0 msgr2=0x560fb42e46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d253bc740 1 -- 172.21.15.33:0/3578434409 shutdown_connections 2024-04-02T13:42:40.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d253bc740 1 -- 172.21.15.33:0/3578434409 wait complete. 2024-04-02T13:42:40.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d253bc740 20 client.16473 trim_cache size 0 max 16384 2024-04-02T13:42:40.927 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:42:40.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d227cd640 1 -- 172.21.15.33:0/4125281780 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1d040a3d20 msgr2=0x7f1d040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.916+0000 7f1d227cd640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1d040a3d20 0x7f1d040a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1d14005fd0 tx=0x7f1d14014af0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.917+0000 7f1d227cd640 1 -- 172.21.15.33:0/4125281780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d1c1003c0 msgr2=0x7f1d1c199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:40.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.917+0000 7f1d227cd640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d1c1003c0 0x7f1d1c199170 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009230 tx=0x7f1d0c02cb30 comp rx=0 tx=0).stop 2024-04-02T13:42:40.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.917+0000 7f1d227cd640 1 -- 172.21.15.33:0/4125281780 shutdown_connections 2024-04-02T13:42:40.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.917+0000 7f1d227cd640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1d040a3d20 0x7f1d040a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.917+0000 7f1d227cd640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d1c105130 0x7f1d1c1a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.917+0000 7f1d227cd640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1d1c100d60 0x7f1d1c1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.917+0000 7f1d227cd640 1 --2- 172.21.15.33:0/4125281780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d1c1003c0 0x7f1d1c199170 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:40.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.917+0000 7f1d227cd640 1 -- 172.21.15.33:0/4125281780 >> 172.21.15.33:0/4125281780 conn(0x7f1d1c075d40 msgr2=0x7f1d1c0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:40.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.917+0000 7f1d227cd640 1 -- 172.21.15.33:0/4125281780 shutdown_connections 2024-04-02T13:42:40.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:40.917+0000 7f1d227cd640 1 -- 172.21.15.33:0/4125281780 wait complete. 2024-04-02T13:42:40.931 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:42:41.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.033+0000 7fcdd630e640 1 -- 172.21.15.33:0/1096281656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd0101370 msgr2=0x7fcdd010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:41.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.033+0000 7fcdd630e640 1 --2- 172.21.15.33:0/1096281656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd0101370 0x7fcdd010d880 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009cd0 tx=0x7fcdc802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.033+0000 7fcdd630e640 1 -- 172.21.15.33:0/1096281656 shutdown_connections 2024-04-02T13:42:41.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.033+0000 7fcdd630e640 1 --2- 172.21.15.33:0/1096281656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd0101370 0x7fcdd010d880 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.033+0000 7fcdd630e640 1 --2- 172.21.15.33:0/1096281656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdd01009b0 0x7fcdd0100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.033+0000 7fcdd630e640 1 --2- 172.21.15.33:0/1096281656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdd01069c0 0x7fcdd0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.033+0000 7fcdd630e640 1 -- 172.21.15.33:0/1096281656 >> 172.21.15.33:0/1096281656 conn(0x7fcdd0076d80 msgr2=0x7fcdd0077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:41.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.033+0000 7fcdd630e640 1 -- 172.21.15.33:0/1096281656 shutdown_connections 2024-04-02T13:42:41.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.033+0000 7fcdd630e640 1 -- 172.21.15.33:0/1096281656 wait complete. 2024-04-02T13:42:41.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.034+0000 7fcdd630e640 1 Processor -- start 2024-04-02T13:42:41.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.034+0000 7fcdd630e640 1 -- start start 2024-04-02T13:42:41.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.034+0000 7fcdd630e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdd01009b0 0x7fcdd0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.034+0000 7fcdd630e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd0101370 0x7fcdd01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.034+0000 7fcdd630e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdd01069c0 0x7fcdd019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.034+0000 7fcdd630e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0111ea0 con 0x7fcdd0101370 2024-04-02T13:42:41.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.034+0000 7fcdd630e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0111d20 con 0x7fcdd01069c0 2024-04-02T13:42:41.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.034+0000 7fcdd630e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0112020 con 0x7fcdd01009b0 2024-04-02T13:42:41.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.034+0000 7fcdcf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd0101370 0x7fcdd01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.034+0000 7fcdcf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd0101370 0x7fcdd01988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39480/0 (socket says 172.21.15.33:39480) 2024-04-02T13:42:41.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.034+0000 7fcdcf7fe640 1 -- 172.21.15.33:0/1356283283 learned_addr learned my addr 172.21.15.33:0/1356283283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:41.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.034+0000 7fcdd48ab640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdd01069c0 0x7fcdd019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.034+0000 7fcdcffff640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdd01009b0 0x7fcdd0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.035+0000 7fcdcf7fe640 1 -- 172.21.15.33:0/1356283283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdd01009b0 msgr2=0x7fcdd0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:41.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.035+0000 7fcdcf7fe640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdd01009b0 0x7fcdd0198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.035+0000 7fcdcf7fe640 1 -- 172.21.15.33:0/1356283283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdd01069c0 msgr2=0x7fcdd019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:41.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.035+0000 7fcdcf7fe640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdd01069c0 0x7fcdd019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.035+0000 7fcdcf7fe640 1 -- 172.21.15.33:0/1356283283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd019d370 con 0x7fcdd0101370 2024-04-02T13:42:41.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.035+0000 7fcdd48ab640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdd01069c0 0x7fcdd019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:41.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.035+0000 7fcdcf7fe640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd0101370 0x7fcdd01988c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400d770 tx=0x7fcdc400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:41.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.035+0000 7fcdcd7fa640 1 -- 172.21.15.33:0/1356283283 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4017070 con 0x7fcdd0101370 2024-04-02T13:42:41.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.035+0000 7fcdd630e640 1 -- 172.21.15.33:0/1356283283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd019d660 con 0x7fcdd0101370 2024-04-02T13:42:41.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.035+0000 7fcdcd7fa640 1 -- 172.21.15.33:0/1356283283 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcdc40054f0 con 0x7fcdd0101370 2024-04-02T13:42:41.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.035+0000 7fcdcd7fa640 1 -- 172.21.15.33:0/1356283283 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4004d60 con 0x7fcdd0101370 2024-04-02T13:42:41.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.035+0000 7fcdd630e640 1 -- 172.21.15.33:0/1356283283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd01a4ef0 con 0x7fcdd0101370 2024-04-02T13:42:41.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.037+0000 7fcdcd7fa640 1 -- 172.21.15.33:0/1356283283 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcdc4004f00 con 0x7fcdd0101370 2024-04-02T13:42:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.037+0000 7fcdd8fd3740 1 Processor -- start 2024-04-02T13:42:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdd8fd3740 1 -- start start 2024-04-02T13:42:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdd8fd3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467c7a0 0x56276467cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdd8fd3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467d0c0 0x562764685760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdd8fd3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764688090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdd8fd3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5627645e07b0 con 0x562764685ca0 2024-04-02T13:42:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdd8fd3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56276457f570 con 0x56276467c7a0 2024-04-02T13:42:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdd8fd3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5627645dc760 con 0x56276467d0c0 2024-04-02T13:42:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdcd7fa640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcdb80a3d20 0x7fcdb80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdcf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764688090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdcf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764688090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39484/0 (socket says 172.21.15.33:39484) 2024-04-02T13:42:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdd48ab640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcdb80a3d20 0x7fcdb80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdcf7fe640 1 -- 172.21.15.33:0/3043331157 learned_addr learned my addr 172.21.15.33:0/3043331157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdcffff640 1 --2- 172.21.15.33:0/3043331157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467d0c0 0x562764685760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdcd7fa640 1 -- 172.21.15.33:0/1356283283 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcdc40d06d0 con 0x7fcdd0101370 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdd48ab640 1 --2- 172.21.15.33:0/3043331157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467c7a0 0x56276467cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdcf7fe640 1 -- 172.21.15.33:0/3043331157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467d0c0 msgr2=0x562764685760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdcf7fe640 1 --2- 172.21.15.33:0/3043331157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467d0c0 0x562764685760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdcf7fe640 1 -- 172.21.15.33:0/3043331157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467c7a0 msgr2=0x56276467cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdcf7fe640 1 --2- 172.21.15.33:0/3043331157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467c7a0 0x56276467cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdcf7fe640 1 -- 172.21.15.33:0/3043331157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627642f43f0 con 0x562764685ca0 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.038+0000 7fcdcf7fe640 1 --2- 172.21.15.33:0/3043331157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764688090 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009f60 tx=0x7fcdc406edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799cae12d45fcf7b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.039+0000 7fcdd48ab640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcdb80a3d20 0x7fcdb80a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8006010 tx=0x7fcdc8032e50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.039+0000 7fcdb67fc640 1 -- 172.21.15.33:0/3043331157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4020070 con 0x562764685ca0 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.039+0000 7fcdb67fc640 1 -- 172.21.15.33:0/3043331157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcdc40056a0 con 0x562764685ca0 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.039+0000 7fcdb67fc640 1 -- 172.21.15.33:0/3043331157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc406ee60 con 0x562764685ca0 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.039+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/3043331157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 msgr2=0x562764688090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:41.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.039+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/3043331157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764688090 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009f60 tx=0x7fcdc406edf0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.039+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/3043331157 shutdown_connections 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.039+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/3043331157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764688090 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.039+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/3043331157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467d0c0 0x562764685760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.039+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/3043331157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467c7a0 0x56276467cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.039+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/3043331157 >> 172.21.15.33:0/3043331157 conn(0x562764676ed0 msgr2=0x5627646772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.039+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/3043331157 shutdown_connections 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.039+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/3043331157 wait complete. 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd8fd3740 1 -- start start 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd8fd3740 1 Processor -- start 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd8fd3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467c7a0 0x562764685750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd8fd3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467d0c0 0x562764682ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd8fd3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764683220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd8fd3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562764688a90 con 0x562764685ca0 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd8fd3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56276459be80 con 0x56276467c7a0 2024-04-02T13:42:41.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd8fd3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562764688c10 con 0x56276467d0c0 2024-04-02T13:42:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd8fd3740 10 client.0 _ll_register_callbacks cb 0x5627645d7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd48ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764683220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd48ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764683220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39498/0 (socket says 172.21.15.33:39498) 2024-04-02T13:42:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd48ab640 1 -- 172.21.15.33:0/304794501 learned_addr learned my addr 172.21.15.33:0/304794501 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdcffff640 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467c7a0 0x562764685750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdcf7fe640 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467d0c0 0x562764682ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd48ab640 1 -- 172.21.15.33:0/304794501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467d0c0 msgr2=0x562764682ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd48ab640 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467d0c0 0x562764682ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd48ab640 1 -- 172.21.15.33:0/304794501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467c7a0 msgr2=0x562764685750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.040+0000 7fcdd48ab640 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467c7a0 0x562764685750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdd48ab640 1 -- 172.21.15.33:0/304794501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627643be7d0 con 0x562764685ca0 2024-04-02T13:42:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdcffff640 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467c7a0 0x562764685750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdd48ab640 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764683220 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8032570 tx=0x7fcdc8045070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b419e7835113f02 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdaf7fe640 1 -- 172.21.15.33:0/304794501 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8053060 con 0x562764685ca0 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdaf7fe640 1 -- 172.21.15.33:0/304794501 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcdc8004790 con 0x562764685ca0 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/304794501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc40056a0 con 0x562764685ca0 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdaf7fe640 1 -- 172.21.15.33:0/304794501 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcdc805a050 con 0x562764685ca0 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdd8fd3740 10 client.16485 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/304794501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562764694b70 con 0x562764685ca0 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdaf7fe640 1 -- 172.21.15.33:0/304794501 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcdc40056a0 con 0x562764685ca0 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdaf7fe640 1 -- 172.21.15.33:0/304794501 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcdc803f030 con 0x562764685ca0 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdd8fd3740 10 client.16485 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdd8fd3740 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdd8fd3740 20 client.16485 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562764696030 0x5627646984d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdd8fd3740 4 client.16485 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:41.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.041+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/304794501 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627645edcf0 con 0x562764696030 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.042+0000 7fcdcffff640 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562764696030 0x5627646984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.042+0000 7fcdcffff640 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562764696030 0x5627646984d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81bb254ac43037a6 server_cookie=d1854288c194be02 in_seq=0 out_seq=0 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.042+0000 7fcdaf7fe640 10 client.16485 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdaf7fe640 1 -- 172.21.15.33:0/304794501 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5627645edcf0 con 0x562764696030 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdaf7fe640 10 client.16485 handle_command_reply: tid=0 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdd8fd3740 1 client.16485 shutdown 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/304794501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 msgr2=0x562764683220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764683220 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8032570 tx=0x7fcdc8045070 comp rx=0 tx=0).stop 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/304794501 shutdown_connections 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562764696030 0x5627646984d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764683220 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467d0c0 0x562764682ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdaf7fe640 0 client.16485 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/304794501 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467c7a0 0x562764685750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/304794501 >> 172.21.15.33:0/304794501 conn(0x562764676ed0 msgr2=0x562764678c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:41.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/304794501 shutdown_connections 2024-04-02T13:42:41.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/304794501 wait complete. 2024-04-02T13:42:41.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.043+0000 7fcdd8fd3740 20 client.16485 trim_cache size 0 max 16384 2024-04-02T13:42:41.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.056+0000 7fcdd8fd3740 1 Processor -- start 2024-04-02T13:42:41.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.056+0000 7fcdd8fd3740 1 -- start start 2024-04-02T13:42:41.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd8fd3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467c7a0 0x562764691980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd8fd3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467d0c0 0x56276467d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd8fd3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764681c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd8fd3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd01b18f0 con 0x562764685ca0 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd8fd3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd01af100 con 0x56276467d0c0 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd8fd3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd01acf30 con 0x56276467c7a0 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd48ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764681c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd48ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764681c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39510/0 (socket says 172.21.15.33:39510) 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd48ab640 1 -- 172.21.15.33:0/1033552776 learned_addr learned my addr 172.21.15.33:0/1033552776 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdcf7fe640 1 --2- 172.21.15.33:0/1033552776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467d0c0 0x56276467d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd48ab640 1 -- 172.21.15.33:0/1033552776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467c7a0 msgr2=0x562764691980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd48ab640 1 --2- 172.21.15.33:0/1033552776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467c7a0 0x562764691980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd48ab640 1 -- 172.21.15.33:0/1033552776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467d0c0 msgr2=0x56276467d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd48ab640 1 --2- 172.21.15.33:0/1033552776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467d0c0 0x56276467d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd48ab640 1 -- 172.21.15.33:0/1033552776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56276467bb20 con 0x562764685ca0 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdd48ab640 1 --2- 172.21.15.33:0/1033552776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764681c40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80413e0 tx=0x7fcdc80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321782818294b49d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:41.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdb5ffb640 1 -- 172.21.15.33:0/1033552776 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8053070 con 0x562764685ca0 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdb5ffb640 1 -- 172.21.15.33:0/1033552776 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcdc8003e70 con 0x562764685ca0 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.057+0000 7fcdb5ffb640 1 -- 172.21.15.33:0/1033552776 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8039040 con 0x562764685ca0 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.058+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/1033552776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 msgr2=0x562764681c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.058+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/1033552776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764681c40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80413e0 tx=0x7fcdc80095c0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.058+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/1033552776 shutdown_connections 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.058+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/1033552776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562764685ca0 0x562764681c40 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.058+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/1033552776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56276467d0c0 0x56276467d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.058+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/1033552776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467c7a0 0x562764691980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.058+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/1033552776 >> 172.21.15.33:0/1033552776 conn(0x562764676ed0 msgr2=0x562764682370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.058+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/1033552776 shutdown_connections 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.058+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/1033552776 wait complete. 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.058+0000 7fcdd8fd3740 1 -- start start 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.058+0000 7fcdd8fd3740 1 Processor -- start 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdd8fd3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467c7a0 0x56276468de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdd8fd3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56276467d0c0 0x56276468e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdd8fd3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562764685ca0 0x56276467f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdd8fd3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd01ad5c0 con 0x56276467d0c0 2024-04-02T13:42:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdd8fd3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd01aea30 con 0x562764685ca0 2024-04-02T13:42:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdd8fd3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd01adc90 con 0x56276467c7a0 2024-04-02T13:42:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdd8fd3740 10 client.0 _ll_register_callbacks cb 0x7fcdc8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdcf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56276467d0c0 0x56276468e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdcf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56276467d0c0 0x56276468e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39518/0 (socket says 172.21.15.33:39518) 2024-04-02T13:42:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdcf7fe640 1 -- 172.21.15.33:0/1105101763 learned_addr learned my addr 172.21.15.33:0/1105101763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdcffff640 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467c7a0 0x56276468de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdd48ab640 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562764685ca0 0x56276467f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdcf7fe640 1 -- 172.21.15.33:0/1105101763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467c7a0 msgr2=0x56276468de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdcf7fe640 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467c7a0 0x56276468de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdcf7fe640 1 -- 172.21.15.33:0/1105101763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562764685ca0 msgr2=0x56276467f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdcf7fe640 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562764685ca0 0x56276467f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdcf7fe640 1 -- 172.21.15.33:0/1105101763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56276467faf0 con 0x56276467d0c0 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdd48ab640 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562764685ca0 0x56276467f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdcf7fe640 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56276467d0c0 0x56276468e3d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4098760 tx=0x7fcdc4005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fc536a71a31494 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdaf7fe640 1 -- 172.21.15.33:0/1105101763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc40b5060 con 0x56276467d0c0 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdaf7fe640 1 -- 172.21.15.33:0/1105101763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcdc40cdd30 con 0x56276467d0c0 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.059+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/1105101763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc8003e70 con 0x56276467d0c0 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.060+0000 7fcdaf7fe640 1 -- 172.21.15.33:0/1105101763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcdc4020040 con 0x56276467d0c0 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.060+0000 7fcdd8fd3740 10 client.16491 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.060+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/1105101763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627646fc210 con 0x56276467d0c0 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.060+0000 7fcdaf7fe640 1 -- 172.21.15.33:0/1105101763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fcdc8003e70 con 0x56276467d0c0 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.060+0000 7fcdaf7fe640 1 -- 172.21.15.33:0/1105101763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcdc408bce0 con 0x56276467d0c0 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.060+0000 7fcdd8fd3740 10 client.16491 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.060+0000 7fcdd8fd3740 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.060+0000 7fcdd8fd3740 20 client.16491 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.060+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5627646fc560 0x5627646fe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.060+0000 7fcdd8fd3740 4 client.16491 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.060+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/1105101763 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627645e8040 con 0x5627646fc560 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.060+0000 7fcdcffff640 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5627646fc560 0x5627646fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdcffff640 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5627646fc560 0x5627646fe950 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c35739ac19c7cdd server_cookie=547f6bbaa0f6f0ae in_seq=0 out_seq=0 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdaf7fe640 10 client.16491 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdaf7fe640 1 -- 172.21.15.33:0/1105101763 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627645e8040 con 0x5627646fc560 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdaf7fe640 10 client.16491 handle_command_reply: tid=0 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdd8fd3740 1 client.16491 shutdown 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/1105101763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56276467d0c0 msgr2=0x56276468e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56276467d0c0 0x56276468e3d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4098760 tx=0x7fcdc4005420 comp rx=0 tx=0).stop 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/1105101763 shutdown_connections 2024-04-02T13:42:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5627646fc560 0x5627646fe950 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562764685ca0 0x56276467f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56276467d0c0 0x56276468e3d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdd8fd3740 1 --2- 172.21.15.33:0/1105101763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56276467c7a0 0x56276468de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/1105101763 >> 172.21.15.33:0/1105101763 conn(0x562764676ed0 msgr2=0x562764675f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdaf7fe640 0 client.16491 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/1105101763 shutdown_connections 2024-04-02T13:42:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdd8fd3740 1 -- 172.21.15.33:0/1105101763 wait complete. 2024-04-02T13:42:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.061+0000 7fcdd8fd3740 20 client.16491 trim_cache size 0 max 16384 2024-04-02T13:42:41.071 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:42:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.062+0000 7fcdd630e640 1 -- 172.21.15.33:0/1356283283 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcdb80a3d20 msgr2=0x7fcdb80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.062+0000 7fcdd630e640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcdb80a3d20 0x7fcdb80a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8006010 tx=0x7fcdc8032e50 comp rx=0 tx=0).stop 2024-04-02T13:42:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.062+0000 7fcdd630e640 1 -- 172.21.15.33:0/1356283283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd0101370 msgr2=0x7fcdd01988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.062+0000 7fcdd630e640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd0101370 0x7fcdd01988c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400d770 tx=0x7fcdc400dc30 comp rx=0 tx=0).stop 2024-04-02T13:42:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.062+0000 7fcdd630e640 1 -- 172.21.15.33:0/1356283283 shutdown_connections 2024-04-02T13:42:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.062+0000 7fcdd630e640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fcdb80a3d20 0x7fcdb80a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.062+0000 7fcdd630e640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdd01069c0 0x7fcdd019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.062+0000 7fcdd630e640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd0101370 0x7fcdd01988c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.062+0000 7fcdd630e640 1 --2- 172.21.15.33:0/1356283283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdd01009b0 0x7fcdd0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:41.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.062+0000 7fcdd630e640 1 -- 172.21.15.33:0/1356283283 >> 172.21.15.33:0/1356283283 conn(0x7fcdd0076d80 msgr2=0x7fcdd00fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:41.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.062+0000 7fcdd630e640 1 -- 172.21.15.33:0/1356283283 shutdown_connections 2024-04-02T13:42:41.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:41.062+0000 7fcdd630e640 1 -- 172.21.15.33:0/1356283283 wait complete. 2024-04-02T13:42:41.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:41 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3413283768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:42:41.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:41 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3413283768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:42:41.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:41 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3413283768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:42:42.077 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:42:42.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:42:42.078 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:42:42.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.195+0000 7f427ac81640 1 -- 172.21.15.33:0/3427693499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274069600 msgr2=0x7f427410df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.195+0000 7f427ac81640 1 --2- 172.21.15.33:0/3427693499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274069600 0x7f427410df60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4268009900 tx=0x7f426802bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.195+0000 7f427ac81640 1 -- 172.21.15.33:0/3427693499 shutdown_connections 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.195+0000 7f427ac81640 1 --2- 172.21.15.33:0/3427693499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f427410e4a0 0x7f4274110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.195+0000 7f427ac81640 1 --2- 172.21.15.33:0/3427693499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274069600 0x7f427410df60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.195+0000 7f427ac81640 1 --2- 172.21.15.33:0/3427693499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4274068ce0 0x7f42740690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.195+0000 7f427ac81640 1 -- 172.21.15.33:0/3427693499 >> 172.21.15.33:0/3427693499 conn(0x7f4274077b00 msgr2=0x7f4274077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.195+0000 7f427ac81640 1 -- 172.21.15.33:0/3427693499 shutdown_connections 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.195+0000 7f427ac81640 1 -- 172.21.15.33:0/3427693499 wait complete. 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.195+0000 7f427ac81640 1 Processor -- start 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.195+0000 7f427ac81640 1 -- start start 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.195+0000 7f427ac81640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4274068ce0 0x7f427419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f427ac81640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274069600 0x7f427419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f427ac81640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f427410e4a0 0x7f427419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f427ac81640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4274114030 con 0x7f4274069600 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f427ac81640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4274113eb0 con 0x7f427410e4a0 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f427ac81640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f42741141b0 con 0x7f4274068ce0 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f4273fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274069600 0x7f427419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f4273fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274069600 0x7f427419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39536/0 (socket says 172.21.15.33:39536) 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f4273fff640 1 -- 172.21.15.33:0/3453219528 learned_addr learned my addr 172.21.15.33:0/3453219528 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f427921e640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f427410e4a0 0x7f427419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f4278a1d640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4274068ce0 0x7f427419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f4273fff640 1 -- 172.21.15.33:0/3453219528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4274068ce0 msgr2=0x7f427419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f4273fff640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4274068ce0 0x7f427419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f4273fff640 1 -- 172.21.15.33:0/3453219528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f427410e4a0 msgr2=0x7f427419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f4273fff640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f427410e4a0 0x7f427419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f4273fff640 1 -- 172.21.15.33:0/3453219528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427419f520 con 0x7f4274069600 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f427921e640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f427410e4a0 0x7f427419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f4273fff640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274069600 0x7f427419aa70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4268005940 tx=0x7f42680045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:42.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f4271ffb640 1 -- 172.21.15.33:0/3453219528 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268043020 con 0x7f4274069600 2024-04-02T13:42:42.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f4271ffb640 1 -- 172.21.15.33:0/3453219528 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f426802caa0 con 0x7f4274069600 2024-04-02T13:42:42.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f4271ffb640 1 -- 172.21.15.33:0/3453219528 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268003b50 con 0x7f4274069600 2024-04-02T13:42:42.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.196+0000 7f427ac81640 1 -- 172.21.15.33:0/3453219528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427419f7b0 con 0x7f4274069600 2024-04-02T13:42:42.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.197+0000 7f427ac81640 1 -- 172.21.15.33:0/3453219528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42741a70a0 con 0x7f4274069600 2024-04-02T13:42:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.199+0000 7f4271ffb640 1 -- 172.21.15.33:0/3453219528 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4268046020 con 0x7f4274069600 2024-04-02T13:42:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.199+0000 7f427d862740 1 Processor -- start 2024-04-02T13:42:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.199+0000 7f427d862740 1 -- start start 2024-04-02T13:42:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.199+0000 7f427d862740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05a2c0 0x55bcde05a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.199+0000 7f427d862740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde05abe0 0x55bcde063280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.199+0000 7f427d862740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde0637c0 0x55bcde065bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.199+0000 7f427d862740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bcddfbe770 con 0x55bcde0637c0 2024-04-02T13:42:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.199+0000 7f427d862740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bcddf66960 con 0x55bcde05a2c0 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.199+0000 7f427d862740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bcddfba8c0 con 0x55bcde05abe0 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f4271ffb640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f425c0a3d20 0x7f425c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f4273fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde0637c0 0x55bcde065bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f4273fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde0637c0 0x55bcde065bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39544/0 (socket says 172.21.15.33:39544) 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f4273fff640 1 -- 172.21.15.33:0/915881357 learned_addr learned my addr 172.21.15.33:0/915881357 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f427921e640 1 --2- 172.21.15.33:0/915881357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05a2c0 0x55bcde05a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f427921e640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f425c0a3d20 0x7f425c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f4278a1d640 1 --2- 172.21.15.33:0/915881357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde05abe0 0x55bcde063280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f4271ffb640 1 -- 172.21.15.33:0/3453219528 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f42680f2700 con 0x7f4274069600 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f4273fff640 1 -- 172.21.15.33:0/915881357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde05abe0 msgr2=0x55bcde063280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f4273fff640 1 --2- 172.21.15.33:0/915881357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde05abe0 0x55bcde063280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f4273fff640 1 -- 172.21.15.33:0/915881357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05a2c0 msgr2=0x55bcde05a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f4273fff640 1 --2- 172.21.15.33:0/915881357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05a2c0 0x55bcde05a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f4273fff640 1 -- 172.21.15.33:0/915881357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcddd9a7e0 con 0x55bcde0637c0 2024-04-02T13:42:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f427921e640 1 --2- 172.21.15.33:0/915881357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05a2c0 0x55bcde05a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f4273fff640 1 --2- 172.21.15.33:0/915881357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde0637c0 0x55bcde065bb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f42680c6b40 tx=0x7f426804d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fd980f5820f8cf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f427921e640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f425c0a3d20 0x7f425c0a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4264005fd0 tx=0x7f42640058e0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.200+0000 7f425affd640 1 -- 172.21.15.33:0/915881357 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42680bdd60 con 0x55bcde0637c0 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.201+0000 7f425affd640 1 -- 172.21.15.33:0/915881357 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f426802cc50 con 0x55bcde0637c0 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.201+0000 7f425affd640 1 -- 172.21.15.33:0/915881357 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42680055e0 con 0x55bcde0637c0 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.201+0000 7f427d862740 1 -- 172.21.15.33:0/915881357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde0637c0 msgr2=0x55bcde065bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.201+0000 7f427d862740 1 --2- 172.21.15.33:0/915881357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde0637c0 0x55bcde065bb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f42680c6b40 tx=0x7f426804d000 comp rx=0 tx=0).stop 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.201+0000 7f427d862740 1 -- 172.21.15.33:0/915881357 shutdown_connections 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.201+0000 7f427d862740 1 --2- 172.21.15.33:0/915881357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde0637c0 0x55bcde065bb0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.201+0000 7f427d862740 1 --2- 172.21.15.33:0/915881357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde05abe0 0x55bcde063280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.201+0000 7f427d862740 1 --2- 172.21.15.33:0/915881357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05a2c0 0x55bcde05a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.201+0000 7f427d862740 1 -- 172.21.15.33:0/915881357 >> 172.21.15.33:0/915881357 conn(0x55bcde0549f0 msgr2=0x55bcde054dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.201+0000 7f427d862740 1 -- 172.21.15.33:0/915881357 shutdown_connections 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.201+0000 7f427d862740 1 -- 172.21.15.33:0/915881357 wait complete. 2024-04-02T13:42:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f427d862740 1 -- start start 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f427d862740 1 Processor -- start 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f427d862740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05a2c0 0x55bcde060780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f427d862740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05abe0 0x55bcde060cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f427d862740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde0637c0 0x55bcde061200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f427d862740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bcddfbe770 con 0x55bcde05abe0 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f427d862740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bcddf79d20 con 0x55bcde05a2c0 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f427d862740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55bcde066370 con 0x55bcde0637c0 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f427d862740 10 client.0 _ll_register_callbacks cb 0x55bcddf46a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f4273fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05abe0 0x55bcde060cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f4273fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05abe0 0x55bcde060cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39560/0 (socket says 172.21.15.33:39560) 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f4273fff640 1 -- 172.21.15.33:0/54846875 learned_addr learned my addr 172.21.15.33:0/54846875 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f4278a1d640 1 --2- 172.21.15.33:0/54846875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05a2c0 0x55bcde060780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f4273fff640 1 -- 172.21.15.33:0/54846875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde0637c0 msgr2=0x55bcde061200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f4273fff640 1 --2- 172.21.15.33:0/54846875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde0637c0 0x55bcde061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f4273fff640 1 -- 172.21.15.33:0/54846875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05a2c0 msgr2=0x55bcde060780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f4273fff640 1 --2- 172.21.15.33:0/54846875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05a2c0 0x55bcde060780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f4273fff640 1 -- 172.21.15.33:0/54846875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcdddfaf70 con 0x55bcde05abe0 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.202+0000 7f4278a1d640 1 --2- 172.21.15.33:0/54846875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05a2c0 0x55bcde060780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f4273fff640 1 --2- 172.21.15.33:0/54846875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05abe0 0x55bcde060cc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f42680badd0 tx=0x7f42680baca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac91b45571689f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f4253fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f4253fff640 1 -- 172.21.15.33:0/54846875 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42680bdd60 con 0x55bcde05abe0 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f4253fff640 1 -- 172.21.15.33:0/54846875 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f42680055c0 con 0x55bcde05abe0 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f427d862740 1 -- 172.21.15.33:0/54846875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bcddfcbcb0 con 0x55bcde05abe0 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f4253fff640 1 -- 172.21.15.33:0/54846875 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42680eff30 con 0x55bcde05abe0 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f427d862740 10 client.16503 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f427d862740 1 -- 172.21.15.33:0/54846875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bcddc92170 con 0x55bcde05abe0 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f4253fff640 1 -- 172.21.15.33:0/54846875 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55bcddfcbcb0 con 0x55bcde05abe0 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f4253fff640 1 -- 172.21.15.33:0/54846875 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f42680c6be0 con 0x55bcde05abe0 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f427d862740 10 client.16503 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f427d862740 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:42.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f427d862740 20 client.16503 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f427d862740 1 --2- 172.21.15.33:0/54846875 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bcde0734a0 0x55bcde0758b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f427d862740 4 client.16503 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.203+0000 7f427d862740 1 -- 172.21.15.33:0/54846875 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcddfc6000 con 0x55bcde0734a0 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.204+0000 7f4278a1d640 1 --2- 172.21.15.33:0/54846875 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bcde0734a0 0x55bcde0758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.204+0000 7f4253fff640 10 client.16503 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.204+0000 7f4278a1d640 1 --2- 172.21.15.33:0/54846875 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bcde0734a0 0x55bcde0758b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b89d6defecea48c server_cookie=6ffd49d97c03607d in_seq=0 out_seq=0 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f4253fff640 1 -- 172.21.15.33:0/54846875 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bcddfc6000 con 0x55bcde0734a0 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f4253fff640 10 client.16503 handle_command_reply: tid=0 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f427d862740 1 client.16503 shutdown 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f427d862740 1 -- 172.21.15.33:0/54846875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05abe0 msgr2=0x55bcde060cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f427d862740 1 --2- 172.21.15.33:0/54846875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05abe0 0x55bcde060cc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f42680badd0 tx=0x7f42680baca0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f427d862740 1 -- 172.21.15.33:0/54846875 shutdown_connections 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f427d862740 1 --2- 172.21.15.33:0/54846875 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bcde0734a0 0x55bcde0758b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f427d862740 1 --2- 172.21.15.33:0/54846875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde0637c0 0x55bcde061200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f427d862740 1 --2- 172.21.15.33:0/54846875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05abe0 0x55bcde060cc0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f4253fff640 0 client.16503 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f427d862740 1 --2- 172.21.15.33:0/54846875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05a2c0 0x55bcde060780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f427d862740 1 -- 172.21.15.33:0/54846875 >> 172.21.15.33:0/54846875 conn(0x55bcde0549f0 msgr2=0x55bcde056740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f427d862740 1 -- 172.21.15.33:0/54846875 shutdown_connections 2024-04-02T13:42:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f427d862740 1 -- 172.21.15.33:0/54846875 wait complete. 2024-04-02T13:42:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.205+0000 7f427d862740 20 client.16503 trim_cache size 0 max 16384 2024-04-02T13:42:42.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.223+0000 7f427d862740 1 Processor -- start 2024-04-02T13:42:42.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.223+0000 7f427d862740 1 -- start start 2024-04-02T13:42:42.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.223+0000 7f427d862740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 0x55bcde062eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.223+0000 7f427d862740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05abe0 0x55bcde06f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.223+0000 7f427d862740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde0637c0 0x55bcde05f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.223+0000 7f427d862740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42741b3af0 con 0x55bcde05a2c0 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.223+0000 7f427d862740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f42741b1300 con 0x55bcde05abe0 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.223+0000 7f427d862740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f42741af130 con 0x55bcde0637c0 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.223+0000 7f4278a1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 0x55bcde062eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.224+0000 7f4278a1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 0x55bcde062eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39562/0 (socket says 172.21.15.33:39562) 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.224+0000 7f4278a1d640 1 -- 172.21.15.33:0/1836781023 learned_addr learned my addr 172.21.15.33:0/1836781023 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.224+0000 7f4273fff640 1 --2- 172.21.15.33:0/1836781023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05abe0 0x55bcde06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.224+0000 7f4278a1d640 1 -- 172.21.15.33:0/1836781023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde0637c0 msgr2=0x55bcde05f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.224+0000 7f4278a1d640 1 --2- 172.21.15.33:0/1836781023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde0637c0 0x55bcde05f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.224+0000 7f4278a1d640 1 -- 172.21.15.33:0/1836781023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05abe0 msgr2=0x55bcde06f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.224+0000 7f4278a1d640 1 --2- 172.21.15.33:0/1836781023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05abe0 0x55bcde06f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.224+0000 7f4278a1d640 1 -- 172.21.15.33:0/1836781023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcde075f30 con 0x55bcde05a2c0 2024-04-02T13:42:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.224+0000 7f4278a1d640 1 --2- 172.21.15.33:0/1836781023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 0x55bcde062eb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55bcde0761b0 tx=0x7f426c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960a3d411c505d3c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.224+0000 7f425a7fc640 1 -- 172.21.15.33:0/1836781023 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c014d40 con 0x55bcde05a2c0 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.224+0000 7f425a7fc640 1 -- 172.21.15.33:0/1836781023 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f426c01f030 con 0x55bcde05a2c0 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.224+0000 7f425a7fc640 1 -- 172.21.15.33:0/1836781023 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c0056b0 con 0x55bcde05a2c0 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.225+0000 7f427d862740 1 -- 172.21.15.33:0/1836781023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 msgr2=0x55bcde062eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.225+0000 7f427d862740 1 --2- 172.21.15.33:0/1836781023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 0x55bcde062eb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55bcde0761b0 tx=0x7f426c0096a0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.225+0000 7f427d862740 1 -- 172.21.15.33:0/1836781023 shutdown_connections 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.225+0000 7f427d862740 1 --2- 172.21.15.33:0/1836781023 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde0637c0 0x55bcde05f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.225+0000 7f427d862740 1 --2- 172.21.15.33:0/1836781023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde05abe0 0x55bcde06f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.225+0000 7f427d862740 1 --2- 172.21.15.33:0/1836781023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 0x55bcde062eb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.225+0000 7f427d862740 1 -- 172.21.15.33:0/1836781023 >> 172.21.15.33:0/1836781023 conn(0x55bcde0549f0 msgr2=0x55bcde060fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.225+0000 7f427d862740 1 -- 172.21.15.33:0/1836781023 shutdown_connections 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.225+0000 7f427d862740 1 -- 172.21.15.33:0/1836781023 wait complete. 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.225+0000 7f427d862740 1 -- start start 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.225+0000 7f427d862740 1 Processor -- start 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f427d862740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 0x55bcde067850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f427d862740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde05abe0 0x55bcde067d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f427d862740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde0637c0 0x55bcde0d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f427d862740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42741af7c0 con 0x55bcde05a2c0 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f427d862740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f42741b0c30 con 0x55bcde0637c0 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f427d862740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f42741afe90 con 0x55bcde05abe0 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f427d862740 10 client.0 _ll_register_callbacks cb 0x7f426400da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4278a1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 0x55bcde067850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4278a1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 0x55bcde067850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39572/0 (socket says 172.21.15.33:39572) 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4278a1d640 1 -- 172.21.15.33:0/2487225517 learned_addr learned my addr 172.21.15.33:0/2487225517 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f427921e640 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde0637c0 0x55bcde0d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4273fff640 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde05abe0 0x55bcde067d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4278a1d640 1 -- 172.21.15.33:0/2487225517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde05abe0 msgr2=0x55bcde067d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4278a1d640 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde05abe0 0x55bcde067d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4278a1d640 1 -- 172.21.15.33:0/2487225517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde0637c0 msgr2=0x55bcde0d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4278a1d640 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde0637c0 0x55bcde0d2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4278a1d640 1 -- 172.21.15.33:0/2487225517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcde0d3080 con 0x55bcde05a2c0 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4273fff640 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde05abe0 0x55bcde067d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f427921e640 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde0637c0 0x55bcde0d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4278a1d640 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 0x55bcde067850 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f426c005c10 tx=0x7f426c0051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1720da4a7f5e241 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4253fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4253fff640 1 -- 172.21.15.33:0/2487225517 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c014e40 con 0x55bcde05a2c0 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f427d862740 1 -- 172.21.15.33:0/2487225517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f426c01f030 con 0x55bcde05a2c0 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.226+0000 7f4253fff640 1 -- 172.21.15.33:0/2487225517 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f426c005330 con 0x55bcde05a2c0 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.227+0000 7f4253fff640 1 -- 172.21.15.33:0/2487225517 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f426c038050 con 0x55bcde05a2c0 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.227+0000 7f427d862740 10 client.16509 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.227+0000 7f427d862740 1 -- 172.21.15.33:0/2487225517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bcde0da470 con 0x55bcde05a2c0 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.227+0000 7f4253fff640 1 -- 172.21.15.33:0/2487225517 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f426c01f030 con 0x55bcde05a2c0 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.227+0000 7f4253fff640 1 -- 172.21.15.33:0/2487225517 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f426c02f450 con 0x55bcde05a2c0 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.227+0000 7f427d862740 10 client.16509 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.227+0000 7f427d862740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.227+0000 7f427d862740 20 client.16509 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:42.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.227+0000 7f427d862740 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bcde0dab30 0x55bcde0dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.227+0000 7f427d862740 4 client.16509 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:42:42.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.227+0000 7f427d862740 1 -- 172.21.15.33:0/2487225517 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bcddf43be0 con 0x55bcde0dab30 2024-04-02T13:42:42.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.227+0000 7f4273fff640 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bcde0dab30 0x55bcde0dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.228+0000 7f4273fff640 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bcde0dab30 0x55bcde0dcf50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf4b3a236a825de2 server_cookie=bc3616359502dbdc in_seq=0 out_seq=0 2024-04-02T13:42:42.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.228+0000 7f4253fff640 10 client.16509 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:42.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f4253fff640 1 -- 172.21.15.33:0/2487225517 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bcddf43be0 con 0x55bcde0dab30 2024-04-02T13:42:42.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f4253fff640 10 client.16509 handle_command_reply: tid=0 2024-04-02T13:42:42.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f427d862740 1 client.16509 shutdown 2024-04-02T13:42:42.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f427d862740 1 -- 172.21.15.33:0/2487225517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 msgr2=0x55bcde067850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f427d862740 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 0x55bcde067850 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f426c005c10 tx=0x7f426c0051a0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f427d862740 1 -- 172.21.15.33:0/2487225517 shutdown_connections 2024-04-02T13:42:42.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f427d862740 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55bcde0dab30 0x55bcde0dcf50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f427d862740 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bcde0637c0 0x55bcde0d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f427d862740 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55bcde05abe0 0x55bcde067d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f4253fff640 0 client.16509 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:42.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f427d862740 1 --2- 172.21.15.33:0/2487225517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bcde05a2c0 0x55bcde067850 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f427d862740 1 -- 172.21.15.33:0/2487225517 >> 172.21.15.33:0/2487225517 conn(0x55bcde0549f0 msgr2=0x55bcde060720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:42.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f427d862740 1 -- 172.21.15.33:0/2487225517 shutdown_connections 2024-04-02T13:42:42.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f427d862740 1 -- 172.21.15.33:0/2487225517 wait complete. 2024-04-02T13:42:42.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.229+0000 7f427d862740 20 client.16509 trim_cache size 0 max 16384 2024-04-02T13:42:42.238 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:42:42.239 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:42:42.239 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "d39bca08-b6a0-48f7-95f1-c88215cfd214", 2024-04-02T13:42:42.239 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:42:42.239 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:42:42.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.230+0000 7f427ac81640 1 -- 172.21.15.33:0/3453219528 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f425c0a3d20 msgr2=0x7f425c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.230+0000 7f427ac81640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f425c0a3d20 0x7f425c0a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4264005fd0 tx=0x7f42640058e0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.230+0000 7f427ac81640 1 -- 172.21.15.33:0/3453219528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274069600 msgr2=0x7f427419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.230+0000 7f427ac81640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274069600 0x7f427419aa70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4268005940 tx=0x7f42680045e0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.230+0000 7f427ac81640 1 -- 172.21.15.33:0/3453219528 shutdown_connections 2024-04-02T13:42:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.230+0000 7f427ac81640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f425c0a3d20 0x7f425c0a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.230+0000 7f427ac81640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f427410e4a0 0x7f427419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.230+0000 7f427ac81640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4274069600 0x7f427419aa70 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.230+0000 7f427ac81640 1 --2- 172.21.15.33:0/3453219528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4274068ce0 0x7f427419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.230+0000 7f427ac81640 1 -- 172.21.15.33:0/3453219528 >> 172.21.15.33:0/3453219528 conn(0x7f4274077b00 msgr2=0x7f427410f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.230+0000 7f427ac81640 1 -- 172.21.15.33:0/3453219528 shutdown_connections 2024-04-02T13:42:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.230+0000 7f427ac81640 1 -- 172.21.15.33:0/3453219528 wait complete. 2024-04-02T13:42:42.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:42:42.245 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:42:42.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.346+0000 7f777cd67640 1 -- 172.21.15.33:0/5445715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781009f0 msgr2=0x7f7778100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.346+0000 7f777cd67640 1 --2- 172.21.15.33:0/5445715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781009f0 0x7f7778100e70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7760009930 tx=0x7f776002bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:42.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.346+0000 7f777cd67640 1 -- 172.21.15.33:0/5445715 shutdown_connections 2024-04-02T13:42:42.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.346+0000 7f777cd67640 1 --2- 172.21.15.33:0/5445715 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77781013b0 0x7f777810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.346+0000 7f777cd67640 1 --2- 172.21.15.33:0/5445715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781009f0 0x7f7778100e70 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.346+0000 7f777cd67640 1 --2- 172.21.15.33:0/5445715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77781069b0 0x7f7778106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.346+0000 7f777cd67640 1 -- 172.21.15.33:0/5445715 >> 172.21.15.33:0/5445715 conn(0x7f7778077b60 msgr2=0x7f7778077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:42.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.346+0000 7f777cd67640 1 -- 172.21.15.33:0/5445715 shutdown_connections 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.346+0000 7f777cd67640 1 -- 172.21.15.33:0/5445715 wait complete. 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.347+0000 7f777cd67640 1 Processor -- start 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.347+0000 7f777cd67640 1 -- start start 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.347+0000 7f777cd67640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781009f0 0x7f77781983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.347+0000 7f777cd67640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77781013b0 0x7f7778198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.347+0000 7f777cd67640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77781069b0 0x7f777819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.347+0000 7f777cd67640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7778110d90 con 0x7f77781009f0 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.347+0000 7f777cd67640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7778110c10 con 0x7f77781013b0 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.347+0000 7f777cd67640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7778110f10 con 0x7f77781069b0 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.347+0000 7f777659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781009f0 0x7f77781983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.347+0000 7f777659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781009f0 0x7f77781983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39594/0 (socket says 172.21.15.33:39594) 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.347+0000 7f777659c640 1 -- 172.21.15.33:0/3085310012 learned_addr learned my addr 172.21.15.33:0/3085310012 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.347+0000 7f776ffff640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77781013b0 0x7f7778198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.348+0000 7f777659c640 1 -- 172.21.15.33:0/3085310012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77781069b0 msgr2=0x7f777819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.348+0000 7f7776d9d640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77781069b0 0x7f777819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.348+0000 7f777659c640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77781069b0 0x7f777819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.348+0000 7f777659c640 1 -- 172.21.15.33:0/3085310012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77781013b0 msgr2=0x7f7778198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.348+0000 7f777659c640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77781013b0 0x7f7778198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.348+0000 7f777659c640 1 -- 172.21.15.33:0/3085310012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777819d3b0 con 0x7f77781009f0 2024-04-02T13:42:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.348+0000 7f777659c640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781009f0 0x7f77781983c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f776800e9f0 tx=0x7f776800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.348+0000 7f776f7fe640 1 -- 172.21.15.33:0/3085310012 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776800cd80 con 0x7f77781009f0 2024-04-02T13:42:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.348+0000 7f777cd67640 1 -- 172.21.15.33:0/3085310012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777819d6a0 con 0x7f77781009f0 2024-04-02T13:42:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.348+0000 7f776f7fe640 1 -- 172.21.15.33:0/3085310012 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7768010900 con 0x7f77781009f0 2024-04-02T13:42:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.348+0000 7f776f7fe640 1 -- 172.21.15.33:0/3085310012 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680189e0 con 0x7f77781009f0 2024-04-02T13:42:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.348+0000 7f777cd67640 1 -- 172.21.15.33:0/3085310012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77781a4f30 con 0x7f77781009f0 2024-04-02T13:42:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.350+0000 7f776f7fe640 1 -- 172.21.15.33:0/3085310012 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7768010ab0 con 0x7f77781009f0 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.350+0000 7f777fa2c740 1 Processor -- start 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777fa2c740 1 -- start start 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777fa2c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644958708d0 0x564495870cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777fa2c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x564495879890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777fa2c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564495879dd0 0x56449587c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777fa2c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5644957d4830 con 0x5644958711f0 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777fa2c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56449577c820 con 0x564495879dd0 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777fa2c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5644957d07e0 con 0x5644958708d0 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x564495879890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x564495879890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39602/0 (socket says 172.21.15.33:39602) 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777659c640 1 -- 172.21.15.33:0/4098117830 learned_addr learned my addr 172.21.15.33:0/4098117830 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f776f7fe640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f77640a3d20 0x7f77640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f776ffff640 1 --2- 172.21.15.33:0/4098117830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564495879dd0 0x56449587c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f7776d9d640 1 --2- 172.21.15.33:0/4098117830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644958708d0 0x564495870cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f7776d9d640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f77640a3d20 0x7f77640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f776f7fe640 1 -- 172.21.15.33:0/3085310012 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f77680d0f20 con 0x7f77781009f0 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777659c640 1 -- 172.21.15.33:0/4098117830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644958708d0 msgr2=0x564495870cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777659c640 1 --2- 172.21.15.33:0/4098117830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644958708d0 0x564495870cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777659c640 1 -- 172.21.15.33:0/4098117830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564495879dd0 msgr2=0x56449587c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777659c640 1 --2- 172.21.15.33:0/4098117830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564495879dd0 0x56449587c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.351+0000 7f777659c640 1 -- 172.21.15.33:0/4098117830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644955b27d0 con 0x5644958711f0 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.352+0000 7f777659c640 1 --2- 172.21.15.33:0/4098117830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x564495879890 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f77680ce230 tx=0x7f7768000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7b7c54f653bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.352+0000 7f7776d9d640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f77640a3d20 0x7f77640a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f777000cfd0 tx=0x7f7770005db0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.352+0000 7f776cff9640 1 -- 172.21.15.33:0/4098117830 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680d45e0 con 0x5644958711f0 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.352+0000 7f776cff9640 1 -- 172.21.15.33:0/4098117830 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f776809d210 con 0x5644958711f0 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.352+0000 7f776cff9640 1 -- 172.21.15.33:0/4098117830 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77680ce540 con 0x5644958711f0 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.352+0000 7f777fa2c740 1 -- 172.21.15.33:0/4098117830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 msgr2=0x564495879890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.352+0000 7f777fa2c740 1 --2- 172.21.15.33:0/4098117830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x564495879890 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f77680ce230 tx=0x7f7768000bc0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.352+0000 7f777fa2c740 1 -- 172.21.15.33:0/4098117830 shutdown_connections 2024-04-02T13:42:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.352+0000 7f777fa2c740 1 --2- 172.21.15.33:0/4098117830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564495879dd0 0x56449587c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.352+0000 7f777fa2c740 1 --2- 172.21.15.33:0/4098117830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x564495879890 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.352+0000 7f777fa2c740 1 --2- 172.21.15.33:0/4098117830 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644958708d0 0x564495870cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f777fa2c740 1 -- 172.21.15.33:0/4098117830 >> 172.21.15.33:0/4098117830 conn(0x56449586b000 msgr2=0x56449586b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f777fa2c740 1 -- 172.21.15.33:0/4098117830 shutdown_connections 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f777fa2c740 1 -- 172.21.15.33:0/4098117830 wait complete. 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f777fa2c740 1 -- start start 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f777fa2c740 1 Processor -- start 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f777fa2c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644958708d0 0x5644958796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f777fa2c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x564495876a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f777fa2c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564495879dd0 0x564495876f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f777fa2c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56449587cb60 con 0x5644958711f0 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f777fa2c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56449578fd30 con 0x5644958708d0 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f777fa2c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56449587cce0 con 0x564495879dd0 2024-04-02T13:42:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f777fa2c740 10 client.0 _ll_register_callbacks cb 0x56449578f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f776ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x564495876a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f776ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x564495876a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39612/0 (socket says 172.21.15.33:39612) 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.353+0000 7f776ffff640 1 -- 172.21.15.33:0/2187645600 learned_addr learned my addr 172.21.15.33:0/2187645600 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f777659c640 1 --2- 172.21.15.33:0/2187645600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644958708d0 0x5644958796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f776ffff640 1 -- 172.21.15.33:0/2187645600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564495879dd0 msgr2=0x564495876f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f776ffff640 1 --2- 172.21.15.33:0/2187645600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564495879dd0 0x564495876f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f776ffff640 1 -- 172.21.15.33:0/2187645600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644958708d0 msgr2=0x5644958796c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f776ffff640 1 --2- 172.21.15.33:0/2187645600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644958708d0 0x5644958796c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f776ffff640 1 -- 172.21.15.33:0/2187645600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644954e83f0 con 0x5644958711f0 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f776ffff640 1 --2- 172.21.15.33:0/2187645600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x564495876a50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7760009230 tx=0x7f7760005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb08a0a85ab86d6e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f775dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f775dffb640 1 -- 172.21.15.33:0/2187645600 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760046070 con 0x5644958711f0 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f775dffb640 1 -- 172.21.15.33:0/2187645600 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f776002c5d0 con 0x5644958711f0 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f777fa2c740 1 -- 172.21.15.33:0/2187645600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f776809d210 con 0x5644958711f0 2024-04-02T13:42:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f775dffb640 1 -- 172.21.15.33:0/2187645600 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f77600055d0 con 0x5644958711f0 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f777fa2c740 10 client.16521 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f777fa2c740 1 -- 172.21.15.33:0/2187645600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644954a80f0 con 0x5644958711f0 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f775dffb640 1 -- 172.21.15.33:0/2187645600 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f776809d210 con 0x5644958711f0 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.354+0000 7f775dffb640 1 -- 172.21.15.33:0/2187645600 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f77600035f0 con 0x5644958711f0 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.355+0000 7f777fa2c740 10 client.16521 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.355+0000 7f777fa2c740 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.355+0000 7f777fa2c740 20 client.16521 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.355+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2187645600 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564495889d70 0x56449588c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.355+0000 7f777fa2c740 4 client.16521 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.355+0000 7f777fa2c740 1 -- 172.21.15.33:0/2187645600 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644957e1d70 con 0x564495889d70 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.355+0000 7f777659c640 1 --2- 172.21.15.33:0/2187645600 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564495889d70 0x56449588c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.355+0000 7f775dffb640 10 client.16521 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.355+0000 7f777659c640 1 --2- 172.21.15.33:0/2187645600 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564495889d70 0x56449588c210 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=210633f9b86457cb server_cookie=8aba95c81496fda8 in_seq=0 out_seq=0 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f775dffb640 1 -- 172.21.15.33:0/2187645600 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644957e1d70 con 0x564495889d70 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f775dffb640 10 client.16521 handle_command_reply: tid=0 2024-04-02T13:42:42.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f777fa2c740 1 client.16521 shutdown 2024-04-02T13:42:42.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f777fa2c740 1 -- 172.21.15.33:0/2187645600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 msgr2=0x564495876a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2187645600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x564495876a50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7760009230 tx=0x7f7760005250 comp rx=0 tx=0).stop 2024-04-02T13:42:42.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f777fa2c740 1 -- 172.21.15.33:0/2187645600 shutdown_connections 2024-04-02T13:42:42.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2187645600 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564495889d70 0x56449588c210 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2187645600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564495879dd0 0x564495876f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2187645600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x564495876a50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f775dffb640 0 client.16521 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:42.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2187645600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644958708d0 0x5644958796c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f777fa2c740 1 -- 172.21.15.33:0/2187645600 >> 172.21.15.33:0/2187645600 conn(0x56449586b000 msgr2=0x56449586cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:42.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f777fa2c740 1 -- 172.21.15.33:0/2187645600 shutdown_connections 2024-04-02T13:42:42.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f777fa2c740 1 -- 172.21.15.33:0/2187645600 wait complete. 2024-04-02T13:42:42.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.356+0000 7f777fa2c740 20 client.16521 trim_cache size 0 max 16384 2024-04-02T13:42:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777fa2c740 1 Processor -- start 2024-04-02T13:42:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777fa2c740 1 -- start start 2024-04-02T13:42:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777fa2c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958708d0 0x564495871e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777fa2c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644958711f0 0x564495879180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777fa2c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564495879dd0 0x5644958796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777fa2c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77781b1950 con 0x5644958708d0 2024-04-02T13:42:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777fa2c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f77781af160 con 0x564495879dd0 2024-04-02T13:42:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777fa2c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f77781acf90 con 0x5644958711f0 2024-04-02T13:42:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958708d0 0x564495871e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958708d0 0x564495871e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39618/0 (socket says 172.21.15.33:39618) 2024-04-02T13:42:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777659c640 1 -- 172.21.15.33:0/2415679108 learned_addr learned my addr 172.21.15.33:0/2415679108 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777659c640 1 -- 172.21.15.33:0/2415679108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644958711f0 msgr2=0x564495879180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777659c640 1 --2- 172.21.15.33:0/2415679108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644958711f0 0x564495879180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777659c640 1 -- 172.21.15.33:0/2415679108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564495879dd0 msgr2=0x5644958796c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f7776d9d640 1 --2- 172.21.15.33:0/2415679108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564495879dd0 0x5644958796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777659c640 1 --2- 172.21.15.33:0/2415679108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564495879dd0 0x5644958796c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777659c640 1 -- 172.21.15.33:0/2415679108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56449586fb70 con 0x5644958708d0 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f7776d9d640 1 --2- 172.21.15.33:0/2415679108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564495879dd0 0x5644958796c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.374+0000 7f777659c640 1 --2- 172.21.15.33:0/2415679108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958708d0 0x564495871e20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f77680994e0 tx=0x7f7768098780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a360ccdfc2ddc8e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f775ffff640 1 -- 172.21.15.33:0/2415679108 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776809ae10 con 0x5644958708d0 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f775ffff640 1 -- 172.21.15.33:0/2415679108 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7768099180 con 0x5644958708d0 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f775ffff640 1 -- 172.21.15.33:0/2415679108 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768018e10 con 0x5644958708d0 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f777fa2c740 1 -- 172.21.15.33:0/2415679108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958708d0 msgr2=0x564495871e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2415679108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958708d0 0x564495871e20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f77680994e0 tx=0x7f7768098780 comp rx=0 tx=0).stop 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f777fa2c740 1 -- 172.21.15.33:0/2415679108 shutdown_connections 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2415679108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564495879dd0 0x5644958796c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2415679108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5644958711f0 0x564495879180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2415679108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958708d0 0x564495871e20 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f777fa2c740 1 -- 172.21.15.33:0/2415679108 >> 172.21.15.33:0/2415679108 conn(0x56449586b000 msgr2=0x564495871710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f777fa2c740 1 -- 172.21.15.33:0/2415679108 shutdown_connections 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f777fa2c740 1 -- 172.21.15.33:0/2415679108 wait complete. 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f777fa2c740 1 -- start start 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.375+0000 7f777fa2c740 1 Processor -- start 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f777fa2c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644958708d0 0x56449587d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f777fa2c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x56449587def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f777fa2c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564495879dd0 0x56449587e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f777fa2c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77781ad620 con 0x5644958711f0 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f777fa2c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f77781aea90 con 0x5644958708d0 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f777fa2c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f77781adcf0 con 0x564495879dd0 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f777fa2c740 10 client.0 _ll_register_callbacks cb 0x7f7760004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f776ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x56449587def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f776ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x56449587def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39628/0 (socket says 172.21.15.33:39628) 2024-04-02T13:42:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f776ffff640 1 -- 172.21.15.33:0/2973646058 learned_addr learned my addr 172.21.15.33:0/2973646058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f7776d9d640 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564495879dd0 0x56449587e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f777659c640 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644958708d0 0x56449587d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f776ffff640 1 -- 172.21.15.33:0/2973646058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564495879dd0 msgr2=0x56449587e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f776ffff640 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564495879dd0 0x56449587e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f776ffff640 1 -- 172.21.15.33:0/2973646058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644958708d0 msgr2=0x56449587d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f776ffff640 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644958708d0 0x56449587d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f776ffff640 1 -- 172.21.15.33:0/2973646058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644958702d0 con 0x5644958711f0 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f777659c640 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644958708d0 0x56449587d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f776ffff640 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x56449587def0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x5644958762b0 tx=0x7f7760003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f69e65e25810fd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.376+0000 7f775dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f775dffb640 1 -- 172.21.15.33:0/2973646058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760046040 con 0x5644958711f0 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f777fa2c740 1 -- 172.21.15.33:0/2973646058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7768099180 con 0x5644958711f0 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f775dffb640 1 -- 172.21.15.33:0/2973646058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f776002c9e0 con 0x5644958711f0 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f775dffb640 1 -- 172.21.15.33:0/2973646058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f776005e050 con 0x5644958711f0 2024-04-02T13:42:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f777fa2c740 10 client.16527 fetch_fsmap learned FSMap version 30 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f777fa2c740 1 -- 172.21.15.33:0/2973646058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644958f0450 con 0x5644958711f0 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f775dffb640 1 -- 172.21.15.33:0/2973646058 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7768099180 con 0x5644958711f0 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f775dffb640 1 -- 172.21.15.33:0/2973646058 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f77600584c0 con 0x5644958711f0 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f777fa2c740 10 client.16527 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f777fa2c740 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f777fa2c740 20 client.16527 populate_metadata read hostname 'smithi033' 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5644958f0b40 0x5644958f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f777fa2c740 4 client.16527 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f777fa2c740 1 -- 172.21.15.33:0/2973646058 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5644957dc0c0 con 0x5644958f0b40 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.377+0000 7f777659c640 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5644958f0b40 0x5644958f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f775dffb640 10 client.16527 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f777659c640 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5644958f0b40 0x5644958f2f60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=813ccd0d78833cdd server_cookie=821935ec81e3ecdb in_seq=0 out_seq=0 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f775dffb640 1 -- 172.21.15.33:0/2973646058 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5644957dc0c0 con 0x5644958f0b40 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f775dffb640 10 client.16527 handle_command_reply: tid=0 2024-04-02T13:42:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f777fa2c740 1 client.16527 shutdown 2024-04-02T13:42:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f777fa2c740 1 -- 172.21.15.33:0/2973646058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 msgr2=0x56449587def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x56449587def0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x5644958762b0 tx=0x7f7760003e30 comp rx=0 tx=0).stop 2024-04-02T13:42:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f777fa2c740 1 -- 172.21.15.33:0/2973646058 shutdown_connections 2024-04-02T13:42:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5644958f0b40 0x5644958f2f60 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564495879dd0 0x56449587e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f775dffb640 0 client.16527 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:42:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644958711f0 0x56449587def0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f777fa2c740 1 --2- 172.21.15.33:0/2973646058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5644958708d0 0x56449587d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f777fa2c740 1 -- 172.21.15.33:0/2973646058 >> 172.21.15.33:0/2973646058 conn(0x56449586b000 msgr2=0x564495871fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f777fa2c740 1 -- 172.21.15.33:0/2973646058 shutdown_connections 2024-04-02T13:42:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f777fa2c740 1 -- 172.21.15.33:0/2973646058 wait complete. 2024-04-02T13:42:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.378+0000 7f777fa2c740 20 client.16527 trim_cache size 0 max 16384 2024-04-02T13:42:42.389 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:42:42.389 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:42:42.389 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:42:42.389 INFO:teuthology.orchestra.run.smithi033.stdout: "d39bca08-b6a0-48f7-95f1-c88215cfd214": { 2024-04-02T13:42:42.389 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:42:42.389 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "d39bca08-b6a0-48f7-95f1-c88215cfd214", 2024-04-02T13:42:42.389 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:42:42.389 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:42:42.389 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:42:42.390 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:42:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.379+0000 7f777cd67640 1 -- 172.21.15.33:0/3085310012 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f77640a3d20 msgr2=0x7f77640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.379+0000 7f777cd67640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f77640a3d20 0x7f77640a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f777000cfd0 tx=0x7f7770005db0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.379+0000 7f777cd67640 1 -- 172.21.15.33:0/3085310012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781009f0 msgr2=0x7f77781983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.379+0000 7f777cd67640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781009f0 0x7f77781983c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f776800e9f0 tx=0x7f776800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.379+0000 7f777cd67640 1 -- 172.21.15.33:0/3085310012 shutdown_connections 2024-04-02T13:42:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.379+0000 7f777cd67640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f77640a3d20 0x7f77640a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.379+0000 7f777cd67640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77781069b0 0x7f777819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.379+0000 7f777cd67640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77781013b0 0x7f7778198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.379+0000 7f777cd67640 1 --2- 172.21.15.33:0/3085310012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781009f0 0x7f77781983c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.379+0000 7f777cd67640 1 -- 172.21.15.33:0/3085310012 >> 172.21.15.33:0/3085310012 conn(0x7f7778077b60 msgr2=0x7f777806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.379+0000 7f777cd67640 1 -- 172.21.15.33:0/3085310012 shutdown_connections 2024-04-02T13:42:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:42:42.379+0000 7f777cd67640 1 -- 172.21.15.33:0/3085310012 wait complete. 2024-04-02T13:42:42.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:42 smithi033 ceph-mon[32390]: pgmap v120: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-02T13:42:42.393 INFO:tasks.cephfs.filesystem:scrub status for tag:d39bca08-b6a0-48f7-95f1-c88215cfd214 - {'path': '/', 'tag': 'd39bca08-b6a0-48f7-95f1-c88215cfd214', 'options': 'recursive,force'} 2024-04-02T13:42:42.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:42 smithi145 ceph-mon[35646]: pgmap v120: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-02T13:42:42.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:42 smithi050 ceph-mon[35524]: pgmap v120: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-02T13:42:43.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:43 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:42:43.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:43 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:43.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:43 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:42:43.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:43 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:43.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:43 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:42:43.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:43 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:43.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:43 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:42:43.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:43 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:43.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:43 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:42:43.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:43 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:43.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:43 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:42:43.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:43 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:44.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:44 smithi145 ceph-mon[35646]: pgmap v121: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-02T13:42:44.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:44 smithi050 ceph-mon[35524]: pgmap v121: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-02T13:42:44.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:44 smithi033 ceph-mon[32390]: pgmap v121: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-02T13:42:46.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:46 smithi145 ceph-mon[35646]: pgmap v122: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:42:46.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:46 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:42:46.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:46 smithi050 ceph-mon[35524]: pgmap v122: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:42:46.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:46 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:42:46.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:46 smithi033 ceph-mon[32390]: pgmap v122: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:42:46.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:46 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:42:48.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:48 smithi145 ceph-mon[35646]: pgmap v123: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:42:48.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:48 smithi050 ceph-mon[35524]: pgmap v123: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:42:48.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:48 smithi033 ceph-mon[32390]: pgmap v123: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:42:50.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:50 smithi145 ceph-mon[35646]: pgmap v124: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:42:50.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:50 smithi050 ceph-mon[35524]: pgmap v124: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:42:50.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:50 smithi033 ceph-mon[32390]: pgmap v124: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:42:52.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:52 smithi145 ceph-mon[35646]: pgmap v125: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:42:52.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:52 smithi050 ceph-mon[35524]: pgmap v125: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:42:52.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:52 smithi033 ceph-mon[32390]: pgmap v125: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:42:54.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:54 smithi145 ceph-mon[35646]: pgmap v126: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:42:54.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:54 smithi050 ceph-mon[35524]: pgmap v126: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:42:54.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:54 smithi033 ceph-mon[32390]: pgmap v126: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:42:55.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:55 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:42:55.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:55 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:42:55.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:55 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:42:56.379 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:56 smithi145 ceph-mon[35646]: pgmap v127: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:42:56.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:56 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:42:56.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:56 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:42:56.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:56 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:42:56.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:56 smithi050 ceph-mon[35524]: pgmap v127: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:42:56.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:56 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:42:56.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:56 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:42:56.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:56 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:42:56.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:56 smithi033 ceph-mon[32390]: pgmap v127: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:42:56.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:56 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:42:56.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:56 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:42:56.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:56 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:42:58.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:42:58 smithi145 ceph-mon[35646]: pgmap v128: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:42:58.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:42:58 smithi050 ceph-mon[35524]: pgmap v128: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:42:58.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:42:58 smithi033 ceph-mon[32390]: pgmap v128: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:00.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:00 smithi145 ceph-mon[35646]: pgmap v129: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:43:00.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:00 smithi050 ceph-mon[35524]: pgmap v129: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:43:00.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:00 smithi033 ceph-mon[32390]: pgmap v129: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:43:02.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:02 smithi145 ceph-mon[35646]: pgmap v130: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:02.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:02 smithi050 ceph-mon[35524]: pgmap v130: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:02.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:02 smithi033 ceph-mon[32390]: pgmap v130: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:04.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:04 smithi145 ceph-mon[35646]: pgmap v131: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:43:04.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:04 smithi050 ceph-mon[35524]: pgmap v131: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:43:04.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:04 smithi033 ceph-mon[32390]: pgmap v131: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:43:06.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:06 smithi145 ceph-mon[35646]: pgmap v132: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:43:06.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:06 smithi050 ceph-mon[35524]: pgmap v132: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:43:06.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:06 smithi033 ceph-mon[32390]: pgmap v132: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:43:08.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:08 smithi145 ceph-mon[35646]: pgmap v133: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:43:08.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:08 smithi050 ceph-mon[35524]: pgmap v133: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:43:08.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:08 smithi033 ceph-mon[32390]: pgmap v133: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:43:10.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:10 smithi145 ceph-mon[35646]: pgmap v134: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:43:10.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:10 smithi050 ceph-mon[35524]: pgmap v134: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:43:10.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:10 smithi033 ceph-mon[32390]: pgmap v134: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:43:12.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:43:12.394 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:43:12.423 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:12 smithi033 ceph-mon[32390]: pgmap v135: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:43:12.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:12 smithi145 ceph-mon[35646]: pgmap v135: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:43:12.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:12 smithi050 ceph-mon[35524]: pgmap v135: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:43:12.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.509+0000 7f5fa2aa5640 1 -- 172.21.15.33:0/103134373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f9c107ab0 msgr2=0x7f5f9c107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.509+0000 7f5fa2aa5640 1 --2- 172.21.15.33:0/103134373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f9c107ab0 0x7f5f9c107e90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009930 tx=0x7f5f9002bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.509+0000 7f5fa2aa5640 1 -- 172.21.15.33:0/103134373 shutdown_connections 2024-04-02T13:43:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.509+0000 7f5fa2aa5640 1 --2- 172.21.15.33:0/103134373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f9c102460 0x7f5f9c10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.509+0000 7f5fa2aa5640 1 --2- 172.21.15.33:0/103134373 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f9c101aa0 0x7f5f9c101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.509+0000 7f5fa2aa5640 1 --2- 172.21.15.33:0/103134373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f9c107ab0 0x7f5f9c107e90 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.509+0000 7f5fa2aa5640 1 -- 172.21.15.33:0/103134373 >> 172.21.15.33:0/103134373 conn(0x7f5f9c0fd530 msgr2=0x7f5f9c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.509+0000 7f5fa2aa5640 1 -- 172.21.15.33:0/103134373 shutdown_connections 2024-04-02T13:43:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.509+0000 7f5fa2aa5640 1 -- 172.21.15.33:0/103134373 wait complete. 2024-04-02T13:43:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.510+0000 7f5fa2aa5640 1 Processor -- start 2024-04-02T13:43:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.510+0000 7f5fa2aa5640 1 -- start start 2024-04-02T13:43:12.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.510+0000 7f5fa2aa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f9c101aa0 0x7f5f9c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.510+0000 7f5fa2aa5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f9c102460 0x7f5f9c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.510+0000 7f5fa2aa5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f9c107ab0 0x7f5f9c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.510+0000 7f5fa2aa5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c111e60 con 0x7f5f9c101aa0 2024-04-02T13:43:12.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.510+0000 7f5fa2aa5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c111ce0 con 0x7f5f9c102460 2024-04-02T13:43:12.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.510+0000 7f5fa2aa5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c111fe0 con 0x7f5f9c107ab0 2024-04-02T13:43:12.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.510+0000 7f5fa0841640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f9c101aa0 0x7f5f9c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.510+0000 7f5fa0841640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f9c101aa0 0x7f5f9c198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48950/0 (socket says 172.21.15.33:48950) 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.510+0000 7f5fa0841640 1 -- 172.21.15.33:0/1412558577 learned_addr learned my addr 172.21.15.33:0/1412558577 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.510+0000 7f5f9bfff640 1 --2- 172.21.15.33:0/1412558577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f9c102460 0x7f5f9c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.510+0000 7f5fa0841640 1 -- 172.21.15.33:0/1412558577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f9c107ab0 msgr2=0x7f5f9c19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.511+0000 7f5fa0841640 1 --2- 172.21.15.33:0/1412558577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f9c107ab0 0x7f5f9c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.511+0000 7f5fa0841640 1 -- 172.21.15.33:0/1412558577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f9c102460 msgr2=0x7f5f9c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.511+0000 7f5fa0841640 1 --2- 172.21.15.33:0/1412558577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f9c102460 0x7f5f9c198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.511+0000 7f5fa0841640 1 -- 172.21.15.33:0/1412558577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9c19d310 con 0x7f5f9c101aa0 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.511+0000 7f5fa0841640 1 --2- 172.21.15.33:0/1412558577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f9c101aa0 0x7f5f9c198320 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5f90000c00 tx=0x7f5f900045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.511+0000 7f5f99ffb640 1 -- 172.21.15.33:0/1412558577 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90043020 con 0x7f5f9c101aa0 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.511+0000 7f5f99ffb640 1 -- 172.21.15.33:0/1412558577 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f9002caa0 con 0x7f5f9c101aa0 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.511+0000 7f5fa2aa5640 1 -- 172.21.15.33:0/1412558577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9c19d5a0 con 0x7f5f9c101aa0 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.511+0000 7f5f99ffb640 1 -- 172.21.15.33:0/1412558577 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90003bf0 con 0x7f5f9c101aa0 2024-04-02T13:43:12.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.511+0000 7f5fa2aa5640 1 -- 172.21.15.33:0/1412558577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9c1a4e90 con 0x7f5f9c101aa0 2024-04-02T13:43:12.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.513+0000 7f5f99ffb640 1 -- 172.21.15.33:0/1412558577 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5f90046020 con 0x7f5f9c101aa0 2024-04-02T13:43:12.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa576a740 1 Processor -- start 2024-04-02T13:43:12.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa576a740 1 -- start start 2024-04-02T13:43:12.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa576a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 0x564c4bc92cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa576a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc9b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa576a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 0x564c4bc9e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa576a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564c4bbf6830 con 0x564c4bc931f0 2024-04-02T13:43:12.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa576a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564c4bb9e820 con 0x564c4bc9bdd0 2024-04-02T13:43:12.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa576a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564c4bbf27e0 con 0x564c4bc928d0 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5f99ffb640 1 --2- 172.21.15.33:0/1412558577 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f8c0a3d20 0x7f5f8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa0841640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc9b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa0841640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc9b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48954/0 (socket says 172.21.15.33:48954) 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa0841640 1 -- 172.21.15.33:0/4039848048 learned_addr learned my addr 172.21.15.33:0/4039848048 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa1042640 1 --2- 172.21.15.33:0/1412558577 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f8c0a3d20 0x7f5f8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5f9bfff640 1 --2- 172.21.15.33:0/4039848048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 0x564c4bc9e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa1042640 1 --2- 172.21.15.33:0/4039848048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 0x564c4bc92cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5f99ffb640 1 -- 172.21.15.33:0/1412558577 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5f900f2380 con 0x7f5f9c101aa0 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa0841640 1 -- 172.21.15.33:0/4039848048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 msgr2=0x564c4bc92cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa0841640 1 --2- 172.21.15.33:0/4039848048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 0x564c4bc92cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa0841640 1 -- 172.21.15.33:0/4039848048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 msgr2=0x564c4bc9e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa0841640 1 --2- 172.21.15.33:0/4039848048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 0x564c4bc9e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa0841640 1 -- 172.21.15.33:0/4039848048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4b9d47d0 con 0x564c4bc931f0 2024-04-02T13:43:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.514+0000 7f5fa1042640 1 --2- 172.21.15.33:0/4039848048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 0x564c4bc92cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.515+0000 7f5fa0841640 1 --2- 172.21.15.33:0/4039848048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc9b890 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5f900ef890 tx=0x7f5f90009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdea83e3e755e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.515+0000 7f5fa1042640 1 --2- 172.21.15.33:0/1412558577 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f8c0a3d20 0x7f5f8c0a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5f940136e0 tx=0x7f5f940092a0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.515+0000 7f5f86ffd640 1 -- 172.21.15.33:0/4039848048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f900be780 con 0x564c4bc931f0 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.515+0000 7f5f86ffd640 1 -- 172.21.15.33:0/4039848048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f900c2430 con 0x564c4bc931f0 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.515+0000 7f5f86ffd640 1 -- 172.21.15.33:0/4039848048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90003d90 con 0x564c4bc931f0 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.515+0000 7f5fa576a740 1 -- 172.21.15.33:0/4039848048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 msgr2=0x564c4bc9b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.516+0000 7f5fa576a740 1 --2- 172.21.15.33:0/4039848048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc9b890 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5f900ef890 tx=0x7f5f90009260 comp rx=0 tx=0).stop 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.516+0000 7f5fa576a740 1 -- 172.21.15.33:0/4039848048 shutdown_connections 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.516+0000 7f5fa576a740 1 --2- 172.21.15.33:0/4039848048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 0x564c4bc9e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.516+0000 7f5fa576a740 1 --2- 172.21.15.33:0/4039848048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc9b890 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.516+0000 7f5fa576a740 1 --2- 172.21.15.33:0/4039848048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 0x564c4bc92cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.516+0000 7f5fa576a740 1 -- 172.21.15.33:0/4039848048 >> 172.21.15.33:0/4039848048 conn(0x564c4bc8d000 msgr2=0x564c4bc8d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.516+0000 7f5fa576a740 1 -- 172.21.15.33:0/4039848048 shutdown_connections 2024-04-02T13:43:12.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.516+0000 7f5fa576a740 1 -- 172.21.15.33:0/4039848048 wait complete. 2024-04-02T13:43:12.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.516+0000 7f5fa576a740 1 -- start start 2024-04-02T13:43:12.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.516+0000 7f5fa576a740 1 Processor -- start 2024-04-02T13:43:12.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.516+0000 7f5fa576a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc928d0 0x564c4bc9b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.516+0000 7f5fa576a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc931f0 0x564c4bc98960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.516+0000 7f5fa576a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc9bdd0 0x564c4bc98ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa576a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564c4bbf6830 con 0x564c4bc928d0 2024-04-02T13:43:12.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa576a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564c4bbb1d30 con 0x564c4bc931f0 2024-04-02T13:43:12.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa576a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564c4bc9e980 con 0x564c4bc9bdd0 2024-04-02T13:43:12.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa576a740 10 client.0 _ll_register_callbacks cb 0x564c4bbb1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:12.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa0841640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc928d0 0x564c4bc9b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa0841640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc928d0 0x564c4bc9b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48968/0 (socket says 172.21.15.33:48968) 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa0841640 1 -- 172.21.15.33:0/1323617697 learned_addr learned my addr 172.21.15.33:0/1323617697 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5f9bfff640 1 --2- 172.21.15.33:0/1323617697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc931f0 0x564c4bc98960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa0841640 1 -- 172.21.15.33:0/1323617697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc9bdd0 msgr2=0x564c4bc98ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa0841640 1 --2- 172.21.15.33:0/1323617697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc9bdd0 0x564c4bc98ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa0841640 1 -- 172.21.15.33:0/1323617697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc931f0 msgr2=0x564c4bc98960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa0841640 1 --2- 172.21.15.33:0/1323617697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc931f0 0x564c4bc98960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa0841640 1 -- 172.21.15.33:0/1323617697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4b90a3f0 con 0x564c4bc928d0 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5f9bfff640 1 --2- 172.21.15.33:0/1323617697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc931f0 0x564c4bc98960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa0841640 1 --2- 172.21.15.33:0/1323617697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc928d0 0x564c4bc9b5d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5f900096c0 tx=0x7f5f900c6d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5aba86ddad516d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5f7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5f7bfff640 1 -- 172.21.15.33:0/1323617697 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f900be080 con 0x564c4bc928d0 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5f7bfff640 1 -- 172.21.15.33:0/1323617697 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5f90005330 con 0x564c4bc928d0 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa576a740 1 -- 172.21.15.33:0/1323617697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f900c2430 con 0x564c4bc928d0 2024-04-02T13:43:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5f7bfff640 1 -- 172.21.15.33:0/1323617697 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f900bc380 con 0x564c4bc928d0 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa576a740 10 client.16539 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.517+0000 7f5fa576a740 1 -- 172.21.15.33:0/1323617697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564c4b8ca0f0 con 0x564c4bc928d0 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.518+0000 7f5f7bfff640 1 -- 172.21.15.33:0/1323617697 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f900c2430 con 0x564c4bc928d0 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.518+0000 7f5f7bfff640 1 -- 172.21.15.33:0/1323617697 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5f900c6860 con 0x564c4bc928d0 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.518+0000 7f5fa576a740 10 client.16539 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.518+0000 7f5fa576a740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.518+0000 7f5fa576a740 20 client.16539 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.518+0000 7f5fa576a740 1 --2- 172.21.15.33:0/1323617697 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564c4bcab840 0x564c4bcadc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.518+0000 7f5fa576a740 4 client.16539 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.518+0000 7f5fa576a740 1 -- 172.21.15.33:0/1323617697 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c4bc03d70 con 0x564c4bcab840 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.518+0000 7f5f9bfff640 1 --2- 172.21.15.33:0/1323617697 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564c4bcab840 0x564c4bcadc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5f9bfff640 1 --2- 172.21.15.33:0/1323617697 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564c4bcab840 0x564c4bcadc90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaabee17d08c9c24 server_cookie=6879909ae824d137 in_seq=0 out_seq=0 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5f7bfff640 10 client.16539 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5f7bfff640 1 -- 172.21.15.33:0/1323617697 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c4bc03d70 con 0x564c4bcab840 2024-04-02T13:43:12.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5f7bfff640 10 client.16539 handle_command_reply: tid=0 2024-04-02T13:43:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5fa576a740 1 client.16539 shutdown 2024-04-02T13:43:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5fa576a740 1 -- 172.21.15.33:0/1323617697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc928d0 msgr2=0x564c4bc9b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5fa576a740 1 --2- 172.21.15.33:0/1323617697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc928d0 0x564c4bc9b5d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5f900096c0 tx=0x7f5f900c6d70 comp rx=0 tx=0).stop 2024-04-02T13:43:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5fa576a740 1 -- 172.21.15.33:0/1323617697 shutdown_connections 2024-04-02T13:43:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5fa576a740 1 --2- 172.21.15.33:0/1323617697 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564c4bcab840 0x564c4bcadc90 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5fa576a740 1 --2- 172.21.15.33:0/1323617697 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc9bdd0 0x564c4bc98ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5fa576a740 1 --2- 172.21.15.33:0/1323617697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc931f0 0x564c4bc98960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5fa576a740 1 --2- 172.21.15.33:0/1323617697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc928d0 0x564c4bc9b5d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5f7bfff640 0 client.16539 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.519+0000 7f5fa576a740 1 -- 172.21.15.33:0/1323617697 >> 172.21.15.33:0/1323617697 conn(0x564c4bc8d000 msgr2=0x564c4bc8ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.520+0000 7f5fa576a740 1 -- 172.21.15.33:0/1323617697 shutdown_connections 2024-04-02T13:43:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.520+0000 7f5fa576a740 1 -- 172.21.15.33:0/1323617697 wait complete. 2024-04-02T13:43:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.520+0000 7f5fa576a740 20 client.16539 trim_cache size 0 max 16384 2024-04-02T13:43:12.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.537+0000 7f5fa576a740 1 Processor -- start 2024-04-02T13:43:12.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.537+0000 7f5fa576a740 1 -- start start 2024-04-02T13:43:12.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.537+0000 7f5fa576a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 0x564c4bca3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5fa576a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc97ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5fa576a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 0x564c4bc8c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5fa576a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c1b1820 con 0x564c4bc931f0 2024-04-02T13:43:12.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5fa576a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c1af030 con 0x564c4bc9bdd0 2024-04-02T13:43:12.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5fa576a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c1ace60 con 0x564c4bc928d0 2024-04-02T13:43:12.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5f9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc97ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5f9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc97ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48978/0 (socket says 172.21.15.33:48978) 2024-04-02T13:43:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5f9bfff640 1 -- 172.21.15.33:0/151726240 learned_addr learned my addr 172.21.15.33:0/151726240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5fa1042640 1 --2- 172.21.15.33:0/151726240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 0x564c4bc8c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5f9bfff640 1 -- 172.21.15.33:0/151726240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 msgr2=0x564c4bca3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5f9bfff640 1 --2- 172.21.15.33:0/151726240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 0x564c4bca3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5f9bfff640 1 -- 172.21.15.33:0/151726240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 msgr2=0x564c4bc8c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5f9bfff640 1 --2- 172.21.15.33:0/151726240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 0x564c4bc8c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5f9bfff640 1 -- 172.21.15.33:0/151726240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4bc8ce30 con 0x564c4bc931f0 2024-04-02T13:43:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5fa1042640 1 --2- 172.21.15.33:0/151726240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 0x564c4bc8c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5f9bfff640 1 --2- 172.21.15.33:0/151726240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc97ac0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x564c4bc8e190 tx=0x7f5f88005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7326e5e630e4e4d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5f867fc640 1 -- 172.21.15.33:0/151726240 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88013b10 con 0x564c4bc931f0 2024-04-02T13:43:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5f867fc640 1 -- 172.21.15.33:0/151726240 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f8801e030 con 0x564c4bc931f0 2024-04-02T13:43:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.538+0000 7f5f867fc640 1 -- 172.21.15.33:0/151726240 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88011a10 con 0x564c4bc931f0 2024-04-02T13:43:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.539+0000 7f5fa576a740 1 -- 172.21.15.33:0/151726240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 msgr2=0x564c4bc97ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.539+0000 7f5fa576a740 1 --2- 172.21.15.33:0/151726240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc97ac0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x564c4bc8e190 tx=0x7f5f88005c50 comp rx=0 tx=0).stop 2024-04-02T13:43:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.539+0000 7f5fa576a740 1 -- 172.21.15.33:0/151726240 shutdown_connections 2024-04-02T13:43:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.539+0000 7f5fa576a740 1 --2- 172.21.15.33:0/151726240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 0x564c4bc8c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.539+0000 7f5fa576a740 1 --2- 172.21.15.33:0/151726240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc97ac0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.539+0000 7f5fa576a740 1 --2- 172.21.15.33:0/151726240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 0x564c4bca3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.539+0000 7f5fa576a740 1 -- 172.21.15.33:0/151726240 >> 172.21.15.33:0/151726240 conn(0x564c4bc8d000 msgr2=0x564c4bc8b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.539+0000 7f5fa576a740 1 -- 172.21.15.33:0/151726240 shutdown_connections 2024-04-02T13:43:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.539+0000 7f5fa576a740 1 -- 172.21.15.33:0/151726240 wait complete. 2024-04-02T13:43:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.539+0000 7f5fa576a740 1 -- start start 2024-04-02T13:43:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.539+0000 7f5fa576a740 1 Processor -- start 2024-04-02T13:43:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5fa576a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 0x564c4bc94bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5fa576a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc95110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5fa576a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 0x564c4bc95850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5fa576a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c1ad4f0 con 0x564c4bc931f0 2024-04-02T13:43:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5fa576a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c1ae960 con 0x564c4bc9bdd0 2024-04-02T13:43:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5fa576a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c1adbc0 con 0x564c4bc928d0 2024-04-02T13:43:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5fa576a740 10 client.0 _ll_register_callbacks cb 0x7f5f900bd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5f9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc95110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5f9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc95110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48986/0 (socket says 172.21.15.33:48986) 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5f9bfff640 1 -- 172.21.15.33:0/3565237416 learned_addr learned my addr 172.21.15.33:0/3565237416 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5fa0841640 1 --2- 172.21.15.33:0/3565237416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 0x564c4bc94bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5fa1042640 1 --2- 172.21.15.33:0/3565237416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 0x564c4bc95850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5f9bfff640 1 -- 172.21.15.33:0/3565237416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 msgr2=0x564c4bc94bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5f9bfff640 1 --2- 172.21.15.33:0/3565237416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 0x564c4bc94bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5f9bfff640 1 -- 172.21.15.33:0/3565237416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 msgr2=0x564c4bc95850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5f9bfff640 1 --2- 172.21.15.33:0/3565237416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 0x564c4bc95850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5f9bfff640 1 -- 172.21.15.33:0/3565237416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4bc96050 con 0x564c4bc931f0 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5f9bfff640 1 --2- 172.21.15.33:0/3565237416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc95110 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x564c4bc8e190 tx=0x7f5f88009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d3f835154ca77c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5f7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5f7bfff640 1 -- 172.21.15.33:0/3565237416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88013ad0 con 0x564c4bc931f0 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5f7bfff640 1 -- 172.21.15.33:0/3565237416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5f88013c70 con 0x564c4bc931f0 2024-04-02T13:43:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.540+0000 7f5fa576a740 1 -- 172.21.15.33:0/3565237416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f8801e030 con 0x564c4bc931f0 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.541+0000 7f5f7bfff640 1 -- 172.21.15.33:0/3565237416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f8801a330 con 0x564c4bc931f0 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.541+0000 7f5fa576a740 10 client.16545 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.541+0000 7f5fa576a740 1 -- 172.21.15.33:0/3565237416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564c4bd16510 con 0x564c4bc931f0 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.541+0000 7f5f7bfff640 1 -- 172.21.15.33:0/3565237416 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5f8801e030 con 0x564c4bc931f0 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.541+0000 7f5f7bfff640 1 -- 172.21.15.33:0/3565237416 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5f8800f040 con 0x564c4bc931f0 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.541+0000 7f5fa576a740 10 client.16545 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.541+0000 7f5fa576a740 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.541+0000 7f5fa576a740 20 client.16545 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.541+0000 7f5fa576a740 1 --2- 172.21.15.33:0/3565237416 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564c4bd16c60 0x564c4bd19080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.541+0000 7f5fa576a740 4 client.16545 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.541+0000 7f5fa576a740 1 -- 172.21.15.33:0/3565237416 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564c4bbfe0c0 con 0x564c4bd16c60 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.541+0000 7f5fa0841640 1 --2- 172.21.15.33:0/3565237416 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564c4bd16c60 0x564c4bd19080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5f7bfff640 10 client.16545 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5fa0841640 1 --2- 172.21.15.33:0/3565237416 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564c4bd16c60 0x564c4bd19080 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb1bb2c942ba443 server_cookie=fbbcfed1fb0af30f in_seq=0 out_seq=0 2024-04-02T13:43:12.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5f7bfff640 1 -- 172.21.15.33:0/3565237416 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564c4bbfe0c0 con 0x564c4bd16c60 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5f7bfff640 10 client.16545 handle_command_reply: tid=0 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5fa576a740 1 client.16545 shutdown 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5fa576a740 1 -- 172.21.15.33:0/3565237416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 msgr2=0x564c4bc95110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5fa576a740 1 --2- 172.21.15.33:0/3565237416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc95110 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x564c4bc8e190 tx=0x7f5f88009ef0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5fa576a740 1 -- 172.21.15.33:0/3565237416 shutdown_connections 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5fa576a740 1 --2- 172.21.15.33:0/3565237416 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564c4bd16c60 0x564c4bd19080 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5fa576a740 1 --2- 172.21.15.33:0/3565237416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564c4bc9bdd0 0x564c4bc95850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5f7bfff640 0 client.16545 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5fa576a740 1 --2- 172.21.15.33:0/3565237416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4bc931f0 0x564c4bc95110 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5fa576a740 1 --2- 172.21.15.33:0/3565237416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564c4bc928d0 0x564c4bc94bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5fa576a740 1 -- 172.21.15.33:0/3565237416 >> 172.21.15.33:0/3565237416 conn(0x564c4bc8d000 msgr2=0x564c4bc8c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5fa576a740 1 -- 172.21.15.33:0/3565237416 shutdown_connections 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5fa576a740 1 -- 172.21.15.33:0/3565237416 wait complete. 2024-04-02T13:43:12.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.542+0000 7f5fa576a740 20 client.16545 trim_cache size 0 max 16384 2024-04-02T13:43:12.557 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:43:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:43:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:43:12.557 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:43:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.543+0000 7f5fa2aa5640 1 -- 172.21.15.33:0/1412558577 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f8c0a3d20 msgr2=0x7f5f8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.543+0000 7f5fa2aa5640 1 --2- 172.21.15.33:0/1412558577 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f8c0a3d20 0x7f5f8c0a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5f940136e0 tx=0x7f5f940092a0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.543+0000 7f5fa2aa5640 1 -- 172.21.15.33:0/1412558577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f9c101aa0 msgr2=0x7f5f9c198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.543+0000 7f5fa2aa5640 1 --2- 172.21.15.33:0/1412558577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f9c101aa0 0x7f5f9c198320 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5f90000c00 tx=0x7f5f900045e0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.543+0000 7f5fa2aa5640 1 -- 172.21.15.33:0/1412558577 shutdown_connections 2024-04-02T13:43:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.543+0000 7f5fa2aa5640 1 --2- 172.21.15.33:0/1412558577 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5f8c0a3d20 0x7f5f8c0a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.543+0000 7f5fa2aa5640 1 --2- 172.21.15.33:0/1412558577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5f9c107ab0 0x7f5f9c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.543+0000 7f5fa2aa5640 1 --2- 172.21.15.33:0/1412558577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f9c102460 0x7f5f9c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.543+0000 7f5fa2aa5640 1 --2- 172.21.15.33:0/1412558577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f9c101aa0 0x7f5f9c198320 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.543+0000 7f5fa2aa5640 1 -- 172.21.15.33:0/1412558577 >> 172.21.15.33:0/1412558577 conn(0x7f5f9c0fd530 msgr2=0x7f5f9c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.543+0000 7f5fa2aa5640 1 -- 172.21.15.33:0/1412558577 shutdown_connections 2024-04-02T13:43:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.543+0000 7f5fa2aa5640 1 -- 172.21.15.33:0/1412558577 wait complete. 2024-04-02T13:43:12.559 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:43:12.560 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:43:12.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.701+0000 7efd83a86640 1 -- 172.21.15.33:0/1722839312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7c101380 msgr2=0x7efd7c10d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.701+0000 7efd83a86640 1 --2- 172.21.15.33:0/1722839312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7c101380 0x7efd7c10d890 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efd6c009cd0 tx=0x7efd6c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.701+0000 7efd83a86640 1 -- 172.21.15.33:0/1722839312 shutdown_connections 2024-04-02T13:43:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.701+0000 7efd83a86640 1 --2- 172.21.15.33:0/1722839312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7c101380 0x7efd7c10d890 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.701+0000 7efd83a86640 1 --2- 172.21.15.33:0/1722839312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd7c1009c0 0x7efd7c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.701+0000 7efd83a86640 1 --2- 172.21.15.33:0/1722839312 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd7c1069d0 0x7efd7c106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.701+0000 7efd83a86640 1 -- 172.21.15.33:0/1722839312 >> 172.21.15.33:0/1722839312 conn(0x7efd7c077aa0 msgr2=0x7efd7c077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.701+0000 7efd83a86640 1 -- 172.21.15.33:0/1722839312 shutdown_connections 2024-04-02T13:43:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.701+0000 7efd83a86640 1 -- 172.21.15.33:0/1722839312 wait complete. 2024-04-02T13:43:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.701+0000 7efd83a86640 1 Processor -- start 2024-04-02T13:43:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd83a86640 1 -- start start 2024-04-02T13:43:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd83a86640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd7c1009c0 0x7efd7c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd83a86640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7c101380 0x7efd7c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd83a86640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd7c1069d0 0x7efd7c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd83a86640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c110db0 con 0x7efd7c101380 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd83a86640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c110c30 con 0x7efd7c1009c0 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd83a86640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c110f30 con 0x7efd7c1069d0 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd81021640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7c101380 0x7efd7c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd81021640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7c101380 0x7efd7c198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49012/0 (socket says 172.21.15.33:49012) 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd81021640 1 -- 172.21.15.33:0/4081072444 learned_addr learned my addr 172.21.15.33:0/4081072444 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd82023640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd7c1069d0 0x7efd7c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd81822640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd7c1009c0 0x7efd7c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd81021640 1 -- 172.21.15.33:0/4081072444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd7c1069d0 msgr2=0x7efd7c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd81021640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd7c1069d0 0x7efd7c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd81021640 1 -- 172.21.15.33:0/4081072444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd7c1009c0 msgr2=0x7efd7c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd81021640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd7c1009c0 0x7efd7c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd81021640 1 -- 172.21.15.33:0/4081072444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c19d2f0 con 0x7efd7c101380 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd81822640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd7c1009c0 0x7efd7c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd81021640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7c101380 0x7efd7c198840 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c920 tx=0x7efd7000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd7affd640 1 -- 172.21.15.33:0/4081072444 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70007c70 con 0x7efd7c101380 2024-04-02T13:43:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd7affd640 1 -- 172.21.15.33:0/4081072444 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efd70007e10 con 0x7efd7c101380 2024-04-02T13:43:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.702+0000 7efd83a86640 1 -- 172.21.15.33:0/4081072444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c19d5e0 con 0x7efd7c101380 2024-04-02T13:43:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.703+0000 7efd7affd640 1 -- 172.21.15.33:0/4081072444 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70018850 con 0x7efd7c101380 2024-04-02T13:43:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.703+0000 7efd83a86640 1 -- 172.21.15.33:0/4081072444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c1a4e70 con 0x7efd7c101380 2024-04-02T13:43:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.703+0000 7efd83a86640 1 -- 172.21.15.33:0/4081072444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7c068d40 con 0x7efd7c101380 2024-04-02T13:43:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.705+0000 7efd7affd640 1 -- 172.21.15.33:0/4081072444 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efd70018a10 con 0x7efd7c101380 2024-04-02T13:43:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.705+0000 7efd7affd640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efd640a3d20 0x7efd640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.705+0000 7efd81822640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efd640a3d20 0x7efd640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.705+0000 7efd7affd640 1 -- 172.21.15.33:0/4081072444 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7efd700d0f20 con 0x7efd7c101380 2024-04-02T13:43:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.706+0000 7efd81822640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efd640a3d20 0x7efd640a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efd74005fd0 tx=0x7efd74005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.709+0000 7efd7affd640 1 -- 172.21.15.33:0/4081072444 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efd70014030 con 0x7efd7c101380 2024-04-02T13:43:12.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.936+0000 7efd83a86640 1 -- 172.21.15.33:0/4081072444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd7c02d370 con 0x7efd7c101380 2024-04-02T13:43:12.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.937+0000 7efd7affd640 1 -- 172.21.15.33:0/4081072444 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11882 (secure 0 0 0) 0x7efd7009d2e0 con 0x7efd7c101380 2024-04-02T13:43:12.941 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:43:12.941 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:41:39.222860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:43:12.941 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-02T13:43:12.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.939+0000 7efd83a86640 1 -- 172.21.15.33:0/4081072444 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efd640a3d20 msgr2=0x7efd640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.939+0000 7efd83a86640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efd640a3d20 0x7efd640a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efd74005fd0 tx=0x7efd74005850 comp rx=0 tx=0).stop 2024-04-02T13:43:12.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.939+0000 7efd83a86640 1 -- 172.21.15.33:0/4081072444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7c101380 msgr2=0x7efd7c198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:12.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.939+0000 7efd83a86640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7c101380 0x7efd7c198840 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c920 tx=0x7efd7000cde0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.939+0000 7efd83a86640 1 -- 172.21.15.33:0/4081072444 shutdown_connections 2024-04-02T13:43:12.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.939+0000 7efd83a86640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efd640a3d20 0x7efd640a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.939+0000 7efd83a86640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd7c1069d0 0x7efd7c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.939+0000 7efd83a86640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd7c101380 0x7efd7c198840 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.939+0000 7efd83a86640 1 --2- 172.21.15.33:0/4081072444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd7c1009c0 0x7efd7c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:12.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.939+0000 7efd83a86640 1 -- 172.21.15.33:0/4081072444 >> 172.21.15.33:0/4081072444 conn(0x7efd7c077aa0 msgr2=0x7efd7c06a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:12.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.939+0000 7efd83a86640 1 -- 172.21.15.33:0/4081072444 shutdown_connections 2024-04-02T13:43:12.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:12.939+0000 7efd83a86640 1 -- 172.21.15.33:0/4081072444 wait complete. 2024-04-02T13:43:12.953 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:43:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.056+0000 7f5b72a55640 1 -- 172.21.15.33:0/3168111585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b6c105130 msgr2=0x7f5b6c105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.056+0000 7f5b72a55640 1 --2- 172.21.15.33:0/3168111585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b6c105130 0x7f5b6c105510 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00aa10 tx=0x7f5b5c02cf70 comp rx=0 tx=0).stop 2024-04-02T13:43:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.056+0000 7f5b72a55640 1 -- 172.21.15.33:0/3168111585 shutdown_connections 2024-04-02T13:43:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.056+0000 7f5b72a55640 1 --2- 172.21.15.33:0/3168111585 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b6c100d60 0x7f5b6c10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.056+0000 7f5b72a55640 1 --2- 172.21.15.33:0/3168111585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b6c1003c0 0x7f5b6c100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.056+0000 7f5b72a55640 1 --2- 172.21.15.33:0/3168111585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b6c105130 0x7f5b6c105510 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.056+0000 7f5b72a55640 1 -- 172.21.15.33:0/3168111585 >> 172.21.15.33:0/3168111585 conn(0x7f5b6c075d40 msgr2=0x7f5b6c076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.056+0000 7f5b72a55640 1 -- 172.21.15.33:0/3168111585 shutdown_connections 2024-04-02T13:43:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.056+0000 7f5b72a55640 1 -- 172.21.15.33:0/3168111585 wait complete. 2024-04-02T13:43:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b72a55640 1 Processor -- start 2024-04-02T13:43:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b72a55640 1 -- start start 2024-04-02T13:43:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b72a55640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b6c1003c0 0x7f5b6c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b72a55640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b6c100d60 0x7f5b6c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b72a55640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b6c105130 0x7f5b6c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b72a55640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10fcb0 con 0x7f5b6c100d60 2024-04-02T13:43:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b72a55640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10fb30 con 0x7f5b6c105130 2024-04-02T13:43:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b72a55640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10fe30 con 0x7f5b6c1003c0 2024-04-02T13:43:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b6c100d60 0x7f5b6c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b6c100d60 0x7f5b6c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49030/0 (socket says 172.21.15.33:49030) 2024-04-02T13:43:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b6b7fe640 1 -- 172.21.15.33:0/4056381483 learned_addr learned my addr 172.21.15.33:0/4056381483 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b70ff2640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b6c105130 0x7f5b6c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b6bfff640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b6c1003c0 0x7f5b6c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b6b7fe640 1 -- 172.21.15.33:0/4056381483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b6c1003c0 msgr2=0x7f5b6c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b6b7fe640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b6c1003c0 0x7f5b6c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b6b7fe640 1 -- 172.21.15.33:0/4056381483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b6c105130 msgr2=0x7f5b6c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b6b7fe640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b6c105130 0x7f5b6c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b6b7fe640 1 -- 172.21.15.33:0/4056381483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c19d3c0 con 0x7f5b6c100d60 2024-04-02T13:43:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.057+0000 7f5b6bfff640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b6c1003c0 0x7f5b6c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.058+0000 7f5b6b7fe640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b6c100d60 0x7f5b6c198910 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000c890 tx=0x7f5b6000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.058+0000 7f5b697fa640 1 -- 172.21.15.33:0/4056381483 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60004220 con 0x7f5b6c100d60 2024-04-02T13:43:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.058+0000 7f5b697fa640 1 -- 172.21.15.33:0/4056381483 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5b600043c0 con 0x7f5b6c100d60 2024-04-02T13:43:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.058+0000 7f5b72a55640 1 -- 172.21.15.33:0/4056381483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c19d6b0 con 0x7f5b6c100d60 2024-04-02T13:43:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.058+0000 7f5b697fa640 1 -- 172.21.15.33:0/4056381483 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60004a00 con 0x7f5b6c100d60 2024-04-02T13:43:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.058+0000 7f5b72a55640 1 -- 172.21.15.33:0/4056381483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c1a4f40 con 0x7f5b6c100d60 2024-04-02T13:43:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b697fa640 1 -- 172.21.15.33:0/4056381483 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5b60004ba0 con 0x7f5b6c100d60 2024-04-02T13:43:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b75638740 1 Processor -- start 2024-04-02T13:43:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b75638740 1 -- start start 2024-04-02T13:43:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b75638740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887887a0 0x562788788b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b75638740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627887890c0 0x562788791760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b75638740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562788791ca0 0x562788794090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b75638740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5627886ec7b0 con 0x5627887887a0 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b75638740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56278868b570 con 0x5627887890c0 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b75638740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5627886e8760 con 0x562788791ca0 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b70ff2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887887a0 0x562788788b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b70ff2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887887a0 0x562788788b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49046/0 (socket says 172.21.15.33:49046) 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b70ff2640 1 -- 172.21.15.33:0/3497750939 learned_addr learned my addr 172.21.15.33:0/3497750939 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b697fa640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5b540a3d20 0x7f5b540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b6bfff640 1 --2- 172.21.15.33:0/3497750939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627887890c0 0x562788791760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b70ff2640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5b540a3d20 0x7f5b540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b697fa640 1 -- 172.21.15.33:0/4056381483 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5b600d08a0 con 0x7f5b6c100d60 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b6b7fe640 1 --2- 172.21.15.33:0/3497750939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562788791ca0 0x562788794090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b70ff2640 1 -- 172.21.15.33:0/3497750939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562788791ca0 msgr2=0x562788794090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b70ff2640 1 --2- 172.21.15.33:0/3497750939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562788791ca0 0x562788794090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.061+0000 7f5b70ff2640 1 -- 172.21.15.33:0/3497750939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627887890c0 msgr2=0x562788791760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.062+0000 7f5b70ff2640 1 --2- 172.21.15.33:0/3497750939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627887890c0 0x562788791760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.062+0000 7f5b70ff2640 1 -- 172.21.15.33:0/3497750939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627884003f0 con 0x5627887887a0 2024-04-02T13:43:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.062+0000 7f5b6b7fe640 1 --2- 172.21.15.33:0/3497750939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562788791ca0 0x562788794090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.062+0000 7f5b6bfff640 1 --2- 172.21.15.33:0/3497750939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627887890c0 0x562788791760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.062+0000 7f5b70ff2640 1 --2- 172.21.15.33:0/3497750939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887887a0 0x562788788b80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5b64013940 tx=0x7f5b64013e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863afe38fd26fcb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.062+0000 7f5b527fc640 1 -- 172.21.15.33:0/3497750939 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64021030 con 0x5627887887a0 2024-04-02T13:43:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.062+0000 7f5b527fc640 1 -- 172.21.15.33:0/3497750939 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5b64003ec0 con 0x5627887887a0 2024-04-02T13:43:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.062+0000 7f5b527fc640 1 -- 172.21.15.33:0/3497750939 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64012650 con 0x5627887887a0 2024-04-02T13:43:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 -- 172.21.15.33:0/3497750939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887887a0 msgr2=0x562788788b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b70ff2640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5b540a3d20 0x7f5b540a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5b64021000 tx=0x7f5b64020000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 --2- 172.21.15.33:0/3497750939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887887a0 0x562788788b80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5b64013940 tx=0x7f5b64013e00 comp rx=0 tx=0).stop 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 -- 172.21.15.33:0/3497750939 shutdown_connections 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 --2- 172.21.15.33:0/3497750939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562788791ca0 0x562788794090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 --2- 172.21.15.33:0/3497750939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627887890c0 0x562788791760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 --2- 172.21.15.33:0/3497750939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887887a0 0x562788788b80 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 -- 172.21.15.33:0/3497750939 >> 172.21.15.33:0/3497750939 conn(0x562788782ed0 msgr2=0x5627887832b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 -- 172.21.15.33:0/3497750939 shutdown_connections 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 -- 172.21.15.33:0/3497750939 wait complete. 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 -- start start 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 Processor -- start 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887887a0 0x562788791750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 0x56278878eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562788791ca0 0x56278878f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562788794a90 con 0x5627887890c0 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5627886a7e80 con 0x562788791ca0 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562788794c10 con 0x5627887887a0 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.063+0000 7f5b75638740 10 client.0 _ll_register_callbacks cb 0x5627886e3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 0x56278878eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 0x56278878eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49062/0 (socket says 172.21.15.33:49062) 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b6b7fe640 1 -- 172.21.15.33:0/1952957003 learned_addr learned my addr 172.21.15.33:0/1952957003 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b70ff2640 1 --2- 172.21.15.33:0/1952957003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562788791ca0 0x56278878f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b6bfff640 1 --2- 172.21.15.33:0/1952957003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887887a0 0x562788791750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b6b7fe640 1 -- 172.21.15.33:0/1952957003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887887a0 msgr2=0x562788791750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b6b7fe640 1 --2- 172.21.15.33:0/1952957003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887887a0 0x562788791750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b6b7fe640 1 -- 172.21.15.33:0/1952957003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562788791ca0 msgr2=0x56278878f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b6b7fe640 1 --2- 172.21.15.33:0/1952957003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562788791ca0 0x56278878f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b6b7fe640 1 -- 172.21.15.33:0/1952957003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627884ca7d0 con 0x5627887890c0 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b6b7fe640 1 --2- 172.21.15.33:0/1952957003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 0x56278878eae0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005fd0 tx=0x7f5b6009bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9a81fa932e1a29 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b4b7fe640 1 -- 172.21.15.33:0/1952957003 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6001d030 con 0x5627887890c0 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b4b7fe640 1 -- 172.21.15.33:0/1952957003 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5b600cdf00 con 0x5627887890c0 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b75638740 1 -- 172.21.15.33:0/1952957003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b64003ec0 con 0x5627887890c0 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b4b7fe640 1 -- 172.21.15.33:0/1952957003 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b600ce100 con 0x5627887890c0 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b75638740 10 client.16563 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b75638740 1 -- 172.21.15.33:0/1952957003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627887a0ab0 con 0x5627887890c0 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.064+0000 7f5b4b7fe640 1 -- 172.21.15.33:0/1952957003 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5b64003ec0 con 0x5627887890c0 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.065+0000 7f5b4b7fe640 1 -- 172.21.15.33:0/1952957003 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5b600bb150 con 0x5627887890c0 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.065+0000 7f5b75638740 10 client.16563 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.065+0000 7f5b75638740 10 client.16563 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.065+0000 7f5b75638740 20 client.16563 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.065+0000 7f5b75638740 1 --2- 172.21.15.33:0/1952957003 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5627887a1f10 0x5627887a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.065+0000 7f5b75638740 4 client.16563 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.065+0000 7f5b75638740 1 -- 172.21.15.33:0/1952957003 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627886f9cf0 con 0x5627887a1f10 2024-04-02T13:43:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.065+0000 7f5b6bfff640 1 --2- 172.21.15.33:0/1952957003 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5627887a1f10 0x5627887a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b6bfff640 1 --2- 172.21.15.33:0/1952957003 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5627887a1f10 0x5627887a4350 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f40010b49094fe18 server_cookie=4cee7c0d466bd844 in_seq=0 out_seq=0 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b4b7fe640 10 client.16563 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b4b7fe640 1 -- 172.21.15.33:0/1952957003 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5627886f9cf0 con 0x5627887a1f10 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b4b7fe640 10 client.16563 handle_command_reply: tid=0 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b75638740 1 client.16563 shutdown 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b75638740 1 -- 172.21.15.33:0/1952957003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 msgr2=0x56278878eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b75638740 1 --2- 172.21.15.33:0/1952957003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 0x56278878eae0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005fd0 tx=0x7f5b6009bef0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b75638740 1 -- 172.21.15.33:0/1952957003 shutdown_connections 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b75638740 1 --2- 172.21.15.33:0/1952957003 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5627887a1f10 0x5627887a4350 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b75638740 1 --2- 172.21.15.33:0/1952957003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562788791ca0 0x56278878f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b75638740 1 --2- 172.21.15.33:0/1952957003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 0x56278878eae0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b4b7fe640 0 client.16563 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b75638740 1 --2- 172.21.15.33:0/1952957003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887887a0 0x562788791750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.066+0000 7f5b75638740 1 -- 172.21.15.33:0/1952957003 >> 172.21.15.33:0/1952957003 conn(0x562788782ed0 msgr2=0x562788784c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.067+0000 7f5b75638740 1 -- 172.21.15.33:0/1952957003 shutdown_connections 2024-04-02T13:43:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.067+0000 7f5b75638740 1 -- 172.21.15.33:0/1952957003 wait complete. 2024-04-02T13:43:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.067+0000 7f5b75638740 20 client.16563 trim_cache size 0 max 16384 2024-04-02T13:43:13.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b75638740 1 Processor -- start 2024-04-02T13:43:13.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b75638740 1 -- start start 2024-04-02T13:43:13.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b75638740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887887a0 0x56278878ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b75638740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 0x56278879d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b75638740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562788791ca0 0x562788791210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b75638740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1b1990 con 0x5627887890c0 2024-04-02T13:43:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b75638740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1af1a0 con 0x562788791ca0 2024-04-02T13:43:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b75638740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1acfd0 con 0x5627887887a0 2024-04-02T13:43:13.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 0x56278879d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 0x56278879d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49074/0 (socket says 172.21.15.33:49074) 2024-04-02T13:43:13.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b6b7fe640 1 -- 172.21.15.33:0/2519434111 learned_addr learned my addr 172.21.15.33:0/2519434111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b70ff2640 1 --2- 172.21.15.33:0/2519434111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562788791ca0 0x562788791210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b6bfff640 1 --2- 172.21.15.33:0/2519434111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887887a0 0x56278878ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b6b7fe640 1 -- 172.21.15.33:0/2519434111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887887a0 msgr2=0x56278878ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b6b7fe640 1 --2- 172.21.15.33:0/2519434111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887887a0 0x56278878ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b6b7fe640 1 -- 172.21.15.33:0/2519434111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562788791ca0 msgr2=0x562788791210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b6b7fe640 1 --2- 172.21.15.33:0/2519434111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562788791ca0 0x562788791210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b6b7fe640 1 -- 172.21.15.33:0/2519434111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562788788ef0 con 0x5627887890c0 2024-04-02T13:43:13.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b70ff2640 1 --2- 172.21.15.33:0/2519434111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562788791ca0 0x562788791210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:13.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.080+0000 7f5b6bfff640 1 --2- 172.21.15.33:0/2519434111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887887a0 0x56278878ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:13.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.081+0000 7f5b6b7fe640 1 --2- 172.21.15.33:0/2519434111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 0x56278879d950 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5b6009a780 tx=0x7f5b6009bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c798fcd6fd754b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.081+0000 7f5b51ffb640 1 -- 172.21.15.33:0/2519434111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60005140 con 0x5627887890c0 2024-04-02T13:43:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.081+0000 7f5b51ffb640 1 -- 172.21.15.33:0/2519434111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5b600bb180 con 0x5627887890c0 2024-04-02T13:43:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.081+0000 7f5b51ffb640 1 -- 172.21.15.33:0/2519434111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b600cdf00 con 0x5627887890c0 2024-04-02T13:43:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.081+0000 7f5b75638740 1 -- 172.21.15.33:0/2519434111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 msgr2=0x56278879d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.081+0000 7f5b75638740 1 --2- 172.21.15.33:0/2519434111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 0x56278879d950 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5b6009a780 tx=0x7f5b6009bf60 comp rx=0 tx=0).stop 2024-04-02T13:43:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.081+0000 7f5b75638740 1 -- 172.21.15.33:0/2519434111 shutdown_connections 2024-04-02T13:43:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.081+0000 7f5b75638740 1 --2- 172.21.15.33:0/2519434111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562788791ca0 0x562788791210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.081+0000 7f5b75638740 1 --2- 172.21.15.33:0/2519434111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627887890c0 0x56278879d950 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.081+0000 7f5b75638740 1 --2- 172.21.15.33:0/2519434111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887887a0 0x56278878ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.081+0000 7f5b75638740 1 -- 172.21.15.33:0/2519434111 >> 172.21.15.33:0/2519434111 conn(0x562788782ed0 msgr2=0x56278878e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.081+0000 7f5b75638740 1 -- 172.21.15.33:0/2519434111 shutdown_connections 2024-04-02T13:43:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.081+0000 7f5b75638740 1 -- 172.21.15.33:0/2519434111 wait complete. 2024-04-02T13:43:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b75638740 1 -- start start 2024-04-02T13:43:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b75638740 1 Processor -- start 2024-04-02T13:43:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b75638740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627887887a0 0x56278878c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b75638740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887890c0 0x562788799e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b75638740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562788791ca0 0x56278879a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b75638740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1ad660 con 0x562788791ca0 2024-04-02T13:43:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b75638740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1aead0 con 0x5627887887a0 2024-04-02T13:43:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b75638740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1add30 con 0x5627887890c0 2024-04-02T13:43:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b75638740 10 client.0 _ll_register_callbacks cb 0x7f5b60005410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b70ff2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562788791ca0 0x56278879a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b70ff2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562788791ca0 0x56278879a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49076/0 (socket says 172.21.15.33:49076) 2024-04-02T13:43:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b70ff2640 1 -- 172.21.15.33:0/1937191000 learned_addr learned my addr 172.21.15.33:0/1937191000 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b6bfff640 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627887887a0 0x56278878c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b6b7fe640 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887890c0 0x562788799e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b70ff2640 1 -- 172.21.15.33:0/1937191000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887890c0 msgr2=0x562788799e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.082+0000 7f5b70ff2640 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887890c0 0x562788799e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b70ff2640 1 -- 172.21.15.33:0/1937191000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627887887a0 msgr2=0x56278878c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b70ff2640 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627887887a0 0x56278878c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b70ff2640 1 -- 172.21.15.33:0/1937191000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562788793090 con 0x562788791ca0 2024-04-02T13:43:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b6bfff640 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627887887a0 0x56278878c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b70ff2640 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562788791ca0 0x56278879a3e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5b64000f80 tx=0x7f5b64013300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b653c11190163d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b4b7fe640 1 -- 172.21.15.33:0/1937191000 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64004c00 con 0x562788791ca0 2024-04-02T13:43:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b4b7fe640 1 -- 172.21.15.33:0/1937191000 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5b64004da0 con 0x562788791ca0 2024-04-02T13:43:13.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b75638740 1 -- 172.21.15.33:0/1937191000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b600bb180 con 0x562788791ca0 2024-04-02T13:43:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b4b7fe640 1 -- 172.21.15.33:0/1937191000 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b6403c050 con 0x562788791ca0 2024-04-02T13:43:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b75638740 10 client.16569 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b75638740 1 -- 172.21.15.33:0/1937191000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627888089f0 con 0x562788791ca0 2024-04-02T13:43:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b4b7fe640 1 -- 172.21.15.33:0/1937191000 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5b600bb180 con 0x562788791ca0 2024-04-02T13:43:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b4b7fe640 1 -- 172.21.15.33:0/1937191000 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5b64004390 con 0x562788791ca0 2024-04-02T13:43:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b75638740 10 client.16569 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b75638740 10 client.16569 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:43:13.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b75638740 20 client.16569 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:13.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b75638740 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x562788808f10 0x56278880b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b75638740 4 client.16569 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:13.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.083+0000 7f5b75638740 1 -- 172.21.15.33:0/1937191000 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627886f4040 con 0x562788808f10 2024-04-02T13:43:13.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.084+0000 7f5b6bfff640 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x562788808f10 0x56278880b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.084+0000 7f5b4b7fe640 10 client.16569 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:43:13.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.084+0000 7f5b6bfff640 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x562788808f10 0x56278880b350 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d196432779d00764 server_cookie=ba5b7e95303be410 in_seq=0 out_seq=0 2024-04-02T13:43:13.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.084+0000 7f5b4b7fe640 1 -- 172.21.15.33:0/1937191000 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627886f4040 con 0x562788808f10 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.084+0000 7f5b4b7fe640 10 client.16569 handle_command_reply: tid=0 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.084+0000 7f5b75638740 1 client.16569 shutdown 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.084+0000 7f5b75638740 1 -- 172.21.15.33:0/1937191000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562788791ca0 msgr2=0x56278879a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.084+0000 7f5b75638740 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562788791ca0 0x56278879a3e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5b64000f80 tx=0x7f5b64013300 comp rx=0 tx=0).stop 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b75638740 1 -- 172.21.15.33:0/1937191000 shutdown_connections 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b75638740 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x562788808f10 0x56278880b350 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b75638740 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562788791ca0 0x56278879a3e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b75638740 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5627887890c0 0x562788799e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b4b7fe640 0 client.16569 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b75638740 1 --2- 172.21.15.33:0/1937191000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627887887a0 0x56278878c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b75638740 1 -- 172.21.15.33:0/1937191000 >> 172.21.15.33:0/1937191000 conn(0x562788782ed0 msgr2=0x5627887842b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b75638740 1 -- 172.21.15.33:0/1937191000 shutdown_connections 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b75638740 1 -- 172.21.15.33:0/1937191000 wait complete. 2024-04-02T13:43:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b75638740 20 client.16569 trim_cache size 0 max 16384 2024-04-02T13:43:13.109 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:43:13.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b72a55640 1 -- 172.21.15.33:0/4056381483 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5b540a3d20 msgr2=0x7f5b540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b72a55640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5b540a3d20 0x7f5b540a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5b64021000 tx=0x7f5b64020000 comp rx=0 tx=0).stop 2024-04-02T13:43:13.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b72a55640 1 -- 172.21.15.33:0/4056381483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b6c100d60 msgr2=0x7f5b6c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b72a55640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b6c100d60 0x7f5b6c198910 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000c890 tx=0x7f5b6000cd50 comp rx=0 tx=0).stop 2024-04-02T13:43:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b72a55640 1 -- 172.21.15.33:0/4056381483 shutdown_connections 2024-04-02T13:43:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b72a55640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5b540a3d20 0x7f5b540a61e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b72a55640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b6c105130 0x7f5b6c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b72a55640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b6c100d60 0x7f5b6c198910 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b72a55640 1 --2- 172.21.15.33:0/4056381483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b6c1003c0 0x7f5b6c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b72a55640 1 -- 172.21.15.33:0/4056381483 >> 172.21.15.33:0/4056381483 conn(0x7f5b6c075d40 msgr2=0x7f5b6c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b72a55640 1 -- 172.21.15.33:0/4056381483 shutdown_connections 2024-04-02T13:43:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.085+0000 7f5b72a55640 1 -- 172.21.15.33:0/4056381483 wait complete. 2024-04-02T13:43:13.111 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:43:13.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.215+0000 7fe36b150640 1 -- 172.21.15.33:0/1913808901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe364100d80 msgr2=0x7fe36410e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.215+0000 7fe36b150640 1 --2- 172.21.15.33:0/1913808901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe364100d80 0x7fe36410e980 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009cd0 tx=0x7fe35c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.215+0000 7fe36b150640 1 -- 172.21.15.33:0/1913808901 shutdown_connections 2024-04-02T13:43:13.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.215+0000 7fe36b150640 1 --2- 172.21.15.33:0/1913808901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe364100d80 0x7fe36410e980 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.215+0000 7fe36b150640 1 --2- 172.21.15.33:0/1913808901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe3641003c0 0x7fe364100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.215+0000 7fe36b150640 1 --2- 172.21.15.33:0/1913808901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe364107d00 0x7fe3641080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.215+0000 7fe36b150640 1 -- 172.21.15.33:0/1913808901 >> 172.21.15.33:0/1913808901 conn(0x7fe364075d40 msgr2=0x7fe364076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.215+0000 7fe36b150640 1 -- 172.21.15.33:0/1913808901 shutdown_connections 2024-04-02T13:43:13.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.215+0000 7fe36b150640 1 -- 172.21.15.33:0/1913808901 wait complete. 2024-04-02T13:43:13.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe36b150640 1 Processor -- start 2024-04-02T13:43:13.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe36b150640 1 -- start start 2024-04-02T13:43:13.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe36b150640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe3641003c0 0x7fe36419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe36b150640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe364100d80 0x7fe36419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe36b150640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe364107d00 0x7fe36419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe36b150640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe364111f30 con 0x7fe364100d80 2024-04-02T13:43:13.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe36b150640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe364111db0 con 0x7fe364107d00 2024-04-02T13:43:13.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe36b150640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe3641120b0 con 0x7fe3641003c0 2024-04-02T13:43:13.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe35bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe364100d80 0x7fe36419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe35bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe364100d80 0x7fe36419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49092/0 (socket says 172.21.15.33:49092) 2024-04-02T13:43:13.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe35bfff640 1 -- 172.21.15.33:0/1463194305 learned_addr learned my addr 172.21.15.33:0/1463194305 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe368eec640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe3641003c0 0x7fe36419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe3696ed640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe364107d00 0x7fe36419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe35bfff640 1 -- 172.21.15.33:0/1463194305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe3641003c0 msgr2=0x7fe36419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe35bfff640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe3641003c0 0x7fe36419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe35bfff640 1 -- 172.21.15.33:0/1463194305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe364107d00 msgr2=0x7fe36419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe35bfff640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe364107d00 0x7fe36419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe35bfff640 1 -- 172.21.15.33:0/1463194305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36419f510 con 0x7fe364100d80 2024-04-02T13:43:13.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe368eec640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe3641003c0 0x7fe36419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:13.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.216+0000 7fe3696ed640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe364107d00 0x7fe36419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:13.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.217+0000 7fe35bfff640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe364100d80 0x7fe36419aa60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe35400c970 tx=0x7fe35400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.217+0000 7fe359ffb640 1 -- 172.21.15.33:0/1463194305 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354007bb0 con 0x7fe364100d80 2024-04-02T13:43:13.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.217+0000 7fe359ffb640 1 -- 172.21.15.33:0/1463194305 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe354007d50 con 0x7fe364100d80 2024-04-02T13:43:13.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.217+0000 7fe36b150640 1 -- 172.21.15.33:0/1463194305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36419f800 con 0x7fe364100d80 2024-04-02T13:43:13.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.217+0000 7fe359ffb640 1 -- 172.21.15.33:0/1463194305 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35400f450 con 0x7fe364100d80 2024-04-02T13:43:13.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.217+0000 7fe36b150640 1 -- 172.21.15.33:0/1463194305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3641a7090 con 0x7fe364100d80 2024-04-02T13:43:13.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.219+0000 7fe359ffb640 1 -- 172.21.15.33:0/1463194305 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe354004fb0 con 0x7fe364100d80 2024-04-02T13:43:13.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.219+0000 7fe36de10740 1 Processor -- start 2024-04-02T13:43:13.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.219+0000 7fe36de10740 1 -- start start 2024-04-02T13:43:13.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe359ffb640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe34c0a3d20 0x7fe34c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe36de10740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24047a0 0x55cee2404b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe36de10740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 0x55cee240d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe36de10740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee240dca0 0x55cee2410090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe36de10740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cee23687b0 con 0x55cee24050c0 2024-04-02T13:43:13.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe36de10740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55cee2307570 con 0x55cee24047a0 2024-04-02T13:43:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe36de10740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55cee2364760 con 0x55cee240dca0 2024-04-02T13:43:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe35bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 0x55cee240d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe3696ed640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe34c0a3d20 0x7fe34c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe35bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 0x55cee240d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49096/0 (socket says 172.21.15.33:49096) 2024-04-02T13:43:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe359ffb640 1 -- 172.21.15.33:0/1463194305 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe3540d0990 con 0x7fe364100d80 2024-04-02T13:43:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe35bfff640 1 -- 172.21.15.33:0/2785301886 learned_addr learned my addr 172.21.15.33:0/2785301886 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe368eec640 1 --2- 172.21.15.33:0/2785301886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24047a0 0x55cee2404b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe3696ed640 1 --2- 172.21.15.33:0/2785301886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee240dca0 0x55cee2410090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe35bfff640 1 -- 172.21.15.33:0/2785301886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee240dca0 msgr2=0x55cee2410090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe35bfff640 1 --2- 172.21.15.33:0/2785301886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee240dca0 0x55cee2410090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe35bfff640 1 -- 172.21.15.33:0/2785301886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24047a0 msgr2=0x55cee2404b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe35bfff640 1 --2- 172.21.15.33:0/2785301886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24047a0 0x55cee2404b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe35bfff640 1 -- 172.21.15.33:0/2785301886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cee207c3f0 con 0x55cee24050c0 2024-04-02T13:43:13.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe35bfff640 1 --2- 172.21.15.33:0/2785301886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 0x55cee240d760 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe3540058c0 tx=0x7fe35401baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4936a82e2213d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe3696ed640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe34c0a3d20 0x7fe34c0a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe35c037040 tx=0x7fe35c032c90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.220+0000 7fe34affd640 1 -- 172.21.15.33:0/2785301886 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35409ccc0 con 0x55cee24050c0 2024-04-02T13:43:13.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.221+0000 7fe34affd640 1 -- 172.21.15.33:0/2785301886 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe35409c5c0 con 0x55cee24050c0 2024-04-02T13:43:13.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.221+0000 7fe34affd640 1 -- 172.21.15.33:0/2785301886 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3540b6100 con 0x55cee24050c0 2024-04-02T13:43:13.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.221+0000 7fe36de10740 1 -- 172.21.15.33:0/2785301886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 msgr2=0x55cee240d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.221+0000 7fe36de10740 1 --2- 172.21.15.33:0/2785301886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 0x55cee240d760 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe3540058c0 tx=0x7fe35401baf0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.221+0000 7fe36de10740 1 -- 172.21.15.33:0/2785301886 shutdown_connections 2024-04-02T13:43:13.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.221+0000 7fe36de10740 1 --2- 172.21.15.33:0/2785301886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee240dca0 0x55cee2410090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.221+0000 7fe36de10740 1 --2- 172.21.15.33:0/2785301886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 0x55cee240d760 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.221+0000 7fe36de10740 1 --2- 172.21.15.33:0/2785301886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24047a0 0x55cee2404b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.221+0000 7fe36de10740 1 -- 172.21.15.33:0/2785301886 >> 172.21.15.33:0/2785301886 conn(0x55cee23feed0 msgr2=0x55cee23ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.221+0000 7fe36de10740 1 -- 172.21.15.33:0/2785301886 shutdown_connections 2024-04-02T13:43:13.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.221+0000 7fe36de10740 1 -- 172.21.15.33:0/2785301886 wait complete. 2024-04-02T13:43:13.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.221+0000 7fe36de10740 1 -- start start 2024-04-02T13:43:13.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe36de10740 1 Processor -- start 2024-04-02T13:43:13.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe36de10740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24047a0 0x55cee240ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe36de10740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24050c0 0x55cee240b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe36de10740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 0x55cee240b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe36de10740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cee2410a30 con 0x55cee240dca0 2024-04-02T13:43:13.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe36de10740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55cee2323e80 con 0x55cee24050c0 2024-04-02T13:43:13.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe36de10740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55cee2410bb0 con 0x55cee24047a0 2024-04-02T13:43:13.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe36de10740 10 client.0 _ll_register_callbacks cb 0x55cee235faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:13.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe3696ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 0x55cee240b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe3696ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 0x55cee240b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49098/0 (socket says 172.21.15.33:49098) 2024-04-02T13:43:13.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe3696ed640 1 -- 172.21.15.33:0/3585879285 learned_addr learned my addr 172.21.15.33:0/3585879285 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe35bfff640 1 --2- 172.21.15.33:0/3585879285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24050c0 0x55cee240b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe368eec640 1 --2- 172.21.15.33:0/3585879285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24047a0 0x55cee240ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe3696ed640 1 -- 172.21.15.33:0/3585879285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24047a0 msgr2=0x55cee240ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe3696ed640 1 --2- 172.21.15.33:0/3585879285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24047a0 0x55cee240ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe3696ed640 1 -- 172.21.15.33:0/3585879285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24050c0 msgr2=0x55cee240b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe3696ed640 1 --2- 172.21.15.33:0/3585879285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24050c0 0x55cee240b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.222+0000 7fe3696ed640 1 -- 172.21.15.33:0/3585879285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cee21467d0 con 0x55cee240dca0 2024-04-02T13:43:13.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe3696ed640 1 --2- 172.21.15.33:0/3585879285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 0x55cee240b590 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe35c036000 tx=0x7fe35c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877e7b3fe3a3b707 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe343fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:13.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe343fff640 1 -- 172.21.15.33:0/3585879285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c051060 con 0x55cee240dca0 2024-04-02T13:43:13.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe343fff640 1 -- 172.21.15.33:0/3585879285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe35c004240 con 0x55cee240dca0 2024-04-02T13:43:13.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe36de10740 1 -- 172.21.15.33:0/3585879285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe35409c5c0 con 0x55cee240dca0 2024-04-02T13:43:13.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe343fff640 1 -- 172.21.15.33:0/3585879285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe35c058050 con 0x55cee240dca0 2024-04-02T13:43:13.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe36de10740 10 client.16581 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:13.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe36de10740 1 -- 172.21.15.33:0/3585879285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cee241cb20 con 0x55cee240dca0 2024-04-02T13:43:13.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe343fff640 1 -- 172.21.15.33:0/3585879285 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe35409c5c0 con 0x55cee240dca0 2024-04-02T13:43:13.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe343fff640 1 -- 172.21.15.33:0/3585879285 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe35c034b10 con 0x55cee240dca0 2024-04-02T13:43:13.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe36de10740 10 client.16581 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:13.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe36de10740 10 client.16581 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:43:13.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe36de10740 20 client.16581 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:13.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe36de10740 1 --2- 172.21.15.33:0/3585879285 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55cee241dfe0 0x55cee2420440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe36de10740 4 client.16581 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:13.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.223+0000 7fe36de10740 1 -- 172.21.15.33:0/3585879285 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cee2375cf0 con 0x55cee241dfe0 2024-04-02T13:43:13.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.224+0000 7fe368eec640 1 --2- 172.21.15.33:0/3585879285 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55cee241dfe0 0x55cee2420440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.224+0000 7fe343fff640 10 client.16581 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:43:13.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.224+0000 7fe368eec640 1 --2- 172.21.15.33:0/3585879285 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55cee241dfe0 0x55cee2420440 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be74d396560dfd8e server_cookie=579415c017f0e9ee in_seq=0 out_seq=0 2024-04-02T13:43:13.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe343fff640 1 -- 172.21.15.33:0/3585879285 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cee2375cf0 con 0x55cee241dfe0 2024-04-02T13:43:13.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe343fff640 10 client.16581 handle_command_reply: tid=0 2024-04-02T13:43:13.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe36de10740 1 client.16581 shutdown 2024-04-02T13:43:13.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe36de10740 1 -- 172.21.15.33:0/3585879285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 msgr2=0x55cee240b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe36de10740 1 --2- 172.21.15.33:0/3585879285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 0x55cee240b590 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe35c036000 tx=0x7fe35c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe36de10740 1 -- 172.21.15.33:0/3585879285 shutdown_connections 2024-04-02T13:43:13.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe36de10740 1 --2- 172.21.15.33:0/3585879285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 0x55cee240b590 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe36de10740 1 --2- 172.21.15.33:0/3585879285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24050c0 0x55cee240b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe36de10740 1 --2- 172.21.15.33:0/3585879285 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55cee241dfe0 0x55cee2420440 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe36de10740 1 --2- 172.21.15.33:0/3585879285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24047a0 0x55cee240ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe343fff640 0 client.16581 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:43:13.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe36de10740 1 -- 172.21.15.33:0/3585879285 >> 172.21.15.33:0/3585879285 conn(0x55cee23feed0 msgr2=0x55cee2400c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe36de10740 1 -- 172.21.15.33:0/3585879285 shutdown_connections 2024-04-02T13:43:13.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.225+0000 7fe36de10740 1 -- 172.21.15.33:0/3585879285 wait complete. 2024-04-02T13:43:13.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.226+0000 7fe36de10740 20 client.16581 trim_cache size 0 max 16384 2024-04-02T13:43:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe36de10740 1 Processor -- start 2024-04-02T13:43:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe36de10740 1 -- start start 2024-04-02T13:43:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe36de10740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24047a0 0x55cee240d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe36de10740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 0x55cee24198f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe36de10740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee240dca0 0x55cee2409c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe36de10740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe3641b39d0 con 0x55cee24050c0 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe36de10740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3641b11e0 con 0x55cee240dca0 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe36de10740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe3641af010 con 0x55cee24047a0 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe35bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 0x55cee24198f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe35bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 0x55cee24198f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49104/0 (socket says 172.21.15.33:49104) 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe35bfff640 1 -- 172.21.15.33:0/1642122560 learned_addr learned my addr 172.21.15.33:0/1642122560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe368eec640 1 --2- 172.21.15.33:0/1642122560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24047a0 0x55cee240d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe3696ed640 1 --2- 172.21.15.33:0/1642122560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee240dca0 0x55cee2409c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe35bfff640 1 -- 172.21.15.33:0/1642122560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24047a0 msgr2=0x55cee240d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe35bfff640 1 --2- 172.21.15.33:0/1642122560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24047a0 0x55cee240d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe35bfff640 1 -- 172.21.15.33:0/1642122560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee240dca0 msgr2=0x55cee2409c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe35bfff640 1 --2- 172.21.15.33:0/1642122560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee240dca0 0x55cee2409c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe35bfff640 1 -- 172.21.15.33:0/1642122560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cee2404ef0 con 0x55cee24050c0 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe3696ed640 1 --2- 172.21.15.33:0/1642122560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee240dca0 0x55cee2409c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.239+0000 7fe368eec640 1 --2- 172.21.15.33:0/1642122560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24047a0 0x55cee240d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.240+0000 7fe35bfff640 1 --2- 172.21.15.33:0/1642122560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 0x55cee24198f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe354098f90 tx=0x7fe3540b5e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a2c381ad1a29e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.240+0000 7fe34a7fc640 1 -- 172.21.15.33:0/1642122560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35401d040 con 0x55cee24050c0 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.240+0000 7fe34a7fc640 1 -- 172.21.15.33:0/1642122560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe35409ccc0 con 0x55cee24050c0 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.240+0000 7fe34a7fc640 1 -- 172.21.15.33:0/1642122560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3540b5040 con 0x55cee24050c0 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.240+0000 7fe36de10740 1 -- 172.21.15.33:0/1642122560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 msgr2=0x55cee24198f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.240+0000 7fe36de10740 1 --2- 172.21.15.33:0/1642122560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 0x55cee24198f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe354098f90 tx=0x7fe3540b5e00 comp rx=0 tx=0).stop 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.240+0000 7fe36de10740 1 -- 172.21.15.33:0/1642122560 shutdown_connections 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.240+0000 7fe36de10740 1 --2- 172.21.15.33:0/1642122560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee240dca0 0x55cee2409c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.240+0000 7fe36de10740 1 --2- 172.21.15.33:0/1642122560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee24050c0 0x55cee24198f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.240+0000 7fe36de10740 1 --2- 172.21.15.33:0/1642122560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24047a0 0x55cee240d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.240+0000 7fe36de10740 1 -- 172.21.15.33:0/1642122560 >> 172.21.15.33:0/1642122560 conn(0x55cee23feed0 msgr2=0x55cee2400300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.240+0000 7fe36de10740 1 -- 172.21.15.33:0/1642122560 shutdown_connections 2024-04-02T13:43:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.240+0000 7fe36de10740 1 -- 172.21.15.33:0/1642122560 wait complete. 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe36de10740 1 -- start start 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe36de10740 1 Processor -- start 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe36de10740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24047a0 0x55cee2415ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe36de10740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24050c0 0x55cee2407340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe36de10740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 0x55cee2407880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe36de10740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe3641af6a0 con 0x55cee240dca0 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe36de10740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3641b0b10 con 0x55cee24047a0 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe36de10740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe3641afd70 con 0x55cee24050c0 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe36de10740 10 client.0 _ll_register_callbacks cb 0x55cee240dbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe3696ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 0x55cee2407880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe3696ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 0x55cee2407880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49116/0 (socket says 172.21.15.33:49116) 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe3696ed640 1 -- 172.21.15.33:0/3218865066 learned_addr learned my addr 172.21.15.33:0/3218865066 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe368eec640 1 --2- 172.21.15.33:0/3218865066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24047a0 0x55cee2415ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe3696ed640 1 -- 172.21.15.33:0/3218865066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24050c0 msgr2=0x55cee2407340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe3696ed640 1 --2- 172.21.15.33:0/3218865066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24050c0 0x55cee2407340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe3696ed640 1 -- 172.21.15.33:0/3218865066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24047a0 msgr2=0x55cee2415ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe3696ed640 1 --2- 172.21.15.33:0/3218865066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24047a0 0x55cee2415ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.241+0000 7fe3696ed640 1 -- 172.21.15.33:0/3218865066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cee2407e40 con 0x55cee240dca0 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe3696ed640 1 --2- 172.21.15.33:0/3218865066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 0x55cee2407880 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe35c02c630 tx=0x7fe35c041830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3307b436b172b2e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe343fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe343fff640 1 -- 172.21.15.33:0/3218865066 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c051040 con 0x55cee240dca0 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe343fff640 1 -- 172.21.15.33:0/3218865066 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe35c049040 con 0x55cee240dca0 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe36de10740 1 -- 172.21.15.33:0/3218865066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe35409ccc0 con 0x55cee240dca0 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe343fff640 1 -- 172.21.15.33:0/3218865066 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe35c068050 con 0x55cee240dca0 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe36de10740 10 client.16587 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe36de10740 1 -- 172.21.15.33:0/3218865066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cee24842c0 con 0x55cee240dca0 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe343fff640 1 -- 172.21.15.33:0/3218865066 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe35409ccc0 con 0x55cee240dca0 2024-04-02T13:43:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe343fff640 1 -- 172.21.15.33:0/3218865066 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe35c004240 con 0x55cee240dca0 2024-04-02T13:43:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe36de10740 10 client.16587 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe36de10740 10 client.16587 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:43:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe36de10740 20 client.16587 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe36de10740 1 --2- 172.21.15.33:0/3218865066 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55cee24849b0 0x55cee2486dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe36de10740 4 client.16587 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.242+0000 7fe36de10740 1 -- 172.21.15.33:0/3218865066 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cee2370040 con 0x55cee24849b0 2024-04-02T13:43:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe368eec640 1 --2- 172.21.15.33:0/3218865066 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55cee24849b0 0x55cee2486dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe343fff640 10 client.16587 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:43:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe368eec640 1 --2- 172.21.15.33:0/3218865066 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55cee24849b0 0x55cee2486dd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f28d96da91233598 server_cookie=8aff85b9ef4d2fe3 in_seq=0 out_seq=0 2024-04-02T13:43:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe343fff640 1 -- 172.21.15.33:0/3218865066 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cee2370040 con 0x55cee24849b0 2024-04-02T13:43:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe343fff640 10 client.16587 handle_command_reply: tid=0 2024-04-02T13:43:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe36de10740 1 client.16587 shutdown 2024-04-02T13:43:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe36de10740 1 -- 172.21.15.33:0/3218865066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 msgr2=0x55cee2407880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe36de10740 1 --2- 172.21.15.33:0/3218865066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 0x55cee2407880 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe35c02c630 tx=0x7fe35c041830 comp rx=0 tx=0).stop 2024-04-02T13:43:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe36de10740 1 -- 172.21.15.33:0/3218865066 shutdown_connections 2024-04-02T13:43:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe36de10740 1 --2- 172.21.15.33:0/3218865066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cee240dca0 0x55cee2407880 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe36de10740 1 --2- 172.21.15.33:0/3218865066 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55cee24849b0 0x55cee2486dd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe36de10740 1 --2- 172.21.15.33:0/3218865066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cee24050c0 0x55cee2407340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe36de10740 1 --2- 172.21.15.33:0/3218865066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cee24047a0 0x55cee2415ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe343fff640 0 client.16587 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:43:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe36de10740 1 -- 172.21.15.33:0/3218865066 >> 172.21.15.33:0/3218865066 conn(0x55cee23feed0 msgr2=0x55cee240aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe36de10740 1 -- 172.21.15.33:0/3218865066 shutdown_connections 2024-04-02T13:43:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe36de10740 1 -- 172.21.15.33:0/3218865066 wait complete. 2024-04-02T13:43:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.243+0000 7fe36de10740 20 client.16587 trim_cache size 0 max 16384 2024-04-02T13:43:13.254 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:43:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.244+0000 7fe36b150640 1 -- 172.21.15.33:0/1463194305 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe34c0a3d20 msgr2=0x7fe34c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.244+0000 7fe36b150640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe34c0a3d20 0x7fe34c0a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe35c037040 tx=0x7fe35c032c90 comp rx=0 tx=0).stop 2024-04-02T13:43:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.244+0000 7fe36b150640 1 -- 172.21.15.33:0/1463194305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe364100d80 msgr2=0x7fe36419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.244+0000 7fe36b150640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe364100d80 0x7fe36419aa60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe35400c970 tx=0x7fe35400ce30 comp rx=0 tx=0).stop 2024-04-02T13:43:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.244+0000 7fe36b150640 1 -- 172.21.15.33:0/1463194305 shutdown_connections 2024-04-02T13:43:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.244+0000 7fe36b150640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe34c0a3d20 0x7fe34c0a61e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.244+0000 7fe36b150640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe364107d00 0x7fe36419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.244+0000 7fe36b150640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe364100d80 0x7fe36419aa60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.244+0000 7fe36b150640 1 --2- 172.21.15.33:0/1463194305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe3641003c0 0x7fe36419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.244+0000 7fe36b150640 1 -- 172.21.15.33:0/1463194305 >> 172.21.15.33:0/1463194305 conn(0x7fe364075d40 msgr2=0x7fe364103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.244+0000 7fe36b150640 1 -- 172.21.15.33:0/1463194305 shutdown_connections 2024-04-02T13:43:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.244+0000 7fe36b150640 1 -- 172.21.15.33:0/1463194305 wait complete. 2024-04-02T13:43:13.256 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:13 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4081072444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:43:13.257 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:43:13.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.359+0000 7f6af644e640 1 -- 172.21.15.33:0/1303607643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af0107aa0 msgr2=0x7f6af0107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.359+0000 7f6af644e640 1 --2- 172.21.15.33:0/1303607643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af0107aa0 0x7f6af0107e80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009930 tx=0x7f6ae002bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 -- 172.21.15.33:0/1303607643 shutdown_connections 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 --2- 172.21.15.33:0/1303607643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6af0102450 0x7f6af010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 --2- 172.21.15.33:0/1303607643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af0101a90 0x7f6af0101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 --2- 172.21.15.33:0/1303607643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af0107aa0 0x7f6af0107e80 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 -- 172.21.15.33:0/1303607643 >> 172.21.15.33:0/1303607643 conn(0x7f6af0078f00 msgr2=0x7f6af0079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 -- 172.21.15.33:0/1303607643 shutdown_connections 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 -- 172.21.15.33:0/1303607643 wait complete. 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 Processor -- start 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 -- start start 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af0101a90 0x7f6af01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6af0102450 0x7f6af01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af0107aa0 0x7f6af019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6af0111e80 con 0x7f6af0107aa0 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6af0111d00 con 0x7f6af0101a90 2024-04-02T13:43:13.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.360+0000 7f6af644e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6af0112000 con 0x7f6af0102450 2024-04-02T13:43:13.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6af49eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af0107aa0 0x7f6af019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6af49eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af0107aa0 0x7f6af019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49132/0 (socket says 172.21.15.33:49132) 2024-04-02T13:43:13.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6af49eb640 1 -- 172.21.15.33:0/2753189291 learned_addr learned my addr 172.21.15.33:0/2753189291 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6af0102450 0x7f6af01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6aeffff640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af0101a90 0x7f6af01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6af49eb640 1 -- 172.21.15.33:0/2753189291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6af0102450 msgr2=0x7f6af01988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6af49eb640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6af0102450 0x7f6af01988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6af49eb640 1 -- 172.21.15.33:0/2753189291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af0101a90 msgr2=0x7f6af01983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6af49eb640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af0101a90 0x7f6af01983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6af49eb640 1 -- 172.21.15.33:0/2753189291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af019d390 con 0x7f6af0107aa0 2024-04-02T13:43:13.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6af0102450 0x7f6af01988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:13.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6aeffff640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af0101a90 0x7f6af01983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:13.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6af49eb640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af0107aa0 0x7f6af019cc60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800eaa0 tx=0x7f6ae800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6aed7fa640 1 -- 172.21.15.33:0/2753189291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae800cdb0 con 0x7f6af0107aa0 2024-04-02T13:43:13.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6aed7fa640 1 -- 172.21.15.33:0/2753189291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6ae8005020 con 0x7f6af0107aa0 2024-04-02T13:43:13.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6af644e640 1 -- 172.21.15.33:0/2753189291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af019d680 con 0x7f6af0107aa0 2024-04-02T13:43:13.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6aed7fa640 1 -- 172.21.15.33:0/2753189291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae80116b0 con 0x7f6af0107aa0 2024-04-02T13:43:13.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.361+0000 7f6af644e640 1 -- 172.21.15.33:0/2753189291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af01a4f10 con 0x7f6af0107aa0 2024-04-02T13:43:13.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.363+0000 7f6aed7fa640 1 -- 172.21.15.33:0/2753189291 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ae80051d0 con 0x7f6af0107aa0 2024-04-02T13:43:13.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.363+0000 7f6af903e740 1 Processor -- start 2024-04-02T13:43:13.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6af903e740 1 -- start start 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6af903e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 0x5626a5240b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6af903e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52410c0 0x5626a5249760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6af903e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a524c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6af903e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5626a51a47b0 con 0x5626a5249ca0 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6af903e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5626a5143570 con 0x5626a52407a0 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6af903e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5626a51a0760 con 0x5626a52410c0 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6aef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a524c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6aef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a524c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49134/0 (socket says 172.21.15.33:49134) 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6aef7fe640 1 -- 172.21.15.33:0/886659478 learned_addr learned my addr 172.21.15.33:0/886659478 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6aed7fa640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6ad80a3d20 0x7f6ad80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6aeffff640 1 --2- 172.21.15.33:0/886659478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52410c0 0x5626a5249760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/886659478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 0x5626a5240b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6af49eb640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6ad80a3d20 0x7f6ad80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6aef7fe640 1 -- 172.21.15.33:0/886659478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52410c0 msgr2=0x5626a5249760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/886659478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52410c0 0x5626a5249760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6aef7fe640 1 -- 172.21.15.33:0/886659478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 msgr2=0x5626a5240b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/886659478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 0x5626a5240b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6aef7fe640 1 -- 172.21.15.33:0/886659478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626a4eb83f0 con 0x5626a5249ca0 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6aeffff640 1 --2- 172.21.15.33:0/886659478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52410c0 0x5626a5249760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.364+0000 7f6aed7fa640 1 -- 172.21.15.33:0/2753189291 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6ae80d0db0 con 0x7f6af0107aa0 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.365+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/886659478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a524c090 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009f70 tx=0x7f6ae4009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a999d784bfc8b45 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.365+0000 7f6ad67fc640 1 -- 172.21.15.33:0/886659478 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4019070 con 0x5626a5249ca0 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.365+0000 7f6af49eb640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6ad80a3d20 0x7f6ad80a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6ae809c6e0 tx=0x7f6ae8098d20 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.365+0000 7f6ad67fc640 1 -- 172.21.15.33:0/886659478 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6ae4004230 con 0x5626a5249ca0 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.365+0000 7f6ad67fc640 1 -- 172.21.15.33:0/886659478 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4011940 con 0x5626a5249ca0 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.365+0000 7f6af903e740 1 -- 172.21.15.33:0/886659478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 msgr2=0x5626a524c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.365+0000 7f6af903e740 1 --2- 172.21.15.33:0/886659478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a524c090 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009f70 tx=0x7f6ae4009720 comp rx=0 tx=0).stop 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.365+0000 7f6af903e740 1 -- 172.21.15.33:0/886659478 shutdown_connections 2024-04-02T13:43:13.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.365+0000 7f6af903e740 1 --2- 172.21.15.33:0/886659478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a524c090 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.365+0000 7f6af903e740 1 --2- 172.21.15.33:0/886659478 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52410c0 0x5626a5249760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.365+0000 7f6af903e740 1 --2- 172.21.15.33:0/886659478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 0x5626a5240b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.365+0000 7f6af903e740 1 -- 172.21.15.33:0/886659478 >> 172.21.15.33:0/886659478 conn(0x5626a523aed0 msgr2=0x5626a523b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.365+0000 7f6af903e740 1 -- 172.21.15.33:0/886659478 shutdown_connections 2024-04-02T13:43:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6af903e740 1 -- 172.21.15.33:0/886659478 wait complete. 2024-04-02T13:43:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6af903e740 1 -- start start 2024-04-02T13:43:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6af903e740 1 Processor -- start 2024-04-02T13:43:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6af903e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52407a0 0x5626a52496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6af903e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52410c0 0x5626a5246a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6af903e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a52471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6af903e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5626a524ca90 con 0x5626a5249ca0 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6af903e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5626a515fe80 con 0x5626a52410c0 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6af903e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5626a524cc10 con 0x5626a52407a0 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6af903e740 10 client.0 _ll_register_callbacks cb 0x5626a519baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6aef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a52471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6aef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a52471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49138/0 (socket says 172.21.15.33:49138) 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6aef7fe640 1 -- 172.21.15.33:0/3495818775 learned_addr learned my addr 172.21.15.33:0/3495818775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.366+0000 7f6aeffff640 1 --2- 172.21.15.33:0/3495818775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52410c0 0x5626a5246a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/3495818775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52407a0 0x5626a52496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6aef7fe640 1 -- 172.21.15.33:0/3495818775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52407a0 msgr2=0x5626a52496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/3495818775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52407a0 0x5626a52496f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6aef7fe640 1 -- 172.21.15.33:0/3495818775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52410c0 msgr2=0x5626a5246a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/3495818775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52410c0 0x5626a5246a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6aef7fe640 1 -- 172.21.15.33:0/3495818775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626a4f827d0 con 0x5626a5249ca0 2024-04-02T13:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/3495818775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a52471c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005e00 tx=0x7f6ae4011520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac5179e78124fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6acf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6acf7fe640 1 -- 172.21.15.33:0/3495818775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4019040 con 0x5626a5249ca0 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6acf7fe640 1 -- 172.21.15.33:0/3495818775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6ae4011ce0 con 0x5626a5249ca0 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6af903e740 1 -- 172.21.15.33:0/3495818775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae4004230 con 0x5626a5249ca0 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6acf7fe640 1 -- 172.21.15.33:0/3495818775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ae4005020 con 0x5626a5249ca0 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6af903e740 10 client.16599 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.367+0000 7f6af903e740 1 -- 172.21.15.33:0/3495818775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5626a5258b40 con 0x5626a5249ca0 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.368+0000 7f6acf7fe640 1 -- 172.21.15.33:0/3495818775 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6ae4004230 con 0x5626a5249ca0 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.368+0000 7f6acf7fe640 1 -- 172.21.15.33:0/3495818775 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6ae4003aa0 con 0x5626a5249ca0 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.368+0000 7f6af903e740 10 client.16599 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.368+0000 7f6af903e740 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.368+0000 7f6af903e740 20 client.16599 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.368+0000 7f6af903e740 1 --2- 172.21.15.33:0/3495818775 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5626a5259e90 0x5626a525c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.368+0000 7f6af903e740 4 client.16599 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.368+0000 7f6af903e740 1 -- 172.21.15.33:0/3495818775 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626a51b1cf0 con 0x5626a5259e90 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.368+0000 7f6aeffff640 1 --2- 172.21.15.33:0/3495818775 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5626a5259e90 0x5626a525c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.368+0000 7f6aeffff640 1 --2- 172.21.15.33:0/3495818775 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5626a5259e90 0x5626a525c280 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1cc8888ef18a26 server_cookie=d8b049ecd8cd1444 in_seq=0 out_seq=0 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.368+0000 7f6acf7fe640 10 client.16599 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.369+0000 7f6acf7fe640 1 -- 172.21.15.33:0/3495818775 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626a51b1cf0 con 0x5626a5259e90 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.369+0000 7f6acf7fe640 10 client.16599 handle_command_reply: tid=0 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.369+0000 7f6af903e740 1 client.16599 shutdown 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.369+0000 7f6af903e740 1 -- 172.21.15.33:0/3495818775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 msgr2=0x5626a52471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.369+0000 7f6af903e740 1 --2- 172.21.15.33:0/3495818775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a52471c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005e00 tx=0x7f6ae4011520 comp rx=0 tx=0).stop 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.369+0000 7f6af903e740 1 -- 172.21.15.33:0/3495818775 shutdown_connections 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.369+0000 7f6af903e740 1 --2- 172.21.15.33:0/3495818775 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5626a5259e90 0x5626a525c280 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.369+0000 7f6af903e740 1 --2- 172.21.15.33:0/3495818775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a52471c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.369+0000 7f6acf7fe640 0 client.16599 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.369+0000 7f6af903e740 1 --2- 172.21.15.33:0/3495818775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52410c0 0x5626a5246a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.369+0000 7f6af903e740 1 --2- 172.21.15.33:0/3495818775 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52407a0 0x5626a52496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.370+0000 7f6af903e740 1 -- 172.21.15.33:0/3495818775 >> 172.21.15.33:0/3495818775 conn(0x5626a523aed0 msgr2=0x5626a523cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.370+0000 7f6af903e740 1 -- 172.21.15.33:0/3495818775 shutdown_connections 2024-04-02T13:43:13.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.370+0000 7f6af903e740 1 -- 172.21.15.33:0/3495818775 wait complete. 2024-04-02T13:43:13.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.370+0000 7f6af903e740 20 client.16599 trim_cache size 0 max 16384 2024-04-02T13:43:13.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.383+0000 7f6af903e740 1 Processor -- start 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.383+0000 7f6af903e740 1 -- start start 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.383+0000 7f6af903e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 0x5626a52559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.383+0000 7f6af903e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52410c0 0x5626a5261880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.383+0000 7f6af903e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a5261dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.383+0000 7f6af903e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b19d0 con 0x5626a5249ca0 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.383+0000 7f6af903e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01af1e0 con 0x5626a52407a0 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.383+0000 7f6af903e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01ad010 con 0x5626a52410c0 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.383+0000 7f6aef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a5261dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.383+0000 7f6aef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a5261dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49154/0 (socket says 172.21.15.33:49154) 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.383+0000 7f6aef7fe640 1 -- 172.21.15.33:0/2544358170 learned_addr learned my addr 172.21.15.33:0/2544358170 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.383+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/2544358170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 0x5626a52559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.384+0000 7f6aeffff640 1 --2- 172.21.15.33:0/2544358170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52410c0 0x5626a5261880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.384+0000 7f6aef7fe640 1 -- 172.21.15.33:0/2544358170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52410c0 msgr2=0x5626a5261880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.384+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/2544358170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52410c0 0x5626a5261880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.384+0000 7f6aef7fe640 1 -- 172.21.15.33:0/2544358170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 msgr2=0x5626a52559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.384+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/2544358170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 0x5626a52559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.384+0000 7f6aef7fe640 1 -- 172.21.15.33:0/2544358170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626a5240ef0 con 0x5626a5249ca0 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.384+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/2544358170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a5261dc0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005dd0 tx=0x7f6ae4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31569492110e5e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.384+0000 7f6ad5ffb640 1 -- 172.21.15.33:0/2544358170 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae403c070 con 0x5626a5249ca0 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.384+0000 7f6ad5ffb640 1 -- 172.21.15.33:0/2544358170 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6ae40041b0 con 0x5626a5249ca0 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.384+0000 7f6ad5ffb640 1 -- 172.21.15.33:0/2544358170 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4019040 con 0x5626a5249ca0 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 -- 172.21.15.33:0/2544358170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 msgr2=0x5626a5261dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 --2- 172.21.15.33:0/2544358170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a5261dc0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005dd0 tx=0x7f6ae4015040 comp rx=0 tx=0).stop 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 -- 172.21.15.33:0/2544358170 shutdown_connections 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 --2- 172.21.15.33:0/2544358170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a5249ca0 0x5626a5261dc0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 --2- 172.21.15.33:0/2544358170 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a52410c0 0x5626a5261880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 --2- 172.21.15.33:0/2544358170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 0x5626a52559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 -- 172.21.15.33:0/2544358170 >> 172.21.15.33:0/2544358170 conn(0x5626a523aed0 msgr2=0x5626a5245350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 -- 172.21.15.33:0/2544358170 shutdown_connections 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 -- 172.21.15.33:0/2544358170 wait complete. 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 -- start start 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 Processor -- start 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 0x5626a5242ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a52410c0 0x5626a5243410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a5249ca0 0x5626a5243950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01ad6a0 con 0x5626a52410c0 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01aeb10 con 0x5626a52407a0 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01add70 con 0x5626a5249ca0 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6af903e740 10 client.0 _ll_register_callbacks cb 0x7f6ae4013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6aeffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a52410c0 0x5626a5243410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6aeffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a52410c0 0x5626a5243410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49168/0 (socket says 172.21.15.33:49168) 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.385+0000 7f6aeffff640 1 -- 172.21.15.33:0/3340623765 learned_addr learned my addr 172.21.15.33:0/3340623765 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:13.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/3340623765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 0x5626a5242ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6aeffff640 1 -- 172.21.15.33:0/3340623765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a5249ca0 msgr2=0x5626a5243950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6aeffff640 1 --2- 172.21.15.33:0/3340623765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a5249ca0 0x5626a5243950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6aeffff640 1 -- 172.21.15.33:0/3340623765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 msgr2=0x5626a5242ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6aeffff640 1 --2- 172.21.15.33:0/3340623765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 0x5626a5242ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6aeffff640 1 -- 172.21.15.33:0/3340623765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626a5251160 con 0x5626a52410c0 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/3340623765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 0x5626a5242ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6aeffff640 1 --2- 172.21.15.33:0/3340623765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a52410c0 0x5626a5243410 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009930 tx=0x7f6ae00043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a13c048051f14e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6acf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6acf7fe640 1 -- 172.21.15.33:0/3340623765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0005230 con 0x5626a52410c0 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6acf7fe640 1 -- 172.21.15.33:0/3340623765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6ae00053d0 con 0x5626a52410c0 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6af903e740 1 -- 172.21.15.33:0/3340623765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae40041b0 con 0x5626a52410c0 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6acf7fe640 1 -- 172.21.15.33:0/3340623765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ae0057050 con 0x5626a52410c0 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6af903e740 10 client.16605 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6af903e740 1 -- 172.21.15.33:0/3340623765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5626a5252460 con 0x5626a52410c0 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.386+0000 7f6acf7fe640 1 -- 172.21.15.33:0/3340623765 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6ae40041b0 con 0x5626a52410c0 2024-04-02T13:43:13.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.387+0000 7f6acf7fe640 1 -- 172.21.15.33:0/3340623765 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6ae00049b0 con 0x5626a52410c0 2024-04-02T13:43:13.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.387+0000 7f6af903e740 10 client.16605 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:13.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.387+0000 7f6af903e740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:13.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.387+0000 7f6af903e740 20 client.16605 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:13.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.387+0000 7f6af903e740 1 --2- 172.21.15.33:0/3340623765 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5626a52c03d0 0x5626a52c27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:13.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.387+0000 7f6af903e740 4 client.16605 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:13.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.387+0000 7f6af903e740 1 -- 172.21.15.33:0/3340623765 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626a51ac040 con 0x5626a52c03d0 2024-04-02T13:43:13.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.387+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/3340623765 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5626a52c03d0 0x5626a52c27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:13.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.387+0000 7f6aef7fe640 1 --2- 172.21.15.33:0/3340623765 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5626a52c03d0 0x5626a52c27f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4898b0729e6ee9a9 server_cookie=71255c4e530a967f in_seq=0 out_seq=0 2024-04-02T13:43:13.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.387+0000 7f6acf7fe640 10 client.16605 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:13.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.387+0000 7f6acf7fe640 1 -- 172.21.15.33:0/3340623765 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626a51ac040 con 0x5626a52c03d0 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.387+0000 7f6acf7fe640 10 client.16605 handle_command_reply: tid=0 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.387+0000 7f6af903e740 1 client.16605 shutdown 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af903e740 1 -- 172.21.15.33:0/3340623765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a52410c0 msgr2=0x5626a5243410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af903e740 1 --2- 172.21.15.33:0/3340623765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a52410c0 0x5626a5243410 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009930 tx=0x7f6ae00043e0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af903e740 1 -- 172.21.15.33:0/3340623765 shutdown_connections 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af903e740 1 --2- 172.21.15.33:0/3340623765 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5626a52c03d0 0x5626a52c27f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af903e740 1 --2- 172.21.15.33:0/3340623765 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5626a5249ca0 0x5626a5243950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6acf7fe640 0 client.16605 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af903e740 1 --2- 172.21.15.33:0/3340623765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5626a52410c0 0x5626a5243410 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af903e740 1 --2- 172.21.15.33:0/3340623765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626a52407a0 0x5626a5242ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af903e740 1 -- 172.21.15.33:0/3340623765 >> 172.21.15.33:0/3340623765 conn(0x5626a523aed0 msgr2=0x5626a52396a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af903e740 1 -- 172.21.15.33:0/3340623765 shutdown_connections 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af903e740 1 -- 172.21.15.33:0/3340623765 wait complete. 2024-04-02T13:43:13.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af903e740 20 client.16605 trim_cache size 0 max 16384 2024-04-02T13:43:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:43:13.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af644e640 1 -- 172.21.15.33:0/2753189291 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6ad80a3d20 msgr2=0x7f6ad80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af644e640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6ad80a3d20 0x7f6ad80a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6ae809c6e0 tx=0x7f6ae8098d20 comp rx=0 tx=0).stop 2024-04-02T13:43:13.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af644e640 1 -- 172.21.15.33:0/2753189291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af0107aa0 msgr2=0x7f6af019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:13.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af644e640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af0107aa0 0x7f6af019cc60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800eaa0 tx=0x7f6ae800ef60 comp rx=0 tx=0).stop 2024-04-02T13:43:13.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af644e640 1 -- 172.21.15.33:0/2753189291 shutdown_connections 2024-04-02T13:43:13.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af644e640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6ad80a3d20 0x7f6ad80a61e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af644e640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af0107aa0 0x7f6af019cc60 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af644e640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6af0102450 0x7f6af01988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af644e640 1 --2- 172.21.15.33:0/2753189291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6af0101a90 0x7f6af01983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:13.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.388+0000 7f6af644e640 1 -- 172.21.15.33:0/2753189291 >> 172.21.15.33:0/2753189291 conn(0x7f6af0078f00 msgr2=0x7f6af0104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:13.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.389+0000 7f6af644e640 1 -- 172.21.15.33:0/2753189291 shutdown_connections 2024-04-02T13:43:13.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:13.389+0000 7f6af644e640 1 -- 172.21.15.33:0/2753189291 wait complete. 2024-04-02T13:43:13.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:13 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4081072444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:43:13.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:13 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4081072444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:43:14.403 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:43:14.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:43:14.403 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:43:14.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:14 smithi033 ceph-mon[32390]: pgmap v136: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:43:14.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:14 smithi145 ceph-mon[35646]: pgmap v136: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:43:14.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:14 smithi050 ceph-mon[35524]: pgmap v136: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:43:14.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.520+0000 7fc4f0b96640 1 -- 172.21.15.33:0/860562391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec102d70 msgr2=0x7fc4ec103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.520+0000 7fc4f0b96640 1 --2- 172.21.15.33:0/860562391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec102d70 0x7fc4ec103150 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009900 tx=0x7fc4dc02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.520+0000 7fc4f0b96640 1 -- 172.21.15.33:0/860562391 shutdown_connections 2024-04-02T13:43:14.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.520+0000 7fc4f0b96640 1 --2- 172.21.15.33:0/860562391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ec10c140 0x7fc4ec10e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.520+0000 7fc4f0b96640 1 --2- 172.21.15.33:0/860562391 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ec103690 0x7fc4ec10bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.520+0000 7fc4f0b96640 1 --2- 172.21.15.33:0/860562391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec102d70 0x7fc4ec103150 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.520+0000 7fc4f0b96640 1 -- 172.21.15.33:0/860562391 >> 172.21.15.33:0/860562391 conn(0x7fc4ec077b60 msgr2=0x7fc4ec077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.520+0000 7fc4f0b96640 1 -- 172.21.15.33:0/860562391 shutdown_connections 2024-04-02T13:43:14.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.520+0000 7fc4f0b96640 1 -- 172.21.15.33:0/860562391 wait complete. 2024-04-02T13:43:14.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.521+0000 7fc4f0b96640 1 Processor -- start 2024-04-02T13:43:14.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.521+0000 7fc4f0b96640 1 -- start start 2024-04-02T13:43:14.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.521+0000 7fc4f0b96640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ec102d70 0x7fc4ec198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.521+0000 7fc4f0b96640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ec103690 0x7fc4ec198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.521+0000 7fc4f0b96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec10c140 0x7fc4ec19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.521+0000 7fc4f0b96640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec111ee0 con 0x7fc4ec10c140 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.521+0000 7fc4f0b96640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec111d60 con 0x7fc4ec103690 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.521+0000 7fc4f0b96640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec112060 con 0x7fc4ec102d70 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.521+0000 7fc4ead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec10c140 0x7fc4ec19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.521+0000 7fc4ead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec10c140 0x7fc4ec19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33852/0 (socket says 172.21.15.33:33852) 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.521+0000 7fc4ead9d640 1 -- 172.21.15.33:0/877944191 learned_addr learned my addr 172.21.15.33:0/877944191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.522+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ec103690 0x7fc4ec198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.522+0000 7fc4ea59c640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ec102d70 0x7fc4ec198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.522+0000 7fc4ead9d640 1 -- 172.21.15.33:0/877944191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ec102d70 msgr2=0x7fc4ec198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.522+0000 7fc4ead9d640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ec102d70 0x7fc4ec198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.522+0000 7fc4ead9d640 1 -- 172.21.15.33:0/877944191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ec103690 msgr2=0x7fc4ec198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.522+0000 7fc4ead9d640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ec103690 0x7fc4ec198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.522+0000 7fc4ead9d640 1 -- 172.21.15.33:0/877944191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec19d430 con 0x7fc4ec10c140 2024-04-02T13:43:14.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.522+0000 7fc4ead9d640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec10c140 0x7fc4ec19cd00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400eaa0 tx=0x7fc4e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.522+0000 7fc4e37fe640 1 -- 172.21.15.33:0/877944191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e400cdb0 con 0x7fc4ec10c140 2024-04-02T13:43:14.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.522+0000 7fc4f0b96640 1 -- 172.21.15.33:0/877944191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec19d720 con 0x7fc4ec10c140 2024-04-02T13:43:14.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.522+0000 7fc4e37fe640 1 -- 172.21.15.33:0/877944191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc4e4005020 con 0x7fc4ec10c140 2024-04-02T13:43:14.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.522+0000 7fc4e37fe640 1 -- 172.21.15.33:0/877944191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40116b0 con 0x7fc4ec10c140 2024-04-02T13:43:14.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.522+0000 7fc4f0b96640 1 -- 172.21.15.33:0/877944191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec1a4fb0 con 0x7fc4ec10c140 2024-04-02T13:43:14.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.525+0000 7fc4e37fe640 1 -- 172.21.15.33:0/877944191 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc4e40051d0 con 0x7fc4ec10c140 2024-04-02T13:43:14.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.525+0000 7fc4f3777740 1 Processor -- start 2024-04-02T13:43:14.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.525+0000 7fc4f3777740 1 -- start start 2024-04-02T13:43:14.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.525+0000 7fc4f3777740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f4542c0 0x559f5f4546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.525+0000 7fc4f3777740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f454be0 0x559f5f45d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.525+0000 7fc4f3777740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 0x559f5f45fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.525+0000 7fc4f3777740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559f5f3b8770 con 0x559f5f45d7c0 2024-04-02T13:43:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.525+0000 7fc4f3777740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559f5f360960 con 0x559f5f454be0 2024-04-02T13:43:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.525+0000 7fc4f3777740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559f5f3b48c0 con 0x559f5f4542c0 2024-04-02T13:43:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 0x559f5f45fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 0x559f5f45fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33856/0 (socket says 172.21.15.33:33856) 2024-04-02T13:43:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e3fff640 1 -- 172.21.15.33:0/1723358579 learned_addr learned my addr 172.21.15.33:0/1723358579 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/1723358579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f4542c0 0x559f5f4546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4ea59c640 1 --2- 172.21.15.33:0/1723358579 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f454be0 0x559f5f45d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e37fe640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc4d80a3d20 0x7fc4d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e3fff640 1 -- 172.21.15.33:0/1723358579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f4542c0 msgr2=0x559f5f4546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/1723358579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f4542c0 0x559f5f4546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e3fff640 1 -- 172.21.15.33:0/1723358579 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f454be0 msgr2=0x559f5f45d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/1723358579 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f454be0 0x559f5f45d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e3fff640 1 -- 172.21.15.33:0/1723358579 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f5f1947e0 con 0x559f5f45d7c0 2024-04-02T13:43:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4ea59c640 1 --2- 172.21.15.33:0/1723358579 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f454be0 0x559f5f45d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4ead9d640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc4d80a3d20 0x7fc4d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e37fe640 1 -- 172.21.15.33:0/877944191 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc4e40d0cc0 con 0x7fc4ec10c140 2024-04-02T13:43:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/1723358579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 0x559f5f45fbb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4011740 tx=0x7fc4d400a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d39dce18ec3ec26 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e0ff9640 1 -- 172.21.15.33:0/1723358579 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4005020 con 0x559f5f45d7c0 2024-04-02T13:43:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4ead9d640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc4d80a3d20 0x7fc4d80a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4099370 tx=0x7fc4e4087040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e0ff9640 1 -- 172.21.15.33:0/1723358579 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc4d40047a0 con 0x559f5f45d7c0 2024-04-02T13:43:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.526+0000 7fc4e0ff9640 1 -- 172.21.15.33:0/1723358579 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d400fa20 con 0x559f5f45d7c0 2024-04-02T13:43:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.527+0000 7fc4f3777740 1 -- 172.21.15.33:0/1723358579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 msgr2=0x559f5f45fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.527+0000 7fc4f3777740 1 --2- 172.21.15.33:0/1723358579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 0x559f5f45fbb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4011740 tx=0x7fc4d400a6a0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.527+0000 7fc4f3777740 1 -- 172.21.15.33:0/1723358579 shutdown_connections 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.527+0000 7fc4f3777740 1 --2- 172.21.15.33:0/1723358579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 0x559f5f45fbb0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.527+0000 7fc4f3777740 1 --2- 172.21.15.33:0/1723358579 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f454be0 0x559f5f45d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.527+0000 7fc4f3777740 1 --2- 172.21.15.33:0/1723358579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f4542c0 0x559f5f4546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.527+0000 7fc4f3777740 1 -- 172.21.15.33:0/1723358579 >> 172.21.15.33:0/1723358579 conn(0x559f5f44e9f0 msgr2=0x559f5f44edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.527+0000 7fc4f3777740 1 -- 172.21.15.33:0/1723358579 shutdown_connections 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.527+0000 7fc4f3777740 1 -- 172.21.15.33:0/1723358579 wait complete. 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.527+0000 7fc4f3777740 1 -- start start 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4f3777740 1 Processor -- start 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4f3777740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 0x559f5f45a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4f3777740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f454be0 0x559f5f45ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4f3777740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f45d7c0 0x559f5f45b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4f3777740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559f5f4605b0 con 0x559f5f4542c0 2024-04-02T13:43:14.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4f3777740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559f5f373d20 con 0x559f5f45d7c0 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4f3777740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x559f5f460730 con 0x559f5f454be0 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4f3777740 10 client.0 _ll_register_callbacks cb 0x559f5f340a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4e3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 0x559f5f45a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4e3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 0x559f5f45a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33860/0 (socket says 172.21.15.33:33860) 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4e3fff640 1 -- 172.21.15.33:0/1737824712 learned_addr learned my addr 172.21.15.33:0/1737824712 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4ea59c640 1 --2- 172.21.15.33:0/1737824712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f454be0 0x559f5f45ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4e3fff640 1 -- 172.21.15.33:0/1737824712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f454be0 msgr2=0x559f5f45ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/1737824712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f454be0 0x559f5f45ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4e3fff640 1 -- 172.21.15.33:0/1737824712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f45d7c0 msgr2=0x559f5f45b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/1737824712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f45d7c0 0x559f5f45b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4e3fff640 1 -- 172.21.15.33:0/1737824712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f5f1f4f70 con 0x559f5f4542c0 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4ea59c640 1 --2- 172.21.15.33:0/1737824712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f454be0 0x559f5f45ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/1737824712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 0x559f5f45a7e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40058c0 tx=0x7fc4d4004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd0923e2d231383 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.528+0000 7fc4d1ffb640 1 -- 172.21.15.33:0/1737824712 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4004020 con 0x559f5f4542c0 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4d1ffb640 1 -- 172.21.15.33:0/1737824712 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc4d40041c0 con 0x559f5f4542c0 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4f3777740 1 -- 172.21.15.33:0/1737824712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4d40047a0 con 0x559f5f4542c0 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4d1ffb640 1 -- 172.21.15.33:0/1737824712 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4d402f050 con 0x559f5f4542c0 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4f3777740 10 client.16617 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4f3777740 1 -- 172.21.15.33:0/1737824712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f5f08c170 con 0x559f5f4542c0 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4d1ffb640 1 -- 172.21.15.33:0/1737824712 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc4d40047a0 con 0x559f5f4542c0 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4d1ffb640 1 -- 172.21.15.33:0/1737824712 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc4d4003820 con 0x559f5f4542c0 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4f3777740 10 client.16617 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4f3777740 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4f3777740 20 client.16617 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4f3777740 1 --2- 172.21.15.33:0/1737824712 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f5f46da10 0x559f5f46fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4f3777740 4 client.16617 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4f3777740 1 -- 172.21.15.33:0/1737824712 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f5f3c5cb0 con 0x559f5f46da10 2024-04-02T13:43:14.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.529+0000 7fc4ea59c640 1 --2- 172.21.15.33:0/1737824712 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f5f46da10 0x559f5f46fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.530+0000 7fc4d1ffb640 10 client.16617 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.530+0000 7fc4ea59c640 1 --2- 172.21.15.33:0/1737824712 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f5f46da10 0x559f5f46fe00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdca659f0e85208a server_cookie=36170e18eee19902 in_seq=0 out_seq=0 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4d1ffb640 1 -- 172.21.15.33:0/1737824712 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f5f3c5cb0 con 0x559f5f46da10 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4d1ffb640 10 client.16617 handle_command_reply: tid=0 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4f3777740 1 client.16617 shutdown 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4f3777740 1 -- 172.21.15.33:0/1737824712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 msgr2=0x559f5f45a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4f3777740 1 --2- 172.21.15.33:0/1737824712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 0x559f5f45a7e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40058c0 tx=0x7fc4d4004e50 comp rx=0 tx=0).stop 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4f3777740 1 -- 172.21.15.33:0/1737824712 shutdown_connections 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4f3777740 1 --2- 172.21.15.33:0/1737824712 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f5f46da10 0x559f5f46fe00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4f3777740 1 --2- 172.21.15.33:0/1737824712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f45d7c0 0x559f5f45b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4d1ffb640 0 client.16617 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4f3777740 1 --2- 172.21.15.33:0/1737824712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f454be0 0x559f5f45ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4f3777740 1 --2- 172.21.15.33:0/1737824712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 0x559f5f45a7e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4f3777740 1 -- 172.21.15.33:0/1737824712 >> 172.21.15.33:0/1737824712 conn(0x559f5f44e9f0 msgr2=0x559f5f450740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4f3777740 1 -- 172.21.15.33:0/1737824712 shutdown_connections 2024-04-02T13:43:14.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4f3777740 1 -- 172.21.15.33:0/1737824712 wait complete. 2024-04-02T13:43:14.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.531+0000 7fc4f3777740 20 client.16617 trim_cache size 0 max 16384 2024-04-02T13:43:14.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4f3777740 1 Processor -- start 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4f3777740 1 -- start start 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4f3777740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 0x559f5f45cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4f3777740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f454be0 0x559f5f4694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4f3777740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f45d7c0 0x559f5f459a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4f3777740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1b18d0 con 0x559f5f4542c0 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4f3777740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1af0e0 con 0x559f5f45d7c0 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4f3777740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1acf10 con 0x559f5f454be0 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4e3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 0x559f5f45cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4e3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 0x559f5f45cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33870/0 (socket says 172.21.15.33:33870) 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4e3fff640 1 -- 172.21.15.33:0/4070810588 learned_addr learned my addr 172.21.15.33:0/4070810588 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4e3fff640 1 -- 172.21.15.33:0/4070810588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f454be0 msgr2=0x559f5f4694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/4070810588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f454be0 0x559f5f4694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4e3fff640 1 -- 172.21.15.33:0/4070810588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f45d7c0 msgr2=0x559f5f459a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/4070810588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f45d7c0 0x559f5f459a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4e3fff640 1 -- 172.21.15.33:0/4070810588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f5f448ec0 con 0x559f5f4542c0 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/4070810588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 0x559f5f45cf10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4000c00 tx=0x7fc4d40052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550dca9224a6a20b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4d3fff640 1 -- 172.21.15.33:0/4070810588 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4004c60 con 0x559f5f4542c0 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4d3fff640 1 -- 172.21.15.33:0/4070810588 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc4d4005540 con 0x559f5f4542c0 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.549+0000 7fc4d3fff640 1 -- 172.21.15.33:0/4070810588 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4004240 con 0x559f5f4542c0 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.550+0000 7fc4f3777740 1 -- 172.21.15.33:0/4070810588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 msgr2=0x559f5f45cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.550+0000 7fc4f3777740 1 --2- 172.21.15.33:0/4070810588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 0x559f5f45cf10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4000c00 tx=0x7fc4d40052c0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.550+0000 7fc4f3777740 1 -- 172.21.15.33:0/4070810588 shutdown_connections 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.550+0000 7fc4f3777740 1 --2- 172.21.15.33:0/4070810588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f45d7c0 0x559f5f459a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.550+0000 7fc4f3777740 1 --2- 172.21.15.33:0/4070810588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f454be0 0x559f5f4694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.550+0000 7fc4f3777740 1 --2- 172.21.15.33:0/4070810588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f4542c0 0x559f5f45cf10 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.550+0000 7fc4f3777740 1 -- 172.21.15.33:0/4070810588 >> 172.21.15.33:0/4070810588 conn(0x559f5f44e9f0 msgr2=0x559f5f450660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.550+0000 7fc4f3777740 1 -- 172.21.15.33:0/4070810588 shutdown_connections 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.550+0000 7fc4f3777740 1 -- 172.21.15.33:0/4070810588 wait complete. 2024-04-02T13:43:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.550+0000 7fc4f3777740 1 -- start start 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.550+0000 7fc4f3777740 1 Processor -- start 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4f3777740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f4542c0 0x559f5f4577e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4f3777740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f454be0 0x559f5f457d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4f3777740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 0x559f5f4582a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4f3777740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1ad5a0 con 0x559f5f45d7c0 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4f3777740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1aea10 con 0x559f5f454be0 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4f3777740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec1adc70 con 0x559f5f4542c0 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4f3777740 10 client.0 _ll_register_callbacks cb 0x7fc4d400ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4e3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 0x559f5f4582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4e3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 0x559f5f4582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33884/0 (socket says 172.21.15.33:33884) 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4e3fff640 1 -- 172.21.15.33:0/3450174780 learned_addr learned my addr 172.21.15.33:0/3450174780 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4ea59c640 1 --2- 172.21.15.33:0/3450174780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f454be0 0x559f5f457d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/3450174780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f4542c0 0x559f5f4577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4e3fff640 1 -- 172.21.15.33:0/3450174780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f4542c0 msgr2=0x559f5f4577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/3450174780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f4542c0 0x559f5f4577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4e3fff640 1 -- 172.21.15.33:0/3450174780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f454be0 msgr2=0x559f5f457d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/3450174780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f454be0 0x559f5f457d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4e3fff640 1 -- 172.21.15.33:0/3450174780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f5f4d30f0 con 0x559f5f45d7c0 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4e3fff640 1 --2- 172.21.15.33:0/3450174780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 0x559f5f4582a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40058c0 tx=0x7fc4d40114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735fddf2e84b3fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4d1ffb640 1 -- 172.21.15.33:0/3450174780 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4004c60 con 0x559f5f45d7c0 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4d1ffb640 1 -- 172.21.15.33:0/3450174780 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc4d4004e00 con 0x559f5f45d7c0 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.551+0000 7fc4f3777740 1 -- 172.21.15.33:0/3450174780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4d4005540 con 0x559f5f45d7c0 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.552+0000 7fc4d1ffb640 1 -- 172.21.15.33:0/3450174780 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4d4011030 con 0x559f5f45d7c0 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.552+0000 7fc4f3777740 10 client.16623 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.552+0000 7fc4f3777740 1 -- 172.21.15.33:0/3450174780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f5f4d4430 con 0x559f5f45d7c0 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.552+0000 7fc4d1ffb640 1 -- 172.21.15.33:0/3450174780 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc4d4005540 con 0x559f5f45d7c0 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.552+0000 7fc4d1ffb640 1 -- 172.21.15.33:0/3450174780 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc4d403c070 con 0x559f5f45d7c0 2024-04-02T13:43:14.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.552+0000 7fc4f3777740 10 client.16623 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:14.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.552+0000 7fc4f3777740 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:14.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.552+0000 7fc4f3777740 20 client.16623 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:14.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.552+0000 7fc4f3777740 1 --2- 172.21.15.33:0/3450174780 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f5f4d4a70 0x559f5f4d6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.552+0000 7fc4f3777740 4 client.16623 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:43:14.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.552+0000 7fc4f3777740 1 -- 172.21.15.33:0/3450174780 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f5f3c0000 con 0x559f5f4d4a70 2024-04-02T13:43:14.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.552+0000 7fc4ea59c640 1 --2- 172.21.15.33:0/3450174780 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f5f4d4a70 0x559f5f4d6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.553+0000 7fc4d1ffb640 10 client.16623 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:14.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.553+0000 7fc4ea59c640 1 --2- 172.21.15.33:0/3450174780 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f5f4d4a70 0x559f5f4d6e90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f945b920bc0df15 server_cookie=5479549c015fa657 in_seq=0 out_seq=0 2024-04-02T13:43:14.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4d1ffb640 1 -- 172.21.15.33:0/3450174780 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f5f3c0000 con 0x559f5f4d4a70 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4d1ffb640 10 client.16623 handle_command_reply: tid=0 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4f3777740 1 client.16623 shutdown 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4f3777740 1 -- 172.21.15.33:0/3450174780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 msgr2=0x559f5f4582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4f3777740 1 --2- 172.21.15.33:0/3450174780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 0x559f5f4582a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40058c0 tx=0x7fc4d40114a0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4f3777740 1 -- 172.21.15.33:0/3450174780 shutdown_connections 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4f3777740 1 --2- 172.21.15.33:0/3450174780 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x559f5f4d4a70 0x559f5f4d6e90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4f3777740 1 --2- 172.21.15.33:0/3450174780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559f5f45d7c0 0x559f5f4582a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4f3777740 1 --2- 172.21.15.33:0/3450174780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559f5f454be0 0x559f5f457d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4d1ffb640 0 client.16623 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4f3777740 1 --2- 172.21.15.33:0/3450174780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559f5f4542c0 0x559f5f4577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4f3777740 1 -- 172.21.15.33:0/3450174780 >> 172.21.15.33:0/3450174780 conn(0x559f5f44e9f0 msgr2=0x559f5f44d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4f3777740 1 -- 172.21.15.33:0/3450174780 shutdown_connections 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4f3777740 1 -- 172.21.15.33:0/3450174780 wait complete. 2024-04-02T13:43:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.554+0000 7fc4f3777740 20 client.16623 trim_cache size 0 max 16384 2024-04-02T13:43:14.563 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:43:14.563 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:43:14.563 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "666fc9a6-33f9-4b1f-9002-a1bcde6af77a", 2024-04-02T13:43:14.563 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:43:14.563 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:43:14.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.555+0000 7fc4f0b96640 1 -- 172.21.15.33:0/877944191 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc4d80a3d20 msgr2=0x7fc4d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.555+0000 7fc4f0b96640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc4d80a3d20 0x7fc4d80a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4099370 tx=0x7fc4e4087040 comp rx=0 tx=0).stop 2024-04-02T13:43:14.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.555+0000 7fc4f0b96640 1 -- 172.21.15.33:0/877944191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec10c140 msgr2=0x7fc4ec19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.555+0000 7fc4f0b96640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec10c140 0x7fc4ec19cd00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400eaa0 tx=0x7fc4e400ef60 comp rx=0 tx=0).stop 2024-04-02T13:43:14.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.555+0000 7fc4f0b96640 1 -- 172.21.15.33:0/877944191 shutdown_connections 2024-04-02T13:43:14.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.555+0000 7fc4f0b96640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc4d80a3d20 0x7fc4d80a61e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.555+0000 7fc4f0b96640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec10c140 0x7fc4ec19cd00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.555+0000 7fc4f0b96640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ec103690 0x7fc4ec198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.555+0000 7fc4f0b96640 1 --2- 172.21.15.33:0/877944191 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ec102d70 0x7fc4ec198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.555+0000 7fc4f0b96640 1 -- 172.21.15.33:0/877944191 >> 172.21.15.33:0/877944191 conn(0x7fc4ec077b60 msgr2=0x7fc4ec0fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.555+0000 7fc4f0b96640 1 -- 172.21.15.33:0/877944191 shutdown_connections 2024-04-02T13:43:14.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.555+0000 7fc4f0b96640 1 -- 172.21.15.33:0/877944191 wait complete. 2024-04-02T13:43:14.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:43:14.569 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:43:14.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.712+0000 7fe8e480d640 1 -- 172.21.15.33:0/765755069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8e010e4a0 msgr2=0x7fe8e0110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.712+0000 7fe8e480d640 1 --2- 172.21.15.33:0/765755069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8e010e4a0 0x7fe8e0110890 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009ca0 tx=0x7fe8d002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.712+0000 7fe8e480d640 1 -- 172.21.15.33:0/765755069 shutdown_connections 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.712+0000 7fe8e480d640 1 --2- 172.21.15.33:0/765755069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8e010e4a0 0x7fe8e0110890 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.712+0000 7fe8e480d640 1 --2- 172.21.15.33:0/765755069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8e0069600 0x7fe8e010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.712+0000 7fe8e480d640 1 --2- 172.21.15.33:0/765755069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e0068ce0 0x7fe8e00690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.712+0000 7fe8e480d640 1 -- 172.21.15.33:0/765755069 >> 172.21.15.33:0/765755069 conn(0x7fe8e0077b00 msgr2=0x7fe8e0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.712+0000 7fe8e480d640 1 -- 172.21.15.33:0/765755069 shutdown_connections 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.712+0000 7fe8e480d640 1 -- 172.21.15.33:0/765755069 wait complete. 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.713+0000 7fe8e480d640 1 Processor -- start 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.713+0000 7fe8e480d640 1 -- start start 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.713+0000 7fe8e480d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8e0068ce0 0x7fe8e019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.713+0000 7fe8e480d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e0069600 0x7fe8e019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.713+0000 7fe8e480d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8e010e4a0 0x7fe8e019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.713+0000 7fe8e480d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e0114030 con 0x7fe8e010e4a0 2024-04-02T13:43:14.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.713+0000 7fe8e480d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e0113eb0 con 0x7fe8e0069600 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.713+0000 7fe8e480d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01141b0 con 0x7fe8e0068ce0 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8ded9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8e010e4a0 0x7fe8e019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8ded9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8e010e4a0 0x7fe8e019edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33904/0 (socket says 172.21.15.33:33904) 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8ded9d640 1 -- 172.21.15.33:0/2411654764 learned_addr learned my addr 172.21.15.33:0/2411654764 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e0069600 0x7fe8e019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8de59c640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8e0068ce0 0x7fe8e019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8ded9d640 1 -- 172.21.15.33:0/2411654764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8e0068ce0 msgr2=0x7fe8e019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8ded9d640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8e0068ce0 0x7fe8e019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8ded9d640 1 -- 172.21.15.33:0/2411654764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e0069600 msgr2=0x7fe8e019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8ded9d640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e0069600 0x7fe8e019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8ded9d640 1 -- 172.21.15.33:0/2411654764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e019f4f0 con 0x7fe8e010e4a0 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8de59c640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8e0068ce0 0x7fe8e019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8ded9d640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8e010e4a0 0x7fe8e019edc0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0004af0 tx=0x7fe8d0034d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8d77fe640 1 -- 172.21.15.33:0/2411654764 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0039050 con 0x7fe8e010e4a0 2024-04-02T13:43:14.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8e480d640 1 -- 172.21.15.33:0/2411654764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e019f780 con 0x7fe8e010e4a0 2024-04-02T13:43:14.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8d77fe640 1 -- 172.21.15.33:0/2411654764 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe8d0045070 con 0x7fe8e010e4a0 2024-04-02T13:43:14.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8d77fe640 1 -- 172.21.15.33:0/2411654764 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d00046b0 con 0x7fe8e010e4a0 2024-04-02T13:43:14.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.714+0000 7fe8e480d640 1 -- 172.21.15.33:0/2411654764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e01a7070 con 0x7fe8e010e4a0 2024-04-02T13:43:14.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.716+0000 7fe8d77fe640 1 -- 172.21.15.33:0/2411654764 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe8d0003970 con 0x7fe8e010e4a0 2024-04-02T13:43:14.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.717+0000 7fe8e73f8740 1 Processor -- start 2024-04-02T13:43:14.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.717+0000 7fe8e73f8740 1 -- start start 2024-04-02T13:43:14.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.717+0000 7fe8e73f8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757a8d0 0x558b1757acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.717+0000 7fe8e73f8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 0x558b17583890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.717+0000 7fe8e73f8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b175861c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.717+0000 7fe8e73f8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b174de830 con 0x558b17583dd0 2024-04-02T13:43:14.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.717+0000 7fe8e73f8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558b17486820 con 0x558b1757b1f0 2024-04-02T13:43:14.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.717+0000 7fe8e73f8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558b174da7e0 con 0x558b1757a8d0 2024-04-02T13:43:14.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.717+0000 7fe8d77fe640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe8cc0a3d20 0x7fe8cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.717+0000 7fe8ddd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b175861c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.717+0000 7fe8ded9d640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe8cc0a3d20 0x7fe8cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.717+0000 7fe8ddd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b175861c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33918/0 (socket says 172.21.15.33:33918) 2024-04-02T13:43:14.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.717+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/2163178676 learned_addr learned my addr 172.21.15.33:0/2163178676 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:14.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.718+0000 7fe8d77fe640 1 -- 172.21.15.33:0/2411654764 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe8d00f3510 con 0x7fe8e010e4a0 2024-04-02T13:43:14.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.718+0000 7fe8de59c640 1 --2- 172.21.15.33:0/2163178676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 0x558b17583890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.718+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/2163178676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757a8d0 0x558b1757acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.718+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/2163178676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757a8d0 msgr2=0x558b1757acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.718+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/2163178676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757a8d0 0x558b1757acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.718+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/2163178676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 msgr2=0x558b17583890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.718+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/2163178676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 0x558b17583890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.718+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/2163178676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b172bc7d0 con 0x558b17583dd0 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.718+0000 7fe8ded9d640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe8cc0a3d20 0x7fe8cc0a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00bef10 tx=0x7fe8d0090040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.718+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/2163178676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b175861c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800bfb0 tx=0x7fe8c800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef158e0423eae088 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.718+0000 7fe8d4ff9640 1 -- 172.21.15.33:0/2163178676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c801e640 con 0x558b17583dd0 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.718+0000 7fe8d4ff9640 1 -- 172.21.15.33:0/2163178676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe8c8010900 con 0x558b17583dd0 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.718+0000 7fe8d4ff9640 1 -- 172.21.15.33:0/2163178676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c801d8a0 con 0x558b17583dd0 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.719+0000 7fe8e73f8740 1 -- 172.21.15.33:0/2163178676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 msgr2=0x558b175861c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.719+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/2163178676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b175861c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800bfb0 tx=0x7fe8c800b7b0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.719+0000 7fe8e73f8740 1 -- 172.21.15.33:0/2163178676 shutdown_connections 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.719+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/2163178676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b175861c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.719+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/2163178676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 0x558b17583890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.719+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/2163178676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757a8d0 0x558b1757acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.719+0000 7fe8e73f8740 1 -- 172.21.15.33:0/2163178676 >> 172.21.15.33:0/2163178676 conn(0x558b17575000 msgr2=0x558b175753e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.719+0000 7fe8e73f8740 1 -- 172.21.15.33:0/2163178676 shutdown_connections 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.719+0000 7fe8e73f8740 1 -- 172.21.15.33:0/2163178676 wait complete. 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.719+0000 7fe8e73f8740 1 -- start start 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.719+0000 7fe8e73f8740 1 Processor -- start 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8e73f8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b1757a8d0 0x558b175836d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8e73f8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 0x558b17580a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8e73f8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b17583dd0 0x558b17580fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8e73f8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b17586bc0 con 0x558b1757a8d0 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8e73f8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558b17499d30 con 0x558b1757b1f0 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8e73f8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558b17586d40 con 0x558b17583dd0 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8e73f8740 10 client.0 _ll_register_callbacks cb 0x558b17499720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:14.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8ddd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b1757a8d0 0x558b175836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8ddd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b1757a8d0 0x558b175836d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33922/0 (socket says 172.21.15.33:33922) 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/4007056057 learned_addr learned my addr 172.21.15.33:0/4007056057 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8de59c640 1 --2- 172.21.15.33:0/4007056057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 0x558b17580a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/4007056057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b17583dd0 0x558b17580fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/4007056057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b17583dd0 msgr2=0x558b17580fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/4007056057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b17583dd0 0x558b17580fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/4007056057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 msgr2=0x558b17580a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/4007056057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 0x558b17580a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/4007056057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b171f23f0 con 0x558b1757a8d0 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/4007056057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b1757a8d0 0x558b175836d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8001eb0 tx=0x7fe8c801d480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365d93261959b60c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8c5ffb640 1 -- 172.21.15.33:0/4007056057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c802f060 con 0x558b1757a8d0 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8c5ffb640 1 -- 172.21.15.33:0/4007056057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe8c8005040 con 0x558b1757a8d0 2024-04-02T13:43:14.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.720+0000 7fe8e73f8740 1 -- 172.21.15.33:0/4007056057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c8010900 con 0x558b1757a8d0 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.721+0000 7fe8c5ffb640 1 -- 172.21.15.33:0/4007056057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe8c8005240 con 0x558b1757a8d0 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.721+0000 7fe8e73f8740 10 client.16635 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.721+0000 7fe8e73f8740 1 -- 172.21.15.33:0/4007056057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558b171b20f0 con 0x558b1757a8d0 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.721+0000 7fe8c5ffb640 1 -- 172.21.15.33:0/4007056057 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe8c8010900 con 0x558b1757a8d0 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.721+0000 7fe8c5ffb640 1 -- 172.21.15.33:0/4007056057 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe8c801c070 con 0x558b1757a8d0 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.721+0000 7fe8e73f8740 10 client.16635 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.721+0000 7fe8e73f8740 10 client.16635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.721+0000 7fe8e73f8740 20 client.16635 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.721+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/4007056057 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b17593d30 0x558b17596120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.721+0000 7fe8e73f8740 4 client.16635 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.721+0000 7fe8e73f8740 1 -- 172.21.15.33:0/4007056057 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b174ebd70 con 0x558b17593d30 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.721+0000 7fe8de59c640 1 --2- 172.21.15.33:0/4007056057 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b17593d30 0x558b17596120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.722+0000 7fe8de59c640 1 --2- 172.21.15.33:0/4007056057 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b17593d30 0x558b17596120 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=105940d565fa50a3 server_cookie=dc7918e732c31c0f in_seq=0 out_seq=0 2024-04-02T13:43:14.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.722+0000 7fe8c5ffb640 10 client.16635 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:14.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.722+0000 7fe8c5ffb640 1 -- 172.21.15.33:0/4007056057 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b174ebd70 con 0x558b17593d30 2024-04-02T13:43:14.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.722+0000 7fe8c5ffb640 10 client.16635 handle_command_reply: tid=0 2024-04-02T13:43:14.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.722+0000 7fe8e73f8740 1 client.16635 shutdown 2024-04-02T13:43:14.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.723+0000 7fe8e73f8740 1 -- 172.21.15.33:0/4007056057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b1757a8d0 msgr2=0x558b175836d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.723+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/4007056057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b1757a8d0 0x558b175836d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8001eb0 tx=0x7fe8c801d480 comp rx=0 tx=0).stop 2024-04-02T13:43:14.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.723+0000 7fe8e73f8740 1 -- 172.21.15.33:0/4007056057 shutdown_connections 2024-04-02T13:43:14.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.723+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/4007056057 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b17593d30 0x558b17596120 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.723+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/4007056057 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b17583dd0 0x558b17580fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.723+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/4007056057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 0x558b17580a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.723+0000 7fe8c5ffb640 0 client.16635 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:14.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.723+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/4007056057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b1757a8d0 0x558b175836d0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.723+0000 7fe8e73f8740 1 -- 172.21.15.33:0/4007056057 >> 172.21.15.33:0/4007056057 conn(0x558b17575000 msgr2=0x558b17576d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.723+0000 7fe8e73f8740 1 -- 172.21.15.33:0/4007056057 shutdown_connections 2024-04-02T13:43:14.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.723+0000 7fe8e73f8740 1 -- 172.21.15.33:0/4007056057 wait complete. 2024-04-02T13:43:14.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.723+0000 7fe8e73f8740 20 client.16635 trim_cache size 0 max 16384 2024-04-02T13:43:14.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.740+0000 7fe8e73f8740 1 Processor -- start 2024-04-02T13:43:14.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.740+0000 7fe8e73f8740 1 -- start start 2024-04-02T13:43:14.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.740+0000 7fe8e73f8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757a8d0 0x558b17574240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.740+0000 7fe8e73f8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757b1f0 0x558b1757be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.740+0000 7fe8e73f8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b17583190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.740+0000 7fe8e73f8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01b3ae0 con 0x558b17583dd0 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.740+0000 7fe8e73f8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01b12f0 con 0x558b1757a8d0 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.740+0000 7fe8e73f8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01af120 con 0x558b1757b1f0 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.740+0000 7fe8ddd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b17583190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.740+0000 7fe8ddd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b17583190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33936/0 (socket says 172.21.15.33:33936) 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.740+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/3680372890 learned_addr learned my addr 172.21.15.33:0/3680372890 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.740+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/3680372890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757a8d0 0x558b17574240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.741+0000 7fe8de59c640 1 --2- 172.21.15.33:0/3680372890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757b1f0 0x558b1757be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.741+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/3680372890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757b1f0 msgr2=0x558b1757be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.741+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/3680372890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757b1f0 0x558b1757be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.741+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/3680372890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757a8d0 msgr2=0x558b17574240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.741+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/3680372890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757a8d0 0x558b17574240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.741+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/3680372890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b1757b020 con 0x558b17583dd0 2024-04-02T13:43:14.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.741+0000 7fe8de59c640 1 --2- 172.21.15.33:0/3680372890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757b1f0 0x558b1757be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:14.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.741+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/3680372890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b17583190 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8003810 tx=0x7fe8c8020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9574d4c6fca9bb5f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.741+0000 7fe8c7fff640 1 -- 172.21.15.33:0/3680372890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8005070 con 0x558b17583dd0 2024-04-02T13:43:14.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.741+0000 7fe8c7fff640 1 -- 172.21.15.33:0/3680372890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe8c8012110 con 0x558b17583dd0 2024-04-02T13:43:14.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.741+0000 7fe8c7fff640 1 -- 172.21.15.33:0/3680372890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c801dc90 con 0x558b17583dd0 2024-04-02T13:43:14.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 -- 172.21.15.33:0/3680372890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 msgr2=0x558b17583190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/3680372890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b17583190 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8003810 tx=0x7fe8c8020040 comp rx=0 tx=0).stop 2024-04-02T13:43:14.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 -- 172.21.15.33:0/3680372890 shutdown_connections 2024-04-02T13:43:14.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/3680372890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b17583190 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/3680372890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757b1f0 0x558b1757be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/3680372890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757a8d0 0x558b17574240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 -- 172.21.15.33:0/3680372890 >> 172.21.15.33:0/3680372890 conn(0x558b17575000 msgr2=0x558b1757b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 -- 172.21.15.33:0/3680372890 shutdown_connections 2024-04-02T13:43:14.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 -- 172.21.15.33:0/3680372890 wait complete. 2024-04-02T13:43:14.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 -- start start 2024-04-02T13:43:14.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 Processor -- start 2024-04-02T13:43:14.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757a8d0 0x558b1758c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 0x558b17587740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b17587c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01af7b0 con 0x558b17583dd0 2024-04-02T13:43:14.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01b0c20 con 0x558b1757b1f0 2024-04-02T13:43:14.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01afe80 con 0x558b1757a8d0 2024-04-02T13:43:14.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8e73f8740 10 client.0 _ll_register_callbacks cb 0x7fe8c800bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:14.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8ddd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b17587c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8de59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 0x558b17587740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8ddd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b17587c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33944/0 (socket says 172.21.15.33:33944) 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.742+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/599259006 learned_addr learned my addr 172.21.15.33:0/599259006 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/599259006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757a8d0 0x558b1758c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/599259006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757a8d0 msgr2=0x558b1758c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/599259006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757a8d0 0x558b1758c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/599259006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 msgr2=0x558b17587740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/599259006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 0x558b17587740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8ddd9b640 1 -- 172.21.15.33:0/599259006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b175884b0 con 0x558b17583dd0 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8de59c640 1 --2- 172.21.15.33:0/599259006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 0x558b17587740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8ddd9b640 1 --2- 172.21.15.33:0/599259006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b17587c80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x558b17574820 tx=0x7fe8c801d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b00ad1046c2eca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8c5ffb640 1 -- 172.21.15.33:0/599259006 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c802f030 con 0x558b17583dd0 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8c5ffb640 1 -- 172.21.15.33:0/599259006 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe8c8003bb0 con 0x558b17583dd0 2024-04-02T13:43:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8e73f8740 1 -- 172.21.15.33:0/599259006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c8012110 con 0x558b17583dd0 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8c5ffb640 1 -- 172.21.15.33:0/599259006 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe8c801d500 con 0x558b17583dd0 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8e73f8740 10 client.16641 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8e73f8740 1 -- 172.21.15.33:0/599259006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558b175fa0e0 con 0x558b17583dd0 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8c5ffb640 1 -- 172.21.15.33:0/599259006 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe8c8012110 con 0x558b17583dd0 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.743+0000 7fe8c5ffb640 1 -- 172.21.15.33:0/599259006 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe8c801c040 con 0x558b17583dd0 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8e73f8740 10 client.16641 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8e73f8740 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8e73f8740 20 client.16641 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/599259006 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b175fa5c0 0x558b175fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8e73f8740 4 client.16641 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8e73f8740 1 -- 172.21.15.33:0/599259006 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b174e60c0 con 0x558b175fa5c0 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8de59c640 1 --2- 172.21.15.33:0/599259006 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b175fa5c0 0x558b175fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8de59c640 1 --2- 172.21.15.33:0/599259006 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b175fa5c0 0x558b175fc9e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7894979ce4b74b8 server_cookie=d30a2f4137239666 in_seq=0 out_seq=0 2024-04-02T13:43:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8c5ffb640 10 client.16641 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8c5ffb640 1 -- 172.21.15.33:0/599259006 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558b174e60c0 con 0x558b175fa5c0 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8c5ffb640 10 client.16641 handle_command_reply: tid=0 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8e73f8740 1 client.16641 shutdown 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8e73f8740 1 -- 172.21.15.33:0/599259006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 msgr2=0x558b17587c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.744+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/599259006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b17587c80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x558b17574820 tx=0x7fe8c801d3e0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e73f8740 1 -- 172.21.15.33:0/599259006 shutdown_connections 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/599259006 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b175fa5c0 0x558b175fc9e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/599259006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b17583dd0 0x558b17587c80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/599259006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b1757b1f0 0x558b17587740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8c5ffb640 0 client.16641 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e73f8740 1 --2- 172.21.15.33:0/599259006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b1757a8d0 0x558b1758c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e73f8740 1 -- 172.21.15.33:0/599259006 >> 172.21.15.33:0/599259006 conn(0x558b17575000 msgr2=0x558b175740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e73f8740 1 -- 172.21.15.33:0/599259006 shutdown_connections 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e73f8740 1 -- 172.21.15.33:0/599259006 wait complete. 2024-04-02T13:43:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e73f8740 20 client.16641 trim_cache size 0 max 16384 2024-04-02T13:43:14.756 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:43:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:43:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:43:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "666fc9a6-33f9-4b1f-9002-a1bcde6af77a": { 2024-04-02T13:43:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:43:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "666fc9a6-33f9-4b1f-9002-a1bcde6af77a", 2024-04-02T13:43:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:43:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:43:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:43:14.756 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:43:14.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e480d640 1 -- 172.21.15.33:0/2411654764 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe8cc0a3d20 msgr2=0x7fe8cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e480d640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe8cc0a3d20 0x7fe8cc0a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00bef10 tx=0x7fe8d0090040 comp rx=0 tx=0).stop 2024-04-02T13:43:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e480d640 1 -- 172.21.15.33:0/2411654764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8e010e4a0 msgr2=0x7fe8e019edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e480d640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8e010e4a0 0x7fe8e019edc0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0004af0 tx=0x7fe8d0034d70 comp rx=0 tx=0).stop 2024-04-02T13:43:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e480d640 1 -- 172.21.15.33:0/2411654764 shutdown_connections 2024-04-02T13:43:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e480d640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe8cc0a3d20 0x7fe8cc0a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e480d640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8e010e4a0 0x7fe8e019edc0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e480d640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8e0069600 0x7fe8e019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e480d640 1 --2- 172.21.15.33:0/2411654764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8e0068ce0 0x7fe8e019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e480d640 1 -- 172.21.15.33:0/2411654764 >> 172.21.15.33:0/2411654764 conn(0x7fe8e0077b00 msgr2=0x7fe8e010f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e480d640 1 -- 172.21.15.33:0/2411654764 shutdown_connections 2024-04-02T13:43:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:14.745+0000 7fe8e480d640 1 -- 172.21.15.33:0/2411654764 wait complete. 2024-04-02T13:43:14.759 INFO:tasks.cephfs.filesystem:scrub status for tag:666fc9a6-33f9-4b1f-9002-a1bcde6af77a - {'path': '/', 'tag': '666fc9a6-33f9-4b1f-9002-a1bcde6af77a', 'options': 'recursive,force'} 2024-04-02T13:43:15.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:15 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:43:15.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:15 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:15.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:15 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:43:15.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:15 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:15.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:15 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:43:15.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:15 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:15.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:15 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:43:15.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:15 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:15.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:15 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:43:15.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:15 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:15.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:15 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:43:15.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:15 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:16.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:16 smithi145 ceph-mon[35646]: pgmap v137: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T13:43:16.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:16 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:43:16.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:16 smithi050 ceph-mon[35524]: pgmap v137: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T13:43:16.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:16 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:43:16.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:16 smithi033 ceph-mon[32390]: pgmap v137: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T13:43:16.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:16 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:43:18.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:18 smithi145 ceph-mon[35646]: pgmap v138: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T13:43:18.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:18 smithi050 ceph-mon[35524]: pgmap v138: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T13:43:18.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:18 smithi033 ceph-mon[32390]: pgmap v138: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T13:43:20.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:20 smithi145 ceph-mon[35646]: pgmap v139: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T13:43:20.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:20 smithi050 ceph-mon[35524]: pgmap v139: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T13:43:20.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:20 smithi033 ceph-mon[32390]: pgmap v139: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T13:43:22.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:22 smithi145 ceph-mon[35646]: pgmap v140: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-02T13:43:22.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:22 smithi050 ceph-mon[35524]: pgmap v140: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-02T13:43:22.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:22 smithi033 ceph-mon[32390]: pgmap v140: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-02T13:43:24.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:24 smithi145 ceph-mon[35646]: pgmap v141: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T13:43:24.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:24 smithi050 ceph-mon[35524]: pgmap v141: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T13:43:24.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:24 smithi033 ceph-mon[32390]: pgmap v141: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T13:43:26.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:26 smithi145 ceph-mon[35646]: pgmap v142: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:26.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:26 smithi050 ceph-mon[35524]: pgmap v142: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:26.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:26 smithi033 ceph-mon[32390]: pgmap v142: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:28.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:28 smithi145 ceph-mon[35646]: pgmap v143: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:28.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:28 smithi050 ceph-mon[35524]: pgmap v143: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:28.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:28 smithi033 ceph-mon[32390]: pgmap v143: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:30.482 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:30 smithi145 ceph-mon[35646]: pgmap v144: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:43:30.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:30 smithi050 ceph-mon[35524]: pgmap v144: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:43:30.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:30 smithi033 ceph-mon[32390]: pgmap v144: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:43:32.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:32 smithi145 ceph-mon[35646]: pgmap v145: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:32.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:32 smithi050 ceph-mon[35524]: pgmap v145: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:32.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:32 smithi033 ceph-mon[32390]: pgmap v145: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:34.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:34 smithi145 ceph-mon[35646]: pgmap v146: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:43:34.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:34 smithi050 ceph-mon[35524]: pgmap v146: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:43:34.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:34 smithi033 ceph-mon[32390]: pgmap v146: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:43:36.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:36 smithi145 ceph-mon[35646]: pgmap v147: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:36.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:36 smithi050 ceph-mon[35524]: pgmap v147: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:36.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:36 smithi033 ceph-mon[32390]: pgmap v147: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:38.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:38 smithi145 ceph-mon[35646]: pgmap v148: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:38.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:38 smithi050 ceph-mon[35524]: pgmap v148: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:38.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:38 smithi033 ceph-mon[32390]: pgmap v148: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:40.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:40 smithi145 ceph-mon[35646]: pgmap v149: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:43:40.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:40 smithi050 ceph-mon[35524]: pgmap v149: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:43:40.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:40 smithi033 ceph-mon[32390]: pgmap v149: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:43:42.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:42 smithi145 ceph-mon[35646]: pgmap v150: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:42.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:42 smithi050 ceph-mon[35524]: pgmap v150: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:42.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:42 smithi033 ceph-mon[32390]: pgmap v150: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:44.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:44 smithi145 ceph-mon[35646]: pgmap v151: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:43:44.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:44 smithi050 ceph-mon[35524]: pgmap v151: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:43:44.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:44 smithi033 ceph-mon[32390]: pgmap v151: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:43:44.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:43:44.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:43:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.875+0000 7f6e1a83c640 1 -- 172.21.15.33:0/164503460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e14107aa0 msgr2=0x7f6e14107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.875+0000 7f6e1a83c640 1 --2- 172.21.15.33:0/164503460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e14107aa0 0x7f6e14107e80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009930 tx=0x7f6dfc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.876+0000 7f6e1a83c640 1 -- 172.21.15.33:0/164503460 shutdown_connections 2024-04-02T13:43:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.876+0000 7f6e1a83c640 1 --2- 172.21.15.33:0/164503460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e14102450 0x7f6e1410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.876+0000 7f6e1a83c640 1 --2- 172.21.15.33:0/164503460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e14101a90 0x7f6e14101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.876+0000 7f6e1a83c640 1 --2- 172.21.15.33:0/164503460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e14107aa0 0x7f6e14107e80 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.876+0000 7f6e1a83c640 1 -- 172.21.15.33:0/164503460 >> 172.21.15.33:0/164503460 conn(0x7f6e14078f00 msgr2=0x7f6e14079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.876+0000 7f6e1a83c640 1 -- 172.21.15.33:0/164503460 shutdown_connections 2024-04-02T13:43:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.876+0000 7f6e1a83c640 1 -- 172.21.15.33:0/164503460 wait complete. 2024-04-02T13:43:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.876+0000 7f6e1a83c640 1 Processor -- start 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.876+0000 7f6e1a83c640 1 -- start start 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e1a83c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e14101a90 0x7f6e141983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e1a83c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e14102450 0x7f6e141988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e1a83c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e14107aa0 0x7f6e1419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e1a83c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14111e80 con 0x7f6e14102450 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e1a83c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14111d00 con 0x7f6e14107aa0 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e1a83c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14112000 con 0x7f6e14101a90 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e14102450 0x7f6e141988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e14102450 0x7f6e141988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41484/0 (socket says 172.21.15.33:41484) 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e137fe640 1 -- 172.21.15.33:0/2828285895 learned_addr learned my addr 172.21.15.33:0/2828285895 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e13fff640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e14101a90 0x7f6e141983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e18dd9640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e14107aa0 0x7f6e1419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e137fe640 1 -- 172.21.15.33:0/2828285895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e14101a90 msgr2=0x7f6e141983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e137fe640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e14101a90 0x7f6e141983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e137fe640 1 -- 172.21.15.33:0/2828285895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e14107aa0 msgr2=0x7f6e1419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e137fe640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e14107aa0 0x7f6e1419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e137fe640 1 -- 172.21.15.33:0/2828285895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1419d390 con 0x7f6e14102450 2024-04-02T13:43:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e137fe640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e14102450 0x7f6e141988e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400e9f0 tx=0x7f6e0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e117fa640 1 -- 172.21.15.33:0/2828285895 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0400cd80 con 0x7f6e14102450 2024-04-02T13:43:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e117fa640 1 -- 172.21.15.33:0/2828285895 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6e04010900 con 0x7f6e14102450 2024-04-02T13:43:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e1a83c640 1 -- 172.21.15.33:0/2828285895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1419d680 con 0x7f6e14102450 2024-04-02T13:43:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.877+0000 7f6e117fa640 1 -- 172.21.15.33:0/2828285895 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e040189e0 con 0x7f6e14102450 2024-04-02T13:43:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.878+0000 7f6e1a83c640 1 -- 172.21.15.33:0/2828285895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e141a4f10 con 0x7f6e14102450 2024-04-02T13:43:44.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.880+0000 7f6e117fa640 1 -- 172.21.15.33:0/2828285895 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6e04010ab0 con 0x7f6e14102450 2024-04-02T13:43:44.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.880+0000 7f6e1d427740 1 Processor -- start 2024-04-02T13:43:44.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.880+0000 7f6e1d427740 1 -- start start 2024-04-02T13:43:44.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.880+0000 7f6e1d427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b9034ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.880+0000 7f6e1d427740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 0x556b90357890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.880+0000 7f6e1d427740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 0x556b9035a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e1d427740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b902b2830 con 0x556b90357dd0 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e1d427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556b9025a820 con 0x556b9034e8d0 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e1d427740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556b902ae7e0 con 0x556b9034f1f0 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e117fa640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6e000a3da0 0x7f6e000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 0x556b9035a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 0x556b9035a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41496/0 (socket says 172.21.15.33:41496) 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e137fe640 1 -- 172.21.15.33:0/3332534272 learned_addr learned my addr 172.21.15.33:0/3332534272 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e18dd9640 1 --2- 172.21.15.33:0/3332534272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b9034ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e18dd9640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6e000a3da0 0x7f6e000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e13fff640 1 --2- 172.21.15.33:0/3332534272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 0x556b90357890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e117fa640 1 -- 172.21.15.33:0/2828285895 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6e040d0e40 con 0x7f6e14102450 2024-04-02T13:43:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e137fe640 1 -- 172.21.15.33:0/3332534272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 msgr2=0x556b90357890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e137fe640 1 --2- 172.21.15.33:0/3332534272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 0x556b90357890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e137fe640 1 -- 172.21.15.33:0/3332534272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 msgr2=0x556b9034ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e137fe640 1 --2- 172.21.15.33:0/3332534272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b9034ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e137fe640 1 -- 172.21.15.33:0/3332534272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b900907d0 con 0x556b90357dd0 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e18dd9640 1 --2- 172.21.15.33:0/3332534272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b9034ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e137fe640 1 --2- 172.21.15.33:0/3332534272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 0x556b9035a1c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400e9c0 tx=0x7f6e0401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300eaeb267a48bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e18dd9640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6e000a3da0 0x7f6e000a6260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005c40 tx=0x7f6e0c009500 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.881+0000 7f6e0a7fc640 1 -- 172.21.15.33:0/3332534272 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04026040 con 0x556b90357dd0 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.882+0000 7f6e0a7fc640 1 -- 172.21.15.33:0/3332534272 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6e0409cae0 con 0x556b90357dd0 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.882+0000 7f6e0a7fc640 1 -- 172.21.15.33:0/3332534272 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0409cd40 con 0x556b90357dd0 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.882+0000 7f6e1d427740 1 -- 172.21.15.33:0/3332534272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 msgr2=0x556b9035a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.882+0000 7f6e1d427740 1 --2- 172.21.15.33:0/3332534272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 0x556b9035a1c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400e9c0 tx=0x7f6e0401f000 comp rx=0 tx=0).stop 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.882+0000 7f6e1d427740 1 -- 172.21.15.33:0/3332534272 shutdown_connections 2024-04-02T13:43:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.882+0000 7f6e1d427740 1 --2- 172.21.15.33:0/3332534272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 0x556b9035a1c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.882+0000 7f6e1d427740 1 --2- 172.21.15.33:0/3332534272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 0x556b90357890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.882+0000 7f6e1d427740 1 --2- 172.21.15.33:0/3332534272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b9034ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.882+0000 7f6e1d427740 1 -- 172.21.15.33:0/3332534272 >> 172.21.15.33:0/3332534272 conn(0x556b90349000 msgr2=0x556b903493e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.882+0000 7f6e1d427740 1 -- 172.21.15.33:0/3332534272 shutdown_connections 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.882+0000 7f6e1d427740 1 -- 172.21.15.33:0/3332534272 wait complete. 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e1d427740 1 -- start start 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e1d427740 1 Processor -- start 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e1d427740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034e8d0 0x556b90357670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e1d427740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 0x556b90354a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e1d427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b90357dd0 0x556b90354f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e1d427740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b9035ab00 con 0x556b9034e8d0 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e1d427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556b9026dd30 con 0x556b90357dd0 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e1d427740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556b9035ac80 con 0x556b9034f1f0 2024-04-02T13:43:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e13fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034e8d0 0x556b90357670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e1d427740 10 client.0 _ll_register_callbacks cb 0x556b9026d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e13fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034e8d0 0x556b90357670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41502/0 (socket says 172.21.15.33:41502) 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e13fff640 1 -- 172.21.15.33:0/592078256 learned_addr learned my addr 172.21.15.33:0/592078256 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e18dd9640 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b90357dd0 0x556b90354f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e137fe640 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 0x556b90354a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.883+0000 7f6e13fff640 1 -- 172.21.15.33:0/592078256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 msgr2=0x556b90354a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6e13fff640 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 0x556b90354a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6e13fff640 1 -- 172.21.15.33:0/592078256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b90357dd0 msgr2=0x556b90354f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6e13fff640 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b90357dd0 0x556b90354f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6e13fff640 1 -- 172.21.15.33:0/592078256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8ffc63f0 con 0x556b9034e8d0 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6e137fe640 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 0x556b90354a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6e13fff640 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034e8d0 0x556b90357670 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009230 tx=0x7f6dfc005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f539e05782ac9f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6df37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6df37fe640 1 -- 172.21.15.33:0/592078256 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc046070 con 0x556b9034e8d0 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6df37fe640 1 -- 172.21.15.33:0/592078256 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6dfc02c5d0 con 0x556b9034e8d0 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6e1d427740 1 -- 172.21.15.33:0/592078256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e0409cae0 con 0x556b9034e8d0 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6df37fe640 1 -- 172.21.15.33:0/592078256 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6dfc052050 con 0x556b9034e8d0 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6e1d427740 10 client.16653 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6e1d427740 1 -- 172.21.15.33:0/592078256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b8ff860f0 con 0x556b9034e8d0 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6df37fe640 1 -- 172.21.15.33:0/592078256 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6e0409cae0 con 0x556b9034e8d0 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6df37fe640 1 -- 172.21.15.33:0/592078256 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6dfc004ad0 con 0x556b9034e8d0 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6e1d427740 10 client.16653 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6e1d427740 10 client.16653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.884+0000 7f6e1d427740 20 client.16653 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.885+0000 7f6e1d427740 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556b90367cd0 0x556b9036a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.885+0000 7f6e1d427740 4 client.16653 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.885+0000 7f6e1d427740 1 -- 172.21.15.33:0/592078256 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b902bfd70 con 0x556b90367cd0 2024-04-02T13:43:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.885+0000 7f6e137fe640 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556b90367cd0 0x556b9036a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.885+0000 7f6e137fe640 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556b90367cd0 0x556b9036a190 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b073290bef4dc3a7 server_cookie=164169c2e807b3d9 in_seq=0 out_seq=0 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.885+0000 7f6df37fe640 10 client.16653 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6df37fe640 1 -- 172.21.15.33:0/592078256 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b902bfd70 con 0x556b90367cd0 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6df37fe640 10 client.16653 handle_command_reply: tid=0 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6e1d427740 1 client.16653 shutdown 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6e1d427740 1 -- 172.21.15.33:0/592078256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034e8d0 msgr2=0x556b90357670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6e1d427740 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034e8d0 0x556b90357670 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009230 tx=0x7f6dfc005250 comp rx=0 tx=0).stop 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6e1d427740 1 -- 172.21.15.33:0/592078256 shutdown_connections 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6e1d427740 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556b90367cd0 0x556b9036a190 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6e1d427740 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b90357dd0 0x556b90354f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6e1d427740 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 0x556b90354a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6df37fe640 0 client.16653 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6e1d427740 1 --2- 172.21.15.33:0/592078256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034e8d0 0x556b90357670 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6e1d427740 1 -- 172.21.15.33:0/592078256 >> 172.21.15.33:0/592078256 conn(0x556b90349000 msgr2=0x556b9034ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6e1d427740 1 -- 172.21.15.33:0/592078256 shutdown_connections 2024-04-02T13:43:44.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6e1d427740 1 -- 172.21.15.33:0/592078256 wait complete. 2024-04-02T13:43:44.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.886+0000 7f6e1d427740 20 client.16653 trim_cache size 0 max 16384 2024-04-02T13:43:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.904+0000 7f6e1d427740 1 Processor -- start 2024-04-02T13:43:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.904+0000 7f6e1d427740 1 -- start start 2024-04-02T13:43:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.904+0000 7f6e1d427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b903639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.904+0000 7f6e1d427740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034f1f0 0x556b9034f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.904+0000 7f6e1d427740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b90357dd0 0x556b90357130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.904+0000 7f6e1d427740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141b18c0 con 0x556b9034f1f0 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.904+0000 7f6e1d427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141af0d0 con 0x556b9034e8d0 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.904+0000 7f6e1d427740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141acf00 con 0x556b90357dd0 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.904+0000 7f6e137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034f1f0 0x556b9034f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.904+0000 7f6e137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034f1f0 0x556b9034f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41504/0 (socket says 172.21.15.33:41504) 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.904+0000 7f6e137fe640 1 -- 172.21.15.33:0/1838077285 learned_addr learned my addr 172.21.15.33:0/1838077285 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e13fff640 1 --2- 172.21.15.33:0/1838077285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b903639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e18dd9640 1 --2- 172.21.15.33:0/1838077285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b90357dd0 0x556b90357130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e137fe640 1 -- 172.21.15.33:0/1838077285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b90357dd0 msgr2=0x556b90357130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e137fe640 1 --2- 172.21.15.33:0/1838077285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b90357dd0 0x556b90357130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e137fe640 1 -- 172.21.15.33:0/1838077285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 msgr2=0x556b903639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e137fe640 1 --2- 172.21.15.33:0/1838077285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b903639f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e137fe640 1 -- 172.21.15.33:0/1838077285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b90344a30 con 0x556b9034f1f0 2024-04-02T13:43:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e137fe640 1 --2- 172.21.15.33:0/1838077285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034f1f0 0x556b9034f830 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009830 tx=0x7f6e04098d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f258734f53c7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e09ffb640 1 -- 172.21.15.33:0/1838077285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0409cae0 con 0x556b9034f1f0 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e09ffb640 1 -- 172.21.15.33:0/1838077285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6e0409ade0 con 0x556b9034f1f0 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e09ffb640 1 -- 172.21.15.33:0/1838077285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04018de0 con 0x556b9034f1f0 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e1d427740 1 -- 172.21.15.33:0/1838077285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034f1f0 msgr2=0x556b9034f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e1d427740 1 --2- 172.21.15.33:0/1838077285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034f1f0 0x556b9034f830 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009830 tx=0x7f6e04098d80 comp rx=0 tx=0).stop 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e1d427740 1 -- 172.21.15.33:0/1838077285 shutdown_connections 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e1d427740 1 --2- 172.21.15.33:0/1838077285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b90357dd0 0x556b90357130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e1d427740 1 --2- 172.21.15.33:0/1838077285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b9034f1f0 0x556b9034f830 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.905+0000 7f6e1d427740 1 --2- 172.21.15.33:0/1838077285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b903639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e1d427740 1 -- 172.21.15.33:0/1838077285 >> 172.21.15.33:0/1838077285 conn(0x556b90349000 msgr2=0x556b90348240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e1d427740 1 -- 172.21.15.33:0/1838077285 shutdown_connections 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e1d427740 1 -- 172.21.15.33:0/1838077285 wait complete. 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e1d427740 1 -- start start 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e1d427740 1 Processor -- start 2024-04-02T13:43:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e1d427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b90350c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e1d427740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 0x556b903511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e1d427740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 0x556b903516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e1d427740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141ad590 con 0x556b90357dd0 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e1d427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141aea00 con 0x556b9034e8d0 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e1d427740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141adc60 con 0x556b9034f1f0 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e1d427740 10 client.0 _ll_register_callbacks cb 0x7f6dfc004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e18dd9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 0x556b903516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e18dd9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 0x556b903516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41512/0 (socket says 172.21.15.33:41512) 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.906+0000 7f6e18dd9640 1 -- 172.21.15.33:0/1852332766 learned_addr learned my addr 172.21.15.33:0/1852332766 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6e13fff640 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b90350c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6e137fe640 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 0x556b903511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6e18dd9640 1 -- 172.21.15.33:0/1852332766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 msgr2=0x556b903511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6e18dd9640 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 0x556b903511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6e18dd9640 1 -- 172.21.15.33:0/1852332766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 msgr2=0x556b90350c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:44.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6e18dd9640 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b90350c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6e18dd9640 1 -- 172.21.15.33:0/1852332766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b90351ce0 con 0x556b90357dd0 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6e13fff640 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b90350c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6e18dd9640 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 0x556b903516f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005fd0 tx=0x7f6e0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc23b9deaa73a5a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6df37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6df37fe640 1 -- 172.21.15.33:0/1852332766 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c019040 con 0x556b90357dd0 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6df37fe640 1 -- 172.21.15.33:0/1852332766 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6e0c010dc0 con 0x556b90357dd0 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6e1d427740 1 -- 172.21.15.33:0/1852332766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e0409ade0 con 0x556b90357dd0 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6df37fe640 1 -- 172.21.15.33:0/1852332766 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6e0c036050 con 0x556b90357dd0 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6e1d427740 10 client.16659 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.907+0000 7f6e1d427740 1 -- 172.21.15.33:0/1852332766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b903d26d0 con 0x556b90357dd0 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.908+0000 7f6df37fe640 1 -- 172.21.15.33:0/1852332766 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6e0409ade0 con 0x556b90357dd0 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.908+0000 7f6df37fe640 1 -- 172.21.15.33:0/1852332766 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6e0c01da00 con 0x556b90357dd0 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.908+0000 7f6e1d427740 10 client.16659 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.908+0000 7f6e1d427740 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.908+0000 7f6e1d427740 20 client.16659 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.908+0000 7f6e1d427740 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556b903d2f30 0x556b903d5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.908+0000 7f6e1d427740 4 client.16659 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.908+0000 7f6e1d427740 1 -- 172.21.15.33:0/1852332766 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b902ba0c0 con 0x556b903d2f30 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.908+0000 7f6e13fff640 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556b903d2f30 0x556b903d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6df37fe640 10 client.16659 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6e13fff640 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556b903d2f30 0x556b903d5350 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce29cb70fb4fbe4 server_cookie=cc137f9677f046d3 in_seq=0 out_seq=0 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6df37fe640 1 -- 172.21.15.33:0/1852332766 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556b902ba0c0 con 0x556b903d2f30 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6df37fe640 10 client.16659 handle_command_reply: tid=0 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6e1d427740 1 client.16659 shutdown 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6e1d427740 1 -- 172.21.15.33:0/1852332766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 msgr2=0x556b903516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6e1d427740 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 0x556b903516f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005fd0 tx=0x7f6e0c004a40 comp rx=0 tx=0).stop 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6e1d427740 1 -- 172.21.15.33:0/1852332766 shutdown_connections 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6e1d427740 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x556b903d2f30 0x556b903d5350 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6e1d427740 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b90357dd0 0x556b903516f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6e1d427740 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556b9034f1f0 0x556b903511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6df37fe640 0 client.16659 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:44.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6e1d427740 1 --2- 172.21.15.33:0/1852332766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556b9034e8d0 0x556b90350c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6e1d427740 1 -- 172.21.15.33:0/1852332766 >> 172.21.15.33:0/1852332766 conn(0x556b90349000 msgr2=0x556b903479b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:44.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6e1d427740 1 -- 172.21.15.33:0/1852332766 shutdown_connections 2024-04-02T13:43:44.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6e1d427740 1 -- 172.21.15.33:0/1852332766 wait complete. 2024-04-02T13:43:44.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.909+0000 7f6e1d427740 20 client.16659 trim_cache size 0 max 16384 2024-04-02T13:43:44.919 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:43:44.919 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:43:44.919 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:43:44.919 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:43:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.910+0000 7f6e1a83c640 1 -- 172.21.15.33:0/2828285895 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6e000a3da0 msgr2=0x7f6e000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.910+0000 7f6e1a83c640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6e000a3da0 0x7f6e000a6260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005c40 tx=0x7f6e0c009500 comp rx=0 tx=0).stop 2024-04-02T13:43:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.910+0000 7f6e1a83c640 1 -- 172.21.15.33:0/2828285895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e14102450 msgr2=0x7f6e141988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.910+0000 7f6e1a83c640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e14102450 0x7f6e141988e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400e9f0 tx=0x7f6e0400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.910+0000 7f6e1a83c640 1 -- 172.21.15.33:0/2828285895 shutdown_connections 2024-04-02T13:43:44.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.910+0000 7f6e1a83c640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f6e000a3da0 0x7f6e000a6260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.910+0000 7f6e1a83c640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e14107aa0 0x7f6e1419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.910+0000 7f6e1a83c640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e14102450 0x7f6e141988e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.910+0000 7f6e1a83c640 1 --2- 172.21.15.33:0/2828285895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e14101a90 0x7f6e141983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.910+0000 7f6e1a83c640 1 -- 172.21.15.33:0/2828285895 >> 172.21.15.33:0/2828285895 conn(0x7f6e14078f00 msgr2=0x7f6e141044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:44.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.910+0000 7f6e1a83c640 1 -- 172.21.15.33:0/2828285895 shutdown_connections 2024-04-02T13:43:44.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:44.910+0000 7f6e1a83c640 1 -- 172.21.15.33:0/2828285895 wait complete. 2024-04-02T13:43:44.924 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:43:44.925 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:43:45.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.024+0000 7f2c0a463640 1 -- 172.21.15.33:0/595700787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c04107b00 msgr2=0x7f2c04107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.024+0000 7f2c0a463640 1 --2- 172.21.15.33:0/595700787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c04107b00 0x7f2c04107ee0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009900 tx=0x7f2bf402bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:45.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.025+0000 7f2c0a463640 1 -- 172.21.15.33:0/595700787 shutdown_connections 2024-04-02T13:43:45.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.025+0000 7f2c0a463640 1 --2- 172.21.15.33:0/595700787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c0406aaf0 0x7f2c0410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.025+0000 7f2c0a463640 1 --2- 172.21.15.33:0/595700787 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c0406a150 0x7f2c0406a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.025+0000 7f2c0a463640 1 --2- 172.21.15.33:0/595700787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c04107b00 0x7f2c04107ee0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.025+0000 7f2c0a463640 1 -- 172.21.15.33:0/595700787 >> 172.21.15.33:0/595700787 conn(0x7f2c04076800 msgr2=0x7f2c04076c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.025+0000 7f2c0a463640 1 -- 172.21.15.33:0/595700787 shutdown_connections 2024-04-02T13:43:45.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.025+0000 7f2c0a463640 1 -- 172.21.15.33:0/595700787 wait complete. 2024-04-02T13:43:45.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.025+0000 7f2c0a463640 1 Processor -- start 2024-04-02T13:43:45.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.025+0000 7f2c0a463640 1 -- start start 2024-04-02T13:43:45.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c0a463640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c0406a150 0x7f2c04100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c0a463640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c0406aaf0 0x7f2c04101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c0a463640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c04107b00 0x7f2c041018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c0a463640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c04111eb0 con 0x7f2c0406aaf0 2024-04-02T13:43:45.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c0a463640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2c04111d30 con 0x7f2c0406a150 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c0a463640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2c04112030 con 0x7f2c04107b00 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c0406aaf0 0x7f2c04101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c0406aaf0 0x7f2c04101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41530/0 (socket says 172.21.15.33:41530) 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c037fe640 1 -- 172.21.15.33:0/724484843 learned_addr learned my addr 172.21.15.33:0/724484843 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c03fff640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c0406a150 0x7f2c04100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c08a00640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c04107b00 0x7f2c041018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c037fe640 1 -- 172.21.15.33:0/724484843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c04107b00 msgr2=0x7f2c041018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c037fe640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c04107b00 0x7f2c041018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c037fe640 1 -- 172.21.15.33:0/724484843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c0406a150 msgr2=0x7f2c04100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c037fe640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c0406a150 0x7f2c04100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c037fe640 1 -- 172.21.15.33:0/724484843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c041a6830 con 0x7f2c0406aaf0 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c037fe640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c0406aaf0 0x7f2c04101360 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800d950 tx=0x7f2bf800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c017fa640 1 -- 172.21.15.33:0/724484843 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8017030 con 0x7f2c0406aaf0 2024-04-02T13:43:45.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c0a463640 1 -- 172.21.15.33:0/724484843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c041a6b20 con 0x7f2c0406aaf0 2024-04-02T13:43:45.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c017fa640 1 -- 172.21.15.33:0/724484843 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2bf8004020 con 0x7f2c0406aaf0 2024-04-02T13:43:45.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c017fa640 1 -- 172.21.15.33:0/724484843 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80047a0 con 0x7f2c0406aaf0 2024-04-02T13:43:45.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.026+0000 7f2c0a463640 1 -- 172.21.15.33:0/724484843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c041a70c0 con 0x7f2c0406aaf0 2024-04-02T13:43:45.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.027+0000 7f2c0a463640 1 -- 172.21.15.33:0/724484843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c041031c0 con 0x7f2c0406aaf0 2024-04-02T13:43:45.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.028+0000 7f2c017fa640 1 -- 172.21.15.33:0/724484843 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2bf80041d0 con 0x7f2c0406aaf0 2024-04-02T13:43:45.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.029+0000 7f2c017fa640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2bec0a3d20 0x7f2bec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.029+0000 7f2c03fff640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2bec0a3d20 0x7f2bec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.029+0000 7f2c017fa640 1 -- 172.21.15.33:0/724484843 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2bf80d05b0 con 0x7f2c0406aaf0 2024-04-02T13:43:45.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.030+0000 7f2c03fff640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2bec0a3d20 0x7f2bec0a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4006010 tx=0x7f2bf4005dd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.034+0000 7f2c017fa640 1 -- 172.21.15.33:0/724484843 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2bf809c8e0 con 0x7f2c0406aaf0 2024-04-02T13:43:45.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.263+0000 7f2c0a463640 1 -- 172.21.15.33:0/724484843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c0402d370 con 0x7f2c0406aaf0 2024-04-02T13:43:45.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.264+0000 7f2c017fa640 1 -- 172.21.15.33:0/724484843 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11882 (secure 0 0 0) 0x7f2bf809c1e0 con 0x7f2c0406aaf0 2024-04-02T13:43:45.268 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:43:45.268 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:41:39.222860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:43:45.268 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-02T13:43:45.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.266+0000 7f2c0a463640 1 -- 172.21.15.33:0/724484843 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2bec0a3d20 msgr2=0x7f2bec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.266+0000 7f2c0a463640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2bec0a3d20 0x7f2bec0a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4006010 tx=0x7f2bf4005dd0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.266+0000 7f2c0a463640 1 -- 172.21.15.33:0/724484843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c0406aaf0 msgr2=0x7f2c04101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.266+0000 7f2c0a463640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c0406aaf0 0x7f2c04101360 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800d950 tx=0x7f2bf800de10 comp rx=0 tx=0).stop 2024-04-02T13:43:45.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.266+0000 7f2c0a463640 1 -- 172.21.15.33:0/724484843 shutdown_connections 2024-04-02T13:43:45.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.266+0000 7f2c0a463640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2bec0a3d20 0x7f2bec0a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.266+0000 7f2c0a463640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c04107b00 0x7f2c041018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.266+0000 7f2c0a463640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c0406aaf0 0x7f2c04101360 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.266+0000 7f2c0a463640 1 --2- 172.21.15.33:0/724484843 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2c0406a150 0x7f2c04100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.266+0000 7f2c0a463640 1 -- 172.21.15.33:0/724484843 >> 172.21.15.33:0/724484843 conn(0x7f2c04076800 msgr2=0x7f2c040fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.266+0000 7f2c0a463640 1 -- 172.21.15.33:0/724484843 shutdown_connections 2024-04-02T13:43:45.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.266+0000 7f2c0a463640 1 -- 172.21.15.33:0/724484843 wait complete. 2024-04-02T13:43:45.281 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:43:45.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.382+0000 7f7804baf640 1 -- 172.21.15.33:0/3776313930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800100ce0 msgr2=0x7f7800101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.382+0000 7f7804baf640 1 --2- 172.21.15.33:0/3776313930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800100ce0 0x7f7800101140 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009900 tx=0x7f77e802bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:45.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.382+0000 7f7804baf640 1 -- 172.21.15.33:0/3776313930 shutdown_connections 2024-04-02T13:43:45.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.382+0000 7f7804baf640 1 --2- 172.21.15.33:0/3776313930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7800101680 0x7f780010d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.382+0000 7f7804baf640 1 --2- 172.21.15.33:0/3776313930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800100ce0 0x7f7800101140 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.382+0000 7f7804baf640 1 --2- 172.21.15.33:0/3776313930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7800106a30 0x7f7800106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.382+0000 7f7804baf640 1 -- 172.21.15.33:0/3776313930 >> 172.21.15.33:0/3776313930 conn(0x7f78000777c0 msgr2=0x7f7800077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.382+0000 7f7804baf640 1 -- 172.21.15.33:0/3776313930 shutdown_connections 2024-04-02T13:43:45.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.382+0000 7f7804baf640 1 -- 172.21.15.33:0/3776313930 wait complete. 2024-04-02T13:43:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f7804baf640 1 Processor -- start 2024-04-02T13:43:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f7804baf640 1 -- start start 2024-04-02T13:43:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f7804baf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800100ce0 0x7f78001990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f7804baf640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7800101680 0x7f78001995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f7804baf640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7800106a30 0x7f78001a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f7804baf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7800110d40 con 0x7f7800100ce0 2024-04-02T13:43:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f7804baf640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7800110bc0 con 0x7f7800106a30 2024-04-02T13:43:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f7804baf640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7800110ec0 con 0x7f7800101680 2024-04-02T13:43:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f77fe59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800100ce0 0x7f78001990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f77fe59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800100ce0 0x7f78001990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41556/0 (socket says 172.21.15.33:41556) 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f77fe59c640 1 -- 172.21.15.33:0/3789923948 learned_addr learned my addr 172.21.15.33:0/3789923948 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f77f7fff640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7800101680 0x7f78001995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f77fed9d640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7800106a30 0x7f78001a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f77fe59c640 1 -- 172.21.15.33:0/3789923948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7800101680 msgr2=0x7f78001995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f77fe59c640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7800101680 0x7f78001995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f77fe59c640 1 -- 172.21.15.33:0/3789923948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7800106a30 msgr2=0x7f78001a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f77fe59c640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7800106a30 0x7f78001a4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f77fe59c640 1 -- 172.21.15.33:0/3789923948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78001a4660 con 0x7f7800100ce0 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.383+0000 7f77fed9d640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7800106a30 0x7f78001a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.384+0000 7f77fe59c640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800100ce0 0x7f78001990a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f77f000d770 tx=0x7f77f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.384+0000 7f77f77fe640 1 -- 172.21.15.33:0/3789923948 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0017070 con 0x7f7800100ce0 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.384+0000 7f7804baf640 1 -- 172.21.15.33:0/3789923948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78001a4830 con 0x7f7800100ce0 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.384+0000 7f77f77fe640 1 -- 172.21.15.33:0/3789923948 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f77f00054f0 con 0x7f7800100ce0 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.384+0000 7f7804baf640 1 -- 172.21.15.33:0/3789923948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78001a4d50 con 0x7f7800100ce0 2024-04-02T13:43:45.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.384+0000 7f77f77fe640 1 -- 172.21.15.33:0/3789923948 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0020060 con 0x7f7800100ce0 2024-04-02T13:43:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.386+0000 7f77f77fe640 1 -- 172.21.15.33:0/3789923948 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f77f0004d00 con 0x7f7800100ce0 2024-04-02T13:43:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.386+0000 7f7807798740 1 Processor -- start 2024-04-02T13:43:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.386+0000 7f7807798740 1 -- start start 2024-04-02T13:43:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f7807798740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 0x560adbb128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f7807798740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb1fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f7807798740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f7807798740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560adba767b0 con 0x560adbb200b0 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f7807798740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560adba15570 con 0x560adbb12df0 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f7807798740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560adba72760 con 0x560adbb124d0 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77f77fe640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f77ec0a3d20 0x7f77ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb224a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41558/0 (socket says 172.21.15.33:41558) 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77f7fff640 1 -- 172.21.15.33:0/1716096616 learned_addr learned my addr 172.21.15.33:0/1716096616 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77fed9d640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f77ec0a3d20 0x7f77ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77fe59c640 1 --2- 172.21.15.33:0/1716096616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb1fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77fed9d640 1 --2- 172.21.15.33:0/1716096616 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 0x560adbb128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77f77fe640 1 -- 172.21.15.33:0/3789923948 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f77f00d0890 con 0x7f7800100ce0 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77f7fff640 1 -- 172.21.15.33:0/1716096616 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 msgr2=0x560adbb128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77f7fff640 1 --2- 172.21.15.33:0/1716096616 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 0x560adbb128b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77f7fff640 1 -- 172.21.15.33:0/1716096616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 msgr2=0x560adbb1fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77f7fff640 1 --2- 172.21.15.33:0/1716096616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb1fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77f7fff640 1 -- 172.21.15.33:0/1716096616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560adb78a3f0 con 0x560adbb200b0 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77fed9d640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f77ec0a3d20 0x7f77ec0a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005fd0 tx=0x7f77f800ad30 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77f7fff640 1 --2- 172.21.15.33:0/1716096616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb224a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009900 tx=0x7f77e8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15b883dbb30311a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.387+0000 7f77f4ff9640 1 -- 172.21.15.33:0/1716096616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8045020 con 0x560adbb200b0 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.388+0000 7f77f4ff9640 1 -- 172.21.15.33:0/1716096616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f77e8005510 con 0x560adbb200b0 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.388+0000 7f77f4ff9640 1 -- 172.21.15.33:0/1716096616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e80046c0 con 0x560adbb200b0 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.388+0000 7f7807798740 1 -- 172.21.15.33:0/1716096616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 msgr2=0x560adbb224a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.388+0000 7f7807798740 1 --2- 172.21.15.33:0/1716096616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb224a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009900 tx=0x7f77e8005c00 comp rx=0 tx=0).stop 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.388+0000 7f7807798740 1 -- 172.21.15.33:0/1716096616 shutdown_connections 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.388+0000 7f7807798740 1 --2- 172.21.15.33:0/1716096616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb224a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.388+0000 7f7807798740 1 --2- 172.21.15.33:0/1716096616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb1fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.388+0000 7f7807798740 1 --2- 172.21.15.33:0/1716096616 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 0x560adbb128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.388+0000 7f7807798740 1 -- 172.21.15.33:0/1716096616 >> 172.21.15.33:0/1716096616 conn(0x560adbb0ced0 msgr2=0x560adbb0d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.388+0000 7f7807798740 1 -- 172.21.15.33:0/1716096616 shutdown_connections 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.388+0000 7f7807798740 1 -- 172.21.15.33:0/1716096616 wait complete. 2024-04-02T13:43:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f7807798740 1 -- start start 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f7807798740 1 Processor -- start 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f7807798740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb124d0 0x560adbb1fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f7807798740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb1ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f7807798740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb200b0 0x560adbb1d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f7807798740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560adbb22e10 con 0x560adbb124d0 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f7807798740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560adba31e80 con 0x560adbb12df0 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f7807798740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560adbb22f90 con 0x560adbb200b0 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f77fe59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb124d0 0x560adbb1fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f7807798740 10 client.0 _ll_register_callbacks cb 0x560adba6daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f77fe59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb124d0 0x560adbb1fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41562/0 (socket says 172.21.15.33:41562) 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f77fe59c640 1 -- 172.21.15.33:0/1395946659 learned_addr learned my addr 172.21.15.33:0/1395946659 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f77f7fff640 1 --2- 172.21.15.33:0/1395946659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb1ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f77fed9d640 1 --2- 172.21.15.33:0/1395946659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb200b0 0x560adbb1d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f77fe59c640 1 -- 172.21.15.33:0/1395946659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb200b0 msgr2=0x560adbb1d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f77fe59c640 1 --2- 172.21.15.33:0/1395946659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb200b0 0x560adbb1d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f77fe59c640 1 -- 172.21.15.33:0/1395946659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 msgr2=0x560adbb1ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f77fe59c640 1 --2- 172.21.15.33:0/1395946659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb1ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.389+0000 7f77fe59c640 1 -- 172.21.15.33:0/1395946659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560adb8547d0 con 0x560adbb124d0 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f77fe59c640 1 --2- 172.21.15.33:0/1395946659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb124d0 0x560adbb1fae0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f77f000d740 tx=0x7f77f0017400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04c4e44021f3447 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f77e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f77e5ffb640 1 -- 172.21.15.33:0/1395946659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0009cc0 con 0x560adbb124d0 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f77e5ffb640 1 -- 172.21.15.33:0/1395946659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f77f00cdef0 con 0x560adbb124d0 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f7807798740 1 -- 172.21.15.33:0/1395946659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77e8005510 con 0x560adbb124d0 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f77e5ffb640 1 -- 172.21.15.33:0/1395946659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f77f00ce0f0 con 0x560adbb124d0 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f7807798740 10 client.16677 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f7807798740 1 -- 172.21.15.33:0/1395946659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560adbb2eec0 con 0x560adbb124d0 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f77e5ffb640 1 -- 172.21.15.33:0/1395946659 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f77e8005510 con 0x560adbb124d0 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f77e5ffb640 1 -- 172.21.15.33:0/1395946659 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f77f009c720 con 0x560adbb124d0 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f7807798740 10 client.16677 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f7807798740 10 client.16677 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f7807798740 20 client.16677 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f7807798740 1 --2- 172.21.15.33:0/1395946659 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560adbb30380 0x560adbb32810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f7807798740 4 client.16677 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.390+0000 7f7807798740 1 -- 172.21.15.33:0/1395946659 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560adba83cf0 con 0x560adbb30380 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.391+0000 7f77f7fff640 1 --2- 172.21.15.33:0/1395946659 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560adbb30380 0x560adbb32810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.391+0000 7f77f7fff640 1 --2- 172.21.15.33:0/1395946659 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560adbb30380 0x560adbb32810 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2370758a4aa83f8a server_cookie=cd27fb8143451937 in_seq=0 out_seq=0 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.391+0000 7f77e5ffb640 10 client.16677 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f77e5ffb640 1 -- 172.21.15.33:0/1395946659 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560adba83cf0 con 0x560adbb30380 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f77e5ffb640 10 client.16677 handle_command_reply: tid=0 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f7807798740 1 client.16677 shutdown 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f7807798740 1 -- 172.21.15.33:0/1395946659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb124d0 msgr2=0x560adbb1fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f7807798740 1 --2- 172.21.15.33:0/1395946659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb124d0 0x560adbb1fae0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f77f000d740 tx=0x7f77f0017400 comp rx=0 tx=0).stop 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f7807798740 1 -- 172.21.15.33:0/1395946659 shutdown_connections 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f7807798740 1 --2- 172.21.15.33:0/1395946659 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560adbb30380 0x560adbb32810 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f7807798740 1 --2- 172.21.15.33:0/1395946659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb200b0 0x560adbb1d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f7807798740 1 --2- 172.21.15.33:0/1395946659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb1ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f7807798740 1 --2- 172.21.15.33:0/1395946659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb124d0 0x560adbb1fae0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f77e5ffb640 0 client.16677 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:43:45.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f7807798740 1 -- 172.21.15.33:0/1395946659 >> 172.21.15.33:0/1395946659 conn(0x560adbb0ced0 msgr2=0x560adbb0ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f7807798740 1 -- 172.21.15.33:0/1395946659 shutdown_connections 2024-04-02T13:43:45.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f7807798740 1 -- 172.21.15.33:0/1395946659 wait complete. 2024-04-02T13:43:45.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.392+0000 7f7807798740 20 client.16677 trim_cache size 0 max 16384 2024-04-02T13:43:45.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.405+0000 7f7807798740 1 Processor -- start 2024-04-02T13:43:45.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.405+0000 7f7807798740 1 -- start start 2024-04-02T13:43:45.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f7807798740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 0x560adbb0ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f7807798740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f7807798740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb38620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f7807798740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f78001af480 con 0x560adbb200b0 2024-04-02T13:43:45.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f7807798740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78001acc90 con 0x560adbb12df0 2024-04-02T13:43:45.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f7807798740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f78001aaac0 con 0x560adbb124d0 2024-04-02T13:43:45.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f77fed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb38620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f77fed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb38620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41570/0 (socket says 172.21.15.33:41570) 2024-04-02T13:43:45.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f77fed9d640 1 -- 172.21.15.33:0/1565766067 learned_addr learned my addr 172.21.15.33:0/1565766067 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f77f7fff640 1 --2- 172.21.15.33:0/1565766067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f77fe59c640 1 --2- 172.21.15.33:0/1565766067 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 0x560adbb0ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f77fed9d640 1 -- 172.21.15.33:0/1565766067 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 msgr2=0x560adbb0ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f77fed9d640 1 --2- 172.21.15.33:0/1565766067 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 0x560adbb0ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f77fed9d640 1 -- 172.21.15.33:0/1565766067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 msgr2=0x560adbb380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f77fed9d640 1 --2- 172.21.15.33:0/1565766067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb380e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f77fed9d640 1 -- 172.21.15.33:0/1565766067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560adbb0e100 con 0x560adbb200b0 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f77fed9d640 1 --2- 172.21.15.33:0/1565766067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb38620 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f77f800a880 tx=0x7f77f801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a261c3edf788208d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f77e7fff640 1 -- 172.21.15.33:0/1565766067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f802d070 con 0x560adbb200b0 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.406+0000 7f77e7fff640 1 -- 172.21.15.33:0/1565766067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f77f8020d90 con 0x560adbb200b0 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.407+0000 7f77e7fff640 1 -- 172.21.15.33:0/1565766067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f801e770 con 0x560adbb200b0 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.407+0000 7f7807798740 1 -- 172.21.15.33:0/1565766067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 msgr2=0x560adbb38620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.407+0000 7f7807798740 1 --2- 172.21.15.33:0/1565766067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb38620 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f77f800a880 tx=0x7f77f801c000 comp rx=0 tx=0).stop 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.407+0000 7f7807798740 1 -- 172.21.15.33:0/1565766067 shutdown_connections 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.407+0000 7f7807798740 1 --2- 172.21.15.33:0/1565766067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb38620 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.407+0000 7f7807798740 1 --2- 172.21.15.33:0/1565766067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.407+0000 7f7807798740 1 --2- 172.21.15.33:0/1565766067 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 0x560adbb0ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.407+0000 7f7807798740 1 -- 172.21.15.33:0/1565766067 >> 172.21.15.33:0/1565766067 conn(0x560adbb0ced0 msgr2=0x560adbb0e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.407+0000 7f7807798740 1 -- 172.21.15.33:0/1565766067 shutdown_connections 2024-04-02T13:43:45.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.407+0000 7f7807798740 1 -- 172.21.15.33:0/1565766067 wait complete. 2024-04-02T13:43:45.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.407+0000 7f7807798740 1 -- start start 2024-04-02T13:43:45.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.407+0000 7f7807798740 1 Processor -- start 2024-04-02T13:43:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f7807798740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 0x560adbb211b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f7807798740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb216f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f7807798740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb21c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f7807798740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f78001ab150 con 0x560adbb200b0 2024-04-02T13:43:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f7807798740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78001ac5c0 con 0x560adbb12df0 2024-04-02T13:43:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f7807798740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f78001ab820 con 0x560adbb124d0 2024-04-02T13:43:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f7807798740 10 client.0 _ll_register_callbacks cb 0x7f77f0016fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77fed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb21c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77fed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb21c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41582/0 (socket says 172.21.15.33:41582) 2024-04-02T13:43:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77fed9d640 1 -- 172.21.15.33:0/437023996 learned_addr learned my addr 172.21.15.33:0/437023996 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77fe59c640 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 0x560adbb211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77f7fff640 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb216f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77fed9d640 1 -- 172.21.15.33:0/437023996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 msgr2=0x560adbb211b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77fed9d640 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 0x560adbb211b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77fed9d640 1 -- 172.21.15.33:0/437023996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 msgr2=0x560adbb216f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77fed9d640 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb216f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77fed9d640 1 -- 172.21.15.33:0/437023996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560adbb268c0 con 0x560adbb200b0 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77f7fff640 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb216f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77fed9d640 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb21c30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f77f802b040 tx=0x7f77f8012490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a2b589cbf21b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77e5ffb640 1 -- 172.21.15.33:0/437023996 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f802d040 con 0x560adbb200b0 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f77e5ffb640 1 -- 172.21.15.33:0/437023996 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f77f8028600 con 0x560adbb200b0 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.408+0000 7f7807798740 1 -- 172.21.15.33:0/437023996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77f8020d90 con 0x560adbb200b0 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.409+0000 7f77e5ffb640 1 -- 172.21.15.33:0/437023996 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f77f8042050 con 0x560adbb200b0 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.409+0000 7f7807798740 10 client.16683 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.409+0000 7f7807798740 1 -- 172.21.15.33:0/437023996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560adbb96ee0 con 0x560adbb200b0 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.409+0000 7f77e5ffb640 1 -- 172.21.15.33:0/437023996 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f77f8020d90 con 0x560adbb200b0 2024-04-02T13:43:45.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.409+0000 7f77e5ffb640 1 -- 172.21.15.33:0/437023996 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f77f8027030 con 0x560adbb200b0 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.409+0000 7f7807798740 10 client.16683 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.409+0000 7f7807798740 10 client.16683 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.409+0000 7f7807798740 20 client.16683 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.409+0000 7f7807798740 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560adbb97630 0x560adbb99a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.409+0000 7f7807798740 4 client.16683 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.409+0000 7f7807798740 1 -- 172.21.15.33:0/437023996 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560adba7e040 con 0x560adbb97630 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.409+0000 7f77fe59c640 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560adbb97630 0x560adbb99a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f77e5ffb640 10 client.16683 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f77fe59c640 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560adbb97630 0x560adbb99a50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a9e4fd7149f6c49 server_cookie=ebb09d67819d3a50 in_seq=0 out_seq=0 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f77e5ffb640 1 -- 172.21.15.33:0/437023996 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560adba7e040 con 0x560adbb97630 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f77e5ffb640 10 client.16683 handle_command_reply: tid=0 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f7807798740 1 client.16683 shutdown 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f7807798740 1 -- 172.21.15.33:0/437023996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 msgr2=0x560adbb21c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f7807798740 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb21c30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f77f802b040 tx=0x7f77f8012490 comp rx=0 tx=0).stop 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f7807798740 1 -- 172.21.15.33:0/437023996 shutdown_connections 2024-04-02T13:43:45.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f7807798740 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560adbb97630 0x560adbb99a50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f7807798740 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560adbb200b0 0x560adbb21c30 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f7807798740 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560adbb12df0 0x560adbb216f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f77e5ffb640 0 client.16683 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:43:45.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f7807798740 1 --2- 172.21.15.33:0/437023996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560adbb124d0 0x560adbb211b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f7807798740 1 -- 172.21.15.33:0/437023996 >> 172.21.15.33:0/437023996 conn(0x560adbb0ced0 msgr2=0x560adbb0eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f7807798740 1 -- 172.21.15.33:0/437023996 shutdown_connections 2024-04-02T13:43:45.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f7807798740 1 -- 172.21.15.33:0/437023996 wait complete. 2024-04-02T13:43:45.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.410+0000 7f7807798740 20 client.16683 trim_cache size 0 max 16384 2024-04-02T13:43:45.420 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:43:45.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.411+0000 7f7804baf640 1 -- 172.21.15.33:0/3789923948 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f77ec0a3d20 msgr2=0x7f77ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.411+0000 7f7804baf640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f77ec0a3d20 0x7f77ec0a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005fd0 tx=0x7f77f800ad30 comp rx=0 tx=0).stop 2024-04-02T13:43:45.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.411+0000 7f7804baf640 1 -- 172.21.15.33:0/3789923948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800100ce0 msgr2=0x7f78001990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.411+0000 7f7804baf640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800100ce0 0x7f78001990a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f77f000d770 tx=0x7f77f000dc30 comp rx=0 tx=0).stop 2024-04-02T13:43:45.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.411+0000 7f7804baf640 1 -- 172.21.15.33:0/3789923948 shutdown_connections 2024-04-02T13:43:45.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.411+0000 7f7804baf640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f77ec0a3d20 0x7f77ec0a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.411+0000 7f7804baf640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7800106a30 0x7f78001a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.411+0000 7f7804baf640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7800101680 0x7f78001995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.411+0000 7f7804baf640 1 --2- 172.21.15.33:0/3789923948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800100ce0 0x7f78001990a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.411+0000 7f7804baf640 1 -- 172.21.15.33:0/3789923948 >> 172.21.15.33:0/3789923948 conn(0x7f78000777c0 msgr2=0x7f78000ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.411+0000 7f7804baf640 1 -- 172.21.15.33:0/3789923948 shutdown_connections 2024-04-02T13:43:45.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.411+0000 7f7804baf640 1 -- 172.21.15.33:0/3789923948 wait complete. 2024-04-02T13:43:45.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:43:45.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.527+0000 7fc30a4ba640 1 -- 172.21.15.33:0/4242361411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc304100970 msgr2=0x7fc304100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.527+0000 7fc30a4ba640 1 --2- 172.21.15.33:0/4242361411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc304100970 0x7fc304100df0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009930 tx=0x7fc2f802bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:45.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.527+0000 7fc30a4ba640 1 -- 172.21.15.33:0/4242361411 shutdown_connections 2024-04-02T13:43:45.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.527+0000 7fc30a4ba640 1 --2- 172.21.15.33:0/4242361411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc304101330 0x7fc30410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.527+0000 7fc30a4ba640 1 --2- 172.21.15.33:0/4242361411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc304100970 0x7fc304100df0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.527+0000 7fc30a4ba640 1 --2- 172.21.15.33:0/4242361411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc304106980 0x7fc304106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.527+0000 7fc30a4ba640 1 -- 172.21.15.33:0/4242361411 >> 172.21.15.33:0/4242361411 conn(0x7fc304077b00 msgr2=0x7fc304077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.528+0000 7fc30a4ba640 1 -- 172.21.15.33:0/4242361411 shutdown_connections 2024-04-02T13:43:45.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.528+0000 7fc30a4ba640 1 -- 172.21.15.33:0/4242361411 wait complete. 2024-04-02T13:43:45.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.528+0000 7fc30a4ba640 1 Processor -- start 2024-04-02T13:43:45.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.528+0000 7fc30a4ba640 1 -- start start 2024-04-02T13:43:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.528+0000 7fc30a4ba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc304100970 0x7fc304198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.528+0000 7fc30a4ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc304101330 0x7fc3041988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.528+0000 7fc30a4ba640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc304106980 0x7fc30419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.528+0000 7fc30a4ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc304111e60 con 0x7fc304101330 2024-04-02T13:43:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.528+0000 7fc30a4ba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc304111ce0 con 0x7fc304100970 2024-04-02T13:43:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.528+0000 7fc30a4ba640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc304111fe0 con 0x7fc304106980 2024-04-02T13:43:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc3037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc304101330 0x7fc3041988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc3037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc304101330 0x7fc3041988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41610/0 (socket says 172.21.15.33:41610) 2024-04-02T13:43:45.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc3037fe640 1 -- 172.21.15.33:0/1105464970 learned_addr learned my addr 172.21.15.33:0/1105464970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc303fff640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc304100970 0x7fc304198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc308a57640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc304106980 0x7fc30419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc3037fe640 1 -- 172.21.15.33:0/1105464970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc304106980 msgr2=0x7fc30419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc3037fe640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc304106980 0x7fc30419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc3037fe640 1 -- 172.21.15.33:0/1105464970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc304100970 msgr2=0x7fc304198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc3037fe640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc304100970 0x7fc304198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc3037fe640 1 -- 172.21.15.33:0/1105464970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30419d320 con 0x7fc304101330 2024-04-02T13:43:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc3037fe640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc304101330 0x7fc3041988c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8000c00 tx=0x7fc2f80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc3017fa640 1 -- 172.21.15.33:0/1105464970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8043020 con 0x7fc304101330 2024-04-02T13:43:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc3017fa640 1 -- 172.21.15.33:0/1105464970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc2f802caa0 con 0x7fc304101330 2024-04-02T13:43:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc30a4ba640 1 -- 172.21.15.33:0/1105464970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30419d5b0 con 0x7fc304101330 2024-04-02T13:43:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc3017fa640 1 -- 172.21.15.33:0/1105464970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8003bf0 con 0x7fc304101330 2024-04-02T13:43:45.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.529+0000 7fc30a4ba640 1 -- 172.21.15.33:0/1105464970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3041a4ea0 con 0x7fc304101330 2024-04-02T13:43:45.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc3017fa640 1 -- 172.21.15.33:0/1105464970 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc2f8046020 con 0x7fc304101330 2024-04-02T13:43:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc30d17d740 1 Processor -- start 2024-04-02T13:43:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc30d17d740 1 -- start start 2024-04-02T13:43:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc30d17d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5c7a0 0x558443c5cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc30d17d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c5d0c0 0x558443c65760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc30d17d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c65ca0 0x558443c68090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc30d17d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558443bc07b0 con 0x558443c5c7a0 2024-04-02T13:43:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc30d17d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558443b5f570 con 0x558443c65ca0 2024-04-02T13:43:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc30d17d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558443bbc760 con 0x558443c5d0c0 2024-04-02T13:43:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc3017fa640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc2ec0a3d20 0x7fc2ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc308a57640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5c7a0 0x558443c5cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc308a57640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5c7a0 0x558443c5cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41620/0 (socket says 172.21.15.33:41620) 2024-04-02T13:43:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc308a57640 1 -- 172.21.15.33:0/2665199872 learned_addr learned my addr 172.21.15.33:0/2665199872 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc303fff640 1 --2- 172.21.15.33:0/2665199872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c5d0c0 0x558443c65760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc308a57640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc2ec0a3d20 0x7fc2ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc3017fa640 1 -- 172.21.15.33:0/1105464970 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc2f80f24a0 con 0x7fc304101330 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.532+0000 7fc3037fe640 1 --2- 172.21.15.33:0/2665199872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c65ca0 0x558443c68090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.533+0000 7fc308a57640 1 -- 172.21.15.33:0/2665199872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c5d0c0 msgr2=0x558443c65760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.533+0000 7fc308a57640 1 --2- 172.21.15.33:0/2665199872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c5d0c0 0x558443c65760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.533+0000 7fc308a57640 1 -- 172.21.15.33:0/2665199872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c65ca0 msgr2=0x558443c68090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.533+0000 7fc308a57640 1 --2- 172.21.15.33:0/2665199872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c65ca0 0x558443c68090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.533+0000 7fc308a57640 1 -- 172.21.15.33:0/2665199872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584438d43f0 con 0x558443c5c7a0 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.533+0000 7fc3037fe640 1 --2- 172.21.15.33:0/2665199872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c65ca0 0x558443c68090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.533+0000 7fc308a57640 1 --2- 172.21.15.33:0/2665199872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5c7a0 0x558443c5cb80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc013a50 tx=0x7fc2fc013f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6be357308580478 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.533+0000 7fc2ea7fc640 1 -- 172.21.15.33:0/2665199872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc013050 con 0x558443c5c7a0 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.533+0000 7fc2ea7fc640 1 -- 172.21.15.33:0/2665199872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc2fc022070 con 0x558443c5c7a0 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.533+0000 7fc2ea7fc640 1 -- 172.21.15.33:0/2665199872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc0126b0 con 0x558443c5c7a0 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.533+0000 7fc308a57640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc2ec0a3d20 0x7fc2ec0a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc021000 tx=0x7fc2fc01f000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 -- 172.21.15.33:0/2665199872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5c7a0 msgr2=0x558443c5cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 --2- 172.21.15.33:0/2665199872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5c7a0 0x558443c5cb80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc013a50 tx=0x7fc2fc013f10 comp rx=0 tx=0).stop 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 -- 172.21.15.33:0/2665199872 shutdown_connections 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 --2- 172.21.15.33:0/2665199872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c65ca0 0x558443c68090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 --2- 172.21.15.33:0/2665199872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c5d0c0 0x558443c65760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 --2- 172.21.15.33:0/2665199872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5c7a0 0x558443c5cb80 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 -- 172.21.15.33:0/2665199872 >> 172.21.15.33:0/2665199872 conn(0x558443c56ed0 msgr2=0x558443c572b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 -- 172.21.15.33:0/2665199872 shutdown_connections 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 -- 172.21.15.33:0/2665199872 wait complete. 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 -- start start 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 Processor -- start 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c5c7a0 0x558443c62b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c63040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c65ca0 0x558443c63580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558443c68a90 con 0x558443c5d0c0 2024-04-02T13:43:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558443b7be80 con 0x558443c65ca0 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558443c68c10 con 0x558443c5c7a0 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.534+0000 7fc30d17d740 10 client.0 _ll_register_callbacks cb 0x558443bb7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc3037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c63040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc3037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c63040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41630/0 (socket says 172.21.15.33:41630) 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc303fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c5c7a0 0x558443c62b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc3037fe640 1 -- 172.21.15.33:0/875031526 learned_addr learned my addr 172.21.15.33:0/875031526 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc308a57640 1 --2- 172.21.15.33:0/875031526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c65ca0 0x558443c63580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc3037fe640 1 -- 172.21.15.33:0/875031526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c5c7a0 msgr2=0x558443c62b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc3037fe640 1 --2- 172.21.15.33:0/875031526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c5c7a0 0x558443c62b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc3037fe640 1 -- 172.21.15.33:0/875031526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c65ca0 msgr2=0x558443c63580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc3037fe640 1 --2- 172.21.15.33:0/875031526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c65ca0 0x558443c63580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc3037fe640 1 -- 172.21.15.33:0/875031526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55844399e7d0 con 0x558443c5d0c0 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc3037fe640 1 --2- 172.21.15.33:0/875031526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c63040 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80bdee0 tx=0x7fc2f80bdb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9715580f183d87da server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc2e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc2e37fe640 1 -- 172.21.15.33:0/875031526 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80c2590 con 0x558443c5d0c0 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc2e37fe640 1 -- 172.21.15.33:0/875031526 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc2f8004c20 con 0x558443c5d0c0 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc30d17d740 1 -- 172.21.15.33:0/875031526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2fc022070 con 0x558443c5d0c0 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc2e37fe640 1 -- 172.21.15.33:0/875031526 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc2f80c6e50 con 0x558443c5d0c0 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc30d17d740 10 client.16695 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc30d17d740 1 -- 172.21.15.33:0/875031526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558443c68a90 con 0x558443c5d0c0 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.535+0000 7fc2e37fe640 1 -- 172.21.15.33:0/875031526 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc2fc022070 con 0x558443c5d0c0 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.536+0000 7fc2e37fe640 1 -- 172.21.15.33:0/875031526 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc2f80efb00 con 0x558443c5d0c0 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.536+0000 7fc30d17d740 10 client.16695 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.536+0000 7fc30d17d740 10 client.16695 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.536+0000 7fc30d17d740 20 client.16695 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.536+0000 7fc30d17d740 1 --2- 172.21.15.33:0/875031526 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558443c75fa0 0x558443c783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.536+0000 7fc30d17d740 4 client.16695 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.536+0000 7fc30d17d740 1 -- 172.21.15.33:0/875031526 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558443bcdcf0 con 0x558443c75fa0 2024-04-02T13:43:45.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.536+0000 7fc303fff640 1 --2- 172.21.15.33:0/875031526 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558443c75fa0 0x558443c783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.536+0000 7fc2e37fe640 10 client.16695 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.536+0000 7fc303fff640 1 --2- 172.21.15.33:0/875031526 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558443c75fa0 0x558443c783c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=303d400749790d73 server_cookie=37af9cde7609f03c in_seq=0 out_seq=0 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc2e37fe640 1 -- 172.21.15.33:0/875031526 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558443bcdcf0 con 0x558443c75fa0 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc2e37fe640 10 client.16695 handle_command_reply: tid=0 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc30d17d740 1 client.16695 shutdown 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc30d17d740 1 -- 172.21.15.33:0/875031526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 msgr2=0x558443c63040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc30d17d740 1 --2- 172.21.15.33:0/875031526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c63040 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80bdee0 tx=0x7fc2f80bdb40 comp rx=0 tx=0).stop 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc30d17d740 1 -- 172.21.15.33:0/875031526 shutdown_connections 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc30d17d740 1 --2- 172.21.15.33:0/875031526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c65ca0 0x558443c63580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc30d17d740 1 --2- 172.21.15.33:0/875031526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c63040 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc30d17d740 1 --2- 172.21.15.33:0/875031526 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558443c75fa0 0x558443c783c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc30d17d740 1 --2- 172.21.15.33:0/875031526 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c5c7a0 0x558443c62b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc2e37fe640 0 client.16695 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc30d17d740 1 -- 172.21.15.33:0/875031526 >> 172.21.15.33:0/875031526 conn(0x558443c56ed0 msgr2=0x558443c58c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc30d17d740 1 -- 172.21.15.33:0/875031526 shutdown_connections 2024-04-02T13:43:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.537+0000 7fc30d17d740 1 -- 172.21.15.33:0/875031526 wait complete. 2024-04-02T13:43:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.538+0000 7fc30d17d740 20 client.16695 trim_cache size 0 max 16384 2024-04-02T13:43:45.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.551+0000 7fc30d17d740 1 Processor -- start 2024-04-02T13:43:45.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.551+0000 7fc30d17d740 1 -- start start 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc30d17d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c5c7a0 0x558443c71950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc30d17d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c61db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc30d17d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c65ca0 0x558443c633a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc30d17d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041b1810 con 0x558443c5d0c0 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc30d17d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041af020 con 0x558443c5c7a0 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc30d17d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041ace50 con 0x558443c65ca0 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc3037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c61db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc3037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c61db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41636/0 (socket says 172.21.15.33:41636) 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc3037fe640 1 -- 172.21.15.33:0/3436853583 learned_addr learned my addr 172.21.15.33:0/3436853583 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc303fff640 1 --2- 172.21.15.33:0/3436853583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c5c7a0 0x558443c71950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc308a57640 1 --2- 172.21.15.33:0/3436853583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c65ca0 0x558443c633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc3037fe640 1 -- 172.21.15.33:0/3436853583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c65ca0 msgr2=0x558443c633a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc3037fe640 1 --2- 172.21.15.33:0/3436853583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c65ca0 0x558443c633a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc3037fe640 1 -- 172.21.15.33:0/3436853583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c5c7a0 msgr2=0x558443c71950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc3037fe640 1 --2- 172.21.15.33:0/3436853583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c5c7a0 0x558443c71950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc3037fe640 1 -- 172.21.15.33:0/3436853583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558443c5c0a0 con 0x558443c5d0c0 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc3037fe640 1 --2- 172.21.15.33:0/3436853583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c61db0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80ba640 tx=0x7fc2f80be360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3051f1c9e7fc8d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc2e9ffb640 1 -- 172.21.15.33:0/3436853583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80c2590 con 0x558443c5d0c0 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc2e9ffb640 1 -- 172.21.15.33:0/3436853583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc2f80be8e0 con 0x558443c5d0c0 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.552+0000 7fc2e9ffb640 1 -- 172.21.15.33:0/3436853583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80bc5b0 con 0x558443c5d0c0 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.553+0000 7fc30d17d740 1 -- 172.21.15.33:0/3436853583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 msgr2=0x558443c61db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.553+0000 7fc30d17d740 1 --2- 172.21.15.33:0/3436853583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c61db0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80ba640 tx=0x7fc2f80be360 comp rx=0 tx=0).stop 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.553+0000 7fc30d17d740 1 -- 172.21.15.33:0/3436853583 shutdown_connections 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.553+0000 7fc30d17d740 1 --2- 172.21.15.33:0/3436853583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c65ca0 0x558443c633a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.553+0000 7fc30d17d740 1 --2- 172.21.15.33:0/3436853583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c61db0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.553+0000 7fc30d17d740 1 --2- 172.21.15.33:0/3436853583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c5c7a0 0x558443c71950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.553+0000 7fc30d17d740 1 -- 172.21.15.33:0/3436853583 >> 172.21.15.33:0/3436853583 conn(0x558443c56ed0 msgr2=0x558443c55f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.553+0000 7fc30d17d740 1 -- 172.21.15.33:0/3436853583 shutdown_connections 2024-04-02T13:43:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.553+0000 7fc30d17d740 1 -- 172.21.15.33:0/3436853583 wait complete. 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.553+0000 7fc30d17d740 1 -- start start 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc30d17d740 1 Processor -- start 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc30d17d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c5c7a0 0x558443c6e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc30d17d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c5f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc30d17d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c65ca0 0x558443c5fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc30d17d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041ad4e0 con 0x558443c5d0c0 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc30d17d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041ae950 con 0x558443c5c7a0 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc30d17d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041adbb0 con 0x558443c65ca0 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc30d17d740 10 client.0 _ll_register_callbacks cb 0x7fc2f8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc3037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c5f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc3037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c5f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41638/0 (socket says 172.21.15.33:41638) 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc3037fe640 1 -- 172.21.15.33:0/1354272167 learned_addr learned my addr 172.21.15.33:0/1354272167 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc308a57640 1 --2- 172.21.15.33:0/1354272167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c65ca0 0x558443c5fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc303fff640 1 --2- 172.21.15.33:0/1354272167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c5c7a0 0x558443c6e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc3037fe640 1 -- 172.21.15.33:0/1354272167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c65ca0 msgr2=0x558443c5fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc3037fe640 1 --2- 172.21.15.33:0/1354272167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c65ca0 0x558443c5fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc3037fe640 1 -- 172.21.15.33:0/1354272167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c5c7a0 msgr2=0x558443c6e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc3037fe640 1 --2- 172.21.15.33:0/1354272167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c5c7a0 0x558443c6e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc3037fe640 1 -- 172.21.15.33:0/1354272167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558443c60260 con 0x558443c5d0c0 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc3037fe640 1 --2- 172.21.15.33:0/1354272167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c5f4f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8005ab0 tx=0x7fc2f80f2150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a36bc2096175f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.554+0000 7fc2e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc2e37fe640 1 -- 172.21.15.33:0/1354272167 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80c2590 con 0x558443c5d0c0 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc2e37fe640 1 -- 172.21.15.33:0/1354272167 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc2f80dd0d0 con 0x558443c5d0c0 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc30d17d740 1 -- 172.21.15.33:0/1354272167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f80be8e0 con 0x558443c5d0c0 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc2e37fe640 1 -- 172.21.15.33:0/1354272167 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc2f80c2190 con 0x558443c5d0c0 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc30d17d740 10 client.16701 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc30d17d740 1 -- 172.21.15.33:0/1354272167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558443c60260 con 0x558443c5d0c0 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc2e37fe640 1 -- 172.21.15.33:0/1354272167 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc2f80be8e0 con 0x558443c5d0c0 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc2e37fe640 1 -- 172.21.15.33:0/1354272167 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc2f80a71f0 con 0x558443c5d0c0 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc30d17d740 10 client.16701 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc30d17d740 10 client.16701 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc30d17d740 20 client.16701 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc30d17d740 1 --2- 172.21.15.33:0/1354272167 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558443cdc7d0 0x558443cdebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc30d17d740 4 client.16701 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc30d17d740 1 -- 172.21.15.33:0/1354272167 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558443bc8040 con 0x558443cdc7d0 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.555+0000 7fc303fff640 1 --2- 172.21.15.33:0/1354272167 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558443cdc7d0 0x558443cdebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc2e37fe640 10 client.16701 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc303fff640 1 --2- 172.21.15.33:0/1354272167 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558443cdc7d0 0x558443cdebf0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91243ef400b36ef4 server_cookie=4f1a415d4d0b1544 in_seq=0 out_seq=0 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc2e37fe640 1 -- 172.21.15.33:0/1354272167 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558443bc8040 con 0x558443cdc7d0 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc2e37fe640 10 client.16701 handle_command_reply: tid=0 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc30d17d740 1 client.16701 shutdown 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc30d17d740 1 -- 172.21.15.33:0/1354272167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 msgr2=0x558443c5f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc30d17d740 1 --2- 172.21.15.33:0/1354272167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c5f4f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8005ab0 tx=0x7fc2f80f2150 comp rx=0 tx=0).stop 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc30d17d740 1 -- 172.21.15.33:0/1354272167 shutdown_connections 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc30d17d740 1 --2- 172.21.15.33:0/1354272167 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558443cdc7d0 0x558443cdebf0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc30d17d740 1 --2- 172.21.15.33:0/1354272167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558443c65ca0 0x558443c5fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc2e37fe640 0 client.16701 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:43:45.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc30d17d740 1 --2- 172.21.15.33:0/1354272167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558443c5d0c0 0x558443c5f4f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc30d17d740 1 --2- 172.21.15.33:0/1354272167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558443c5c7a0 0x558443c6e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc30d17d740 1 -- 172.21.15.33:0/1354272167 >> 172.21.15.33:0/1354272167 conn(0x558443c56ed0 msgr2=0x558443c556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc30d17d740 1 -- 172.21.15.33:0/1354272167 shutdown_connections 2024-04-02T13:43:45.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc30d17d740 1 -- 172.21.15.33:0/1354272167 wait complete. 2024-04-02T13:43:45.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.556+0000 7fc30d17d740 20 client.16701 trim_cache size 0 max 16384 2024-04-02T13:43:45.566 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:43:45.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.557+0000 7fc30a4ba640 1 -- 172.21.15.33:0/1105464970 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc2ec0a3d20 msgr2=0x7fc2ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.557+0000 7fc30a4ba640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc2ec0a3d20 0x7fc2ec0a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc021000 tx=0x7fc2fc01f000 comp rx=0 tx=0).stop 2024-04-02T13:43:45.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.557+0000 7fc30a4ba640 1 -- 172.21.15.33:0/1105464970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc304101330 msgr2=0x7fc3041988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.557+0000 7fc30a4ba640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc304101330 0x7fc3041988c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8000c00 tx=0x7fc2f80045e0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.557+0000 7fc30a4ba640 1 -- 172.21.15.33:0/1105464970 shutdown_connections 2024-04-02T13:43:45.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.557+0000 7fc30a4ba640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc2ec0a3d20 0x7fc2ec0a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.557+0000 7fc30a4ba640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc304106980 0x7fc30419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.557+0000 7fc30a4ba640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc304101330 0x7fc3041988c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.557+0000 7fc30a4ba640 1 --2- 172.21.15.33:0/1105464970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc304100970 0x7fc304198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.557+0000 7fc30a4ba640 1 -- 172.21.15.33:0/1105464970 >> 172.21.15.33:0/1105464970 conn(0x7fc304077b00 msgr2=0x7fc304103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.557+0000 7fc30a4ba640 1 -- 172.21.15.33:0/1105464970 shutdown_connections 2024-04-02T13:43:45.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.557+0000 7fc30a4ba640 1 -- 172.21.15.33:0/1105464970 wait complete. 2024-04-02T13:43:45.571 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:43:45.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.672+0000 7f035e814640 1 -- 172.21.15.33:0/989691159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03581069a0 msgr2=0x7f0358106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.672+0000 7f035e814640 1 --2- 172.21.15.33:0/989691159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03581069a0 0x7f0358106d80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0348009930 tx=0x7f034802bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:45.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.672+0000 7f035e814640 1 -- 172.21.15.33:0/989691159 shutdown_connections 2024-04-02T13:43:45.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.672+0000 7f035e814640 1 --2- 172.21.15.33:0/989691159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0358101350 0x7f035810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.672+0000 7f035e814640 1 --2- 172.21.15.33:0/989691159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0358100990 0x7f0358100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.672+0000 7f035e814640 1 --2- 172.21.15.33:0/989691159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03581069a0 0x7f0358106d80 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.672+0000 7f035e814640 1 -- 172.21.15.33:0/989691159 >> 172.21.15.33:0/989691159 conn(0x7f0358077ad0 msgr2=0x7f0358077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.672+0000 7f035e814640 1 -- 172.21.15.33:0/989691159 shutdown_connections 2024-04-02T13:43:45.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.672+0000 7f035e814640 1 -- 172.21.15.33:0/989691159 wait complete. 2024-04-02T13:43:45.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f035e814640 1 Processor -- start 2024-04-02T13:43:45.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f035e814640 1 -- start start 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f035e814640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0358100990 0x7f03581983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f035e814640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0358101350 0x7f0358198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f035e814640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03581069a0 0x7f035819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f035e814640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0358111e80 con 0x7f0358101350 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f035e814640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0358111d00 con 0x7f0358100990 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f035e814640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0358112000 con 0x7f03581069a0 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f03577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0358101350 0x7f0358198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f03577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0358101350 0x7f0358198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41656/0 (socket says 172.21.15.33:41656) 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f03577fe640 1 -- 172.21.15.33:0/1055050887 learned_addr learned my addr 172.21.15.33:0/1055050887 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f035cdb1640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03581069a0 0x7f035819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f0357fff640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0358100990 0x7f03581983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f03577fe640 1 -- 172.21.15.33:0/1055050887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03581069a0 msgr2=0x7f035819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f03577fe640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03581069a0 0x7f035819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f03577fe640 1 -- 172.21.15.33:0/1055050887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0358100990 msgr2=0x7f03581983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f03577fe640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0358100990 0x7f03581983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.673+0000 7f03577fe640 1 -- 172.21.15.33:0/1055050887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035819d3b0 con 0x7f0358101350 2024-04-02T13:43:45.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.674+0000 7f03577fe640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0358101350 0x7f0358198900 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f034c00d770 tx=0x7f034c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.674+0000 7f03557fa640 1 -- 172.21.15.33:0/1055050887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c017070 con 0x7f0358101350 2024-04-02T13:43:45.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.674+0000 7f03557fa640 1 -- 172.21.15.33:0/1055050887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f034c0054f0 con 0x7f0358101350 2024-04-02T13:43:45.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.674+0000 7f035e814640 1 -- 172.21.15.33:0/1055050887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035819d6a0 con 0x7f0358101350 2024-04-02T13:43:45.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.674+0000 7f03557fa640 1 -- 172.21.15.33:0/1055050887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c004d60 con 0x7f0358101350 2024-04-02T13:43:45.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.674+0000 7f035e814640 1 -- 172.21.15.33:0/1055050887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03581a4f30 con 0x7f0358101350 2024-04-02T13:43:45.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.676+0000 7f03557fa640 1 -- 172.21.15.33:0/1055050887 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f034c004f00 con 0x7f0358101350 2024-04-02T13:43:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.676+0000 7f03613fe740 1 Processor -- start 2024-04-02T13:43:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.676+0000 7f03613fe740 1 -- start start 2024-04-02T13:43:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03557fa640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f03400a3d20 0x7f03400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03613fe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea6f7a0 0x5629aea6fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03613fe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea700c0 0x5629aea78760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03613fe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03613fe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5629ae9d37b0 con 0x5629aea78ca0 2024-04-02T13:43:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03613fe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5629ae972570 con 0x5629aea6f7a0 2024-04-02T13:43:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03613fe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5629ae9cf760 con 0x5629aea700c0 2024-04-02T13:43:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f035cdb1640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f03400a3d20 0x7f03400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03577fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea700c0 0x5629aea78760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f035cdb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03557fa640 1 -- 172.21.15.33:0/1055050887 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f034c0d0660 con 0x7f0358101350 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03577fe640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea700c0 0x5629aea78760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:40910/0 (socket says 172.21.15.33:40910) 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03577fe640 1 -- 172.21.15.33:0/3235423545 learned_addr learned my addr 172.21.15.33:0/3235423545 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f0357fff640 1 --2- 172.21.15.33:0/3235423545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea6f7a0 0x5629aea6fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03577fe640 1 -- 172.21.15.33:0/3235423545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea6f7a0 msgr2=0x5629aea6fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03577fe640 1 --2- 172.21.15.33:0/3235423545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea6f7a0 0x5629aea6fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03577fe640 1 -- 172.21.15.33:0/3235423545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 msgr2=0x5629aea7b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03577fe640 1 --2- 172.21.15.33:0/3235423545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea7b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03577fe640 1 -- 172.21.15.33:0/3235423545 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629ae6e73f0 con 0x5629aea700c0 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f035cdb1640 1 --2- 172.21.15.33:0/3235423545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea7b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f035cdb1640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f03400a3d20 0x7f03400a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f035000afd0 tx=0x7f035000ad80 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.677+0000 7f03577fe640 1 --2- 172.21.15.33:0/3235423545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea700c0 0x5629aea78760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f034c098c20 tx=0x7f034c051440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fb1280b50b626e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.678+0000 7f033e7fc640 1 -- 172.21.15.33:0/3235423545 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c020070 con 0x5629aea700c0 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.678+0000 7f033e7fc640 1 -- 172.21.15.33:0/3235423545 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f034c0056a0 con 0x5629aea700c0 2024-04-02T13:43:45.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.678+0000 7f033e7fc640 1 -- 172.21.15.33:0/3235423545 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c0af0e0 con 0x5629aea700c0 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.678+0000 7f03613fe740 1 -- 172.21.15.33:0/3235423545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea700c0 msgr2=0x5629aea78760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.678+0000 7f03613fe740 1 --2- 172.21.15.33:0/3235423545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea700c0 0x5629aea78760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f034c098c20 tx=0x7f034c051440 comp rx=0 tx=0).stop 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.678+0000 7f03613fe740 1 -- 172.21.15.33:0/3235423545 shutdown_connections 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.678+0000 7f03613fe740 1 --2- 172.21.15.33:0/3235423545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea7b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.678+0000 7f03613fe740 1 --2- 172.21.15.33:0/3235423545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea700c0 0x5629aea78760 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.678+0000 7f03613fe740 1 --2- 172.21.15.33:0/3235423545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea6f7a0 0x5629aea6fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.678+0000 7f03613fe740 1 -- 172.21.15.33:0/3235423545 >> 172.21.15.33:0/3235423545 conn(0x5629aea69ed0 msgr2=0x5629aea6a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.678+0000 7f03613fe740 1 -- 172.21.15.33:0/3235423545 shutdown_connections 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.678+0000 7f03613fe740 1 -- 172.21.15.33:0/3235423545 wait complete. 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f03613fe740 1 -- start start 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f03613fe740 1 Processor -- start 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f03613fe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea6f7a0 0x5629aea78690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f03613fe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea700c0 0x5629aea75a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f03613fe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea78ca0 0x5629aea76030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f03613fe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5629aea7ba30 con 0x5629aea6f7a0 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f03613fe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5629ae98ee80 con 0x5629aea700c0 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f03613fe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5629aea7bbb0 con 0x5629aea78ca0 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f03613fe740 10 client.0 _ll_register_callbacks cb 0x5629ae9caaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f0357fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea6f7a0 0x5629aea78690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f0357fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea6f7a0 0x5629aea78690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41676/0 (socket says 172.21.15.33:41676) 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f0357fff640 1 -- 172.21.15.33:0/1815618065 learned_addr learned my addr 172.21.15.33:0/1815618065 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f035cdb1640 1 --2- 172.21.15.33:0/1815618065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea78ca0 0x5629aea76030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f03577fe640 1 --2- 172.21.15.33:0/1815618065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea700c0 0x5629aea75a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f0357fff640 1 -- 172.21.15.33:0/1815618065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea78ca0 msgr2=0x5629aea76030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f0357fff640 1 --2- 172.21.15.33:0/1815618065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea78ca0 0x5629aea76030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f0357fff640 1 -- 172.21.15.33:0/1815618065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea700c0 msgr2=0x5629aea75a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f0357fff640 1 --2- 172.21.15.33:0/1815618065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea700c0 0x5629aea75a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.679+0000 7f0357fff640 1 -- 172.21.15.33:0/1815618065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629ae7b17d0 con 0x5629aea6f7a0 2024-04-02T13:43:45.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f0357fff640 1 --2- 172.21.15.33:0/1815618065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea6f7a0 0x5629aea78690 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0348005940 tx=0x7f0348005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd45f5a242ee9bef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03377fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03377fe640 1 -- 172.21.15.33:0/1815618065 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348045070 con 0x5629aea6f7a0 2024-04-02T13:43:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03377fe640 1 -- 172.21.15.33:0/1815618065 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f034802c7d0 con 0x5629aea6f7a0 2024-04-02T13:43:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03613fe740 1 -- 172.21.15.33:0/1815618065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f034c0056a0 con 0x5629aea6f7a0 2024-04-02T13:43:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03377fe640 1 -- 172.21.15.33:0/1815618065 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0348051050 con 0x5629aea6f7a0 2024-04-02T13:43:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03613fe740 10 client.16713 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03613fe740 1 -- 172.21.15.33:0/1815618065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5629aea87a50 con 0x5629aea6f7a0 2024-04-02T13:43:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03377fe640 1 -- 172.21.15.33:0/1815618065 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f034c0056a0 con 0x5629aea6f7a0 2024-04-02T13:43:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03377fe640 1 -- 172.21.15.33:0/1815618065 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f03480035f0 con 0x5629aea6f7a0 2024-04-02T13:43:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03613fe740 10 client.16713 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03613fe740 10 client.16713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03613fe740 20 client.16713 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:45.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03613fe740 1 --2- 172.21.15.33:0/1815618065 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5629aea88f20 0x5629aea8b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03613fe740 4 client.16713 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.680+0000 7f03613fe740 1 -- 172.21.15.33:0/1815618065 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629ae9e0cf0 con 0x5629aea88f20 2024-04-02T13:43:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.681+0000 7f03577fe640 1 --2- 172.21.15.33:0/1815618065 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5629aea88f20 0x5629aea8b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.681+0000 7f03577fe640 1 --2- 172.21.15.33:0/1815618065 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5629aea88f20 0x5629aea8b3c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20fd3e40a00fa548 server_cookie=389f9bf88af3fcb6 in_seq=0 out_seq=0 2024-04-02T13:43:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.681+0000 7f03377fe640 10 client.16713 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03377fe640 1 -- 172.21.15.33:0/1815618065 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629ae9e0cf0 con 0x5629aea88f20 2024-04-02T13:43:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03377fe640 10 client.16713 handle_command_reply: tid=0 2024-04-02T13:43:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03613fe740 1 client.16713 shutdown 2024-04-02T13:43:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03613fe740 1 -- 172.21.15.33:0/1815618065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea6f7a0 msgr2=0x5629aea78690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03613fe740 1 --2- 172.21.15.33:0/1815618065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea6f7a0 0x5629aea78690 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0348005940 tx=0x7f0348005270 comp rx=0 tx=0).stop 2024-04-02T13:43:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03613fe740 1 -- 172.21.15.33:0/1815618065 shutdown_connections 2024-04-02T13:43:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03613fe740 1 --2- 172.21.15.33:0/1815618065 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5629aea88f20 0x5629aea8b3c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03613fe740 1 --2- 172.21.15.33:0/1815618065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea78ca0 0x5629aea76030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03613fe740 1 --2- 172.21.15.33:0/1815618065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea700c0 0x5629aea75a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03613fe740 1 --2- 172.21.15.33:0/1815618065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea6f7a0 0x5629aea78690 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03377fe640 0 client.16713 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:45.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03613fe740 1 -- 172.21.15.33:0/1815618065 >> 172.21.15.33:0/1815618065 conn(0x5629aea69ed0 msgr2=0x5629aea6bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03613fe740 1 -- 172.21.15.33:0/1815618065 shutdown_connections 2024-04-02T13:43:45.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03613fe740 1 -- 172.21.15.33:0/1815618065 wait complete. 2024-04-02T13:43:45.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.682+0000 7f03613fe740 20 client.16713 trim_cache size 0 max 16384 2024-04-02T13:43:45.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.695+0000 7f03613fe740 1 Processor -- start 2024-04-02T13:43:45.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.695+0000 7f03613fe740 1 -- start start 2024-04-02T13:43:45.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.695+0000 7f03613fe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea6f7a0 0x5629aea84920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.695+0000 7f03613fe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea700c0 0x5629aea74a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.695+0000 7f03613fe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea78150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.695+0000 7f03613fe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03581b18d0 con 0x5629aea78ca0 2024-04-02T13:43:45.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f03613fe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f03581af0e0 con 0x5629aea6f7a0 2024-04-02T13:43:45.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f03613fe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f03581acf10 con 0x5629aea700c0 2024-04-02T13:43:45.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f035cdb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea78150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f035cdb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea78150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41678/0 (socket says 172.21.15.33:41678) 2024-04-02T13:43:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f035cdb1640 1 -- 172.21.15.33:0/926412762 learned_addr learned my addr 172.21.15.33:0/926412762 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f03577fe640 1 --2- 172.21.15.33:0/926412762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea700c0 0x5629aea74a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f0357fff640 1 --2- 172.21.15.33:0/926412762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea6f7a0 0x5629aea84920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f035cdb1640 1 -- 172.21.15.33:0/926412762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea700c0 msgr2=0x5629aea74a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f035cdb1640 1 --2- 172.21.15.33:0/926412762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea700c0 0x5629aea74a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f035cdb1640 1 -- 172.21.15.33:0/926412762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea6f7a0 msgr2=0x5629aea84920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f035cdb1640 1 --2- 172.21.15.33:0/926412762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea6f7a0 0x5629aea84920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f035cdb1640 1 -- 172.21.15.33:0/926412762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629aea763a0 con 0x5629aea78ca0 2024-04-02T13:43:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f035cdb1640 1 --2- 172.21.15.33:0/926412762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea78150 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f03500142f0 tx=0x7f0350004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4329c1fbc88eb67 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f033dffb640 1 -- 172.21.15.33:0/926412762 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350025070 con 0x5629aea78ca0 2024-04-02T13:43:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f033dffb640 1 -- 172.21.15.33:0/926412762 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0350012d60 con 0x5629aea78ca0 2024-04-02T13:43:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.696+0000 7f033dffb640 1 -- 172.21.15.33:0/926412762 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035000f630 con 0x5629aea78ca0 2024-04-02T13:43:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.697+0000 7f03613fe740 1 -- 172.21.15.33:0/926412762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 msgr2=0x5629aea78150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.697+0000 7f03613fe740 1 --2- 172.21.15.33:0/926412762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea78150 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f03500142f0 tx=0x7f0350004ab0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.697+0000 7f03613fe740 1 -- 172.21.15.33:0/926412762 shutdown_connections 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.697+0000 7f03613fe740 1 --2- 172.21.15.33:0/926412762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea78150 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.697+0000 7f03613fe740 1 --2- 172.21.15.33:0/926412762 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea700c0 0x5629aea74a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.697+0000 7f03613fe740 1 --2- 172.21.15.33:0/926412762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea6f7a0 0x5629aea84920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.697+0000 7f03613fe740 1 -- 172.21.15.33:0/926412762 >> 172.21.15.33:0/926412762 conn(0x5629aea69ed0 msgr2=0x5629aea742f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.697+0000 7f03613fe740 1 -- 172.21.15.33:0/926412762 shutdown_connections 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.697+0000 7f03613fe740 1 -- 172.21.15.33:0/926412762 wait complete. 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.697+0000 7f03613fe740 1 -- start start 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.697+0000 7f03613fe740 1 Processor -- start 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f03613fe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea6f7a0 0x5629aea80e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f03613fe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea700c0 0x5629aea81370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f03613fe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea72cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f03613fe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03581ad5a0 con 0x5629aea78ca0 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f03613fe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f03581aea10 con 0x5629aea700c0 2024-04-02T13:43:45.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f03613fe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f03581adc70 con 0x5629aea6f7a0 2024-04-02T13:43:45.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f03613fe740 10 client.0 _ll_register_callbacks cb 0x7f0348005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:45.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f035cdb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea72cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f035cdb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea72cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41694/0 (socket says 172.21.15.33:41694) 2024-04-02T13:43:45.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f0357fff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea6f7a0 0x5629aea80e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f035cdb1640 1 -- 172.21.15.33:0/3969801652 learned_addr learned my addr 172.21.15.33:0/3969801652 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:45.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f03577fe640 1 --2- 172.21.15.33:0/3969801652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea700c0 0x5629aea81370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f035cdb1640 1 -- 172.21.15.33:0/3969801652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea6f7a0 msgr2=0x5629aea80e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f035cdb1640 1 --2- 172.21.15.33:0/3969801652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea6f7a0 0x5629aea80e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f035cdb1640 1 -- 172.21.15.33:0/3969801652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea700c0 msgr2=0x5629aea81370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:45.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f035cdb1640 1 --2- 172.21.15.33:0/3969801652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea700c0 0x5629aea81370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f035cdb1640 1 -- 172.21.15.33:0/3969801652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629aea734f0 con 0x5629aea78ca0 2024-04-02T13:43:45.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f035cdb1640 1 --2- 172.21.15.33:0/3969801652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea72cc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0350015010 tx=0x7f0350021b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d5ee1f14b1fe24 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:45.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f03377fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f03377fe640 1 -- 172.21.15.33:0/3969801652 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350025040 con 0x5629aea78ca0 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f03377fe640 1 -- 172.21.15.33:0/3969801652 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0350003970 con 0x5629aea78ca0 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f03613fe740 1 -- 172.21.15.33:0/3969801652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0350012d60 con 0x5629aea78ca0 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.698+0000 7f03377fe640 1 -- 172.21.15.33:0/3969801652 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0350003b70 con 0x5629aea78ca0 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.699+0000 7f03613fe740 10 client.16719 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.699+0000 7f03613fe740 1 -- 172.21.15.33:0/3969801652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5629aea79ff0 con 0x5629aea78ca0 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.699+0000 7f03377fe640 1 -- 172.21.15.33:0/3969801652 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0350012d60 con 0x5629aea78ca0 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.699+0000 7f03377fe640 1 -- 172.21.15.33:0/3969801652 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0350031070 con 0x5629aea78ca0 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.699+0000 7f03613fe740 10 client.16719 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.699+0000 7f03613fe740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.699+0000 7f03613fe740 20 client.16719 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.699+0000 7f03613fe740 1 --2- 172.21.15.33:0/3969801652 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5629aeaef7e0 0x5629aeaf1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.699+0000 7f03613fe740 4 client.16719 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.699+0000 7f03613fe740 1 -- 172.21.15.33:0/3969801652 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629ae9db040 con 0x5629aeaef7e0 2024-04-02T13:43:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.699+0000 7f0357fff640 1 --2- 172.21.15.33:0/3969801652 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5629aeaef7e0 0x5629aeaf1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03377fe640 10 client.16719 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f0357fff640 1 --2- 172.21.15.33:0/3969801652 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5629aeaef7e0 0x5629aeaf1c00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7638b511b10e4eb7 server_cookie=d878d85069a3274e in_seq=0 out_seq=0 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03377fe640 1 -- 172.21.15.33:0/3969801652 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629ae9db040 con 0x5629aeaef7e0 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03377fe640 10 client.16719 handle_command_reply: tid=0 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03613fe740 1 client.16719 shutdown 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03613fe740 1 -- 172.21.15.33:0/3969801652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 msgr2=0x5629aea72cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03613fe740 1 --2- 172.21.15.33:0/3969801652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea72cc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0350015010 tx=0x7f0350021b90 comp rx=0 tx=0).stop 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03613fe740 1 -- 172.21.15.33:0/3969801652 shutdown_connections 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03613fe740 1 --2- 172.21.15.33:0/3969801652 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5629aeaef7e0 0x5629aeaf1c00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03613fe740 1 --2- 172.21.15.33:0/3969801652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629aea78ca0 0x5629aea72cc0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03613fe740 1 --2- 172.21.15.33:0/3969801652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629aea700c0 0x5629aea81370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03377fe640 0 client.16719 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03613fe740 1 --2- 172.21.15.33:0/3969801652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629aea6f7a0 0x5629aea80e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03613fe740 1 -- 172.21.15.33:0/3969801652 >> 172.21.15.33:0/3969801652 conn(0x5629aea69ed0 msgr2=0x5629aea686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03613fe740 1 -- 172.21.15.33:0/3969801652 shutdown_connections 2024-04-02T13:43:45.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03613fe740 1 -- 172.21.15.33:0/3969801652 wait complete. 2024-04-02T13:43:45.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.700+0000 7f03613fe740 20 client.16719 trim_cache size 0 max 16384 2024-04-02T13:43:45.711 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:43:45.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.701+0000 7f035e814640 1 -- 172.21.15.33:0/1055050887 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f03400a3d20 msgr2=0x7f03400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.701+0000 7f035e814640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f03400a3d20 0x7f03400a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f035000afd0 tx=0x7f035000ad80 comp rx=0 tx=0).stop 2024-04-02T13:43:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.701+0000 7f035e814640 1 -- 172.21.15.33:0/1055050887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0358101350 msgr2=0x7f0358198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.701+0000 7f035e814640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0358101350 0x7f0358198900 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f034c00d770 tx=0x7f034c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:43:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.701+0000 7f035e814640 1 -- 172.21.15.33:0/1055050887 shutdown_connections 2024-04-02T13:43:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.701+0000 7f035e814640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f03400a3d20 0x7f03400a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.701+0000 7f035e814640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03581069a0 0x7f035819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.701+0000 7f035e814640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0358101350 0x7f0358198900 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.701+0000 7f035e814640 1 --2- 172.21.15.33:0/1055050887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0358100990 0x7f03581983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.701+0000 7f035e814640 1 -- 172.21.15.33:0/1055050887 >> 172.21.15.33:0/1055050887 conn(0x7f0358077ad0 msgr2=0x7f0358103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.701+0000 7f035e814640 1 -- 172.21.15.33:0/1055050887 shutdown_connections 2024-04-02T13:43:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:45.701+0000 7f035e814640 1 -- 172.21.15.33:0/1055050887 wait complete. 2024-04-02T13:43:46.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:46 smithi145 ceph-mon[35646]: pgmap v152: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:46.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:46 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/724484843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:43:46.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:46 smithi050 ceph-mon[35524]: pgmap v152: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:46.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:46 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/724484843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:43:46.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:46 smithi033 ceph-mon[32390]: pgmap v152: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:46.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:46 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/724484843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:43:46.716 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:43:46.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:43:46.716 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:43:46.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.833+0000 7f022c2ba640 1 -- 172.21.15.33:0/375294327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224102450 msgr2=0x7f022410e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.833+0000 7f022c2ba640 1 --2- 172.21.15.33:0/375294327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224102450 0x7f022410e960 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f021c009cd0 tx=0x7f021c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.833+0000 7f022c2ba640 1 -- 172.21.15.33:0/375294327 shutdown_connections 2024-04-02T13:43:46.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.833+0000 7f022c2ba640 1 --2- 172.21.15.33:0/375294327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224102450 0x7f022410e960 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.833+0000 7f022c2ba640 1 --2- 172.21.15.33:0/375294327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0224101a90 0x7f0224101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.833+0000 7f022c2ba640 1 --2- 172.21.15.33:0/375294327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0224107aa0 0x7f0224107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.833+0000 7f022c2ba640 1 -- 172.21.15.33:0/375294327 >> 172.21.15.33:0/375294327 conn(0x7f02240fd5a0 msgr2=0x7f02240ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.833+0000 7f022c2ba640 1 -- 172.21.15.33:0/375294327 shutdown_connections 2024-04-02T13:43:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.833+0000 7f022c2ba640 1 -- 172.21.15.33:0/375294327 wait complete. 2024-04-02T13:43:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022c2ba640 1 Processor -- start 2024-04-02T13:43:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022c2ba640 1 -- start start 2024-04-02T13:43:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022c2ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224101a90 0x7f02241983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022c2ba640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0224102450 0x7f0224198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022c2ba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0224107aa0 0x7f022419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022c2ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0224111e80 con 0x7f0224101a90 2024-04-02T13:43:46.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022c2ba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0224111d00 con 0x7f0224107aa0 2024-04-02T13:43:46.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022c2ba640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0224112000 con 0x7f0224102450 2024-04-02T13:43:46.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022a056640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224101a90 0x7f02241983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022a056640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224101a90 0x7f02241983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41722/0 (socket says 172.21.15.33:41722) 2024-04-02T13:43:46.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022a056640 1 -- 172.21.15.33:0/2338892411 learned_addr learned my addr 172.21.15.33:0/2338892411 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:46.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022a857640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0224107aa0 0x7f022419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022a056640 1 -- 172.21.15.33:0/2338892411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0224102450 msgr2=0x7f0224198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f0229855640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0224102450 0x7f0224198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022a056640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0224102450 0x7f0224198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022a056640 1 -- 172.21.15.33:0/2338892411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0224107aa0 msgr2=0x7f022419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.834+0000 7f022a056640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0224107aa0 0x7f022419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.835+0000 7f022a056640 1 -- 172.21.15.33:0/2338892411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022419d3b0 con 0x7f0224101a90 2024-04-02T13:43:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.835+0000 7f022a857640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0224107aa0 0x7f022419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.835+0000 7f0229855640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0224102450 0x7f0224198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.835+0000 7f022a056640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224101a90 0x7f02241983c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f021800d950 tx=0x7f021800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.835+0000 7f02177fe640 1 -- 172.21.15.33:0/2338892411 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218017030 con 0x7f0224101a90 2024-04-02T13:43:46.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.835+0000 7f02177fe640 1 -- 172.21.15.33:0/2338892411 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0218004020 con 0x7f0224101a90 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.835+0000 7f022c2ba640 1 -- 172.21.15.33:0/2338892411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022419d6a0 con 0x7f0224101a90 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.835+0000 7f02177fe640 1 -- 172.21.15.33:0/2338892411 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180047a0 con 0x7f0224101a90 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.835+0000 7f022c2ba640 1 -- 172.21.15.33:0/2338892411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02241a4f30 con 0x7f0224101a90 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.837+0000 7f02177fe640 1 -- 172.21.15.33:0/2338892411 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f02180041d0 con 0x7f0224101a90 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f022ef7e740 1 Processor -- start 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f022ef7e740 1 -- start start 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f02177fe640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f02100a3d20 0x7f02100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f022ef7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x5629404676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f022ef7e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562940467be0 0x562940470280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f022ef7e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940472bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f022ef7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5629403cb770 con 0x5629404672c0 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f022ef7e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562940373960 con 0x562940467be0 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f022ef7e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5629403c78c0 con 0x5629404707c0 2024-04-02T13:43:46.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f022a056640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x5629404676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f022a056640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x5629404676a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41728/0 (socket says 172.21.15.33:41728) 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f022a056640 1 -- 172.21.15.33:0/4174580908 learned_addr learned my addr 172.21.15.33:0/4174580908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f0229855640 1 --2- 172.21.15.33:0/4174580908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562940467be0 0x562940470280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f022a857640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f02100a3d20 0x7f02100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.838+0000 7f02177fe640 1 -- 172.21.15.33:0/2338892411 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f02180d0600 con 0x7f0224101a90 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.839+0000 7f022a857640 1 --2- 172.21.15.33:0/4174580908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940472bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.839+0000 7f022a056640 1 -- 172.21.15.33:0/4174580908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 msgr2=0x562940472bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.839+0000 7f022a056640 1 --2- 172.21.15.33:0/4174580908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940472bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.839+0000 7f022a056640 1 -- 172.21.15.33:0/4174580908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562940467be0 msgr2=0x562940470280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.839+0000 7f022a056640 1 --2- 172.21.15.33:0/4174580908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562940467be0 0x562940470280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.839+0000 7f022a056640 1 -- 172.21.15.33:0/4174580908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629401a77e0 con 0x5629404672c0 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.839+0000 7f022a857640 1 --2- 172.21.15.33:0/4174580908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940472bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.839+0000 7f0229855640 1 --2- 172.21.15.33:0/4174580908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562940467be0 0x562940470280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.839+0000 7f022a056640 1 --2- 172.21.15.33:0/4174580908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x5629404676a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0218005fd0 tx=0x7f02180207f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fb36f585c04e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.839+0000 7f022a857640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f02100a3d20 0x7f02100a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x562940462b80 tx=0x7f021c0323d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:46.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.839+0000 7f0214bfe640 1 -- 172.21.15.33:0/4174580908 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021809bb90 con 0x5629404672c0 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.839+0000 7f0214bfe640 1 -- 172.21.15.33:0/4174580908 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f021809c930 con 0x5629404672c0 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.839+0000 7f0214bfe640 1 -- 172.21.15.33:0/4174580908 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218021cc0 con 0x5629404672c0 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.840+0000 7f022ef7e740 1 -- 172.21.15.33:0/4174580908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 msgr2=0x5629404676a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.840+0000 7f022ef7e740 1 --2- 172.21.15.33:0/4174580908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x5629404676a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0218005fd0 tx=0x7f02180207f0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.840+0000 7f022ef7e740 1 -- 172.21.15.33:0/4174580908 shutdown_connections 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.840+0000 7f022ef7e740 1 --2- 172.21.15.33:0/4174580908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940472bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.840+0000 7f022ef7e740 1 --2- 172.21.15.33:0/4174580908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562940467be0 0x562940470280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.840+0000 7f022ef7e740 1 --2- 172.21.15.33:0/4174580908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x5629404676a0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.840+0000 7f022ef7e740 1 -- 172.21.15.33:0/4174580908 >> 172.21.15.33:0/4174580908 conn(0x5629404619f0 msgr2=0x562940461dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.840+0000 7f022ef7e740 1 -- 172.21.15.33:0/4174580908 shutdown_connections 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.840+0000 7f022ef7e740 1 -- 172.21.15.33:0/4174580908 wait complete. 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.840+0000 7f022ef7e740 1 -- start start 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.840+0000 7f022ef7e740 1 Processor -- start 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022ef7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x56294046d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022ef7e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562940467be0 0x56294046dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022ef7e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629404707c0 0x56294046e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022ef7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562940473550 con 0x5629404672c0 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022ef7e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562940386d20 con 0x5629404707c0 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022ef7e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5629404736d0 con 0x562940467be0 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022ef7e740 10 client.0 _ll_register_callbacks cb 0x562940353a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022a056640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x56294046d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022a056640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x56294046d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41740/0 (socket says 172.21.15.33:41740) 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022a056640 1 -- 172.21.15.33:0/1469940811 learned_addr learned my addr 172.21.15.33:0/1469940811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f0229855640 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562940467be0 0x56294046dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022a056640 1 -- 172.21.15.33:0/1469940811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562940467be0 msgr2=0x56294046dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022a857640 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629404707c0 0x56294046e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022a056640 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562940467be0 0x56294046dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022a056640 1 -- 172.21.15.33:0/1469940811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629404707c0 msgr2=0x56294046e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022a056640 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629404707c0 0x56294046e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022a056640 1 -- 172.21.15.33:0/1469940811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562940207f70 con 0x5629404672c0 2024-04-02T13:43:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022a857640 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629404707c0 0x56294046e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022a056640 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x56294046d7e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f02180a4970 tx=0x7f0218098b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f299d9a4dd7079 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f020dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f020dffb640 1 -- 172.21.15.33:0/1469940811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180cde70 con 0x5629404672c0 2024-04-02T13:43:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f020dffb640 1 -- 172.21.15.33:0/1469940811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f02180a4a10 con 0x5629404672c0 2024-04-02T13:43:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.841+0000 7f022ef7e740 1 -- 172.21.15.33:0/1469940811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f021809c930 con 0x5629404672c0 2024-04-02T13:43:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.842+0000 7f020dffb640 1 -- 172.21.15.33:0/1469940811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02180a05e0 con 0x5629404672c0 2024-04-02T13:43:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.842+0000 7f022ef7e740 10 client.16731 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.842+0000 7f022ef7e740 1 -- 172.21.15.33:0/1469940811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56294009f170 con 0x5629404672c0 2024-04-02T13:43:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.842+0000 7f020dffb640 1 -- 172.21.15.33:0/1469940811 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f021809c930 con 0x5629404672c0 2024-04-02T13:43:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.842+0000 7f020dffb640 1 -- 172.21.15.33:0/1469940811 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f02180a0240 con 0x5629404672c0 2024-04-02T13:43:46.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.842+0000 7f022ef7e740 10 client.16731 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.842+0000 7f022ef7e740 10 client.16731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.842+0000 7f022ef7e740 20 client.16731 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.842+0000 7f022ef7e740 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562940480950 0x562940482d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.842+0000 7f022ef7e740 4 client.16731 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.842+0000 7f022ef7e740 1 -- 172.21.15.33:0/1469940811 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629403d8cb0 con 0x562940480950 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.842+0000 7f0229855640 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562940480950 0x562940482d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.843+0000 7f020dffb640 10 client.16731 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.843+0000 7f0229855640 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562940480950 0x562940482d40 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d75875da03e40c65 server_cookie=a1b4c16fbd3eabca in_seq=0 out_seq=0 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f020dffb640 1 -- 172.21.15.33:0/1469940811 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629403d8cb0 con 0x562940480950 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f020dffb640 10 client.16731 handle_command_reply: tid=0 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f022ef7e740 1 client.16731 shutdown 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f022ef7e740 1 -- 172.21.15.33:0/1469940811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 msgr2=0x56294046d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f022ef7e740 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x56294046d7e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f02180a4970 tx=0x7f0218098b30 comp rx=0 tx=0).stop 2024-04-02T13:43:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f022ef7e740 1 -- 172.21.15.33:0/1469940811 shutdown_connections 2024-04-02T13:43:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f022ef7e740 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562940480950 0x562940482d40 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f022ef7e740 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629404707c0 0x56294046e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f022ef7e740 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562940467be0 0x56294046dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f022ef7e740 1 --2- 172.21.15.33:0/1469940811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x56294046d7e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f020dffb640 0 client.16731 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f022ef7e740 1 -- 172.21.15.33:0/1469940811 >> 172.21.15.33:0/1469940811 conn(0x5629404619f0 msgr2=0x562940463740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f022ef7e740 1 -- 172.21.15.33:0/1469940811 shutdown_connections 2024-04-02T13:43:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f022ef7e740 1 -- 172.21.15.33:0/1469940811 wait complete. 2024-04-02T13:43:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.844+0000 7f022ef7e740 20 client.16731 trim_cache size 0 max 16384 2024-04-02T13:43:46.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.862+0000 7f022ef7e740 1 Processor -- start 2024-04-02T13:43:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.862+0000 7f022ef7e740 1 -- start start 2024-04-02T13:43:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.862+0000 7f022ef7e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629404672c0 0x56294046ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.862+0000 7f022ef7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562940467be0 0x56294047c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.862+0000 7f022ef7e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940485d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.862+0000 7f022ef7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f02241b19c0 con 0x562940467be0 2024-04-02T13:43:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.862+0000 7f022ef7e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f02241af1d0 con 0x5629404672c0 2024-04-02T13:43:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.862+0000 7f022ef7e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f02241ad000 con 0x5629404707c0 2024-04-02T13:43:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.862+0000 7f0229855640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562940467be0 0x56294047c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.862+0000 7f0229855640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562940467be0 0x56294047c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41754/0 (socket says 172.21.15.33:41754) 2024-04-02T13:43:46.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.862+0000 7f0229855640 1 -- 172.21.15.33:0/3418950310 learned_addr learned my addr 172.21.15.33:0/3418950310 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:46.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.862+0000 7f022a056640 1 --2- 172.21.15.33:0/3418950310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629404672c0 0x56294046ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.862+0000 7f022a857640 1 --2- 172.21.15.33:0/3418950310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940485d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f0229855640 1 -- 172.21.15.33:0/3418950310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 msgr2=0x562940485d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:46.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f0229855640 1 --2- 172.21.15.33:0/3418950310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940485d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f0229855640 1 -- 172.21.15.33:0/3418950310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629404672c0 msgr2=0x56294046ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:46.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f0229855640 1 --2- 172.21.15.33:0/3418950310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629404672c0 0x56294046ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f0229855640 1 -- 172.21.15.33:0/3418950310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562940461230 con 0x562940467be0 2024-04-02T13:43:46.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f022a857640 1 --2- 172.21.15.33:0/3418950310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940485d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:46.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f022a056640 1 --2- 172.21.15.33:0/3418950310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629404672c0 0x56294046ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:46.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f0229855640 1 --2- 172.21.15.33:0/3418950310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562940467be0 0x56294047c0c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0220005fd0 tx=0x7f02200094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67dcde368b8fd209 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:46.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f020ffff640 1 -- 172.21.15.33:0/3418950310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220013c50 con 0x562940467be0 2024-04-02T13:43:46.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f020ffff640 1 -- 172.21.15.33:0/3418950310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0220013df0 con 0x562940467be0 2024-04-02T13:43:46.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f020ffff640 1 -- 172.21.15.33:0/3418950310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220004730 con 0x562940467be0 2024-04-02T13:43:46.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f022ef7e740 1 -- 172.21.15.33:0/3418950310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562940467be0 msgr2=0x56294047c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f022ef7e740 1 --2- 172.21.15.33:0/3418950310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562940467be0 0x56294047c0c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0220005fd0 tx=0x7f02200094a0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f022ef7e740 1 -- 172.21.15.33:0/3418950310 shutdown_connections 2024-04-02T13:43:46.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f022ef7e740 1 --2- 172.21.15.33:0/3418950310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940485d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f022ef7e740 1 --2- 172.21.15.33:0/3418950310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562940467be0 0x56294047c0c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f022ef7e740 1 --2- 172.21.15.33:0/3418950310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5629404672c0 0x56294046ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.863+0000 7f022ef7e740 1 -- 172.21.15.33:0/3418950310 >> 172.21.15.33:0/3418950310 conn(0x5629404619f0 msgr2=0x56294047b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:46.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022ef7e740 1 -- 172.21.15.33:0/3418950310 shutdown_connections 2024-04-02T13:43:46.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022ef7e740 1 -- 172.21.15.33:0/3418950310 wait complete. 2024-04-02T13:43:46.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022ef7e740 1 -- start start 2024-04-02T13:43:46.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022ef7e740 1 Processor -- start 2024-04-02T13:43:46.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022ef7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x562940483980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022ef7e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562940467be0 0x562940483ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022ef7e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940484400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022ef7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f02241ad690 con 0x5629404672c0 2024-04-02T13:43:46.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022ef7e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f02241aeb00 con 0x562940467be0 2024-04-02T13:43:46.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022ef7e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f02241add60 con 0x5629404707c0 2024-04-02T13:43:46.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022ef7e740 10 client.0 _ll_register_callbacks cb 0x7f0220009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:46.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022a056640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x562940483980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022a056640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x562940483980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41756/0 (socket says 172.21.15.33:41756) 2024-04-02T13:43:46.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022a056640 1 -- 172.21.15.33:0/3197806889 learned_addr learned my addr 172.21.15.33:0/3197806889 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:46.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f0229855640 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562940467be0 0x562940483ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022a857640 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940484400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022a056640 1 -- 172.21.15.33:0/3197806889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 msgr2=0x562940484400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:46.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022a056640 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940484400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.864+0000 7f022a056640 1 -- 172.21.15.33:0/3197806889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562940467be0 msgr2=0x562940483ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:46.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f022a056640 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562940467be0 0x562940483ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f022a056640 1 -- 172.21.15.33:0/3197806889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629404e5ce0 con 0x5629404672c0 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f0229855640 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562940467be0 0x562940483ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f022a056640 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x562940483980 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0218098c00 tx=0x7f021809a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac36aaafc1629c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f020dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f020dffb640 1 -- 172.21.15.33:0/3197806889 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180cde70 con 0x5629404672c0 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f020dffb640 1 -- 172.21.15.33:0/3197806889 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f02180a0120 con 0x5629404672c0 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f022ef7e740 1 -- 172.21.15.33:0/3197806889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0220013df0 con 0x5629404672c0 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f020dffb640 1 -- 172.21.15.33:0/3197806889 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f021809bb90 con 0x5629404672c0 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f022ef7e740 10 client.16737 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f022ef7e740 1 -- 172.21.15.33:0/3197806889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5629404e5ce0 con 0x5629404672c0 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f020dffb640 1 -- 172.21.15.33:0/3197806889 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f0220013df0 con 0x5629404672c0 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f020dffb640 1 -- 172.21.15.33:0/3197806889 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f02180980a0 con 0x5629404672c0 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f022ef7e740 10 client.16737 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:46.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f022ef7e740 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f022ef7e740 20 client.16737 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f022ef7e740 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5629404e72e0 0x5629404e9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f022ef7e740 4 client.16737 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.865+0000 7f022ef7e740 1 -- 172.21.15.33:0/3197806889 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5629403d3000 con 0x5629404e72e0 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.866+0000 7f0229855640 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5629404e72e0 0x5629404e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.866+0000 7f020dffb640 10 client.16737 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.866+0000 7f0229855640 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5629404e72e0 0x5629404e9700 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8c05afb562cd07e server_cookie=d7f1b0c21953cc12 in_seq=0 out_seq=0 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.867+0000 7f020dffb640 1 -- 172.21.15.33:0/3197806889 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5629403d3000 con 0x5629404e72e0 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.867+0000 7f020dffb640 10 client.16737 handle_command_reply: tid=0 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.867+0000 7f022ef7e740 1 client.16737 shutdown 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.867+0000 7f022ef7e740 1 -- 172.21.15.33:0/3197806889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 msgr2=0x562940483980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.867+0000 7f022ef7e740 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x562940483980 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0218098c00 tx=0x7f021809a740 comp rx=0 tx=0).stop 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.867+0000 7f022ef7e740 1 -- 172.21.15.33:0/3197806889 shutdown_connections 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.867+0000 7f022ef7e740 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5629404e72e0 0x5629404e9700 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022ef7e740 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5629404707c0 0x562940484400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022ef7e740 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562940467be0 0x562940483ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022ef7e740 1 --2- 172.21.15.33:0/3197806889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629404672c0 0x562940483980 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022ef7e740 1 -- 172.21.15.33:0/3197806889 >> 172.21.15.33:0/3197806889 conn(0x5629404619f0 msgr2=0x562940460ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:46.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f020dffb640 0 client.16737 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:46.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022ef7e740 1 -- 172.21.15.33:0/3197806889 shutdown_connections 2024-04-02T13:43:46.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022ef7e740 1 -- 172.21.15.33:0/3197806889 wait complete. 2024-04-02T13:43:46.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022ef7e740 20 client.16737 trim_cache size 0 max 16384 2024-04-02T13:43:46.881 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:43:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:43:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "b11a345e-1cf6-4f10-9cfe-c0d63739c0b7", 2024-04-02T13:43:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:43:46.881 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:43:46.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022c2ba640 1 -- 172.21.15.33:0/2338892411 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f02100a3d20 msgr2=0x7f02100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022c2ba640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f02100a3d20 0x7f02100a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x562940462b80 tx=0x7f021c0323d0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022c2ba640 1 -- 172.21.15.33:0/2338892411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224101a90 msgr2=0x7f02241983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022c2ba640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224101a90 0x7f02241983c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f021800d950 tx=0x7f021800de10 comp rx=0 tx=0).stop 2024-04-02T13:43:46.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022c2ba640 1 -- 172.21.15.33:0/2338892411 shutdown_connections 2024-04-02T13:43:46.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022c2ba640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f02100a3d20 0x7f02100a61e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022c2ba640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0224107aa0 0x7f022419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022c2ba640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0224102450 0x7f0224198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022c2ba640 1 --2- 172.21.15.33:0/2338892411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0224101a90 0x7f02241983c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022c2ba640 1 -- 172.21.15.33:0/2338892411 >> 172.21.15.33:0/2338892411 conn(0x7f02240fd5a0 msgr2=0x7f02240fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:46.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022c2ba640 1 -- 172.21.15.33:0/2338892411 shutdown_connections 2024-04-02T13:43:46.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.868+0000 7f022c2ba640 1 -- 172.21.15.33:0/2338892411 wait complete. 2024-04-02T13:43:46.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:43:46.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:43:46.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.985+0000 7f8aa994a640 1 -- 172.21.15.33:0/794406420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aa4101370 msgr2=0x7f8aa410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.985+0000 7f8aa994a640 1 --2- 172.21.15.33:0/794406420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aa4101370 0x7f8aa410d880 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009cd0 tx=0x7f8a9402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.986+0000 7f8aa994a640 1 -- 172.21.15.33:0/794406420 shutdown_connections 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.986+0000 7f8aa994a640 1 --2- 172.21.15.33:0/794406420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aa4101370 0x7f8aa410d880 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.986+0000 7f8aa994a640 1 --2- 172.21.15.33:0/794406420 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8aa41009b0 0x7f8aa4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.986+0000 7f8aa994a640 1 --2- 172.21.15.33:0/794406420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aa41069c0 0x7f8aa4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.986+0000 7f8aa994a640 1 -- 172.21.15.33:0/794406420 >> 172.21.15.33:0/794406420 conn(0x7f8aa4076d80 msgr2=0x7f8aa4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.986+0000 7f8aa994a640 1 -- 172.21.15.33:0/794406420 shutdown_connections 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.986+0000 7f8aa994a640 1 -- 172.21.15.33:0/794406420 wait complete. 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.986+0000 7f8aa994a640 1 Processor -- start 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.986+0000 7f8aa994a640 1 -- start start 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa994a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aa41009b0 0x7f8aa4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa994a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8aa4101370 0x7f8aa41996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa994a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aa41069c0 0x7f8aa41a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa994a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4110da0 con 0x7f8aa41069c0 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa994a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4110c20 con 0x7f8aa41009b0 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa994a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4110f20 con 0x7f8aa4101370 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aa41069c0 0x7f8aa41a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aa41069c0 0x7f8aa41a41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41782/0 (socket says 172.21.15.33:41782) 2024-04-02T13:43:46.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa37fe640 1 -- 172.21.15.33:0/3015648925 learned_addr learned my addr 172.21.15.33:0/3015648925 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:46.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aa41009b0 0x7f8aa4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8aa4101370 0x7f8aa41996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa37fe640 1 -- 172.21.15.33:0/3015648925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8aa4101370 msgr2=0x7f8aa41996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa37fe640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8aa4101370 0x7f8aa41996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa37fe640 1 -- 172.21.15.33:0/3015648925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aa41009b0 msgr2=0x7f8aa4199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa37fe640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aa41009b0 0x7f8aa4199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa37fe640 1 -- 172.21.15.33:0/3015648925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa41a4720 con 0x7f8aa41069c0 2024-04-02T13:43:46.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa37fe640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aa41069c0 0x7f8aa41a41e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8a94005cc0 tx=0x7f8a94034d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:46.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa8948640 1 -- 172.21.15.33:0/3015648925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94039050 con 0x7f8aa41069c0 2024-04-02T13:43:46.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa994a640 1 -- 172.21.15.33:0/3015648925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa41a48f0 con 0x7f8aa41069c0 2024-04-02T13:43:46.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.987+0000 7f8aa8948640 1 -- 172.21.15.33:0/3015648925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8a94045070 con 0x7f8aa41069c0 2024-04-02T13:43:46.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.988+0000 7f8aa8948640 1 -- 172.21.15.33:0/3015648925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94004610 con 0x7f8aa41069c0 2024-04-02T13:43:46.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.988+0000 7f8aa994a640 1 -- 172.21.15.33:0/3015648925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa41a4d60 con 0x7f8aa41069c0 2024-04-02T13:43:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aa8948640 1 -- 172.21.15.33:0/3015648925 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a94003970 con 0x7f8aa41069c0 2024-04-02T13:43:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aac605740 1 Processor -- start 2024-04-02T13:43:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aac605740 1 -- start start 2024-04-02T13:43:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aac605740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 0x55f1761739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aac605740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173f20 0x55f176180ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aac605740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1761811e0 0x55f1761835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aac605740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f1760d7830 con 0x55f176173f20 2024-04-02T13:43:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aac605740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f17607f820 con 0x55f1761811e0 2024-04-02T13:43:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aac605740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f1760d37e0 con 0x55f176173600 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173f20 0x55f176180ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173f20 0x55f176180ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41794/0 (socket says 172.21.15.33:41794) 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aa2ffd640 1 -- 172.21.15.33:0/2878425141 learned_addr learned my addr 172.21.15.33:0/2878425141 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aa8948640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8a900a3d20 0x7f8a900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/2878425141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 0x55f1761739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aa37fe640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8a900a3d20 0x7f8a900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aa2ffd640 1 -- 172.21.15.33:0/2878425141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 msgr2=0x55f1761739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/2878425141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 0x55f1761739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/2878425141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1761811e0 0x55f1761835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.990+0000 7f8aa2ffd640 1 -- 172.21.15.33:0/2878425141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1761811e0 msgr2=0x55f1761835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.991+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/2878425141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1761811e0 0x55f1761835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.991+0000 7f8aa2ffd640 1 -- 172.21.15.33:0/2878425141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f175eb57d0 con 0x55f176173f20 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.991+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/2878425141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1761811e0 0x55f1761835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.991+0000 7f8aa8948640 1 -- 172.21.15.33:0/3015648925 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8a940f3480 con 0x7f8aa41069c0 2024-04-02T13:43:47.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.991+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/2878425141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173f20 0x55f176180ca0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00ac90 tx=0x7f8a9c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac5e5db01d5ec9b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.991+0000 7f8a99ffb640 1 -- 172.21.15.33:0/2878425141 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c01d070 con 0x55f176173f20 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.991+0000 7f8aa37fe640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8a900a3d20 0x7f8a900a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a940c79a0 tx=0x7f8a94091d50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.991+0000 7f8a99ffb640 1 -- 172.21.15.33:0/2878425141 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8a9c018b10 con 0x55f176173f20 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.991+0000 7f8a99ffb640 1 -- 172.21.15.33:0/2878425141 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c010b00 con 0x55f176173f20 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.992+0000 7f8aac605740 1 -- 172.21.15.33:0/2878425141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173f20 msgr2=0x55f176180ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.992+0000 7f8aac605740 1 --2- 172.21.15.33:0/2878425141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173f20 0x55f176180ca0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00ac90 tx=0x7f8a9c00f790 comp rx=0 tx=0).stop 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.992+0000 7f8aac605740 1 -- 172.21.15.33:0/2878425141 shutdown_connections 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.992+0000 7f8aac605740 1 --2- 172.21.15.33:0/2878425141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1761811e0 0x55f1761835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.992+0000 7f8aac605740 1 --2- 172.21.15.33:0/2878425141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173f20 0x55f176180ca0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.992+0000 7f8aac605740 1 --2- 172.21.15.33:0/2878425141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 0x55f1761739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.992+0000 7f8aac605740 1 -- 172.21.15.33:0/2878425141 >> 172.21.15.33:0/2878425141 conn(0x55f17616e000 msgr2=0x55f17616e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.992+0000 7f8aac605740 1 -- 172.21.15.33:0/2878425141 shutdown_connections 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.992+0000 7f8aac605740 1 -- 172.21.15.33:0/2878425141 wait complete. 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.992+0000 7f8aac605740 1 -- start start 2024-04-02T13:43:47.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.992+0000 7f8aac605740 1 Processor -- start 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.992+0000 7f8aac605740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 0x55f176180ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aac605740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f176173f20 0x55f17617de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aac605740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1761811e0 0x55f17617e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aac605740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f176183fa0 con 0x55f1761811e0 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aac605740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f176092d30 con 0x55f176173f20 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aac605740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f176184120 con 0x55f176173600 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aac605740 10 client.0 _ll_register_callbacks cb 0x55f176092720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aa27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1761811e0 0x55f17617e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aa27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1761811e0 0x55f17617e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41810/0 (socket says 172.21.15.33:41810) 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aa27fc640 1 -- 172.21.15.33:0/3892900332 learned_addr learned my addr 172.21.15.33:0/3892900332 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f176173f20 0x55f17617de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 0x55f176180ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aa27fc640 1 -- 172.21.15.33:0/3892900332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 msgr2=0x55f176180ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 0x55f176180ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aa27fc640 1 -- 172.21.15.33:0/3892900332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f176173f20 msgr2=0x55f17617de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f176173f20 0x55f17617de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aa27fc640 1 -- 172.21.15.33:0/3892900332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f175deb3f0 con 0x55f1761811e0 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f176173f20 0x55f17617de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1761811e0 0x55f17617e390 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c012fd0 tx=0x7f8a8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217c1c97b0826ff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8a82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8a82ffd640 1 -- 172.21.15.33:0/3892900332 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c012110 con 0x55f1761811e0 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aac605740 1 -- 172.21.15.33:0/3892900332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a9c018b10 con 0x55f1761811e0 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8a82ffd640 1 -- 172.21.15.33:0/3892900332 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8a8c004020 con 0x55f1761811e0 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8a82ffd640 1 -- 172.21.15.33:0/3892900332 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8a8c025050 con 0x55f1761811e0 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aac605740 10 client.16749 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.993+0000 7f8aac605740 1 -- 172.21.15.33:0/3892900332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f175dab0f0 con 0x55f1761811e0 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.994+0000 7f8a82ffd640 1 -- 172.21.15.33:0/3892900332 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8a9c018b10 con 0x55f1761811e0 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.994+0000 7f8a82ffd640 1 -- 172.21.15.33:0/3892900332 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8a8c003be0 con 0x55f1761811e0 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.994+0000 7f8aac605740 10 client.16749 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:47.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.994+0000 7f8aac605740 10 client.16749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.994+0000 7f8aac605740 20 client.16749 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.994+0000 7f8aac605740 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f176191250 0x55f1761936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.994+0000 7f8aac605740 4 client.16749 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.994+0000 7f8aac605740 1 -- 172.21.15.33:0/3892900332 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1760e4d70 con 0x55f176191250 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.994+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f176191250 0x55f1761936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.995+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f176191250 0x55f1761936c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df315bea465106b7 server_cookie=9010b066af19d072 in_seq=0 out_seq=0 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.995+0000 7f8a82ffd640 10 client.16749 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.995+0000 7f8a82ffd640 1 -- 172.21.15.33:0/3892900332 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1760e4d70 con 0x55f176191250 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.995+0000 7f8a82ffd640 10 client.16749 handle_command_reply: tid=0 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.995+0000 7f8aac605740 1 client.16749 shutdown 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.996+0000 7f8aac605740 1 -- 172.21.15.33:0/3892900332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1761811e0 msgr2=0x55f17617e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.996+0000 7f8aac605740 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1761811e0 0x55f17617e390 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c012fd0 tx=0x7f8a8c0046c0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.996+0000 7f8aac605740 1 -- 172.21.15.33:0/3892900332 shutdown_connections 2024-04-02T13:43:47.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.996+0000 7f8aac605740 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f176191250 0x55f1761936c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.996+0000 7f8aac605740 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1761811e0 0x55f17617e390 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.996+0000 7f8aac605740 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f176173f20 0x55f17617de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.996+0000 7f8a82ffd640 0 client.16749 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:47.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.996+0000 7f8aac605740 1 --2- 172.21.15.33:0/3892900332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 0x55f176180ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.996+0000 7f8aac605740 1 -- 172.21.15.33:0/3892900332 >> 172.21.15.33:0/3892900332 conn(0x55f17616e000 msgr2=0x55f17616fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:47.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.996+0000 7f8aac605740 1 -- 172.21.15.33:0/3892900332 shutdown_connections 2024-04-02T13:43:47.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.996+0000 7f8aac605740 1 -- 172.21.15.33:0/3892900332 wait complete. 2024-04-02T13:43:47.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:46.996+0000 7f8aac605740 20 client.16749 trim_cache size 0 max 16384 2024-04-02T13:43:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aac605740 1 Processor -- start 2024-04-02T13:43:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aac605740 1 -- start start 2024-04-02T13:43:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aac605740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f17616e000 0x55f17618ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aac605740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173600 0x55f17617cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aac605740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173f20 0x55f176180580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aac605740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41af600 con 0x55f176173600 2024-04-02T13:43:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aac605740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41ace10 con 0x55f17616e000 2024-04-02T13:43:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aac605740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41aac40 con 0x55f176173f20 2024-04-02T13:43:47.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173600 0x55f17617cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173600 0x55f17617cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41820/0 (socket says 172.21.15.33:41820) 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aa2ffd640 1 -- 172.21.15.33:0/263713519 learned_addr learned my addr 172.21.15.33:0/263713519 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/263713519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f17616e000 0x55f17618ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aa2ffd640 1 -- 172.21.15.33:0/263713519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173f20 msgr2=0x55f176180580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/263713519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173f20 0x55f176180580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/263713519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173f20 0x55f176180580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aa2ffd640 1 -- 172.21.15.33:0/263713519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f17616e000 msgr2=0x55f17618ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/263713519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f17616e000 0x55f17618ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aa2ffd640 1 -- 172.21.15.33:0/263713519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17616de30 con 0x55f176173600 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.014+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/263713519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173600 0x55f17617cd90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00fc90 tx=0x7f8a9c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f6ac9c31a4c5a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.015+0000 7f8a997fa640 1 -- 172.21.15.33:0/263713519 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c01d040 con 0x55f176173600 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.015+0000 7f8a997fa640 1 -- 172.21.15.33:0/263713519 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8a9c02d020 con 0x55f176173600 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.015+0000 7f8a997fa640 1 -- 172.21.15.33:0/263713519 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c010d90 con 0x55f176173600 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.015+0000 7f8aac605740 1 -- 172.21.15.33:0/263713519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173600 msgr2=0x55f17617cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.015+0000 7f8aac605740 1 --2- 172.21.15.33:0/263713519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173600 0x55f17617cd90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00fc90 tx=0x7f8a9c00af90 comp rx=0 tx=0).stop 2024-04-02T13:43:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.015+0000 7f8aac605740 1 -- 172.21.15.33:0/263713519 shutdown_connections 2024-04-02T13:43:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.015+0000 7f8aac605740 1 --2- 172.21.15.33:0/263713519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173f20 0x55f176180580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.015+0000 7f8aac605740 1 --2- 172.21.15.33:0/263713519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f176173600 0x55f17617cd90 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.015+0000 7f8aac605740 1 --2- 172.21.15.33:0/263713519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f17616e000 0x55f17618ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.015+0000 7f8aac605740 1 -- 172.21.15.33:0/263713519 >> 172.21.15.33:0/263713519 conn(0x55f17617e4f0 msgr2=0x55f1761794d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.015+0000 7f8aac605740 1 -- 172.21.15.33:0/263713519 shutdown_connections 2024-04-02T13:43:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.015+0000 7f8aac605740 1 -- 172.21.15.33:0/263713519 wait complete. 2024-04-02T13:43:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aac605740 1 -- start start 2024-04-02T13:43:47.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aac605740 1 Processor -- start 2024-04-02T13:43:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aac605740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f17616e000 0x55f1761898e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aac605740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 0x55f17617a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aac605740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f176173f20 0x55f17617b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aac605740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41ab2d0 con 0x55f17616e000 2024-04-02T13:43:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aac605740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41ac740 con 0x55f176173f20 2024-04-02T13:43:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aac605740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41ab9a0 con 0x55f176173600 2024-04-02T13:43:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aac605740 10 client.0 _ll_register_callbacks cb 0x7f8a8c012c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aa27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f17616e000 0x55f1761898e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aa27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f17616e000 0x55f1761898e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41826/0 (socket says 172.21.15.33:41826) 2024-04-02T13:43:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aa27fc640 1 -- 172.21.15.33:0/3678971082 learned_addr learned my addr 172.21.15.33:0/3678971082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:43:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/3678971082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 0x55f17617a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.016+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/3678971082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f176173f20 0x55f17617b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8aa27fc640 1 -- 172.21.15.33:0/3678971082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 msgr2=0x55f17617a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/3678971082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 0x55f17617a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8aa27fc640 1 -- 172.21.15.33:0/3678971082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f176173f20 msgr2=0x55f17617b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/3678971082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f176173f20 0x55f17617b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8aa27fc640 1 -- 172.21.15.33:0/3678971082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1761f6800 con 0x55f17616e000 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8aa27fc640 1 --2- 172.21.15.33:0/3678971082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f17616e000 0x55f1761898e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c001eb0 tx=0x7f8a8c0127f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451df2bc4ddfabd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8a82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8a82ffd640 1 -- 172.21.15.33:0/3678971082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c00dba0 con 0x55f17616e000 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8aac605740 1 -- 172.21.15.33:0/3678971082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a9c02d020 con 0x55f17616e000 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8a82ffd640 1 -- 172.21.15.33:0/3678971082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8a8c0041c0 con 0x55f17616e000 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8a82ffd640 1 -- 172.21.15.33:0/3678971082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8a8c03a050 con 0x55f17616e000 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8aac605740 10 client.16755 fetch_fsmap learned FSMap version 30 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8aac605740 1 -- 172.21.15.33:0/3678971082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f1761f7980 con 0x55f17616e000 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.017+0000 7f8a82ffd640 1 -- 172.21.15.33:0/3678971082 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8a9c02d020 con 0x55f17616e000 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.018+0000 7f8a82ffd640 1 -- 172.21.15.33:0/3678971082 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8a8c0050d0 con 0x55f17616e000 2024-04-02T13:43:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.018+0000 7f8aac605740 10 client.16755 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.018+0000 7f8aac605740 10 client.16755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.018+0000 7f8aac605740 20 client.16755 populate_metadata read hostname 'smithi033' 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.018+0000 7f8aac605740 1 --2- 172.21.15.33:0/3678971082 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f1761811e0 0x55f1761fa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.018+0000 7f8aac605740 4 client.16755 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.018+0000 7f8aac605740 1 -- 172.21.15.33:0/3678971082 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f1760df0c0 con 0x55f1761811e0 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.018+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/3678971082 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f1761811e0 0x55f1761fa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.018+0000 7f8aa2ffd640 1 --2- 172.21.15.33:0/3678971082 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f1761811e0 0x55f1761fa140 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51fa67d9d587edb4 server_cookie=5d4e008e62d67aff in_seq=0 out_seq=0 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.018+0000 7f8a82ffd640 10 client.16755 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8a82ffd640 1 -- 172.21.15.33:0/3678971082 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f1760df0c0 con 0x55f1761811e0 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8a82ffd640 10 client.16755 handle_command_reply: tid=0 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8aac605740 1 client.16755 shutdown 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8aac605740 1 -- 172.21.15.33:0/3678971082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f17616e000 msgr2=0x55f1761898e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8aac605740 1 --2- 172.21.15.33:0/3678971082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f17616e000 0x55f1761898e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c001eb0 tx=0x7f8a8c0127f0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8aac605740 1 -- 172.21.15.33:0/3678971082 shutdown_connections 2024-04-02T13:43:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8aac605740 1 --2- 172.21.15.33:0/3678971082 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f1761811e0 0x55f1761fa140 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8aac605740 1 --2- 172.21.15.33:0/3678971082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f176173f20 0x55f17617b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8aac605740 1 --2- 172.21.15.33:0/3678971082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f176173600 0x55f17617a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8a82ffd640 0 client.16755 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:43:47.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8aac605740 1 --2- 172.21.15.33:0/3678971082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f17616e000 0x55f1761898e0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8aac605740 1 -- 172.21.15.33:0/3678971082 >> 172.21.15.33:0/3678971082 conn(0x55f17617e4f0 msgr2=0x55f17616c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:47.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8aac605740 1 -- 172.21.15.33:0/3678971082 shutdown_connections 2024-04-02T13:43:47.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8aac605740 1 -- 172.21.15.33:0/3678971082 wait complete. 2024-04-02T13:43:47.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.019+0000 7f8aac605740 20 client.16755 trim_cache size 0 max 16384 2024-04-02T13:43:47.028 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:43:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:43:47.028 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:43:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "b11a345e-1cf6-4f10-9cfe-c0d63739c0b7": { 2024-04-02T13:43:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:43:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "b11a345e-1cf6-4f10-9cfe-c0d63739c0b7", 2024-04-02T13:43:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:43:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:43:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:43:47.029 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:43:47.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.020+0000 7f8aa994a640 1 -- 172.21.15.33:0/3015648925 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8a900a3d20 msgr2=0x7f8a900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:47.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.020+0000 7f8aa994a640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8a900a3d20 0x7f8a900a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a940c79a0 tx=0x7f8a94091d50 comp rx=0 tx=0).stop 2024-04-02T13:43:47.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.020+0000 7f8aa994a640 1 -- 172.21.15.33:0/3015648925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aa41069c0 msgr2=0x7f8aa41a41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:47.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.020+0000 7f8aa994a640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aa41069c0 0x7f8aa41a41e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8a94005cc0 tx=0x7f8a94034d70 comp rx=0 tx=0).stop 2024-04-02T13:43:47.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.020+0000 7f8aa994a640 1 -- 172.21.15.33:0/3015648925 shutdown_connections 2024-04-02T13:43:47.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.020+0000 7f8aa994a640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8a900a3d20 0x7f8a900a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.020+0000 7f8aa994a640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8aa41069c0 0x7f8aa41a41e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.020+0000 7f8aa994a640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8aa4101370 0x7f8aa41996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.020+0000 7f8aa994a640 1 --2- 172.21.15.33:0/3015648925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aa41009b0 0x7f8aa4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:47.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.020+0000 7f8aa994a640 1 -- 172.21.15.33:0/3015648925 >> 172.21.15.33:0/3015648925 conn(0x7f8aa4076d80 msgr2=0x7f8aa40fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:47.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.020+0000 7f8aa994a640 1 -- 172.21.15.33:0/3015648925 shutdown_connections 2024-04-02T13:43:47.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:43:47.020+0000 7f8aa994a640 1 -- 172.21.15.33:0/3015648925 wait complete. 2024-04-02T13:43:47.034 INFO:tasks.cephfs.filesystem:scrub status for tag:b11a345e-1cf6-4f10-9cfe-c0d63739c0b7 - {'path': '/', 'tag': 'b11a345e-1cf6-4f10-9cfe-c0d63739c0b7', 'options': 'recursive,force'} 2024-04-02T13:43:48.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:48 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:43:48.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:48 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:48.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:48 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:43:48.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:48 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:48.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:48 smithi145 ceph-mon[35646]: pgmap v153: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:48.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:48 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:43:48.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:48 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:48.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:48 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:43:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:48 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:48.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:48 smithi050 ceph-mon[35524]: pgmap v153: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:48.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:48 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:43:48.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:48 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:48.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:48 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:43:48.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:48 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:48.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:48 smithi033 ceph-mon[32390]: pgmap v153: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:43:50.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:50 smithi145 ceph-mon[35646]: pgmap v154: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:43:50.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:50 smithi050 ceph-mon[35524]: pgmap v154: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:43:50.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:50 smithi033 ceph-mon[32390]: pgmap v154: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:43:51.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:51 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:43:51.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:51 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:43:51.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:51 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:43:52.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:52 smithi050 ceph-mon[35524]: pgmap v155: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:52.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:52 smithi033 ceph-mon[32390]: pgmap v155: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:52.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:52 smithi145 ceph-mon[35646]: pgmap v155: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:54.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:54 smithi050 ceph-mon[35524]: pgmap v156: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:43:54.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:54 smithi033 ceph-mon[32390]: pgmap v156: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:43:54.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:54 smithi145 ceph-mon[35646]: pgmap v156: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:43:56.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:56 smithi050 ceph-mon[35524]: pgmap v157: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:56.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:56 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:43:56.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:56 smithi033 ceph-mon[32390]: pgmap v157: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:56.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:56 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:43:56.530 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:56 smithi145 ceph-mon[35646]: pgmap v157: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:56.530 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:56 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:43:57.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:57 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:43:57.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:57 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:43:57.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:57 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:43:57.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:57 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:43:57.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:57 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:43:57.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:57 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:43:57.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:57 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:43:57.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:57 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:43:57.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:57 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:43:58.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:43:58 smithi145 ceph-mon[35646]: pgmap v158: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:58.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:43:58 smithi050 ceph-mon[35524]: pgmap v158: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:43:58.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:43:58 smithi033 ceph-mon[32390]: pgmap v158: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:00.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:00 smithi145 ceph-mon[35646]: pgmap v159: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:44:00.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:00 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:00.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:00 smithi050 ceph-mon[35524]: pgmap v159: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:44:00.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:00 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:00.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:00 smithi033 ceph-mon[32390]: pgmap v159: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:44:00.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:00 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:02.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:02 smithi145 ceph-mon[35646]: pgmap v160: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:02.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:02 smithi050 ceph-mon[35524]: pgmap v160: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:02 smithi033 ceph-mon[32390]: pgmap v160: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:04.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:04 smithi145 ceph-mon[35646]: pgmap v161: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:44:04.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:04 smithi050 ceph-mon[35524]: pgmap v161: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:44:04.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:04 smithi033 ceph-mon[32390]: pgmap v161: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:44:06.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:06 smithi145 ceph-mon[35646]: pgmap v162: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:44:06.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:06 smithi050 ceph-mon[35524]: pgmap v162: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:44:06.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:06 smithi033 ceph-mon[32390]: pgmap v162: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:44:08.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:08 smithi145 ceph-mon[35646]: pgmap v163: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:44:08.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:08 smithi050 ceph-mon[35524]: pgmap v163: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:44:08.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:08 smithi033 ceph-mon[32390]: pgmap v163: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:44:10.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:10 smithi145 ceph-mon[35646]: pgmap v164: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:44:10.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:10 smithi050 ceph-mon[35524]: pgmap v164: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:44:10.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:10 smithi033 ceph-mon[32390]: pgmap v164: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:44:12.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:12 smithi145 ceph-mon[35646]: pgmap v165: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:44:12.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:12 smithi050 ceph-mon[35524]: pgmap v165: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:44:12.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:12 smithi033 ceph-mon[32390]: pgmap v165: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-02T13:44:14.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:14 smithi145 ceph-mon[35646]: pgmap v166: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:44:14.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:14 smithi050 ceph-mon[35524]: pgmap v166: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:44:14.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:14 smithi033 ceph-mon[32390]: pgmap v166: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:44:16.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:16 smithi145 ceph-mon[35646]: pgmap v167: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T13:44:16.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:16 smithi050 ceph-mon[35524]: pgmap v167: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T13:44:16.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:16 smithi033 ceph-mon[32390]: pgmap v167: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T13:44:17.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:44:17.035 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:44:17.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.150+0000 7fe6341d6640 1 -- 172.21.15.33:0/2534006676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c10c140 msgr2=0x7fe62c10e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.150+0000 7fe6341d6640 1 --2- 172.21.15.33:0/2534006676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c10c140 0x7fe62c10e5b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe624009ca0 tx=0x7fe62402c180 comp rx=0 tx=0).stop 2024-04-02T13:44:17.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.150+0000 7fe6341d6640 1 -- 172.21.15.33:0/2534006676 shutdown_connections 2024-04-02T13:44:17.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.150+0000 7fe6341d6640 1 --2- 172.21.15.33:0/2534006676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c10c140 0x7fe62c10e5b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.150+0000 7fe6341d6640 1 --2- 172.21.15.33:0/2534006676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe62c103690 0x7fe62c10bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.150+0000 7fe6341d6640 1 --2- 172.21.15.33:0/2534006676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe62c102d70 0x7fe62c103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.150+0000 7fe6341d6640 1 -- 172.21.15.33:0/2534006676 >> 172.21.15.33:0/2534006676 conn(0x7fe62c077b60 msgr2=0x7fe62c077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.150+0000 7fe6341d6640 1 -- 172.21.15.33:0/2534006676 shutdown_connections 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.150+0000 7fe6341d6640 1 -- 172.21.15.33:0/2534006676 wait complete. 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe6341d6640 1 Processor -- start 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe6341d6640 1 -- start start 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe6341d6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe62c102d70 0x7fe62c1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe6341d6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe62c103690 0x7fe62c100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe6341d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c10c140 0x7fe62c101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe6341d6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c111ee0 con 0x7fe62c10c140 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe6341d6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c111d60 con 0x7fe62c103690 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe6341d6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c112060 con 0x7fe62c102d70 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe632773640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c10c140 0x7fe62c101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe632773640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c10c140 0x7fe62c101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59668/0 (socket says 172.21.15.33:59668) 2024-04-02T13:44:17.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe632773640 1 -- 172.21.15.33:0/1930031745 learned_addr learned my addr 172.21.15.33:0/1930031745 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe631771640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe62c103690 0x7fe62c100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe631f72640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe62c102d70 0x7fe62c1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.151+0000 7fe632773640 1 -- 172.21.15.33:0/1930031745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe62c102d70 msgr2=0x7fe62c1009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.152+0000 7fe632773640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe62c102d70 0x7fe62c1009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.152+0000 7fe632773640 1 -- 172.21.15.33:0/1930031745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe62c103690 msgr2=0x7fe62c100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.152+0000 7fe632773640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe62c103690 0x7fe62c100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.152+0000 7fe632773640 1 -- 172.21.15.33:0/1930031745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c1a6820 con 0x7fe62c10c140 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.152+0000 7fe631f72640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe62c102d70 0x7fe62c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.152+0000 7fe632773640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c10c140 0x7fe62c101440 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe624004af0 tx=0x7fe624034ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.152+0000 7fe622ffd640 1 -- 172.21.15.33:0/1930031745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624039050 con 0x7fe62c10c140 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.152+0000 7fe622ffd640 1 -- 172.21.15.33:0/1930031745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe624045070 con 0x7fe62c10c140 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.152+0000 7fe622ffd640 1 -- 172.21.15.33:0/1930031745 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624032b30 con 0x7fe62c10c140 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.152+0000 7fe6341d6640 1 -- 172.21.15.33:0/1930031745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c1a6ab0 con 0x7fe62c10c140 2024-04-02T13:44:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.152+0000 7fe6341d6640 1 -- 172.21.15.33:0/1930031745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c1a6ff0 con 0x7fe62c10c140 2024-04-02T13:44:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.154+0000 7fe622ffd640 1 -- 172.21.15.33:0/1930031745 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe6240046f0 con 0x7fe62c10c140 2024-04-02T13:44:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.154+0000 7fe636dbd740 1 Processor -- start 2024-04-02T13:44:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe636dbd740 1 -- start start 2024-04-02T13:44:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe622ffd640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe6180a3dd0 0x7fe6180a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe636dbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 0x56447c4c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe636dbd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4c21f0 0x56447c4ca890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe636dbd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4cadd0 0x56447c4cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe636dbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56447c425830 con 0x56447c4c18d0 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe636dbd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56447c3cd820 con 0x56447c4cadd0 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe636dbd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56447c4217e0 con 0x56447c4c21f0 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe631f72640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 0x56447c4c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe622ffd640 1 -- 172.21.15.33:0/1930031745 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe6240f34d0 con 0x7fe62c10c140 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe631771640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe6180a3dd0 0x7fe6180a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe631f72640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 0x56447c4c1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59670/0 (socket says 172.21.15.33:59670) 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe631f72640 1 -- 172.21.15.33:0/1869526789 learned_addr learned my addr 172.21.15.33:0/1869526789 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe632773640 1 --2- 172.21.15.33:0/1869526789 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4cadd0 0x56447c4cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.155+0000 7fe631771640 1 --2- 172.21.15.33:0/1869526789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4c21f0 0x56447c4ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.156+0000 7fe631f72640 1 -- 172.21.15.33:0/1869526789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4c21f0 msgr2=0x56447c4ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.156+0000 7fe631f72640 1 --2- 172.21.15.33:0/1869526789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4c21f0 0x56447c4ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.156+0000 7fe631f72640 1 -- 172.21.15.33:0/1869526789 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4cadd0 msgr2=0x56447c4cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.156+0000 7fe631f72640 1 --2- 172.21.15.33:0/1869526789 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4cadd0 0x56447c4cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.156+0000 7fe631f72640 1 -- 172.21.15.33:0/1869526789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56447c2037d0 con 0x56447c4c18d0 2024-04-02T13:44:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.156+0000 7fe631771640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe6180a3dd0 0x7fe6180a6290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0058c0 tx=0x7fe61c0137d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.156+0000 7fe631f72640 1 --2- 172.21.15.33:0/1869526789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 0x56447c4c1cb0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe62800bb70 tx=0x7fe62800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3053bb44d131270 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.156+0000 7fe617fff640 1 -- 172.21.15.33:0/1869526789 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628017620 con 0x56447c4c18d0 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.156+0000 7fe617fff640 1 -- 172.21.15.33:0/1869526789 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe628017c90 con 0x56447c4c18d0 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.156+0000 7fe617fff640 1 -- 172.21.15.33:0/1869526789 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62800faf0 con 0x56447c4c18d0 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.157+0000 7fe636dbd740 1 -- 172.21.15.33:0/1869526789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 msgr2=0x56447c4c1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.157+0000 7fe636dbd740 1 --2- 172.21.15.33:0/1869526789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 0x56447c4c1cb0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe62800bb70 tx=0x7fe62800d710 comp rx=0 tx=0).stop 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.157+0000 7fe636dbd740 1 -- 172.21.15.33:0/1869526789 shutdown_connections 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.157+0000 7fe636dbd740 1 --2- 172.21.15.33:0/1869526789 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4cadd0 0x56447c4cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.157+0000 7fe636dbd740 1 --2- 172.21.15.33:0/1869526789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4c21f0 0x56447c4ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.157+0000 7fe636dbd740 1 --2- 172.21.15.33:0/1869526789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 0x56447c4c1cb0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.157+0000 7fe636dbd740 1 -- 172.21.15.33:0/1869526789 >> 172.21.15.33:0/1869526789 conn(0x56447c4bc000 msgr2=0x56447c4bc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.157+0000 7fe636dbd740 1 -- 172.21.15.33:0/1869526789 shutdown_connections 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.157+0000 7fe636dbd740 1 -- 172.21.15.33:0/1869526789 wait complete. 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.157+0000 7fe636dbd740 1 -- start start 2024-04-02T13:44:17.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.157+0000 7fe636dbd740 1 Processor -- start 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe636dbd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4c18d0 0x56447c4ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe636dbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 0x56447c4c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe636dbd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4cadd0 0x56447c4c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe636dbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56447c4cdbc0 con 0x56447c4c21f0 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe636dbd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56447c3e0d30 con 0x56447c4cadd0 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe636dbd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56447c4cdd40 con 0x56447c4c18d0 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe636dbd740 10 client.0 _ll_register_callbacks cb 0x56447c3e0720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe631771640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 0x56447c4c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe631771640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 0x56447c4c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59684/0 (socket says 172.21.15.33:59684) 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe631f72640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4c18d0 0x56447c4ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe631771640 1 -- 172.21.15.33:0/2621989431 learned_addr learned my addr 172.21.15.33:0/2621989431 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe632773640 1 --2- 172.21.15.33:0/2621989431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4cadd0 0x56447c4c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe631771640 1 -- 172.21.15.33:0/2621989431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4c18d0 msgr2=0x56447c4ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe631771640 1 --2- 172.21.15.33:0/2621989431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4c18d0 0x56447c4ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe631771640 1 -- 172.21.15.33:0/2621989431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4cadd0 msgr2=0x56447c4c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe631771640 1 --2- 172.21.15.33:0/2621989431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4cadd0 0x56447c4c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe631771640 1 -- 172.21.15.33:0/2621989431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56447c1393f0 con 0x56447c4c21f0 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe631771640 1 --2- 172.21.15.33:0/2621989431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 0x56447c4c79e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe61c011940 tx=0x7fe61c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8294c918e1ebfc11 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.158+0000 7fe6157fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe6157fa640 1 -- 172.21.15.33:0/2621989431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c025070 con 0x56447c4c21f0 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe636dbd740 1 -- 172.21.15.33:0/2621989431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe628017c90 con 0x56447c4c21f0 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe6157fa640 1 -- 172.21.15.33:0/2621989431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe61c00fd00 con 0x56447c4c21f0 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe6157fa640 1 -- 172.21.15.33:0/2621989431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe61c030050 con 0x56447c4c21f0 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe636dbd740 10 client.16767 fetch_fsmap learned FSMap version 31 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe636dbd740 1 -- 172.21.15.33:0/2621989431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56447c0f90f0 con 0x56447c4c21f0 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe6157fa640 1 -- 172.21.15.33:0/2621989431 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe628017c90 con 0x56447c4c21f0 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe6157fa640 1 -- 172.21.15.33:0/2621989431 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fe61c00f050 con 0x56447c4c21f0 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe636dbd740 10 client.16767 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T13:44:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe636dbd740 10 client.16767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe636dbd740 20 client.16767 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe636dbd740 1 --2- 172.21.15.33:0/2621989431 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56447c4dad50 0x56447c4dd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe636dbd740 4 client.16767 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.159+0000 7fe636dbd740 1 -- 172.21.15.33:0/2621989431 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56447c432d70 con 0x56447c4dad50 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.160+0000 7fe631f72640 1 --2- 172.21.15.33:0/2621989431 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56447c4dad50 0x56447c4dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.160+0000 7fe631f72640 1 --2- 172.21.15.33:0/2621989431 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56447c4dad50 0x56447c4dd170 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf337ab7fcead9f server_cookie=5d4b746a14638c95 in_seq=0 out_seq=0 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.160+0000 7fe6157fa640 10 client.16767 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe6157fa640 1 -- 172.21.15.33:0/2621989431 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56447c432d70 con 0x56447c4dad50 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe6157fa640 10 client.16767 handle_command_reply: tid=0 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe636dbd740 1 client.16767 shutdown 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe636dbd740 1 -- 172.21.15.33:0/2621989431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 msgr2=0x56447c4c79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe636dbd740 1 --2- 172.21.15.33:0/2621989431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 0x56447c4c79e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe61c011940 tx=0x7fe61c019000 comp rx=0 tx=0).stop 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe636dbd740 1 -- 172.21.15.33:0/2621989431 shutdown_connections 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe636dbd740 1 --2- 172.21.15.33:0/2621989431 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56447c4dad50 0x56447c4dd170 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe636dbd740 1 --2- 172.21.15.33:0/2621989431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4cadd0 0x56447c4c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe636dbd740 1 --2- 172.21.15.33:0/2621989431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 0x56447c4c79e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe6157fa640 0 client.16767 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:17.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe636dbd740 1 --2- 172.21.15.33:0/2621989431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4c18d0 0x56447c4ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe636dbd740 1 -- 172.21.15.33:0/2621989431 >> 172.21.15.33:0/2621989431 conn(0x56447c4bc000 msgr2=0x56447c4bdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe636dbd740 1 -- 172.21.15.33:0/2621989431 shutdown_connections 2024-04-02T13:44:17.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe636dbd740 1 -- 172.21.15.33:0/2621989431 wait complete. 2024-04-02T13:44:17.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.161+0000 7fe636dbd740 20 client.16767 trim_cache size 0 max 16384 2024-04-02T13:44:17.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe636dbd740 1 Processor -- start 2024-04-02T13:44:17.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe636dbd740 1 -- start start 2024-04-02T13:44:17.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe636dbd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4c18d0 0x56447c4d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe636dbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 0x56447c4bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe636dbd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4cadd0 0x56447c4ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe636dbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c1b39f0 con 0x56447c4c21f0 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe636dbd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c1b1230 con 0x56447c4c18d0 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe636dbd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c1af060 con 0x56447c4cadd0 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe631771640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 0x56447c4bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe631771640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 0x56447c4bb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59686/0 (socket says 172.21.15.33:59686) 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe631771640 1 -- 172.21.15.33:0/470372297 learned_addr learned my addr 172.21.15.33:0/470372297 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe631771640 1 -- 172.21.15.33:0/470372297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4cadd0 msgr2=0x56447c4ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe631771640 1 --2- 172.21.15.33:0/470372297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4cadd0 0x56447c4ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe631f72640 1 --2- 172.21.15.33:0/470372297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4c18d0 0x56447c4d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe631771640 1 -- 172.21.15.33:0/470372297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4c18d0 msgr2=0x56447c4d6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe631771640 1 --2- 172.21.15.33:0/470372297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4c18d0 0x56447c4d6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe631771640 1 -- 172.21.15.33:0/470372297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56447c4bbe30 con 0x56447c4c21f0 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.179+0000 7fe631f72640 1 --2- 172.21.15.33:0/470372297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4c18d0 0x56447c4d6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.180+0000 7fe631771640 1 --2- 172.21.15.33:0/470372297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 0x56447c4bb240 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe61c02e040 tx=0x7fe61c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abe80043695b913 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.180+0000 7fe6177fe640 1 -- 172.21.15.33:0/470372297 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c025040 con 0x56447c4c21f0 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.180+0000 7fe6177fe640 1 -- 172.21.15.33:0/470372297 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe61c00f050 con 0x56447c4c21f0 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.180+0000 7fe6177fe640 1 -- 172.21.15.33:0/470372297 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c004020 con 0x56447c4c21f0 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.180+0000 7fe636dbd740 1 -- 172.21.15.33:0/470372297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 msgr2=0x56447c4bb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.180+0000 7fe636dbd740 1 --2- 172.21.15.33:0/470372297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 0x56447c4bb240 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe61c02e040 tx=0x7fe61c00b8f0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.180+0000 7fe636dbd740 1 -- 172.21.15.33:0/470372297 shutdown_connections 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.180+0000 7fe636dbd740 1 --2- 172.21.15.33:0/470372297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4cadd0 0x56447c4ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.180+0000 7fe636dbd740 1 --2- 172.21.15.33:0/470372297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c21f0 0x56447c4bb240 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.180+0000 7fe636dbd740 1 --2- 172.21.15.33:0/470372297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4c18d0 0x56447c4d6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.180+0000 7fe636dbd740 1 -- 172.21.15.33:0/470372297 >> 172.21.15.33:0/470372297 conn(0x56447c4bc000 msgr2=0x56447c4c2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.180+0000 7fe636dbd740 1 -- 172.21.15.33:0/470372297 shutdown_connections 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.180+0000 7fe636dbd740 1 -- 172.21.15.33:0/470372297 wait complete. 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe636dbd740 1 -- start start 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe636dbd740 1 Processor -- start 2024-04-02T13:44:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe636dbd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 0x56447c4d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe636dbd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4c21f0 0x56447c4c4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe636dbd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4cadd0 0x56447c4c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe636dbd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c1af6f0 con 0x56447c4c18d0 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe636dbd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c1b0b60 con 0x56447c4c21f0 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe636dbd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c1afdc0 con 0x56447c4cadd0 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe636dbd740 10 client.0 _ll_register_callbacks cb 0x7fe61c011820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe631f72640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 0x56447c4d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe631f72640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 0x56447c4d3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59698/0 (socket says 172.21.15.33:59698) 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe631f72640 1 -- 172.21.15.33:0/2584354134 learned_addr learned my addr 172.21.15.33:0/2584354134 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe631771640 1 --2- 172.21.15.33:0/2584354134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4c21f0 0x56447c4c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe631f72640 1 -- 172.21.15.33:0/2584354134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4cadd0 msgr2=0x56447c4c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe631f72640 1 --2- 172.21.15.33:0/2584354134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4cadd0 0x56447c4c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe631f72640 1 -- 172.21.15.33:0/2584354134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4c21f0 msgr2=0x56447c4c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe631f72640 1 --2- 172.21.15.33:0/2584354134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4c21f0 0x56447c4c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.181+0000 7fe631f72640 1 -- 172.21.15.33:0/2584354134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56447c4c5350 con 0x56447c4c18d0 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe631f72640 1 --2- 172.21.15.33:0/2584354134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 0x56447c4d3420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe62800b310 tx=0x7fe628000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf7eebb34d8c94b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe6157fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe6157fa640 1 -- 172.21.15.33:0/2584354134 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62802add0 con 0x56447c4c18d0 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe6157fa640 1 -- 172.21.15.33:0/2584354134 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe62802a3e0 con 0x56447c4c18d0 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe636dbd740 1 -- 172.21.15.33:0/2584354134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe61c00f050 con 0x56447c4c18d0 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe6157fa640 1 -- 172.21.15.33:0/2584354134 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe628038050 con 0x56447c4c18d0 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe636dbd740 10 client.16773 fetch_fsmap learned FSMap version 31 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe636dbd740 1 -- 172.21.15.33:0/2584354134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56447c5413d0 con 0x56447c4c18d0 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe6157fa640 1 -- 172.21.15.33:0/2584354134 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe61c00f050 con 0x56447c4c18d0 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe6157fa640 1 -- 172.21.15.33:0/2584354134 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fe62800faa0 con 0x56447c4c18d0 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe636dbd740 10 client.16773 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe636dbd740 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe636dbd740 20 client.16773 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe636dbd740 1 --2- 172.21.15.33:0/2584354134 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56447c541920 0x56447c543d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe636dbd740 4 client.16773 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:44:17.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.182+0000 7fe636dbd740 1 -- 172.21.15.33:0/2584354134 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56447c42d0c0 con 0x56447c541920 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe631771640 1 --2- 172.21.15.33:0/2584354134 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56447c541920 0x56447c543d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe631771640 1 --2- 172.21.15.33:0/2584354134 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56447c541920 0x56447c543d40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e424d4194b31d661 server_cookie=773c8d22f3bb77ab in_seq=0 out_seq=0 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe6157fa640 10 client.16773 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe6157fa640 1 -- 172.21.15.33:0/2584354134 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56447c42d0c0 con 0x56447c541920 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe6157fa640 10 client.16773 handle_command_reply: tid=0 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe636dbd740 1 client.16773 shutdown 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe636dbd740 1 -- 172.21.15.33:0/2584354134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 msgr2=0x56447c4d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe636dbd740 1 --2- 172.21.15.33:0/2584354134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 0x56447c4d3420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe62800b310 tx=0x7fe628000bc0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe636dbd740 1 -- 172.21.15.33:0/2584354134 shutdown_connections 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe636dbd740 1 --2- 172.21.15.33:0/2584354134 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56447c541920 0x56447c543d40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe636dbd740 1 --2- 172.21.15.33:0/2584354134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56447c4cadd0 0x56447c4c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe636dbd740 1 --2- 172.21.15.33:0/2584354134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56447c4c21f0 0x56447c4c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe6157fa640 0 client.16773 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe636dbd740 1 --2- 172.21.15.33:0/2584354134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56447c4c18d0 0x56447c4d3420 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe636dbd740 1 -- 172.21.15.33:0/2584354134 >> 172.21.15.33:0/2584354134 conn(0x56447c4bc000 msgr2=0x56447c4c6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe636dbd740 1 -- 172.21.15.33:0/2584354134 shutdown_connections 2024-04-02T13:44:17.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.183+0000 7fe636dbd740 1 -- 172.21.15.33:0/2584354134 wait complete. 2024-04-02T13:44:17.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.184+0000 7fe636dbd740 20 client.16773 trim_cache size 0 max 16384 2024-04-02T13:44:17.193 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:44:17.193 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:44:17.193 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:44:17.193 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:44:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.184+0000 7fe6341d6640 1 -- 172.21.15.33:0/1930031745 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe6180a3dd0 msgr2=0x7fe6180a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.184+0000 7fe6341d6640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe6180a3dd0 0x7fe6180a6290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0058c0 tx=0x7fe61c0137d0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.184+0000 7fe6341d6640 1 -- 172.21.15.33:0/1930031745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c10c140 msgr2=0x7fe62c101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.184+0000 7fe6341d6640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c10c140 0x7fe62c101440 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe624004af0 tx=0x7fe624034ed0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.184+0000 7fe6341d6640 1 -- 172.21.15.33:0/1930031745 shutdown_connections 2024-04-02T13:44:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.184+0000 7fe6341d6640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe6180a3dd0 0x7fe6180a6290 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.184+0000 7fe6341d6640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe62c10c140 0x7fe62c101440 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.184+0000 7fe6341d6640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe62c103690 0x7fe62c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.184+0000 7fe6341d6640 1 --2- 172.21.15.33:0/1930031745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe62c102d70 0x7fe62c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.184+0000 7fe6341d6640 1 -- 172.21.15.33:0/1930031745 >> 172.21.15.33:0/1930031745 conn(0x7fe62c077b60 msgr2=0x7fe62c0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.184+0000 7fe6341d6640 1 -- 172.21.15.33:0/1930031745 shutdown_connections 2024-04-02T13:44:17.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.184+0000 7fe6341d6640 1 -- 172.21.15.33:0/1930031745 wait complete. 2024-04-02T13:44:17.198 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:44:17.198 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:44:17.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.298+0000 7f916863c640 1 -- 172.21.15.33:0/3795259314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9160103690 msgr2=0x7f916010bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.298+0000 7f916863c640 1 --2- 172.21.15.33:0/3795259314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9160103690 0x7f916010bb70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9154009900 tx=0x7f915402bc30 comp rx=0 tx=0).stop 2024-04-02T13:44:17.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 -- 172.21.15.33:0/3795259314 shutdown_connections 2024-04-02T13:44:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 --2- 172.21.15.33:0/3795259314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f916010c140 0x7f916010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 --2- 172.21.15.33:0/3795259314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9160103690 0x7f916010bb70 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 --2- 172.21.15.33:0/3795259314 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9160102d70 0x7f9160103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 -- 172.21.15.33:0/3795259314 >> 172.21.15.33:0/3795259314 conn(0x7f9160077b00 msgr2=0x7f9160077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 -- 172.21.15.33:0/3795259314 shutdown_connections 2024-04-02T13:44:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 -- 172.21.15.33:0/3795259314 wait complete. 2024-04-02T13:44:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 Processor -- start 2024-04-02T13:44:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 -- start start 2024-04-02T13:44:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9160102d70 0x7f9160198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9160103690 0x7f9160198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f916010c140 0x7f916019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9160111f10 con 0x7f916010c140 2024-04-02T13:44:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9160111d90 con 0x7f9160102d70 2024-04-02T13:44:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.299+0000 7f916863c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9160112090 con 0x7f9160103690 2024-04-02T13:44:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f9166bd9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f916010c140 0x7f916019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f9166bd9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f916010c140 0x7f916019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59716/0 (socket says 172.21.15.33:59716) 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f9166bd9640 1 -- 172.21.15.33:0/1029575657 learned_addr learned my addr 172.21.15.33:0/1029575657 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f91663d8640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9160102d70 0x7f9160198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f9165bd7640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9160103690 0x7f9160198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f9166bd9640 1 -- 172.21.15.33:0/1029575657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9160103690 msgr2=0x7f9160198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f9166bd9640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9160103690 0x7f9160198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f9166bd9640 1 -- 172.21.15.33:0/1029575657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9160102d70 msgr2=0x7f9160198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f9166bd9640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9160102d70 0x7f9160198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f9166bd9640 1 -- 172.21.15.33:0/1029575657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916019d420 con 0x7f916010c140 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f9166bd9640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f916010c140 0x7f916019ccf0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f915800d750 tx=0x7f915800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f915f7fe640 1 -- 172.21.15.33:0/1029575657 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158017070 con 0x7f916010c140 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f915f7fe640 1 -- 172.21.15.33:0/1029575657 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f91580054f0 con 0x7f916010c140 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f916863c640 1 -- 172.21.15.33:0/1029575657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916019d710 con 0x7f916010c140 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f915f7fe640 1 -- 172.21.15.33:0/1029575657 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158004cf0 con 0x7f916010c140 2024-04-02T13:44:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.300+0000 7f916863c640 1 -- 172.21.15.33:0/1029575657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91601a4fa0 con 0x7f916010c140 2024-04-02T13:44:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.301+0000 7f916863c640 1 -- 172.21.15.33:0/1029575657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91600688f0 con 0x7f916010c140 2024-04-02T13:44:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.302+0000 7f915f7fe640 1 -- 172.21.15.33:0/1029575657 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9158004e90 con 0x7f916010c140 2024-04-02T13:44:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.303+0000 7f915f7fe640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f91480a3d20 0x7f91480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.303+0000 7f91663d8640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f91480a3d20 0x7f91480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.303+0000 7f915f7fe640 1 -- 172.21.15.33:0/1029575657 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f91580d0970 con 0x7f916010c140 2024-04-02T13:44:17.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.303+0000 7f91663d8640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f91480a3d20 0x7f91480a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9150005fd0 tx=0x7f9150009680 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.307+0000 7f915f7fe640 1 -- 172.21.15.33:0/1029575657 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f915809cc60 con 0x7f916010c140 2024-04-02T13:44:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.536+0000 7f916863c640 1 -- 172.21.15.33:0/1029575657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9160107230 con 0x7f916010c140 2024-04-02T13:44:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.537+0000 7f915f7fe640 1 -- 172.21.15.33:0/1029575657 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11880 (secure 0 0 0) 0x7f915809c560 con 0x7f916010c140 2024-04-02T13:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:44:17.540 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:43:59.225834+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24473,24512]},"id":1}]} 2024-04-02T13:44:17.540 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 31 2024-04-02T13:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.539+0000 7f916863c640 1 -- 172.21.15.33:0/1029575657 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f91480a3d20 msgr2=0x7f91480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.539+0000 7f916863c640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f91480a3d20 0x7f91480a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9150005fd0 tx=0x7f9150009680 comp rx=0 tx=0).stop 2024-04-02T13:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.539+0000 7f916863c640 1 -- 172.21.15.33:0/1029575657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f916010c140 msgr2=0x7f916019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.539+0000 7f916863c640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f916010c140 0x7f916019ccf0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f915800d750 tx=0x7f915800dc10 comp rx=0 tx=0).stop 2024-04-02T13:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.539+0000 7f916863c640 1 -- 172.21.15.33:0/1029575657 shutdown_connections 2024-04-02T13:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.539+0000 7f916863c640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f91480a3d20 0x7f91480a61e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.539+0000 7f916863c640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f916010c140 0x7f916019ccf0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.539+0000 7f916863c640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9160103690 0x7f9160198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.539+0000 7f916863c640 1 --2- 172.21.15.33:0/1029575657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9160102d70 0x7f9160198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.539+0000 7f916863c640 1 -- 172.21.15.33:0/1029575657 >> 172.21.15.33:0/1029575657 conn(0x7f9160077b00 msgr2=0x7f91600fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.539+0000 7f916863c640 1 -- 172.21.15.33:0/1029575657 shutdown_connections 2024-04-02T13:44:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.539+0000 7f916863c640 1 -- 172.21.15.33:0/1029575657 wait complete. 2024-04-02T13:44:17.553 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:44:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.655+0000 7f3e77e33640 1 -- 172.21.15.33:0/3083755377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e70101a90 msgr2=0x7f3e70101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.655+0000 7f3e77e33640 1 --2- 172.21.15.33:0/3083755377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e70101a90 0x7f3e70101f10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009930 tx=0x7f3e6002bc30 comp rx=0 tx=0).stop 2024-04-02T13:44:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.655+0000 7f3e77e33640 1 -- 172.21.15.33:0/3083755377 shutdown_connections 2024-04-02T13:44:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.655+0000 7f3e77e33640 1 --2- 172.21.15.33:0/3083755377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e70102450 0x7f3e7010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.655+0000 7f3e77e33640 1 --2- 172.21.15.33:0/3083755377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e70101a90 0x7f3e70101f10 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.655+0000 7f3e77e33640 1 --2- 172.21.15.33:0/3083755377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e70107aa0 0x7f3e70107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.655+0000 7f3e77e33640 1 -- 172.21.15.33:0/3083755377 >> 172.21.15.33:0/3083755377 conn(0x7f3e70078f00 msgr2=0x7f3e70079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.655+0000 7f3e77e33640 1 -- 172.21.15.33:0/3083755377 shutdown_connections 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.655+0000 7f3e77e33640 1 -- 172.21.15.33:0/3083755377 wait complete. 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e77e33640 1 Processor -- start 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e77e33640 1 -- start start 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e77e33640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e70101a90 0x7f3e701983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e77e33640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e70102450 0x7f3e70198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e77e33640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e70107aa0 0x7f3e7019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e77e33640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70111e80 con 0x7f3e70102450 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e77e33640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70111d00 con 0x7f3e70107aa0 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e77e33640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70112000 con 0x7f3e70101a90 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e753ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e70102450 0x7f3e70198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e753ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e70102450 0x7f3e70198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59748/0 (socket says 172.21.15.33:59748) 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e753ce640 1 -- 172.21.15.33:0/4122124721 learned_addr learned my addr 172.21.15.33:0/4122124721 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e763d0640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e70107aa0 0x7f3e7019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e75bcf640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e70101a90 0x7f3e701983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e753ce640 1 -- 172.21.15.33:0/4122124721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e70101a90 msgr2=0x7f3e701983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.656+0000 7f3e753ce640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e70101a90 0x7f3e701983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.657+0000 7f3e753ce640 1 -- 172.21.15.33:0/4122124721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e70107aa0 msgr2=0x7f3e7019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.657+0000 7f3e753ce640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e70107aa0 0x7f3e7019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.657+0000 7f3e753ce640 1 -- 172.21.15.33:0/4122124721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7019d3b0 con 0x7f3e70102450 2024-04-02T13:44:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.657+0000 7f3e763d0640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e70107aa0 0x7f3e7019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.657+0000 7f3e75bcf640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e70101a90 0x7f3e701983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.657+0000 7f3e753ce640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e70102450 0x7f3e70198900 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3e60000c00 tx=0x7f3e600045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.657+0000 7f3e66ffd640 1 -- 172.21.15.33:0/4122124721 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60043020 con 0x7f3e70102450 2024-04-02T13:44:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.657+0000 7f3e66ffd640 1 -- 172.21.15.33:0/4122124721 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3e6002caa0 con 0x7f3e70102450 2024-04-02T13:44:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.657+0000 7f3e77e33640 1 -- 172.21.15.33:0/4122124721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7019d640 con 0x7f3e70102450 2024-04-02T13:44:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.657+0000 7f3e66ffd640 1 -- 172.21.15.33:0/4122124721 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60003bf0 con 0x7f3e70102450 2024-04-02T13:44:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.657+0000 7f3e77e33640 1 -- 172.21.15.33:0/4122124721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e701a4f30 con 0x7f3e70102450 2024-04-02T13:44:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.660+0000 7f3e66ffd640 1 -- 172.21.15.33:0/4122124721 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3e60046020 con 0x7f3e70102450 2024-04-02T13:44:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.660+0000 7f3e7aa17740 1 Processor -- start 2024-04-02T13:44:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.660+0000 7f3e7aa17740 1 -- start start 2024-04-02T13:44:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.660+0000 7f3e7aa17740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0d7a0 0x563b95e0db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.660+0000 7f3e7aa17740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0e0c0 0x563b95e16760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.660+0000 7f3e7aa17740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 0x563b95e19090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.660+0000 7f3e7aa17740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563b95d717b0 con 0x563b95e16ca0 2024-04-02T13:44:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.660+0000 7f3e7aa17740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563b95d10570 con 0x563b95e0e0c0 2024-04-02T13:44:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.660+0000 7f3e7aa17740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563b95d6d760 con 0x563b95e0d7a0 2024-04-02T13:44:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e753ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 0x563b95e19090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e75bcf640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0e0c0 0x563b95e16760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e753ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 0x563b95e19090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59752/0 (socket says 172.21.15.33:59752) 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e753ce640 1 -- 172.21.15.33:0/603681161 learned_addr learned my addr 172.21.15.33:0/603681161 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e75bcf640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0e0c0 0x563b95e16760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:33192/0 (socket says 172.21.15.33:33192) 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e763d0640 1 --2- 172.21.15.33:0/603681161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0d7a0 0x563b95e0db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e66ffd640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3e5c0a3d20 0x7f3e5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e753ce640 1 -- 172.21.15.33:0/603681161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0d7a0 msgr2=0x563b95e0db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e753ce640 1 --2- 172.21.15.33:0/603681161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0d7a0 0x563b95e0db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e753ce640 1 -- 172.21.15.33:0/603681161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0e0c0 msgr2=0x563b95e16760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e753ce640 1 --2- 172.21.15.33:0/603681161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0e0c0 0x563b95e16760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e753ce640 1 -- 172.21.15.33:0/603681161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b95a853f0 con 0x563b95e16ca0 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e763d0640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3e5c0a3d20 0x7f3e5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e66ffd640 1 -- 172.21.15.33:0/4122124721 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3e600f25d0 con 0x7f3e70102450 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e753ce640 1 --2- 172.21.15.33:0/603681161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 0x563b95e19090 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3e600efa00 tx=0x7f3e6004d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa8becc406a9136 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e5bfff640 1 -- 172.21.15.33:0/603681161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e600bde50 con 0x563b95e16ca0 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e763d0640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3e5c0a3d20 0x7f3e5c0a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e68005fd0 tx=0x7f3e68004a40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e5bfff640 1 -- 172.21.15.33:0/603681161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3e600bea10 con 0x563b95e16ca0 2024-04-02T13:44:17.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.661+0000 7f3e5bfff640 1 -- 172.21.15.33:0/603681161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e600bdc70 con 0x563b95e16ca0 2024-04-02T13:44:17.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.662+0000 7f3e7aa17740 1 -- 172.21.15.33:0/603681161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 msgr2=0x563b95e19090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.662+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/603681161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 0x563b95e19090 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3e600efa00 tx=0x7f3e6004d000 comp rx=0 tx=0).stop 2024-04-02T13:44:17.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.662+0000 7f3e7aa17740 1 -- 172.21.15.33:0/603681161 shutdown_connections 2024-04-02T13:44:17.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.662+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/603681161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 0x563b95e19090 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.662+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/603681161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0e0c0 0x563b95e16760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.662+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/603681161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0d7a0 0x563b95e0db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.662+0000 7f3e7aa17740 1 -- 172.21.15.33:0/603681161 >> 172.21.15.33:0/603681161 conn(0x563b95e07ed0 msgr2=0x563b95e082b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.662+0000 7f3e7aa17740 1 -- 172.21.15.33:0/603681161 shutdown_connections 2024-04-02T13:44:17.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.662+0000 7f3e7aa17740 1 -- 172.21.15.33:0/603681161 wait complete. 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e7aa17740 1 -- start start 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e7aa17740 1 Processor -- start 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e7aa17740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0d7a0 0x563b95e166e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e7aa17740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0e0c0 0x563b95e13ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e7aa17740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e16ca0 0x563b95e14220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e7aa17740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563b95d717b0 con 0x563b95e0d7a0 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e7aa17740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563b95d2ce80 con 0x563b95e16ca0 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e7aa17740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563b95e19850 con 0x563b95e0e0c0 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e7aa17740 10 client.0 _ll_register_callbacks cb 0x563b95d68aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e75bcf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0d7a0 0x563b95e166e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e75bcf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0d7a0 0x563b95e166e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59760/0 (socket says 172.21.15.33:59760) 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e75bcf640 1 -- 172.21.15.33:0/1297989595 learned_addr learned my addr 172.21.15.33:0/1297989595 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e753ce640 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0e0c0 0x563b95e13ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.663+0000 7f3e763d0640 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e16ca0 0x563b95e14220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e75bcf640 1 -- 172.21.15.33:0/1297989595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0e0c0 msgr2=0x563b95e13ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e75bcf640 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0e0c0 0x563b95e13ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e75bcf640 1 -- 172.21.15.33:0/1297989595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e16ca0 msgr2=0x563b95e14220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e75bcf640 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e16ca0 0x563b95e14220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e75bcf640 1 -- 172.21.15.33:0/1297989595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b95b4f7d0 con 0x563b95e0d7a0 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e763d0640 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e16ca0 0x563b95e14220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e75bcf640 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0d7a0 0x563b95e166e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009f50 tx=0x7f3e6c011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4671ae2588fad632 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e597fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e597fa640 1 -- 172.21.15.33:0/1297989595 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c012920 con 0x563b95e0d7a0 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e597fa640 1 -- 172.21.15.33:0/1297989595 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3e6c012ac0 con 0x563b95e0d7a0 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1297989595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e600bea10 con 0x563b95e0d7a0 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e597fa640 1 -- 172.21.15.33:0/1297989595 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3e6c025050 con 0x563b95e0d7a0 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e7aa17740 10 client.16791 fetch_fsmap learned FSMap version 31 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1297989595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563b95e25750 con 0x563b95e0d7a0 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e597fa640 1 -- 172.21.15.33:0/1297989595 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3e600bea10 con 0x563b95e0d7a0 2024-04-02T13:44:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e597fa640 1 -- 172.21.15.33:0/1297989595 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f3e6c00f040 con 0x563b95e0d7a0 2024-04-02T13:44:17.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e7aa17740 10 client.16791 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T13:44:17.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e7aa17740 10 client.16791 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:44:17.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e7aa17740 20 client.16791 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:17.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x563b95e26d00 0x563b95e29180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e7aa17740 4 client.16791 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:17.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.664+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1297989595 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b95d7ecf0 con 0x563b95e26d00 2024-04-02T13:44:17.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.665+0000 7f3e753ce640 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x563b95e26d00 0x563b95e29180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.665+0000 7f3e753ce640 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x563b95e26d00 0x563b95e29180 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef1161f9802cc5fb server_cookie=cd7c5837871949cf in_seq=0 out_seq=0 2024-04-02T13:44:17.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.665+0000 7f3e597fa640 10 client.16791 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:44:17.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e597fa640 1 -- 172.21.15.33:0/1297989595 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b95d7ecf0 con 0x563b95e26d00 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e597fa640 10 client.16791 handle_command_reply: tid=0 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e7aa17740 1 client.16791 shutdown 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1297989595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0d7a0 msgr2=0x563b95e166e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0d7a0 0x563b95e166e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009f50 tx=0x7f3e6c011870 comp rx=0 tx=0).stop 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1297989595 shutdown_connections 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x563b95e26d00 0x563b95e29180 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e16ca0 0x563b95e14220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0e0c0 0x563b95e13ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1297989595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0d7a0 0x563b95e166e0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e597fa640 0 client.16791 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1297989595 >> 172.21.15.33:0/1297989595 conn(0x563b95e07ed0 msgr2=0x563b95e09c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1297989595 shutdown_connections 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1297989595 wait complete. 2024-04-02T13:44:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.666+0000 7f3e7aa17740 20 client.16791 trim_cache size 0 max 16384 2024-04-02T13:44:17.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.679+0000 7f3e7aa17740 1 Processor -- start 2024-04-02T13:44:17.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.679+0000 7f3e7aa17740 1 -- start start 2024-04-02T13:44:17.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e7aa17740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0d7a0 0x563b95e0e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e7aa17740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0e0c0 0x563b95e133c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e7aa17740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e16ca0 0x563b95e09a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e7aa17740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701b1860 con 0x563b95e0e0c0 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e7aa17740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701af0a0 con 0x563b95e0d7a0 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e7aa17740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701aced0 con 0x563b95e16ca0 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e753ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0e0c0 0x563b95e133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e753ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0e0c0 0x563b95e133c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59774/0 (socket says 172.21.15.33:59774) 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e753ce640 1 -- 172.21.15.33:0/1341243235 learned_addr learned my addr 172.21.15.33:0/1341243235 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e75bcf640 1 --2- 172.21.15.33:0/1341243235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0d7a0 0x563b95e0e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e763d0640 1 --2- 172.21.15.33:0/1341243235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e16ca0 0x563b95e09a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e753ce640 1 -- 172.21.15.33:0/1341243235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e16ca0 msgr2=0x563b95e09a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e753ce640 1 --2- 172.21.15.33:0/1341243235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e16ca0 0x563b95e09a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e753ce640 1 -- 172.21.15.33:0/1341243235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0d7a0 msgr2=0x563b95e0e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e753ce640 1 --2- 172.21.15.33:0/1341243235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0d7a0 0x563b95e0e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e753ce640 1 -- 172.21.15.33:0/1341243235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b95e29800 con 0x563b95e0e0c0 2024-04-02T13:44:17.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e753ce640 1 --2- 172.21.15.33:0/1341243235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0e0c0 0x563b95e133c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3e6002ce60 tx=0x7f3e60042f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517a79d139c1142 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e5b7fe640 1 -- 172.21.15.33:0/1341243235 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e600bdc70 con 0x563b95e0e0c0 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e5b7fe640 1 -- 172.21.15.33:0/1341243235 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3e600bea10 con 0x563b95e0e0c0 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.680+0000 7f3e5b7fe640 1 -- 172.21.15.33:0/1341243235 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e600bde50 con 0x563b95e0e0c0 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.681+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1341243235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0e0c0 msgr2=0x563b95e133c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.681+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1341243235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0e0c0 0x563b95e133c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3e6002ce60 tx=0x7f3e60042f30 comp rx=0 tx=0).stop 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.681+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1341243235 shutdown_connections 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.681+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1341243235 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e16ca0 0x563b95e09a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.681+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1341243235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e0e0c0 0x563b95e133c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.681+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1341243235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0d7a0 0x563b95e0e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.681+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1341243235 >> 172.21.15.33:0/1341243235 conn(0x563b95e07ed0 msgr2=0x563b95e09320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.681+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1341243235 shutdown_connections 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.681+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1341243235 wait complete. 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.681+0000 7f3e7aa17740 1 -- start start 2024-04-02T13:44:17.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.681+0000 7f3e7aa17740 1 Processor -- start 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e7aa17740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0d7a0 0x563b95e18ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e7aa17740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0e0c0 0x563b95e17080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e7aa17740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 0x563b95e17770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e7aa17740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701ad560 con 0x563b95e16ca0 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e7aa17740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701ae9d0 con 0x563b95e0e0c0 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e7aa17740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701adc30 con 0x563b95e0d7a0 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e7aa17740 10 client.0 _ll_register_callbacks cb 0x7f3e6c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e763d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 0x563b95e17770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e75bcf640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0d7a0 0x563b95e18ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e763d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 0x563b95e17770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59782/0 (socket says 172.21.15.33:59782) 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e763d0640 1 -- 172.21.15.33:0/1181050630 learned_addr learned my addr 172.21.15.33:0/1181050630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e753ce640 1 --2- 172.21.15.33:0/1181050630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0e0c0 0x563b95e17080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e763d0640 1 -- 172.21.15.33:0/1181050630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0d7a0 msgr2=0x563b95e18ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e763d0640 1 --2- 172.21.15.33:0/1181050630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0d7a0 0x563b95e18ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e763d0640 1 -- 172.21.15.33:0/1181050630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0e0c0 msgr2=0x563b95e17080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e763d0640 1 --2- 172.21.15.33:0/1181050630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0e0c0 0x563b95e17080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e763d0640 1 -- 172.21.15.33:0/1181050630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b95e1d980 con 0x563b95e16ca0 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e763d0640 1 --2- 172.21.15.33:0/1181050630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 0x563b95e17770 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3e68013800 tx=0x7f3e68004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209ad846a96f3c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e597fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e597fa640 1 -- 172.21.15.33:0/1181050630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68013e40 con 0x563b95e16ca0 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e597fa640 1 -- 172.21.15.33:0/1181050630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3e6801fb80 con 0x563b95e16ca0 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.682+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1181050630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e600bea10 con 0x563b95e16ca0 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.683+0000 7f3e597fa640 1 -- 172.21.15.33:0/1181050630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3e6801fd80 con 0x563b95e16ca0 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.683+0000 7f3e7aa17740 10 client.16797 fetch_fsmap learned FSMap version 31 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.683+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1181050630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563b95e1ece0 con 0x563b95e16ca0 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.683+0000 7f3e597fa640 1 -- 172.21.15.33:0/1181050630 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3e600bea10 con 0x563b95e16ca0 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.683+0000 7f3e597fa640 1 -- 172.21.15.33:0/1181050630 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f3e6800f040 con 0x563b95e16ca0 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.683+0000 7f3e7aa17740 10 client.16797 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.683+0000 7f3e7aa17740 10 client.16797 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:44:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.683+0000 7f3e7aa17740 20 client.16797 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.683+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1181050630 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x563b95e8d2c0 0x563b95e8f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.683+0000 7f3e7aa17740 4 client.16797 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.683+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1181050630 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b95d79040 con 0x563b95e8d2c0 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.683+0000 7f3e75bcf640 1 --2- 172.21.15.33:0/1181050630 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x563b95e8d2c0 0x563b95e8f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e597fa640 10 client.16797 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e75bcf640 1 --2- 172.21.15.33:0/1181050630 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x563b95e8d2c0 0x563b95e8f6b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7755aff5582457 server_cookie=183310b7a05157aa in_seq=0 out_seq=0 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e597fa640 1 -- 172.21.15.33:0/1181050630 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b95d79040 con 0x563b95e8d2c0 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e597fa640 10 client.16797 handle_command_reply: tid=0 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e7aa17740 1 client.16797 shutdown 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1181050630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 msgr2=0x563b95e17770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1181050630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 0x563b95e17770 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3e68013800 tx=0x7f3e68004ab0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1181050630 shutdown_connections 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1181050630 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x563b95e8d2c0 0x563b95e8f6b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1181050630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b95e16ca0 0x563b95e17770 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1181050630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b95e0e0c0 0x563b95e17080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e7aa17740 1 --2- 172.21.15.33:0/1181050630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b95e0d7a0 0x563b95e18ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e597fa640 0 client.16797 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:44:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1181050630 >> 172.21.15.33:0/1181050630 conn(0x563b95e07ed0 msgr2=0x563b95e09b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1181050630 shutdown_connections 2024-04-02T13:44:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e7aa17740 1 -- 172.21.15.33:0/1181050630 wait complete. 2024-04-02T13:44:17.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.684+0000 7f3e7aa17740 20 client.16797 trim_cache size 0 max 16384 2024-04-02T13:44:17.694 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:44:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.685+0000 7f3e77e33640 1 -- 172.21.15.33:0/4122124721 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3e5c0a3d20 msgr2=0x7f3e5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.685+0000 7f3e77e33640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3e5c0a3d20 0x7f3e5c0a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e68005fd0 tx=0x7f3e68004a40 comp rx=0 tx=0).stop 2024-04-02T13:44:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.685+0000 7f3e77e33640 1 -- 172.21.15.33:0/4122124721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e70102450 msgr2=0x7f3e70198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.685+0000 7f3e77e33640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e70102450 0x7f3e70198900 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3e60000c00 tx=0x7f3e600045e0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.685+0000 7f3e77e33640 1 -- 172.21.15.33:0/4122124721 shutdown_connections 2024-04-02T13:44:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.685+0000 7f3e77e33640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3e5c0a3d20 0x7f3e5c0a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.685+0000 7f3e77e33640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3e70107aa0 0x7f3e7019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.685+0000 7f3e77e33640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e70102450 0x7f3e70198900 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.685+0000 7f3e77e33640 1 --2- 172.21.15.33:0/4122124721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3e70101a90 0x7f3e701983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.685+0000 7f3e77e33640 1 -- 172.21.15.33:0/4122124721 >> 172.21.15.33:0/4122124721 conn(0x7f3e70078f00 msgr2=0x7f3e701044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.685+0000 7f3e77e33640 1 -- 172.21.15.33:0/4122124721 shutdown_connections 2024-04-02T13:44:17.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.685+0000 7f3e77e33640 1 -- 172.21.15.33:0/4122124721 wait complete. 2024-04-02T13:44:17.698 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:44:17.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.801+0000 7f17d050e640 1 -- 172.21.15.33:0/2055126169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c81009b0 msgr2=0x7f17c8100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.801+0000 7f17d050e640 1 --2- 172.21.15.33:0/2055126169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c81009b0 0x7f17c8100e30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009930 tx=0x7f17b802bc30 comp rx=0 tx=0).stop 2024-04-02T13:44:17.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.801+0000 7f17d050e640 1 -- 172.21.15.33:0/2055126169 shutdown_connections 2024-04-02T13:44:17.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.801+0000 7f17d050e640 1 --2- 172.21.15.33:0/2055126169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17c8101370 0x7f17c810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.801+0000 7f17d050e640 1 --2- 172.21.15.33:0/2055126169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c81009b0 0x7f17c8100e30 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.801+0000 7f17d050e640 1 --2- 172.21.15.33:0/2055126169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c81069c0 0x7f17c8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.801+0000 7f17d050e640 1 -- 172.21.15.33:0/2055126169 >> 172.21.15.33:0/2055126169 conn(0x7f17c8076d80 msgr2=0x7f17c8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.801+0000 7f17d050e640 1 -- 172.21.15.33:0/2055126169 shutdown_connections 2024-04-02T13:44:17.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.801+0000 7f17d050e640 1 -- 172.21.15.33:0/2055126169 wait complete. 2024-04-02T13:44:17.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.801+0000 7f17d050e640 1 Processor -- start 2024-04-02T13:44:17.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.801+0000 7f17d050e640 1 -- start start 2024-04-02T13:44:17.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.801+0000 7f17d050e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c81009b0 0x7f17c8197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17d050e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c8101370 0x7f17c8197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17d050e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17c81069c0 0x7f17c819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17d050e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8110d70 con 0x7f17c8101370 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17d050e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8110bf0 con 0x7f17c81069c0 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17d050e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8110ef0 con 0x7f17c81009b0 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17cdaa9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c8101370 0x7f17c8197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17cdaa9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c8101370 0x7f17c8197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59794/0 (socket says 172.21.15.33:59794) 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17cdaa9640 1 -- 172.21.15.33:0/1647186034 learned_addr learned my addr 172.21.15.33:0/1647186034 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17ceaab640 1 --2- 172.21.15.33:0/1647186034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17c81069c0 0x7f17c819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17cdaa9640 1 -- 172.21.15.33:0/1647186034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c81009b0 msgr2=0x7f17c8197330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17cdaa9640 1 --2- 172.21.15.33:0/1647186034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c81009b0 0x7f17c8197330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17cdaa9640 1 -- 172.21.15.33:0/1647186034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17c81069c0 msgr2=0x7f17c819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17cdaa9640 1 --2- 172.21.15.33:0/1647186034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17c81069c0 0x7f17c819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17cdaa9640 1 -- 172.21.15.33:0/1647186034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c819c320 con 0x7f17c8101370 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17cdaa9640 1 --2- 172.21.15.33:0/1647186034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c8101370 0x7f17c8197870 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f17b8000c00 tx=0x7f17b80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17bf7fe640 1 -- 172.21.15.33:0/1647186034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8043020 con 0x7f17c8101370 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17bf7fe640 1 -- 172.21.15.33:0/1647186034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f17b802caa0 con 0x7f17c8101370 2024-04-02T13:44:17.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17d050e640 1 -- 172.21.15.33:0/1647186034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c819c5b0 con 0x7f17c8101370 2024-04-02T13:44:17.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17bf7fe640 1 -- 172.21.15.33:0/1647186034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8003bf0 con 0x7f17c8101370 2024-04-02T13:44:17.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.802+0000 7f17d050e640 1 -- 172.21.15.33:0/1647186034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c81a3ea0 con 0x7f17c8101370 2024-04-02T13:44:17.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.805+0000 7f17bf7fe640 1 -- 172.21.15.33:0/1647186034 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f17b8046020 con 0x7f17c8101370 2024-04-02T13:44:17.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.805+0000 7f17d31cd740 1 Processor -- start 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.805+0000 7f17d31cd740 1 -- start start 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17d31cd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b5f7a0 0x562fd5b5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17d31cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b600c0 0x562fd5b68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17d31cd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b68ca0 0x562fd5b6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17d31cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562fd5ac37b0 con 0x562fd5b600c0 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17d31cd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562fd5a62570 con 0x562fd5b5f7a0 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17d31cd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562fd5abf760 con 0x562fd5b68ca0 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17bf7fe640 1 --2- 172.21.15.33:0/1647186034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f17b40a3d20 0x7f17b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17ce2aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b600c0 0x562fd5b68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17ce2aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b600c0 0x562fd5b68760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59804/0 (socket says 172.21.15.33:59804) 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17ce2aa640 1 -- 172.21.15.33:0/2220014814 learned_addr learned my addr 172.21.15.33:0/2220014814 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17ceaab640 1 --2- 172.21.15.33:0/2220014814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b5f7a0 0x562fd5b5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17cdaa9640 1 --2- 172.21.15.33:0/2220014814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b68ca0 0x562fd5b6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17ceaab640 1 --2- 172.21.15.33:0/1647186034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f17b40a3d20 0x7f17b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17bf7fe640 1 -- 172.21.15.33:0/1647186034 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f17b80f2550 con 0x7f17c8101370 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17ce2aa640 1 -- 172.21.15.33:0/2220014814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b68ca0 msgr2=0x562fd5b6b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17ce2aa640 1 --2- 172.21.15.33:0/2220014814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b68ca0 0x562fd5b6b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17ce2aa640 1 -- 172.21.15.33:0/2220014814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b5f7a0 msgr2=0x562fd5b5fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17ce2aa640 1 --2- 172.21.15.33:0/2220014814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b5f7a0 0x562fd5b5fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17ce2aa640 1 -- 172.21.15.33:0/2220014814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fd57d73f0 con 0x562fd5b600c0 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17ceaab640 1 --2- 172.21.15.33:0/1647186034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f17b40a3d20 0x7f17b40a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17c0005fd0 tx=0x7f17c0009970 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.806+0000 7f17ce2aa640 1 --2- 172.21.15.33:0/2220014814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b600c0 0x562fd5b68760 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f17c400ba50 tx=0x7f17c400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9a24c7656179a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.807+0000 7f17bcbfe640 1 -- 172.21.15.33:0/2220014814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4016070 con 0x562fd5b600c0 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.807+0000 7f17bcbfe640 1 -- 172.21.15.33:0/2220014814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f17c400de30 con 0x562fd5b600c0 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.807+0000 7f17bcbfe640 1 -- 172.21.15.33:0/2220014814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4003ce0 con 0x562fd5b600c0 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.807+0000 7f17d31cd740 1 -- 172.21.15.33:0/2220014814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b600c0 msgr2=0x562fd5b68760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.807+0000 7f17d31cd740 1 --2- 172.21.15.33:0/2220014814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b600c0 0x562fd5b68760 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f17c400ba50 tx=0x7f17c400be50 comp rx=0 tx=0).stop 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.807+0000 7f17d31cd740 1 -- 172.21.15.33:0/2220014814 shutdown_connections 2024-04-02T13:44:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.807+0000 7f17d31cd740 1 --2- 172.21.15.33:0/2220014814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b68ca0 0x562fd5b6b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.807+0000 7f17d31cd740 1 --2- 172.21.15.33:0/2220014814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b600c0 0x562fd5b68760 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.807+0000 7f17d31cd740 1 --2- 172.21.15.33:0/2220014814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b5f7a0 0x562fd5b5fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.807+0000 7f17d31cd740 1 -- 172.21.15.33:0/2220014814 >> 172.21.15.33:0/2220014814 conn(0x562fd5b59ed0 msgr2=0x562fd5b5a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.807+0000 7f17d31cd740 1 -- 172.21.15.33:0/2220014814 shutdown_connections 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.807+0000 7f17d31cd740 1 -- 172.21.15.33:0/2220014814 wait complete. 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17d31cd740 1 -- start start 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17d31cd740 1 Processor -- start 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17d31cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 0x562fd5b68730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17d31cd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b600c0 0x562fd5b65ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17d31cd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b68ca0 0x562fd5b661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17d31cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562fd5b6ba90 con 0x562fd5b5f7a0 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17d31cd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562fd5a7ee80 con 0x562fd5b68ca0 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17d31cd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562fd5b6bc10 con 0x562fd5b600c0 2024-04-02T13:44:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17d31cd740 10 client.0 _ll_register_callbacks cb 0x562fd5abaaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17ce2aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 0x562fd5b68730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17ce2aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 0x562fd5b68730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59816/0 (socket says 172.21.15.33:59816) 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17ce2aa640 1 -- 172.21.15.33:0/1643628558 learned_addr learned my addr 172.21.15.33:0/1643628558 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17cdaa9640 1 --2- 172.21.15.33:0/1643628558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b600c0 0x562fd5b65ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17ceaab640 1 --2- 172.21.15.33:0/1643628558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b68ca0 0x562fd5b661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17ce2aa640 1 -- 172.21.15.33:0/1643628558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b600c0 msgr2=0x562fd5b65ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17ce2aa640 1 --2- 172.21.15.33:0/1643628558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b600c0 0x562fd5b65ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17ce2aa640 1 -- 172.21.15.33:0/1643628558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b68ca0 msgr2=0x562fd5b661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17ce2aa640 1 --2- 172.21.15.33:0/1643628558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b68ca0 0x562fd5b661b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.808+0000 7f17ce2aa640 1 -- 172.21.15.33:0/1643628558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fd58a17d0 con 0x562fd5b5f7a0 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17ce2aa640 1 --2- 172.21.15.33:0/1643628558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 0x562fd5b68730 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f17c4000c00 tx=0x7f17c400d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b7707951977528 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17b1ffb640 1 -- 172.21.15.33:0/1643628558 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4016070 con 0x562fd5b5f7a0 2024-04-02T13:44:17.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17b1ffb640 1 -- 172.21.15.33:0/1643628558 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f17c4005240 con 0x562fd5b5f7a0 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17d31cd740 1 -- 172.21.15.33:0/1643628558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c400de30 con 0x562fd5b5f7a0 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17b1ffb640 1 -- 172.21.15.33:0/1643628558 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f17c402f050 con 0x562fd5b5f7a0 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17d31cd740 10 client.16809 fetch_fsmap learned FSMap version 31 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17d31cd740 1 -- 172.21.15.33:0/1643628558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562fd5b77ba0 con 0x562fd5b5f7a0 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17b1ffb640 1 -- 172.21.15.33:0/1643628558 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f17c400de30 con 0x562fd5b5f7a0 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17b1ffb640 1 -- 172.21.15.33:0/1643628558 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f17c4024020 con 0x562fd5b5f7a0 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17d31cd740 10 client.16809 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17d31cd740 10 client.16809 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17d31cd740 20 client.16809 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17d31cd740 1 --2- 172.21.15.33:0/1643628558 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x562fd5b79000 0x562fd5b7b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17d31cd740 4 client.16809 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.809+0000 7f17d31cd740 1 -- 172.21.15.33:0/1643628558 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fd5ad0cf0 con 0x562fd5b79000 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.810+0000 7f17cdaa9640 1 --2- 172.21.15.33:0/1643628558 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x562fd5b79000 0x562fd5b7b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.810+0000 7f17b1ffb640 10 client.16809 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:44:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.810+0000 7f17cdaa9640 1 --2- 172.21.15.33:0/1643628558 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x562fd5b79000 0x562fd5b7b3f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a86e06867d277056 server_cookie=78374753097bda8c in_seq=0 out_seq=0 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17b1ffb640 1 -- 172.21.15.33:0/1643628558 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562fd5ad0cf0 con 0x562fd5b79000 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17b1ffb640 10 client.16809 handle_command_reply: tid=0 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17d31cd740 1 client.16809 shutdown 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17d31cd740 1 -- 172.21.15.33:0/1643628558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 msgr2=0x562fd5b68730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17d31cd740 1 --2- 172.21.15.33:0/1643628558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 0x562fd5b68730 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f17c4000c00 tx=0x7f17c400d960 comp rx=0 tx=0).stop 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17d31cd740 1 -- 172.21.15.33:0/1643628558 shutdown_connections 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17d31cd740 1 --2- 172.21.15.33:0/1643628558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b68ca0 0x562fd5b661b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17d31cd740 1 --2- 172.21.15.33:0/1643628558 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x562fd5b79000 0x562fd5b7b3f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17d31cd740 1 --2- 172.21.15.33:0/1643628558 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b600c0 0x562fd5b65ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17d31cd740 1 --2- 172.21.15.33:0/1643628558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 0x562fd5b68730 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17b1ffb640 0 client.16809 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17d31cd740 1 -- 172.21.15.33:0/1643628558 >> 172.21.15.33:0/1643628558 conn(0x562fd5b59ed0 msgr2=0x562fd5b5bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17d31cd740 1 -- 172.21.15.33:0/1643628558 shutdown_connections 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17d31cd740 1 -- 172.21.15.33:0/1643628558 wait complete. 2024-04-02T13:44:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.811+0000 7f17d31cd740 20 client.16809 trim_cache size 0 max 16384 2024-04-02T13:44:17.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.824+0000 7f17d31cd740 1 Processor -- start 2024-04-02T13:44:17.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.824+0000 7f17d31cd740 1 -- start start 2024-04-02T13:44:17.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17d31cd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b5f7a0 0x562fd5b74980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17d31cd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b600c0 0x562fd5b64aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17d31cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b68ca0 0x562fd5b681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17d31cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81af6a0 con 0x562fd5b68ca0 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17d31cd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81acee0 con 0x562fd5b600c0 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17d31cd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81aad10 con 0x562fd5b5f7a0 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17ceaab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b68ca0 0x562fd5b681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17ceaab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b68ca0 0x562fd5b681f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59818/0 (socket says 172.21.15.33:59818) 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17ceaab640 1 -- 172.21.15.33:0/1198453397 learned_addr learned my addr 172.21.15.33:0/1198453397 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17cdaa9640 1 --2- 172.21.15.33:0/1198453397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b600c0 0x562fd5b64aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17ceaab640 1 -- 172.21.15.33:0/1198453397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b5f7a0 msgr2=0x562fd5b74980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17ce2aa640 1 --2- 172.21.15.33:0/1198453397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b5f7a0 0x562fd5b74980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17ceaab640 1 --2- 172.21.15.33:0/1198453397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b5f7a0 0x562fd5b74980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17ceaab640 1 -- 172.21.15.33:0/1198453397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b600c0 msgr2=0x562fd5b64aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17ceaab640 1 --2- 172.21.15.33:0/1198453397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b600c0 0x562fd5b64aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17ceaab640 1 -- 172.21.15.33:0/1198453397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fd5b7b980 con 0x562fd5b68ca0 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17cdaa9640 1 --2- 172.21.15.33:0/1198453397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b600c0 0x562fd5b64aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17ceaab640 1 --2- 172.21.15.33:0/1198453397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b68ca0 0x562fd5b681f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f17c0012640 tx=0x7f17c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd6d1344328c319 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17b3fff640 1 -- 172.21.15.33:0/1198453397 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0004b20 con 0x562fd5b68ca0 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17b3fff640 1 -- 172.21.15.33:0/1198453397 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f17c0004cc0 con 0x562fd5b68ca0 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.825+0000 7f17b3fff640 1 -- 172.21.15.33:0/1198453397 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c00053a0 con 0x562fd5b68ca0 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 -- 172.21.15.33:0/1198453397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b68ca0 msgr2=0x562fd5b681f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 --2- 172.21.15.33:0/1198453397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b68ca0 0x562fd5b681f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f17c0012640 tx=0x7f17c0004ab0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 -- 172.21.15.33:0/1198453397 shutdown_connections 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 --2- 172.21.15.33:0/1198453397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b68ca0 0x562fd5b681f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 --2- 172.21.15.33:0/1198453397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b600c0 0x562fd5b64aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 --2- 172.21.15.33:0/1198453397 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b5f7a0 0x562fd5b74980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 -- 172.21.15.33:0/1198453397 >> 172.21.15.33:0/1198453397 conn(0x562fd5b59ed0 msgr2=0x562fd5b64390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 -- 172.21.15.33:0/1198453397 shutdown_connections 2024-04-02T13:44:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 -- 172.21.15.33:0/1198453397 wait complete. 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 -- start start 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 Processor -- start 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 0x562fd5b70ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b600c0 0x562fd5b713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b68ca0 0x562fd5b6c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81ab3a0 con 0x562fd5b5f7a0 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81ac810 con 0x562fd5b600c0 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81aba70 con 0x562fd5b68ca0 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17d31cd740 10 client.0 _ll_register_callbacks cb 0x7f17c0012730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.826+0000 7f17ce2aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 0x562fd5b70ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17ce2aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 0x562fd5b70ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59824/0 (socket says 172.21.15.33:59824) 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17ce2aa640 1 -- 172.21.15.33:0/3478631123 learned_addr learned my addr 172.21.15.33:0/3478631123 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17ceaab640 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b68ca0 0x562fd5b6c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17cdaa9640 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b600c0 0x562fd5b713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17ce2aa640 1 -- 172.21.15.33:0/3478631123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b68ca0 msgr2=0x562fd5b6c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17ce2aa640 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b68ca0 0x562fd5b6c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17ce2aa640 1 -- 172.21.15.33:0/3478631123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b600c0 msgr2=0x562fd5b713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17ce2aa640 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b600c0 0x562fd5b713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17ce2aa640 1 -- 172.21.15.33:0/3478631123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fd5b6cd90 con 0x562fd5b5f7a0 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17cdaa9640 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b600c0 0x562fd5b713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17ce2aa640 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 0x562fd5b70ea0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f17c400b9a0 tx=0x7f17c4005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463a59cea8d3efd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17b1ffb640 1 -- 172.21.15.33:0/3478631123 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4016040 con 0x562fd5b5f7a0 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17b1ffb640 1 -- 172.21.15.33:0/3478631123 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f17c401c610 con 0x562fd5b5f7a0 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17d31cd740 1 -- 172.21.15.33:0/3478631123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c0004cc0 con 0x562fd5b5f7a0 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17b1ffb640 1 -- 172.21.15.33:0/3478631123 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f17c403b050 con 0x562fd5b5f7a0 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17d31cd740 10 client.16815 fetch_fsmap learned FSMap version 31 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17d31cd740 1 -- 172.21.15.33:0/3478631123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562fd5bdf0d0 con 0x562fd5b5f7a0 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.827+0000 7f17b1ffb640 1 -- 172.21.15.33:0/3478631123 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f17c0004cc0 con 0x562fd5b5f7a0 2024-04-02T13:44:17.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.828+0000 7f17b1ffb640 1 -- 172.21.15.33:0/3478631123 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f17c4024030 con 0x562fd5b5f7a0 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.828+0000 7f17d31cd740 10 client.16815 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.828+0000 7f17d31cd740 10 client.16815 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.828+0000 7f17d31cd740 20 client.16815 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.828+0000 7f17d31cd740 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x562fd5bdf880 0x562fd5be1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.828+0000 7f17d31cd740 4 client.16815 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.828+0000 7f17d31cd740 1 -- 172.21.15.33:0/3478631123 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562fd5acb040 con 0x562fd5bdf880 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.828+0000 7f17cdaa9640 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x562fd5bdf880 0x562fd5be1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.828+0000 7f17cdaa9640 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x562fd5bdf880 0x562fd5be1ca0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cdee4f0779571264 server_cookie=6d79e750069c85aa in_seq=0 out_seq=0 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.828+0000 7f17b1ffb640 10 client.16815 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17b1ffb640 1 -- 172.21.15.33:0/3478631123 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562fd5acb040 con 0x562fd5bdf880 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17b1ffb640 10 client.16815 handle_command_reply: tid=0 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d31cd740 1 client.16815 shutdown 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d31cd740 1 -- 172.21.15.33:0/3478631123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 msgr2=0x562fd5b70ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d31cd740 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 0x562fd5b70ea0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f17c400b9a0 tx=0x7f17c4005760 comp rx=0 tx=0).stop 2024-04-02T13:44:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d31cd740 1 -- 172.21.15.33:0/3478631123 shutdown_connections 2024-04-02T13:44:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d31cd740 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x562fd5bdf880 0x562fd5be1ca0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d31cd740 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562fd5b68ca0 0x562fd5b6c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17b1ffb640 0 client.16815 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:44:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d31cd740 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fd5b600c0 0x562fd5b713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d31cd740 1 --2- 172.21.15.33:0/3478631123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562fd5b5f7a0 0x562fd5b70ea0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d31cd740 1 -- 172.21.15.33:0/3478631123 >> 172.21.15.33:0/3478631123 conn(0x562fd5b59ed0 msgr2=0x562fd5b58f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d31cd740 1 -- 172.21.15.33:0/3478631123 shutdown_connections 2024-04-02T13:44:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d31cd740 1 -- 172.21.15.33:0/3478631123 wait complete. 2024-04-02T13:44:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d31cd740 20 client.16815 trim_cache size 0 max 16384 2024-04-02T13:44:17.839 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:44:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d050e640 1 -- 172.21.15.33:0/1647186034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f17b40a3d20 msgr2=0x7f17b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d050e640 1 --2- 172.21.15.33:0/1647186034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f17b40a3d20 0x7f17b40a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17c0005fd0 tx=0x7f17c0009970 comp rx=0 tx=0).stop 2024-04-02T13:44:17.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d050e640 1 -- 172.21.15.33:0/1647186034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c8101370 msgr2=0x7f17c8197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d050e640 1 --2- 172.21.15.33:0/1647186034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c8101370 0x7f17c8197870 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f17b8000c00 tx=0x7f17b80045e0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d050e640 1 -- 172.21.15.33:0/1647186034 shutdown_connections 2024-04-02T13:44:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d050e640 1 --2- 172.21.15.33:0/1647186034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f17b40a3d20 0x7f17b40a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d050e640 1 --2- 172.21.15.33:0/1647186034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17c81069c0 0x7f17c819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d050e640 1 --2- 172.21.15.33:0/1647186034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17c8101370 0x7f17c8197870 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d050e640 1 --2- 172.21.15.33:0/1647186034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17c81009b0 0x7f17c8197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d050e640 1 -- 172.21.15.33:0/1647186034 >> 172.21.15.33:0/1647186034 conn(0x7f17c8076d80 msgr2=0x7f17c80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d050e640 1 -- 172.21.15.33:0/1647186034 shutdown_connections 2024-04-02T13:44:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.829+0000 7f17d050e640 1 -- 172.21.15.33:0/1647186034 wait complete. 2024-04-02T13:44:17.842 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:44:17.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.947+0000 7f6832915640 1 -- 172.21.15.33:0/60967232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682c106980 msgr2=0x7f682c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.947+0000 7f6832915640 1 --2- 172.21.15.33:0/60967232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682c106980 0x7f682c106d60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f681c009930 tx=0x7f681c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:44:17.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.947+0000 7f6832915640 1 -- 172.21.15.33:0/60967232 shutdown_connections 2024-04-02T13:44:17.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.947+0000 7f6832915640 1 --2- 172.21.15.33:0/60967232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f682c101330 0x7f682c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.947+0000 7f6832915640 1 --2- 172.21.15.33:0/60967232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f682c100970 0x7f682c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.947+0000 7f6832915640 1 --2- 172.21.15.33:0/60967232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682c106980 0x7f682c106d60 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.947+0000 7f6832915640 1 -- 172.21.15.33:0/60967232 >> 172.21.15.33:0/60967232 conn(0x7f682c077b00 msgr2=0x7f682c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.947+0000 7f6832915640 1 -- 172.21.15.33:0/60967232 shutdown_connections 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.947+0000 7f6832915640 1 -- 172.21.15.33:0/60967232 wait complete. 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.947+0000 7f6832915640 1 Processor -- start 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.947+0000 7f6832915640 1 -- start start 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f6832915640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682c100970 0x7f682c102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f6832915640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f682c101330 0x7f682c103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f6832915640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f682c106980 0x7f682c1039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f6832915640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f682c110d60 con 0x7f682c100970 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f6832915640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f682c110be0 con 0x7f682c101330 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f6832915640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f682c110ee0 con 0x7f682c106980 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f682bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682c100970 0x7f682c102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f682bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682c100970 0x7f682c102f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59852/0 (socket says 172.21.15.33:59852) 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f682bfff640 1 -- 172.21.15.33:0/3103258640 learned_addr learned my addr 172.21.15.33:0/3103258640 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f6830eb2640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f682c106980 0x7f682c1039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f682b7fe640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f682c101330 0x7f682c103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f682bfff640 1 -- 172.21.15.33:0/3103258640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f682c106980 msgr2=0x7f682c1039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f682bfff640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f682c106980 0x7f682c1039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f682bfff640 1 -- 172.21.15.33:0/3103258640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f682c101330 msgr2=0x7f682c103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f682bfff640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f682c101330 0x7f682c103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f682bfff640 1 -- 172.21.15.33:0/3103258640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682c1aad90 con 0x7f682c100970 2024-04-02T13:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f682b7fe640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f682c101330 0x7f682c103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f682bfff640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682c100970 0x7f682c102f40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f681c000c00 tx=0x7f681c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f68297fa640 1 -- 172.21.15.33:0/3103258640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c043020 con 0x7f682c100970 2024-04-02T13:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f68297fa640 1 -- 172.21.15.33:0/3103258640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f681c02c5d0 con 0x7f682c100970 2024-04-02T13:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f6832915640 1 -- 172.21.15.33:0/3103258640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682c1aaf60 con 0x7f682c100970 2024-04-02T13:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.948+0000 7f68297fa640 1 -- 172.21.15.33:0/3103258640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c003970 con 0x7f682c100970 2024-04-02T13:44:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.949+0000 7f6832915640 1 -- 172.21.15.33:0/3103258640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f682c1ab450 con 0x7f682c100970 2024-04-02T13:44:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.951+0000 7f68297fa640 1 -- 172.21.15.33:0/3103258640 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f681c046020 con 0x7f682c100970 2024-04-02T13:44:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.951+0000 7f68355d8740 1 Processor -- start 2024-04-02T13:44:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.951+0000 7f68355d8740 1 -- start start 2024-04-02T13:44:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.951+0000 7f68355d8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 0x55c20910ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.951+0000 7f68297fa640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f68140a3d20 0x7f68140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.951+0000 7f68355d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910b0c0 0x55c209113760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.951+0000 7f68355d8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c209113ca0 0x55c209116090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.951+0000 7f68355d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c20906e7b0 con 0x55c20910b0c0 2024-04-02T13:44:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.951+0000 7f68355d8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c20900d570 con 0x55c20910a7a0 2024-04-02T13:44:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.951+0000 7f68355d8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c20906a760 con 0x55c209113ca0 2024-04-02T13:44:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.951+0000 7f682b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910b0c0 0x55c209113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f682b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910b0c0 0x55c209113760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59868/0 (socket says 172.21.15.33:59868) 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f682b7fe640 1 -- 172.21.15.33:0/700354274 learned_addr learned my addr 172.21.15.33:0/700354274 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f68297fa640 1 -- 172.21.15.33:0/3103258640 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f681c0f2430 con 0x7f682c100970 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f682bfff640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f68140a3d20 0x7f68140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f6830eb2640 1 --2- 172.21.15.33:0/700354274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 0x55c20910ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f6830eb2640 1 --2- 172.21.15.33:0/700354274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c209113ca0 0x55c209116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f682b7fe640 1 -- 172.21.15.33:0/700354274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c209113ca0 msgr2=0x55c209116090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f682b7fe640 1 --2- 172.21.15.33:0/700354274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c209113ca0 0x55c209116090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f682b7fe640 1 -- 172.21.15.33:0/700354274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 msgr2=0x55c20910ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f682b7fe640 1 --2- 172.21.15.33:0/700354274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 0x55c20910ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f682b7fe640 1 -- 172.21.15.33:0/700354274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c208d823f0 con 0x55c20910b0c0 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f6830eb2640 1 --2- 172.21.15.33:0/700354274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 0x55c20910ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f682bfff640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f68140a3d20 0x7f68140a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f681c02ce60 tx=0x7f681c009260 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f682b7fe640 1 --2- 172.21.15.33:0/700354274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910b0c0 0x55c209113760 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6820009c70 tx=0x7f68200117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577989ca7aa5d45c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f68127fc640 1 -- 172.21.15.33:0/700354274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820004030 con 0x55c20910b0c0 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f68127fc640 1 -- 172.21.15.33:0/700354274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f68200041d0 con 0x55c20910b0c0 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.952+0000 7f68127fc640 1 -- 172.21.15.33:0/700354274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820012780 con 0x55c20910b0c0 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.953+0000 7f68355d8740 1 -- 172.21.15.33:0/700354274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910b0c0 msgr2=0x55c209113760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.953+0000 7f68355d8740 1 --2- 172.21.15.33:0/700354274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910b0c0 0x55c209113760 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6820009c70 tx=0x7f68200117b0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.953+0000 7f68355d8740 1 -- 172.21.15.33:0/700354274 shutdown_connections 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.953+0000 7f68355d8740 1 --2- 172.21.15.33:0/700354274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c209113ca0 0x55c209116090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.953+0000 7f68355d8740 1 --2- 172.21.15.33:0/700354274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910b0c0 0x55c209113760 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.953+0000 7f68355d8740 1 --2- 172.21.15.33:0/700354274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 0x55c20910ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.953+0000 7f68355d8740 1 -- 172.21.15.33:0/700354274 >> 172.21.15.33:0/700354274 conn(0x55c209104ed0 msgr2=0x55c2091052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.953+0000 7f68355d8740 1 -- 172.21.15.33:0/700354274 shutdown_connections 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.953+0000 7f68355d8740 1 -- 172.21.15.33:0/700354274 wait complete. 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.953+0000 7f68355d8740 1 -- start start 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.953+0000 7f68355d8740 1 Processor -- start 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f68355d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910a7a0 0x55c209113740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f68355d8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910b0c0 0x55c209110ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f68355d8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c209113ca0 0x55c2091111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f68355d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c209116a90 con 0x55c20910a7a0 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f68355d8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c209029e80 con 0x55c20910b0c0 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f68355d8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c209116c10 con 0x55c209113ca0 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f68355d8740 10 client.0 _ll_register_callbacks cb 0x55c209065aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f6830eb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910a7a0 0x55c209113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f6830eb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910a7a0 0x55c209113740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59878/0 (socket says 172.21.15.33:59878) 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f6830eb2640 1 -- 172.21.15.33:0/168388428 learned_addr learned my addr 172.21.15.33:0/168388428 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f682b7fe640 1 --2- 172.21.15.33:0/168388428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910b0c0 0x55c209110ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f6830eb2640 1 -- 172.21.15.33:0/168388428 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c209113ca0 msgr2=0x55c2091111c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f6830eb2640 1 --2- 172.21.15.33:0/168388428 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c209113ca0 0x55c2091111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f6830eb2640 1 -- 172.21.15.33:0/168388428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910b0c0 msgr2=0x55c209110ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f6830eb2640 1 --2- 172.21.15.33:0/168388428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910b0c0 0x55c209110ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f6830eb2640 1 -- 172.21.15.33:0/168388428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c208e4c7d0 con 0x55c20910a7a0 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f6830eb2640 1 --2- 172.21.15.33:0/168388428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910a7a0 0x55c209113740 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6824012c80 tx=0x7f682400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f9bffe93731e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f680b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f680b7fe640 1 -- 172.21.15.33:0/168388428 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824003970 con 0x55c20910a7a0 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f680b7fe640 1 -- 172.21.15.33:0/168388428 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6824003b10 con 0x55c20910a7a0 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.954+0000 7f68355d8740 1 -- 172.21.15.33:0/168388428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68200041d0 con 0x55c20910a7a0 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.955+0000 7f680b7fe640 1 -- 172.21.15.33:0/168388428 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6824027050 con 0x55c20910a7a0 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.955+0000 7f68355d8740 10 client.16827 fetch_fsmap learned FSMap version 31 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.955+0000 7f68355d8740 1 -- 172.21.15.33:0/168388428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c209122b70 con 0x55c20910a7a0 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.955+0000 7f680b7fe640 1 -- 172.21.15.33:0/168388428 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f68200041d0 con 0x55c20910a7a0 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.955+0000 7f680b7fe640 1 -- 172.21.15.33:0/168388428 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f682401c430 con 0x55c20910a7a0 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.955+0000 7f68355d8740 10 client.16827 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.955+0000 7f68355d8740 10 client.16827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.955+0000 7f68355d8740 20 client.16827 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.955+0000 7f68355d8740 1 --2- 172.21.15.33:0/168388428 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c209124080 0x55c2091264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.955+0000 7f68355d8740 4 client.16827 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.955+0000 7f68355d8740 1 -- 172.21.15.33:0/168388428 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c20907bcf0 con 0x55c209124080 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.955+0000 7f682b7fe640 1 --2- 172.21.15.33:0/168388428 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c209124080 0x55c2091264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.956+0000 7f682b7fe640 1 --2- 172.21.15.33:0/168388428 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c209124080 0x55c2091264a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1e3618ee6a9760f server_cookie=910fda370b41993d in_seq=0 out_seq=0 2024-04-02T13:44:17.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.956+0000 7f680b7fe640 10 client.16827 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.956+0000 7f680b7fe640 1 -- 172.21.15.33:0/168388428 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c20907bcf0 con 0x55c209124080 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.956+0000 7f680b7fe640 10 client.16827 handle_command_reply: tid=0 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.957+0000 7f68355d8740 1 client.16827 shutdown 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.957+0000 7f68355d8740 1 -- 172.21.15.33:0/168388428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910a7a0 msgr2=0x55c209113740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.957+0000 7f68355d8740 1 --2- 172.21.15.33:0/168388428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910a7a0 0x55c209113740 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6824012c80 tx=0x7f682400f790 comp rx=0 tx=0).stop 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.957+0000 7f68355d8740 1 -- 172.21.15.33:0/168388428 shutdown_connections 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.957+0000 7f68355d8740 1 --2- 172.21.15.33:0/168388428 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c209124080 0x55c2091264a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.957+0000 7f68355d8740 1 --2- 172.21.15.33:0/168388428 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c209113ca0 0x55c2091111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.957+0000 7f68355d8740 1 --2- 172.21.15.33:0/168388428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910b0c0 0x55c209110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.957+0000 7f68355d8740 1 --2- 172.21.15.33:0/168388428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c20910a7a0 0x55c209113740 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.957+0000 7f680b7fe640 0 client.16827 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.957+0000 7f68355d8740 1 -- 172.21.15.33:0/168388428 >> 172.21.15.33:0/168388428 conn(0x55c209104ed0 msgr2=0x55c209106c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.957+0000 7f68355d8740 1 -- 172.21.15.33:0/168388428 shutdown_connections 2024-04-02T13:44:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.957+0000 7f68355d8740 1 -- 172.21.15.33:0/168388428 wait complete. 2024-04-02T13:44:17.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.957+0000 7f68355d8740 20 client.16827 trim_cache size 0 max 16384 2024-04-02T13:44:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.970+0000 7f68355d8740 1 Processor -- start 2024-04-02T13:44:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.970+0000 7f68355d8740 1 -- start start 2024-04-02T13:44:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.970+0000 7f68355d8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 0x55c20911f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.970+0000 7f68355d8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c20910b0c0 0x55c209113200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.970+0000 7f68355d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 0x55c209113740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.970+0000 7f68355d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1b7da0 con 0x55c209113ca0 2024-04-02T13:44:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.970+0000 7f68355d8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1b55e0 con 0x55c20910a7a0 2024-04-02T13:44:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.970+0000 7f68355d8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1b3410 con 0x55c20910b0c0 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.970+0000 7f6830eb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 0x55c209113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.970+0000 7f6830eb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 0x55c209113740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59884/0 (socket says 172.21.15.33:59884) 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.971+0000 7f6830eb2640 1 -- 172.21.15.33:0/1893317175 learned_addr learned my addr 172.21.15.33:0/1893317175 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.971+0000 7f6830eb2640 1 --2- 172.21.15.33:0/1893317175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 0x55c20911f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.971+0000 7f6830eb2640 1 -- 172.21.15.33:0/1893317175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c20910b0c0 msgr2=0x55c209113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.971+0000 7f6830eb2640 1 --2- 172.21.15.33:0/1893317175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c20910b0c0 0x55c209113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.971+0000 7f6830eb2640 1 -- 172.21.15.33:0/1893317175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 msgr2=0x55c20911f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.971+0000 7f6830eb2640 1 --2- 172.21.15.33:0/1893317175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 0x55c20911f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.971+0000 7f6830eb2640 1 -- 172.21.15.33:0/1893317175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c20910aef0 con 0x55c209113ca0 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.971+0000 7f6830eb2640 1 --2- 172.21.15.33:0/1893317175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 0x55c209113740 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6824002230 tx=0x7f6824014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7088e46a94af2ff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.971+0000 7f6811ffb640 1 -- 172.21.15.33:0/1893317175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682401c430 con 0x55c209113ca0 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.971+0000 7f6811ffb640 1 -- 172.21.15.33:0/1893317175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f682401c5d0 con 0x55c209113ca0 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.971+0000 7f6811ffb640 1 -- 172.21.15.33:0/1893317175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824003af0 con 0x55c209113ca0 2024-04-02T13:44:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 -- 172.21.15.33:0/1893317175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 msgr2=0x55c209113740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 --2- 172.21.15.33:0/1893317175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 0x55c209113740 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6824002230 tx=0x7f6824014040 comp rx=0 tx=0).stop 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 -- 172.21.15.33:0/1893317175 shutdown_connections 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 --2- 172.21.15.33:0/1893317175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 0x55c209113740 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 --2- 172.21.15.33:0/1893317175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c20910b0c0 0x55c209113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 --2- 172.21.15.33:0/1893317175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 0x55c20911f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 -- 172.21.15.33:0/1893317175 >> 172.21.15.33:0/1893317175 conn(0x55c209104ed0 msgr2=0x55c2091052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 -- 172.21.15.33:0/1893317175 shutdown_connections 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 -- 172.21.15.33:0/1893317175 wait complete. 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 -- start start 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 Processor -- start 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 0x55c20911c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c20910b0c0 0x55c209114830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 0x55c209114d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1b3aa0 con 0x55c209113ca0 2024-04-02T13:44:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1b4f10 con 0x55c20910a7a0 2024-04-02T13:44:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.972+0000 7f68355d8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f682c1b4170 con 0x55c20910b0c0 2024-04-02T13:44:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f68355d8740 10 client.0 _ll_register_callbacks cb 0x7f6824009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f6830eb2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 0x55c20911c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f6830eb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 0x55c209114d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f6830eb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 0x55c209114d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59888/0 (socket says 172.21.15.33:59888) 2024-04-02T13:44:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f6830eb2640 1 -- 172.21.15.33:0/2448972915 learned_addr learned my addr 172.21.15.33:0/2448972915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f682b7fe640 1 --2- 172.21.15.33:0/2448972915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c20910b0c0 0x55c209114830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f6830eb2640 1 -- 172.21.15.33:0/2448972915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c20910b0c0 msgr2=0x55c209114830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f6830eb2640 1 --2- 172.21.15.33:0/2448972915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c20910b0c0 0x55c209114830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f6830eb2640 1 -- 172.21.15.33:0/2448972915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 msgr2=0x55c20911c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f6830eb2640 1 --2- 172.21.15.33:0/2448972915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 0x55c20911c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f6830eb2640 1 -- 172.21.15.33:0/2448972915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2091155a0 con 0x55c209113ca0 2024-04-02T13:44:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f6830eb2640 1 --2- 172.21.15.33:0/2448972915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 0x55c209114d70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6824005d30 tx=0x7f6824009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370066169a5bc1b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f680b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f680b7fe640 1 -- 172.21.15.33:0/2448972915 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682401ca70 con 0x55c209113ca0 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f68355d8740 1 -- 172.21.15.33:0/2448972915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f682401c5d0 con 0x55c209113ca0 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f680b7fe640 1 -- 172.21.15.33:0/2448972915 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f682401cc10 con 0x55c209113ca0 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f680b7fe640 1 -- 172.21.15.33:0/2448972915 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6824005080 con 0x55c209113ca0 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f68355d8740 10 client.16833 fetch_fsmap learned FSMap version 31 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.973+0000 7f68355d8740 1 -- 172.21.15.33:0/2448972915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c20918a620 con 0x55c209113ca0 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.974+0000 7f680b7fe640 1 -- 172.21.15.33:0/2448972915 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f682401c5d0 con 0x55c209113ca0 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.974+0000 7f680b7fe640 1 -- 172.21.15.33:0/2448972915 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f6824038040 con 0x55c209113ca0 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.974+0000 7f68355d8740 10 client.16833 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.974+0000 7f68355d8740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.974+0000 7f68355d8740 20 client.16833 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.974+0000 7f68355d8740 1 --2- 172.21.15.33:0/2448972915 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c20918ac90 0x55c20918d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.974+0000 7f68355d8740 4 client.16833 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.974+0000 7f68355d8740 1 -- 172.21.15.33:0/2448972915 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c209076040 con 0x55c20918ac90 2024-04-02T13:44:17.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.974+0000 7f682b7fe640 1 --2- 172.21.15.33:0/2448972915 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c20918ac90 0x55c20918d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f680b7fe640 10 client.16833 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f682b7fe640 1 --2- 172.21.15.33:0/2448972915 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c20918ac90 0x55c20918d0b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c23e1218e9371c50 server_cookie=7a7cd5ccc97f7a25 in_seq=0 out_seq=0 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f680b7fe640 1 -- 172.21.15.33:0/2448972915 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c209076040 con 0x55c20918ac90 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f680b7fe640 10 client.16833 handle_command_reply: tid=0 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f68355d8740 1 client.16833 shutdown 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f68355d8740 1 -- 172.21.15.33:0/2448972915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 msgr2=0x55c209114d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f68355d8740 1 --2- 172.21.15.33:0/2448972915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 0x55c209114d70 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6824005d30 tx=0x7f6824009f90 comp rx=0 tx=0).stop 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f68355d8740 1 -- 172.21.15.33:0/2448972915 shutdown_connections 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f68355d8740 1 --2- 172.21.15.33:0/2448972915 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c20918ac90 0x55c20918d0b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f68355d8740 1 --2- 172.21.15.33:0/2448972915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c209113ca0 0x55c209114d70 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f68355d8740 1 --2- 172.21.15.33:0/2448972915 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c20910b0c0 0x55c209114830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f680b7fe640 0 client.16833 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f68355d8740 1 --2- 172.21.15.33:0/2448972915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c20910a7a0 0x55c20911c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f68355d8740 1 -- 172.21.15.33:0/2448972915 >> 172.21.15.33:0/2448972915 conn(0x55c209104ed0 msgr2=0x55c209106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f68355d8740 1 -- 172.21.15.33:0/2448972915 shutdown_connections 2024-04-02T13:44:17.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f68355d8740 1 -- 172.21.15.33:0/2448972915 wait complete. 2024-04-02T13:44:17.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.975+0000 7f68355d8740 20 client.16833 trim_cache size 0 max 16384 2024-04-02T13:44:17.985 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:44:17.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.976+0000 7f6832915640 1 -- 172.21.15.33:0/3103258640 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f68140a3d20 msgr2=0x7f68140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.976+0000 7f6832915640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f68140a3d20 0x7f68140a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f681c02ce60 tx=0x7f681c009260 comp rx=0 tx=0).stop 2024-04-02T13:44:17.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.976+0000 7f6832915640 1 -- 172.21.15.33:0/3103258640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682c100970 msgr2=0x7f682c102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.976+0000 7f6832915640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682c100970 0x7f682c102f40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f681c000c00 tx=0x7f681c004570 comp rx=0 tx=0).stop 2024-04-02T13:44:17.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.976+0000 7f6832915640 1 -- 172.21.15.33:0/3103258640 shutdown_connections 2024-04-02T13:44:17.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.976+0000 7f6832915640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f68140a3d20 0x7f68140a61e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.976+0000 7f6832915640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f682c106980 0x7f682c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.976+0000 7f6832915640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f682c101330 0x7f682c103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.976+0000 7f6832915640 1 --2- 172.21.15.33:0/3103258640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f682c100970 0x7f682c102f40 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.976+0000 7f6832915640 1 -- 172.21.15.33:0/3103258640 >> 172.21.15.33:0/3103258640 conn(0x7f682c077b00 msgr2=0x7f682c069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.976+0000 7f6832915640 1 -- 172.21.15.33:0/3103258640 shutdown_connections 2024-04-02T13:44:17.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:17.976+0000 7f6832915640 1 -- 172.21.15.33:0/3103258640 wait complete. 2024-04-02T13:44:18.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:18 smithi145 ceph-mon[35646]: pgmap v168: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T13:44:18.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:18 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1029575657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:44:18.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:18 smithi050 ceph-mon[35524]: pgmap v168: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T13:44:18.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:18 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1029575657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:44:18.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:18 smithi033 ceph-mon[32390]: pgmap v168: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T13:44:18.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:18 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1029575657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:44:18.989 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:44:18.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:44:18.990 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:44:19.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.107+0000 7faa5561a640 1 -- 172.21.15.33:0/1554163333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50103690 msgr2=0x7faa5010bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.107+0000 7faa5561a640 1 --2- 172.21.15.33:0/1554163333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50103690 0x7faa5010bb70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7faa44009900 tx=0x7faa4402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.107+0000 7faa5561a640 1 -- 172.21.15.33:0/1554163333 shutdown_connections 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.107+0000 7faa5561a640 1 --2- 172.21.15.33:0/1554163333 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa5010c140 0x7faa5010e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.107+0000 7faa5561a640 1 --2- 172.21.15.33:0/1554163333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50103690 0x7faa5010bb70 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.107+0000 7faa5561a640 1 --2- 172.21.15.33:0/1554163333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa50102d70 0x7faa50103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.107+0000 7faa5561a640 1 -- 172.21.15.33:0/1554163333 >> 172.21.15.33:0/1554163333 conn(0x7faa50077b60 msgr2=0x7faa50077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.107+0000 7faa5561a640 1 -- 172.21.15.33:0/1554163333 shutdown_connections 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.107+0000 7faa5561a640 1 -- 172.21.15.33:0/1554163333 wait complete. 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.108+0000 7faa5561a640 1 Processor -- start 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.108+0000 7faa5561a640 1 -- start start 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.108+0000 7faa5561a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa50102d70 0x7faa50198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.108+0000 7faa5561a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50103690 0x7faa50198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.108+0000 7faa5561a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa5010c140 0x7faa5019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.108+0000 7faa5561a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa50111f40 con 0x7faa50103690 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.108+0000 7faa5561a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa50111dc0 con 0x7faa5010c140 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.108+0000 7faa5561a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faa501120c0 con 0x7faa50102d70 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.108+0000 7faa4e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50103690 0x7faa50198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.108+0000 7faa4e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50103690 0x7faa50198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59902/0 (socket says 172.21.15.33:59902) 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.108+0000 7faa4e7fc640 1 -- 172.21.15.33:0/2042166521 learned_addr learned my addr 172.21.15.33:0/2042166521 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa4f7fe640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa5010c140 0x7faa5019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa4effd640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa50102d70 0x7faa50198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa4e7fc640 1 -- 172.21.15.33:0/2042166521 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa50102d70 msgr2=0x7faa50198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa4e7fc640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa50102d70 0x7faa50198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa4e7fc640 1 -- 172.21.15.33:0/2042166521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa5010c140 msgr2=0x7faa5019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa4e7fc640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa5010c140 0x7faa5019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa4e7fc640 1 -- 172.21.15.33:0/2042166521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5019d3f0 con 0x7faa50103690 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa4effd640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa50102d70 0x7faa50198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa4e7fc640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50103690 0x7faa50198940 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7faa44005940 tx=0x7faa440050a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa37fff640 1 -- 172.21.15.33:0/2042166521 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44043020 con 0x7faa50103690 2024-04-02T13:44:19.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa37fff640 1 -- 172.21.15.33:0/2042166521 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7faa440054a0 con 0x7faa50103690 2024-04-02T13:44:19.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa5561a640 1 -- 172.21.15.33:0/2042166521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5019d680 con 0x7faa50103690 2024-04-02T13:44:19.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa37fff640 1 -- 172.21.15.33:0/2042166521 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44003c30 con 0x7faa50103690 2024-04-02T13:44:19.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.109+0000 7faa5561a640 1 -- 172.21.15.33:0/2042166521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa501a4f70 con 0x7faa50103690 2024-04-02T13:44:19.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.111+0000 7faa37fff640 1 -- 172.21.15.33:0/2042166521 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa44046020 con 0x7faa50103690 2024-04-02T13:44:19.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa581fd740 1 Processor -- start 2024-04-02T13:44:19.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa581fd740 1 -- start start 2024-04-02T13:44:19.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa581fd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e348fff0 0x55b0e34903d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa581fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e3490910 0x55b0e349d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa581fd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e349dbd0 0x55b0e349ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa581fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b0e33f4770 con 0x55b0e3490910 2024-04-02T13:44:19.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa581fd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b0e339c960 con 0x55b0e349dbd0 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa581fd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b0e33f08c0 con 0x55b0e348fff0 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa4effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e3490910 0x55b0e349d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa37fff640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7faa380a3da0 0x7faa380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa4effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e3490910 0x55b0e349d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59910/0 (socket says 172.21.15.33:59910) 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa4effd640 1 -- 172.21.15.33:0/2674117934 learned_addr learned my addr 172.21.15.33:0/2674117934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa4e7fc640 1 --2- 172.21.15.33:0/2674117934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e349dbd0 0x55b0e349ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa4f7fe640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7faa380a3da0 0x7faa380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa4f7fe640 1 --2- 172.21.15.33:0/2674117934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e348fff0 0x55b0e34903d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa4effd640 1 -- 172.21.15.33:0/2674117934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e348fff0 msgr2=0x55b0e34903d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa37fff640 1 -- 172.21.15.33:0/2042166521 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faa440f2d40 con 0x7faa50103690 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa4effd640 1 --2- 172.21.15.33:0/2674117934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e348fff0 0x55b0e34903d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.112+0000 7faa4effd640 1 -- 172.21.15.33:0/2674117934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e349dbd0 msgr2=0x55b0e349ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.113+0000 7faa4effd640 1 --2- 172.21.15.33:0/2674117934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e349dbd0 0x55b0e349ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.113+0000 7faa4effd640 1 -- 172.21.15.33:0/2674117934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e31d07e0 con 0x55b0e3490910 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.113+0000 7faa4e7fc640 1 --2- 172.21.15.33:0/2674117934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e349dbd0 0x55b0e349ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.113+0000 7faa4f7fe640 1 --2- 172.21.15.33:0/2674117934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e348fff0 0x55b0e34903d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.113+0000 7faa4effd640 1 --2- 172.21.15.33:0/2674117934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e3490910 0x55b0e349d690 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7faa4000bc20 tx=0x7faa4000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16c8a50b429df8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.113+0000 7faa4f7fe640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7faa380a3da0 0x7faa380a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faa4800a400 tx=0x7faa48009540 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.113+0000 7faa357fa640 1 -- 172.21.15.33:0/2674117934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40017620 con 0x55b0e3490910 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.113+0000 7faa357fa640 1 -- 172.21.15.33:0/2674117934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7faa40017c90 con 0x55b0e3490910 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.113+0000 7faa357fa640 1 -- 172.21.15.33:0/2674117934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4000fc10 con 0x55b0e3490910 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 -- 172.21.15.33:0/2674117934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e3490910 msgr2=0x55b0e349d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 --2- 172.21.15.33:0/2674117934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e3490910 0x55b0e349d690 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7faa4000bc20 tx=0x7faa4000d710 comp rx=0 tx=0).stop 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 -- 172.21.15.33:0/2674117934 shutdown_connections 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 --2- 172.21.15.33:0/2674117934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e349dbd0 0x55b0e349ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 --2- 172.21.15.33:0/2674117934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e3490910 0x55b0e349d690 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 --2- 172.21.15.33:0/2674117934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e348fff0 0x55b0e34903d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 -- 172.21.15.33:0/2674117934 >> 172.21.15.33:0/2674117934 conn(0x55b0e348a9f0 msgr2=0x55b0e348add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 -- 172.21.15.33:0/2674117934 shutdown_connections 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 -- 172.21.15.33:0/2674117934 wait complete. 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 -- start start 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 Processor -- start 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 0x55b0e349ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e3490910 0x55b0e349b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e349dbd0 0x55b0e349b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b0e34a0990 con 0x55b0e348fff0 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b0e33afd20 con 0x55b0e349dbd0 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b0e34a0b10 con 0x55b0e3490910 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa581fd740 10 client.0 _ll_register_callbacks cb 0x55b0e337ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa4effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 0x55b0e349ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.114+0000 7faa4effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 0x55b0e349ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59918/0 (socket says 172.21.15.33:59918) 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa4effd640 1 -- 172.21.15.33:0/3904875494 learned_addr learned my addr 172.21.15.33:0/3904875494 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa4f7fe640 1 --2- 172.21.15.33:0/3904875494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e349dbd0 0x55b0e349b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa4e7fc640 1 --2- 172.21.15.33:0/3904875494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e3490910 0x55b0e349b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa4effd640 1 -- 172.21.15.33:0/3904875494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e3490910 msgr2=0x55b0e349b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa4effd640 1 --2- 172.21.15.33:0/3904875494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e3490910 0x55b0e349b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa4effd640 1 -- 172.21.15.33:0/3904875494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e349dbd0 msgr2=0x55b0e349b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa4effd640 1 --2- 172.21.15.33:0/3904875494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e349dbd0 0x55b0e349b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa4effd640 1 -- 172.21.15.33:0/3904875494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e3230f70 con 0x55b0e348fff0 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa4effd640 1 --2- 172.21.15.33:0/3904875494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 0x55b0e349ab80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7faa40017650 tx=0x7faa40017680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69eb6a43156eb542 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa32ff5640 1 -- 172.21.15.33:0/3904875494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40025020 con 0x55b0e348fff0 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa32ff5640 1 -- 172.21.15.33:0/3904875494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faa40017a30 con 0x55b0e348fff0 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa581fd740 1 -- 172.21.15.33:0/3904875494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa40017c90 con 0x55b0e348fff0 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa32ff5640 1 -- 172.21.15.33:0/3904875494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faa40030050 con 0x55b0e348fff0 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa581fd740 10 client.16845 fetch_fsmap learned FSMap version 31 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa581fd740 1 -- 172.21.15.33:0/3904875494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b0e30c8170 con 0x55b0e348fff0 2024-04-02T13:44:19.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.115+0000 7faa32ff5640 1 -- 172.21.15.33:0/3904875494 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7faa40017c90 con 0x55b0e348fff0 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.116+0000 7faa32ff5640 1 -- 172.21.15.33:0/3904875494 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7faa400046c0 con 0x55b0e348fff0 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.116+0000 7faa581fd740 10 client.16845 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.116+0000 7faa581fd740 10 client.16845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.116+0000 7faa581fd740 20 client.16845 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.116+0000 7faa581fd740 1 --2- 172.21.15.33:0/3904875494 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b0e34adee0 0x55b0e34b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.116+0000 7faa581fd740 4 client.16845 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.116+0000 7faa581fd740 1 -- 172.21.15.33:0/3904875494 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0e3401cb0 con 0x55b0e34adee0 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.116+0000 7faa4e7fc640 1 --2- 172.21.15.33:0/3904875494 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b0e34adee0 0x55b0e34b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.116+0000 7faa4e7fc640 1 --2- 172.21.15.33:0/3904875494 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b0e34adee0 0x55b0e34b02d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cc085d51423b1d5 server_cookie=b75a8b1b555e11b7 in_seq=0 out_seq=0 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.116+0000 7faa32ff5640 10 client.16845 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa32ff5640 1 -- 172.21.15.33:0/3904875494 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0e3401cb0 con 0x55b0e34adee0 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa32ff5640 10 client.16845 handle_command_reply: tid=0 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa581fd740 1 client.16845 shutdown 2024-04-02T13:44:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa581fd740 1 -- 172.21.15.33:0/3904875494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 msgr2=0x55b0e349ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa581fd740 1 --2- 172.21.15.33:0/3904875494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 0x55b0e349ab80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7faa40017650 tx=0x7faa40017680 comp rx=0 tx=0).stop 2024-04-02T13:44:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa581fd740 1 -- 172.21.15.33:0/3904875494 shutdown_connections 2024-04-02T13:44:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa581fd740 1 --2- 172.21.15.33:0/3904875494 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b0e34adee0 0x55b0e34b02d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa581fd740 1 --2- 172.21.15.33:0/3904875494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e349dbd0 0x55b0e349b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa32ff5640 0 client.16845 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa581fd740 1 --2- 172.21.15.33:0/3904875494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e3490910 0x55b0e349b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa581fd740 1 --2- 172.21.15.33:0/3904875494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 0x55b0e349ab80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa581fd740 1 -- 172.21.15.33:0/3904875494 >> 172.21.15.33:0/3904875494 conn(0x55b0e348a9f0 msgr2=0x55b0e348c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa581fd740 1 -- 172.21.15.33:0/3904875494 shutdown_connections 2024-04-02T13:44:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.117+0000 7faa581fd740 1 -- 172.21.15.33:0/3904875494 wait complete. 2024-04-02T13:44:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.118+0000 7faa581fd740 20 client.16845 trim_cache size 0 max 16384 2024-04-02T13:44:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.135+0000 7faa581fd740 1 Processor -- start 2024-04-02T13:44:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.135+0000 7faa581fd740 1 -- start start 2024-04-02T13:44:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.135+0000 7faa581fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 0x55b0e349d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.135+0000 7faa581fd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e3490910 0x55b0e34a9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.135+0000 7faa581fd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e349dbd0 0x55b0e34b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.135+0000 7faa581fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa501af670 con 0x55b0e348fff0 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.135+0000 7faa581fd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa501aceb0 con 0x55b0e3490910 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.135+0000 7faa581fd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faa501aace0 con 0x55b0e349dbd0 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.136+0000 7faa4effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 0x55b0e349d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.136+0000 7faa4effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 0x55b0e349d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59934/0 (socket says 172.21.15.33:59934) 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.136+0000 7faa4effd640 1 -- 172.21.15.33:0/266759066 learned_addr learned my addr 172.21.15.33:0/266759066 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.136+0000 7faa4effd640 1 -- 172.21.15.33:0/266759066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e349dbd0 msgr2=0x55b0e34b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.136+0000 7faa4effd640 1 --2- 172.21.15.33:0/266759066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e349dbd0 0x55b0e34b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.136+0000 7faa4effd640 1 -- 172.21.15.33:0/266759066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e3490910 msgr2=0x55b0e34a9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.136+0000 7faa4effd640 1 --2- 172.21.15.33:0/266759066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e3490910 0x55b0e34a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.136+0000 7faa4effd640 1 -- 172.21.15.33:0/266759066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e348a820 con 0x55b0e348fff0 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.136+0000 7faa4effd640 1 --2- 172.21.15.33:0/266759066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 0x55b0e349d2b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7faa40004ae0 tx=0x7faa40005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673c609bbce53196 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.136+0000 7faa34ff9640 1 -- 172.21.15.33:0/266759066 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40039070 con 0x55b0e348fff0 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.136+0000 7faa34ff9640 1 -- 172.21.15.33:0/266759066 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7faa4000dc10 con 0x55b0e348fff0 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.136+0000 7faa34ff9640 1 -- 172.21.15.33:0/266759066 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40025070 con 0x55b0e348fff0 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 -- 172.21.15.33:0/266759066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 msgr2=0x55b0e349d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 --2- 172.21.15.33:0/266759066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 0x55b0e349d2b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7faa40004ae0 tx=0x7faa40005ed0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 -- 172.21.15.33:0/266759066 shutdown_connections 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 --2- 172.21.15.33:0/266759066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e349dbd0 0x55b0e34b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 --2- 172.21.15.33:0/266759066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e3490910 0x55b0e34a9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 --2- 172.21.15.33:0/266759066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e348fff0 0x55b0e349d2b0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 -- 172.21.15.33:0/266759066 >> 172.21.15.33:0/266759066 conn(0x55b0e348a9f0 msgr2=0x55b0e348c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 -- 172.21.15.33:0/266759066 shutdown_connections 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 -- 172.21.15.33:0/266759066 wait complete. 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 -- start start 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 Processor -- start 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e348fff0 0x55b0e34a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e3490910 0x55b0e34a6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e349dbd0 0x55b0e34a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa501ab370 con 0x55b0e349dbd0 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa501ac7e0 con 0x55b0e348fff0 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faa501aba40 con 0x55b0e3490910 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa581fd740 10 client.0 _ll_register_callbacks cb 0x7faa4000ba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e349dbd0 0x55b0e34a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e349dbd0 0x55b0e34a2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59938/0 (socket says 172.21.15.33:59938) 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.137+0000 7faa4f7fe640 1 -- 172.21.15.33:0/1408667582 learned_addr learned my addr 172.21.15.33:0/1408667582 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa4effd640 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e348fff0 0x55b0e34a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa4e7fc640 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e3490910 0x55b0e34a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa4f7fe640 1 -- 172.21.15.33:0/1408667582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e3490910 msgr2=0x55b0e34a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa4f7fe640 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e3490910 0x55b0e34a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa4f7fe640 1 -- 172.21.15.33:0/1408667582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e348fff0 msgr2=0x55b0e34a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa4f7fe640 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e348fff0 0x55b0e34a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa4f7fe640 1 -- 172.21.15.33:0/1408667582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e349ffc0 con 0x55b0e349dbd0 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa4effd640 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e348fff0 0x55b0e34a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa4e7fc640 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e3490910 0x55b0e34a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa4f7fe640 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e349dbd0 0x55b0e34a2e00 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7faa48016dc0 tx=0x7faa48004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6c3a91a5441e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa32ff5640 1 -- 172.21.15.33:0/1408667582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48026020 con 0x55b0e349dbd0 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa32ff5640 1 -- 172.21.15.33:0/1408667582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faa48014ab0 con 0x55b0e349dbd0 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa581fd740 1 -- 172.21.15.33:0/1408667582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa4000dc10 con 0x55b0e349dbd0 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa32ff5640 1 -- 172.21.15.33:0/1408667582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faa48014cb0 con 0x55b0e349dbd0 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa581fd740 10 client.16851 fetch_fsmap learned FSMap version 31 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa581fd740 1 -- 172.21.15.33:0/1408667582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b0e3514bc0 con 0x55b0e349dbd0 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.138+0000 7faa32ff5640 1 -- 172.21.15.33:0/1408667582 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7faa4000dc10 con 0x55b0e349dbd0 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.139+0000 7faa32ff5640 1 -- 172.21.15.33:0/1408667582 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7faa4802a4a0 con 0x55b0e349dbd0 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.139+0000 7faa581fd740 10 client.16851 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.139+0000 7faa581fd740 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.139+0000 7faa581fd740 20 client.16851 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.139+0000 7faa581fd740 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b0e3515420 0x55b0e3517840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.139+0000 7faa581fd740 4 client.16851 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:44:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.139+0000 7faa581fd740 1 -- 172.21.15.33:0/1408667582 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b0e33fc000 con 0x55b0e3515420 2024-04-02T13:44:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.139+0000 7faa4effd640 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b0e3515420 0x55b0e3517840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.139+0000 7faa4effd640 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b0e3515420 0x55b0e3517840 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=720b7a1bd6cf5d52 server_cookie=f624fbe8139c57ae in_seq=0 out_seq=0 2024-04-02T13:44:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.139+0000 7faa32ff5640 10 client.16851 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa32ff5640 1 -- 172.21.15.33:0/1408667582 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b0e33fc000 con 0x55b0e3515420 2024-04-02T13:44:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa32ff5640 10 client.16851 handle_command_reply: tid=0 2024-04-02T13:44:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa581fd740 1 client.16851 shutdown 2024-04-02T13:44:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa581fd740 1 -- 172.21.15.33:0/1408667582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e349dbd0 msgr2=0x55b0e34a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa581fd740 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e349dbd0 0x55b0e34a2e00 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7faa48016dc0 tx=0x7faa48004b50 comp rx=0 tx=0).stop 2024-04-02T13:44:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa581fd740 1 -- 172.21.15.33:0/1408667582 shutdown_connections 2024-04-02T13:44:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa581fd740 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b0e3515420 0x55b0e3517840 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa581fd740 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0e349dbd0 0x55b0e34a2e00 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa32ff5640 0 client.16851 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa581fd740 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b0e3490910 0x55b0e34a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa581fd740 1 --2- 172.21.15.33:0/1408667582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0e348fff0 0x55b0e34a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa581fd740 1 -- 172.21.15.33:0/1408667582 >> 172.21.15.33:0/1408667582 conn(0x55b0e348a9f0 msgr2=0x55b0e348bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa581fd740 1 -- 172.21.15.33:0/1408667582 shutdown_connections 2024-04-02T13:44:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa581fd740 1 -- 172.21.15.33:0/1408667582 wait complete. 2024-04-02T13:44:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa581fd740 20 client.16851 trim_cache size 0 max 16384 2024-04-02T13:44:19.149 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:44:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:44:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "f5e072fb-6988-469c-9581-f0ef6bbdf79c", 2024-04-02T13:44:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:44:19.150 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:44:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa5561a640 1 -- 172.21.15.33:0/2042166521 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7faa380a3da0 msgr2=0x7faa380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa5561a640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7faa380a3da0 0x7faa380a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faa4800a400 tx=0x7faa48009540 comp rx=0 tx=0).stop 2024-04-02T13:44:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa5561a640 1 -- 172.21.15.33:0/2042166521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50103690 msgr2=0x7faa50198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa5561a640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50103690 0x7faa50198940 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7faa44005940 tx=0x7faa440050a0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa5561a640 1 -- 172.21.15.33:0/2042166521 shutdown_connections 2024-04-02T13:44:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.141+0000 7faa5561a640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7faa380a3da0 0x7faa380a6260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.142+0000 7faa5561a640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa5010c140 0x7faa5019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.142+0000 7faa5561a640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa50103690 0x7faa50198940 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.142+0000 7faa5561a640 1 --2- 172.21.15.33:0/2042166521 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faa50102d70 0x7faa50198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.142+0000 7faa5561a640 1 -- 172.21.15.33:0/2042166521 >> 172.21.15.33:0/2042166521 conn(0x7faa50077b60 msgr2=0x7faa500fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.142+0000 7faa5561a640 1 -- 172.21.15.33:0/2042166521 shutdown_connections 2024-04-02T13:44:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.142+0000 7faa5561a640 1 -- 172.21.15.33:0/2042166521 wait complete. 2024-04-02T13:44:19.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:44:19.155 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:44:19.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.257+0000 7fdae12d6640 1 -- 172.21.15.33:0/4040163776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdadc101330 msgr2=0x7fdadc10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.258+0000 7fdae12d6640 1 --2- 172.21.15.33:0/4040163776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdadc101330 0x7fdadc10d840 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009cd0 tx=0x7fdad402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.258+0000 7fdae12d6640 1 -- 172.21.15.33:0/4040163776 shutdown_connections 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.258+0000 7fdae12d6640 1 --2- 172.21.15.33:0/4040163776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdadc101330 0x7fdadc10d840 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.258+0000 7fdae12d6640 1 --2- 172.21.15.33:0/4040163776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdadc100970 0x7fdadc100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.258+0000 7fdae12d6640 1 --2- 172.21.15.33:0/4040163776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdadc106980 0x7fdadc106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.258+0000 7fdae12d6640 1 -- 172.21.15.33:0/4040163776 >> 172.21.15.33:0/4040163776 conn(0x7fdadc077b00 msgr2=0x7fdadc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.258+0000 7fdae12d6640 1 -- 172.21.15.33:0/4040163776 shutdown_connections 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.258+0000 7fdae12d6640 1 -- 172.21.15.33:0/4040163776 wait complete. 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.258+0000 7fdae12d6640 1 Processor -- start 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.258+0000 7fdae12d6640 1 -- start start 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.258+0000 7fdae12d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdadc100970 0x7fdadc197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdae12d6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdadc101330 0x7fdadc197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdae12d6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdadc106980 0x7fdadc19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdae12d6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc110d60 con 0x7fdadc100970 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdae12d6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc110be0 con 0x7fdadc106980 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdae12d6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc110ee0 con 0x7fdadc101330 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdadaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdadc100970 0x7fdadc197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdadaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdadc100970 0x7fdadc197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59954/0 (socket says 172.21.15.33:59954) 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdadaffd640 1 -- 172.21.15.33:0/277302064 learned_addr learned my addr 172.21.15.33:0/277302064 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdadb7fe640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdadc106980 0x7fdadc19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdada7fc640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdadc101330 0x7fdadc197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdadaffd640 1 -- 172.21.15.33:0/277302064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdadc101330 msgr2=0x7fdadc197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdadaffd640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdadc101330 0x7fdadc197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdadaffd640 1 -- 172.21.15.33:0/277302064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdadc106980 msgr2=0x7fdadc19bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdadaffd640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdadc106980 0x7fdadc19bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdadaffd640 1 -- 172.21.15.33:0/277302064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc19c230 con 0x7fdadc100970 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdadb7fe640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdadc106980 0x7fdadc19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdadaffd640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdadc100970 0x7fdadc197240 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009860 tx=0x7fdacc009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdac3fff640 1 -- 172.21.15.33:0/277302064 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc005020 con 0x7fdadc100970 2024-04-02T13:44:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdae12d6640 1 -- 172.21.15.33:0/277302064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdadc19c520 con 0x7fdadc100970 2024-04-02T13:44:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdac3fff640 1 -- 172.21.15.33:0/277302064 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdacc0047a0 con 0x7fdadc100970 2024-04-02T13:44:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdac3fff640 1 -- 172.21.15.33:0/277302064 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc007ba0 con 0x7fdadc100970 2024-04-02T13:44:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.259+0000 7fdae12d6640 1 -- 172.21.15.33:0/277302064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdadc068c60 con 0x7fdadc100970 2024-04-02T13:44:19.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.261+0000 7fdac3fff640 1 -- 172.21.15.33:0/277302064 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdacc004950 con 0x7fdadc100970 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdae3f9a740 1 Processor -- start 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdae3f9a740 1 -- start start 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdac3fff640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fdac40a3d20 0x7fdac40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdae3f9a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 0x5638404e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdae3f9a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e71f0 0x5638404ef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdae3f9a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x5638404f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdae3f9a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56384044a830 con 0x5638404efdd0 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdae3f9a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5638403f2820 con 0x5638404e71f0 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdae3f9a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5638404467e0 con 0x5638404e68d0 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdadb7fe640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fdac40a3d20 0x7fdac40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdac3fff640 1 -- 172.21.15.33:0/277302064 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fdacc0d2080 con 0x7fdadc100970 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdadb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x5638404f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdadb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x5638404f21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59968/0 (socket says 172.21.15.33:59968) 2024-04-02T13:44:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdadb7fe640 1 -- 172.21.15.33:0/3871010015 learned_addr learned my addr 172.21.15.33:0/3871010015 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdadaffd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 0x5638404e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.262+0000 7fdada7fc640 1 --2- 172.21.15.33:0/3871010015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e71f0 0x5638404ef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.263+0000 7fdadb7fe640 1 -- 172.21.15.33:0/3871010015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 msgr2=0x5638404e6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.263+0000 7fdadb7fe640 1 --2- 172.21.15.33:0/3871010015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 0x5638404e6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.263+0000 7fdadb7fe640 1 -- 172.21.15.33:0/3871010015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e71f0 msgr2=0x5638404ef890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.263+0000 7fdadb7fe640 1 --2- 172.21.15.33:0/3871010015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e71f0 0x5638404ef890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.263+0000 7fdadb7fe640 1 -- 172.21.15.33:0/3871010015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638402287d0 con 0x5638404efdd0 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.263+0000 7fdadaffd640 1 --2- 172.21.15.33:0/3871010015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 0x5638404e6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.263+0000 7fdadb7fe640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fdac40a3d20 0x7fdac40a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdad4036000 tx=0x7fdad4009880 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.263+0000 7fdadb7fe640 1 --2- 172.21.15.33:0/3871010015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x5638404f21c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdad402c630 tx=0x7fdad4046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75055aae90f54238 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.263+0000 7fdac17fa640 1 -- 172.21.15.33:0/3871010015 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4004b40 con 0x5638404efdd0 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.263+0000 7fdac17fa640 1 -- 172.21.15.33:0/3871010015 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdad403a050 con 0x5638404efdd0 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.263+0000 7fdac17fa640 1 -- 172.21.15.33:0/3871010015 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad40044b0 con 0x5638404efdd0 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 -- 172.21.15.33:0/3871010015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 msgr2=0x5638404f21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/3871010015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x5638404f21c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdad402c630 tx=0x7fdad4046000 comp rx=0 tx=0).stop 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 -- 172.21.15.33:0/3871010015 shutdown_connections 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/3871010015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x5638404f21c0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/3871010015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e71f0 0x5638404ef890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/3871010015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 0x5638404e6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 -- 172.21.15.33:0/3871010015 >> 172.21.15.33:0/3871010015 conn(0x5638404e1000 msgr2=0x5638404e13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 -- 172.21.15.33:0/3871010015 shutdown_connections 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 -- 172.21.15.33:0/3871010015 wait complete. 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 -- start start 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 Processor -- start 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e68d0 0x5638404ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e71f0 0x5638404eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x5638404ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5638404f2b60 con 0x5638404efdd0 2024-04-02T13:44:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563840405d30 con 0x5638404e68d0 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5638404f2ce0 con 0x5638404e71f0 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdae3f9a740 10 client.0 _ll_register_callbacks cb 0x563840405720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdadb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x5638404ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdadb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x5638404ecfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59980/0 (socket says 172.21.15.33:59980) 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.264+0000 7fdadb7fe640 1 -- 172.21.15.33:0/3300707157 learned_addr learned my addr 172.21.15.33:0/3300707157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdadaffd640 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e68d0 0x5638404ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdada7fc640 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e71f0 0x5638404eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdadb7fe640 1 -- 172.21.15.33:0/3300707157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e71f0 msgr2=0x5638404eca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdadb7fe640 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e71f0 0x5638404eca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdadb7fe640 1 -- 172.21.15.33:0/3300707157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e68d0 msgr2=0x5638404ef6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdadb7fe640 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e68d0 0x5638404ef6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdadb7fe640 1 -- 172.21.15.33:0/3300707157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56384015e3f0 con 0x5638404efdd0 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdada7fc640 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e71f0 0x5638404eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdadb7fe640 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x5638404ecfb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009cd0 tx=0x7fdad402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912f045e86d5ff48 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdabeff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdabeff5640 1 -- 172.21.15.33:0/3300707157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad405e060 con 0x5638404efdd0 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdabeff5640 1 -- 172.21.15.33:0/3300707157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fdad4032950 con 0x5638404efdd0 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdae3f9a740 1 -- 172.21.15.33:0/3300707157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad403a050 con 0x5638404efdd0 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdabeff5640 1 -- 172.21.15.33:0/3300707157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdad4065050 con 0x5638404efdd0 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdae3f9a740 10 client.16863 fetch_fsmap learned FSMap version 31 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdae3f9a740 1 -- 172.21.15.33:0/3300707157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56384011e0f0 con 0x5638404efdd0 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.265+0000 7fdabeff5640 1 -- 172.21.15.33:0/3300707157 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdad403a050 con 0x5638404efdd0 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.266+0000 7fdabeff5640 1 -- 172.21.15.33:0/3300707157 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fdad404f610 con 0x5638404efdd0 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.266+0000 7fdae3f9a740 10 client.16863 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.266+0000 7fdae3f9a740 10 client.16863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.266+0000 7fdae3f9a740 20 client.16863 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.266+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5638404ffc30 0x563840502020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.266+0000 7fdae3f9a740 4 client.16863 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.266+0000 7fdae3f9a740 1 -- 172.21.15.33:0/3300707157 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563840457d70 con 0x5638404ffc30 2024-04-02T13:44:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.266+0000 7fdadaffd640 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5638404ffc30 0x563840502020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.266+0000 7fdadaffd640 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5638404ffc30 0x563840502020 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f4973d7a18d668 server_cookie=df6090817bdf7f8d in_seq=0 out_seq=0 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.266+0000 7fdabeff5640 10 client.16863 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdabeff5640 1 -- 172.21.15.33:0/3300707157 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563840457d70 con 0x5638404ffc30 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdabeff5640 10 client.16863 handle_command_reply: tid=0 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdae3f9a740 1 client.16863 shutdown 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdae3f9a740 1 -- 172.21.15.33:0/3300707157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 msgr2=0x5638404ecfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x5638404ecfb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009cd0 tx=0x7fdad402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdae3f9a740 1 -- 172.21.15.33:0/3300707157 shutdown_connections 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5638404ffc30 0x563840502020 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x5638404ecfb0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e71f0 0x5638404eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdabeff5640 0 client.16863 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/3300707157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e68d0 0x5638404ef6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdae3f9a740 1 -- 172.21.15.33:0/3300707157 >> 172.21.15.33:0/3300707157 conn(0x5638404e1000 msgr2=0x5638404e2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdae3f9a740 1 -- 172.21.15.33:0/3300707157 shutdown_connections 2024-04-02T13:44:19.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdae3f9a740 1 -- 172.21.15.33:0/3300707157 wait complete. 2024-04-02T13:44:19.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.267+0000 7fdae3f9a740 20 client.16863 trim_cache size 0 max 16384 2024-04-02T13:44:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.284+0000 7fdae3f9a740 1 Processor -- start 2024-04-02T13:44:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdae3f9a740 1 -- start start 2024-04-02T13:44:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdae3f9a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 0x5638404fba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdae3f9a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404e71f0 0x5638404e00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdae3f9a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404efdd0 0x5638405075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdae3f9a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc1af6f0 con 0x5638404e71f0 2024-04-02T13:44:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdae3f9a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc1acf30 con 0x5638404efdd0 2024-04-02T13:44:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdae3f9a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc1aad60 con 0x5638404e68d0 2024-04-02T13:44:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdada7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404e71f0 0x5638404e00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdada7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404e71f0 0x5638404e00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59992/0 (socket says 172.21.15.33:59992) 2024-04-02T13:44:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdada7fc640 1 -- 172.21.15.33:0/1126953430 learned_addr learned my addr 172.21.15.33:0/1126953430 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdada7fc640 1 -- 172.21.15.33:0/1126953430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 msgr2=0x5638404fba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdada7fc640 1 --2- 172.21.15.33:0/1126953430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 0x5638404fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdada7fc640 1 -- 172.21.15.33:0/1126953430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404efdd0 msgr2=0x5638405075f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdada7fc640 1 --2- 172.21.15.33:0/1126953430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404efdd0 0x5638405075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdada7fc640 1 -- 172.21.15.33:0/1126953430 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638404e0e30 con 0x5638404e71f0 2024-04-02T13:44:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdada7fc640 1 --2- 172.21.15.33:0/1126953430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404e71f0 0x5638404e00c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009a40 tx=0x7fdad0009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a149cf0564481913 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdac0ff9640 1 -- 172.21.15.33:0/1126953430 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0012b10 con 0x5638404e71f0 2024-04-02T13:44:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdac0ff9640 1 -- 172.21.15.33:0/1126953430 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdad0012cb0 con 0x5638404e71f0 2024-04-02T13:44:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.285+0000 7fdac0ff9640 1 -- 172.21.15.33:0/1126953430 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad001b410 con 0x5638404e71f0 2024-04-02T13:44:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.286+0000 7fdae3f9a740 1 -- 172.21.15.33:0/1126953430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404e71f0 msgr2=0x5638404e00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.286+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/1126953430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404e71f0 0x5638404e00c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009a40 tx=0x7fdad0009f00 comp rx=0 tx=0).stop 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.286+0000 7fdae3f9a740 1 -- 172.21.15.33:0/1126953430 shutdown_connections 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.286+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/1126953430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404efdd0 0x5638405075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.286+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/1126953430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404e71f0 0x5638404e00c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.286+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/1126953430 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 0x5638404fba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.286+0000 7fdae3f9a740 1 -- 172.21.15.33:0/1126953430 >> 172.21.15.33:0/1126953430 conn(0x5638404e1000 msgr2=0x5638404df9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.286+0000 7fdae3f9a740 1 -- 172.21.15.33:0/1126953430 shutdown_connections 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.286+0000 7fdae3f9a740 1 -- 172.21.15.33:0/1126953430 wait complete. 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.286+0000 7fdae3f9a740 1 -- start start 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.286+0000 7fdae3f9a740 1 Processor -- start 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdae3f9a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 0x5638404e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdae3f9a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e71f0 0x5638404ea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdae3f9a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x563840564e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdae3f9a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc1ab3f0 con 0x5638404efdd0 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdae3f9a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc1ac860 con 0x5638404e71f0 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdae3f9a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc1abac0 con 0x5638404e68d0 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdae3f9a740 10 client.0 _ll_register_callbacks cb 0x7fdad0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdadb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x563840564e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdadb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x563840564e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60002/0 (socket says 172.21.15.33:60002) 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdadb7fe640 1 -- 172.21.15.33:0/288894768 learned_addr learned my addr 172.21.15.33:0/288894768 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdadaffd640 1 --2- 172.21.15.33:0/288894768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 0x5638404e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdada7fc640 1 --2- 172.21.15.33:0/288894768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e71f0 0x5638404ea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdadb7fe640 1 -- 172.21.15.33:0/288894768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 msgr2=0x5638404e9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdadb7fe640 1 --2- 172.21.15.33:0/288894768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 0x5638404e9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdadb7fe640 1 -- 172.21.15.33:0/288894768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e71f0 msgr2=0x5638404ea2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdadb7fe640 1 --2- 172.21.15.33:0/288894768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e71f0 0x5638404ea2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdadb7fe640 1 -- 172.21.15.33:0/288894768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638405653a0 con 0x5638404efdd0 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdadb7fe640 1 --2- 172.21.15.33:0/288894768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x563840564e60 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdad4006010 tx=0x7fdad402c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288cefdfd77d61ff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdabeff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdabeff5640 1 -- 172.21.15.33:0/288894768 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad405e040 con 0x5638404efdd0 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdabeff5640 1 -- 172.21.15.33:0/288894768 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fdad404f480 con 0x5638404efdd0 2024-04-02T13:44:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdae3f9a740 1 -- 172.21.15.33:0/288894768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad0012cb0 con 0x5638404efdd0 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdabeff5640 1 -- 172.21.15.33:0/288894768 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdad4032660 con 0x5638404efdd0 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdae3f9a740 10 client.16869 fetch_fsmap learned FSMap version 31 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.287+0000 7fdae3f9a740 1 -- 172.21.15.33:0/288894768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563840566610 con 0x5638404efdd0 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.288+0000 7fdabeff5640 1 -- 172.21.15.33:0/288894768 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdad0012cb0 con 0x5638404efdd0 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.288+0000 7fdabeff5640 1 -- 172.21.15.33:0/288894768 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fdad4005030 con 0x5638404efdd0 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.288+0000 7fdae3f9a740 10 client.16869 fetch_fsmap finished waiting for FSMap version 31 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.288+0000 7fdae3f9a740 10 client.16869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.288+0000 7fdae3f9a740 20 client.16869 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.288+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/288894768 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5638404ffc30 0x563840568c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.288+0000 7fdae3f9a740 4 client.16869 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.288+0000 7fdae3f9a740 1 -- 172.21.15.33:0/288894768 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5638404520c0 con 0x5638404ffc30 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.288+0000 7fdadaffd640 1 --2- 172.21.15.33:0/288894768 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5638404ffc30 0x563840568c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdabeff5640 10 client.16869 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdadaffd640 1 --2- 172.21.15.33:0/288894768 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5638404ffc30 0x563840568c30 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da355ee7e3019761 server_cookie=22e3561aeac71a3e in_seq=0 out_seq=0 2024-04-02T13:44:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdabeff5640 1 -- 172.21.15.33:0/288894768 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5638404520c0 con 0x5638404ffc30 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdabeff5640 10 client.16869 handle_command_reply: tid=0 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae3f9a740 1 client.16869 shutdown 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae3f9a740 1 -- 172.21.15.33:0/288894768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 msgr2=0x563840564e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/288894768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x563840564e60 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdad4006010 tx=0x7fdad402c900 comp rx=0 tx=0).stop 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae3f9a740 1 -- 172.21.15.33:0/288894768 shutdown_connections 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/288894768 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5638404ffc30 0x563840568c30 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/288894768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638404efdd0 0x563840564e60 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/288894768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638404e71f0 0x5638404ea2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdabeff5640 0 client.16869 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae3f9a740 1 --2- 172.21.15.33:0/288894768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5638404e68d0 0x5638404e9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae3f9a740 1 -- 172.21.15.33:0/288894768 >> 172.21.15.33:0/288894768 conn(0x5638404e1000 msgr2=0x5638404e0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae3f9a740 1 -- 172.21.15.33:0/288894768 shutdown_connections 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae3f9a740 1 -- 172.21.15.33:0/288894768 wait complete. 2024-04-02T13:44:19.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae3f9a740 20 client.16869 trim_cache size 0 max 16384 2024-04-02T13:44:19.299 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:44:19.299 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:44:19.299 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:44:19.299 INFO:teuthology.orchestra.run.smithi033.stdout: "f5e072fb-6988-469c-9581-f0ef6bbdf79c": { 2024-04-02T13:44:19.299 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:44:19.299 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "f5e072fb-6988-469c-9581-f0ef6bbdf79c", 2024-04-02T13:44:19.299 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:44:19.299 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:44:19.299 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:44:19.299 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:44:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae12d6640 1 -- 172.21.15.33:0/277302064 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fdac40a3d20 msgr2=0x7fdac40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae12d6640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fdac40a3d20 0x7fdac40a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdad4036000 tx=0x7fdad4009880 comp rx=0 tx=0).stop 2024-04-02T13:44:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae12d6640 1 -- 172.21.15.33:0/277302064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdadc100970 msgr2=0x7fdadc197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.289+0000 7fdae12d6640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdadc100970 0x7fdadc197240 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009860 tx=0x7fdacc009d20 comp rx=0 tx=0).stop 2024-04-02T13:44:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.290+0000 7fdae12d6640 1 -- 172.21.15.33:0/277302064 shutdown_connections 2024-04-02T13:44:19.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.290+0000 7fdae12d6640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fdac40a3d20 0x7fdac40a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.290+0000 7fdae12d6640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdadc106980 0x7fdadc19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.290+0000 7fdae12d6640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdadc101330 0x7fdadc197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.290+0000 7fdae12d6640 1 --2- 172.21.15.33:0/277302064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdadc100970 0x7fdadc197240 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.290+0000 7fdae12d6640 1 -- 172.21.15.33:0/277302064 >> 172.21.15.33:0/277302064 conn(0x7fdadc077b00 msgr2=0x7fdadc103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.290+0000 7fdae12d6640 1 -- 172.21.15.33:0/277302064 shutdown_connections 2024-04-02T13:44:19.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:19.290+0000 7fdae12d6640 1 -- 172.21.15.33:0/277302064 wait complete. 2024-04-02T13:44:19.304 INFO:tasks.cephfs.filesystem:scrub status for tag:f5e072fb-6988-469c-9581-f0ef6bbdf79c - {'path': '/', 'tag': 'f5e072fb-6988-469c-9581-f0ef6bbdf79c', 'options': 'recursive,force'} 2024-04-02T13:44:20.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:20 smithi145 ceph-mon[35646]: pgmap v169: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T13:44:20.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:20 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:44:20.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:20 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:20.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:20 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:44:20.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:20 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:20.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:20 smithi050 ceph-mon[35524]: pgmap v169: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T13:44:20.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:20 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:44:20.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:20 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:20.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:20 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:44:20.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:20 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:20.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:20 smithi033 ceph-mon[32390]: pgmap v169: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-02T13:44:20.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:20 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:44:20.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:20 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:20.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:20 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:44:20.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:20 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:21.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:21 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:44:21.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:21 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:44:21.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:21 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:44:22.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:22 smithi145 ceph-mon[35646]: pgmap v170: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-02T13:44:22.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:22 smithi050 ceph-mon[35524]: pgmap v170: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-02T13:44:22.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:22 smithi033 ceph-mon[32390]: pgmap v170: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-02T13:44:24.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:24 smithi145 ceph-mon[35646]: pgmap v171: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T13:44:24.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:24 smithi050 ceph-mon[35524]: pgmap v171: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T13:44:24.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:24 smithi033 ceph-mon[32390]: pgmap v171: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T13:44:26.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:26 smithi145 ceph-mon[35646]: pgmap v172: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:26.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:26 smithi050 ceph-mon[35524]: pgmap v172: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:26.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:26 smithi033 ceph-mon[32390]: pgmap v172: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:28.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:28 smithi145 ceph-mon[35646]: pgmap v173: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:28.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:28 smithi050 ceph-mon[35524]: pgmap v173: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:29.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:28 smithi033 ceph-mon[32390]: pgmap v173: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:30.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:30 smithi145 ceph-mon[35646]: pgmap v174: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:44:30.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:30 smithi050 ceph-mon[35524]: pgmap v174: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:44:31.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:30 smithi033 ceph-mon[32390]: pgmap v174: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:44:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:32 smithi145 ceph-mon[35646]: pgmap v175: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:32 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:32 smithi050 ceph-mon[35524]: pgmap v175: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:32 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:33.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:32 smithi033 ceph-mon[32390]: pgmap v175: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:33.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:32 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:34.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:34 smithi145 ceph-mon[35646]: pgmap v176: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:44:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:34 smithi050 ceph-mon[35524]: pgmap v176: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:44:35.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:34 smithi033 ceph-mon[32390]: pgmap v176: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:44:36.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:36 smithi145 ceph-mon[35646]: pgmap v177: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:36 smithi050 ceph-mon[35524]: pgmap v177: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:37.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:36 smithi033 ceph-mon[32390]: pgmap v177: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:37.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:37 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:37.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:37 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:38.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:37 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:38.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:38 smithi145 ceph-mon[35646]: pgmap v178: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:38 smithi050 ceph-mon[35524]: pgmap v178: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:39.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:38 smithi033 ceph-mon[32390]: pgmap v178: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:40.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:40 smithi145 ceph-mon[35646]: pgmap v179: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:44:40.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:40 smithi050 ceph-mon[35524]: pgmap v179: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:44:41.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:40 smithi033 ceph-mon[32390]: pgmap v179: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:44:42.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:42 smithi145 ceph-mon[35646]: pgmap v180: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:42.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:42 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:42.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:42 smithi050 ceph-mon[35524]: pgmap v180: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:42.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:42 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:43.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:42 smithi033 ceph-mon[32390]: pgmap v180: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:43.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:42 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:44.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:44 smithi145 ceph-mon[35646]: pgmap v181: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:44:44.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:44 smithi050 ceph-mon[35524]: pgmap v181: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:44:45.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:44 smithi033 ceph-mon[32390]: pgmap v181: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:44:46.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:46 smithi145 ceph-mon[35646]: pgmap v182: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:46 smithi050 ceph-mon[35524]: pgmap v182: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:47.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:46 smithi033 ceph-mon[32390]: pgmap v182: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:48.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:48 smithi145 ceph-mon[35646]: pgmap v183: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:48.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:48 smithi050 ceph-mon[35524]: pgmap v183: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:49.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:48 smithi033 ceph-mon[32390]: pgmap v183: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:49.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:44:49.306 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:44:49.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.420+0000 7fedd8c63640 1 -- 172.21.15.33:0/3440715046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedd4100970 msgr2=0x7fedd4100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.420+0000 7fedd8c63640 1 --2- 172.21.15.33:0/3440715046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedd4100970 0x7fedd4100df0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009930 tx=0x7fedc802bc30 comp rx=0 tx=0).stop 2024-04-02T13:44:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.420+0000 7fedd8c63640 1 -- 172.21.15.33:0/3440715046 shutdown_connections 2024-04-02T13:44:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.420+0000 7fedd8c63640 1 --2- 172.21.15.33:0/3440715046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedd4101330 0x7fedd410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.420+0000 7fedd8c63640 1 --2- 172.21.15.33:0/3440715046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedd4100970 0x7fedd4100df0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.420+0000 7fedd8c63640 1 --2- 172.21.15.33:0/3440715046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fedd4106980 0x7fedd4106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.420+0000 7fedd8c63640 1 -- 172.21.15.33:0/3440715046 >> 172.21.15.33:0/3440715046 conn(0x7fedd4077b00 msgr2=0x7fedd4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.420+0000 7fedd8c63640 1 -- 172.21.15.33:0/3440715046 shutdown_connections 2024-04-02T13:44:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.420+0000 7fedd8c63640 1 -- 172.21.15.33:0/3440715046 wait complete. 2024-04-02T13:44:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.420+0000 7fedd8c63640 1 Processor -- start 2024-04-02T13:44:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.420+0000 7fedd8c63640 1 -- start start 2024-04-02T13:44:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd8c63640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedd4100970 0x7fedd4102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd8c63640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedd4101330 0x7fedd4103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd8c63640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fedd4106980 0x7fedd4103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd8c63640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4110d60 con 0x7fedd4101330 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd8c63640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4110be0 con 0x7fedd4100970 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd8c63640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4110ee0 con 0x7fedd4106980 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedd4101330 0x7fedd4103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedd4101330 0x7fedd4103520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36140/0 (socket says 172.21.15.33:36140) 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd1d9b640 1 -- 172.21.15.33:0/3537576896 learned_addr learned my addr 172.21.15.33:0/3537576896 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd2d9d640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fedd4106980 0x7fedd4103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd259c640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedd4100970 0x7fedd4102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd1d9b640 1 -- 172.21.15.33:0/3537576896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fedd4106980 msgr2=0x7fedd4103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd1d9b640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fedd4106980 0x7fedd4103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd1d9b640 1 -- 172.21.15.33:0/3537576896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedd4100970 msgr2=0x7fedd4102fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd1d9b640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedd4100970 0x7fedd4102fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd1d9b640 1 -- 172.21.15.33:0/3537576896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd41a6860 con 0x7fedd4101330 2024-04-02T13:44:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd259c640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedd4100970 0x7fedd4102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.421+0000 7fedd1d9b640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedd4101330 0x7fedd4103520 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fedc8005ab0 tx=0x7fedc8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.422+0000 7fedc37fe640 1 -- 172.21.15.33:0/3537576896 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8043020 con 0x7fedd4101330 2024-04-02T13:44:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.422+0000 7fedc37fe640 1 -- 172.21.15.33:0/3537576896 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fedc802c5d0 con 0x7fedd4101330 2024-04-02T13:44:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.422+0000 7fedd8c63640 1 -- 172.21.15.33:0/3537576896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd41a6af0 con 0x7fedd4101330 2024-04-02T13:44:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.422+0000 7fedc37fe640 1 -- 172.21.15.33:0/3537576896 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc80039f0 con 0x7fedd4101330 2024-04-02T13:44:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.422+0000 7fedd8c63640 1 -- 172.21.15.33:0/3537576896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd41a7090 con 0x7fedd4101330 2024-04-02T13:44:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.424+0000 7fedc37fe640 1 -- 172.21.15.33:0/3537576896 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fedc8046020 con 0x7fedd4101330 2024-04-02T13:44:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.424+0000 7feddb853740 1 Processor -- start 2024-04-02T13:44:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7feddb853740 1 -- start start 2024-04-02T13:44:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedc37fe640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fedbc0a3da0 0x7fedbc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7feddb853740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ac8d0 0x55dabc0accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7feddb853740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0ad1f0 0x55dabc0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7feddb853740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7feddb853740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dabc010830 con 0x55dabc0b5dd0 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7feddb853740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55dabbfb8820 con 0x55dabc0ac8d0 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7feddb853740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55dabc00c7e0 con 0x55dabc0ad1f0 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedd2d9d640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fedbc0a3da0 0x7fedbc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedc37fe640 1 -- 172.21.15.33:0/3537576896 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fedc80f2460 con 0x7fedd4101330 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedd2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedd259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ac8d0 0x55dabc0accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedd2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36142/0 (socket says 172.21.15.33:36142) 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedd259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ac8d0 0x55dabc0accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:36816/0 (socket says 172.21.15.33:36816) 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedd2d9d640 1 -- 172.21.15.33:0/708334925 learned_addr learned my addr 172.21.15.33:0/708334925 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedd1d9b640 1 --2- 172.21.15.33:0/708334925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0ad1f0 0x55dabc0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedd2d9d640 1 -- 172.21.15.33:0/708334925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0ad1f0 msgr2=0x55dabc0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedd2d9d640 1 --2- 172.21.15.33:0/708334925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0ad1f0 0x55dabc0b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedd2d9d640 1 -- 172.21.15.33:0/708334925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ac8d0 msgr2=0x55dabc0accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedd2d9d640 1 --2- 172.21.15.33:0/708334925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ac8d0 0x55dabc0accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedd2d9d640 1 -- 172.21.15.33:0/708334925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dabbdee7d0 con 0x55dabc0b5dd0 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.425+0000 7fedd259c640 1 --2- 172.21.15.33:0/708334925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ac8d0 0x55dabc0accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.426+0000 7fedd2d9d640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fedbc0a3da0 0x7fedbc0a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fedc4011d00 tx=0x7fedc4011440 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.426+0000 7fedd2d9d640 1 --2- 172.21.15.33:0/708334925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b81c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fedc4011670 tx=0x7fedc401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba0922807371254 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.426+0000 7fedc0ff9640 1 -- 172.21.15.33:0/708334925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4024070 con 0x55dabc0b5dd0 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.426+0000 7fedc0ff9640 1 -- 172.21.15.33:0/708334925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fedc4011df0 con 0x55dabc0b5dd0 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.426+0000 7fedc0ff9640 1 -- 172.21.15.33:0/708334925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc401fe10 con 0x55dabc0b5dd0 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.426+0000 7feddb853740 1 -- 172.21.15.33:0/708334925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 msgr2=0x55dabc0b81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.426+0000 7feddb853740 1 --2- 172.21.15.33:0/708334925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b81c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fedc4011670 tx=0x7fedc401d000 comp rx=0 tx=0).stop 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 -- 172.21.15.33:0/708334925 shutdown_connections 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 --2- 172.21.15.33:0/708334925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b81c0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 --2- 172.21.15.33:0/708334925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0ad1f0 0x55dabc0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 --2- 172.21.15.33:0/708334925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ac8d0 0x55dabc0accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 -- 172.21.15.33:0/708334925 >> 172.21.15.33:0/708334925 conn(0x55dabc0a7000 msgr2=0x55dabc0a73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 -- 172.21.15.33:0/708334925 shutdown_connections 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 -- 172.21.15.33:0/708334925 wait complete. 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 -- start start 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 Processor -- start 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ac8d0 0x55dabc0b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ad1f0 0x55dabc0b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dabc0b8b60 con 0x55dabc0ac8d0 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55dabbfcbd30 con 0x55dabc0ad1f0 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55dabc0b8ce0 con 0x55dabc0b5dd0 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7feddb853740 10 client.0 _ll_register_callbacks cb 0x55dabbfcb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7fedd259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ac8d0 0x55dabc0b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.427+0000 7fedd259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ac8d0 0x55dabc0b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36146/0 (socket says 172.21.15.33:36146) 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7fedd259c640 1 -- 172.21.15.33:0/963346320 learned_addr learned my addr 172.21.15.33:0/963346320 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7fedd1d9b640 1 --2- 172.21.15.33:0/963346320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ad1f0 0x55dabc0b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7fedd2d9d640 1 --2- 172.21.15.33:0/963346320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7fedd259c640 1 -- 172.21.15.33:0/963346320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0b5dd0 msgr2=0x55dabc0b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7fedd259c640 1 --2- 172.21.15.33:0/963346320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7fedd259c640 1 -- 172.21.15.33:0/963346320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ad1f0 msgr2=0x55dabc0b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7fedd259c640 1 --2- 172.21.15.33:0/963346320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ad1f0 0x55dabc0b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7fedd259c640 1 -- 172.21.15.33:0/963346320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dabbd243f0 con 0x55dabc0ac8d0 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7fedd259c640 1 --2- 172.21.15.33:0/963346320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ac8d0 0x55dabc0b56c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00bf80 tx=0x7fedcc0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877c4643c1e3b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7fedb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7fedb9ffb640 1 -- 172.21.15.33:0/963346320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc01f060 con 0x55dabc0ac8d0 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7fedb9ffb640 1 -- 172.21.15.33:0/963346320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fedcc0109c0 con 0x55dabc0ac8d0 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7feddb853740 1 -- 172.21.15.33:0/963346320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc4011df0 con 0x55dabc0ac8d0 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7fedb9ffb640 1 -- 172.21.15.33:0/963346320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fedcc026050 con 0x55dabc0ac8d0 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7feddb853740 10 client.16881 fetch_fsmap learned FSMap version 34 2024-04-02T13:44:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.428+0000 7feddb853740 1 -- 172.21.15.33:0/963346320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dabbce40f0 con 0x55dabc0ac8d0 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.429+0000 7fedb9ffb640 1 -- 172.21.15.33:0/963346320 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fedc4011df0 con 0x55dabc0ac8d0 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.429+0000 7fedb9ffb640 1 -- 172.21.15.33:0/963346320 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fedcc019030 con 0x55dabc0ac8d0 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.429+0000 7feddb853740 10 client.16881 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.429+0000 7feddb853740 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.429+0000 7feddb853740 20 client.16881 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.429+0000 7feddb853740 1 --2- 172.21.15.33:0/963346320 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dabc0c5d90 0x55dabc0c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.429+0000 7feddb853740 4 client.16881 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.429+0000 7feddb853740 1 -- 172.21.15.33:0/963346320 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dabc01dd70 con 0x55dabc0c5d90 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.429+0000 7fedd1d9b640 1 --2- 172.21.15.33:0/963346320 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dabc0c5d90 0x55dabc0c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.429+0000 7fedb9ffb640 10 client.16881 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.429+0000 7fedd1d9b640 1 --2- 172.21.15.33:0/963346320 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dabc0c5d90 0x55dabc0c81e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efbca018d276da46 server_cookie=4893880379406c8e in_seq=0 out_seq=0 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7fedb9ffb640 1 -- 172.21.15.33:0/963346320 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dabc01dd70 con 0x55dabc0c5d90 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7fedb9ffb640 10 client.16881 handle_command_reply: tid=0 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7feddb853740 1 client.16881 shutdown 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7feddb853740 1 -- 172.21.15.33:0/963346320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ac8d0 msgr2=0x55dabc0b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7feddb853740 1 --2- 172.21.15.33:0/963346320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ac8d0 0x55dabc0b56c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00bf80 tx=0x7fedcc0128e0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7feddb853740 1 -- 172.21.15.33:0/963346320 shutdown_connections 2024-04-02T13:44:49.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7feddb853740 1 --2- 172.21.15.33:0/963346320 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dabc0c5d90 0x55dabc0c81e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7feddb853740 1 --2- 172.21.15.33:0/963346320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7feddb853740 1 --2- 172.21.15.33:0/963346320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ad1f0 0x55dabc0b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7feddb853740 1 --2- 172.21.15.33:0/963346320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ac8d0 0x55dabc0b56c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7fedb9ffb640 0 client.16881 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:49.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7feddb853740 1 -- 172.21.15.33:0/963346320 >> 172.21.15.33:0/963346320 conn(0x55dabc0a7000 msgr2=0x55dabc0a8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7feddb853740 1 -- 172.21.15.33:0/963346320 shutdown_connections 2024-04-02T13:44:49.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.430+0000 7feddb853740 1 -- 172.21.15.33:0/963346320 wait complete. 2024-04-02T13:44:49.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.431+0000 7feddb853740 20 client.16881 trim_cache size 0 max 16384 2024-04-02T13:44:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.448+0000 7feddb853740 1 Processor -- start 2024-04-02T13:44:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.448+0000 7feddb853740 1 -- start start 2024-04-02T13:44:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.448+0000 7feddb853740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0ac8d0 0x55dabc0c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.448+0000 7feddb853740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ad1f0 0x55dabc0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.448+0000 7feddb853740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.448+0000 7feddb853740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b39a0 con 0x55dabc0b5dd0 2024-04-02T13:44:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.448+0000 7feddb853740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b1210 con 0x55dabc0ad1f0 2024-04-02T13:44:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.448+0000 7feddb853740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41af040 con 0x55dabc0ac8d0 2024-04-02T13:44:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.448+0000 7fedd2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.449+0000 7fedd2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36158/0 (socket says 172.21.15.33:36158) 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.449+0000 7fedd2d9d640 1 -- 172.21.15.33:0/1440563091 learned_addr learned my addr 172.21.15.33:0/1440563091 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.449+0000 7fedd1d9b640 1 --2- 172.21.15.33:0/1440563091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ad1f0 0x55dabc0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.449+0000 7fedd259c640 1 --2- 172.21.15.33:0/1440563091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0ac8d0 0x55dabc0c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.449+0000 7fedd2d9d640 1 -- 172.21.15.33:0/1440563091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0ac8d0 msgr2=0x55dabc0c1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.449+0000 7fedd2d9d640 1 --2- 172.21.15.33:0/1440563091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0ac8d0 0x55dabc0c1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.449+0000 7fedd2d9d640 1 -- 172.21.15.33:0/1440563091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ad1f0 msgr2=0x55dabc0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.449+0000 7fedd2d9d640 1 --2- 172.21.15.33:0/1440563091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ad1f0 0x55dabc0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.449+0000 7fedd2d9d640 1 -- 172.21.15.33:0/1440563091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dabc0a6e30 con 0x55dabc0b5dd0 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.449+0000 7fedd2d9d640 1 --2- 172.21.15.33:0/1440563091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b2c40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fedc4005f40 tx=0x7fedc400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f82d7230fe9bcbd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.449+0000 7fedbbfff640 1 -- 172.21.15.33:0/1440563091 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4024040 con 0x55dabc0b5dd0 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.449+0000 7fedbbfff640 1 -- 172.21.15.33:0/1440563091 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fedc40117c0 con 0x55dabc0b5dd0 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.449+0000 7fedbbfff640 1 -- 172.21.15.33:0/1440563091 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4003d70 con 0x55dabc0b5dd0 2024-04-02T13:44:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 -- 172.21.15.33:0/1440563091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 msgr2=0x55dabc0b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 --2- 172.21.15.33:0/1440563091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b2c40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fedc4005f40 tx=0x7fedc400ff70 comp rx=0 tx=0).stop 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 -- 172.21.15.33:0/1440563091 shutdown_connections 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 --2- 172.21.15.33:0/1440563091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0b5dd0 0x55dabc0b2c40 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 --2- 172.21.15.33:0/1440563091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ad1f0 0x55dabc0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 --2- 172.21.15.33:0/1440563091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0ac8d0 0x55dabc0c1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 -- 172.21.15.33:0/1440563091 >> 172.21.15.33:0/1440563091 conn(0x55dabc0a7000 msgr2=0x55dabc0a8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 -- 172.21.15.33:0/1440563091 shutdown_connections 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 -- 172.21.15.33:0/1440563091 wait complete. 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 -- start start 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 Processor -- start 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ac8d0 0x55dabc0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ad1f0 0x55dabc0affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0b5dd0 0x55dabc12b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41af6d0 con 0x55dabc0ad1f0 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b0b40 con 0x55dabc0ac8d0 2024-04-02T13:44:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41afda0 con 0x55dabc0b5dd0 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.450+0000 7feddb853740 10 client.0 _ll_register_callbacks cb 0x7fedcc00bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedd1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ad1f0 0x55dabc0affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedd1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ad1f0 0x55dabc0affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36162/0 (socket says 172.21.15.33:36162) 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedd1d9b640 1 -- 172.21.15.33:0/4211805461 learned_addr learned my addr 172.21.15.33:0/4211805461 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedd259c640 1 --2- 172.21.15.33:0/4211805461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ac8d0 0x55dabc0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedd2d9d640 1 --2- 172.21.15.33:0/4211805461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0b5dd0 0x55dabc12b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedd1d9b640 1 -- 172.21.15.33:0/4211805461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0b5dd0 msgr2=0x55dabc12b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedd1d9b640 1 --2- 172.21.15.33:0/4211805461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0b5dd0 0x55dabc12b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedd1d9b640 1 -- 172.21.15.33:0/4211805461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ac8d0 msgr2=0x55dabc0afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedd1d9b640 1 --2- 172.21.15.33:0/4211805461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ac8d0 0x55dabc0afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedd1d9b640 1 -- 172.21.15.33:0/4211805461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dabc0b81c0 con 0x55dabc0ad1f0 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedd1d9b640 1 --2- 172.21.15.33:0/4211805461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ad1f0 0x55dabc0affd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fedc8004fd0 tx=0x7fedc8005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9135a864718cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedb9ffb640 1 -- 172.21.15.33:0/4211805461 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc80be160 con 0x55dabc0ad1f0 2024-04-02T13:44:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedb9ffb640 1 -- 172.21.15.33:0/4211805461 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fedc802cb10 con 0x55dabc0ad1f0 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7feddb853740 1 -- 172.21.15.33:0/4211805461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dabc0180c0 con 0x55dabc0ad1f0 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedb9ffb640 1 -- 172.21.15.33:0/4211805461 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fedc80c2510 con 0x55dabc0ad1f0 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7feddb853740 10 client.16887 fetch_fsmap learned FSMap version 34 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7feddb853740 1 -- 172.21.15.33:0/4211805461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dabc12c7c0 con 0x55dabc0ad1f0 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.451+0000 7fedb9ffb640 1 -- 172.21.15.33:0/4211805461 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55dabc0180c0 con 0x55dabc0ad1f0 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.452+0000 7fedb9ffb640 1 -- 172.21.15.33:0/4211805461 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fedc8098ce0 con 0x55dabc0ad1f0 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.452+0000 7feddb853740 10 client.16887 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.452+0000 7feddb853740 10 client.16887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.452+0000 7feddb853740 20 client.16887 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.452+0000 7feddb853740 1 --2- 172.21.15.33:0/4211805461 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dabc12cef0 0x55dabc12f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.452+0000 7feddb853740 4 client.16887 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.452+0000 7feddb853740 1 -- 172.21.15.33:0/4211805461 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dabbf95560 con 0x55dabc12cef0 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.452+0000 7fedd259c640 1 --2- 172.21.15.33:0/4211805461 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dabc12cef0 0x55dabc12f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.452+0000 7fedd259c640 1 --2- 172.21.15.33:0/4211805461 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dabc12cef0 0x55dabc12f310 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9da4ed18f735c99 server_cookie=67374998a0a601b3 in_seq=0 out_seq=0 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.452+0000 7fedb9ffb640 10 client.16887 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.452+0000 7fedb9ffb640 1 -- 172.21.15.33:0/4211805461 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dabbf95560 con 0x55dabc12cef0 2024-04-02T13:44:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.452+0000 7fedb9ffb640 10 client.16887 handle_command_reply: tid=0 2024-04-02T13:44:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.452+0000 7feddb853740 1 client.16887 shutdown 2024-04-02T13:44:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7feddb853740 1 -- 172.21.15.33:0/4211805461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ad1f0 msgr2=0x55dabc0affd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7feddb853740 1 --2- 172.21.15.33:0/4211805461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ad1f0 0x55dabc0affd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fedc8004fd0 tx=0x7fedc8005890 comp rx=0 tx=0).stop 2024-04-02T13:44:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7feddb853740 1 -- 172.21.15.33:0/4211805461 shutdown_connections 2024-04-02T13:44:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7feddb853740 1 --2- 172.21.15.33:0/4211805461 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55dabc12cef0 0x55dabc12f310 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7feddb853740 1 --2- 172.21.15.33:0/4211805461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55dabc0b5dd0 0x55dabc12b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7fedb9ffb640 0 client.16887 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7feddb853740 1 --2- 172.21.15.33:0/4211805461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dabc0ad1f0 0x55dabc0affd0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7feddb853740 1 --2- 172.21.15.33:0/4211805461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dabc0ac8d0 0x55dabc0afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7feddb853740 1 -- 172.21.15.33:0/4211805461 >> 172.21.15.33:0/4211805461 conn(0x55dabc0a7000 msgr2=0x55dabc0ad710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7feddb853740 1 -- 172.21.15.33:0/4211805461 shutdown_connections 2024-04-02T13:44:49.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7feddb853740 1 -- 172.21.15.33:0/4211805461 wait complete. 2024-04-02T13:44:49.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7feddb853740 20 client.16887 trim_cache size 0 max 16384 2024-04-02T13:44:49.463 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:44:49.463 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:44:49.463 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:44:49.463 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:44:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7fedd8c63640 1 -- 172.21.15.33:0/3537576896 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fedbc0a3da0 msgr2=0x7fedbc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7fedd8c63640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fedbc0a3da0 0x7fedbc0a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fedc4011d00 tx=0x7fedc4011440 comp rx=0 tx=0).stop 2024-04-02T13:44:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7fedd8c63640 1 -- 172.21.15.33:0/3537576896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedd4101330 msgr2=0x7fedd4103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7fedd8c63640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedd4101330 0x7fedd4103520 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fedc8005ab0 tx=0x7fedc8004600 comp rx=0 tx=0).stop 2024-04-02T13:44:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7fedd8c63640 1 -- 172.21.15.33:0/3537576896 shutdown_connections 2024-04-02T13:44:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7fedd8c63640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fedbc0a3da0 0x7fedbc0a6260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7fedd8c63640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fedd4106980 0x7fedd4103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7fedd8c63640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedd4101330 0x7fedd4103520 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7fedd8c63640 1 --2- 172.21.15.33:0/3537576896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fedd4100970 0x7fedd4102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7fedd8c63640 1 -- 172.21.15.33:0/3537576896 >> 172.21.15.33:0/3537576896 conn(0x7fedd4077b00 msgr2=0x7fedd4068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7fedd8c63640 1 -- 172.21.15.33:0/3537576896 shutdown_connections 2024-04-02T13:44:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.453+0000 7fedd8c63640 1 -- 172.21.15.33:0/3537576896 wait complete. 2024-04-02T13:44:49.468 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:44:49.468 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:44:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.569+0000 7fe9b1baf640 1 -- 172.21.15.33:0/1634227298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9ac10c130 msgr2=0x7fe9ac10e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.569+0000 7fe9b1baf640 1 --2- 172.21.15.33:0/1634227298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9ac10c130 0x7fe9ac10e560 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009ca0 tx=0x7fe9a402c180 comp rx=0 tx=0).stop 2024-04-02T13:44:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.569+0000 7fe9b1baf640 1 -- 172.21.15.33:0/1634227298 shutdown_connections 2024-04-02T13:44:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.569+0000 7fe9b1baf640 1 --2- 172.21.15.33:0/1634227298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9ac10c130 0x7fe9ac10e560 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.569+0000 7fe9b1baf640 1 --2- 172.21.15.33:0/1634227298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ac103680 0x7fe9ac10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.569+0000 7fe9b1baf640 1 --2- 172.21.15.33:0/1634227298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9ac102d60 0x7fe9ac103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.569+0000 7fe9b1baf640 1 -- 172.21.15.33:0/1634227298 >> 172.21.15.33:0/1634227298 conn(0x7fe9ac076d80 msgr2=0x7fe9ac077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.569+0000 7fe9b1baf640 1 -- 172.21.15.33:0/1634227298 shutdown_connections 2024-04-02T13:44:49.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.569+0000 7fe9b1baf640 1 -- 172.21.15.33:0/1634227298 wait complete. 2024-04-02T13:44:49.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9b1baf640 1 Processor -- start 2024-04-02T13:44:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9b1baf640 1 -- start start 2024-04-02T13:44:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9b1baf640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ac102d60 0x7fe9ac198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9b1baf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9ac103680 0x7fe9ac1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9b1baf640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9ac10c130 0x7fe9ac19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9b1baf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac111ef0 con 0x7fe9ac103680 2024-04-02T13:44:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9b1baf640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac111d70 con 0x7fe9ac102d60 2024-04-02T13:44:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9b1baf640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac112070 con 0x7fe9ac10c130 2024-04-02T13:44:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9ac103680 0x7fe9ac1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9ac103680 0x7fe9ac1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36180/0 (socket says 172.21.15.33:36180) 2024-04-02T13:44:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9aaffd640 1 -- 172.21.15.33:0/3197936786 learned_addr learned my addr 172.21.15.33:0/3197936786 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9abfff640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9ac10c130 0x7fe9ac19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9ab7fe640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ac102d60 0x7fe9ac198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9aaffd640 1 -- 172.21.15.33:0/3197936786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9ac10c130 msgr2=0x7fe9ac19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9aaffd640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9ac10c130 0x7fe9ac19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9aaffd640 1 -- 172.21.15.33:0/3197936786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ac102d60 msgr2=0x7fe9ac198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9aaffd640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ac102d60 0x7fe9ac198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.570+0000 7fe9aaffd640 1 -- 172.21.15.33:0/3197936786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac19d410 con 0x7fe9ac103680 2024-04-02T13:44:49.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.571+0000 7fe9aaffd640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9ac103680 0x7fe9ac1989b0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000d950 tx=0x7fe9a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.571+0000 7fe9a8ff9640 1 -- 172.21.15.33:0/3197936786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0017030 con 0x7fe9ac103680 2024-04-02T13:44:49.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.571+0000 7fe9b1baf640 1 -- 172.21.15.33:0/3197936786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac19d700 con 0x7fe9ac103680 2024-04-02T13:44:49.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.571+0000 7fe9a8ff9640 1 -- 172.21.15.33:0/3197936786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe9a0004020 con 0x7fe9ac103680 2024-04-02T13:44:49.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.571+0000 7fe9a8ff9640 1 -- 172.21.15.33:0/3197936786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a00047a0 con 0x7fe9ac103680 2024-04-02T13:44:49.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.571+0000 7fe9b1baf640 1 -- 172.21.15.33:0/3197936786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac1a4f90 con 0x7fe9ac103680 2024-04-02T13:44:49.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.572+0000 7fe9b1baf640 1 -- 172.21.15.33:0/3197936786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ac068d40 con 0x7fe9ac103680 2024-04-02T13:44:49.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.573+0000 7fe9a8ff9640 1 -- 172.21.15.33:0/3197936786 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe9a00041d0 con 0x7fe9ac103680 2024-04-02T13:44:49.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.573+0000 7fe9a8ff9640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe9940a3d20 0x7fe9940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.574+0000 7fe9ab7fe640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe9940a3d20 0x7fe9940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.574+0000 7fe9a8ff9640 1 -- 172.21.15.33:0/3197936786 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe9a00d06c0 con 0x7fe9ac103680 2024-04-02T13:44:49.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.574+0000 7fe9ab7fe640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe9940a3d20 0x7fe9940a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00a7f0 tx=0x7fe99c008040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.578+0000 7fe9a8ff9640 1 -- 172.21.15.33:0/3197936786 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe9a009c9b0 con 0x7fe9ac103680 2024-04-02T13:44:49.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.806+0000 7fe9b1baf640 1 -- 172.21.15.33:0/3197936786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9ac1a5220 con 0x7fe9ac103680 2024-04-02T13:44:49.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.807+0000 7fe9a8ff9640 1 -- 172.21.15.33:0/3197936786 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11874 (secure 0 0 0) 0x7fe9a0096e20 con 0x7fe9ac103680 2024-04-02T13:44:49.811 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:44:49.811 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:44:49.227025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24512,24551,24473]},"id":1}]} 2024-04-02T13:44:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2024-04-02T13:44:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.809+0000 7fe9b1baf640 1 -- 172.21.15.33:0/3197936786 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe9940a3d20 msgr2=0x7fe9940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.809+0000 7fe9b1baf640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe9940a3d20 0x7fe9940a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00a7f0 tx=0x7fe99c008040 comp rx=0 tx=0).stop 2024-04-02T13:44:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.809+0000 7fe9b1baf640 1 -- 172.21.15.33:0/3197936786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9ac103680 msgr2=0x7fe9ac1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.809+0000 7fe9b1baf640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9ac103680 0x7fe9ac1989b0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000d950 tx=0x7fe9a000de10 comp rx=0 tx=0).stop 2024-04-02T13:44:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.809+0000 7fe9b1baf640 1 -- 172.21.15.33:0/3197936786 shutdown_connections 2024-04-02T13:44:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.809+0000 7fe9b1baf640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe9940a3d20 0x7fe9940a61e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.809+0000 7fe9b1baf640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9ac10c130 0x7fe9ac19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.809+0000 7fe9b1baf640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9ac103680 0x7fe9ac1989b0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.809+0000 7fe9b1baf640 1 --2- 172.21.15.33:0/3197936786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9ac102d60 0x7fe9ac198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.809+0000 7fe9b1baf640 1 -- 172.21.15.33:0/3197936786 >> 172.21.15.33:0/3197936786 conn(0x7fe9ac076d80 msgr2=0x7fe9ac0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.809+0000 7fe9b1baf640 1 -- 172.21.15.33:0/3197936786 shutdown_connections 2024-04-02T13:44:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.809+0000 7fe9b1baf640 1 -- 172.21.15.33:0/3197936786 wait complete. 2024-04-02T13:44:49.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:44:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.925+0000 7f99ad82a640 1 -- 172.21.15.33:0/3605957800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a810c110 msgr2=0x7f99a810e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.925+0000 7f99ad82a640 1 --2- 172.21.15.33:0/3605957800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a810c110 0x7f99a810e540 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009ca0 tx=0x7f99a002c180 comp rx=0 tx=0).stop 2024-04-02T13:44:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.925+0000 7f99ad82a640 1 -- 172.21.15.33:0/3605957800 shutdown_connections 2024-04-02T13:44:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.925+0000 7f99ad82a640 1 --2- 172.21.15.33:0/3605957800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a810c110 0x7f99a810e540 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.925+0000 7f99ad82a640 1 --2- 172.21.15.33:0/3605957800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99a8103660 0x7f99a810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.925+0000 7f99ad82a640 1 --2- 172.21.15.33:0/3605957800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a8102d40 0x7f99a8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.925+0000 7f99ad82a640 1 -- 172.21.15.33:0/3605957800 >> 172.21.15.33:0/3605957800 conn(0x7f99a8076d80 msgr2=0x7f99a8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.925+0000 7f99ad82a640 1 -- 172.21.15.33:0/3605957800 shutdown_connections 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.925+0000 7f99ad82a640 1 -- 172.21.15.33:0/3605957800 wait complete. 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99ad82a640 1 Processor -- start 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99ad82a640 1 -- start start 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99ad82a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99a8102d40 0x7f99a8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99ad82a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a8103660 0x7f99a8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99ad82a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a810c110 0x7f99a8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99ad82a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8111e70 con 0x7f99a810c110 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99ad82a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8111cf0 con 0x7f99a8103660 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99ad82a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8111ff0 con 0x7f99a8102d40 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a810c110 0x7f99a8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a810c110 0x7f99a8101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36190/0 (socket says 172.21.15.33:36190) 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99a77fe640 1 -- 172.21.15.33:0/2198759593 learned_addr learned my addr 172.21.15.33:0/2198759593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99a6ffd640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99a8102d40 0x7f99a8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f999ffff640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a8103660 0x7f99a8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99a77fe640 1 -- 172.21.15.33:0/2198759593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99a8102d40 msgr2=0x7f99a8100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99a77fe640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99a8102d40 0x7f99a8100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99a77fe640 1 -- 172.21.15.33:0/2198759593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a8103660 msgr2=0x7f99a8100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99a77fe640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a8103660 0x7f99a8100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.926+0000 7f99a77fe640 1 -- 172.21.15.33:0/2198759593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a81a6890 con 0x7f99a810c110 2024-04-02T13:44:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.927+0000 7f99a77fe640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a810c110 0x7f99a8101410 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f99a0004af0 tx=0x7f99a0034ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.927+0000 7f99ac828640 1 -- 172.21.15.33:0/2198759593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0039050 con 0x7f99a810c110 2024-04-02T13:44:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.927+0000 7f99ac828640 1 -- 172.21.15.33:0/2198759593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f99a0045070 con 0x7f99a810c110 2024-04-02T13:44:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.927+0000 7f99ad82a640 1 -- 172.21.15.33:0/2198759593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a81a6b20 con 0x7f99a810c110 2024-04-02T13:44:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.927+0000 7f99ac828640 1 -- 172.21.15.33:0/2198759593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0032b70 con 0x7f99a810c110 2024-04-02T13:44:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.927+0000 7f99ad82a640 1 -- 172.21.15.33:0/2198759593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a81a7060 con 0x7f99a810c110 2024-04-02T13:44:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.929+0000 7f99ac828640 1 -- 172.21.15.33:0/2198759593 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f99a0032d10 con 0x7f99a810c110 2024-04-02T13:44:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.929+0000 7f99b0413740 1 Processor -- start 2024-04-02T13:44:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.929+0000 7f99b0413740 1 -- start start 2024-04-02T13:44:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.929+0000 7f99b0413740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b838db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.929+0000 7f99b0413740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b838e0c0 0x5637b8396760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.929+0000 7f99b0413740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b8396ca0 0x5637b8399090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.929+0000 7f99b0413740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5637b82f17b0 con 0x5637b838d7a0 2024-04-02T13:44:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.929+0000 7f99b0413740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5637b8290570 con 0x5637b8396ca0 2024-04-02T13:44:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.929+0000 7f99b0413740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5637b82ed760 con 0x5637b838e0c0 2024-04-02T13:44:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f999ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b838db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f999ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b838db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36196/0 (socket says 172.21.15.33:36196) 2024-04-02T13:44:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f999ffff640 1 -- 172.21.15.33:0/3427886969 learned_addr learned my addr 172.21.15.33:0/3427886969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f99a6ffd640 1 --2- 172.21.15.33:0/3427886969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b838e0c0 0x5637b8396760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f99ac828640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f99940a3da0 0x7f99940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f999ffff640 1 --2- 172.21.15.33:0/3427886969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b8396ca0 0x5637b8399090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f99a77fe640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f99940a3da0 0x7f99940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f999ffff640 1 -- 172.21.15.33:0/3427886969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b838e0c0 msgr2=0x5637b8396760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f999ffff640 1 --2- 172.21.15.33:0/3427886969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b838e0c0 0x5637b8396760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f999ffff640 1 -- 172.21.15.33:0/3427886969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b8396ca0 msgr2=0x5637b8399090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f999ffff640 1 --2- 172.21.15.33:0/3427886969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b8396ca0 0x5637b8399090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f999ffff640 1 -- 172.21.15.33:0/3427886969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b80053f0 con 0x5637b838d7a0 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f99ac828640 1 -- 172.21.15.33:0/2198759593 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f99a00f3390 con 0x7f99a810c110 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f999ffff640 1 --2- 172.21.15.33:0/3427886969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b838db80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f999000bb40 tx=0x7f99900041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23094c62e2c72335 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f999dffb640 1 -- 172.21.15.33:0/3427886969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990012480 con 0x5637b838d7a0 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f99a77fe640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f99940a3da0 0x7f99940a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f99a00bed90 tx=0x7f99a008da40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f999dffb640 1 -- 172.21.15.33:0/3427886969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9990012af0 con 0x5637b838d7a0 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.930+0000 7f999dffb640 1 -- 172.21.15.33:0/3427886969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99900109b0 con 0x5637b838d7a0 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.931+0000 7f99b0413740 1 -- 172.21.15.33:0/3427886969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 msgr2=0x5637b838db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.931+0000 7f99b0413740 1 --2- 172.21.15.33:0/3427886969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b838db80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f999000bb40 tx=0x7f99900041d0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.931+0000 7f99b0413740 1 -- 172.21.15.33:0/3427886969 shutdown_connections 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.931+0000 7f99b0413740 1 --2- 172.21.15.33:0/3427886969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b8396ca0 0x5637b8399090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.931+0000 7f99b0413740 1 --2- 172.21.15.33:0/3427886969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b838e0c0 0x5637b8396760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.931+0000 7f99b0413740 1 --2- 172.21.15.33:0/3427886969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b838db80 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.931+0000 7f99b0413740 1 -- 172.21.15.33:0/3427886969 >> 172.21.15.33:0/3427886969 conn(0x5637b8387ed0 msgr2=0x5637b83882b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.931+0000 7f99b0413740 1 -- 172.21.15.33:0/3427886969 shutdown_connections 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.931+0000 7f99b0413740 1 -- 172.21.15.33:0/3427886969 wait complete. 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.931+0000 7f99b0413740 1 -- start start 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f99b0413740 1 Processor -- start 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f99b0413740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b83966b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f99b0413740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838e0c0 0x5637b8393a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f99b0413740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b8396ca0 0x5637b8394050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f99b0413740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5637b8399a30 con 0x5637b838d7a0 2024-04-02T13:44:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f99b0413740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5637b82ace80 con 0x5637b838e0c0 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f99b0413740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5637b8399bb0 con 0x5637b8396ca0 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f99b0413740 10 client.0 _ll_register_callbacks cb 0x5637b82e8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f999ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b83966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f999ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b83966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36208/0 (socket says 172.21.15.33:36208) 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f999ffff640 1 -- 172.21.15.33:0/1271604555 learned_addr learned my addr 172.21.15.33:0/1271604555 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f999ffff640 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b8396ca0 0x5637b8394050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f99a6ffd640 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838e0c0 0x5637b8393a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f999ffff640 1 -- 172.21.15.33:0/1271604555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b8396ca0 msgr2=0x5637b8394050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f999ffff640 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b8396ca0 0x5637b8394050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f999ffff640 1 -- 172.21.15.33:0/1271604555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838e0c0 msgr2=0x5637b8393a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f999ffff640 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838e0c0 0x5637b8393a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f999ffff640 1 -- 172.21.15.33:0/1271604555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b80cf7d0 con 0x5637b838d7a0 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f99a6ffd640 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838e0c0 0x5637b8393a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f999ffff640 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b83966b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f999000b3f0 tx=0x7f9990005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5151c02058ecf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f9986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f9986ffd640 1 -- 172.21.15.33:0/1271604555 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990023070 con 0x5637b838d7a0 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f9986ffd640 1 -- 172.21.15.33:0/1271604555 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9990010750 con 0x5637b838d7a0 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.932+0000 7f99b0413740 1 -- 172.21.15.33:0/1271604555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9990012af0 con 0x5637b838d7a0 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.933+0000 7f9986ffd640 1 -- 172.21.15.33:0/1271604555 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9990012950 con 0x5637b838d7a0 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.933+0000 7f99b0413740 10 client.16905 fetch_fsmap learned FSMap version 35 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.933+0000 7f99b0413740 1 -- 172.21.15.33:0/1271604555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5637b83a5a90 con 0x5637b838d7a0 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.933+0000 7f9986ffd640 1 -- 172.21.15.33:0/1271604555 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9990012af0 con 0x5637b838d7a0 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.933+0000 7f9986ffd640 1 -- 172.21.15.33:0/1271604555 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f999001d070 con 0x5637b838d7a0 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.933+0000 7f99b0413740 10 client.16905 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.933+0000 7f99b0413740 10 client.16905 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.933+0000 7f99b0413740 20 client.16905 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.933+0000 7f99b0413740 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5637b83a6de0 0x5637b83a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.933+0000 7f99b0413740 4 client.16905 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.933+0000 7f99b0413740 1 -- 172.21.15.33:0/1271604555 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637b82fecf0 con 0x5637b83a6de0 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.933+0000 7f99a6ffd640 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5637b83a6de0 0x5637b83a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.934+0000 7f99a6ffd640 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5637b83a6de0 0x5637b83a91d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfc49df92baf7a95 server_cookie=88b1a899eca3e246 in_seq=0 out_seq=0 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.934+0000 7f9986ffd640 10 client.16905 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.934+0000 7f9986ffd640 1 -- 172.21.15.33:0/1271604555 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637b82fecf0 con 0x5637b83a6de0 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.934+0000 7f9986ffd640 10 client.16905 handle_command_reply: tid=0 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.934+0000 7f99b0413740 1 client.16905 shutdown 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.935+0000 7f99b0413740 1 -- 172.21.15.33:0/1271604555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 msgr2=0x5637b83966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.935+0000 7f99b0413740 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b83966b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f999000b3f0 tx=0x7f9990005f30 comp rx=0 tx=0).stop 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.935+0000 7f99b0413740 1 -- 172.21.15.33:0/1271604555 shutdown_connections 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.935+0000 7f99b0413740 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5637b83a6de0 0x5637b83a91d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.935+0000 7f99b0413740 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b8396ca0 0x5637b8394050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.935+0000 7f99b0413740 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838e0c0 0x5637b8393a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.935+0000 7f9986ffd640 0 client.16905 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.935+0000 7f99b0413740 1 --2- 172.21.15.33:0/1271604555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b83966b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.935+0000 7f99b0413740 1 -- 172.21.15.33:0/1271604555 >> 172.21.15.33:0/1271604555 conn(0x5637b8387ed0 msgr2=0x5637b8389c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.935+0000 7f99b0413740 1 -- 172.21.15.33:0/1271604555 shutdown_connections 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.935+0000 7f99b0413740 1 -- 172.21.15.33:0/1271604555 wait complete. 2024-04-02T13:44:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.935+0000 7f99b0413740 20 client.16905 trim_cache size 0 max 16384 2024-04-02T13:44:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.948+0000 7f99b0413740 1 Processor -- start 2024-04-02T13:44:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.948+0000 7f99b0413740 1 -- start start 2024-04-02T13:44:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.948+0000 7f99b0413740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838d7a0 0x5637b838e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.948+0000 7f99b0413740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b838e0c0 0x5637b83a2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.948+0000 7f99b0413740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b8396ca0 0x5637b83882b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.948+0000 7f99b0413740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99a81b39e0 con 0x5637b8396ca0 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.948+0000 7f99b0413740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f99a81b1280 con 0x5637b838d7a0 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.948+0000 7f99b0413740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99a81af0b0 con 0x5637b838e0c0 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.948+0000 7f999ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b8396ca0 0x5637b83882b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.948+0000 7f999ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b8396ca0 0x5637b83882b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36220/0 (socket says 172.21.15.33:36220) 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.948+0000 7f999ffff640 1 -- 172.21.15.33:0/1995857620 learned_addr learned my addr 172.21.15.33:0/1995857620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.948+0000 7f99a6ffd640 1 --2- 172.21.15.33:0/1995857620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b838e0c0 0x5637b83a2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.948+0000 7f999ffff640 1 --2- 172.21.15.33:0/1995857620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838d7a0 0x5637b838e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f999ffff640 1 -- 172.21.15.33:0/1995857620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b838e0c0 msgr2=0x5637b83a2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f999ffff640 1 --2- 172.21.15.33:0/1995857620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b838e0c0 0x5637b83a2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f999ffff640 1 -- 172.21.15.33:0/1995857620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838d7a0 msgr2=0x5637b838e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f999ffff640 1 --2- 172.21.15.33:0/1995857620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838d7a0 0x5637b838e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f999ffff640 1 -- 172.21.15.33:0/1995857620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b83943c0 con 0x5637b8396ca0 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f999ffff640 1 --2- 172.21.15.33:0/1995857620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b8396ca0 0x5637b83882b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f999000b520 tx=0x7f999001ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbea4bd336b12fd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f999d7fa640 1 -- 172.21.15.33:0/1995857620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990023040 con 0x5637b8396ca0 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f999d7fa640 1 -- 172.21.15.33:0/1995857620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9990012af0 con 0x5637b8396ca0 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f999d7fa640 1 -- 172.21.15.33:0/1995857620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990005290 con 0x5637b8396ca0 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f99b0413740 1 -- 172.21.15.33:0/1995857620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b8396ca0 msgr2=0x5637b83882b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f99b0413740 1 --2- 172.21.15.33:0/1995857620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b8396ca0 0x5637b83882b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f999000b520 tx=0x7f999001ef50 comp rx=0 tx=0).stop 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f99b0413740 1 -- 172.21.15.33:0/1995857620 shutdown_connections 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f99b0413740 1 --2- 172.21.15.33:0/1995857620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b8396ca0 0x5637b83882b0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f99b0413740 1 --2- 172.21.15.33:0/1995857620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b838e0c0 0x5637b83a2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f99b0413740 1 --2- 172.21.15.33:0/1995857620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838d7a0 0x5637b838e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f99b0413740 1 -- 172.21.15.33:0/1995857620 >> 172.21.15.33:0/1995857620 conn(0x5637b8387ed0 msgr2=0x5637b83932d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f99b0413740 1 -- 172.21.15.33:0/1995857620 shutdown_connections 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.949+0000 7f99b0413740 1 -- 172.21.15.33:0/1995857620 wait complete. 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f99b0413740 1 -- start start 2024-04-02T13:44:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f99b0413740 1 Processor -- start 2024-04-02T13:44:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f99b0413740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b839ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f99b0413740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838e0c0 0x5637b839f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f99b0413740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b8396ca0 0x5637b8390d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f99b0413740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99a81af740 con 0x5637b838d7a0 2024-04-02T13:44:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f99b0413740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f99a81b0bb0 con 0x5637b838e0c0 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f99b0413740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99a81afe10 con 0x5637b8396ca0 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f99b0413740 10 client.0 _ll_register_callbacks cb 0x7f999800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f999ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b839ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f999ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b839ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36234/0 (socket says 172.21.15.33:36234) 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f999ffff640 1 -- 172.21.15.33:0/880237547 learned_addr learned my addr 172.21.15.33:0/880237547 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f99a6ffd640 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838e0c0 0x5637b839f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.950+0000 7f999ffff640 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b8396ca0 0x5637b8390d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f999ffff640 1 -- 172.21.15.33:0/880237547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b8396ca0 msgr2=0x5637b8390d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f999ffff640 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b8396ca0 0x5637b8390d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f999ffff640 1 -- 172.21.15.33:0/880237547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838e0c0 msgr2=0x5637b839f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f999ffff640 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838e0c0 0x5637b839f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f999ffff640 1 -- 172.21.15.33:0/880237547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b83a9540 con 0x5637b838d7a0 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f99a6ffd640 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838e0c0 0x5637b839f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f999ffff640 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b839ee30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9990014010 tx=0x7f9990005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62ea7a529494078 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f9986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f9986ffd640 1 -- 172.21.15.33:0/880237547 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999003f020 con 0x5637b838d7a0 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f9986ffd640 1 -- 172.21.15.33:0/880237547 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9990005040 con 0x5637b838d7a0 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f99b0413740 1 -- 172.21.15.33:0/880237547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9990012af0 con 0x5637b838d7a0 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f9986ffd640 1 -- 172.21.15.33:0/880237547 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9990023070 con 0x5637b838d7a0 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f99b0413740 10 client.16911 fetch_fsmap learned FSMap version 35 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f99b0413740 1 -- 172.21.15.33:0/880237547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5637b8397e10 con 0x5637b838d7a0 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f9986ffd640 1 -- 172.21.15.33:0/880237547 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9990012af0 con 0x5637b838d7a0 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f9986ffd640 1 -- 172.21.15.33:0/880237547 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f999001d070 con 0x5637b838d7a0 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f99b0413740 10 client.16911 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f99b0413740 10 client.16911 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f99b0413740 20 client.16911 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f99b0413740 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5637b840d240 0x5637b840f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f99b0413740 4 client.16911 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.951+0000 7f99b0413740 1 -- 172.21.15.33:0/880237547 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637b82f9040 con 0x5637b840d240 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.952+0000 7f99a6ffd640 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5637b840d240 0x5637b840f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.952+0000 7f99a6ffd640 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5637b840d240 0x5637b840f630 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c5602e71c594dc41 server_cookie=7bc1a48a5d2e36b9 in_seq=0 out_seq=0 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.952+0000 7f9986ffd640 10 client.16911 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.952+0000 7f9986ffd640 1 -- 172.21.15.33:0/880237547 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637b82f9040 con 0x5637b840d240 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.952+0000 7f9986ffd640 10 client.16911 handle_command_reply: tid=0 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.952+0000 7f99b0413740 1 client.16911 shutdown 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99b0413740 1 -- 172.21.15.33:0/880237547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 msgr2=0x5637b839ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99b0413740 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b839ee30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9990014010 tx=0x7f9990005f30 comp rx=0 tx=0).stop 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99b0413740 1 -- 172.21.15.33:0/880237547 shutdown_connections 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99b0413740 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5637b840d240 0x5637b840f630 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99b0413740 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5637b8396ca0 0x5637b8390d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99b0413740 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5637b838e0c0 0x5637b839f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f9986ffd640 0 client.16911 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:44:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99b0413740 1 --2- 172.21.15.33:0/880237547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637b838d7a0 0x5637b839ee30 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99b0413740 1 -- 172.21.15.33:0/880237547 >> 172.21.15.33:0/880237547 conn(0x5637b8387ed0 msgr2=0x5637b83882b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99b0413740 1 -- 172.21.15.33:0/880237547 shutdown_connections 2024-04-02T13:44:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99b0413740 1 -- 172.21.15.33:0/880237547 wait complete. 2024-04-02T13:44:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99b0413740 20 client.16911 trim_cache size 0 max 16384 2024-04-02T13:44:49.963 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:44:49.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99ad82a640 1 -- 172.21.15.33:0/2198759593 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f99940a3da0 msgr2=0x7f99940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99ad82a640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f99940a3da0 0x7f99940a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f99a00bed90 tx=0x7f99a008da40 comp rx=0 tx=0).stop 2024-04-02T13:44:49.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99ad82a640 1 -- 172.21.15.33:0/2198759593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a810c110 msgr2=0x7f99a8101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:49.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99ad82a640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a810c110 0x7f99a8101410 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f99a0004af0 tx=0x7f99a0034ed0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99ad82a640 1 -- 172.21.15.33:0/2198759593 shutdown_connections 2024-04-02T13:44:49.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99ad82a640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f99940a3da0 0x7f99940a6260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99ad82a640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a810c110 0x7f99a8101410 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99ad82a640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a8103660 0x7f99a8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99ad82a640 1 --2- 172.21.15.33:0/2198759593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99a8102d40 0x7f99a8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:49.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99ad82a640 1 -- 172.21.15.33:0/2198759593 >> 172.21.15.33:0/2198759593 conn(0x7f99a8076d80 msgr2=0x7f99a80fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:49.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.953+0000 7f99ad82a640 1 -- 172.21.15.33:0/2198759593 shutdown_connections 2024-04-02T13:44:49.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:49.954+0000 7f99ad82a640 1 -- 172.21.15.33:0/2198759593 wait complete. 2024-04-02T13:44:49.968 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:44:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.069+0000 7f14855ba640 1 -- 172.21.15.33:0/700677200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1480107aa0 msgr2=0x7f1480107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.069+0000 7f14855ba640 1 --2- 172.21.15.33:0/700677200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1480107aa0 0x7f1480107e80 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f1470009930 tx=0x7f147002bc30 comp rx=0 tx=0).stop 2024-04-02T13:44:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.069+0000 7f14855ba640 1 -- 172.21.15.33:0/700677200 shutdown_connections 2024-04-02T13:44:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.069+0000 7f14855ba640 1 --2- 172.21.15.33:0/700677200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1480102450 0x7f148010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.069+0000 7f14855ba640 1 --2- 172.21.15.33:0/700677200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1480101a90 0x7f1480101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.069+0000 7f14855ba640 1 --2- 172.21.15.33:0/700677200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1480107aa0 0x7f1480107e80 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.069+0000 7f14855ba640 1 -- 172.21.15.33:0/700677200 >> 172.21.15.33:0/700677200 conn(0x7f1480078f00 msgr2=0x7f1480079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.069+0000 7f14855ba640 1 -- 172.21.15.33:0/700677200 shutdown_connections 2024-04-02T13:44:50.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.069+0000 7f14855ba640 1 -- 172.21.15.33:0/700677200 wait complete. 2024-04-02T13:44:50.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.069+0000 7f14855ba640 1 Processor -- start 2024-04-02T13:44:50.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f14855ba640 1 -- start start 2024-04-02T13:44:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f14855ba640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1480101a90 0x7f14801983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f14855ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1480102450 0x7f1480198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f14855ba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1480107aa0 0x7f148019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f14855ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1480111e80 con 0x7f1480102450 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f14855ba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1480111d00 con 0x7f1480107aa0 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f14855ba640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1480112000 con 0x7f1480101a90 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f147e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1480102450 0x7f1480198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f147e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1480102450 0x7f1480198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36258/0 (socket says 172.21.15.33:36258) 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f147e7fc640 1 -- 172.21.15.33:0/1466059439 learned_addr learned my addr 172.21.15.33:0/1466059439 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f147effd640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1480101a90 0x7f14801983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f147f7fe640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1480107aa0 0x7f148019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f147e7fc640 1 -- 172.21.15.33:0/1466059439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1480101a90 msgr2=0x7f14801983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f147e7fc640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1480101a90 0x7f14801983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f147e7fc640 1 -- 172.21.15.33:0/1466059439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1480107aa0 msgr2=0x7f148019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f147e7fc640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1480107aa0 0x7f148019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f147e7fc640 1 -- 172.21.15.33:0/1466059439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148019d3b0 con 0x7f1480102450 2024-04-02T13:44:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.070+0000 7f147f7fe640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1480107aa0 0x7f148019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.071+0000 7f147e7fc640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1480102450 0x7f1480198900 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f147400e9f0 tx=0x7f147400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.071+0000 7f1467fff640 1 -- 172.21.15.33:0/1466059439 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147400cd80 con 0x7f1480102450 2024-04-02T13:44:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.071+0000 7f14855ba640 1 -- 172.21.15.33:0/1466059439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148019d6a0 con 0x7f1480102450 2024-04-02T13:44:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.071+0000 7f1467fff640 1 -- 172.21.15.33:0/1466059439 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f14740054f0 con 0x7f1480102450 2024-04-02T13:44:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.071+0000 7f1467fff640 1 -- 172.21.15.33:0/1466059439 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474010690 con 0x7f1480102450 2024-04-02T13:44:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.071+0000 7f14855ba640 1 -- 172.21.15.33:0/1466059439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14801a4f30 con 0x7f1480102450 2024-04-02T13:44:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.073+0000 7f1467fff640 1 -- 172.21.15.33:0/1466059439 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1474004f20 con 0x7f1480102450 2024-04-02T13:44:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.073+0000 7f14881a2740 1 Processor -- start 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.073+0000 7f14881a2740 1 -- start start 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.073+0000 7f14881a2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64450 0x55e9b5d64830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.073+0000 7f14881a2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 0x55e9b5d71b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f14881a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 0x55e9b5d744a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f14881a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e9b5cc87b0 con 0x55e9b5d720b0 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f14881a2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e9b5c67570 con 0x55e9b5d64450 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f14881a2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e9b5cc4760 con 0x55e9b5d64d70 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f1467fff640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f14680a3d20 0x7f14680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f147e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 0x55e9b5d744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f147e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 0x55e9b5d744a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36260/0 (socket says 172.21.15.33:36260) 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f147e7fc640 1 -- 172.21.15.33:0/1342880231 learned_addr learned my addr 172.21.15.33:0/1342880231 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f147f7fe640 1 --2- 172.21.15.33:0/1342880231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64450 0x55e9b5d64830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f147effd640 1 --2- 172.21.15.33:0/1342880231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 0x55e9b5d71b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f147f7fe640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f14680a3d20 0x7f14680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f1467fff640 1 -- 172.21.15.33:0/1466059439 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f14740d1050 con 0x7f1480102450 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f147e7fc640 1 -- 172.21.15.33:0/1342880231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 msgr2=0x55e9b5d71b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f147e7fc640 1 --2- 172.21.15.33:0/1342880231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 0x55e9b5d71b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f147e7fc640 1 -- 172.21.15.33:0/1342880231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64450 msgr2=0x55e9b5d64830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f147e7fc640 1 --2- 172.21.15.33:0/1342880231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64450 0x55e9b5d64830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f147e7fc640 1 -- 172.21.15.33:0/1342880231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b59dc3f0 con 0x55e9b5d720b0 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f147e7fc640 1 --2- 172.21.15.33:0/1342880231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 0x55e9b5d744a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f147409d2a0 tx=0x7f1474004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e09e0e61d1e685 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f147f7fe640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f14680a3d20 0x7f14680a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1478005c40 tx=0x7f1478009200 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f14657fa640 1 -- 172.21.15.33:0/1342880231 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147409d340 con 0x55e9b5d720b0 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.074+0000 7f14657fa640 1 -- 172.21.15.33:0/1342880231 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1474087070 con 0x55e9b5d720b0 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.075+0000 7f14657fa640 1 -- 172.21.15.33:0/1342880231 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474005210 con 0x55e9b5d720b0 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.075+0000 7f14881a2740 1 -- 172.21.15.33:0/1342880231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 msgr2=0x55e9b5d744a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.075+0000 7f14881a2740 1 --2- 172.21.15.33:0/1342880231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 0x55e9b5d744a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f147409d2a0 tx=0x7f1474004990 comp rx=0 tx=0).stop 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.075+0000 7f14881a2740 1 -- 172.21.15.33:0/1342880231 shutdown_connections 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.075+0000 7f14881a2740 1 --2- 172.21.15.33:0/1342880231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 0x55e9b5d744a0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.075+0000 7f14881a2740 1 --2- 172.21.15.33:0/1342880231 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 0x55e9b5d71b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.075+0000 7f14881a2740 1 --2- 172.21.15.33:0/1342880231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64450 0x55e9b5d64830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.075+0000 7f14881a2740 1 -- 172.21.15.33:0/1342880231 >> 172.21.15.33:0/1342880231 conn(0x55e9b5d5eed0 msgr2=0x55e9b5d5f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.075+0000 7f14881a2740 1 -- 172.21.15.33:0/1342880231 shutdown_connections 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.075+0000 7f14881a2740 1 -- 172.21.15.33:0/1342880231 wait complete. 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.075+0000 7f14881a2740 1 -- start start 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f14881a2740 1 Processor -- start 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f14881a2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64450 0x55e9b5d71ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f14881a2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 0x55e9b5d6ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f14881a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 0x55e9b5d6f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f14881a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e9b5d74e30 con 0x55e9b5d720b0 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f14881a2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e9b5c83e80 con 0x55e9b5d64450 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f14881a2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e9b5d74fb0 con 0x55e9b5d64d70 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f14881a2740 10 client.0 _ll_register_callbacks cb 0x55e9b5cbfaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f147f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 0x55e9b5d6f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f147f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 0x55e9b5d6f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36264/0 (socket says 172.21.15.33:36264) 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f147f7fe640 1 -- 172.21.15.33:0/601329396 learned_addr learned my addr 172.21.15.33:0/601329396 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f147effd640 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64450 0x55e9b5d71ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f147e7fc640 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 0x55e9b5d6ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f147f7fe640 1 -- 172.21.15.33:0/601329396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 msgr2=0x55e9b5d6ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f147f7fe640 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 0x55e9b5d6ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f147f7fe640 1 -- 172.21.15.33:0/601329396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64450 msgr2=0x55e9b5d71ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f147f7fe640 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64450 0x55e9b5d71ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f147f7fe640 1 -- 172.21.15.33:0/601329396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b5aa67d0 con 0x55e9b5d720b0 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f147e7fc640 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 0x55e9b5d6ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f147effd640 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64450 0x55e9b5d71ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f147f7fe640 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 0x55e9b5d6f5b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f14780142e0 tx=0x7f1478004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96776b411a69a9ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f1462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.076+0000 7f1462ff5640 1 -- 172.21.15.33:0/601329396 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478025070 con 0x55e9b5d720b0 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f14881a2740 1 -- 172.21.15.33:0/601329396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1474087070 con 0x55e9b5d720b0 2024-04-02T13:44:50.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f1462ff5640 1 -- 172.21.15.33:0/601329396 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f147800fba0 con 0x55e9b5d720b0 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f1462ff5640 1 -- 172.21.15.33:0/601329396 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1478031050 con 0x55e9b5d720b0 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f14881a2740 10 client.16923 fetch_fsmap learned FSMap version 35 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f14881a2740 1 -- 172.21.15.33:0/601329396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e9b5d80f70 con 0x55e9b5d720b0 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f1462ff5640 1 -- 172.21.15.33:0/601329396 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1474087070 con 0x55e9b5d720b0 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f1462ff5640 1 -- 172.21.15.33:0/601329396 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f1478013400 con 0x55e9b5d720b0 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f14881a2740 10 client.16923 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f14881a2740 10 client.16923 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f14881a2740 20 client.16923 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f14881a2740 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55e9b5d82380 0x55e9b5d847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f14881a2740 4 client.16923 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f14881a2740 1 -- 172.21.15.33:0/601329396 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9b5cd5cf0 con 0x55e9b5d82380 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.077+0000 7f147effd640 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55e9b5d82380 0x55e9b5d847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.078+0000 7f147effd640 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55e9b5d82380 0x55e9b5d847f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d591d9a35e90e54 server_cookie=1272888a70faac1c in_seq=0 out_seq=0 2024-04-02T13:44:50.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.078+0000 7f1462ff5640 10 client.16923 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f1462ff5640 1 -- 172.21.15.33:0/601329396 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9b5cd5cf0 con 0x55e9b5d82380 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f1462ff5640 10 client.16923 handle_command_reply: tid=0 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f14881a2740 1 client.16923 shutdown 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f14881a2740 1 -- 172.21.15.33:0/601329396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 msgr2=0x55e9b5d6f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f14881a2740 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 0x55e9b5d6f5b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f14780142e0 tx=0x7f1478004a40 comp rx=0 tx=0).stop 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f14881a2740 1 -- 172.21.15.33:0/601329396 shutdown_connections 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f14881a2740 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d720b0 0x55e9b5d6f5b0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f14881a2740 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55e9b5d82380 0x55e9b5d847f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f14881a2740 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 0x55e9b5d6ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f14881a2740 1 --2- 172.21.15.33:0/601329396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64450 0x55e9b5d71ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f1462ff5640 0 client.16923 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f14881a2740 1 -- 172.21.15.33:0/601329396 >> 172.21.15.33:0/601329396 conn(0x55e9b5d5eed0 msgr2=0x55e9b5d60c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f14881a2740 1 -- 172.21.15.33:0/601329396 shutdown_connections 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f14881a2740 1 -- 172.21.15.33:0/601329396 wait complete. 2024-04-02T13:44:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.079+0000 7f14881a2740 20 client.16923 trim_cache size 0 max 16384 2024-04-02T13:44:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.092+0000 7f14881a2740 1 Processor -- start 2024-04-02T13:44:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.092+0000 7f14881a2740 1 -- start start 2024-04-02T13:44:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.092+0000 7f14881a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 0x55e9b5d7dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.092+0000 7f14881a2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64d70 0x55e9b5d6de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.092+0000 7f14881a2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d720b0 0x55e9b5d8a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.092+0000 7f14881a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14801af6e0 con 0x55e9b5d64450 2024-04-02T13:44:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.092+0000 7f14881a2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f14801acf80 con 0x55e9b5d64d70 2024-04-02T13:44:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.092+0000 7f14881a2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f14801aadb0 con 0x55e9b5d720b0 2024-04-02T13:44:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.092+0000 7f147effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 0x55e9b5d7dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.092+0000 7f147effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 0x55e9b5d7dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36278/0 (socket says 172.21.15.33:36278) 2024-04-02T13:44:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.092+0000 7f147effd640 1 -- 172.21.15.33:0/3074015940 learned_addr learned my addr 172.21.15.33:0/3074015940 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.093+0000 7f147effd640 1 -- 172.21.15.33:0/3074015940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d720b0 msgr2=0x55e9b5d8a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.093+0000 7f147effd640 1 --2- 172.21.15.33:0/3074015940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d720b0 0x55e9b5d8a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.093+0000 7f147effd640 1 -- 172.21.15.33:0/3074015940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64d70 msgr2=0x55e9b5d6de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.093+0000 7f147effd640 1 --2- 172.21.15.33:0/3074015940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64d70 0x55e9b5d6de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.093+0000 7f147effd640 1 -- 172.21.15.33:0/3074015940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b5d65230 con 0x55e9b5d64450 2024-04-02T13:44:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.093+0000 7f147effd640 1 --2- 172.21.15.33:0/3074015940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 0x55e9b5d7dd60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1470005ab0 tx=0x7f1470005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0650b3bd1f80789 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.093+0000 7f1464ff9640 1 -- 172.21.15.33:0/3074015940 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147004e020 con 0x55e9b5d64450 2024-04-02T13:44:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.093+0000 7f1464ff9640 1 -- 172.21.15.33:0/3074015940 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1470005510 con 0x55e9b5d64450 2024-04-02T13:44:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.093+0000 7f1464ff9640 1 -- 172.21.15.33:0/3074015940 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14700046c0 con 0x55e9b5d64450 2024-04-02T13:44:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 -- 172.21.15.33:0/3074015940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 msgr2=0x55e9b5d7dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 --2- 172.21.15.33:0/3074015940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 0x55e9b5d7dd60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1470005ab0 tx=0x7f1470005fd0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 -- 172.21.15.33:0/3074015940 shutdown_connections 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 --2- 172.21.15.33:0/3074015940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d720b0 0x55e9b5d8a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 --2- 172.21.15.33:0/3074015940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d64d70 0x55e9b5d6de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 --2- 172.21.15.33:0/3074015940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 0x55e9b5d7dd60 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 -- 172.21.15.33:0/3074015940 >> 172.21.15.33:0/3074015940 conn(0x55e9b5d5eed0 msgr2=0x55e9b5d6d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 -- 172.21.15.33:0/3074015940 shutdown_connections 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 -- 172.21.15.33:0/3074015940 wait complete. 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 -- start start 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 Processor -- start 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 0x55e9b5d7a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 0x55e9b5d75830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d720b0 0x55e9b5d75d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14801ab440 con 0x55e9b5d64450 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f14801ac8b0 con 0x55e9b5d720b0 2024-04-02T13:44:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f14801abb10 con 0x55e9b5d64d70 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f14881a2740 10 client.0 _ll_register_callbacks cb 0x7f1478014f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f147effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 0x55e9b5d7a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f147effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 0x55e9b5d7a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36292/0 (socket says 172.21.15.33:36292) 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f147effd640 1 -- 172.21.15.33:0/2572456533 learned_addr learned my addr 172.21.15.33:0/2572456533 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f147f7fe640 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d720b0 0x55e9b5d75d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.094+0000 7f147e7fc640 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 0x55e9b5d75830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f147effd640 1 -- 172.21.15.33:0/2572456533 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 msgr2=0x55e9b5d75830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f147effd640 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 0x55e9b5d75830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f147effd640 1 -- 172.21.15.33:0/2572456533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d720b0 msgr2=0x55e9b5d75d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f147effd640 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d720b0 0x55e9b5d75d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f147effd640 1 -- 172.21.15.33:0/2572456533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b5d76360 con 0x55e9b5d64450 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f147f7fe640 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d720b0 0x55e9b5d75d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f147effd640 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 0x55e9b5d7a420 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1470036040 tx=0x7f14700045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96291f6af2f93b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f1462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f1462ff5640 1 -- 172.21.15.33:0/2572456533 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14700035f0 con 0x55e9b5d64450 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f14881a2740 1 -- 172.21.15.33:0/2572456533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1470005510 con 0x55e9b5d64450 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f1462ff5640 1 -- 172.21.15.33:0/2572456533 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1470003790 con 0x55e9b5d64450 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f1462ff5640 1 -- 172.21.15.33:0/2572456533 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1470003990 con 0x55e9b5d64450 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f14881a2740 10 client.16929 fetch_fsmap learned FSMap version 35 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f14881a2740 1 -- 172.21.15.33:0/2572456533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e9b5de8710 con 0x55e9b5d64450 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f1462ff5640 1 -- 172.21.15.33:0/2572456533 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1470005510 con 0x55e9b5d64450 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f1462ff5640 1 -- 172.21.15.33:0/2572456533 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f1470049070 con 0x55e9b5d64450 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f14881a2740 10 client.16929 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f14881a2740 10 client.16929 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f14881a2740 20 client.16929 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f14881a2740 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55e9b5de8ed0 0x55e9b5deb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.095+0000 7f14881a2740 4 client.16929 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f14881a2740 1 -- 172.21.15.33:0/2572456533 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9b5cd0040 con 0x55e9b5de8ed0 2024-04-02T13:44:50.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f147e7fc640 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55e9b5de8ed0 0x55e9b5deb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f1462ff5640 10 client.16929 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f147e7fc640 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55e9b5de8ed0 0x55e9b5deb2f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b85ad03dc2e73a2 server_cookie=e02080dceb59cd45 in_seq=0 out_seq=0 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f1462ff5640 1 -- 172.21.15.33:0/2572456533 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9b5cd0040 con 0x55e9b5de8ed0 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f1462ff5640 10 client.16929 handle_command_reply: tid=0 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f14881a2740 1 client.16929 shutdown 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f14881a2740 1 -- 172.21.15.33:0/2572456533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 msgr2=0x55e9b5d7a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f14881a2740 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 0x55e9b5d7a420 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1470036040 tx=0x7f14700045b0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f14881a2740 1 -- 172.21.15.33:0/2572456533 shutdown_connections 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f14881a2740 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9b5d720b0 0x55e9b5d75d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f14881a2740 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55e9b5de8ed0 0x55e9b5deb2f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f14881a2740 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e9b5d64d70 0x55e9b5d75830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f1462ff5640 0 client.16929 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f14881a2740 1 --2- 172.21.15.33:0/2572456533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9b5d64450 0x55e9b5d7a420 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.096+0000 7f14881a2740 1 -- 172.21.15.33:0/2572456533 >> 172.21.15.33:0/2572456533 conn(0x55e9b5d5eed0 msgr2=0x55e9b5d609c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14881a2740 1 -- 172.21.15.33:0/2572456533 shutdown_connections 2024-04-02T13:44:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14881a2740 1 -- 172.21.15.33:0/2572456533 wait complete. 2024-04-02T13:44:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14881a2740 20 client.16929 trim_cache size 0 max 16384 2024-04-02T13:44:50.110 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:44:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14855ba640 1 -- 172.21.15.33:0/1466059439 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f14680a3d20 msgr2=0x7f14680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14855ba640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f14680a3d20 0x7f14680a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1478005c40 tx=0x7f1478009200 comp rx=0 tx=0).stop 2024-04-02T13:44:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14855ba640 1 -- 172.21.15.33:0/1466059439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1480102450 msgr2=0x7f1480198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14855ba640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1480102450 0x7f1480198900 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f147400e9f0 tx=0x7f147400edf0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14855ba640 1 -- 172.21.15.33:0/1466059439 shutdown_connections 2024-04-02T13:44:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14855ba640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f14680a3d20 0x7f14680a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14855ba640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1480107aa0 0x7f148019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14855ba640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1480102450 0x7f1480198900 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14855ba640 1 --2- 172.21.15.33:0/1466059439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1480101a90 0x7f14801983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14855ba640 1 -- 172.21.15.33:0/1466059439 >> 172.21.15.33:0/1466059439 conn(0x7f1480078f00 msgr2=0x7f14801044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14855ba640 1 -- 172.21.15.33:0/1466059439 shutdown_connections 2024-04-02T13:44:50.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.097+0000 7f14855ba640 1 -- 172.21.15.33:0/1466059439 wait complete. 2024-04-02T13:44:50.113 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:44:50.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 -- 172.21.15.33:0/1906033716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f347810c110 msgr2=0x7f347810e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 --2- 172.21.15.33:0/1906033716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f347810c110 0x7f347810e540 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3474009ca0 tx=0x7f347402c180 comp rx=0 tx=0).stop 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 -- 172.21.15.33:0/1906033716 shutdown_connections 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 --2- 172.21.15.33:0/1906033716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f347810c110 0x7f347810e540 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 --2- 172.21.15.33:0/1906033716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3478103660 0x7f347810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 --2- 172.21.15.33:0/1906033716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3478102d40 0x7f3478103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 -- 172.21.15.33:0/1906033716 >> 172.21.15.33:0/1906033716 conn(0x7f3478076d80 msgr2=0x7f3478077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 -- 172.21.15.33:0/1906033716 shutdown_connections 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 -- 172.21.15.33:0/1906033716 wait complete. 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 Processor -- start 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 -- start start 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3478102d40 0x7f3478198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3478103660 0x7f34781988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f347810c110 0x7f347819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3478111e70 con 0x7f3478102d40 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3478111cf0 con 0x7f3478103660 2024-04-02T13:44:50.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.214+0000 7f347fc55640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3478111ff0 con 0x7f347810c110 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347d9f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3478102d40 0x7f3478198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347d9f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3478102d40 0x7f3478198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36302/0 (socket says 172.21.15.33:36302) 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347d9f1640 1 -- 172.21.15.33:0/3396386730 learned_addr learned my addr 172.21.15.33:0/3396386730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347d1f0640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3478103660 0x7f34781988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347e1f2640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f347810c110 0x7f347819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347d9f1640 1 -- 172.21.15.33:0/3396386730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f347810c110 msgr2=0x7f347819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347d9f1640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f347810c110 0x7f347819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347d9f1640 1 -- 172.21.15.33:0/3396386730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3478103660 msgr2=0x7f34781988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347d9f1640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3478103660 0x7f34781988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347d9f1640 1 -- 172.21.15.33:0/3396386730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347819d360 con 0x7f3478102d40 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347e1f2640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f347810c110 0x7f347819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347d1f0640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3478103660 0x7f34781988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347d9f1640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3478102d40 0x7f3478198370 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f347000d950 tx=0x7f347000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f346effd640 1 -- 172.21.15.33:0/3396386730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470017030 con 0x7f3478102d40 2024-04-02T13:44:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347fc55640 1 -- 172.21.15.33:0/3396386730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347819d650 con 0x7f3478102d40 2024-04-02T13:44:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f346effd640 1 -- 172.21.15.33:0/3396386730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3470004020 con 0x7f3478102d40 2024-04-02T13:44:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f346effd640 1 -- 172.21.15.33:0/3396386730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34700047a0 con 0x7f3478102d40 2024-04-02T13:44:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.215+0000 7f347fc55640 1 -- 172.21.15.33:0/3396386730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34781a4ee0 con 0x7f3478102d40 2024-04-02T13:44:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f346effd640 1 -- 172.21.15.33:0/3396386730 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f34700041d0 con 0x7f3478102d40 2024-04-02T13:44:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f3482839740 1 Processor -- start 2024-04-02T13:44:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f3482839740 1 -- start start 2024-04-02T13:44:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f3482839740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 0x55e424440b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f3482839740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 0x55e424449760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f3482839740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e42444c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f3482839740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e4243a47b0 con 0x55e424449ca0 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f3482839740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e424343570 con 0x55e4244407a0 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f3482839740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e4243a0760 con 0x55e4244410c0 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f346effd640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f34640a3d20 0x7f34640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f347d1f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e42444c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f347d1f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e42444c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36312/0 (socket says 172.21.15.33:36312) 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f347d1f0640 1 -- 172.21.15.33:0/2021948443 learned_addr learned my addr 172.21.15.33:0/2021948443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f347e1f2640 1 --2- 172.21.15.33:0/2021948443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 0x55e424440b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f347e1f2640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f34640a3d20 0x7f34640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.218+0000 7f346effd640 1 -- 172.21.15.33:0/3396386730 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f34700d0680 con 0x7f3478102d40 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.219+0000 7f347d9f1640 1 --2- 172.21.15.33:0/2021948443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 0x55e424449760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.219+0000 7f347d1f0640 1 -- 172.21.15.33:0/2021948443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 msgr2=0x55e424449760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.219+0000 7f347d1f0640 1 --2- 172.21.15.33:0/2021948443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 0x55e424449760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.219+0000 7f347d1f0640 1 -- 172.21.15.33:0/2021948443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 msgr2=0x55e424440b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.219+0000 7f347d1f0640 1 --2- 172.21.15.33:0/2021948443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 0x55e424440b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.219+0000 7f347d1f0640 1 -- 172.21.15.33:0/2021948443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4240b83f0 con 0x55e424449ca0 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.219+0000 7f347d1f0640 1 --2- 172.21.15.33:0/2021948443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e42444c090 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f346800dc30 tx=0x7f346800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843eff8f76fea12d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.219+0000 7f347e1f2640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f34640a3d20 0x7f34640a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f347402c6c0 tx=0x7f34740095c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.219+0000 7f3463fff640 1 -- 172.21.15.33:0/2021948443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468005020 con 0x55e424449ca0 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.219+0000 7f3463fff640 1 -- 172.21.15.33:0/2021948443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f34680047a0 con 0x55e424449ca0 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.219+0000 7f3463fff640 1 -- 172.21.15.33:0/2021948443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34680116a0 con 0x55e424449ca0 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.220+0000 7f3482839740 1 -- 172.21.15.33:0/2021948443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 msgr2=0x55e42444c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.220+0000 7f3482839740 1 --2- 172.21.15.33:0/2021948443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e42444c090 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f346800dc30 tx=0x7f346800f730 comp rx=0 tx=0).stop 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.220+0000 7f3482839740 1 -- 172.21.15.33:0/2021948443 shutdown_connections 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.220+0000 7f3482839740 1 --2- 172.21.15.33:0/2021948443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e42444c090 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.220+0000 7f3482839740 1 --2- 172.21.15.33:0/2021948443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 0x55e424449760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.220+0000 7f3482839740 1 --2- 172.21.15.33:0/2021948443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 0x55e424440b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.220+0000 7f3482839740 1 -- 172.21.15.33:0/2021948443 >> 172.21.15.33:0/2021948443 conn(0x55e42443aed0 msgr2=0x55e42443b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.220+0000 7f3482839740 1 -- 172.21.15.33:0/2021948443 shutdown_connections 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.220+0000 7f3482839740 1 -- 172.21.15.33:0/2021948443 wait complete. 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.220+0000 7f3482839740 1 -- start start 2024-04-02T13:44:50.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.220+0000 7f3482839740 1 Processor -- start 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f3482839740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4244407a0 0x55e4244496e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f3482839740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244410c0 0x55e424446a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f3482839740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e424449ca0 0x55e4244471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f3482839740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e42444ca90 con 0x55e4244407a0 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f3482839740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e42435fe80 con 0x55e4244410c0 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f3482839740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e42444cc10 con 0x55e424449ca0 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f3482839740 10 client.0 _ll_register_callbacks cb 0x55e42439baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f347d9f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4244407a0 0x55e4244496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f347d9f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4244407a0 0x55e4244496e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36324/0 (socket says 172.21.15.33:36324) 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f347d9f1640 1 -- 172.21.15.33:0/2224721365 learned_addr learned my addr 172.21.15.33:0/2224721365 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f347d1f0640 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244410c0 0x55e424446a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f347e1f2640 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e424449ca0 0x55e4244471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f347d9f1640 1 -- 172.21.15.33:0/2224721365 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e424449ca0 msgr2=0x55e4244471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f347d9f1640 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e424449ca0 0x55e4244471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f347d9f1640 1 -- 172.21.15.33:0/2224721365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244410c0 msgr2=0x55e424446a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f347d9f1640 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244410c0 0x55e424446a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f347d9f1640 1 -- 172.21.15.33:0/2224721365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4241827d0 con 0x55e4244407a0 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f347e1f2640 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e424449ca0 0x55e4244471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f347d9f1640 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4244407a0 0x55e4244496e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3470098710 tx=0x7f347009bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5578102024f66784 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f34617fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f34617fa640 1 -- 172.21.15.33:0/2224721365 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34700b5060 con 0x55e4244407a0 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f34617fa640 1 -- 172.21.15.33:0/2224721365 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f34700d0290 con 0x55e4244407a0 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.221+0000 7f3482839740 1 -- 172.21.15.33:0/2224721365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34680047a0 con 0x55e4244407a0 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.222+0000 7f34617fa640 1 -- 172.21.15.33:0/2224721365 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f34700cdee0 con 0x55e4244407a0 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.222+0000 7f3482839740 10 client.16941 fetch_fsmap learned FSMap version 35 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.222+0000 7f3482839740 1 -- 172.21.15.33:0/2224721365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e424458b20 con 0x55e4244407a0 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.222+0000 7f34617fa640 1 -- 172.21.15.33:0/2224721365 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f34680047a0 con 0x55e4244407a0 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.222+0000 7f34617fa640 1 -- 172.21.15.33:0/2224721365 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f34700a0170 con 0x55e4244407a0 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.222+0000 7f3482839740 10 client.16941 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.222+0000 7f3482839740 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:50.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.222+0000 7f3482839740 20 client.16941 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.222+0000 7f3482839740 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e424459f30 0x55e42445c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.222+0000 7f3482839740 4 client.16941 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.222+0000 7f3482839740 1 -- 172.21.15.33:0/2224721365 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4243b1cf0 con 0x55e424459f30 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.222+0000 7f347d1f0640 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e424459f30 0x55e42445c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.223+0000 7f347d1f0640 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e424459f30 0x55e42445c380 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e541b5b764fee931 server_cookie=a959a3efd2cfaca1 in_seq=0 out_seq=0 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.223+0000 7f34617fa640 10 client.16941 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.223+0000 7f34617fa640 1 -- 172.21.15.33:0/2224721365 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4243b1cf0 con 0x55e424459f30 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.223+0000 7f34617fa640 10 client.16941 handle_command_reply: tid=0 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.223+0000 7f3482839740 1 client.16941 shutdown 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.224+0000 7f3482839740 1 -- 172.21.15.33:0/2224721365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4244407a0 msgr2=0x55e4244496e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.224+0000 7f3482839740 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4244407a0 0x55e4244496e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3470098710 tx=0x7f347009bd00 comp rx=0 tx=0).stop 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.224+0000 7f3482839740 1 -- 172.21.15.33:0/2224721365 shutdown_connections 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.224+0000 7f3482839740 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e424459f30 0x55e42445c380 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.224+0000 7f3482839740 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e424449ca0 0x55e4244471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.224+0000 7f3482839740 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244410c0 0x55e424446a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.224+0000 7f3482839740 1 --2- 172.21.15.33:0/2224721365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4244407a0 0x55e4244496e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.224+0000 7f34617fa640 0 client.16941 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.224+0000 7f3482839740 1 -- 172.21.15.33:0/2224721365 >> 172.21.15.33:0/2224721365 conn(0x55e42443aed0 msgr2=0x55e42443cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.224+0000 7f3482839740 1 -- 172.21.15.33:0/2224721365 shutdown_connections 2024-04-02T13:44:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.224+0000 7f3482839740 1 -- 172.21.15.33:0/2224721365 wait complete. 2024-04-02T13:44:50.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.224+0000 7f3482839740 20 client.16941 trim_cache size 0 max 16384 2024-04-02T13:44:50.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.237+0000 7f3482839740 1 Processor -- start 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.237+0000 7f3482839740 1 -- start start 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.237+0000 7f3482839740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 0x55e424455950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.237+0000 7f3482839740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 0x55e424461c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.237+0000 7f3482839740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e424462190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.237+0000 7f3482839740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34781b18f0 con 0x55e424449ca0 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.237+0000 7f3482839740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34781af190 con 0x55e4244407a0 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.237+0000 7f3482839740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f34781acfc0 con 0x55e4244410c0 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.237+0000 7f347e1f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e424462190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.237+0000 7f347e1f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e424462190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36328/0 (socket says 172.21.15.33:36328) 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.237+0000 7f347e1f2640 1 -- 172.21.15.33:0/2756914767 learned_addr learned my addr 172.21.15.33:0/2756914767 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.237+0000 7f347d9f1640 1 --2- 172.21.15.33:0/2756914767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 0x55e424455950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.238+0000 7f347e1f2640 1 -- 172.21.15.33:0/2756914767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 msgr2=0x55e424461c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.238+0000 7f347e1f2640 1 --2- 172.21.15.33:0/2756914767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 0x55e424461c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.238+0000 7f347e1f2640 1 -- 172.21.15.33:0/2756914767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 msgr2=0x55e424455950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.238+0000 7f347e1f2640 1 --2- 172.21.15.33:0/2756914767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 0x55e424455950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.238+0000 7f347e1f2640 1 -- 172.21.15.33:0/2756914767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e424447520 con 0x55e424449ca0 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.238+0000 7f347d9f1640 1 --2- 172.21.15.33:0/2756914767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 0x55e424455950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.238+0000 7f347e1f2640 1 --2- 172.21.15.33:0/2756914767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e424462190 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3474003f40 tx=0x7f347403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3690ccb1e08cfe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.238+0000 7f34637fe640 1 -- 172.21.15.33:0/2756914767 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474039070 con 0x55e424449ca0 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.238+0000 7f34637fe640 1 -- 172.21.15.33:0/2756914767 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3474054070 con 0x55e424449ca0 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.238+0000 7f34637fe640 1 -- 172.21.15.33:0/2756914767 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474032cf0 con 0x55e424449ca0 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 -- 172.21.15.33:0/2756914767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 msgr2=0x55e424462190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 --2- 172.21.15.33:0/2756914767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e424462190 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3474003f40 tx=0x7f347403f000 comp rx=0 tx=0).stop 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 -- 172.21.15.33:0/2756914767 shutdown_connections 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 --2- 172.21.15.33:0/2756914767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e424462190 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 --2- 172.21.15.33:0/2756914767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 0x55e424461c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 --2- 172.21.15.33:0/2756914767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 0x55e424455950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 -- 172.21.15.33:0/2756914767 >> 172.21.15.33:0/2756914767 conn(0x55e42443aed0 msgr2=0x55e424441de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 -- 172.21.15.33:0/2756914767 shutdown_connections 2024-04-02T13:44:50.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 -- 172.21.15.33:0/2756914767 wait complete. 2024-04-02T13:44:50.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 -- start start 2024-04-02T13:44:50.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 Processor -- start 2024-04-02T13:44:50.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 0x55e4244439a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 0x55e424443ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e424444420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34781ad650 con 0x55e424449ca0 2024-04-02T13:44:50.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34781aeac0 con 0x55e4244407a0 2024-04-02T13:44:50.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f34781add20 con 0x55e4244410c0 2024-04-02T13:44:50.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f3482839740 10 client.0 _ll_register_callbacks cb 0x7f3470005130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:50.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.239+0000 7f347e1f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e424444420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f347d9f1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 0x55e4244439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f347e1f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e424444420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36340/0 (socket says 172.21.15.33:36340) 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f347e1f2640 1 -- 172.21.15.33:0/953257687 learned_addr learned my addr 172.21.15.33:0/953257687 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f347d1f0640 1 --2- 172.21.15.33:0/953257687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 0x55e424443ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f347e1f2640 1 -- 172.21.15.33:0/953257687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 msgr2=0x55e424443ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f347e1f2640 1 --2- 172.21.15.33:0/953257687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 0x55e424443ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f347e1f2640 1 -- 172.21.15.33:0/953257687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 msgr2=0x55e4244439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f347e1f2640 1 --2- 172.21.15.33:0/953257687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 0x55e4244439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f347e1f2640 1 -- 172.21.15.33:0/953257687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e424450510 con 0x55e424449ca0 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f347d1f0640 1 --2- 172.21.15.33:0/953257687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 0x55e424443ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f347e1f2640 1 --2- 172.21.15.33:0/953257687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e424444420 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3474009a60 tx=0x7f3474003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7834c436bfaaa072 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f34617fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f34617fa640 1 -- 172.21.15.33:0/953257687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347405c070 con 0x55e424449ca0 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f34617fa640 1 -- 172.21.15.33:0/953257687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3474004610 con 0x55e424449ca0 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f3482839740 1 -- 172.21.15.33:0/953257687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3474054070 con 0x55e424449ca0 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f34617fa640 1 -- 172.21.15.33:0/953257687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3474039040 con 0x55e424449ca0 2024-04-02T13:44:50.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f3482839740 10 client.16947 fetch_fsmap learned FSMap version 35 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f3482839740 1 -- 172.21.15.33:0/953257687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e4244c4e40 con 0x55e424449ca0 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.240+0000 7f34617fa640 1 -- 172.21.15.33:0/953257687 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3474054070 con 0x55e424449ca0 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f34617fa640 1 -- 172.21.15.33:0/953257687 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f3474005580 con 0x55e424449ca0 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f3482839740 10 client.16947 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f3482839740 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f3482839740 20 client.16947 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f3482839740 1 --2- 172.21.15.33:0/953257687 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e4244c5590 0x55e4244c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f3482839740 4 client.16947 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f3482839740 1 -- 172.21.15.33:0/953257687 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4243ac040 con 0x55e4244c5590 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f347d9f1640 1 --2- 172.21.15.33:0/953257687 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e4244c5590 0x55e4244c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f34617fa640 10 client.16947 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f347d9f1640 1 --2- 172.21.15.33:0/953257687 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e4244c5590 0x55e4244c79b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f0c65ccf1ae07d1 server_cookie=4a2a513497caa959 in_seq=0 out_seq=0 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f34617fa640 1 -- 172.21.15.33:0/953257687 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4243ac040 con 0x55e4244c5590 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f34617fa640 10 client.16947 handle_command_reply: tid=0 2024-04-02T13:44:50.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f3482839740 1 client.16947 shutdown 2024-04-02T13:44:50.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f3482839740 1 -- 172.21.15.33:0/953257687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 msgr2=0x55e424444420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.241+0000 7f3482839740 1 --2- 172.21.15.33:0/953257687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e424444420 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3474009a60 tx=0x7f3474003ce0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f3482839740 1 -- 172.21.15.33:0/953257687 shutdown_connections 2024-04-02T13:44:50.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f3482839740 1 --2- 172.21.15.33:0/953257687 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e4244c5590 0x55e4244c79b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f3482839740 1 --2- 172.21.15.33:0/953257687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e424449ca0 0x55e424444420 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f3482839740 1 --2- 172.21.15.33:0/953257687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e4244410c0 0x55e424443ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f3482839740 1 --2- 172.21.15.33:0/953257687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4244407a0 0x55e4244439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f34617fa640 0 client.16947 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:50.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f3482839740 1 -- 172.21.15.33:0/953257687 >> 172.21.15.33:0/953257687 conn(0x55e42443aed0 msgr2=0x55e4244396a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:50.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f3482839740 1 -- 172.21.15.33:0/953257687 shutdown_connections 2024-04-02T13:44:50.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f3482839740 1 -- 172.21.15.33:0/953257687 wait complete. 2024-04-02T13:44:50.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f3482839740 20 client.16947 trim_cache size 0 max 16384 2024-04-02T13:44:50.252 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:44:50.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f347fc55640 1 -- 172.21.15.33:0/3396386730 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f34640a3d20 msgr2=0x7f34640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f347fc55640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f34640a3d20 0x7f34640a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f347402c6c0 tx=0x7f34740095c0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f347fc55640 1 -- 172.21.15.33:0/3396386730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3478102d40 msgr2=0x7f3478198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:50.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f347fc55640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3478102d40 0x7f3478198370 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f347000d950 tx=0x7f347000de10 comp rx=0 tx=0).stop 2024-04-02T13:44:50.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f347fc55640 1 -- 172.21.15.33:0/3396386730 shutdown_connections 2024-04-02T13:44:50.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f347fc55640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f34640a3d20 0x7f34640a61e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f347fc55640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f347810c110 0x7f347819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f347fc55640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3478103660 0x7f34781988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f347fc55640 1 --2- 172.21.15.33:0/3396386730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3478102d40 0x7f3478198370 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:50.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f347fc55640 1 -- 172.21.15.33:0/3396386730 >> 172.21.15.33:0/3396386730 conn(0x7f3478076d80 msgr2=0x7f34780fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:50.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f347fc55640 1 -- 172.21.15.33:0/3396386730 shutdown_connections 2024-04-02T13:44:50.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:50.242+0000 7f347fc55640 1 -- 172.21.15.33:0/3396386730 wait complete. 2024-04-02T13:44:50.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:50 smithi145 ceph-mon[35646]: pgmap v184: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:44:50.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:50 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:50.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:50 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3197936786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:44:50.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:50 smithi050 ceph-mon[35524]: pgmap v184: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:44:50.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:50 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:50.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:50 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3197936786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:44:51.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:50 smithi033 ceph-mon[32390]: pgmap v184: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:44:51.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:50 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:51.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:50 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3197936786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:44:51.257 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:44:51.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:44:51.257 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:44:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.373+0000 7f52c35cd640 1 -- 172.21.15.33:0/1626367956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52bc103690 msgr2=0x7f52bc10bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.373+0000 7f52c35cd640 1 --2- 172.21.15.33:0/1626367956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52bc103690 0x7f52bc10bb70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009900 tx=0x7f52b002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.373+0000 7f52c35cd640 1 -- 172.21.15.33:0/1626367956 shutdown_connections 2024-04-02T13:44:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.373+0000 7f52c35cd640 1 --2- 172.21.15.33:0/1626367956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52bc10c140 0x7f52bc10e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.373+0000 7f52c35cd640 1 --2- 172.21.15.33:0/1626367956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52bc103690 0x7f52bc10bb70 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.373+0000 7f52c35cd640 1 --2- 172.21.15.33:0/1626367956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52bc102d70 0x7f52bc103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.373+0000 7f52c35cd640 1 -- 172.21.15.33:0/1626367956 >> 172.21.15.33:0/1626367956 conn(0x7f52bc077b60 msgr2=0x7f52bc077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.374+0000 7f52c35cd640 1 -- 172.21.15.33:0/1626367956 shutdown_connections 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.374+0000 7f52c35cd640 1 -- 172.21.15.33:0/1626367956 wait complete. 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.374+0000 7f52c35cd640 1 Processor -- start 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.374+0000 7f52c35cd640 1 -- start start 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.374+0000 7f52c35cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52bc102d70 0x7f52bc1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.374+0000 7f52c35cd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52bc103690 0x7f52bc100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.374+0000 7f52c35cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52bc10c140 0x7f52bc101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.374+0000 7f52c35cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc111ee0 con 0x7f52bc10c140 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.374+0000 7f52c35cd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc111d60 con 0x7f52bc103690 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.374+0000 7f52c35cd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc112060 con 0x7f52bc102d70 2024-04-02T13:44:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.374+0000 7f52c1b6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52bc10c140 0x7f52bc101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.374+0000 7f52c1b6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52bc10c140 0x7f52bc101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36362/0 (socket says 172.21.15.33:36362) 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.374+0000 7f52c1b6a640 1 -- 172.21.15.33:0/1766483784 learned_addr learned my addr 172.21.15.33:0/1766483784 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52c0b68640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52bc103690 0x7f52bc100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52c1369640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52bc102d70 0x7f52bc1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52c1b6a640 1 -- 172.21.15.33:0/1766483784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52bc102d70 msgr2=0x7f52bc1009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52c1b6a640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52bc102d70 0x7f52bc1009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52c1b6a640 1 -- 172.21.15.33:0/1766483784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52bc103690 msgr2=0x7f52bc100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52c1b6a640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52bc103690 0x7f52bc100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52c1b6a640 1 -- 172.21.15.33:0/1766483784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc1a68e0 con 0x7f52bc10c140 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52c1369640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52bc102d70 0x7f52bc1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52c1b6a640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52bc10c140 0x7f52bc101440 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f52b400cce0 tx=0x7f52b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52ae7fc640 1 -- 172.21.15.33:0/1766483784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b400f450 con 0x7f52bc10c140 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52ae7fc640 1 -- 172.21.15.33:0/1766483784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f52b400fac0 con 0x7f52bc10c140 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52c35cd640 1 -- 172.21.15.33:0/1766483784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52bc1a6bd0 con 0x7f52bc10c140 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52ae7fc640 1 -- 172.21.15.33:0/1766483784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4018be0 con 0x7f52bc10c140 2024-04-02T13:44:51.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.375+0000 7f52c35cd640 1 -- 172.21.15.33:0/1766483784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52bc1a70a0 con 0x7f52bc10c140 2024-04-02T13:44:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.377+0000 7f52ae7fc640 1 -- 172.21.15.33:0/1766483784 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f52b400fc70 con 0x7f52bc10c140 2024-04-02T13:44:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.378+0000 7f52c61b1740 1 Processor -- start 2024-04-02T13:44:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.378+0000 7f52c61b1740 1 -- start start 2024-04-02T13:44:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.378+0000 7f52c61b1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f02c0 0x55d8db1f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.378+0000 7f52c61b1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f0be0 0x55d8db1f9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.378+0000 7f52c61b1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f97c0 0x55d8db1fbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.378+0000 7f52c61b1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d8db154770 con 0x55d8db1f02c0 2024-04-02T13:44:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.378+0000 7f52c61b1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d8db0fc960 con 0x55d8db1f0be0 2024-04-02T13:44:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.378+0000 7f52c61b1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d8db1508c0 con 0x55d8db1f97c0 2024-04-02T13:44:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.378+0000 7f52ae7fc640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f52a80a3d20 0x7f52a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c0b68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f02c0 0x55d8db1f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c0b68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f02c0 0x55d8db1f06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36366/0 (socket says 172.21.15.33:36366) 2024-04-02T13:44:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c0b68640 1 -- 172.21.15.33:0/3680494790 learned_addr learned my addr 172.21.15.33:0/3680494790 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c1b6a640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f52a80a3d20 0x7f52a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c1369640 1 --2- 172.21.15.33:0/3680494790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f0be0 0x55d8db1f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52ae7fc640 1 -- 172.21.15.33:0/1766483784 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f52b40d1220 con 0x7f52bc10c140 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c0b68640 1 --2- 172.21.15.33:0/3680494790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f97c0 0x55d8db1fbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c0b68640 1 -- 172.21.15.33:0/3680494790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f97c0 msgr2=0x55d8db1fbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c0b68640 1 --2- 172.21.15.33:0/3680494790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f97c0 0x55d8db1fbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c0b68640 1 -- 172.21.15.33:0/3680494790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f0be0 msgr2=0x55d8db1f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c0b68640 1 --2- 172.21.15.33:0/3680494790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f0be0 0x55d8db1f9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c0b68640 1 -- 172.21.15.33:0/3680494790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8daf307e0 con 0x55d8db1f02c0 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c1369640 1 --2- 172.21.15.33:0/3680494790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f0be0 0x55d8db1f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c1b6a640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f52a80a3d20 0x7f52a80a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f52b40997e0 tx=0x7f52b406e850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52c0b68640 1 --2- 172.21.15.33:0/3680494790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f02c0 0x55d8db1f06a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f52b0036040 tx=0x7f52b0005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc9e74f41bd5d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52a77fe640 1 -- 172.21.15.33:0/3680494790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0047020 con 0x55d8db1f02c0 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52a77fe640 1 -- 172.21.15.33:0/3680494790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f52b0005510 con 0x55d8db1f02c0 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.379+0000 7f52a77fe640 1 -- 172.21.15.33:0/3680494790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b00048c0 con 0x55d8db1f02c0 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.380+0000 7f52c61b1740 1 -- 172.21.15.33:0/3680494790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f02c0 msgr2=0x55d8db1f06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.380+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3680494790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f02c0 0x55d8db1f06a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f52b0036040 tx=0x7f52b0005d90 comp rx=0 tx=0).stop 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.380+0000 7f52c61b1740 1 -- 172.21.15.33:0/3680494790 shutdown_connections 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.380+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3680494790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f97c0 0x55d8db1fbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.380+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3680494790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f0be0 0x55d8db1f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.380+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3680494790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f02c0 0x55d8db1f06a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.380+0000 7f52c61b1740 1 -- 172.21.15.33:0/3680494790 >> 172.21.15.33:0/3680494790 conn(0x55d8db1ea9f0 msgr2=0x55d8db1eadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.380+0000 7f52c61b1740 1 -- 172.21.15.33:0/3680494790 shutdown_connections 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.380+0000 7f52c61b1740 1 -- 172.21.15.33:0/3680494790 wait complete. 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.380+0000 7f52c61b1740 1 -- start start 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c61b1740 1 Processor -- start 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c61b1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 0x55d8db1f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c61b1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db1f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c61b1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 0x55d8db1f7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c61b1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d8db1fc550 con 0x55d8db1f0be0 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c61b1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d8db10fd20 con 0x55d8db1f97c0 2024-04-02T13:44:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c61b1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d8db1fc6d0 con 0x55d8db1f02c0 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c61b1740 10 client.0 _ll_register_callbacks cb 0x55d8db0dca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c1369640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db1f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c1369640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db1f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36380/0 (socket says 172.21.15.33:36380) 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c1369640 1 -- 172.21.15.33:0/2938348606 learned_addr learned my addr 172.21.15.33:0/2938348606 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c0b68640 1 --2- 172.21.15.33:0/2938348606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 0x55d8db1f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c1369640 1 -- 172.21.15.33:0/2938348606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 msgr2=0x55d8db1f67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c1369640 1 --2- 172.21.15.33:0/2938348606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 0x55d8db1f67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c1369640 1 -- 172.21.15.33:0/2938348606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 msgr2=0x55d8db1f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c1369640 1 --2- 172.21.15.33:0/2938348606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 0x55d8db1f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c1369640 1 -- 172.21.15.33:0/2938348606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8daf90f70 con 0x55d8db1f0be0 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.381+0000 7f52c0b68640 1 --2- 172.21.15.33:0/2938348606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 0x55d8db1f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52c1369640 1 --2- 172.21.15.33:0/2938348606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db1f6d20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f52b8012480 tx=0x7f52b8012940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199e3577dd18389e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52a4ff9640 1 -- 172.21.15.33:0/2938348606 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b801f060 con 0x55d8db1f0be0 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52a4ff9640 1 -- 172.21.15.33:0/2938348606 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f52b80109e0 con 0x55d8db1f0be0 2024-04-02T13:44:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52c61b1740 1 -- 172.21.15.33:0/2938348606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b0005510 con 0x55d8db1f0be0 2024-04-02T13:44:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52a4ff9640 1 -- 172.21.15.33:0/2938348606 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f52b8026050 con 0x55d8db1f0be0 2024-04-02T13:44:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52c61b1740 10 client.16959 fetch_fsmap learned FSMap version 35 2024-04-02T13:44:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52c61b1740 1 -- 172.21.15.33:0/2938348606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d8dae28170 con 0x55d8db1f0be0 2024-04-02T13:44:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52a4ff9640 1 -- 172.21.15.33:0/2938348606 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f52b0005510 con 0x55d8db1f0be0 2024-04-02T13:44:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52a4ff9640 1 -- 172.21.15.33:0/2938348606 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f52b8019030 con 0x55d8db1f0be0 2024-04-02T13:44:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52c61b1740 10 client.16959 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T13:44:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52c61b1740 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52c61b1740 20 client.16959 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52c61b1740 1 --2- 172.21.15.33:0/2938348606 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d8db209d00 0x55d8db20c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52c61b1740 4 client.16959 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.382+0000 7f52c61b1740 1 -- 172.21.15.33:0/2938348606 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8db161cb0 con 0x55d8db209d00 2024-04-02T13:44:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.383+0000 7f52c0b68640 1 --2- 172.21.15.33:0/2938348606 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d8db209d00 0x55d8db20c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.383+0000 7f52c0b68640 1 --2- 172.21.15.33:0/2938348606 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d8db209d00 0x55d8db20c120 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6da66dffdc91517 server_cookie=567cfcd6758438d1 in_seq=0 out_seq=0 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.383+0000 7f52a4ff9640 10 client.16959 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52a4ff9640 1 -- 172.21.15.33:0/2938348606 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8db161cb0 con 0x55d8db209d00 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52a4ff9640 10 client.16959 handle_command_reply: tid=0 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52c61b1740 1 client.16959 shutdown 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52c61b1740 1 -- 172.21.15.33:0/2938348606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 msgr2=0x55d8db1f6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52c61b1740 1 --2- 172.21.15.33:0/2938348606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db1f6d20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f52b8012480 tx=0x7f52b8012940 comp rx=0 tx=0).stop 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52c61b1740 1 -- 172.21.15.33:0/2938348606 shutdown_connections 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52c61b1740 1 --2- 172.21.15.33:0/2938348606 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d8db209d00 0x55d8db20c120 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52c61b1740 1 --2- 172.21.15.33:0/2938348606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 0x55d8db1f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52c61b1740 1 --2- 172.21.15.33:0/2938348606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db1f6d20 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52a4ff9640 0 client.16959 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52c61b1740 1 --2- 172.21.15.33:0/2938348606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 0x55d8db1f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52c61b1740 1 -- 172.21.15.33:0/2938348606 >> 172.21.15.33:0/2938348606 conn(0x55d8db1ea9f0 msgr2=0x55d8db1ec740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52c61b1740 1 -- 172.21.15.33:0/2938348606 shutdown_connections 2024-04-02T13:44:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52c61b1740 1 -- 172.21.15.33:0/2938348606 wait complete. 2024-04-02T13:44:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.384+0000 7f52c61b1740 20 client.16959 trim_cache size 0 max 16384 2024-04-02T13:44:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.402+0000 7f52c61b1740 1 Processor -- start 2024-04-02T13:44:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.402+0000 7f52c61b1740 1 -- start start 2024-04-02T13:44:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.402+0000 7f52c61b1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 0x55d8db1f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.402+0000 7f52c61b1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db1f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.402+0000 7f52c61b1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 0x55d8db211a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.402+0000 7f52c61b1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1b37a0 con 0x55d8db1f0be0 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.402+0000 7f52c61b1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1b1040 con 0x55d8db1f97c0 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.402+0000 7f52c61b1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1aee70 con 0x55d8db1f02c0 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.402+0000 7f52c1369640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db1f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.402+0000 7f52c1369640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db1f7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36390/0 (socket says 172.21.15.33:36390) 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.402+0000 7f52c1369640 1 -- 172.21.15.33:0/3964895198 learned_addr learned my addr 172.21.15.33:0/3964895198 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52c0b68640 1 --2- 172.21.15.33:0/3964895198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 0x55d8db211a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52c1369640 1 -- 172.21.15.33:0/3964895198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 msgr2=0x55d8db1f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52c1369640 1 --2- 172.21.15.33:0/3964895198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 0x55d8db1f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52c1369640 1 -- 172.21.15.33:0/3964895198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 msgr2=0x55d8db211a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52c1369640 1 --2- 172.21.15.33:0/3964895198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 0x55d8db211a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52c1369640 1 -- 172.21.15.33:0/3964895198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8db1ea120 con 0x55d8db1f0be0 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52c0b68640 1 --2- 172.21.15.33:0/3964895198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 0x55d8db211a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52c1369640 1 --2- 172.21.15.33:0/3964895198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db1f7080 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f52b801aea0 tx=0x7f52b800f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0fe083087b416f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52a6ffd640 1 -- 172.21.15.33:0/3964895198 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b802e020 con 0x55d8db1f0be0 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52a6ffd640 1 -- 172.21.15.33:0/3964895198 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f52b800fe50 con 0x55d8db1f0be0 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52a6ffd640 1 -- 172.21.15.33:0/3964895198 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8000e60 con 0x55d8db1f0be0 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52c61b1740 1 -- 172.21.15.33:0/3964895198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 msgr2=0x55d8db1f7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3964895198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db1f7080 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f52b801aea0 tx=0x7f52b800f6f0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52c61b1740 1 -- 172.21.15.33:0/3964895198 shutdown_connections 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.403+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3964895198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 0x55d8db211a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3964895198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db1f7080 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3964895198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 0x55d8db1f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 1 -- 172.21.15.33:0/3964895198 >> 172.21.15.33:0/3964895198 conn(0x55d8db1ea9f0 msgr2=0x55d8db1ebcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 1 -- 172.21.15.33:0/3964895198 shutdown_connections 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 1 -- 172.21.15.33:0/3964895198 wait complete. 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 1 -- start start 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 1 Processor -- start 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 0x55d8db2019b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db201ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 0x55d8db1fe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1af500 con 0x55d8db1f0be0 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1b0970 con 0x55d8db1f97c0 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1afbd0 con 0x55d8db1f02c0 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c61b1740 10 client.0 _ll_register_callbacks cb 0x7f52b800b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c1369640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db201ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c1369640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db201ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36394/0 (socket says 172.21.15.33:36394) 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c1369640 1 -- 172.21.15.33:0/3241599691 learned_addr learned my addr 172.21.15.33:0/3241599691 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.404+0000 7f52c0b68640 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 0x55d8db1fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52c0b68640 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 0x55d8db2019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52c1369640 1 -- 172.21.15.33:0/3241599691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 msgr2=0x55d8db2019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52c1369640 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 0x55d8db2019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52c1369640 1 -- 172.21.15.33:0/3241599691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 msgr2=0x55d8db1fe980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52c1369640 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 0x55d8db1fe980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52c1369640 1 -- 172.21.15.33:0/3241599691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8db1ff180 con 0x55d8db1f0be0 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52c0b68640 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 0x55d8db2019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52c1369640 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db201ef0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f52b8012480 tx=0x7f52b800b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ee1a541c4feeab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52a4ff9640 1 -- 172.21.15.33:0/3241599691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8037020 con 0x55d8db1f0be0 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52a4ff9640 1 -- 172.21.15.33:0/3241599691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f52b8004030 con 0x55d8db1f0be0 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52c61b1740 1 -- 172.21.15.33:0/3241599691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b800fe50 con 0x55d8db1f0be0 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52a4ff9640 1 -- 172.21.15.33:0/3241599691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f52b803f050 con 0x55d8db1f0be0 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52c61b1740 10 client.16965 fetch_fsmap learned FSMap version 35 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52c61b1740 1 -- 172.21.15.33:0/3241599691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d8db270ca0 con 0x55d8db1f0be0 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.405+0000 7f52a4ff9640 1 -- 172.21.15.33:0/3241599691 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f52b800fe50 con 0x55d8db1f0be0 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.406+0000 7f52a4ff9640 1 -- 172.21.15.33:0/3241599691 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f52b8019070 con 0x55d8db1f0be0 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.406+0000 7f52c61b1740 10 client.16965 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.406+0000 7f52c61b1740 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.406+0000 7f52c61b1740 20 client.16965 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.406+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d8db2712e0 0x55d8db273700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.406+0000 7f52c61b1740 4 client.16965 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:44:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.406+0000 7f52c61b1740 1 -- 172.21.15.33:0/3241599691 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d8db15c000 con 0x55d8db2712e0 2024-04-02T13:44:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.406+0000 7f52c0b68640 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d8db2712e0 0x55d8db273700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.406+0000 7f52c0b68640 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d8db2712e0 0x55d8db273700 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d89671a397b71a5 server_cookie=f9f1229ba46da79c in_seq=0 out_seq=0 2024-04-02T13:44:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.406+0000 7f52a4ff9640 10 client.16965 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:51.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.407+0000 7f52a4ff9640 1 -- 172.21.15.33:0/3241599691 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d8db15c000 con 0x55d8db2712e0 2024-04-02T13:44:51.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.407+0000 7f52a4ff9640 10 client.16965 handle_command_reply: tid=0 2024-04-02T13:44:51.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.407+0000 7f52c61b1740 1 client.16965 shutdown 2024-04-02T13:44:51.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c61b1740 1 -- 172.21.15.33:0/3241599691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 msgr2=0x55d8db201ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db201ef0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f52b8012480 tx=0x7f52b800b5a0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c61b1740 1 -- 172.21.15.33:0/3241599691 shutdown_connections 2024-04-02T13:44:51.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55d8db2712e0 0x55d8db273700 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d8db1f97c0 0x55d8db1fe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52a4ff9640 0 client.16965 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:51.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8db1f0be0 0x55d8db201ef0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c61b1740 1 --2- 172.21.15.33:0/3241599691 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d8db1f02c0 0x55d8db2019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c61b1740 1 -- 172.21.15.33:0/3241599691 >> 172.21.15.33:0/3241599691 conn(0x55d8db1ea9f0 msgr2=0x55d8db1ec410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:51.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c61b1740 1 -- 172.21.15.33:0/3241599691 shutdown_connections 2024-04-02T13:44:51.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c61b1740 1 -- 172.21.15.33:0/3241599691 wait complete. 2024-04-02T13:44:51.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c61b1740 20 client.16965 trim_cache size 0 max 16384 2024-04-02T13:44:51.417 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:44:51.418 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:44:51.418 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "9201f237-2612-4579-baae-ffde0de44404", 2024-04-02T13:44:51.418 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:44:51.418 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:44:51.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c35cd640 1 -- 172.21.15.33:0/1766483784 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f52a80a3d20 msgr2=0x7f52a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c35cd640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f52a80a3d20 0x7f52a80a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f52b40997e0 tx=0x7f52b406e850 comp rx=0 tx=0).stop 2024-04-02T13:44:51.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c35cd640 1 -- 172.21.15.33:0/1766483784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52bc10c140 msgr2=0x7f52bc101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c35cd640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52bc10c140 0x7f52bc101440 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f52b400cce0 tx=0x7f52b400a620 comp rx=0 tx=0).stop 2024-04-02T13:44:51.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c35cd640 1 -- 172.21.15.33:0/1766483784 shutdown_connections 2024-04-02T13:44:51.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c35cd640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f52a80a3d20 0x7f52a80a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c35cd640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52bc10c140 0x7f52bc101440 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c35cd640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52bc103690 0x7f52bc100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c35cd640 1 --2- 172.21.15.33:0/1766483784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52bc102d70 0x7f52bc1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.408+0000 7f52c35cd640 1 -- 172.21.15.33:0/1766483784 >> 172.21.15.33:0/1766483784 conn(0x7f52bc077b60 msgr2=0x7f52bc0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:51.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.409+0000 7f52c35cd640 1 -- 172.21.15.33:0/1766483784 shutdown_connections 2024-04-02T13:44:51.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.409+0000 7f52c35cd640 1 -- 172.21.15.33:0/1766483784 wait complete. 2024-04-02T13:44:51.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:44:51.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:44:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.524+0000 7ff3cb596640 1 -- 172.21.15.33:0/1688442738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c410e4a0 msgr2=0x7ff3c4110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.524+0000 7ff3cb596640 1 --2- 172.21.15.33:0/1688442738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c410e4a0 0x7ff3c4110890 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009ca0 tx=0x7ff3c002c180 comp rx=0 tx=0).stop 2024-04-02T13:44:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.524+0000 7ff3cb596640 1 -- 172.21.15.33:0/1688442738 shutdown_connections 2024-04-02T13:44:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.524+0000 7ff3cb596640 1 --2- 172.21.15.33:0/1688442738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c410e4a0 0x7ff3c4110890 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.524+0000 7ff3cb596640 1 --2- 172.21.15.33:0/1688442738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3c4069620 0x7ff3c410df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.524+0000 7ff3cb596640 1 --2- 172.21.15.33:0/1688442738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3c4068d00 0x7ff3c40690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.524+0000 7ff3cb596640 1 -- 172.21.15.33:0/1688442738 >> 172.21.15.33:0/1688442738 conn(0x7ff3c4077b00 msgr2=0x7ff3c4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.524+0000 7ff3cb596640 1 -- 172.21.15.33:0/1688442738 shutdown_connections 2024-04-02T13:44:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.525+0000 7ff3cb596640 1 -- 172.21.15.33:0/1688442738 wait complete. 2024-04-02T13:44:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.525+0000 7ff3cb596640 1 Processor -- start 2024-04-02T13:44:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.525+0000 7ff3cb596640 1 -- start start 2024-04-02T13:44:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.525+0000 7ff3cb596640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3c4068d00 0x7ff3c419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.525+0000 7ff3cb596640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3c4069620 0x7ff3c419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.525+0000 7ff3cb596640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c410e4a0 0x7ff3c419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.525+0000 7ff3cb596640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c4114040 con 0x7ff3c410e4a0 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.525+0000 7ff3cb596640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c4113ec0 con 0x7ff3c4068d00 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.525+0000 7ff3cb596640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41141c0 con 0x7ff3c4069620 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.525+0000 7ff3c9b33640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c410e4a0 0x7ff3c419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.525+0000 7ff3c9b33640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c410e4a0 0x7ff3c419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36424/0 (socket says 172.21.15.33:36424) 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.525+0000 7ff3c9b33640 1 -- 172.21.15.33:0/3423427620 learned_addr learned my addr 172.21.15.33:0/3423427620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.525+0000 7ff3c9332640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3c4068d00 0x7ff3c419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.526+0000 7ff3c8b31640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3c4069620 0x7ff3c419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.526+0000 7ff3c9b33640 1 -- 172.21.15.33:0/3423427620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3c4069620 msgr2=0x7ff3c419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.526+0000 7ff3c9b33640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3c4069620 0x7ff3c419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.526+0000 7ff3c9b33640 1 -- 172.21.15.33:0/3423427620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3c4068d00 msgr2=0x7ff3c419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.526+0000 7ff3c9b33640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3c4068d00 0x7ff3c419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.526+0000 7ff3c9b33640 1 -- 172.21.15.33:0/3423427620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c419f490 con 0x7ff3c410e4a0 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.526+0000 7ff3c9b33640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c410e4a0 0x7ff3c419ed60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0004af0 tx=0x7ff3c0034ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:51.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.526+0000 7ff3ba7fc640 1 -- 172.21.15.33:0/3423427620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0039050 con 0x7ff3c410e4a0 2024-04-02T13:44:51.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.526+0000 7ff3ba7fc640 1 -- 172.21.15.33:0/3423427620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff3c0045070 con 0x7ff3c410e4a0 2024-04-02T13:44:51.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.526+0000 7ff3cb596640 1 -- 172.21.15.33:0/3423427620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c419f720 con 0x7ff3c410e4a0 2024-04-02T13:44:51.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.526+0000 7ff3ba7fc640 1 -- 172.21.15.33:0/3423427620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0032b30 con 0x7ff3c410e4a0 2024-04-02T13:44:51.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.526+0000 7ff3cb596640 1 -- 172.21.15.33:0/3423427620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c41a7010 con 0x7ff3c410e4a0 2024-04-02T13:44:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.528+0000 7ff3ba7fc640 1 -- 172.21.15.33:0/3423427620 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff3c00046f0 con 0x7ff3c410e4a0 2024-04-02T13:44:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3ce177740 1 Processor -- start 2024-04-02T13:44:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3ce177740 1 -- start start 2024-04-02T13:44:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3ce177740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f248d0 0x55cf88f24cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3ce177740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f251f0 0x55cf88f2d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3ce177740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f2ddd0 0x55cf88f301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3ce177740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cf88e88830 con 0x55cf88f2ddd0 2024-04-02T13:44:51.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3ce177740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55cf88e30820 con 0x55cf88f251f0 2024-04-02T13:44:51.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3ce177740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55cf88e847e0 con 0x55cf88f248d0 2024-04-02T13:44:51.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3ba7fc640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff3b00a3da0 0x7ff3b00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3c8b31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f2ddd0 0x55cf88f301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3c8b31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f2ddd0 0x55cf88f301c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36428/0 (socket says 172.21.15.33:36428) 2024-04-02T13:44:51.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3c8b31640 1 -- 172.21.15.33:0/4275200321 learned_addr learned my addr 172.21.15.33:0/4275200321 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:51.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3c9332640 1 --2- 172.21.15.33:0/4275200321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f251f0 0x55cf88f2d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3c9b33640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff3b00a3da0 0x7ff3b00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3c8b31640 1 --2- 172.21.15.33:0/4275200321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f248d0 0x55cf88f24cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3ba7fc640 1 -- 172.21.15.33:0/3423427620 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff3c00f3420 con 0x7ff3c410e4a0 2024-04-02T13:44:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3c9332640 1 -- 172.21.15.33:0/4275200321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f248d0 msgr2=0x55cf88f24cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3c9332640 1 --2- 172.21.15.33:0/4275200321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f248d0 0x55cf88f24cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3c9332640 1 -- 172.21.15.33:0/4275200321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f2ddd0 msgr2=0x55cf88f301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3c9332640 1 --2- 172.21.15.33:0/4275200321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f2ddd0 0x55cf88f301c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3c9332640 1 -- 172.21.15.33:0/4275200321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf88c667d0 con 0x55cf88f251f0 2024-04-02T13:44:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.529+0000 7ff3c8b31640 1 --2- 172.21.15.33:0/4275200321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f2ddd0 0x55cf88f301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.530+0000 7ff3c9332640 1 --2- 172.21.15.33:0/4275200321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f251f0 0x55cf88f2d890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00ab40 tx=0x7ff3bc00f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88200e1dc07f8433 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.530+0000 7ff3c9b33640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff3b00a3da0 0x7ff3b00a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00bbb30 tx=0x7ff3c0089c00 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.530+0000 7ff3af7fe640 1 -- 172.21.15.33:0/4275200321 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc004020 con 0x55cf88f251f0 2024-04-02T13:44:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.530+0000 7ff3af7fe640 1 -- 172.21.15.33:0/4275200321 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff3bc0041c0 con 0x55cf88f251f0 2024-04-02T13:44:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.530+0000 7ff3af7fe640 1 -- 172.21.15.33:0/4275200321 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc018770 con 0x55cf88f251f0 2024-04-02T13:44:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 -- 172.21.15.33:0/4275200321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f251f0 msgr2=0x55cf88f2d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 --2- 172.21.15.33:0/4275200321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f251f0 0x55cf88f2d890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00ab40 tx=0x7ff3bc00f3c0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 -- 172.21.15.33:0/4275200321 shutdown_connections 2024-04-02T13:44:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 --2- 172.21.15.33:0/4275200321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f2ddd0 0x55cf88f301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 --2- 172.21.15.33:0/4275200321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f251f0 0x55cf88f2d890 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 --2- 172.21.15.33:0/4275200321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f248d0 0x55cf88f24cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 -- 172.21.15.33:0/4275200321 >> 172.21.15.33:0/4275200321 conn(0x55cf88f1f000 msgr2=0x55cf88f1f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 -- 172.21.15.33:0/4275200321 shutdown_connections 2024-04-02T13:44:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 -- 172.21.15.33:0/4275200321 wait complete. 2024-04-02T13:44:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 -- start start 2024-04-02T13:44:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 Processor -- start 2024-04-02T13:44:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f248d0 0x55cf88f2d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f251f0 0x55cf88f2a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f2ddd0 0x55cf88f2aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cf88f30bc0 con 0x55cf88f248d0 2024-04-02T13:44:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55cf88e43d30 con 0x55cf88f251f0 2024-04-02T13:44:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55cf88f30d40 con 0x55cf88f2ddd0 2024-04-02T13:44:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.531+0000 7ff3ce177740 10 client.0 _ll_register_callbacks cb 0x55cf88e43720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:51.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3c8b31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f248d0 0x55cf88f2d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3c8b31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f248d0 0x55cf88f2d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36444/0 (socket says 172.21.15.33:36444) 2024-04-02T13:44:51.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3c8b31640 1 -- 172.21.15.33:0/3572198712 learned_addr learned my addr 172.21.15.33:0/3572198712 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:51.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3c9332640 1 --2- 172.21.15.33:0/3572198712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f251f0 0x55cf88f2a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3c8b31640 1 -- 172.21.15.33:0/3572198712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f2ddd0 msgr2=0x55cf88f2aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3c8b31640 1 --2- 172.21.15.33:0/3572198712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f2ddd0 0x55cf88f2aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3c8b31640 1 -- 172.21.15.33:0/3572198712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f251f0 msgr2=0x55cf88f2a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3c8b31640 1 --2- 172.21.15.33:0/3572198712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f251f0 0x55cf88f2a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3c8b31640 1 -- 172.21.15.33:0/3572198712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf88b9c3f0 con 0x55cf88f248d0 2024-04-02T13:44:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3c8b31640 1 --2- 172.21.15.33:0/3572198712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f248d0 0x55cf88f2d5f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4005a70 tx=0x7ff3b40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fc09b061d38560 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3acff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3acff9640 1 -- 172.21.15.33:0/3572198712 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4005220 con 0x55cf88f248d0 2024-04-02T13:44:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3acff9640 1 -- 172.21.15.33:0/3572198712 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff3b40053c0 con 0x55cf88f248d0 2024-04-02T13:44:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3ce177740 1 -- 172.21.15.33:0/3572198712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3bc0041c0 con 0x55cf88f248d0 2024-04-02T13:44:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3acff9640 1 -- 172.21.15.33:0/3572198712 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff3b40055c0 con 0x55cf88f248d0 2024-04-02T13:44:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3ce177740 10 client.16977 fetch_fsmap learned FSMap version 35 2024-04-02T13:44:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.532+0000 7ff3ce177740 1 -- 172.21.15.33:0/3572198712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cf88b5c0f0 con 0x55cf88f248d0 2024-04-02T13:44:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.533+0000 7ff3acff9640 1 -- 172.21.15.33:0/3572198712 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff3bc0041c0 con 0x55cf88f248d0 2024-04-02T13:44:51.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.533+0000 7ff3acff9640 1 -- 172.21.15.33:0/3572198712 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7ff3b400f040 con 0x55cf88f248d0 2024-04-02T13:44:51.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.533+0000 7ff3ce177740 10 client.16977 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T13:44:51.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.533+0000 7ff3ce177740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:51.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.533+0000 7ff3ce177740 20 client.16977 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:51.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.533+0000 7ff3ce177740 1 --2- 172.21.15.33:0/3572198712 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cf88f3dd10 0x55cf88f40140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.533+0000 7ff3ce177740 4 client.16977 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:51.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.533+0000 7ff3ce177740 1 -- 172.21.15.33:0/3572198712 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf88e95d70 con 0x55cf88f3dd10 2024-04-02T13:44:51.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.533+0000 7ff3c9332640 1 --2- 172.21.15.33:0/3572198712 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cf88f3dd10 0x55cf88f40140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.533+0000 7ff3acff9640 10 client.16977 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:51.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.533+0000 7ff3c9332640 1 --2- 172.21.15.33:0/3572198712 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cf88f3dd10 0x55cf88f40140 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33a7710a1d3f832c server_cookie=aef8df1dee1f9957 in_seq=0 out_seq=0 2024-04-02T13:44:51.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3acff9640 1 -- 172.21.15.33:0/3572198712 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf88e95d70 con 0x55cf88f3dd10 2024-04-02T13:44:51.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3acff9640 10 client.16977 handle_command_reply: tid=0 2024-04-02T13:44:51.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3ce177740 1 client.16977 shutdown 2024-04-02T13:44:51.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3ce177740 1 -- 172.21.15.33:0/3572198712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f248d0 msgr2=0x55cf88f2d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3ce177740 1 --2- 172.21.15.33:0/3572198712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f248d0 0x55cf88f2d5f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4005a70 tx=0x7ff3b40117b0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3ce177740 1 -- 172.21.15.33:0/3572198712 shutdown_connections 2024-04-02T13:44:51.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3ce177740 1 --2- 172.21.15.33:0/3572198712 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cf88f3dd10 0x55cf88f40140 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3ce177740 1 --2- 172.21.15.33:0/3572198712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f2ddd0 0x55cf88f2aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3ce177740 1 --2- 172.21.15.33:0/3572198712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f251f0 0x55cf88f2a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3acff9640 0 client.16977 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:51.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3ce177740 1 --2- 172.21.15.33:0/3572198712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f248d0 0x55cf88f2d5f0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3ce177740 1 -- 172.21.15.33:0/3572198712 >> 172.21.15.33:0/3572198712 conn(0x55cf88f1f000 msgr2=0x55cf88f20d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:51.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3ce177740 1 -- 172.21.15.33:0/3572198712 shutdown_connections 2024-04-02T13:44:51.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3ce177740 1 -- 172.21.15.33:0/3572198712 wait complete. 2024-04-02T13:44:51.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.534+0000 7ff3ce177740 20 client.16977 trim_cache size 0 max 16384 2024-04-02T13:44:51.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.552+0000 7ff3ce177740 1 Processor -- start 2024-04-02T13:44:51.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.552+0000 7ff3ce177740 1 -- start start 2024-04-02T13:44:51.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.552+0000 7ff3ce177740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f248d0 0x55cf88f39ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.552+0000 7ff3ce177740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 0x55cf88f1d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.552+0000 7ff3ce177740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f2ddd0 0x55cf88f2d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.552+0000 7ff3ce177740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41b3930 con 0x55cf88f251f0 2024-04-02T13:44:51.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.552+0000 7ff3ce177740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41b11d0 con 0x55cf88f248d0 2024-04-02T13:44:51.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.552+0000 7ff3ce177740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41af000 con 0x55cf88f2ddd0 2024-04-02T13:44:51.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.552+0000 7ff3c9332640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 0x55cf88f1d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.552+0000 7ff3c9332640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 0x55cf88f1d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36454/0 (socket says 172.21.15.33:36454) 2024-04-02T13:44:51.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.552+0000 7ff3c9332640 1 -- 172.21.15.33:0/840445374 learned_addr learned my addr 172.21.15.33:0/840445374 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:51.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3c8b31640 1 --2- 172.21.15.33:0/840445374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f248d0 0x55cf88f39ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3c9332640 1 -- 172.21.15.33:0/840445374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f2ddd0 msgr2=0x55cf88f2d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3c9332640 1 --2- 172.21.15.33:0/840445374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f2ddd0 0x55cf88f2d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3c9332640 1 -- 172.21.15.33:0/840445374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f248d0 msgr2=0x55cf88f39ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3c9332640 1 --2- 172.21.15.33:0/840445374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f248d0 0x55cf88f39ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3c9332640 1 -- 172.21.15.33:0/840445374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf88f1ee30 con 0x55cf88f251f0 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3c8b31640 1 --2- 172.21.15.33:0/840445374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f248d0 0x55cf88f39ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3c9332640 1 --2- 172.21.15.33:0/840445374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 0x55cf88f1d9b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc003e70 tx=0x7ff3bc003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a5137304acd7b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3aeffd640 1 -- 172.21.15.33:0/840445374 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc030070 con 0x55cf88f251f0 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3aeffd640 1 -- 172.21.15.33:0/840445374 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff3bc00fe00 con 0x55cf88f251f0 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3aeffd640 1 -- 172.21.15.33:0/840445374 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0052a0 con 0x55cf88f251f0 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3ce177740 1 -- 172.21.15.33:0/840445374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 msgr2=0x55cf88f1d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3ce177740 1 --2- 172.21.15.33:0/840445374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 0x55cf88f1d9b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc003e70 tx=0x7ff3bc003bd0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3ce177740 1 -- 172.21.15.33:0/840445374 shutdown_connections 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3ce177740 1 --2- 172.21.15.33:0/840445374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f2ddd0 0x55cf88f2d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.553+0000 7ff3ce177740 1 --2- 172.21.15.33:0/840445374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 0x55cf88f1d9b0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3ce177740 1 --2- 172.21.15.33:0/840445374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f248d0 0x55cf88f39ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3ce177740 1 -- 172.21.15.33:0/840445374 >> 172.21.15.33:0/840445374 conn(0x55cf88f1f000 msgr2=0x55cf88f299b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3ce177740 1 -- 172.21.15.33:0/840445374 shutdown_connections 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3ce177740 1 -- 172.21.15.33:0/840445374 wait complete. 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3ce177740 1 -- start start 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3ce177740 1 Processor -- start 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3ce177740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f248d0 0x55cf88f318e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3ce177740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 0x55cf88f31e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3ce177740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f2ddd0 0x55cf88f32360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3ce177740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41af690 con 0x55cf88f251f0 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3ce177740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41b0b00 con 0x55cf88f248d0 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3ce177740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41afd60 con 0x55cf88f2ddd0 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3ce177740 10 client.0 _ll_register_callbacks cb 0x7ff3b4013930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3c9332640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 0x55cf88f31e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3c9332640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 0x55cf88f31e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36470/0 (socket says 172.21.15.33:36470) 2024-04-02T13:44:51.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.554+0000 7ff3c9332640 1 -- 172.21.15.33:0/209131745 learned_addr learned my addr 172.21.15.33:0/209131745 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3c8b31640 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f248d0 0x55cf88f318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3c8b31640 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f2ddd0 0x55cf88f32360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3c9332640 1 -- 172.21.15.33:0/209131745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f2ddd0 msgr2=0x55cf88f32360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3c9332640 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f2ddd0 0x55cf88f32360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3c9332640 1 -- 172.21.15.33:0/209131745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f248d0 msgr2=0x55cf88f318e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3c9332640 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f248d0 0x55cf88f318e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3c9332640 1 -- 172.21.15.33:0/209131745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf88fa3280 con 0x55cf88f251f0 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3c8b31640 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f248d0 0x55cf88f318e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3c9332640 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 0x55cf88f31e20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0058c0 tx=0x7ff3bc004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f35e3e2194b927 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3acff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3acff9640 1 -- 172.21.15.33:0/209131745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc030030 con 0x55cf88f251f0 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3acff9640 1 -- 172.21.15.33:0/209131745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff3bc0040b0 con 0x55cf88f251f0 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3ce177740 1 -- 172.21.15.33:0/209131745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3bc00fe00 con 0x55cf88f251f0 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3acff9640 1 -- 172.21.15.33:0/209131745 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff3bc0035f0 con 0x55cf88f251f0 2024-04-02T13:44:51.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3ce177740 10 client.16983 fetch_fsmap learned FSMap version 35 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3ce177740 1 -- 172.21.15.33:0/209131745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cf88fa4400 con 0x55cf88f251f0 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3acff9640 1 -- 172.21.15.33:0/209131745 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff3bc00fe00 con 0x55cf88f251f0 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.555+0000 7ff3acff9640 1 -- 172.21.15.33:0/209131745 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7ff3bc02b040 con 0x55cf88f251f0 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.556+0000 7ff3ce177740 10 client.16983 fetch_fsmap finished waiting for FSMap version 35 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.556+0000 7ff3ce177740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.556+0000 7ff3ce177740 20 client.16983 populate_metadata read hostname 'smithi033' 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.556+0000 7ff3ce177740 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cf88fa4a10 0x55cf88fa6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.556+0000 7ff3ce177740 4 client.16983 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.556+0000 7ff3ce177740 1 -- 172.21.15.33:0/209131745 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cf88e900c0 con 0x55cf88fa4a10 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.556+0000 7ff3c8b31640 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cf88fa4a10 0x55cf88fa6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.556+0000 7ff3c8b31640 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cf88fa4a10 0x55cf88fa6e30 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dad196a700b07237 server_cookie=4c63b7401b9a0d74 in_seq=0 out_seq=0 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.556+0000 7ff3acff9640 10 client.16983 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3acff9640 1 -- 172.21.15.33:0/209131745 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cf88e900c0 con 0x55cf88fa4a10 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3acff9640 10 client.16983 handle_command_reply: tid=0 2024-04-02T13:44:51.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3ce177740 1 client.16983 shutdown 2024-04-02T13:44:51.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3ce177740 1 -- 172.21.15.33:0/209131745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 msgr2=0x55cf88f31e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3ce177740 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 0x55cf88f31e20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0058c0 tx=0x7ff3bc004350 comp rx=0 tx=0).stop 2024-04-02T13:44:51.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3ce177740 1 -- 172.21.15.33:0/209131745 shutdown_connections 2024-04-02T13:44:51.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3ce177740 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55cf88fa4a10 0x55cf88fa6e30 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3ce177740 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55cf88f2ddd0 0x55cf88f32360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3acff9640 0 client.16983 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:44:51.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3ce177740 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf88f251f0 0x55cf88f31e20 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3ce177740 1 --2- 172.21.15.33:0/209131745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cf88f248d0 0x55cf88f318e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3ce177740 1 -- 172.21.15.33:0/209131745 >> 172.21.15.33:0/209131745 conn(0x55cf88f1f000 msgr2=0x55cf88f29120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:51.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3ce177740 1 -- 172.21.15.33:0/209131745 shutdown_connections 2024-04-02T13:44:51.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3ce177740 1 -- 172.21.15.33:0/209131745 wait complete. 2024-04-02T13:44:51.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3ce177740 20 client.16983 trim_cache size 0 max 16384 2024-04-02T13:44:51.572 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:44:51.572 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:44:51.572 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:44:51.572 INFO:teuthology.orchestra.run.smithi033.stdout: "9201f237-2612-4579-baae-ffde0de44404": { 2024-04-02T13:44:51.572 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:44:51.572 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "9201f237-2612-4579-baae-ffde0de44404", 2024-04-02T13:44:51.572 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:44:51.572 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:44:51.572 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:44:51.572 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:44:51.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3cb596640 1 -- 172.21.15.33:0/3423427620 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff3b00a3da0 msgr2=0x7ff3b00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3cb596640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff3b00a3da0 0x7ff3b00a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00bbb30 tx=0x7ff3c0089c00 comp rx=0 tx=0).stop 2024-04-02T13:44:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3cb596640 1 -- 172.21.15.33:0/3423427620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c410e4a0 msgr2=0x7ff3c419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3cb596640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c410e4a0 0x7ff3c419ed60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0004af0 tx=0x7ff3c0034ed0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3cb596640 1 -- 172.21.15.33:0/3423427620 shutdown_connections 2024-04-02T13:44:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3cb596640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff3b00a3da0 0x7ff3b00a6260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3cb596640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c410e4a0 0x7ff3c419ed60 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3cb596640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3c4069620 0x7ff3c419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.557+0000 7ff3cb596640 1 --2- 172.21.15.33:0/3423427620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3c4068d00 0x7ff3c419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.558+0000 7ff3cb596640 1 -- 172.21.15.33:0/3423427620 >> 172.21.15.33:0/3423427620 conn(0x7ff3c4077b00 msgr2=0x7ff3c410f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.558+0000 7ff3cb596640 1 -- 172.21.15.33:0/3423427620 shutdown_connections 2024-04-02T13:44:51.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:44:51.558+0000 7ff3cb596640 1 -- 172.21.15.33:0/3423427620 wait complete. 2024-04-02T13:44:51.575 INFO:tasks.cephfs.filesystem:scrub status for tag:9201f237-2612-4579-baae-ffde0de44404 - {'path': '/', 'tag': '9201f237-2612-4579-baae-ffde0de44404', 'options': 'recursive,force'} 2024-04-02T13:44:52.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:52 smithi145 ceph-mon[35646]: pgmap v185: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:52.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:52 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:44:52.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:52 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:52.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:52 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:44:52.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:52 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:52.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:52 smithi050 ceph-mon[35524]: pgmap v185: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:52.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:52 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:44:52.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:52 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:52.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:52 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:44:52.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:52 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:53.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:52 smithi033 ceph-mon[32390]: pgmap v185: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:44:53.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:52 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:44:53.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:52 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:53.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:52 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:44:53.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:52 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:54.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:54 smithi145 ceph-mon[35646]: pgmap v186: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:44:54.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:54 smithi050 ceph-mon[35524]: pgmap v186: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:44:55.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:54 smithi033 ceph-mon[32390]: pgmap v186: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:44:56.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:56 smithi145 ceph-mon[35646]: pgmap v187: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:56.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:56 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:44:56.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:56 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:56.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:56 smithi050 ceph-mon[35524]: pgmap v187: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:56.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:56 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:44:56.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:56 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:57.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:56 smithi033 ceph-mon[32390]: pgmap v187: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:57.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:56 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:44:57.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:56 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:57.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:57 smithi145 ceph-mon[35646]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:57.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:57 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:57.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:57 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:44:57.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:57 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:44:57.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:57 smithi050 ceph-mon[35524]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:57.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:57 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:57.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:57 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:44:57.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:57 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:44:58.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:57 smithi033 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-04-02T13:44:58.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:57 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:58.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:57 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:44:58.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:57 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:44:58.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:44:58 smithi145 ceph-mon[35646]: pgmap v188: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:58.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:44:58 smithi050 ceph-mon[35524]: pgmap v188: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:44:59.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:44:58 smithi033 ceph-mon[32390]: pgmap v188: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:45:00.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:00 smithi145 ceph-mon[35646]: pgmap v189: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:45:00.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:00 smithi050 ceph-mon[35524]: pgmap v189: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:45:01.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:00 smithi033 ceph-mon[32390]: pgmap v189: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:45:02.983 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:02 smithi145 ceph-mon[35646]: pgmap v190: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:45:02.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:02 smithi050 ceph-mon[35524]: pgmap v190: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:45:03.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:02 smithi033 ceph-mon[32390]: pgmap v190: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:45:04.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:04 smithi145 ceph-mon[35646]: pgmap v191: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-02T13:45:04.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:04 smithi050 ceph-mon[35524]: pgmap v191: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-02T13:45:05.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:04 smithi033 ceph-mon[32390]: pgmap v191: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-02T13:45:06.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:06 smithi145 ceph-mon[35646]: pgmap v192: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-02T13:45:06.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:06 smithi050 ceph-mon[35524]: pgmap v192: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-02T13:45:07.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:06 smithi033 ceph-mon[32390]: pgmap v192: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-02T13:45:08.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:08 smithi145 ceph-mon[35646]: pgmap v193: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-02T13:45:08.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:08 smithi050 ceph-mon[35524]: pgmap v193: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-02T13:45:09.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:08 smithi033 ceph-mon[32390]: pgmap v193: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-02T13:45:10.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:10 smithi145 ceph-mon[35646]: pgmap v194: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-02T13:45:10.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:10 smithi050 ceph-mon[35524]: pgmap v194: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-02T13:45:11.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:10 smithi033 ceph-mon[32390]: pgmap v194: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-02T13:45:12.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:12 smithi145 ceph-mon[35646]: pgmap v195: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-02T13:45:12.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:12 smithi145 ceph-mon[35646]: osdmap e96: 12 total, 12 up, 12 in 2024-04-02T13:45:12.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:12 smithi050 ceph-mon[35524]: pgmap v195: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-02T13:45:12.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:12 smithi050 ceph-mon[35524]: osdmap e96: 12 total, 12 up, 12 in 2024-04-02T13:45:13.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:12 smithi033 ceph-mon[32390]: pgmap v195: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-02T13:45:13.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:12 smithi033 ceph-mon[32390]: osdmap e96: 12 total, 12 up, 12 in 2024-04-02T13:45:14.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:14 smithi145 ceph-mon[35646]: pgmap v197: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:45:14.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:14 smithi050 ceph-mon[35524]: pgmap v197: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:45:15.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:14 smithi033 ceph-mon[32390]: pgmap v197: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:45:16.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:16 smithi145 ceph-mon[35646]: pgmap v198: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:45:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:16 smithi145 ceph-mon[35646]: osdmap e97: 12 total, 12 up, 12 in 2024-04-02T13:45:16.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:16 smithi050 ceph-mon[35524]: pgmap v198: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:45:16.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:16 smithi050 ceph-mon[35524]: osdmap e97: 12 total, 12 up, 12 in 2024-04-02T13:45:17.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:16 smithi033 ceph-mon[32390]: pgmap v198: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:45:17.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:16 smithi033 ceph-mon[32390]: osdmap e97: 12 total, 12 up, 12 in 2024-04-02T13:45:17.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:17 smithi050 ceph-mon[35524]: pgmap v200: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-02T13:45:18.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:17 smithi033 ceph-mon[32390]: pgmap v200: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-02T13:45:18.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:17 smithi145 ceph-mon[35646]: pgmap v200: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-02T13:45:20.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:20 smithi145 ceph-mon[35646]: pgmap v201: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:45:20.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:20 smithi050 ceph-mon[35524]: pgmap v201: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:45:20.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:20 smithi033 ceph-mon[32390]: pgmap v201: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:45:21.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:45:21.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:45:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.692+0000 7f925c255640 1 -- 172.21.15.33:0/4265946835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92541009b0 msgr2=0x7f9254100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:21.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.692+0000 7f925c255640 1 --2- 172.21.15.33:0/4265946835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92541009b0 0x7f9254100e30 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9244009930 tx=0x7f924402bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:21.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.692+0000 7f925c255640 1 -- 172.21.15.33:0/4265946835 shutdown_connections 2024-04-02T13:45:21.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.692+0000 7f925c255640 1 --2- 172.21.15.33:0/4265946835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9254101370 0x7f925410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.692+0000 7f925c255640 1 --2- 172.21.15.33:0/4265946835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92541009b0 0x7f9254100e30 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.692+0000 7f925c255640 1 --2- 172.21.15.33:0/4265946835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92541069c0 0x7f9254106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.692+0000 7f925c255640 1 -- 172.21.15.33:0/4265946835 >> 172.21.15.33:0/4265946835 conn(0x7f9254076d80 msgr2=0x7f9254077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:21.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.692+0000 7f925c255640 1 -- 172.21.15.33:0/4265946835 shutdown_connections 2024-04-02T13:45:21.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.692+0000 7f925c255640 1 -- 172.21.15.33:0/4265946835 wait complete. 2024-04-02T13:45:21.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f925c255640 1 Processor -- start 2024-04-02T13:45:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f925c255640 1 -- start start 2024-04-02T13:45:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f925c255640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92541009b0 0x7f92541983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f925c255640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9254101370 0x7f9254198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f925c255640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92541069c0 0x7f925419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f925c255640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9254111ef0 con 0x7f92541009b0 2024-04-02T13:45:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f925c255640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9254111d70 con 0x7f9254101370 2024-04-02T13:45:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f925c255640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9254112070 con 0x7f92541069c0 2024-04-02T13:45:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f9259ff1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92541009b0 0x7f92541983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f9259ff1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92541009b0 0x7f92541983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40124/0 (socket says 172.21.15.33:40124) 2024-04-02T13:45:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f9259ff1640 1 -- 172.21.15.33:0/1416895500 learned_addr learned my addr 172.21.15.33:0/1416895500 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f92597f0640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9254101370 0x7f9254198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f925a7f2640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92541069c0 0x7f925419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f9259ff1640 1 -- 172.21.15.33:0/1416895500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92541069c0 msgr2=0x7f925419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f9259ff1640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92541069c0 0x7f925419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f9259ff1640 1 -- 172.21.15.33:0/1416895500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9254101370 msgr2=0x7f9254198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f9259ff1640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9254101370 0x7f9254198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f9259ff1640 1 -- 172.21.15.33:0/1416895500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925419d3e0 con 0x7f92541009b0 2024-04-02T13:45:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.693+0000 7f925a7f2640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92541069c0 0x7f925419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.694+0000 7f9259ff1640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92541009b0 0x7f92541983f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f925000c970 tx=0x7f925000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.694+0000 7f924affd640 1 -- 172.21.15.33:0/1416895500 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250007bb0 con 0x7f92541009b0 2024-04-02T13:45:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.694+0000 7f924affd640 1 -- 172.21.15.33:0/1416895500 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9250007d50 con 0x7f92541009b0 2024-04-02T13:45:21.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.694+0000 7f925c255640 1 -- 172.21.15.33:0/1416895500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925419d6d0 con 0x7f92541009b0 2024-04-02T13:45:21.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.694+0000 7f924affd640 1 -- 172.21.15.33:0/1416895500 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925000f450 con 0x7f92541009b0 2024-04-02T13:45:21.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.694+0000 7f925c255640 1 -- 172.21.15.33:0/1416895500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92541a4f60 con 0x7f92541009b0 2024-04-02T13:45:21.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.696+0000 7f924affd640 1 -- 172.21.15.33:0/1416895500 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9250004fb0 con 0x7f92541009b0 2024-04-02T13:45:21.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.696+0000 7f925ee44740 1 Processor -- start 2024-04-02T13:45:21.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f925ee44740 1 -- start start 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f924affd640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f92400a3da0 0x7f92400a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f925ee44740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c58d0 0x55a2816c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f925ee44740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 0x55a2816ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f925ee44740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816cedd0 0x55a2816d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f925ee44740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a281629830 con 0x55a2816c61f0 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f925ee44740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a2815d1820 con 0x55a2816c58d0 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f925ee44740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a2816257e0 con 0x55a2816cedd0 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f925a7f2640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f92400a3da0 0x7f92400a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f92597f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 0x55a2816ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f924affd640 1 -- 172.21.15.33:0/1416895500 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f92500d0820 con 0x7f92541009b0 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f92597f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 0x55a2816ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40126/0 (socket says 172.21.15.33:40126) 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f92597f0640 1 -- 172.21.15.33:0/2935941605 learned_addr learned my addr 172.21.15.33:0/2935941605 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f9259ff1640 1 --2- 172.21.15.33:0/2935941605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c58d0 0x55a2816c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f92597f0640 1 -- 172.21.15.33:0/2935941605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816cedd0 msgr2=0x55a2816d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f92597f0640 1 --2- 172.21.15.33:0/2935941605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816cedd0 0x55a2816d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f92597f0640 1 -- 172.21.15.33:0/2935941605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c58d0 msgr2=0x55a2816c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f92597f0640 1 --2- 172.21.15.33:0/2935941605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c58d0 0x55a2816c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f92597f0640 1 -- 172.21.15.33:0/2935941605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2814077d0 con 0x55a2816c61f0 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.697+0000 7f925a7f2640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f92400a3da0 0x7f92400a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f924c005fd0 tx=0x7f924c009980 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.698+0000 7f92597f0640 1 --2- 172.21.15.33:0/2935941605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 0x55a2816ce890 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9244009930 tx=0x7f9244004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55a898ba090dba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.698+0000 7f923ffff640 1 -- 172.21.15.33:0/2935941605 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244045020 con 0x55a2816c61f0 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.698+0000 7f923ffff640 1 -- 172.21.15.33:0/2935941605 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9244005510 con 0x55a2816c61f0 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.698+0000 7f923ffff640 1 -- 172.21.15.33:0/2935941605 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92440046c0 con 0x55a2816c61f0 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.698+0000 7f925ee44740 1 -- 172.21.15.33:0/2935941605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 msgr2=0x55a2816ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.698+0000 7f925ee44740 1 --2- 172.21.15.33:0/2935941605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 0x55a2816ce890 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9244009930 tx=0x7f9244004170 comp rx=0 tx=0).stop 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.698+0000 7f925ee44740 1 -- 172.21.15.33:0/2935941605 shutdown_connections 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.698+0000 7f925ee44740 1 --2- 172.21.15.33:0/2935941605 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816cedd0 0x55a2816d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.698+0000 7f925ee44740 1 --2- 172.21.15.33:0/2935941605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 0x55a2816ce890 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.698+0000 7f925ee44740 1 --2- 172.21.15.33:0/2935941605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c58d0 0x55a2816c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.698+0000 7f925ee44740 1 -- 172.21.15.33:0/2935941605 >> 172.21.15.33:0/2935941605 conn(0x55a2816c0000 msgr2=0x55a2816c03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.699+0000 7f925ee44740 1 -- 172.21.15.33:0/2935941605 shutdown_connections 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.699+0000 7f925ee44740 1 -- 172.21.15.33:0/2935941605 wait complete. 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.699+0000 7f925ee44740 1 -- start start 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.699+0000 7f925ee44740 1 Processor -- start 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.699+0000 7f925ee44740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c58d0 0x55a2816ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.699+0000 7f925ee44740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 0x55a2816cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.699+0000 7f925ee44740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816cedd0 0x55a2816cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.699+0000 7f925ee44740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a2816d1b60 con 0x55a2816c61f0 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.699+0000 7f925ee44740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a2815e4d30 con 0x55a2816cedd0 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.699+0000 7f925ee44740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a2816d1ce0 con 0x55a2816c58d0 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f925ee44740 10 client.0 _ll_register_callbacks cb 0x55a2815e4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f92597f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 0x55a2816cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f92597f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 0x55a2816cb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40132/0 (socket says 172.21.15.33:40132) 2024-04-02T13:45:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f92597f0640 1 -- 172.21.15.33:0/3223369602 learned_addr learned my addr 172.21.15.33:0/3223369602 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f925a7f2640 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816cedd0 0x55a2816cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f9259ff1640 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c58d0 0x55a2816ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f92597f0640 1 -- 172.21.15.33:0/3223369602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c58d0 msgr2=0x55a2816ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f92597f0640 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c58d0 0x55a2816ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f92597f0640 1 -- 172.21.15.33:0/3223369602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816cedd0 msgr2=0x55a2816cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f92597f0640 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816cedd0 0x55a2816cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f92597f0640 1 -- 172.21.15.33:0/3223369602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a28133d3f0 con 0x55a2816c61f0 2024-04-02T13:45:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f9259ff1640 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c58d0 0x55a2816ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f92597f0640 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 0x55a2816cb990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f92440096c0 tx=0x7f92440045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7148bd2d77d6b419 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f923d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f923d7fa640 1 -- 172.21.15.33:0/3223369602 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924404e070 con 0x55a2816c61f0 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f923d7fa640 1 -- 172.21.15.33:0/3223369602 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f9244003a00 con 0x55a2816c61f0 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f925ee44740 1 -- 172.21.15.33:0/3223369602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9244005510 con 0x55a2816c61f0 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f923d7fa640 1 -- 172.21.15.33:0/3223369602 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f924405a050 con 0x55a2816c61f0 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f925ee44740 10 client.16995 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.700+0000 7f925ee44740 1 -- 172.21.15.33:0/3223369602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a2812fd0f0 con 0x55a2816c61f0 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.701+0000 7f923d7fa640 1 -- 172.21.15.33:0/3223369602 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9244005510 con 0x55a2816c61f0 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.701+0000 7f923d7fa640 1 -- 172.21.15.33:0/3223369602 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f9244041c80 con 0x55a2816c61f0 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.701+0000 7f925ee44740 10 client.16995 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.701+0000 7f925ee44740 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.701+0000 7f925ee44740 20 client.16995 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.701+0000 7f925ee44740 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a2816deb60 0x55a2816e0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.701+0000 7f925ee44740 4 client.16995 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.701+0000 7f925ee44740 1 -- 172.21.15.33:0/3223369602 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a281636d70 con 0x55a2816deb60 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.701+0000 7f9259ff1640 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a2816deb60 0x55a2816e0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.702+0000 7f923d7fa640 10 client.16995 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.702+0000 7f9259ff1640 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a2816deb60 0x55a2816e0f50 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddaa2a25a6c514d9 server_cookie=b98d1615d5186fe1 in_seq=0 out_seq=0 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.702+0000 7f923d7fa640 1 -- 172.21.15.33:0/3223369602 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a281636d70 con 0x55a2816deb60 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.702+0000 7f923d7fa640 10 client.16995 handle_command_reply: tid=0 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.702+0000 7f925ee44740 1 client.16995 shutdown 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.703+0000 7f925ee44740 1 -- 172.21.15.33:0/3223369602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 msgr2=0x55a2816cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.703+0000 7f925ee44740 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 0x55a2816cb990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f92440096c0 tx=0x7f92440045d0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.703+0000 7f925ee44740 1 -- 172.21.15.33:0/3223369602 shutdown_connections 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.703+0000 7f925ee44740 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a2816deb60 0x55a2816e0f50 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.703+0000 7f925ee44740 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816cedd0 0x55a2816cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.703+0000 7f925ee44740 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c61f0 0x55a2816cb990 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.703+0000 7f923d7fa640 0 client.16995 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.703+0000 7f925ee44740 1 --2- 172.21.15.33:0/3223369602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c58d0 0x55a2816ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.703+0000 7f925ee44740 1 -- 172.21.15.33:0/3223369602 >> 172.21.15.33:0/3223369602 conn(0x55a2816c0000 msgr2=0x55a2816c1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.703+0000 7f925ee44740 1 -- 172.21.15.33:0/3223369602 shutdown_connections 2024-04-02T13:45:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.703+0000 7f925ee44740 1 -- 172.21.15.33:0/3223369602 wait complete. 2024-04-02T13:45:21.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.703+0000 7f925ee44740 20 client.16995 trim_cache size 0 max 16384 2024-04-02T13:45:21.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f925ee44740 1 Processor -- start 2024-04-02T13:45:21.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f925ee44740 1 -- start start 2024-04-02T13:45:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f925ee44740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c0000 0x55a2816daa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f925ee44740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c58d0 0x55a2816ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f925ee44740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c61f0 0x55a2816ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f925ee44740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f92541b1840 con 0x55a2816c0000 2024-04-02T13:45:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f925ee44740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f92541af110 con 0x55a2816c61f0 2024-04-02T13:45:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f925ee44740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f92541acf80 con 0x55a2816c58d0 2024-04-02T13:45:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f9259ff1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c0000 0x55a2816daa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f9259ff1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c0000 0x55a2816daa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40148/0 (socket says 172.21.15.33:40148) 2024-04-02T13:45:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f9259ff1640 1 -- 172.21.15.33:0/2762808054 learned_addr learned my addr 172.21.15.33:0/2762808054 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f92597f0640 1 --2- 172.21.15.33:0/2762808054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c58d0 0x55a2816ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f9259ff1640 1 -- 172.21.15.33:0/2762808054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c58d0 msgr2=0x55a2816ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f925a7f2640 1 --2- 172.21.15.33:0/2762808054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c61f0 0x55a2816ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f9259ff1640 1 --2- 172.21.15.33:0/2762808054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c58d0 0x55a2816ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f9259ff1640 1 -- 172.21.15.33:0/2762808054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c61f0 msgr2=0x55a2816ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f9259ff1640 1 --2- 172.21.15.33:0/2762808054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c61f0 0x55a2816ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f9259ff1640 1 -- 172.21.15.33:0/2762808054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2816bfe30 con 0x55a2816c0000 2024-04-02T13:45:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f92597f0640 1 --2- 172.21.15.33:0/2762808054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c58d0 0x55a2816ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.721+0000 7f9259ff1640 1 --2- 172.21.15.33:0/2762808054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c0000 0x55a2816daa80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f92500980d0 tx=0x7f925000a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f664a7a43ae571ed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.722+0000 7f923f7fe640 1 -- 172.21.15.33:0/2762808054 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925001d040 con 0x55a2816c0000 2024-04-02T13:45:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.722+0000 7f923f7fe640 1 -- 172.21.15.33:0/2762808054 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f92500a5060 con 0x55a2816c0000 2024-04-02T13:45:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.722+0000 7f923f7fe640 1 -- 172.21.15.33:0/2762808054 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250098a60 con 0x55a2816c0000 2024-04-02T13:45:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.722+0000 7f925ee44740 1 -- 172.21.15.33:0/2762808054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c0000 msgr2=0x55a2816daa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.722+0000 7f925ee44740 1 --2- 172.21.15.33:0/2762808054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c0000 0x55a2816daa80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f92500980d0 tx=0x7f925000a540 comp rx=0 tx=0).stop 2024-04-02T13:45:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.722+0000 7f925ee44740 1 -- 172.21.15.33:0/2762808054 shutdown_connections 2024-04-02T13:45:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.722+0000 7f925ee44740 1 --2- 172.21.15.33:0/2762808054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c61f0 0x55a2816ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.722+0000 7f925ee44740 1 --2- 172.21.15.33:0/2762808054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c58d0 0x55a2816ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.722+0000 7f925ee44740 1 --2- 172.21.15.33:0/2762808054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c0000 0x55a2816daa80 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.722+0000 7f925ee44740 1 -- 172.21.15.33:0/2762808054 >> 172.21.15.33:0/2762808054 conn(0x55a2816ca5d0 msgr2=0x55a2816c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.722+0000 7f925ee44740 1 -- 172.21.15.33:0/2762808054 shutdown_connections 2024-04-02T13:45:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.722+0000 7f925ee44740 1 -- 172.21.15.33:0/2762808054 wait complete. 2024-04-02T13:45:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.722+0000 7f925ee44740 1 -- start start 2024-04-02T13:45:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f925ee44740 1 Processor -- start 2024-04-02T13:45:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f925ee44740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c0000 0x55a2816d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f925ee44740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c58d0 0x55a2816d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f925ee44740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c61f0 0x55a281743d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f925ee44740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f92541ad5d0 con 0x55a2816c58d0 2024-04-02T13:45:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f925ee44740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f92541aea40 con 0x55a2816c61f0 2024-04-02T13:45:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f925ee44740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f92541adca0 con 0x55a2816c0000 2024-04-02T13:45:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f925ee44740 10 client.0 _ll_register_callbacks cb 0x7f924402c960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f92597f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c58d0 0x55a2816d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f92597f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c58d0 0x55a2816d3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40150/0 (socket says 172.21.15.33:40150) 2024-04-02T13:45:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f92597f0640 1 -- 172.21.15.33:0/3160286174 learned_addr learned my addr 172.21.15.33:0/3160286174 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f9259ff1640 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c0000 0x55a2816d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f925a7f2640 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c61f0 0x55a281743d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f92597f0640 1 -- 172.21.15.33:0/3160286174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c0000 msgr2=0x55a2816d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f92597f0640 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c0000 0x55a2816d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f92597f0640 1 -- 172.21.15.33:0/3160286174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c61f0 msgr2=0x55a281743d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f92597f0640 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c61f0 0x55a281743d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f92597f0640 1 -- 172.21.15.33:0/3160286174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2816d3b30 con 0x55a2816c58d0 2024-04-02T13:45:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f9259ff1640 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c0000 0x55a2816d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.723+0000 7f925a7f2640 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c61f0 0x55a281743d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f92597f0640 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c58d0 0x55a2816d3230 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f924402c170 tx=0x7f924402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da9f9b60deaf256 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f923d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f923d7fa640 1 -- 172.21.15.33:0/3160286174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924404e040 con 0x55a2816c58d0 2024-04-02T13:45:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f923d7fa640 1 -- 172.21.15.33:0/3160286174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f924404ab20 con 0x55a2816c58d0 2024-04-02T13:45:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f925ee44740 1 -- 172.21.15.33:0/3160286174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92500a5060 con 0x55a2816c58d0 2024-04-02T13:45:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f923d7fa640 1 -- 172.21.15.33:0/3160286174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9244067050 con 0x55a2816c58d0 2024-04-02T13:45:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f925ee44740 10 client.17001 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f925ee44740 1 -- 172.21.15.33:0/3160286174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a281745320 con 0x55a2816c58d0 2024-04-02T13:45:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f923d7fa640 1 -- 172.21.15.33:0/3160286174 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f92500a5060 con 0x55a2816c58d0 2024-04-02T13:45:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f923d7fa640 1 -- 172.21.15.33:0/3160286174 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f9244041510 con 0x55a2816c58d0 2024-04-02T13:45:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f925ee44740 10 client.17001 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f925ee44740 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f925ee44740 20 client.17001 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f925ee44740 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a2816cedd0 0x55a281747a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f925ee44740 4 client.17001 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:45:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f925ee44740 1 -- 172.21.15.33:0/3160286174 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a2816310c0 con 0x55a2816cedd0 2024-04-02T13:45:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.724+0000 7f9259ff1640 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a2816cedd0 0x55a281747a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f9259ff1640 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a2816cedd0 0x55a281747a20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ab65750679995a server_cookie=6c3b46e11e69574d in_seq=0 out_seq=0 2024-04-02T13:45:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f923d7fa640 10 client.17001 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f923d7fa640 1 -- 172.21.15.33:0/3160286174 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a2816310c0 con 0x55a2816cedd0 2024-04-02T13:45:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f923d7fa640 10 client.17001 handle_command_reply: tid=0 2024-04-02T13:45:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f925ee44740 1 client.17001 shutdown 2024-04-02T13:45:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f925ee44740 1 -- 172.21.15.33:0/3160286174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c58d0 msgr2=0x55a2816d3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f925ee44740 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c58d0 0x55a2816d3230 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f924402c170 tx=0x7f924402c7e0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f925ee44740 1 -- 172.21.15.33:0/3160286174 shutdown_connections 2024-04-02T13:45:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f925ee44740 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55a2816cedd0 0x55a281747a20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f925ee44740 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2816c61f0 0x55a281743d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f925ee44740 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a2816c58d0 0x55a2816d3230 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f925ee44740 1 --2- 172.21.15.33:0/3160286174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a2816c0000 0x55a2816d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f923d7fa640 0 client.17001 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f925ee44740 1 -- 172.21.15.33:0/3160286174 >> 172.21.15.33:0/3160286174 conn(0x55a2816ca5d0 msgr2=0x55a2816be9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f925ee44740 1 -- 172.21.15.33:0/3160286174 shutdown_connections 2024-04-02T13:45:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f925ee44740 1 -- 172.21.15.33:0/3160286174 wait complete. 2024-04-02T13:45:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.725+0000 7f925ee44740 20 client.17001 trim_cache size 0 max 16384 2024-04-02T13:45:21.737 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:45:21.737 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:45:21.737 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:45:21.737 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:45:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.726+0000 7f925c255640 1 -- 172.21.15.33:0/1416895500 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f92400a3da0 msgr2=0x7f92400a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.726+0000 7f925c255640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f92400a3da0 0x7f92400a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f924c005fd0 tx=0x7f924c009980 comp rx=0 tx=0).stop 2024-04-02T13:45:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.726+0000 7f925c255640 1 -- 172.21.15.33:0/1416895500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92541009b0 msgr2=0x7f92541983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.726+0000 7f925c255640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92541009b0 0x7f92541983f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f925000c970 tx=0x7f925000ce30 comp rx=0 tx=0).stop 2024-04-02T13:45:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.726+0000 7f925c255640 1 -- 172.21.15.33:0/1416895500 shutdown_connections 2024-04-02T13:45:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.726+0000 7f925c255640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f92400a3da0 0x7f92400a6260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.726+0000 7f925c255640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92541069c0 0x7f925419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.726+0000 7f925c255640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9254101370 0x7f9254198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.726+0000 7f925c255640 1 --2- 172.21.15.33:0/1416895500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92541009b0 0x7f92541983f0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.726+0000 7f925c255640 1 -- 172.21.15.33:0/1416895500 >> 172.21.15.33:0/1416895500 conn(0x7f9254076d80 msgr2=0x7f92540fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.726+0000 7f925c255640 1 -- 172.21.15.33:0/1416895500 shutdown_connections 2024-04-02T13:45:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.726+0000 7f925c255640 1 -- 172.21.15.33:0/1416895500 wait complete. 2024-04-02T13:45:21.740 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:45:21.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:45:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.841+0000 7f3dce3c8640 1 -- 172.21.15.33:0/2497637301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc8107ad0 msgr2=0x7f3dc8107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.841+0000 7f3dce3c8640 1 --2- 172.21.15.33:0/2497637301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc8107ad0 0x7f3dc8107eb0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009930 tx=0x7f3db802bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.841+0000 7f3dce3c8640 1 -- 172.21.15.33:0/2497637301 shutdown_connections 2024-04-02T13:45:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.841+0000 7f3dce3c8640 1 --2- 172.21.15.33:0/2497637301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dc8102480 0x7f3dc810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.841+0000 7f3dce3c8640 1 --2- 172.21.15.33:0/2497637301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3dc8101ac0 0x7f3dc8101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.841+0000 7f3dce3c8640 1 --2- 172.21.15.33:0/2497637301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc8107ad0 0x7f3dc8107eb0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.841+0000 7f3dce3c8640 1 -- 172.21.15.33:0/2497637301 >> 172.21.15.33:0/2497637301 conn(0x7f3dc80fd570 msgr2=0x7f3dc80ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.841+0000 7f3dce3c8640 1 -- 172.21.15.33:0/2497637301 shutdown_connections 2024-04-02T13:45:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.841+0000 7f3dce3c8640 1 -- 172.21.15.33:0/2497637301 wait complete. 2024-04-02T13:45:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dce3c8640 1 Processor -- start 2024-04-02T13:45:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dce3c8640 1 -- start start 2024-04-02T13:45:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dce3c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc8101ac0 0x7f3dc81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dce3c8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3dc8102480 0x7f3dc81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dce3c8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dc8107ad0 0x7f3dc819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dce3c8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8111ed0 con 0x7f3dc8101ac0 2024-04-02T13:45:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dce3c8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8111d50 con 0x7f3dc8102480 2024-04-02T13:45:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dce3c8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8112050 con 0x7f3dc8107ad0 2024-04-02T13:45:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc8101ac0 0x7f3dc81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc8101ac0 0x7f3dc81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40162/0 (socket says 172.21.15.33:40162) 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dc7fff640 1 -- 172.21.15.33:0/109343309 learned_addr learned my addr 172.21.15.33:0/109343309 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dc7fff640 1 -- 172.21.15.33:0/109343309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dc8107ad0 msgr2=0x7f3dc819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dc7fff640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dc8107ad0 0x7f3dc819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dc77fe640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3dc8102480 0x7f3dc81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dc7fff640 1 -- 172.21.15.33:0/109343309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3dc8102480 msgr2=0x7f3dc81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dc7fff640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3dc8102480 0x7f3dc81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dc7fff640 1 -- 172.21.15.33:0/109343309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc819d350 con 0x7f3dc8101ac0 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.842+0000 7f3dc77fe640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3dc8102480 0x7f3dc81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.843+0000 7f3dc7fff640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc8101ac0 0x7f3dc81983b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3db8000c00 tx=0x7f3db80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.843+0000 7f3dc57fa640 1 -- 172.21.15.33:0/109343309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8043020 con 0x7f3dc8101ac0 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.843+0000 7f3dce3c8640 1 -- 172.21.15.33:0/109343309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc819d5e0 con 0x7f3dc8101ac0 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.843+0000 7f3dc57fa640 1 -- 172.21.15.33:0/109343309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3db802caa0 con 0x7f3dc8101ac0 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.843+0000 7f3dc57fa640 1 -- 172.21.15.33:0/109343309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8003c20 con 0x7f3dc8101ac0 2024-04-02T13:45:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.843+0000 7f3dce3c8640 1 -- 172.21.15.33:0/109343309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc81a4ed0 con 0x7f3dc8101ac0 2024-04-02T13:45:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.844+0000 7f3dce3c8640 1 -- 172.21.15.33:0/109343309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc8068d40 con 0x7f3dc8101ac0 2024-04-02T13:45:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.845+0000 7f3dc57fa640 1 -- 172.21.15.33:0/109343309 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3db8046020 con 0x7f3dc8101ac0 2024-04-02T13:45:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.845+0000 7f3dc57fa640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3db00a3d20 0x7f3db00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:21.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.846+0000 7f3dc77fe640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3db00a3d20 0x7f3db00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:21.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.846+0000 7f3dc57fa640 1 -- 172.21.15.33:0/109343309 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f3db80f2790 con 0x7f3dc8101ac0 2024-04-02T13:45:21.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.846+0000 7f3dc77fe640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3db00a3d20 0x7f3db00a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3dc81070c0 tx=0x7f3dbc005d90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:21.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:21.850+0000 7f3dc57fa640 1 -- 172.21.15.33:0/109343309 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3db804d070 con 0x7f3dc8101ac0 2024-04-02T13:45:22.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.079+0000 7f3dce3c8640 1 -- 172.21.15.33:0/109343309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dc802d370 con 0x7f3dc8101ac0 2024-04-02T13:45:22.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.079+0000 7f3dc57fa640 1 -- 172.21.15.33:0/109343309 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11873 (secure 0 0 0) 0x7f3db80beb70 con 0x7f3dc8101ac0 2024-04-02T13:45:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:45:22.082 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:44:55.702357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:45:22.083 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 36 2024-04-02T13:45:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.082+0000 7f3dce3c8640 1 -- 172.21.15.33:0/109343309 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3db00a3d20 msgr2=0x7f3db00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.082+0000 7f3dce3c8640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3db00a3d20 0x7f3db00a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3dc81070c0 tx=0x7f3dbc005d90 comp rx=0 tx=0).stop 2024-04-02T13:45:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.082+0000 7f3dce3c8640 1 -- 172.21.15.33:0/109343309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc8101ac0 msgr2=0x7f3dc81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.082+0000 7f3dce3c8640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc8101ac0 0x7f3dc81983b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3db8000c00 tx=0x7f3db80045e0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.082+0000 7f3dce3c8640 1 -- 172.21.15.33:0/109343309 shutdown_connections 2024-04-02T13:45:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.082+0000 7f3dce3c8640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3db00a3d20 0x7f3db00a61e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.082+0000 7f3dce3c8640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dc8107ad0 0x7f3dc819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.082+0000 7f3dce3c8640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3dc8102480 0x7f3dc81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.082+0000 7f3dce3c8640 1 --2- 172.21.15.33:0/109343309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc8101ac0 0x7f3dc81983b0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.082+0000 7f3dce3c8640 1 -- 172.21.15.33:0/109343309 >> 172.21.15.33:0/109343309 conn(0x7f3dc80fd570 msgr2=0x7f3dc80fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.082+0000 7f3dce3c8640 1 -- 172.21.15.33:0/109343309 shutdown_connections 2024-04-02T13:45:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.082+0000 7f3dce3c8640 1 -- 172.21.15.33:0/109343309 wait complete. 2024-04-02T13:45:22.096 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:45:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.198+0000 7f66fe860640 1 -- 172.21.15.33:0/1241863089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f8107ab0 msgr2=0x7f66f8107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.198+0000 7f66fe860640 1 --2- 172.21.15.33:0/1241863089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f8107ab0 0x7f66f8107e90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009930 tx=0x7f66e802bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.198+0000 7f66fe860640 1 -- 172.21.15.33:0/1241863089 shutdown_connections 2024-04-02T13:45:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.198+0000 7f66fe860640 1 --2- 172.21.15.33:0/1241863089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66f8102460 0x7f66f810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.198+0000 7f66fe860640 1 --2- 172.21.15.33:0/1241863089 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66f8101aa0 0x7f66f8101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.198+0000 7f66fe860640 1 --2- 172.21.15.33:0/1241863089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f8107ab0 0x7f66f8107e90 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.198+0000 7f66fe860640 1 -- 172.21.15.33:0/1241863089 >> 172.21.15.33:0/1241863089 conn(0x7f66f80fd530 msgr2=0x7f66f80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.198+0000 7f66fe860640 1 -- 172.21.15.33:0/1241863089 shutdown_connections 2024-04-02T13:45:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.198+0000 7f66fe860640 1 -- 172.21.15.33:0/1241863089 wait complete. 2024-04-02T13:45:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.198+0000 7f66fe860640 1 Processor -- start 2024-04-02T13:45:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.198+0000 7f66fe860640 1 -- start start 2024-04-02T13:45:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66fe860640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f8101aa0 0x7f66f8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66fe860640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66f8102460 0x7f66f8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66fe860640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66f8107ab0 0x7f66f819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66fe860640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f66f8111e90 con 0x7f66f8101aa0 2024-04-02T13:45:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66fe860640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f66f8111d10 con 0x7f66f8107ab0 2024-04-02T13:45:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66fe860640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f66f8112010 con 0x7f66f8102460 2024-04-02T13:45:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f8101aa0 0x7f66f8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f8101aa0 0x7f66f8198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40182/0 (socket says 172.21.15.33:40182) 2024-04-02T13:45:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66f7fff640 1 -- 172.21.15.33:0/326169252 learned_addr learned my addr 172.21.15.33:0/326169252 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66fcdfd640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66f8107ab0 0x7f66f819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66f77fe640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66f8102460 0x7f66f8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66f7fff640 1 -- 172.21.15.33:0/326169252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66f8102460 msgr2=0x7f66f8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66f7fff640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66f8102460 0x7f66f8198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66f7fff640 1 -- 172.21.15.33:0/326169252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66f8107ab0 msgr2=0x7f66f819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66f7fff640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66f8107ab0 0x7f66f819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66f7fff640 1 -- 172.21.15.33:0/326169252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f819d400 con 0x7f66f8101aa0 2024-04-02T13:45:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66fcdfd640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66f8107ab0 0x7f66f819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66f7fff640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f8101aa0 0x7f66f8198410 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f66e8000c00 tx=0x7f66e80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.199+0000 7f66f57fa640 1 -- 172.21.15.33:0/326169252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8043020 con 0x7f66f8101aa0 2024-04-02T13:45:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.200+0000 7f66fe860640 1 -- 172.21.15.33:0/326169252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f819d690 con 0x7f66f8101aa0 2024-04-02T13:45:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.200+0000 7f66f57fa640 1 -- 172.21.15.33:0/326169252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f66e802caa0 con 0x7f66f8101aa0 2024-04-02T13:45:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.200+0000 7f66f57fa640 1 -- 172.21.15.33:0/326169252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8003bf0 con 0x7f66f8101aa0 2024-04-02T13:45:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.200+0000 7f66fe860640 1 -- 172.21.15.33:0/326169252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f81a4f80 con 0x7f66f8101aa0 2024-04-02T13:45:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f66f57fa640 1 -- 172.21.15.33:0/326169252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66e8046020 con 0x7f66f8101aa0 2024-04-02T13:45:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f6701447740 1 Processor -- start 2024-04-02T13:45:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f6701447740 1 -- start start 2024-04-02T13:45:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f6701447740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990f450 0x55a27990f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f6701447740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 0x55a27991cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f6701447740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27991d0b0 0x55a27991f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f6701447740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a2798737b0 con 0x55a27990fd70 2024-04-02T13:45:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f6701447740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a279812570 con 0x55a27990f450 2024-04-02T13:45:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f6701447740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a27986f760 con 0x55a27991d0b0 2024-04-02T13:45:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f66f57fa640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f66e00a3d20 0x7f66e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 0x55a27991cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 0x55a27991cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40196/0 (socket says 172.21.15.33:40196) 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f66fcdfd640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f66e00a3d20 0x7f66e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f66f7fff640 1 -- 172.21.15.33:0/2465418256 learned_addr learned my addr 172.21.15.33:0/2465418256 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.202+0000 7f66fcdfd640 1 --2- 172.21.15.33:0/2465418256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990f450 0x55a27990f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.203+0000 7f66f77fe640 1 --2- 172.21.15.33:0/2465418256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27991d0b0 0x55a27991f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.203+0000 7f66f57fa640 1 -- 172.21.15.33:0/326169252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f66e80f2630 con 0x7f66f8101aa0 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.203+0000 7f66f7fff640 1 -- 172.21.15.33:0/2465418256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27991d0b0 msgr2=0x55a27991f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.203+0000 7f66f7fff640 1 --2- 172.21.15.33:0/2465418256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27991d0b0 0x55a27991f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.203+0000 7f66f7fff640 1 -- 172.21.15.33:0/2465418256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990f450 msgr2=0x55a27990f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.203+0000 7f66f7fff640 1 --2- 172.21.15.33:0/2465418256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990f450 0x55a27990f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.203+0000 7f66f7fff640 1 -- 172.21.15.33:0/2465418256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2795873f0 con 0x55a27990fd70 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.203+0000 7f66f77fe640 1 --2- 172.21.15.33:0/2465418256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27991d0b0 0x55a27991f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.203+0000 7f66fcdfd640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f66e00a3d20 0x7f66e00a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009370 tx=0x7f66f0013ba0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.203+0000 7f66f7fff640 1 --2- 172.21.15.33:0/2465418256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 0x55a27991cb70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f66e80c6a50 tx=0x7f66e802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f61e18488b45c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.203+0000 7f66de7fc640 1 -- 172.21.15.33:0/2465418256 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e80be0e0 con 0x55a27990fd70 2024-04-02T13:45:22.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.203+0000 7f66de7fc640 1 -- 172.21.15.33:0/2465418256 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f66e80bea10 con 0x55a27990fd70 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.203+0000 7f66de7fc640 1 -- 172.21.15.33:0/2465418256 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e80bdc70 con 0x55a27990fd70 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.204+0000 7f6701447740 1 -- 172.21.15.33:0/2465418256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 msgr2=0x55a27991cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.204+0000 7f6701447740 1 --2- 172.21.15.33:0/2465418256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 0x55a27991cb70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f66e80c6a50 tx=0x7f66e802bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.204+0000 7f6701447740 1 -- 172.21.15.33:0/2465418256 shutdown_connections 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.204+0000 7f6701447740 1 --2- 172.21.15.33:0/2465418256 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27991d0b0 0x55a27991f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.204+0000 7f6701447740 1 --2- 172.21.15.33:0/2465418256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 0x55a27991cb70 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.204+0000 7f6701447740 1 --2- 172.21.15.33:0/2465418256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990f450 0x55a27990f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.204+0000 7f6701447740 1 -- 172.21.15.33:0/2465418256 >> 172.21.15.33:0/2465418256 conn(0x55a279909ed0 msgr2=0x55a27990a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.204+0000 7f6701447740 1 -- 172.21.15.33:0/2465418256 shutdown_connections 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.204+0000 7f6701447740 1 -- 172.21.15.33:0/2465418256 wait complete. 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.204+0000 7f6701447740 1 -- start start 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.204+0000 7f6701447740 1 Processor -- start 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f6701447740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27990f450 0x55a27991ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f6701447740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 0x55a279919df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f6701447740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27991d0b0 0x55a27991a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f6701447740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a2798737b0 con 0x55a27990fd70 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f6701447740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a27982ee80 con 0x55a27991d0b0 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f6701447740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a27991fcb0 con 0x55a27990f450 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f6701447740 10 client.0 _ll_register_callbacks cb 0x55a27986aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 0x55a279919df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 0x55a279919df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40212/0 (socket says 172.21.15.33:40212) 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66f77fe640 1 -- 172.21.15.33:0/293700567 learned_addr learned my addr 172.21.15.33:0/293700567 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66fcdfd640 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27991d0b0 0x55a27991a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66f7fff640 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27990f450 0x55a27991ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66f77fe640 1 -- 172.21.15.33:0/293700567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27990f450 msgr2=0x55a27991ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66f77fe640 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27990f450 0x55a27991ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66f77fe640 1 -- 172.21.15.33:0/293700567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27991d0b0 msgr2=0x55a27991a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66f77fe640 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27991d0b0 0x55a27991a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66f77fe640 1 -- 172.21.15.33:0/293700567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2796517d0 con 0x55a27990fd70 2024-04-02T13:45:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66fcdfd640 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27991d0b0 0x55a27991a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66f77fe640 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 0x55a279919df0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00f6c0 tx=0x7f66ec00fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa3c2c760070cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.205+0000 7f66d77fe640 1 -- 172.21.15.33:0/293700567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec016070 con 0x55a27990fd70 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f66d77fe640 1 -- 172.21.15.33:0/293700567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f66ec00bb00 con 0x55a27990fd70 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f6701447740 1 -- 172.21.15.33:0/293700567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66e80bea10 con 0x55a27990fd70 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f66d77fe640 1 -- 172.21.15.33:0/293700567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f66ec025050 con 0x55a27990fd70 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f6701447740 10 client.17019 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f6701447740 1 -- 172.21.15.33:0/293700567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a27992bb90 con 0x55a27990fd70 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f66d77fe640 1 -- 172.21.15.33:0/293700567 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f66e80bea10 con 0x55a27990fd70 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f66d77fe640 1 -- 172.21.15.33:0/293700567 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f66ec01a400 con 0x55a27990fd70 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f6701447740 10 client.17019 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f6701447740 10 client.17019 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f6701447740 20 client.17019 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f6701447740 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a27992d140 0x55a27992f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f6701447740 4 client.17019 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f6701447740 1 -- 172.21.15.33:0/293700567 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a279880cf0 con 0x55a27992d140 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.206+0000 7f66f7fff640 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a27992d140 0x55a27992f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.207+0000 7f66f7fff640 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a27992d140 0x55a27992f560 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=320eb4fc7ede3d22 server_cookie=600108e3c7daa6fb in_seq=0 out_seq=0 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.207+0000 7f66d77fe640 10 client.17019 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f66d77fe640 1 -- 172.21.15.33:0/293700567 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a279880cf0 con 0x55a27992d140 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f66d77fe640 10 client.17019 handle_command_reply: tid=0 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f6701447740 1 client.17019 shutdown 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f6701447740 1 -- 172.21.15.33:0/293700567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 msgr2=0x55a279919df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f6701447740 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 0x55a279919df0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00f6c0 tx=0x7f66ec00fb80 comp rx=0 tx=0).stop 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f6701447740 1 -- 172.21.15.33:0/293700567 shutdown_connections 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f6701447740 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a27992d140 0x55a27992f560 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f6701447740 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27991d0b0 0x55a27991a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f6701447740 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990fd70 0x55a279919df0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f6701447740 1 --2- 172.21.15.33:0/293700567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27990f450 0x55a27991ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f66d77fe640 0 client.17019 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:45:22.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f6701447740 1 -- 172.21.15.33:0/293700567 >> 172.21.15.33:0/293700567 conn(0x55a279909ed0 msgr2=0x55a27990bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f6701447740 1 -- 172.21.15.33:0/293700567 shutdown_connections 2024-04-02T13:45:22.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f6701447740 1 -- 172.21.15.33:0/293700567 wait complete. 2024-04-02T13:45:22.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.208+0000 7f6701447740 20 client.17019 trim_cache size 0 max 16384 2024-04-02T13:45:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.221+0000 7f6701447740 1 Processor -- start 2024-04-02T13:45:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.221+0000 7f6701447740 1 -- start start 2024-04-02T13:45:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.221+0000 7f6701447740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990f450 0x55a279919d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.221+0000 7f6701447740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27990fd70 0x55a2799289f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.221+0000 7f6701447740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27991d0b0 0x55a27991c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.221+0000 7f6701447740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81af700 con 0x55a27991d0b0 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.221+0000 7f6701447740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81acfd0 con 0x55a27990f450 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.221+0000 7f6701447740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81aae40 con 0x55a27990fd70 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.221+0000 7f66fcdfd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27991d0b0 0x55a27991c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.221+0000 7f66fcdfd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27991d0b0 0x55a27991c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40224/0 (socket says 172.21.15.33:40224) 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.221+0000 7f66fcdfd640 1 -- 172.21.15.33:0/4248634127 learned_addr learned my addr 172.21.15.33:0/4248634127 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.222+0000 7f66f7fff640 1 --2- 172.21.15.33:0/4248634127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990f450 0x55a279919d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.222+0000 7f66fcdfd640 1 -- 172.21.15.33:0/4248634127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27990fd70 msgr2=0x55a2799289f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.222+0000 7f66fcdfd640 1 --2- 172.21.15.33:0/4248634127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27990fd70 0x55a2799289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.222+0000 7f66fcdfd640 1 -- 172.21.15.33:0/4248634127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990f450 msgr2=0x55a279919d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.222+0000 7f66fcdfd640 1 --2- 172.21.15.33:0/4248634127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990f450 0x55a279919d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.222+0000 7f66fcdfd640 1 -- 172.21.15.33:0/4248634127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a27990fba0 con 0x55a27991d0b0 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.222+0000 7f66fcdfd640 1 --2- 172.21.15.33:0/4248634127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27991d0b0 0x55a27991c520 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f66f0011c70 tx=0x7f66f000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c7f730d54bde22 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.222+0000 7f66ddffb640 1 -- 172.21.15.33:0/4248634127 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f002b070 con 0x55a27991d0b0 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.222+0000 7f66ddffb640 1 -- 172.21.15.33:0/4248634127 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f66f0011ca0 con 0x55a27991d0b0 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.222+0000 7f66ddffb640 1 -- 172.21.15.33:0/4248634127 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f000f3d0 con 0x55a27991d0b0 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 -- 172.21.15.33:0/4248634127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27991d0b0 msgr2=0x55a27991c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 --2- 172.21.15.33:0/4248634127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27991d0b0 0x55a27991c520 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f66f0011c70 tx=0x7f66f000b000 comp rx=0 tx=0).stop 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 -- 172.21.15.33:0/4248634127 shutdown_connections 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 --2- 172.21.15.33:0/4248634127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27991d0b0 0x55a27991c520 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 --2- 172.21.15.33:0/4248634127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27990fd70 0x55a2799289f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 --2- 172.21.15.33:0/4248634127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990f450 0x55a279919d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 -- 172.21.15.33:0/4248634127 >> 172.21.15.33:0/4248634127 conn(0x55a279909ed0 msgr2=0x55a279919680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 -- 172.21.15.33:0/4248634127 shutdown_connections 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 -- 172.21.15.33:0/4248634127 wait complete. 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 -- start start 2024-04-02T13:45:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 Processor -- start 2024-04-02T13:45:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990f450 0x55a27991de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990fd70 0x55a27991e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27991d0b0 0x55a27991e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81ab490 con 0x55a27990f450 2024-04-02T13:45:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81ac900 con 0x55a27990fd70 2024-04-02T13:45:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81abb60 con 0x55a27991d0b0 2024-04-02T13:45:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f6701447740 10 client.0 _ll_register_callbacks cb 0x7f66ec005d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.223+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990f450 0x55a27991de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990f450 0x55a27991de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40226/0 (socket says 172.21.15.33:40226) 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66f7fff640 1 -- 172.21.15.33:0/234054620 learned_addr learned my addr 172.21.15.33:0/234054620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66f77fe640 1 --2- 172.21.15.33:0/234054620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990fd70 0x55a27991e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66fcdfd640 1 --2- 172.21.15.33:0/234054620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27991d0b0 0x55a27991e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66f7fff640 1 -- 172.21.15.33:0/234054620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27991d0b0 msgr2=0x55a27991e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66f7fff640 1 --2- 172.21.15.33:0/234054620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27991d0b0 0x55a27991e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66f7fff640 1 -- 172.21.15.33:0/234054620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990fd70 msgr2=0x55a27991e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66f7fff640 1 --2- 172.21.15.33:0/234054620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990fd70 0x55a27991e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66f7fff640 1 -- 172.21.15.33:0/234054620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a279923200 con 0x55a27990f450 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66f7fff640 1 --2- 172.21.15.33:0/234054620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990f450 0x55a27991de70 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f66e802c4e0 tx=0x7f66e8003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24842ae23026b415 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66d77fe640 1 -- 172.21.15.33:0/234054620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e80bde50 con 0x55a27990f450 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66d77fe640 1 -- 172.21.15.33:0/234054620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f66e8005550 con 0x55a27990f450 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f6701447740 1 -- 172.21.15.33:0/234054620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66f0011ca0 con 0x55a27990f450 2024-04-02T13:45:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66d77fe640 1 -- 172.21.15.33:0/234054620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f66e80bdc70 con 0x55a27990f450 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f6701447740 10 client.17025 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f6701447740 1 -- 172.21.15.33:0/234054620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a279993a00 con 0x55a27990f450 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.224+0000 7f66d77fe640 1 -- 172.21.15.33:0/234054620 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f66f0011ca0 con 0x55a27990f450 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.225+0000 7f66d77fe640 1 -- 172.21.15.33:0/234054620 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f66e80ebce0 con 0x55a27990f450 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.225+0000 7f6701447740 10 client.17025 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.225+0000 7f6701447740 10 client.17025 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.225+0000 7f6701447740 20 client.17025 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.225+0000 7f6701447740 1 --2- 172.21.15.33:0/234054620 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a279993dc0 0x55a2799961e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.225+0000 7f6701447740 4 client.17025 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.225+0000 7f6701447740 1 -- 172.21.15.33:0/234054620 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a27987b040 con 0x55a279993dc0 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.225+0000 7f66f77fe640 1 --2- 172.21.15.33:0/234054620 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a279993dc0 0x55a2799961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.225+0000 7f66f77fe640 1 --2- 172.21.15.33:0/234054620 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a279993dc0 0x55a2799961e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3250d24466f68c36 server_cookie=bd0ec589355d455d in_seq=0 out_seq=0 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.225+0000 7f66d77fe640 10 client.17025 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f66d77fe640 1 -- 172.21.15.33:0/234054620 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a27987b040 con 0x55a279993dc0 2024-04-02T13:45:22.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f66d77fe640 10 client.17025 handle_command_reply: tid=0 2024-04-02T13:45:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f6701447740 1 client.17025 shutdown 2024-04-02T13:45:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f6701447740 1 -- 172.21.15.33:0/234054620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990f450 msgr2=0x55a27991de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f6701447740 1 --2- 172.21.15.33:0/234054620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990f450 0x55a27991de70 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f66e802c4e0 tx=0x7f66e8003f70 comp rx=0 tx=0).stop 2024-04-02T13:45:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f6701447740 1 -- 172.21.15.33:0/234054620 shutdown_connections 2024-04-02T13:45:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f6701447740 1 --2- 172.21.15.33:0/234054620 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a279993dc0 0x55a2799961e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f6701447740 1 --2- 172.21.15.33:0/234054620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a27991d0b0 0x55a27991e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f6701447740 1 --2- 172.21.15.33:0/234054620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a27990fd70 0x55a27991e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f66d77fe640 0 client.17025 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:45:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f6701447740 1 --2- 172.21.15.33:0/234054620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a27990f450 0x55a27991de70 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f6701447740 1 -- 172.21.15.33:0/234054620 >> 172.21.15.33:0/234054620 conn(0x55a279909ed0 msgr2=0x55a27990b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f6701447740 1 -- 172.21.15.33:0/234054620 shutdown_connections 2024-04-02T13:45:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f6701447740 1 -- 172.21.15.33:0/234054620 wait complete. 2024-04-02T13:45:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f6701447740 20 client.17025 trim_cache size 0 max 16384 2024-04-02T13:45:22.235 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:45:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f66fe860640 1 -- 172.21.15.33:0/326169252 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f66e00a3d20 msgr2=0x7f66e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f66fe860640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f66e00a3d20 0x7f66e00a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009370 tx=0x7f66f0013ba0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f66fe860640 1 -- 172.21.15.33:0/326169252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f8101aa0 msgr2=0x7f66f8198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.226+0000 7f66fe860640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f8101aa0 0x7f66f8198410 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f66e8000c00 tx=0x7f66e80045e0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.227+0000 7f66fe860640 1 -- 172.21.15.33:0/326169252 shutdown_connections 2024-04-02T13:45:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.227+0000 7f66fe860640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f66e00a3d20 0x7f66e00a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.227+0000 7f66fe860640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66f8107ab0 0x7f66f819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.227+0000 7f66fe860640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66f8102460 0x7f66f8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.227+0000 7f66fe860640 1 --2- 172.21.15.33:0/326169252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66f8101aa0 0x7f66f8198410 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.227+0000 7f66fe860640 1 -- 172.21.15.33:0/326169252 >> 172.21.15.33:0/326169252 conn(0x7f66f80fd530 msgr2=0x7f66f80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.227+0000 7f66fe860640 1 -- 172.21.15.33:0/326169252 shutdown_connections 2024-04-02T13:45:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.227+0000 7f66fe860640 1 -- 172.21.15.33:0/326169252 wait complete. 2024-04-02T13:45:22.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:45:22.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.342+0000 7f5cceb75640 1 -- 172.21.15.33:0/487449307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cc8100970 msgr2=0x7f5cc8100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.342+0000 7f5cceb75640 1 --2- 172.21.15.33:0/487449307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cc8100970 0x7f5cc8100df0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009cd0 tx=0x7f5cb802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.342+0000 7f5cceb75640 1 -- 172.21.15.33:0/487449307 shutdown_connections 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.342+0000 7f5cceb75640 1 --2- 172.21.15.33:0/487449307 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5cc8101330 0x7f5cc810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.342+0000 7f5cceb75640 1 --2- 172.21.15.33:0/487449307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cc8100970 0x7f5cc8100df0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.342+0000 7f5cceb75640 1 --2- 172.21.15.33:0/487449307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc8106980 0x7f5cc8106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.342+0000 7f5cceb75640 1 -- 172.21.15.33:0/487449307 >> 172.21.15.33:0/487449307 conn(0x7f5cc8077b00 msgr2=0x7f5cc8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.343+0000 7f5cceb75640 1 -- 172.21.15.33:0/487449307 shutdown_connections 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.343+0000 7f5cceb75640 1 -- 172.21.15.33:0/487449307 wait complete. 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.343+0000 7f5cceb75640 1 Processor -- start 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.343+0000 7f5cceb75640 1 -- start start 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.343+0000 7f5cceb75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cc8100970 0x7f5cc8199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.343+0000 7f5cceb75640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc8101330 0x7f5cc8199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.343+0000 7f5cceb75640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5cc8106980 0x7f5cc81a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.343+0000 7f5cceb75640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8110d60 con 0x7f5cc8100970 2024-04-02T13:45:22.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.343+0000 7f5cceb75640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8110be0 con 0x7f5cc8101330 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.343+0000 7f5cceb75640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8110ee0 con 0x7f5cc8106980 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.343+0000 7f5ccc911640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cc8100970 0x7f5cc8199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.343+0000 7f5ccc911640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cc8100970 0x7f5cc8199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40240/0 (socket says 172.21.15.33:40240) 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.343+0000 7f5ccc911640 1 -- 172.21.15.33:0/451240084 learned_addr learned my addr 172.21.15.33:0/451240084 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5cc7fff640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc8101330 0x7f5cc8199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5ccd112640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5cc8106980 0x7f5cc81a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5ccc911640 1 -- 172.21.15.33:0/451240084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5cc8106980 msgr2=0x7f5cc81a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5ccc911640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5cc8106980 0x7f5cc81a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5ccc911640 1 -- 172.21.15.33:0/451240084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc8101330 msgr2=0x7f5cc8199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5ccc911640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc8101330 0x7f5cc8199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5ccc911640 1 -- 172.21.15.33:0/451240084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc81a47c0 con 0x7f5cc8100970 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5cc7fff640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc8101330 0x7f5cc8199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5ccd112640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5cc8106980 0x7f5cc81a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5ccc911640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cc8100970 0x7f5cc8199200 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000e9f0 tx=0x7f5cc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5cc5ffb640 1 -- 172.21.15.33:0/451240084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc000cd80 con 0x7f5cc8100970 2024-04-02T13:45:22.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5cc5ffb640 1 -- 172.21.15.33:0/451240084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5cc0010900 con 0x7f5cc8100970 2024-04-02T13:45:22.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5cceb75640 1 -- 172.21.15.33:0/451240084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc81a4990 con 0x7f5cc8100970 2024-04-02T13:45:22.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5cc5ffb640 1 -- 172.21.15.33:0/451240084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc00189e0 con 0x7f5cc8100970 2024-04-02T13:45:22.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.344+0000 7f5cceb75640 1 -- 172.21.15.33:0/451240084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc81a4eb0 con 0x7f5cc8100970 2024-04-02T13:45:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.346+0000 7f5cc5ffb640 1 -- 172.21.15.33:0/451240084 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5cc0010ab0 con 0x7f5cc8100970 2024-04-02T13:45:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5cd1838740 1 Processor -- start 2024-04-02T13:45:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5cd1838740 1 -- start start 2024-04-02T13:45:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5cd1838740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 0x55c3d019db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5cd1838740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019e0c0 0x55c3d01a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5cd1838740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5cd1838740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c3d01017b0 con 0x55c3d019e0c0 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5cd1838740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c3d00a0570 con 0x55c3d019d7a0 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5cd1838740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c3d00fd760 con 0x55c3d01a6ca0 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5cc5ffb640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5cb40a3d20 0x7f5cb40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5ccc911640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019e0c0 0x55c3d01a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5ccc911640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019e0c0 0x55c3d01a6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40248/0 (socket says 172.21.15.33:40248) 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5ccc911640 1 -- 172.21.15.33:0/2723735481 learned_addr learned my addr 172.21.15.33:0/2723735481 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5ccd112640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5cb40a3d20 0x7f5cb40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5cc7fff640 1 --2- 172.21.15.33:0/2723735481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5ccd112640 1 --2- 172.21.15.33:0/2723735481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 0x55c3d019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5cc5ffb640 1 -- 172.21.15.33:0/451240084 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f5cc00d0ff0 con 0x7f5cc8100970 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5ccc911640 1 -- 172.21.15.33:0/2723735481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d01a6ca0 msgr2=0x55c3d01a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5ccc911640 1 --2- 172.21.15.33:0/2723735481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5ccc911640 1 -- 172.21.15.33:0/2723735481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 msgr2=0x55c3d019db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.347+0000 7f5ccc911640 1 --2- 172.21.15.33:0/2723735481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 0x55c3d019db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.348+0000 7f5ccc911640 1 -- 172.21.15.33:0/2723735481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3cfe153f0 con 0x55c3d019e0c0 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.348+0000 7f5ccd112640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5cb40a3d20 0x7f5cb40a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0013650 tx=0x7f5cb000c7f0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.348+0000 7f5ccc911640 1 --2- 172.21.15.33:0/2723735481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019e0c0 0x55c3d01a6760 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000e9c0 tx=0x7f5cc000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c42d53c1e876e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.348+0000 7f5cbeffd640 1 -- 172.21.15.33:0/2723735481 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0026040 con 0x55c3d019e0c0 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.348+0000 7f5cbeffd640 1 -- 172.21.15.33:0/2723735481 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5cc009d280 con 0x55c3d019e0c0 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.348+0000 7f5cbeffd640 1 -- 172.21.15.33:0/2723735481 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc00991f0 con 0x55c3d019e0c0 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.348+0000 7f5cd1838740 1 -- 172.21.15.33:0/2723735481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019e0c0 msgr2=0x55c3d01a6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.348+0000 7f5cd1838740 1 --2- 172.21.15.33:0/2723735481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019e0c0 0x55c3d01a6760 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000e9c0 tx=0x7f5cc000cf20 comp rx=0 tx=0).stop 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.348+0000 7f5cd1838740 1 -- 172.21.15.33:0/2723735481 shutdown_connections 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.348+0000 7f5cd1838740 1 --2- 172.21.15.33:0/2723735481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.348+0000 7f5cd1838740 1 --2- 172.21.15.33:0/2723735481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019e0c0 0x55c3d01a6760 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.348+0000 7f5cd1838740 1 --2- 172.21.15.33:0/2723735481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 0x55c3d019db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5cd1838740 1 -- 172.21.15.33:0/2723735481 >> 172.21.15.33:0/2723735481 conn(0x55c3d0197ed0 msgr2=0x55c3d01982b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5cd1838740 1 -- 172.21.15.33:0/2723735481 shutdown_connections 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5cd1838740 1 -- 172.21.15.33:0/2723735481 wait complete. 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5cd1838740 1 -- start start 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5cd1838740 1 Processor -- start 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5cd1838740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 0x55c3d01a6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5cd1838740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 0x55c3d01a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5cd1838740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5cd1838740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c3d01a9a30 con 0x55c3d01a6ca0 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5cd1838740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c3d00bce80 con 0x55c3d019d7a0 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5cd1838740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c3d01a9bb0 con 0x55c3d019e0c0 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5cd1838740 10 client.0 _ll_register_callbacks cb 0x55c3d00f8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5ccd112640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5ccd112640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40264/0 (socket says 172.21.15.33:40264) 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5ccd112640 1 -- 172.21.15.33:0/1444456588 learned_addr learned my addr 172.21.15.33:0/1444456588 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.349+0000 7f5ccc911640 1 --2- 172.21.15.33:0/1444456588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 0x55c3d01a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5cc7fff640 1 --2- 172.21.15.33:0/1444456588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 0x55c3d01a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5ccd112640 1 -- 172.21.15.33:0/1444456588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 msgr2=0x55c3d01a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5ccd112640 1 --2- 172.21.15.33:0/1444456588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 0x55c3d01a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5ccd112640 1 -- 172.21.15.33:0/1444456588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 msgr2=0x55c3d01a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5ccd112640 1 --2- 172.21.15.33:0/1444456588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 0x55c3d01a6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5ccd112640 1 -- 172.21.15.33:0/1444456588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3cfedf7d0 con 0x55c3d01a6ca0 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5ccd112640 1 --2- 172.21.15.33:0/1444456588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a41b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0011b60 tx=0x7f5cb0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebde3754e49a41f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5ca7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5ca7fff640 1 -- 172.21.15.33:0/1444456588 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0025070 con 0x55c3d01a6ca0 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5ca7fff640 1 -- 172.21.15.33:0/1444456588 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f5cb000fd60 con 0x55c3d01a6ca0 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5cd1838740 1 -- 172.21.15.33:0/1444456588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cc009d280 con 0x55c3d01a6ca0 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5ca7fff640 1 -- 172.21.15.33:0/1444456588 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5cb0030050 con 0x55c3d01a6ca0 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5cd1838740 10 client.17037 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5cd1838740 1 -- 172.21.15.33:0/1444456588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c3d01b5ac0 con 0x55c3d01a6ca0 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.350+0000 7f5ca7fff640 1 -- 172.21.15.33:0/1444456588 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5cc009d280 con 0x55c3d01a6ca0 2024-04-02T13:45:22.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.351+0000 7f5ca7fff640 1 -- 172.21.15.33:0/1444456588 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f5cb0004ce0 con 0x55c3d01a6ca0 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.351+0000 7f5cd1838740 10 client.17037 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.351+0000 7f5cd1838740 10 client.17037 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.351+0000 7f5cd1838740 20 client.17037 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.351+0000 7f5cd1838740 1 --2- 172.21.15.33:0/1444456588 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55c3d01b6ed0 0x55c3d01b9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.351+0000 7f5cd1838740 4 client.17037 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.351+0000 7f5cd1838740 1 -- 172.21.15.33:0/1444456588 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3d010ecf0 con 0x55c3d01b6ed0 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.351+0000 7f5ccc911640 1 --2- 172.21.15.33:0/1444456588 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55c3d01b6ed0 0x55c3d01b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.351+0000 7f5ccc911640 1 --2- 172.21.15.33:0/1444456588 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55c3d01b6ed0 0x55c3d01b9330 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b1a444ee68cdb59 server_cookie=5d0d155997b6a010 in_seq=0 out_seq=0 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.351+0000 7f5ca7fff640 10 client.17037 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5ca7fff640 1 -- 172.21.15.33:0/1444456588 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c3d010ecf0 con 0x55c3d01b6ed0 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5ca7fff640 10 client.17037 handle_command_reply: tid=0 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5cd1838740 1 client.17037 shutdown 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5cd1838740 1 -- 172.21.15.33:0/1444456588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 msgr2=0x55c3d01a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5cd1838740 1 --2- 172.21.15.33:0/1444456588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a41b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0011b60 tx=0x7f5cb0019000 comp rx=0 tx=0).stop 2024-04-02T13:45:22.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5cd1838740 1 -- 172.21.15.33:0/1444456588 shutdown_connections 2024-04-02T13:45:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5cd1838740 1 --2- 172.21.15.33:0/1444456588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a41b0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5cd1838740 1 --2- 172.21.15.33:0/1444456588 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55c3d01b6ed0 0x55c3d01b9330 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5cd1838740 1 --2- 172.21.15.33:0/1444456588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 0x55c3d01a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5cd1838740 1 --2- 172.21.15.33:0/1444456588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 0x55c3d01a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5ca7fff640 0 client.17037 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:45:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5cd1838740 1 -- 172.21.15.33:0/1444456588 >> 172.21.15.33:0/1444456588 conn(0x55c3d0197ed0 msgr2=0x55c3d0199c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5cd1838740 1 -- 172.21.15.33:0/1444456588 shutdown_connections 2024-04-02T13:45:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5cd1838740 1 -- 172.21.15.33:0/1444456588 wait complete. 2024-04-02T13:45:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.352+0000 7f5cd1838740 20 client.17037 trim_cache size 0 max 16384 2024-04-02T13:45:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5cd1838740 1 Processor -- start 2024-04-02T13:45:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5cd1838740 1 -- start start 2024-04-02T13:45:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5cd1838740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 0x55c3d01b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5cd1838740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 0x55c3d01a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5cd1838740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5cd1838740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81b1960 con 0x55c3d01a6ca0 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5cd1838740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81af230 con 0x55c3d019d7a0 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5cd1838740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81ad0a0 con 0x55c3d019e0c0 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5ccd112640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5ccd112640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40278/0 (socket says 172.21.15.33:40278) 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5ccd112640 1 -- 172.21.15.33:0/3901705165 learned_addr learned my addr 172.21.15.33:0/3901705165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5ccc911640 1 --2- 172.21.15.33:0/3901705165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 0x55c3d01b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5ccd112640 1 -- 172.21.15.33:0/3901705165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 msgr2=0x55c3d01a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5ccd112640 1 --2- 172.21.15.33:0/3901705165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 0x55c3d01a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5ccd112640 1 -- 172.21.15.33:0/3901705165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 msgr2=0x55c3d01b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5ccd112640 1 --2- 172.21.15.33:0/3901705165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 0x55c3d01b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.366+0000 7f5ccd112640 1 -- 172.21.15.33:0/3901705165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d019f690 con 0x55c3d01a6ca0 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.367+0000 7f5ccd112640 1 --2- 172.21.15.33:0/3901705165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a6730 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0000c00 tx=0x7f5cb000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deba337d2a2f8fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.367+0000 7f5cbe7fc640 1 -- 172.21.15.33:0/3901705165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0025040 con 0x55c3d01a6ca0 2024-04-02T13:45:22.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.367+0000 7f5cbe7fc640 1 -- 172.21.15.33:0/3901705165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5cb0011970 con 0x55c3d01a6ca0 2024-04-02T13:45:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.367+0000 7f5cbe7fc640 1 -- 172.21.15.33:0/3901705165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0034810 con 0x55c3d01a6ca0 2024-04-02T13:45:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.367+0000 7f5cd1838740 1 -- 172.21.15.33:0/3901705165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 msgr2=0x55c3d01a6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.367+0000 7f5cd1838740 1 --2- 172.21.15.33:0/3901705165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a6730 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0000c00 tx=0x7f5cb000c860 comp rx=0 tx=0).stop 2024-04-02T13:45:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.367+0000 7f5cd1838740 1 -- 172.21.15.33:0/3901705165 shutdown_connections 2024-04-02T13:45:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.367+0000 7f5cd1838740 1 --2- 172.21.15.33:0/3901705165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d01a6ca0 0x55c3d01a6730 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.367+0000 7f5cd1838740 1 --2- 172.21.15.33:0/3901705165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 0x55c3d01a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.367+0000 7f5cd1838740 1 --2- 172.21.15.33:0/3901705165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d019d7a0 0x55c3d01b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.367+0000 7f5cd1838740 1 -- 172.21.15.33:0/3901705165 >> 172.21.15.33:0/3901705165 conn(0x55c3d0197ed0 msgr2=0x55c3d01982b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.367+0000 7f5cd1838740 1 -- 172.21.15.33:0/3901705165 shutdown_connections 2024-04-02T13:45:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.367+0000 7f5cd1838740 1 -- 172.21.15.33:0/3901705165 wait complete. 2024-04-02T13:45:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5cd1838740 1 -- start start 2024-04-02T13:45:22.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5cd1838740 1 Processor -- start 2024-04-02T13:45:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5cd1838740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019d7a0 0x55c3d01a8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5cd1838740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 0x55c3d01a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5cd1838740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d01a6ca0 0x55c3d01ad550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5cd1838740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81ad6f0 con 0x55c3d019d7a0 2024-04-02T13:45:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5cd1838740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81aeb60 con 0x55c3d01a6ca0 2024-04-02T13:45:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5cd1838740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81addc0 con 0x55c3d019e0c0 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5cd1838740 10 client.0 _ll_register_callbacks cb 0x7f5cb000ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5ccc911640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019d7a0 0x55c3d01a8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5ccc911640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019d7a0 0x55c3d01a8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40290/0 (socket says 172.21.15.33:40290) 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5ccc911640 1 -- 172.21.15.33:0/2095451084 learned_addr learned my addr 172.21.15.33:0/2095451084 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5ccd112640 1 --2- 172.21.15.33:0/2095451084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d01a6ca0 0x55c3d01ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5cc7fff640 1 --2- 172.21.15.33:0/2095451084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 0x55c3d01a87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5ccc911640 1 -- 172.21.15.33:0/2095451084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 msgr2=0x55c3d01a87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5ccc911640 1 --2- 172.21.15.33:0/2095451084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 0x55c3d01a87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5ccc911640 1 -- 172.21.15.33:0/2095451084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d01a6ca0 msgr2=0x55c3d01ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.368+0000 7f5ccc911640 1 --2- 172.21.15.33:0/2095451084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d01a6ca0 0x55c3d01ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5ccc911640 1 -- 172.21.15.33:0/2095451084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d01adb90 con 0x55c3d019d7a0 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5ccc911640 1 --2- 172.21.15.33:0/2095451084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019d7a0 0x55c3d01a8290 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5cc009cac0 tx=0x7f5cc0098e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa21a9cb87a7ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5ca7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5ca7fff640 1 -- 172.21.15.33:0/2095451084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc00a0f30 con 0x55c3d019d7a0 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5ca7fff640 1 -- 172.21.15.33:0/2095451084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f5cc00bd0b0 con 0x55c3d019d7a0 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5cd1838740 1 -- 172.21.15.33:0/2095451084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb0011970 con 0x55c3d019d7a0 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5ca7fff640 1 -- 172.21.15.33:0/2095451084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5cc00a57f0 con 0x55c3d019d7a0 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5cd1838740 10 client.17043 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5cd1838740 1 -- 172.21.15.33:0/2095451084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c3d02221b0 con 0x55c3d019d7a0 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5ca7fff640 1 -- 172.21.15.33:0/2095451084 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5cb0011970 con 0x55c3d019d7a0 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5ca7fff640 1 -- 172.21.15.33:0/2095451084 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f5cc0061bc0 con 0x55c3d019d7a0 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5cd1838740 10 client.17043 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5cd1838740 10 client.17043 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5cd1838740 20 client.17043 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5cd1838740 1 --2- 172.21.15.33:0/2095451084 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55c3d02224f0 0x55c3d02248e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5cd1838740 4 client.17043 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.369+0000 7f5cd1838740 1 -- 172.21.15.33:0/2095451084 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3d0109040 con 0x55c3d02224f0 2024-04-02T13:45:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5cc7fff640 1 --2- 172.21.15.33:0/2095451084 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55c3d02224f0 0x55c3d02248e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5ca7fff640 10 client.17043 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5cc7fff640 1 --2- 172.21.15.33:0/2095451084 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55c3d02224f0 0x55c3d02248e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ff5f2bc98515142 server_cookie=2ee20e9f3ad44a59 in_seq=0 out_seq=0 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5ca7fff640 1 -- 172.21.15.33:0/2095451084 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3d0109040 con 0x55c3d02224f0 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5ca7fff640 10 client.17043 handle_command_reply: tid=0 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5cd1838740 1 client.17043 shutdown 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5cd1838740 1 -- 172.21.15.33:0/2095451084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019d7a0 msgr2=0x55c3d01a8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5cd1838740 1 --2- 172.21.15.33:0/2095451084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019d7a0 0x55c3d01a8290 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5cc009cac0 tx=0x7f5cc0098e20 comp rx=0 tx=0).stop 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5cd1838740 1 -- 172.21.15.33:0/2095451084 shutdown_connections 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5cd1838740 1 --2- 172.21.15.33:0/2095451084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3d01a6ca0 0x55c3d01ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5cd1838740 1 --2- 172.21.15.33:0/2095451084 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55c3d02224f0 0x55c3d02248e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5cd1838740 1 --2- 172.21.15.33:0/2095451084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3d019e0c0 0x55c3d01a87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5ca7fff640 0 client.17043 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5cd1838740 1 --2- 172.21.15.33:0/2095451084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3d019d7a0 0x55c3d01a8290 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.370+0000 7f5cd1838740 1 -- 172.21.15.33:0/2095451084 >> 172.21.15.33:0/2095451084 conn(0x55c3d0197ed0 msgr2=0x55c3d0199300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cd1838740 1 -- 172.21.15.33:0/2095451084 shutdown_connections 2024-04-02T13:45:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cd1838740 1 -- 172.21.15.33:0/2095451084 wait complete. 2024-04-02T13:45:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cd1838740 20 client.17043 trim_cache size 0 max 16384 2024-04-02T13:45:22.380 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:45:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cceb75640 1 -- 172.21.15.33:0/451240084 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5cb40a3d20 msgr2=0x7f5cb40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cceb75640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5cb40a3d20 0x7f5cb40a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0013650 tx=0x7f5cb000c7f0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cceb75640 1 -- 172.21.15.33:0/451240084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cc8100970 msgr2=0x7f5cc8199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cceb75640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cc8100970 0x7f5cc8199200 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000e9f0 tx=0x7f5cc000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cceb75640 1 -- 172.21.15.33:0/451240084 shutdown_connections 2024-04-02T13:45:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cceb75640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5cb40a3d20 0x7f5cb40a61e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cceb75640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5cc8106980 0x7f5cc81a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cceb75640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cc8101330 0x7f5cc8199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cceb75640 1 --2- 172.21.15.33:0/451240084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cc8100970 0x7f5cc8199200 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cceb75640 1 -- 172.21.15.33:0/451240084 >> 172.21.15.33:0/451240084 conn(0x7f5cc8077b00 msgr2=0x7f5cc8103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cceb75640 1 -- 172.21.15.33:0/451240084 shutdown_connections 2024-04-02T13:45:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.371+0000 7f5cceb75640 1 -- 172.21.15.33:0/451240084 wait complete. 2024-04-02T13:45:22.386 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:22 smithi033 ceph-mon[32390]: pgmap v202: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 219 B/s wr, 7 op/s 2024-04-02T13:45:22.387 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:45:22.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:22 smithi145 ceph-mon[35646]: pgmap v202: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 219 B/s wr, 7 op/s 2024-04-02T13:45:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.489+0000 7efdb61cd640 1 -- 172.21.15.33:0/3675202707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdb0101370 msgr2=0x7efdb010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.489+0000 7efdb61cd640 1 --2- 172.21.15.33:0/3675202707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdb0101370 0x7efdb010d880 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7efda8009cd0 tx=0x7efda802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.489+0000 7efdb61cd640 1 -- 172.21.15.33:0/3675202707 shutdown_connections 2024-04-02T13:45:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.489+0000 7efdb61cd640 1 --2- 172.21.15.33:0/3675202707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdb0101370 0x7efdb010d880 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.489+0000 7efdb61cd640 1 --2- 172.21.15.33:0/3675202707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb01009b0 0x7efdb0100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.489+0000 7efdb61cd640 1 --2- 172.21.15.33:0/3675202707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb01069c0 0x7efdb0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.489+0000 7efdb61cd640 1 -- 172.21.15.33:0/3675202707 >> 172.21.15.33:0/3675202707 conn(0x7efdb0076d80 msgr2=0x7efdb0077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.489+0000 7efdb61cd640 1 -- 172.21.15.33:0/3675202707 shutdown_connections 2024-04-02T13:45:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.489+0000 7efdb61cd640 1 -- 172.21.15.33:0/3675202707 wait complete. 2024-04-02T13:45:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.489+0000 7efdb61cd640 1 Processor -- start 2024-04-02T13:45:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdb61cd640 1 -- start start 2024-04-02T13:45:22.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdb61cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdb01009b0 0x7efdb0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdb61cd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0101370 0x7efdb01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdb61cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb01069c0 0x7efdb019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdb61cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efdb0110da0 con 0x7efdb01009b0 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdb61cd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efdb0110c20 con 0x7efdb0101370 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdb61cd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efdb0110f20 con 0x7efdb01069c0 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdaf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdb01009b0 0x7efdb0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdaf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdb01009b0 0x7efdb0198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40300/0 (socket says 172.21.15.33:40300) 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdaf7fe640 1 -- 172.21.15.33:0/618426613 learned_addr learned my addr 172.21.15.33:0/618426613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdaeffd640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0101370 0x7efdb01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdaffff640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb01069c0 0x7efdb019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdaf7fe640 1 -- 172.21.15.33:0/618426613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb01069c0 msgr2=0x7efdb019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdaf7fe640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb01069c0 0x7efdb019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdaf7fe640 1 -- 172.21.15.33:0/618426613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0101370 msgr2=0x7efdb01988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdaf7fe640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0101370 0x7efdb01988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.490+0000 7efdaf7fe640 1 -- 172.21.15.33:0/618426613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb019d350 con 0x7efdb01009b0 2024-04-02T13:45:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.491+0000 7efdaf7fe640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdb01009b0 0x7efdb0198360 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7efda000d950 tx=0x7efda000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.491+0000 7efdacff9640 1 -- 172.21.15.33:0/618426613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0017030 con 0x7efdb01009b0 2024-04-02T13:45:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.491+0000 7efdb61cd640 1 -- 172.21.15.33:0/618426613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb019d640 con 0x7efdb01009b0 2024-04-02T13:45:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.491+0000 7efdacff9640 1 -- 172.21.15.33:0/618426613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efda0004020 con 0x7efdb01009b0 2024-04-02T13:45:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.491+0000 7efdacff9640 1 -- 172.21.15.33:0/618426613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda00047a0 con 0x7efdb01009b0 2024-04-02T13:45:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.491+0000 7efdb61cd640 1 -- 172.21.15.33:0/618426613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb01a4ed0 con 0x7efdb01009b0 2024-04-02T13:45:22.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:22 smithi050 ceph-mon[35524]: pgmap v202: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 219 B/s wr, 7 op/s 2024-04-02T13:45:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.493+0000 7efdacff9640 1 -- 172.21.15.33:0/618426613 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efda00041d0 con 0x7efdb01009b0 2024-04-02T13:45:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.493+0000 7efdb8db3740 1 Processor -- start 2024-04-02T13:45:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.493+0000 7efdb8db3740 1 -- start start 2024-04-02T13:45:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.493+0000 7efdb8db3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff787a0 0x558b4ff78b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.493+0000 7efdb8db3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff790c0 0x558b4ff81760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.493+0000 7efdb8db3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff81ca0 0x558b4ff84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.493+0000 7efdb8db3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b4fedc7b0 con 0x558b4ff81ca0 2024-04-02T13:45:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.493+0000 7efdb8db3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558b4fe7b570 con 0x558b4ff790c0 2024-04-02T13:45:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.493+0000 7efdb8db3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558b4fed8760 con 0x558b4ff787a0 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff81ca0 0x558b4ff84090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdacff9640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efd980a3d20 0x7efd980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff790c0 0x558b4ff81760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff81ca0 0x558b4ff84090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40312/0 (socket says 172.21.15.33:40312) 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaeffd640 1 -- 172.21.15.33:0/1258879485 learned_addr learned my addr 172.21.15.33:0/1258879485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaffff640 1 --2- 172.21.15.33:0/1258879485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff787a0 0x558b4ff78b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaffff640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efd980a3d20 0x7efd980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdacff9640 1 -- 172.21.15.33:0/618426613 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7efda00d0660 con 0x7efdb01009b0 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaeffd640 1 -- 172.21.15.33:0/1258879485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff787a0 msgr2=0x558b4ff78b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaeffd640 1 --2- 172.21.15.33:0/1258879485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff787a0 0x558b4ff78b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaeffd640 1 -- 172.21.15.33:0/1258879485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff790c0 msgr2=0x558b4ff81760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaeffd640 1 --2- 172.21.15.33:0/1258879485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff790c0 0x558b4ff81760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaeffd640 1 -- 172.21.15.33:0/1258879485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b4fbf03f0 con 0x558b4ff81ca0 2024-04-02T13:45:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaf7fe640 1 --2- 172.21.15.33:0/1258879485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff790c0 0x558b4ff81760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaeffd640 1 --2- 172.21.15.33:0/1258879485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff81ca0 0x558b4ff84090 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7efda4009c20 tx=0x7efda40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9d46ba94aaf3c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efdaffff640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efd980a3d20 0x7efd980a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efda8037040 tx=0x7efda80095c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efd95ffb640 1 -- 172.21.15.33:0/1258879485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4012450 con 0x558b4ff81ca0 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efd95ffb640 1 -- 172.21.15.33:0/1258879485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efda4012ac0 con 0x558b4ff81ca0 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.494+0000 7efd95ffb640 1 -- 172.21.15.33:0/1258879485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda400fbf0 con 0x558b4ff81ca0 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.495+0000 7efdb8db3740 1 -- 172.21.15.33:0/1258879485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff81ca0 msgr2=0x558b4ff84090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.495+0000 7efdb8db3740 1 --2- 172.21.15.33:0/1258879485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff81ca0 0x558b4ff84090 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7efda4009c20 tx=0x7efda40117b0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.495+0000 7efdb8db3740 1 -- 172.21.15.33:0/1258879485 shutdown_connections 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.495+0000 7efdb8db3740 1 --2- 172.21.15.33:0/1258879485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff81ca0 0x558b4ff84090 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.495+0000 7efdb8db3740 1 --2- 172.21.15.33:0/1258879485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff790c0 0x558b4ff81760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.495+0000 7efdb8db3740 1 --2- 172.21.15.33:0/1258879485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff787a0 0x558b4ff78b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.495+0000 7efdb8db3740 1 -- 172.21.15.33:0/1258879485 >> 172.21.15.33:0/1258879485 conn(0x558b4ff72ed0 msgr2=0x558b4ff732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.495+0000 7efdb8db3740 1 -- 172.21.15.33:0/1258879485 shutdown_connections 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.495+0000 7efdb8db3740 1 -- 172.21.15.33:0/1258879485 wait complete. 2024-04-02T13:45:22.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.495+0000 7efdb8db3740 1 -- start start 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdb8db3740 1 Processor -- start 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdb8db3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff787a0 0x558b4ff81670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdb8db3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 0x558b4ff7ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdb8db3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 0x558b4ff7f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdb8db3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b4ff84a90 con 0x558b4ff790c0 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdb8db3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558b4fe97e80 con 0x558b4ff787a0 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdb8db3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558b4ff84c10 con 0x558b4ff81ca0 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdb8db3740 10 client.0 _ll_register_callbacks cb 0x558b4fed3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdaeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 0x558b4ff7ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdaeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 0x558b4ff7ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40324/0 (socket says 172.21.15.33:40324) 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdaeffd640 1 -- 172.21.15.33:0/3390673205 learned_addr learned my addr 172.21.15.33:0/3390673205 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdaf7fe640 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff787a0 0x558b4ff81670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdaffff640 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 0x558b4ff7f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdaeffd640 1 -- 172.21.15.33:0/3390673205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 msgr2=0x558b4ff7f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdaeffd640 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 0x558b4ff7f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdaeffd640 1 -- 172.21.15.33:0/3390673205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff787a0 msgr2=0x558b4ff81670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdaeffd640 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff787a0 0x558b4ff81670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdaeffd640 1 -- 172.21.15.33:0/3390673205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b4fcba7d0 con 0x558b4ff790c0 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.496+0000 7efdaffff640 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 0x558b4ff7f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efdaeffd640 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 0x558b4ff7ea00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7efda400f590 tx=0x7efda4012f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5487daaff67339 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efd8effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efd8effd640 1 -- 172.21.15.33:0/3390673205 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4028070 con 0x558b4ff790c0 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efd8effd640 1 -- 172.21.15.33:0/3390673205 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7efda4004450 con 0x558b4ff790c0 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efdb8db3740 1 -- 172.21.15.33:0/3390673205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda4012ac0 con 0x558b4ff790c0 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efd8effd640 1 -- 172.21.15.33:0/3390673205 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efda4034050 con 0x558b4ff790c0 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efdb8db3740 10 client.17055 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efdb8db3740 1 -- 172.21.15.33:0/3390673205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558b4ff90a60 con 0x558b4ff790c0 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efd8effd640 1 -- 172.21.15.33:0/3390673205 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efda4012ac0 con 0x558b4ff790c0 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efd8effd640 1 -- 172.21.15.33:0/3390673205 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7efda40035f0 con 0x558b4ff790c0 2024-04-02T13:45:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efdb8db3740 10 client.17055 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efdb8db3740 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efdb8db3740 20 client.17055 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efdb8db3740 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b4ff91e10 0x558b4ff94200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efdb8db3740 4 client.17055 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.497+0000 7efdb8db3740 1 -- 172.21.15.33:0/3390673205 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b4fee9cf0 con 0x558b4ff91e10 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.498+0000 7efdaf7fe640 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b4ff91e10 0x558b4ff94200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.498+0000 7efdaf7fe640 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b4ff91e10 0x558b4ff94200 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=508330cee7e76205 server_cookie=a04834e447f2a781 in_seq=0 out_seq=0 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.498+0000 7efd8effd640 10 client.17055 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efd8effd640 1 -- 172.21.15.33:0/3390673205 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b4fee9cf0 con 0x558b4ff91e10 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efd8effd640 10 client.17055 handle_command_reply: tid=0 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efdb8db3740 1 client.17055 shutdown 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efdb8db3740 1 -- 172.21.15.33:0/3390673205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 msgr2=0x558b4ff7ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efdb8db3740 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 0x558b4ff7ea00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7efda400f590 tx=0x7efda4012f20 comp rx=0 tx=0).stop 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efdb8db3740 1 -- 172.21.15.33:0/3390673205 shutdown_connections 2024-04-02T13:45:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efdb8db3740 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b4ff91e10 0x558b4ff94200 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efdb8db3740 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 0x558b4ff7f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efd8effd640 0 client.17055 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:22.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efdb8db3740 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 0x558b4ff7ea00 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efdb8db3740 1 --2- 172.21.15.33:0/3390673205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff787a0 0x558b4ff81670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efdb8db3740 1 -- 172.21.15.33:0/3390673205 >> 172.21.15.33:0/3390673205 conn(0x558b4ff72ed0 msgr2=0x558b4ff74c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efdb8db3740 1 -- 172.21.15.33:0/3390673205 shutdown_connections 2024-04-02T13:45:22.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efdb8db3740 1 -- 172.21.15.33:0/3390673205 wait complete. 2024-04-02T13:45:22.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.499+0000 7efdb8db3740 20 client.17055 trim_cache size 0 max 16384 2024-04-02T13:45:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.512+0000 7efdb8db3740 1 Processor -- start 2024-04-02T13:45:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.512+0000 7efdb8db3740 1 -- start start 2024-04-02T13:45:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.512+0000 7efdb8db3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff787a0 0x558b4ff8d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.512+0000 7efdb8db3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff790c0 0x558b4ff7d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.512+0000 7efdb8db3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 0x558b4ff81130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.512+0000 7efdb8db3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01b1990 con 0x558b4ff787a0 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.512+0000 7efdb8db3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01af260 con 0x558b4ff790c0 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.512+0000 7efdb8db3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01ad0d0 con 0x558b4ff81ca0 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.512+0000 7efdaf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff787a0 0x558b4ff8d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.512+0000 7efdaf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff787a0 0x558b4ff8d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40330/0 (socket says 172.21.15.33:40330) 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.512+0000 7efdaf7fe640 1 -- 172.21.15.33:0/403750797 learned_addr learned my addr 172.21.15.33:0/403750797 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.513+0000 7efdaeffd640 1 --2- 172.21.15.33:0/403750797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff790c0 0x558b4ff7d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.513+0000 7efdaf7fe640 1 -- 172.21.15.33:0/403750797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 msgr2=0x558b4ff81130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.513+0000 7efdaf7fe640 1 --2- 172.21.15.33:0/403750797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 0x558b4ff81130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.513+0000 7efdaf7fe640 1 -- 172.21.15.33:0/403750797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff790c0 msgr2=0x558b4ff7d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.513+0000 7efdaf7fe640 1 --2- 172.21.15.33:0/403750797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff790c0 0x558b4ff7d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.513+0000 7efdaf7fe640 1 -- 172.21.15.33:0/403750797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b4ff78ef0 con 0x558b4ff787a0 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.513+0000 7efdaeffd640 1 --2- 172.21.15.33:0/403750797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff790c0 0x558b4ff7d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.513+0000 7efdaf7fe640 1 --2- 172.21.15.33:0/403750797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff787a0 0x558b4ff8d950 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efda00cd980 tx=0x7efda00d5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0971fb1e0eb0860 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.513+0000 7efd957fa640 1 -- 172.21.15.33:0/403750797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda00a05e0 con 0x558b4ff787a0 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.513+0000 7efd957fa640 1 -- 172.21.15.33:0/403750797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efda00a4ea0 con 0x558b4ff787a0 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.513+0000 7efd957fa640 1 -- 172.21.15.33:0/403750797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda00d0270 con 0x558b4ff787a0 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 -- 172.21.15.33:0/403750797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff787a0 msgr2=0x558b4ff8d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 --2- 172.21.15.33:0/403750797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff787a0 0x558b4ff8d950 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efda00cd980 tx=0x7efda00d5040 comp rx=0 tx=0).stop 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 -- 172.21.15.33:0/403750797 shutdown_connections 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 --2- 172.21.15.33:0/403750797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 0x558b4ff81130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 --2- 172.21.15.33:0/403750797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff790c0 0x558b4ff7d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 --2- 172.21.15.33:0/403750797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff787a0 0x558b4ff8d950 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 -- 172.21.15.33:0/403750797 >> 172.21.15.33:0/403750797 conn(0x558b4ff72ed0 msgr2=0x558b4ff7d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 -- 172.21.15.33:0/403750797 shutdown_connections 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 -- 172.21.15.33:0/403750797 wait complete. 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 -- start start 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 Processor -- start 2024-04-02T13:45:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff787a0 0x558b4ff82680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 0x558b4ff82bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 0x558b4ff88e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01ad720 con 0x558b4ff790c0 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01aeb90 con 0x558b4ff787a0 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efdb01addf0 con 0x558b4ff81ca0 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.514+0000 7efdb8db3740 10 client.0 _ll_register_callbacks cb 0x7efda009c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdaeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 0x558b4ff82bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdaeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 0x558b4ff82bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40340/0 (socket says 172.21.15.33:40340) 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdaeffd640 1 -- 172.21.15.33:0/262744266 learned_addr learned my addr 172.21.15.33:0/262744266 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdaf7fe640 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff787a0 0x558b4ff82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdaffff640 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 0x558b4ff88e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdaeffd640 1 -- 172.21.15.33:0/262744266 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 msgr2=0x558b4ff88e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdaeffd640 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 0x558b4ff88e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdaeffd640 1 -- 172.21.15.33:0/262744266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff787a0 msgr2=0x558b4ff82680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdaeffd640 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff787a0 0x558b4ff82680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdaeffd640 1 -- 172.21.15.33:0/262744266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b4ff893a0 con 0x558b4ff790c0 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdaffff640 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 0x558b4ff88e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdaeffd640 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 0x558b4ff82bc0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x558b4ff7ffd0 tx=0x7efda40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce3d82fb180e36a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efd8effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efd8effd640 1 -- 172.21.15.33:0/262744266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda403d060 con 0x558b4ff790c0 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efd8effd640 1 -- 172.21.15.33:0/262744266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7efda40035f0 con 0x558b4ff790c0 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdb8db3740 1 -- 172.21.15.33:0/262744266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda00a4ea0 con 0x558b4ff790c0 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efd8effd640 1 -- 172.21.15.33:0/262744266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efda4044050 con 0x558b4ff790c0 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdb8db3740 10 client.17061 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.515+0000 7efdb8db3740 1 -- 172.21.15.33:0/262744266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558b4ff8a610 con 0x558b4ff790c0 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.516+0000 7efd8effd640 1 -- 172.21.15.33:0/262744266 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7efda00a4ea0 con 0x558b4ff790c0 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.516+0000 7efd8effd640 1 -- 172.21.15.33:0/262744266 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7efda40053e0 con 0x558b4ff790c0 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.516+0000 7efdb8db3740 10 client.17061 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.516+0000 7efdb8db3740 10 client.17061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.516+0000 7efdb8db3740 20 client.17061 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.516+0000 7efdb8db3740 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b4ff91e10 0x558b4fffe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.516+0000 7efdb8db3740 4 client.17061 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.516+0000 7efdb8db3740 1 -- 172.21.15.33:0/262744266 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b4fee4040 con 0x558b4ff91e10 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.516+0000 7efdaf7fe640 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b4ff91e10 0x558b4fffe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.516+0000 7efd8effd640 10 client.17061 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.516+0000 7efdaf7fe640 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b4ff91e10 0x558b4fffe9f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c0dfb99e3cb3901 server_cookie=6bb4e00030ea76b9 in_seq=0 out_seq=0 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efd8effd640 1 -- 172.21.15.33:0/262744266 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b4fee4040 con 0x558b4ff91e10 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efd8effd640 10 client.17061 handle_command_reply: tid=0 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb8db3740 1 client.17061 shutdown 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb8db3740 1 -- 172.21.15.33:0/262744266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 msgr2=0x558b4ff82bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb8db3740 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 0x558b4ff82bc0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x558b4ff7ffd0 tx=0x7efda40057f0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb8db3740 1 -- 172.21.15.33:0/262744266 shutdown_connections 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb8db3740 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558b4ff91e10 0x558b4fffe9f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb8db3740 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b4ff81ca0 0x558b4ff88e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb8db3740 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b4ff790c0 0x558b4ff82bc0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efd8effd640 0 client.17061 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:22.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb8db3740 1 --2- 172.21.15.33:0/262744266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558b4ff787a0 0x558b4ff82680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb8db3740 1 -- 172.21.15.33:0/262744266 >> 172.21.15.33:0/262744266 conn(0x558b4ff72ed0 msgr2=0x558b4ff716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb8db3740 1 -- 172.21.15.33:0/262744266 shutdown_connections 2024-04-02T13:45:22.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb8db3740 1 -- 172.21.15.33:0/262744266 wait complete. 2024-04-02T13:45:22.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb8db3740 20 client.17061 trim_cache size 0 max 16384 2024-04-02T13:45:22.526 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:45:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb61cd640 1 -- 172.21.15.33:0/618426613 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efd980a3d20 msgr2=0x7efd980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb61cd640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efd980a3d20 0x7efd980a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efda8037040 tx=0x7efda80095c0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb61cd640 1 -- 172.21.15.33:0/618426613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdb01009b0 msgr2=0x7efdb0198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb61cd640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdb01009b0 0x7efdb0198360 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7efda000d950 tx=0x7efda000de10 comp rx=0 tx=0).stop 2024-04-02T13:45:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb61cd640 1 -- 172.21.15.33:0/618426613 shutdown_connections 2024-04-02T13:45:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb61cd640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7efd980a3d20 0x7efd980a61e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb61cd640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efdb01069c0 0x7efdb019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb61cd640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdb0101370 0x7efdb01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb61cd640 1 --2- 172.21.15.33:0/618426613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdb01009b0 0x7efdb0198360 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.517+0000 7efdb61cd640 1 -- 172.21.15.33:0/618426613 >> 172.21.15.33:0/618426613 conn(0x7efdb0076d80 msgr2=0x7efdb006a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.518+0000 7efdb61cd640 1 -- 172.21.15.33:0/618426613 shutdown_connections 2024-04-02T13:45:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:22.518+0000 7efdb61cd640 1 -- 172.21.15.33:0/618426613 wait complete. 2024-04-02T13:45:23.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:23 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/109343309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:45:23.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:23 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/109343309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:45:23.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:23 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/109343309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:45:23.532 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:45:23.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:45:23.532 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:45:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.649+0000 7f5ca1d0e640 1 -- 172.21.15.33:0/3958754918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c9c102d40 msgr2=0x7f5c9c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.649+0000 7f5ca1d0e640 1 --2- 172.21.15.33:0/3958754918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c9c102d40 0x7f5c9c103120 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009900 tx=0x7f5c8c02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.649+0000 7f5ca1d0e640 1 -- 172.21.15.33:0/3958754918 shutdown_connections 2024-04-02T13:45:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.649+0000 7f5ca1d0e640 1 --2- 172.21.15.33:0/3958754918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c9c10c110 0x7f5c9c10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.649+0000 7f5ca1d0e640 1 --2- 172.21.15.33:0/3958754918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c9c103660 0x7f5c9c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.649+0000 7f5ca1d0e640 1 --2- 172.21.15.33:0/3958754918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c9c102d40 0x7f5c9c103120 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.649+0000 7f5ca1d0e640 1 -- 172.21.15.33:0/3958754918 >> 172.21.15.33:0/3958754918 conn(0x7f5c9c076d80 msgr2=0x7f5c9c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.649+0000 7f5ca1d0e640 1 -- 172.21.15.33:0/3958754918 shutdown_connections 2024-04-02T13:45:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.649+0000 7f5ca1d0e640 1 -- 172.21.15.33:0/3958754918 wait complete. 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.650+0000 7f5ca1d0e640 1 Processor -- start 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.650+0000 7f5ca1d0e640 1 -- start start 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.650+0000 7f5ca1d0e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c9c102d40 0x7f5c9c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.650+0000 7f5ca1d0e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c9c103660 0x7f5c9c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.650+0000 7f5ca1d0e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c9c10c110 0x7f5c9c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.650+0000 7f5ca1d0e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c111e90 con 0x7f5c9c103660 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.650+0000 7f5ca1d0e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c111d10 con 0x7f5c9c102d40 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.650+0000 7f5ca1d0e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c112010 con 0x7f5c9c10c110 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.650+0000 7f5c9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c9c103660 0x7f5c9c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.650+0000 7f5c9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c9c103660 0x7f5c9c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49308/0 (socket says 172.21.15.33:49308) 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.650+0000 7f5c9affd640 1 -- 172.21.15.33:0/1323582282 learned_addr learned my addr 172.21.15.33:0/1323582282 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.650+0000 7f5c9b7fe640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c9c102d40 0x7f5c9c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.650+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c9c10c110 0x7f5c9c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.651+0000 7f5c9affd640 1 -- 172.21.15.33:0/1323582282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c9c10c110 msgr2=0x7f5c9c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.651+0000 7f5c9affd640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c9c10c110 0x7f5c9c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.651+0000 7f5c9affd640 1 -- 172.21.15.33:0/1323582282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c9c102d40 msgr2=0x7f5c9c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.651+0000 7f5c9affd640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c9c102d40 0x7f5c9c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.651+0000 7f5c9affd640 1 -- 172.21.15.33:0/1323582282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c19d380 con 0x7f5c9c103660 2024-04-02T13:45:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.651+0000 7f5c9affd640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c9c103660 0x7f5c9c1988d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000d770 tx=0x7f5c9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.651+0000 7f5c98ff9640 1 -- 172.21.15.33:0/1323582282 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90017070 con 0x7f5c9c103660 2024-04-02T13:45:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.651+0000 7f5c98ff9640 1 -- 172.21.15.33:0/1323582282 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5c900054f0 con 0x7f5c9c103660 2024-04-02T13:45:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.651+0000 7f5ca1d0e640 1 -- 172.21.15.33:0/1323582282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c19d670 con 0x7f5c9c103660 2024-04-02T13:45:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.651+0000 7f5c98ff9640 1 -- 172.21.15.33:0/1323582282 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90004d60 con 0x7f5c9c103660 2024-04-02T13:45:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.651+0000 7f5ca1d0e640 1 -- 172.21.15.33:0/1323582282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c1a4f00 con 0x7f5c9c103660 2024-04-02T13:45:23.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.653+0000 7f5c98ff9640 1 -- 172.21.15.33:0/1323582282 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5c90004f00 con 0x7f5c9c103660 2024-04-02T13:45:23.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5ca49d3740 1 Processor -- start 2024-04-02T13:45:23.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5ca49d3740 1 -- start start 2024-04-02T13:45:23.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5ca49d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x56088588b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5ca49d3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588bbe0 0x560885894280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5c98ff9640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5c840a3d20 0x7f5c840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5ca49d3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5608858947c0 0x560885896bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5ca49d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5608857ef770 con 0x56088588b2c0 2024-04-02T13:45:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5ca49d3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560885797960 con 0x56088588bbe0 2024-04-02T13:45:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5ca49d3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5608857eb8c0 con 0x5608858947c0 2024-04-02T13:45:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x56088588b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5c9affd640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5c840a3d20 0x7f5c840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x56088588b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49316/0 (socket says 172.21.15.33:49316) 2024-04-02T13:45:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5c9bfff640 1 -- 172.21.15.33:0/3387537128 learned_addr learned my addr 172.21.15.33:0/3387537128 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5c9b7fe640 1 --2- 172.21.15.33:0/3387537128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588bbe0 0x560885894280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5c98ff9640 1 -- 172.21.15.33:0/1323582282 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f5c900d0790 con 0x7f5c9c103660 2024-04-02T13:45:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5c9bfff640 1 -- 172.21.15.33:0/3387537128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5608858947c0 msgr2=0x560885896bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/3387537128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5608858947c0 0x560885896bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5c9bfff640 1 -- 172.21.15.33:0/3387537128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588bbe0 msgr2=0x560885894280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.654+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/3387537128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588bbe0 0x560885894280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.655+0000 7f5c9bfff640 1 -- 172.21.15.33:0/3387537128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608855cb7e0 con 0x56088588b2c0 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.655+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/3387537128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x56088588b6a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009ee0 tx=0x7f5c94012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b8c64242e06b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.655+0000 7f5c9affd640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5c840a3d20 0x7f5c840a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5c9009c000 tx=0x7f5c9006dd40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.655+0000 7f5c81ffb640 1 -- 172.21.15.33:0/3387537128 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9401b600 con 0x56088588b2c0 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.655+0000 7f5c81ffb640 1 -- 172.21.15.33:0/3387537128 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5c940138d0 con 0x56088588b2c0 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.655+0000 7f5c81ffb640 1 -- 172.21.15.33:0/3387537128 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94010910 con 0x56088588b2c0 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.655+0000 7f5ca49d3740 1 -- 172.21.15.33:0/3387537128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 msgr2=0x56088588b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.655+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/3387537128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x56088588b6a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009ee0 tx=0x7f5c94012760 comp rx=0 tx=0).stop 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 -- 172.21.15.33:0/3387537128 shutdown_connections 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/3387537128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5608858947c0 0x560885896bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/3387537128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588bbe0 0x560885894280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/3387537128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x56088588b6a0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 -- 172.21.15.33:0/3387537128 >> 172.21.15.33:0/3387537128 conn(0x5608858859f0 msgr2=0x560885885dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 -- 172.21.15.33:0/3387537128 shutdown_connections 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 -- 172.21.15.33:0/3387537128 wait complete. 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 -- start start 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 Processor -- start 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x560885891800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56088588bbe0 0x560885891d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5608858947c0 0x560885892280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5608858975b0 con 0x56088588b2c0 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5608857aad20 con 0x5608858947c0 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560885897730 con 0x56088588bbe0 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.656+0000 7f5ca49d3740 10 client.0 _ll_register_callbacks cb 0x560885777a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x560885891800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x560885891800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49330/0 (socket says 172.21.15.33:49330) 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c9bfff640 1 -- 172.21.15.33:0/3256542098 learned_addr learned my addr 172.21.15.33:0/3256542098 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c9b7fe640 1 --2- 172.21.15.33:0/3256542098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56088588bbe0 0x560885891d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/3256542098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5608858947c0 0x560885892280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c9bfff640 1 -- 172.21.15.33:0/3256542098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56088588bbe0 msgr2=0x560885891d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/3256542098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56088588bbe0 0x560885891d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c9bfff640 1 -- 172.21.15.33:0/3256542098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5608858947c0 msgr2=0x560885892280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/3256542098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5608858947c0 0x560885892280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c9bfff640 1 -- 172.21.15.33:0/3256542098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56088562bf70 con 0x56088588b2c0 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/3256542098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x560885891800 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5c94010590 tx=0x7f5c94005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd101ae1646cff66 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c7affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c7affd640 1 -- 172.21.15.33:0/3256542098 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94013dc0 con 0x56088588b2c0 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c7affd640 1 -- 172.21.15.33:0/3256542098 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f5c940106f0 con 0x56088588b2c0 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5ca49d3740 1 -- 172.21.15.33:0/3256542098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c940138d0 con 0x56088588b2c0 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5c7affd640 1 -- 172.21.15.33:0/3256542098 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5c94033050 con 0x56088588b2c0 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5ca49d3740 10 client.17073 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.657+0000 7f5ca49d3740 1 -- 172.21.15.33:0/3256542098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5608854c3170 con 0x56088588b2c0 2024-04-02T13:45:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.658+0000 7f5c7affd640 1 -- 172.21.15.33:0/3256542098 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5c940138d0 con 0x56088588b2c0 2024-04-02T13:45:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.658+0000 7f5c7affd640 1 -- 172.21.15.33:0/3256542098 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f5c9400f040 con 0x56088588b2c0 2024-04-02T13:45:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.658+0000 7f5ca49d3740 10 client.17073 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.658+0000 7f5ca49d3740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.658+0000 7f5ca49d3740 20 client.17073 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.658+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/3256542098 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5608858a4b00 0x5608858a6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.658+0000 7f5ca49d3740 4 client.17073 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.658+0000 7f5ca49d3740 1 -- 172.21.15.33:0/3256542098 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608857fccb0 con 0x5608858a4b00 2024-04-02T13:45:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.658+0000 7f5c9b7fe640 1 --2- 172.21.15.33:0/3256542098 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5608858a4b00 0x5608858a6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.659+0000 7f5c9b7fe640 1 --2- 172.21.15.33:0/3256542098 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5608858a4b00 0x5608858a6ef0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a1cbfe562b9fb7 server_cookie=75ff46273cb23f33 in_seq=0 out_seq=0 2024-04-02T13:45:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.659+0000 7f5c7affd640 10 client.17073 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:23.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.659+0000 7f5c7affd640 1 -- 172.21.15.33:0/3256542098 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608857fccb0 con 0x5608858a4b00 2024-04-02T13:45:23.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.659+0000 7f5c7affd640 10 client.17073 handle_command_reply: tid=0 2024-04-02T13:45:23.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.659+0000 7f5ca49d3740 1 client.17073 shutdown 2024-04-02T13:45:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.660+0000 7f5ca49d3740 1 -- 172.21.15.33:0/3256542098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 msgr2=0x560885891800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.660+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/3256542098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x560885891800 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5c94010590 tx=0x7f5c94005f70 comp rx=0 tx=0).stop 2024-04-02T13:45:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.660+0000 7f5ca49d3740 1 -- 172.21.15.33:0/3256542098 shutdown_connections 2024-04-02T13:45:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.660+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/3256542098 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5608858a4b00 0x5608858a6ef0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.660+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/3256542098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5608858947c0 0x560885892280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.660+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/3256542098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56088588bbe0 0x560885891d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.660+0000 7f5c7affd640 0 client.17073 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.660+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/3256542098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x560885891800 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.660+0000 7f5ca49d3740 1 -- 172.21.15.33:0/3256542098 >> 172.21.15.33:0/3256542098 conn(0x5608858859f0 msgr2=0x560885887740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.660+0000 7f5ca49d3740 1 -- 172.21.15.33:0/3256542098 shutdown_connections 2024-04-02T13:45:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.660+0000 7f5ca49d3740 1 -- 172.21.15.33:0/3256542098 wait complete. 2024-04-02T13:45:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.660+0000 7f5ca49d3740 20 client.17073 trim_cache size 0 max 16384 2024-04-02T13:45:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5ca49d3740 1 Processor -- start 2024-04-02T13:45:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5ca49d3740 1 -- start start 2024-04-02T13:45:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5ca49d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x560885893f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5ca49d3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588bbe0 0x5608858a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5ca49d3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5608858947c0 0x5608858ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5ca49d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1b1840 con 0x56088588b2c0 2024-04-02T13:45:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5ca49d3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1af110 con 0x56088588bbe0 2024-04-02T13:45:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5ca49d3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1acf80 con 0x5608858947c0 2024-04-02T13:45:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x560885893f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x560885893f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49342/0 (socket says 172.21.15.33:49342) 2024-04-02T13:45:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5c9bfff640 1 -- 172.21.15.33:0/4057285805 learned_addr learned my addr 172.21.15.33:0/4057285805 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5c9b7fe640 1 --2- 172.21.15.33:0/4057285805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588bbe0 0x5608858a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5c9bfff640 1 -- 172.21.15.33:0/4057285805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5608858947c0 msgr2=0x5608858ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/4057285805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5608858947c0 0x5608858ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5c9bfff640 1 -- 172.21.15.33:0/4057285805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588bbe0 msgr2=0x5608858a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/4057285805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588bbe0 0x5608858a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5c9bfff640 1 -- 172.21.15.33:0/4057285805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56088588ba10 con 0x56088588b2c0 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.678+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/4057285805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x560885893f30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5c94010560 tx=0x7f5c94013400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4850cda4c097efd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.679+0000 7f5c817fa640 1 -- 172.21.15.33:0/4057285805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94009580 con 0x56088588b2c0 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.679+0000 7f5c817fa640 1 -- 172.21.15.33:0/4057285805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5c9402f070 con 0x56088588b2c0 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.679+0000 7f5c817fa640 1 -- 172.21.15.33:0/4057285805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9401b400 con 0x56088588b2c0 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.679+0000 7f5ca49d3740 1 -- 172.21.15.33:0/4057285805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 msgr2=0x560885893f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.679+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/4057285805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x560885893f30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5c94010560 tx=0x7f5c94013400 comp rx=0 tx=0).stop 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.679+0000 7f5ca49d3740 1 -- 172.21.15.33:0/4057285805 shutdown_connections 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.679+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/4057285805 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5608858947c0 0x5608858ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.679+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/4057285805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588bbe0 0x5608858a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.679+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/4057285805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588b2c0 0x560885893f30 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.679+0000 7f5ca49d3740 1 -- 172.21.15.33:0/4057285805 >> 172.21.15.33:0/4057285805 conn(0x5608858859f0 msgr2=0x560885884ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.679+0000 7f5ca49d3740 1 -- 172.21.15.33:0/4057285805 shutdown_connections 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.679+0000 7f5ca49d3740 1 -- 172.21.15.33:0/4057285805 wait complete. 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.679+0000 7f5ca49d3740 1 -- start start 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5ca49d3740 1 Processor -- start 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5ca49d3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588b2c0 0x560885898c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5ca49d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588bbe0 0x5608858991b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5ca49d3740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5608858947c0 0x560885909df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5ca49d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1ad5d0 con 0x56088588bbe0 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5ca49d3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1aea40 con 0x56088588b2c0 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5ca49d3740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1adca0 con 0x5608858947c0 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5ca49d3740 10 client.0 _ll_register_callbacks cb 0x7f5c8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5c9b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588bbe0 0x5608858991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5c9b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588bbe0 0x5608858991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49344/0 (socket says 172.21.15.33:49344) 2024-04-02T13:45:23.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5c9b7fe640 1 -- 172.21.15.33:0/1849025310 learned_addr learned my addr 172.21.15.33:0/1849025310 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/1849025310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588b2c0 0x560885898c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/1849025310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5608858947c0 0x560885909df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5c9b7fe640 1 -- 172.21.15.33:0/1849025310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5608858947c0 msgr2=0x560885909df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5c9b7fe640 1 --2- 172.21.15.33:0/1849025310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5608858947c0 0x560885909df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5c9b7fe640 1 -- 172.21.15.33:0/1849025310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588b2c0 msgr2=0x560885898c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5c9b7fe640 1 --2- 172.21.15.33:0/1849025310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588b2c0 0x560885898c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5c9b7fe640 1 -- 172.21.15.33:0/1849025310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56088590a3e0 con 0x56088588bbe0 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5c9b7fe640 1 --2- 172.21.15.33:0/1849025310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588bbe0 0x5608858991b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c005ab0 tx=0x7f5c8c02c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3640cb28d7f387 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.680+0000 7f5c7affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5c7affd640 1 -- 172.21.15.33:0/1849025310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c005140 con 0x56088588bbe0 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5c7affd640 1 -- 172.21.15.33:0/1849025310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f5c8c0052e0 con 0x56088588bbe0 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5ca49d3740 1 -- 172.21.15.33:0/1849025310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c9402f070 con 0x56088588bbe0 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5c7affd640 1 -- 172.21.15.33:0/1849025310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5c8c057050 con 0x56088588bbe0 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5ca49d3740 10 client.17079 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:23.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5ca49d3740 1 -- 172.21.15.33:0/1849025310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56088590b790 con 0x56088588bbe0 2024-04-02T13:45:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5c7affd640 1 -- 172.21.15.33:0/1849025310 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5c9402f070 con 0x56088588bbe0 2024-04-02T13:45:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5c7affd640 1 -- 172.21.15.33:0/1849025310 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f5c8c004a30 con 0x56088588bbe0 2024-04-02T13:45:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5ca49d3740 10 client.17079 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5ca49d3740 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5ca49d3740 20 client.17079 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/1849025310 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56088590bf90 0x56088590e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5ca49d3740 4 client.17079 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:45:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5ca49d3740 1 -- 172.21.15.33:0/1849025310 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5608857f7000 con 0x56088590bf90 2024-04-02T13:45:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.681+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/1849025310 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56088590bf90 0x56088590e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.682+0000 7f5c9bfff640 1 --2- 172.21.15.33:0/1849025310 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56088590bf90 0x56088590e3b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d6f1612fa68cc65 server_cookie=9b578872b2237a2 in_seq=0 out_seq=0 2024-04-02T13:45:23.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.682+0000 7f5c7affd640 10 client.17079 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:23.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5c7affd640 1 -- 172.21.15.33:0/1849025310 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5608857f7000 con 0x56088590bf90 2024-04-02T13:45:23.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5c7affd640 10 client.17079 handle_command_reply: tid=0 2024-04-02T13:45:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5ca49d3740 1 client.17079 shutdown 2024-04-02T13:45:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5ca49d3740 1 -- 172.21.15.33:0/1849025310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588bbe0 msgr2=0x5608858991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/1849025310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588bbe0 0x5608858991b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c005ab0 tx=0x7f5c8c02c350 comp rx=0 tx=0).stop 2024-04-02T13:45:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5ca49d3740 1 -- 172.21.15.33:0/1849025310 shutdown_connections 2024-04-02T13:45:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/1849025310 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x56088590bf90 0x56088590e3b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/1849025310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5608858947c0 0x560885909df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/1849025310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56088588bbe0 0x5608858991b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5c7affd640 0 client.17079 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5ca49d3740 1 --2- 172.21.15.33:0/1849025310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56088588b2c0 0x560885898c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5ca49d3740 1 -- 172.21.15.33:0/1849025310 >> 172.21.15.33:0/1849025310 conn(0x5608858859f0 msgr2=0x560885886d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5ca49d3740 1 -- 172.21.15.33:0/1849025310 shutdown_connections 2024-04-02T13:45:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5ca49d3740 1 -- 172.21.15.33:0/1849025310 wait complete. 2024-04-02T13:45:23.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.683+0000 7f5ca49d3740 20 client.17079 trim_cache size 0 max 16384 2024-04-02T13:45:23.692 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:45:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:45:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "100fd986-4556-4448-8e0c-065713cbf498", 2024-04-02T13:45:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:45:23.692 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:45:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.684+0000 7f5ca1d0e640 1 -- 172.21.15.33:0/1323582282 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5c840a3d20 msgr2=0x7f5c840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.684+0000 7f5ca1d0e640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5c840a3d20 0x7f5c840a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5c9009c000 tx=0x7f5c9006dd40 comp rx=0 tx=0).stop 2024-04-02T13:45:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.684+0000 7f5ca1d0e640 1 -- 172.21.15.33:0/1323582282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c9c103660 msgr2=0x7f5c9c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.684+0000 7f5ca1d0e640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c9c103660 0x7f5c9c1988d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000d770 tx=0x7f5c9000dc30 comp rx=0 tx=0).stop 2024-04-02T13:45:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.684+0000 7f5ca1d0e640 1 -- 172.21.15.33:0/1323582282 shutdown_connections 2024-04-02T13:45:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.684+0000 7f5ca1d0e640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5c840a3d20 0x7f5c840a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.684+0000 7f5ca1d0e640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5c9c10c110 0x7f5c9c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.684+0000 7f5ca1d0e640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c9c103660 0x7f5c9c1988d0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.684+0000 7f5ca1d0e640 1 --2- 172.21.15.33:0/1323582282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c9c102d40 0x7f5c9c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.684+0000 7f5ca1d0e640 1 -- 172.21.15.33:0/1323582282 >> 172.21.15.33:0/1323582282 conn(0x7f5c9c076d80 msgr2=0x7f5c9c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.684+0000 7f5ca1d0e640 1 -- 172.21.15.33:0/1323582282 shutdown_connections 2024-04-02T13:45:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.684+0000 7f5ca1d0e640 1 -- 172.21.15.33:0/1323582282 wait complete. 2024-04-02T13:45:23.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:45:23.698 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:45:23.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.800+0000 7f8095e86640 1 -- 172.21.15.33:0/1853644216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8090102d70 msgr2=0x7f8090103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.800+0000 7f8095e86640 1 --2- 172.21.15.33:0/1853644216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8090102d70 0x7f8090103150 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8080009900 tx=0x7f808002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.800+0000 7f8095e86640 1 -- 172.21.15.33:0/1853644216 shutdown_connections 2024-04-02T13:45:23.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.800+0000 7f8095e86640 1 --2- 172.21.15.33:0/1853644216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809010c140 0x7f809010e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.800+0000 7f8095e86640 1 --2- 172.21.15.33:0/1853644216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8090103690 0x7f809010bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.800+0000 7f8095e86640 1 --2- 172.21.15.33:0/1853644216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8090102d70 0x7f8090103150 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.800+0000 7f8095e86640 1 -- 172.21.15.33:0/1853644216 >> 172.21.15.33:0/1853644216 conn(0x7f8090077b60 msgr2=0x7f8090077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.800+0000 7f8095e86640 1 -- 172.21.15.33:0/1853644216 shutdown_connections 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.800+0000 7f8095e86640 1 -- 172.21.15.33:0/1853644216 wait complete. 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.801+0000 7f8095e86640 1 Processor -- start 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.801+0000 7f8095e86640 1 -- start start 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.801+0000 7f8095e86640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8090102d70 0x7f80901983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.801+0000 7f8095e86640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8090103690 0x7f80901988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.801+0000 7f8095e86640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809010c140 0x7f809019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.801+0000 7f8095e86640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8090111ee0 con 0x7f8090103690 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.801+0000 7f8095e86640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8090111d60 con 0x7f809010c140 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.801+0000 7f8095e86640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8090112060 con 0x7f8090102d70 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.801+0000 7f808effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8090103690 0x7f80901988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.801+0000 7f808effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8090103690 0x7f80901988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49366/0 (socket says 172.21.15.33:49366) 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.801+0000 7f808effd640 1 -- 172.21.15.33:0/3289961783 learned_addr learned my addr 172.21.15.33:0/3289961783 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f808ffff640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809010c140 0x7f809019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f808f7fe640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8090102d70 0x7f80901983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f808effd640 1 -- 172.21.15.33:0/3289961783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8090102d70 msgr2=0x7f80901983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f808effd640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8090102d70 0x7f80901983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f808effd640 1 -- 172.21.15.33:0/3289961783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809010c140 msgr2=0x7f809019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f808effd640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809010c140 0x7f809019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f808effd640 1 -- 172.21.15.33:0/3289961783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809019d340 con 0x7f8090103690 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f808ffff640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809010c140 0x7f809019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f808effd640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8090103690 0x7f80901988e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f807800c920 tx=0x7f807800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f808cff9640 1 -- 172.21.15.33:0/3289961783 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078007c70 con 0x7f8090103690 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f8095e86640 1 -- 172.21.15.33:0/3289961783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809019d630 con 0x7f8090103690 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f808cff9640 1 -- 172.21.15.33:0/3289961783 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8078007e10 con 0x7f8090103690 2024-04-02T13:45:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f808cff9640 1 -- 172.21.15.33:0/3289961783 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078018800 con 0x7f8090103690 2024-04-02T13:45:23.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.802+0000 7f8095e86640 1 -- 172.21.15.33:0/3289961783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80901a4ec0 con 0x7f8090103690 2024-04-02T13:45:23.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.805+0000 7f8098a6e740 1 Processor -- start 2024-04-02T13:45:23.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.805+0000 7f8098a6e740 1 -- start start 2024-04-02T13:45:23.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.805+0000 7f808cff9640 1 -- 172.21.15.33:0/3289961783 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8078018a20 con 0x7f8090103690 2024-04-02T13:45:23.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.805+0000 7f8098a6e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29508d0 0x563ae2950cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.805+0000 7f8098a6e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29511f0 0x563ae2959890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.805+0000 7f8098a6e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 0x563ae295c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.805+0000 7f8098a6e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563ae28b4830 con 0x563ae29508d0 2024-04-02T13:45:23.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.805+0000 7f8098a6e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563ae285c820 con 0x563ae29511f0 2024-04-02T13:45:23.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.805+0000 7f8098a6e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563ae28b07e0 con 0x563ae2959dd0 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.805+0000 7f808ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29508d0 0x563ae2950cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.805+0000 7f808ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29508d0 0x563ae2950cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49374/0 (socket says 172.21.15.33:49374) 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.805+0000 7f808ffff640 1 -- 172.21.15.33:0/2376736026 learned_addr learned my addr 172.21.15.33:0/2376736026 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.805+0000 7f808f7fe640 1 --2- 172.21.15.33:0/2376736026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29511f0 0x563ae2959890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.806+0000 7f808effd640 1 --2- 172.21.15.33:0/2376736026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 0x563ae295c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.806+0000 7f808ffff640 1 -- 172.21.15.33:0/2376736026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 msgr2=0x563ae295c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.806+0000 7f808ffff640 1 --2- 172.21.15.33:0/2376736026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 0x563ae295c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.806+0000 7f808ffff640 1 -- 172.21.15.33:0/2376736026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29511f0 msgr2=0x563ae2959890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.806+0000 7f808ffff640 1 --2- 172.21.15.33:0/2376736026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29511f0 0x563ae2959890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.806+0000 7f808cff9640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f807c0a3d20 0x7f807c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.806+0000 7f808ffff640 1 -- 172.21.15.33:0/2376736026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae26927d0 con 0x563ae29508d0 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.806+0000 7f808effd640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f807c0a3d20 0x7f807c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.806+0000 7f808ffff640 1 --2- 172.21.15.33:0/2376736026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29508d0 0x563ae2950cb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8088009fd0 tx=0x7f808800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c537404c4b12d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.806+0000 7f808cff9640 1 -- 172.21.15.33:0/3289961783 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f80780d0de0 con 0x7f8090103690 2024-04-02T13:45:23.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.806+0000 7f8085ffb640 1 -- 172.21.15.33:0/2376736026 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088015070 con 0x563ae29508d0 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.806+0000 7f8085ffb640 1 -- 172.21.15.33:0/2376736026 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8088011bd0 con 0x563ae29508d0 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.806+0000 7f8085ffb640 1 -- 172.21.15.33:0/2376736026 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088010c00 con 0x563ae29508d0 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.807+0000 7f8098a6e740 1 -- 172.21.15.33:0/2376736026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29508d0 msgr2=0x563ae2950cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.807+0000 7f808effd640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f807c0a3d20 0x7f807c0a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f80780ce210 tx=0x7f8078087040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.807+0000 7f8098a6e740 1 --2- 172.21.15.33:0/2376736026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29508d0 0x563ae2950cb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8088009fd0 tx=0x7f808800f7f0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.807+0000 7f8098a6e740 1 -- 172.21.15.33:0/2376736026 shutdown_connections 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.807+0000 7f8098a6e740 1 --2- 172.21.15.33:0/2376736026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 0x563ae295c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.807+0000 7f8098a6e740 1 --2- 172.21.15.33:0/2376736026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29511f0 0x563ae2959890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.807+0000 7f8098a6e740 1 --2- 172.21.15.33:0/2376736026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29508d0 0x563ae2950cb0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.807+0000 7f8098a6e740 1 -- 172.21.15.33:0/2376736026 >> 172.21.15.33:0/2376736026 conn(0x563ae294b000 msgr2=0x563ae294b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.807+0000 7f8098a6e740 1 -- 172.21.15.33:0/2376736026 shutdown_connections 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.807+0000 7f8098a6e740 1 -- 172.21.15.33:0/2376736026 wait complete. 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.807+0000 7f8098a6e740 1 -- start start 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.807+0000 7f8098a6e740 1 Processor -- start 2024-04-02T13:45:23.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f8098a6e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 0x563ae2959720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f8098a6e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 0x563ae2956ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f8098a6e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 0x563ae2957010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f8098a6e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563ae295cbc0 con 0x563ae29511f0 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f8098a6e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563ae286fd30 con 0x563ae29508d0 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f8098a6e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563ae295cd40 con 0x563ae2959dd0 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f8098a6e740 10 client.0 _ll_register_callbacks cb 0x563ae286f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f808f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 0x563ae2956ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f808f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 0x563ae2956ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49384/0 (socket says 172.21.15.33:49384) 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f808f7fe640 1 -- 172.21.15.33:0/3359658979 learned_addr learned my addr 172.21.15.33:0/3359658979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f808ffff640 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 0x563ae2957010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f808ffff640 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 0x563ae2959720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f808f7fe640 1 -- 172.21.15.33:0/3359658979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 msgr2=0x563ae2957010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f808f7fe640 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 0x563ae2957010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f808f7fe640 1 -- 172.21.15.33:0/3359658979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 msgr2=0x563ae2959720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f808f7fe640 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 0x563ae2959720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f808f7fe640 1 -- 172.21.15.33:0/3359658979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae25c83f0 con 0x563ae29511f0 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f808ffff640 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 0x563ae2959720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f808f7fe640 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 0x563ae2956ab0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8080009900 tx=0x7f8080005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d444c9ec8fed92 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f806effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f806effd640 1 -- 172.21.15.33:0/3359658979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080043030 con 0x563ae29511f0 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f806effd640 1 -- 172.21.15.33:0/3359658979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f808002c670 con 0x563ae29511f0 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.808+0000 7f8098a6e740 1 -- 172.21.15.33:0/3359658979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8088011bd0 con 0x563ae29511f0 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.809+0000 7f806effd640 1 -- 172.21.15.33:0/3359658979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8080050050 con 0x563ae29511f0 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.809+0000 7f8098a6e740 10 client.17091 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.809+0000 7f8098a6e740 1 -- 172.21.15.33:0/3359658979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563ae25880f0 con 0x563ae29511f0 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.809+0000 7f806effd640 1 -- 172.21.15.33:0/3359658979 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8088011bd0 con 0x563ae29511f0 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.809+0000 7f806effd640 1 -- 172.21.15.33:0/3359658979 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f8080004c60 con 0x563ae29511f0 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.809+0000 7f8098a6e740 10 client.17091 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.809+0000 7f8098a6e740 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:23.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.809+0000 7f8098a6e740 20 client.17091 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.809+0000 7f8098a6e740 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563ae2969e20 0x563ae296c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.809+0000 7f8098a6e740 4 client.17091 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.809+0000 7f8098a6e740 1 -- 172.21.15.33:0/3359658979 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ae28c1d70 con 0x563ae2969e20 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.809+0000 7f808ffff640 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563ae2969e20 0x563ae296c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.810+0000 7f806effd640 10 client.17091 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.810+0000 7f808ffff640 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563ae2969e20 0x563ae296c260 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceebdcc402298165 server_cookie=3c6bbeb899be8dc1 in_seq=0 out_seq=0 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.810+0000 7f806effd640 1 -- 172.21.15.33:0/3359658979 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ae28c1d70 con 0x563ae2969e20 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.810+0000 7f806effd640 10 client.17091 handle_command_reply: tid=0 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.811+0000 7f8098a6e740 1 client.17091 shutdown 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.811+0000 7f8098a6e740 1 -- 172.21.15.33:0/3359658979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 msgr2=0x563ae2956ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.811+0000 7f8098a6e740 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 0x563ae2956ab0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8080009900 tx=0x7f8080005250 comp rx=0 tx=0).stop 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.811+0000 7f8098a6e740 1 -- 172.21.15.33:0/3359658979 shutdown_connections 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.811+0000 7f8098a6e740 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563ae2969e20 0x563ae296c260 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.811+0000 7f8098a6e740 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 0x563ae2957010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.811+0000 7f8098a6e740 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 0x563ae2956ab0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.811+0000 7f806effd640 0 client.17091 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:23.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.811+0000 7f8098a6e740 1 --2- 172.21.15.33:0/3359658979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 0x563ae2959720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.811+0000 7f8098a6e740 1 -- 172.21.15.33:0/3359658979 >> 172.21.15.33:0/3359658979 conn(0x563ae294b000 msgr2=0x563ae294cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:23.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.811+0000 7f8098a6e740 1 -- 172.21.15.33:0/3359658979 shutdown_connections 2024-04-02T13:45:23.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.811+0000 7f8098a6e740 1 -- 172.21.15.33:0/3359658979 wait complete. 2024-04-02T13:45:23.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.811+0000 7f8098a6e740 20 client.17091 trim_cache size 0 max 16384 2024-04-02T13:45:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f8098a6e740 1 Processor -- start 2024-04-02T13:45:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f8098a6e740 1 -- start start 2024-04-02T13:45:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f8098a6e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 0x563ae294a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f8098a6e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 0x563ae294caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f8098a6e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 0x563ae29591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f8098a6e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80901b1890 con 0x563ae29511f0 2024-04-02T13:45:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f8098a6e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f80901af160 con 0x563ae29508d0 2024-04-02T13:45:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f8098a6e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f80901acfd0 con 0x563ae2959dd0 2024-04-02T13:45:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f808f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 0x563ae294caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f808f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 0x563ae294caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49398/0 (socket says 172.21.15.33:49398) 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f808f7fe640 1 -- 172.21.15.33:0/7503562 learned_addr learned my addr 172.21.15.33:0/7503562 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f808ffff640 1 --2- 172.21.15.33:0/7503562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 0x563ae294a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f808ffff640 1 --2- 172.21.15.33:0/7503562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 0x563ae29591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f808f7fe640 1 -- 172.21.15.33:0/7503562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 msgr2=0x563ae29591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f808f7fe640 1 --2- 172.21.15.33:0/7503562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 0x563ae29591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f808f7fe640 1 -- 172.21.15.33:0/7503562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 msgr2=0x563ae294a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f808f7fe640 1 --2- 172.21.15.33:0/7503562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 0x563ae294a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f808f7fe640 1 -- 172.21.15.33:0/7503562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae294ae30 con 0x563ae29511f0 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f808ffff640 1 --2- 172.21.15.33:0/7503562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 0x563ae29591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.829+0000 7f808f7fe640 1 --2- 172.21.15.33:0/7503562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 0x563ae294caf0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f808002c210 tx=0x7f808002bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50561192c3e0a67f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.830+0000 7f80857fa640 1 -- 172.21.15.33:0/7503562 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080043040 con 0x563ae29511f0 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.830+0000 7f80857fa640 1 -- 172.21.15.33:0/7503562 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f80800055d0 con 0x563ae29511f0 2024-04-02T13:45:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.830+0000 7f80857fa640 1 -- 172.21.15.33:0/7503562 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80800046c0 con 0x563ae29511f0 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.830+0000 7f8098a6e740 1 -- 172.21.15.33:0/7503562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 msgr2=0x563ae294caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.830+0000 7f8098a6e740 1 --2- 172.21.15.33:0/7503562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 0x563ae294caf0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f808002c210 tx=0x7f808002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.830+0000 7f8098a6e740 1 -- 172.21.15.33:0/7503562 shutdown_connections 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.830+0000 7f8098a6e740 1 --2- 172.21.15.33:0/7503562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae2959dd0 0x563ae29591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.830+0000 7f8098a6e740 1 --2- 172.21.15.33:0/7503562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae29511f0 0x563ae294caf0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.830+0000 7f8098a6e740 1 --2- 172.21.15.33:0/7503562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 0x563ae294a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.830+0000 7f8098a6e740 1 -- 172.21.15.33:0/7503562 >> 172.21.15.33:0/7503562 conn(0x563ae294b000 msgr2=0x563ae294c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.830+0000 7f8098a6e740 1 -- 172.21.15.33:0/7503562 shutdown_connections 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.830+0000 7f8098a6e740 1 -- 172.21.15.33:0/7503562 wait complete. 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f8098a6e740 1 -- start start 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f8098a6e740 1 Processor -- start 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f8098a6e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 0x563ae295daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f8098a6e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae29511f0 0x563ae295dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f8098a6e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae2959dd0 0x563ae295e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f8098a6e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80901ad620 con 0x563ae2959dd0 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f8098a6e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f80901aea90 con 0x563ae29508d0 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f8098a6e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f80901adcf0 con 0x563ae29511f0 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f8098a6e740 10 client.0 _ll_register_callbacks cb 0x7f8080004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f808ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae2959dd0 0x563ae295e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f808ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae2959dd0 0x563ae295e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49406/0 (socket says 172.21.15.33:49406) 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f808ffff640 1 -- 172.21.15.33:0/3867376481 learned_addr learned my addr 172.21.15.33:0/3867376481 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f808ffff640 1 --2- 172.21.15.33:0/3867376481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 0x563ae295daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f808f7fe640 1 --2- 172.21.15.33:0/3867376481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae29511f0 0x563ae295dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f808ffff640 1 -- 172.21.15.33:0/3867376481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae29511f0 msgr2=0x563ae295dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f808ffff640 1 --2- 172.21.15.33:0/3867376481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae29511f0 0x563ae295dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.831+0000 7f808ffff640 1 -- 172.21.15.33:0/3867376481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 msgr2=0x563ae295daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f808ffff640 1 --2- 172.21.15.33:0/3867376481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 0x563ae295daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f808ffff640 1 -- 172.21.15.33:0/3867376481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae29cf4a0 con 0x563ae2959dd0 2024-04-02T13:45:23.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f808ffff640 1 --2- 172.21.15.33:0/3867376481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae2959dd0 0x563ae295e520 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8088010630 tx=0x7f8080004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d979a8169e31cb37 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f806effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f806effd640 1 -- 172.21.15.33:0/3867376481 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088024040 con 0x563ae2959dd0 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f806effd640 1 -- 172.21.15.33:0/3867376481 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f8088004be0 con 0x563ae2959dd0 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f8098a6e740 1 -- 172.21.15.33:0/3867376481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80800055d0 con 0x563ae2959dd0 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f806effd640 1 -- 172.21.15.33:0/3867376481 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8088015040 con 0x563ae2959dd0 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f8098a6e740 10 client.17097 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f8098a6e740 1 -- 172.21.15.33:0/3867376481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563ae29d0760 con 0x563ae2959dd0 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f806effd640 1 -- 172.21.15.33:0/3867376481 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f80800055d0 con 0x563ae2959dd0 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f806effd640 1 -- 172.21.15.33:0/3867376481 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f8088005030 con 0x563ae2959dd0 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f8098a6e740 10 client.17097 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f8098a6e740 10 client.17097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f8098a6e740 20 client.17097 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f8098a6e740 1 --2- 172.21.15.33:0/3867376481 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563ae29d0da0 0x563ae29d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f8098a6e740 4 client.17097 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.832+0000 7f8098a6e740 1 -- 172.21.15.33:0/3867376481 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ae28bc0c0 con 0x563ae29d0da0 2024-04-02T13:45:23.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.833+0000 7f808f7fe640 1 --2- 172.21.15.33:0/3867376481 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563ae29d0da0 0x563ae29d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.833+0000 7f806effd640 10 client.17097 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.833+0000 7f808f7fe640 1 --2- 172.21.15.33:0/3867376481 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563ae29d0da0 0x563ae29d31c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dafbd7ff2c579f78 server_cookie=a01726529f4ffcc0 in_seq=0 out_seq=0 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.833+0000 7f806effd640 1 -- 172.21.15.33:0/3867376481 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563ae28bc0c0 con 0x563ae29d0da0 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.833+0000 7f806effd640 10 client.17097 handle_command_reply: tid=0 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.833+0000 7f8098a6e740 1 client.17097 shutdown 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.833+0000 7f8098a6e740 1 -- 172.21.15.33:0/3867376481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae2959dd0 msgr2=0x563ae295e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.833+0000 7f8098a6e740 1 --2- 172.21.15.33:0/3867376481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae2959dd0 0x563ae295e520 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8088010630 tx=0x7f8080004490 comp rx=0 tx=0).stop 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8098a6e740 1 -- 172.21.15.33:0/3867376481 shutdown_connections 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8098a6e740 1 --2- 172.21.15.33:0/3867376481 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x563ae29d0da0 0x563ae29d31c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8098a6e740 1 --2- 172.21.15.33:0/3867376481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ae2959dd0 0x563ae295e520 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8098a6e740 1 --2- 172.21.15.33:0/3867376481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563ae29511f0 0x563ae295dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f806effd640 0 client.17097 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8098a6e740 1 --2- 172.21.15.33:0/3867376481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ae29508d0 0x563ae295daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8098a6e740 1 -- 172.21.15.33:0/3867376481 >> 172.21.15.33:0/3867376481 conn(0x563ae294b000 msgr2=0x563ae2951fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8098a6e740 1 -- 172.21.15.33:0/3867376481 shutdown_connections 2024-04-02T13:45:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8098a6e740 1 -- 172.21.15.33:0/3867376481 wait complete. 2024-04-02T13:45:23.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8098a6e740 20 client.17097 trim_cache size 0 max 16384 2024-04-02T13:45:23.843 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:45:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:45:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:45:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "100fd986-4556-4448-8e0c-065713cbf498": { 2024-04-02T13:45:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:45:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "100fd986-4556-4448-8e0c-065713cbf498", 2024-04-02T13:45:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:45:23.844 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:45:23.844 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:45:23.844 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:45:23.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8095e86640 1 -- 172.21.15.33:0/3289961783 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f807c0a3d20 msgr2=0x7f807c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8095e86640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f807c0a3d20 0x7f807c0a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f80780ce210 tx=0x7f8078087040 comp rx=0 tx=0).stop 2024-04-02T13:45:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8095e86640 1 -- 172.21.15.33:0/3289961783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8090103690 msgr2=0x7f80901988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8095e86640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8090103690 0x7f80901988e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f807800c920 tx=0x7f807800cde0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8095e86640 1 -- 172.21.15.33:0/3289961783 shutdown_connections 2024-04-02T13:45:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8095e86640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f807c0a3d20 0x7f807c0a61e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8095e86640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f809010c140 0x7f809019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8095e86640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8090103690 0x7f80901988e0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8095e86640 1 --2- 172.21.15.33:0/3289961783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8090102d70 0x7f80901983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8095e86640 1 -- 172.21.15.33:0/3289961783 >> 172.21.15.33:0/3289961783 conn(0x7f8090077b60 msgr2=0x7f80900fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8095e86640 1 -- 172.21.15.33:0/3289961783 shutdown_connections 2024-04-02T13:45:23.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:23.834+0000 7f8095e86640 1 -- 172.21.15.33:0/3289961783 wait complete. 2024-04-02T13:45:23.848 INFO:tasks.cephfs.filesystem:scrub status for tag:100fd986-4556-4448-8e0c-065713cbf498 - {'path': '/', 'tag': '100fd986-4556-4448-8e0c-065713cbf498', 'options': 'recursive,force'} 2024-04-02T13:45:24.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:24 smithi033 ceph-mon[32390]: pgmap v203: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:45:24.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:24 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:45:24.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:24 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:24.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:24 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:45:24.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:24 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:24.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:24 smithi145 ceph-mon[35646]: pgmap v203: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:45:24.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:24 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:45:24.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:24 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:24.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:24 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:45:24.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:24 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:24.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:24 smithi050 ceph-mon[35524]: pgmap v203: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:45:24.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:24 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:45:24.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:24 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:24.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:24 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:45:24.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:24 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:26.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:26 smithi145 ceph-mon[35646]: pgmap v204: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:45:26.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:26 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:45:26.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:26 smithi050 ceph-mon[35524]: pgmap v204: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:45:26.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:26 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:45:26.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:26 smithi033 ceph-mon[32390]: pgmap v204: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:45:26.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:26 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:45:28.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:28 smithi145 ceph-mon[35646]: pgmap v205: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 98 B/s wr, 6 op/s 2024-04-02T13:45:28.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:28 smithi050 ceph-mon[35524]: pgmap v205: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 98 B/s wr, 6 op/s 2024-04-02T13:45:28.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:28 smithi033 ceph-mon[32390]: pgmap v205: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 98 B/s wr, 6 op/s 2024-04-02T13:45:30.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:30 smithi145 ceph-mon[35646]: pgmap v206: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T13:45:30.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:30 smithi050 ceph-mon[35524]: pgmap v206: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T13:45:30.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:30 smithi033 ceph-mon[32390]: pgmap v206: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-02T13:45:32.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:32 smithi145 ceph-mon[35646]: pgmap v207: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:45:32.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:32 smithi050 ceph-mon[35524]: pgmap v207: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:45:32.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:32 smithi033 ceph-mon[32390]: pgmap v207: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:45:34.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:34 smithi145 ceph-mon[35646]: pgmap v208: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:45:34.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:34 smithi050 ceph-mon[35524]: pgmap v208: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:45:34.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:34 smithi033 ceph-mon[32390]: pgmap v208: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:45:36.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:36 smithi145 ceph-mon[35646]: pgmap v209: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:45:36.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:36 smithi050 ceph-mon[35524]: pgmap v209: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:45:36.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:36 smithi033 ceph-mon[32390]: pgmap v209: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:45:38.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:38 smithi145 ceph-mon[35646]: pgmap v210: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:45:38.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:38 smithi050 ceph-mon[35524]: pgmap v210: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:45:38.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:38 smithi033 ceph-mon[32390]: pgmap v210: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:45:40.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:40 smithi145 ceph-mon[35646]: pgmap v211: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:45:40.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:40 smithi050 ceph-mon[35524]: pgmap v211: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:45:40.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:40 smithi033 ceph-mon[32390]: pgmap v211: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:45:42.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:42 smithi145 ceph-mon[35646]: pgmap v212: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:42.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:42 smithi050 ceph-mon[35524]: pgmap v212: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:42.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:42 smithi033 ceph-mon[32390]: pgmap v212: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:44.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:44 smithi145 ceph-mon[35646]: pgmap v213: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:45:44.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:44 smithi050 ceph-mon[35524]: pgmap v213: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:45:44.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:44 smithi033 ceph-mon[32390]: pgmap v213: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:45:46.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:46 smithi145 ceph-mon[35646]: pgmap v214: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:46.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:46 smithi050 ceph-mon[35524]: pgmap v214: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:46.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:46 smithi033 ceph-mon[32390]: pgmap v214: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:48.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:48 smithi145 ceph-mon[35646]: pgmap v215: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:48.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:48 smithi050 ceph-mon[35524]: pgmap v215: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:48.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:48 smithi033 ceph-mon[32390]: pgmap v215: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:50.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:50 smithi145 ceph-mon[35646]: pgmap v216: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:45:50.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:50 smithi050 ceph-mon[35524]: pgmap v216: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:45:50.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:50 smithi033 ceph-mon[32390]: pgmap v216: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:45:52.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:52 smithi145 ceph-mon[35646]: pgmap v217: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:52.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:52 smithi050 ceph-mon[35524]: pgmap v217: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:52.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:52 smithi033 ceph-mon[32390]: pgmap v217: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:53.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:45:53.849 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:45:53.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.963+0000 7fd1b5a06640 1 -- 172.21.15.33:0/39788233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b0106a30 msgr2=0x7fd1b0106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:53.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.963+0000 7fd1b5a06640 1 --2- 172.21.15.33:0/39788233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b0106a30 0x7fd1b0106e10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009930 tx=0x7fd1a002bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:53.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.963+0000 7fd1b5a06640 1 -- 172.21.15.33:0/39788233 shutdown_connections 2024-04-02T13:45:53.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.963+0000 7fd1b5a06640 1 --2- 172.21.15.33:0/39788233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1b01013e0 0x7fd1b010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.963+0000 7fd1b5a06640 1 --2- 172.21.15.33:0/39788233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1b0100a20 0x7fd1b0100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.963+0000 7fd1b5a06640 1 --2- 172.21.15.33:0/39788233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b0106a30 0x7fd1b0106e10 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.963+0000 7fd1b5a06640 1 -- 172.21.15.33:0/39788233 >> 172.21.15.33:0/39788233 conn(0x7fd1b0077b60 msgr2=0x7fd1b0077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.963+0000 7fd1b5a06640 1 -- 172.21.15.33:0/39788233 shutdown_connections 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.963+0000 7fd1b5a06640 1 -- 172.21.15.33:0/39788233 wait complete. 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1b5a06640 1 Processor -- start 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1b5a06640 1 -- start start 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1b5a06640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1b0100a20 0x7fd1b0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1b5a06640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b01013e0 0x7fd1b0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1b5a06640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1b0106a30 0x7fd1b019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1b5a06640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b0111f10 con 0x7fd1b01013e0 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1b5a06640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b0111d90 con 0x7fd1b0106a30 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1b5a06640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b0112090 con 0x7fd1b0100a20 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b01013e0 0x7fd1b0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b01013e0 0x7fd1b0198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60132/0 (socket says 172.21.15.33:60132) 2024-04-02T13:45:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1a7fff640 1 -- 172.21.15.33:0/1750388488 learned_addr learned my addr 172.21.15.33:0/1750388488 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1aeffd640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1b0100a20 0x7fd1b0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1af7fe640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1b0106a30 0x7fd1b019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1a7fff640 1 -- 172.21.15.33:0/1750388488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1b0100a20 msgr2=0x7fd1b0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1a7fff640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1b0100a20 0x7fd1b0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1a7fff640 1 -- 172.21.15.33:0/1750388488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1b0106a30 msgr2=0x7fd1b019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1a7fff640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1b0106a30 0x7fd1b019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1a7fff640 1 -- 172.21.15.33:0/1750388488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b019d3d0 con 0x7fd1b01013e0 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.964+0000 7fd1aeffd640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1b0100a20 0x7fd1b0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.965+0000 7fd1a7fff640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b01013e0 0x7fd1b0198970 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd19800d770 tx=0x7fd19800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.965+0000 7fd1b4a04640 1 -- 172.21.15.33:0/1750388488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198017070 con 0x7fd1b01013e0 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.965+0000 7fd1b5a06640 1 -- 172.21.15.33:0/1750388488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b019d6c0 con 0x7fd1b01013e0 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.965+0000 7fd1b4a04640 1 -- 172.21.15.33:0/1750388488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd1980054f0 con 0x7fd1b01013e0 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.965+0000 7fd1b4a04640 1 -- 172.21.15.33:0/1750388488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198004d60 con 0x7fd1b01013e0 2024-04-02T13:45:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.965+0000 7fd1b5a06640 1 -- 172.21.15.33:0/1750388488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b01a4f50 con 0x7fd1b01013e0 2024-04-02T13:45:53.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.967+0000 7fd1b4a04640 1 -- 172.21.15.33:0/1750388488 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd198004f00 con 0x7fd1b01013e0 2024-04-02T13:45:53.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.967+0000 7fd1b85ed740 1 Processor -- start 2024-04-02T13:45:53.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.967+0000 7fd1b85ed740 1 -- start start 2024-04-02T13:45:53.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1b85ed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf8dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:53.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1b85ed740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 0x55f55bf96890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:53.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1b85ed740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf96dd0 0x55f55bf991c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:53.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1b4a04640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd19c0a3d20 0x7fd19c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:53.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1b85ed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f55bef1830 con 0x55f55bf8d8d0 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1b85ed740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f55be99820 con 0x55f55bf96dd0 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1b85ed740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f55beed7e0 con 0x55f55bf8e1f0 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf8dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf8dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60146/0 (socket says 172.21.15.33:60146) 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1af7fe640 1 -- 172.21.15.33:0/2327506671 learned_addr learned my addr 172.21.15.33:0/2327506671 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1aeffd640 1 --2- 172.21.15.33:0/2327506671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 0x55f55bf96890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1af7fe640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd19c0a3d20 0x7fd19c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1b4a04640 1 -- 172.21.15.33:0/1750388488 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fd1980d0730 con 0x7fd1b01013e0 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1a7fff640 1 --2- 172.21.15.33:0/2327506671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf96dd0 0x55f55bf991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1af7fe640 1 -- 172.21.15.33:0/2327506671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 msgr2=0x55f55bf96890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1af7fe640 1 --2- 172.21.15.33:0/2327506671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 0x55f55bf96890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1af7fe640 1 -- 172.21.15.33:0/2327506671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf96dd0 msgr2=0x55f55bf991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1af7fe640 1 --2- 172.21.15.33:0/2327506671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf96dd0 0x55f55bf991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1af7fe640 1 -- 172.21.15.33:0/2327506671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f55bccf7d0 con 0x55f55bf8d8d0 2024-04-02T13:45:53.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1af7fe640 1 --2- 172.21.15.33:0/2327506671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf8dcb0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80094d0 tx=0x7fd1a80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4b8c7862bb0034 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.968+0000 7fd1af7fe640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd19c0a3d20 0x7fd19c0a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005c40 tx=0x7fd1a801d000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.969+0000 7fd1a5ffb640 1 -- 172.21.15.33:0/2327506671 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8012840 con 0x55f55bf8d8d0 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.969+0000 7fd1a5ffb640 1 -- 172.21.15.33:0/2327506671 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd1a8009d60 con 0x55f55bf8d8d0 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.969+0000 7fd1a5ffb640 1 -- 172.21.15.33:0/2327506671 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8004030 con 0x55f55bf8d8d0 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.969+0000 7fd1b85ed740 1 -- 172.21.15.33:0/2327506671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 msgr2=0x55f55bf8dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.969+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/2327506671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf8dcb0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80094d0 tx=0x7fd1a80107a0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.969+0000 7fd1b85ed740 1 -- 172.21.15.33:0/2327506671 shutdown_connections 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.969+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/2327506671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf96dd0 0x55f55bf991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.969+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/2327506671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 0x55f55bf96890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.969+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/2327506671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf8dcb0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.969+0000 7fd1b85ed740 1 -- 172.21.15.33:0/2327506671 >> 172.21.15.33:0/2327506671 conn(0x55f55bf88000 msgr2=0x55f55bf883e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.969+0000 7fd1b85ed740 1 -- 172.21.15.33:0/2327506671 shutdown_connections 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.969+0000 7fd1b85ed740 1 -- 172.21.15.33:0/2327506671 wait complete. 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1b85ed740 1 -- start start 2024-04-02T13:45:53.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1b85ed740 1 Processor -- start 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1b85ed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf96720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1b85ed740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 0x55f55bf93ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1b85ed740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf96dd0 0x55f55bf94010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1b85ed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f55bf99b60 con 0x55f55bf8d8d0 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1b85ed740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f55beacd30 con 0x55f55bf96dd0 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1b85ed740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f55bf99ce0 con 0x55f55bf8e1f0 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1b85ed740 10 client.0 _ll_register_callbacks cb 0x55f55beac720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf96720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf96720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60148/0 (socket says 172.21.15.33:60148) 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1aeffd640 1 -- 172.21.15.33:0/1638115440 learned_addr learned my addr 172.21.15.33:0/1638115440 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1a7fff640 1 --2- 172.21.15.33:0/1638115440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 0x55f55bf93ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.970+0000 7fd1af7fe640 1 --2- 172.21.15.33:0/1638115440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf96dd0 0x55f55bf94010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd1aeffd640 1 -- 172.21.15.33:0/1638115440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 msgr2=0x55f55bf93ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd1aeffd640 1 --2- 172.21.15.33:0/1638115440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 0x55f55bf93ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd1aeffd640 1 -- 172.21.15.33:0/1638115440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf96dd0 msgr2=0x55f55bf94010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd1aeffd640 1 --2- 172.21.15.33:0/1638115440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf96dd0 0x55f55bf94010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd1aeffd640 1 -- 172.21.15.33:0/1638115440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f55bc053f0 con 0x55f55bf8d8d0 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd1aeffd640 1 --2- 172.21.15.33:0/1638115440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf96720 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0005ab0 tx=0x7fd1a0005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad66fa01d1de0ad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd18effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd18effd640 1 -- 172.21.15.33:0/1638115440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a002c7d0 con 0x55f55bf8d8d0 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd18effd640 1 -- 172.21.15.33:0/1638115440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fd1a002c970 con 0x55f55bf8d8d0 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd1b85ed740 1 -- 172.21.15.33:0/1638115440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f55befed70 con 0x55f55bf8d8d0 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd18effd640 1 -- 172.21.15.33:0/1638115440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd1a002cb70 con 0x55f55bf8d8d0 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd1b85ed740 10 client.17109 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd1b85ed740 1 -- 172.21.15.33:0/1638115440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f55bbc50f0 con 0x55f55bf8d8d0 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.971+0000 7fd18effd640 1 -- 172.21.15.33:0/1638115440 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55f55befed70 con 0x55f55bf8d8d0 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.972+0000 7fd18effd640 1 -- 172.21.15.33:0/1638115440 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fd1a0003790 con 0x55f55bf8d8d0 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.972+0000 7fd1b85ed740 10 client.17109 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:53.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.972+0000 7fd1b85ed740 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.972+0000 7fd1b85ed740 20 client.17109 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.972+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/1638115440 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f55bfa6dc0 0x55f55bfa91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.972+0000 7fd1b85ed740 4 client.17109 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.972+0000 7fd1b85ed740 1 -- 172.21.15.33:0/1638115440 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f55bef90c0 con 0x55f55bfa6dc0 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.972+0000 7fd1a7fff640 1 --2- 172.21.15.33:0/1638115440 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f55bfa6dc0 0x55f55bfa91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.972+0000 7fd18effd640 10 client.17109 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.972+0000 7fd1a7fff640 1 --2- 172.21.15.33:0/1638115440 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f55bfa6dc0 0x55f55bfa91b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6beb515a1ca848f5 server_cookie=7e02482817975eb5 in_seq=0 out_seq=0 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd18effd640 1 -- 172.21.15.33:0/1638115440 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f55bef90c0 con 0x55f55bfa6dc0 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd18effd640 10 client.17109 handle_command_reply: tid=0 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd1b85ed740 1 client.17109 shutdown 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd1b85ed740 1 -- 172.21.15.33:0/1638115440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 msgr2=0x55f55bf96720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/1638115440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf96720 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0005ab0 tx=0x7fd1a0005250 comp rx=0 tx=0).stop 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd1b85ed740 1 -- 172.21.15.33:0/1638115440 shutdown_connections 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/1638115440 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f55bfa6dc0 0x55f55bfa91b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/1638115440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf96dd0 0x55f55bf94010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd18effd640 0 client.17109 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:53.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/1638115440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 0x55f55bf93ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/1638115440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf96720 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:53.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd1b85ed740 1 -- 172.21.15.33:0/1638115440 >> 172.21.15.33:0/1638115440 conn(0x55f55bf88000 msgr2=0x55f55bf89d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:53.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd1b85ed740 1 -- 172.21.15.33:0/1638115440 shutdown_connections 2024-04-02T13:45:53.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd1b85ed740 1 -- 172.21.15.33:0/1638115440 wait complete. 2024-04-02T13:45:53.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.973+0000 7fd1b85ed740 20 client.17109 trim_cache size 0 max 16384 2024-04-02T13:45:54.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1b85ed740 1 Processor -- start 2024-04-02T13:45:54.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1b85ed740 1 -- start start 2024-04-02T13:45:54.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1b85ed740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf8d8d0 0x55f55bf869b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1b85ed740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 0x55f55bf961e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1b85ed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf96dd0 0x55f55bf96720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1b85ed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b01b1890 con 0x55f55bf96dd0 2024-04-02T13:45:54.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1b85ed740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b01af160 con 0x55f55bf8d8d0 2024-04-02T13:45:54.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1b85ed740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b01acfd0 con 0x55f55bf8e1f0 2024-04-02T13:45:54.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf96dd0 0x55f55bf96720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf96dd0 0x55f55bf96720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60158/0 (socket says 172.21.15.33:60158) 2024-04-02T13:45:54.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1af7fe640 1 -- 172.21.15.33:0/692881263 learned_addr learned my addr 172.21.15.33:0/692881263 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1aeffd640 1 --2- 172.21.15.33:0/692881263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf8d8d0 0x55f55bf869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1af7fe640 1 -- 172.21.15.33:0/692881263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 msgr2=0x55f55bf961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1af7fe640 1 --2- 172.21.15.33:0/692881263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 0x55f55bf961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1af7fe640 1 -- 172.21.15.33:0/692881263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf8d8d0 msgr2=0x55f55bf869b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.991+0000 7fd1af7fe640 1 --2- 172.21.15.33:0/692881263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf8d8d0 0x55f55bf869b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.992+0000 7fd1af7fe640 1 -- 172.21.15.33:0/692881263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f55bf94380 con 0x55f55bf96dd0 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.992+0000 7fd1aeffd640 1 --2- 172.21.15.33:0/692881263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf8d8d0 0x55f55bf869b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.992+0000 7fd1af7fe640 1 --2- 172.21.15.33:0/692881263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf96dd0 0x55f55bf96720 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8006010 tx=0x7fd1a8010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c751af786cdbb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.992+0000 7fd1a57fa640 1 -- 172.21.15.33:0/692881263 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8012840 con 0x55f55bf96dd0 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.992+0000 7fd1a57fa640 1 -- 172.21.15.33:0/692881263 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd1a8012050 con 0x55f55bf96dd0 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.992+0000 7fd1a57fa640 1 -- 172.21.15.33:0/692881263 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8005350 con 0x55f55bf96dd0 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.992+0000 7fd1b85ed740 1 -- 172.21.15.33:0/692881263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf96dd0 msgr2=0x55f55bf96720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.992+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/692881263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf96dd0 0x55f55bf96720 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8006010 tx=0x7fd1a8010f70 comp rx=0 tx=0).stop 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.992+0000 7fd1b85ed740 1 -- 172.21.15.33:0/692881263 shutdown_connections 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.992+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/692881263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf96dd0 0x55f55bf96720 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.992+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/692881263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf8e1f0 0x55f55bf961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.992+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/692881263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf8d8d0 0x55f55bf869b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.992+0000 7fd1b85ed740 1 -- 172.21.15.33:0/692881263 >> 172.21.15.33:0/692881263 conn(0x55f55bf88000 msgr2=0x55f55bf8efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1b85ed740 1 -- 172.21.15.33:0/692881263 shutdown_connections 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1b85ed740 1 -- 172.21.15.33:0/692881263 wait complete. 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1b85ed740 1 -- start start 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1b85ed740 1 Processor -- start 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1b85ed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf9f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1b85ed740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf8e1f0 0x55f55bf9b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1b85ed740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf96dd0 0x55f55bf9ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1b85ed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b01ad620 con 0x55f55bf8d8d0 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1b85ed740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b01aea90 con 0x55f55bf8e1f0 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1b85ed740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b01adcf0 con 0x55f55bf96dd0 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1b85ed740 10 client.0 _ll_register_callbacks cb 0x7fd1a0004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:54.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf9f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf9f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60174/0 (socket says 172.21.15.33:60174) 2024-04-02T13:45:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1aeffd640 1 -- 172.21.15.33:0/2180753082 learned_addr learned my addr 172.21.15.33:0/2180753082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.993+0000 7fd1a7fff640 1 --2- 172.21.15.33:0/2180753082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf8e1f0 0x55f55bf9b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd1af7fe640 1 --2- 172.21.15.33:0/2180753082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf96dd0 0x55f55bf9ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd1aeffd640 1 -- 172.21.15.33:0/2180753082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf96dd0 msgr2=0x55f55bf9ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd1aeffd640 1 --2- 172.21.15.33:0/2180753082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf96dd0 0x55f55bf9ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd1aeffd640 1 -- 172.21.15.33:0/2180753082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf8e1f0 msgr2=0x55f55bf9b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd1aeffd640 1 --2- 172.21.15.33:0/2180753082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf8e1f0 0x55f55bf9b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd1aeffd640 1 -- 172.21.15.33:0/2180753082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f55bf9c090 con 0x55f55bf8d8d0 2024-04-02T13:45:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd1aeffd640 1 --2- 172.21.15.33:0/2180753082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf9f140 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0000c00 tx=0x7fd1a0005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d8ee9732cd8ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd18effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd18effd640 1 -- 172.21.15.33:0/2180753082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a002cd10 con 0x55f55bf8d8d0 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd18effd640 1 -- 172.21.15.33:0/2180753082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fd1a00046c0 con 0x55f55bf8d8d0 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd1b85ed740 1 -- 172.21.15.33:0/2180753082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1a8012050 con 0x55f55bf8d8d0 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd18effd640 1 -- 172.21.15.33:0/2180753082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd1a0062050 con 0x55f55bf8d8d0 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd1b85ed740 10 client.17115 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd1b85ed740 1 -- 172.21.15.33:0/2180753082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f55c012490 con 0x55f55bf8d8d0 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.994+0000 7fd18effd640 1 -- 172.21.15.33:0/2180753082 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd1a8012050 con 0x55f55bf8d8d0 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.995+0000 7fd18effd640 1 -- 172.21.15.33:0/2180753082 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fd1a005d030 con 0x55f55bf8d8d0 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.995+0000 7fd1b85ed740 10 client.17115 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.995+0000 7fd1b85ed740 10 client.17115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.995+0000 7fd1b85ed740 20 client.17115 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.995+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/2180753082 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f55c012c60 0x55f55c015080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.995+0000 7fd1b85ed740 4 client.17115 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:45:54.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.995+0000 7fd1b85ed740 1 -- 172.21.15.33:0/2180753082 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f55bf87860 con 0x55f55c012c60 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.995+0000 7fd1a7fff640 1 --2- 172.21.15.33:0/2180753082 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f55c012c60 0x55f55c015080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.995+0000 7fd18effd640 10 client.17115 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.995+0000 7fd1a7fff640 1 --2- 172.21.15.33:0/2180753082 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f55c012c60 0x55f55c015080 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ec9688f56dfecd server_cookie=1e1c8ab34f3a52ed in_seq=0 out_seq=0 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.995+0000 7fd18effd640 1 -- 172.21.15.33:0/2180753082 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f55bf87860 con 0x55f55c012c60 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.995+0000 7fd18effd640 10 client.17115 handle_command_reply: tid=0 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.995+0000 7fd1b85ed740 1 client.17115 shutdown 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b85ed740 1 -- 172.21.15.33:0/2180753082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 msgr2=0x55f55bf9f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/2180753082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf9f140 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0000c00 tx=0x7fd1a0005e10 comp rx=0 tx=0).stop 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b85ed740 1 -- 172.21.15.33:0/2180753082 shutdown_connections 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/2180753082 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f55c012c60 0x55f55c015080 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/2180753082 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f55bf96dd0 0x55f55bf9ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/2180753082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f55bf8e1f0 0x55f55bf9b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd18effd640 0 client.17115 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b85ed740 1 --2- 172.21.15.33:0/2180753082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f55bf8d8d0 0x55f55bf9f140 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b85ed740 1 -- 172.21.15.33:0/2180753082 >> 172.21.15.33:0/2180753082 conn(0x55f55bf88000 msgr2=0x55f55bf892e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b85ed740 1 -- 172.21.15.33:0/2180753082 shutdown_connections 2024-04-02T13:45:54.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b85ed740 1 -- 172.21.15.33:0/2180753082 wait complete. 2024-04-02T13:45:54.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b85ed740 20 client.17115 trim_cache size 0 max 16384 2024-04-02T13:45:54.006 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:45:54.006 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:45:54.006 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:45:54.006 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:45:54.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b5a06640 1 -- 172.21.15.33:0/1750388488 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd19c0a3d20 msgr2=0x7fd19c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b5a06640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd19c0a3d20 0x7fd19c0a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005c40 tx=0x7fd1a801d000 comp rx=0 tx=0).stop 2024-04-02T13:45:54.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b5a06640 1 -- 172.21.15.33:0/1750388488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b01013e0 msgr2=0x7fd1b0198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b5a06640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b01013e0 0x7fd1b0198970 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd19800d770 tx=0x7fd19800dc30 comp rx=0 tx=0).stop 2024-04-02T13:45:54.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b5a06640 1 -- 172.21.15.33:0/1750388488 shutdown_connections 2024-04-02T13:45:54.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b5a06640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd19c0a3d20 0x7fd19c0a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b5a06640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1b0106a30 0x7fd1b019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b5a06640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b01013e0 0x7fd1b0198970 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b5a06640 1 --2- 172.21.15.33:0/1750388488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd1b0100a20 0x7fd1b0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b5a06640 1 -- 172.21.15.33:0/1750388488 >> 172.21.15.33:0/1750388488 conn(0x7fd1b0077b60 msgr2=0x7fd1b0103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.996+0000 7fd1b5a06640 1 -- 172.21.15.33:0/1750388488 shutdown_connections 2024-04-02T13:45:54.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:53.997+0000 7fd1b5a06640 1 -- 172.21.15.33:0/1750388488 wait complete. 2024-04-02T13:45:54.011 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:45:54.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:45:54.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.111+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2951766212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e4106a10 msgr2=0x7fd5e4106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.111+0000 7fd5e89b5640 1 --2- 172.21.15.33:0/2951766212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e4106a10 0x7fd5e4106df0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009930 tx=0x7fd5d402bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:54.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.112+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2951766212 shutdown_connections 2024-04-02T13:45:54.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.112+0000 7fd5e89b5640 1 --2- 172.21.15.33:0/2951766212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e41013c0 0x7fd5e410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.112+0000 7fd5e89b5640 1 --2- 172.21.15.33:0/2951766212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5e4100a00 0x7fd5e4100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.112+0000 7fd5e89b5640 1 --2- 172.21.15.33:0/2951766212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e4106a10 0x7fd5e4106df0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.112+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2951766212 >> 172.21.15.33:0/2951766212 conn(0x7fd5e4077b00 msgr2=0x7fd5e4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.112+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2951766212 shutdown_connections 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.112+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2951766212 wait complete. 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.112+0000 7fd5e89b5640 1 Processor -- start 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.112+0000 7fd5e89b5640 1 -- start start 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e89b5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5e4100a00 0x7fd5e4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e89b5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e41013c0 0x7fd5e4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e89b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e4106a10 0x7fd5e419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e89b5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4110df0 con 0x7fd5e4106a10 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e89b5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4110c70 con 0x7fd5e41013c0 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e89b5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4110f70 con 0x7fd5e4100a00 2024-04-02T13:45:54.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e4106a10 0x7fd5e419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e4106a10 0x7fd5e419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60196/0 (socket says 172.21.15.33:60196) 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e2d9d640 1 -- 172.21.15.33:0/2376447356 learned_addr learned my addr 172.21.15.33:0/2376447356 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e259c640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5e4100a00 0x7fd5e4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e1d9b640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e41013c0 0x7fd5e4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e2d9d640 1 -- 172.21.15.33:0/2376447356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5e4100a00 msgr2=0x7fd5e4197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e2d9d640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5e4100a00 0x7fd5e4197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e2d9d640 1 -- 172.21.15.33:0/2376447356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e41013c0 msgr2=0x7fd5e4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e2d9d640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e41013c0 0x7fd5e4197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e2d9d640 1 -- 172.21.15.33:0/2376447356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e419c2d0 con 0x7fd5e4106a10 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e2d9d640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e4106a10 0x7fd5e419bba0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009fd0 tx=0x7fd5dc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5db7fe640 1 -- 172.21.15.33:0/2376447356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc017070 con 0x7fd5e4106a10 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5db7fe640 1 -- 172.21.15.33:0/2376447356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd5dc003e40 con 0x7fd5e4106a10 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2376447356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e419c5c0 con 0x7fd5e4106a10 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5db7fe640 1 -- 172.21.15.33:0/2376447356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0106c0 con 0x7fd5e4106a10 2024-04-02T13:45:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.113+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2376447356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e41a3e50 con 0x7fd5e4106a10 2024-04-02T13:45:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.115+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2376447356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e40688f0 con 0x7fd5e4106a10 2024-04-02T13:45:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.115+0000 7fd5db7fe640 1 -- 172.21.15.33:0/2376447356 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd5dc003970 con 0x7fd5e4106a10 2024-04-02T13:45:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.116+0000 7fd5db7fe640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd5d00a3d20 0x7fd5d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.116+0000 7fd5db7fe640 1 -- 172.21.15.33:0/2376447356 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fd5dc0d0850 con 0x7fd5e4106a10 2024-04-02T13:45:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.116+0000 7fd5e259c640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd5d00a3d20 0x7fd5d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.116+0000 7fd5e259c640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd5d00a3d20 0x7fd5d00a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4000c00 tx=0x7fd5d4005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.122+0000 7fd5db7fe640 1 -- 172.21.15.33:0/2376447356 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd5dc003c60 con 0x7fd5e4106a10 2024-04-02T13:45:54.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:54 smithi033 ceph-mon[32390]: pgmap v218: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:45:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.345+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2376447356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5e4108f70 con 0x7fd5e4106a10 2024-04-02T13:45:54.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.346+0000 7fd5db7fe640 1 -- 172.21.15.33:0/2376447356 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11873 (secure 0 0 0) 0x7fd5dc09c4d0 con 0x7fd5e4106a10 2024-04-02T13:45:54.349 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:45:54.350 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:44:55.702357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:45:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 36 2024-04-02T13:45:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.348+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2376447356 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd5d00a3d20 msgr2=0x7fd5d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.348+0000 7fd5e89b5640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd5d00a3d20 0x7fd5d00a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4000c00 tx=0x7fd5d4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.348+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2376447356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e4106a10 msgr2=0x7fd5e419bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.348+0000 7fd5e89b5640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e4106a10 0x7fd5e419bba0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009fd0 tx=0x7fd5dc00b260 comp rx=0 tx=0).stop 2024-04-02T13:45:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.348+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2376447356 shutdown_connections 2024-04-02T13:45:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.348+0000 7fd5e89b5640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd5d00a3d20 0x7fd5d00a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.348+0000 7fd5e89b5640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5e4106a10 0x7fd5e419bba0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.348+0000 7fd5e89b5640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e41013c0 0x7fd5e4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.348+0000 7fd5e89b5640 1 --2- 172.21.15.33:0/2376447356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5e4100a00 0x7fd5e4197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.348+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2376447356 >> 172.21.15.33:0/2376447356 conn(0x7fd5e4077b00 msgr2=0x7fd5e41032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.348+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2376447356 shutdown_connections 2024-04-02T13:45:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.348+0000 7fd5e89b5640 1 -- 172.21.15.33:0/2376447356 wait complete. 2024-04-02T13:45:54.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:45:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.463+0000 7ff3d88b8640 1 -- 172.21.15.33:0/2693772992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d4100c90 msgr2=0x7ff3d41067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.463+0000 7ff3d88b8640 1 --2- 172.21.15.33:0/2693772992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d4100c90 0x7ff3d41067c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009900 tx=0x7ff3bc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.463+0000 7ff3d88b8640 1 -- 172.21.15.33:0/2693772992 shutdown_connections 2024-04-02T13:45:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.463+0000 7ff3d88b8640 1 --2- 172.21.15.33:0/2693772992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3d4106ef0 0x7ff3d410ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.463+0000 7ff3d88b8640 1 --2- 172.21.15.33:0/2693772992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d4100c90 0x7ff3d41067c0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.463+0000 7ff3d88b8640 1 --2- 172.21.15.33:0/2693772992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3d41002e0 0x7ff3d41006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.463+0000 7ff3d88b8640 1 -- 172.21.15.33:0/2693772992 >> 172.21.15.33:0/2693772992 conn(0x7ff3d4077030 msgr2=0x7ff3d40fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.463+0000 7ff3d88b8640 1 -- 172.21.15.33:0/2693772992 shutdown_connections 2024-04-02T13:45:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.463+0000 7ff3d88b8640 1 -- 172.21.15.33:0/2693772992 wait complete. 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.464+0000 7ff3d88b8640 1 Processor -- start 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.464+0000 7ff3d88b8640 1 -- start start 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.464+0000 7ff3d88b8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3d41002e0 0x7ff3d4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.464+0000 7ff3d88b8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3d4106ef0 0x7ff3d41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.464+0000 7ff3d88b8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d419cd20 0x7ff3d419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.464+0000 7ff3d88b8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d4111f20 con 0x7ff3d419cd20 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.464+0000 7ff3d88b8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d4111da0 con 0x7ff3d4106ef0 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.464+0000 7ff3d88b8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41120a0 con 0x7ff3d41002e0 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.464+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d419cd20 0x7ff3d419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.464+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d419cd20 0x7ff3d419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60202/0 (socket says 172.21.15.33:60202) 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.464+0000 7ff3d2d9d640 1 -- 172.21.15.33:0/4082829244 learned_addr learned my addr 172.21.15.33:0/4082829244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.464+0000 7ff3d259c640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3d41002e0 0x7ff3d4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.464+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3d4106ef0 0x7ff3d41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.465+0000 7ff3d2d9d640 1 -- 172.21.15.33:0/4082829244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3d41002e0 msgr2=0x7ff3d4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.465+0000 7ff3d2d9d640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3d41002e0 0x7ff3d4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.465+0000 7ff3d2d9d640 1 -- 172.21.15.33:0/4082829244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3d4106ef0 msgr2=0x7ff3d41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.465+0000 7ff3d2d9d640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3d4106ef0 0x7ff3d41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.465+0000 7ff3d2d9d640 1 -- 172.21.15.33:0/4082829244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d419d810 con 0x7ff3d419cd20 2024-04-02T13:45:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.465+0000 7ff3d259c640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3d41002e0 0x7ff3d4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.465+0000 7ff3d2d9d640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d419cd20 0x7ff3d419d1d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00eaa0 tx=0x7ff3cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.465+0000 7ff3cb7fe640 1 -- 172.21.15.33:0/4082829244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc00cdb0 con 0x7ff3d419cd20 2024-04-02T13:45:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.465+0000 7ff3d88b8640 1 -- 172.21.15.33:0/4082829244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d419db00 con 0x7ff3d419cd20 2024-04-02T13:45:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.465+0000 7ff3cb7fe640 1 -- 172.21.15.33:0/4082829244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff3cc005020 con 0x7ff3d419cd20 2024-04-02T13:45:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.465+0000 7ff3cb7fe640 1 -- 172.21.15.33:0/4082829244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc0116b0 con 0x7ff3d419cd20 2024-04-02T13:45:54.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.465+0000 7ff3d88b8640 1 -- 172.21.15.33:0/4082829244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d41a5270 con 0x7ff3d419cd20 2024-04-02T13:45:54.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.467+0000 7ff3cb7fe640 1 -- 172.21.15.33:0/4082829244 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff3cc0051d0 con 0x7ff3d419cd20 2024-04-02T13:45:54.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.467+0000 7ff3db578740 1 Processor -- start 2024-04-02T13:45:54.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3db578740 1 -- start start 2024-04-02T13:45:54.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3db578740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567337a0 0x55d756733b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3db578740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567340c0 0x55d75673c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3db578740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d75673cca0 0x55d75673f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3db578740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d7566977b0 con 0x55d7567337a0 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3db578740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d756636570 con 0x55d7567340c0 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3db578740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d756693760 con 0x55d75673cca0 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3cb7fe640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff3c00a3d20 0x7ff3c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567337a0 0x55d756733b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3d259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567340c0 0x55d75673c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567337a0 0x55d756733b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60208/0 (socket says 172.21.15.33:60208) 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3cbfff640 1 -- 172.21.15.33:0/3951939129 learned_addr learned my addr 172.21.15.33:0/3951939129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3d2d9d640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff3c00a3d20 0x7ff3c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3cb7fe640 1 -- 172.21.15.33:0/4082829244 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7ff3cc0d0ee0 con 0x7ff3d419cd20 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/3951939129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d75673cca0 0x55d75673f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3cbfff640 1 -- 172.21.15.33:0/3951939129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d75673cca0 msgr2=0x55d75673f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/3951939129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d75673cca0 0x55d75673f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3cbfff640 1 -- 172.21.15.33:0/3951939129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567340c0 msgr2=0x55d75673c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/3951939129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567340c0 0x55d75673c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3cbfff640 1 -- 172.21.15.33:0/3951939129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7563ab3f0 con 0x55d7567337a0 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3d259c640 1 --2- 172.21.15.33:0/3951939129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567340c0 0x55d75673c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3d2d9d640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff3c00a3d20 0x7ff3c00a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc098eb0 tx=0x7ff3cc085f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.468+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/3951939129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567337a0 0x55d756733b80 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc005940 tx=0x7ff3bc0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a083246b83eec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.469+0000 7ff3c8ff9640 1 -- 172.21.15.33:0/3951939129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc004470 con 0x55d7567337a0 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.469+0000 7ff3c8ff9640 1 -- 172.21.15.33:0/3951939129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff3bc005510 con 0x55d7567337a0 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.469+0000 7ff3c8ff9640 1 -- 172.21.15.33:0/3951939129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc004920 con 0x55d7567337a0 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.469+0000 7ff3db578740 1 -- 172.21.15.33:0/3951939129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567337a0 msgr2=0x55d756733b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.469+0000 7ff3db578740 1 --2- 172.21.15.33:0/3951939129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567337a0 0x55d756733b80 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc005940 tx=0x7ff3bc0058b0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.469+0000 7ff3db578740 1 -- 172.21.15.33:0/3951939129 shutdown_connections 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.469+0000 7ff3db578740 1 --2- 172.21.15.33:0/3951939129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d75673cca0 0x55d75673f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.469+0000 7ff3db578740 1 --2- 172.21.15.33:0/3951939129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567340c0 0x55d75673c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.469+0000 7ff3db578740 1 --2- 172.21.15.33:0/3951939129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567337a0 0x55d756733b80 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.469+0000 7ff3db578740 1 -- 172.21.15.33:0/3951939129 >> 172.21.15.33:0/3951939129 conn(0x55d75672ded0 msgr2=0x55d75672e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.469+0000 7ff3db578740 1 -- 172.21.15.33:0/3951939129 shutdown_connections 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.469+0000 7ff3db578740 1 -- 172.21.15.33:0/3951939129 wait complete. 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3db578740 1 -- start start 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3db578740 1 Processor -- start 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3db578740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567337a0 0x55d756739b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3db578740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567340c0 0x55d75673a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3db578740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 0x55d75673a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3db578740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d75673fa30 con 0x55d75673cca0 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3db578740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d756652e80 con 0x55d7567340c0 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3db578740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d75673fbb0 con 0x55d7567337a0 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3db578740 10 client.0 _ll_register_callbacks cb 0x55d75668eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 0x55d75673a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 0x55d75673a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60224/0 (socket says 172.21.15.33:60224) 2024-04-02T13:45:54.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3cbfff640 1 -- 172.21.15.33:0/3953050738 learned_addr learned my addr 172.21.15.33:0/3953050738 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3d259c640 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567340c0 0x55d75673a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.470+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567337a0 0x55d756739b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3cbfff640 1 -- 172.21.15.33:0/3953050738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567337a0 msgr2=0x55d756739b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567337a0 0x55d756739b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3cbfff640 1 -- 172.21.15.33:0/3953050738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567340c0 msgr2=0x55d75673a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567340c0 0x55d75673a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3cbfff640 1 -- 172.21.15.33:0/3953050738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7564757d0 con 0x55d75673cca0 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3d259c640 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567340c0 0x55d75673a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 0x55d75673a5c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc005c40 tx=0x7ff3bc02cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862902af443ca8be server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3b9ffb640 1 -- 172.21.15.33:0/3953050738 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc057060 con 0x55d75673cca0 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3b9ffb640 1 -- 172.21.15.33:0/3953050738 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7ff3bc02c7e0 con 0x55d75673cca0 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3db578740 1 -- 172.21.15.33:0/3953050738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3bc005510 con 0x55d75673cca0 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3b9ffb640 1 -- 172.21.15.33:0/3953050738 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff3bc05e050 con 0x55d75673cca0 2024-04-02T13:45:54.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3db578740 10 client.17133 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3db578740 1 -- 172.21.15.33:0/3953050738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d75674bb40 con 0x55d75673cca0 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3b9ffb640 1 -- 172.21.15.33:0/3953050738 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff3bc005510 con 0x55d75673cca0 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3b9ffb640 1 -- 172.21.15.33:0/3953050738 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7ff3bc04e070 con 0x55d75673cca0 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3db578740 10 client.17133 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3db578740 10 client.17133 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3db578740 20 client.17133 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3db578740 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d75674ce90 0x55d75674f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3db578740 4 client.17133 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.471+0000 7ff3db578740 1 -- 172.21.15.33:0/3953050738 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7566a4cf0 con 0x55d75674ce90 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.472+0000 7ff3d259c640 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d75674ce90 0x55d75674f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.472+0000 7ff3b9ffb640 10 client.17133 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.472+0000 7ff3d259c640 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d75674ce90 0x55d75674f280 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2cc5aff09a2d9e3e server_cookie=c5416580fbdbb51d in_seq=0 out_seq=0 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3b9ffb640 1 -- 172.21.15.33:0/3953050738 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7566a4cf0 con 0x55d75674ce90 2024-04-02T13:45:54.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3b9ffb640 10 client.17133 handle_command_reply: tid=0 2024-04-02T13:45:54.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3db578740 1 client.17133 shutdown 2024-04-02T13:45:54.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3db578740 1 -- 172.21.15.33:0/3953050738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 msgr2=0x55d75673a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3db578740 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 0x55d75673a5c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc005c40 tx=0x7ff3bc02cbb0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3db578740 1 -- 172.21.15.33:0/3953050738 shutdown_connections 2024-04-02T13:45:54.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3db578740 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d75674ce90 0x55d75674f280 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3db578740 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 0x55d75673a5c0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3db578740 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567340c0 0x55d75673a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3db578740 1 --2- 172.21.15.33:0/3953050738 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567337a0 0x55d756739b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3b9ffb640 0 client.17133 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:45:54.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3db578740 1 -- 172.21.15.33:0/3953050738 >> 172.21.15.33:0/3953050738 conn(0x55d75672ded0 msgr2=0x55d75672fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3db578740 1 -- 172.21.15.33:0/3953050738 shutdown_connections 2024-04-02T13:45:54.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3db578740 1 -- 172.21.15.33:0/3953050738 wait complete. 2024-04-02T13:45:54.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.473+0000 7ff3db578740 20 client.17133 trim_cache size 0 max 16384 2024-04-02T13:45:54.485 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:54 smithi145 ceph-mon[35646]: pgmap v218: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:45:54.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:54 smithi050 ceph-mon[35524]: pgmap v218: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:45:54.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.486+0000 7ff3db578740 1 Processor -- start 2024-04-02T13:45:54.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3db578740 1 -- start start 2024-04-02T13:45:54.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3db578740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567337a0 0x55d75673c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3db578740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567340c0 0x55d756748510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3db578740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 0x55d756736dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3db578740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41b17d0 con 0x55d75673cca0 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3db578740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41af0a0 con 0x55d7567337a0 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3db578740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41acf10 con 0x55d7567340c0 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 0x55d756736dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 0x55d756736dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60232/0 (socket says 172.21.15.33:60232) 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3cbfff640 1 -- 172.21.15.33:0/373570112 learned_addr learned my addr 172.21.15.33:0/373570112 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3d259c640 1 --2- 172.21.15.33:0/373570112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567340c0 0x55d756748510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/373570112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567337a0 0x55d75673c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3cbfff640 1 -- 172.21.15.33:0/373570112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567340c0 msgr2=0x55d756748510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/373570112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567340c0 0x55d756748510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3cbfff640 1 -- 172.21.15.33:0/373570112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567337a0 msgr2=0x55d75673c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/373570112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567337a0 0x55d75673c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3cbfff640 1 -- 172.21.15.33:0/373570112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d756733ef0 con 0x55d75673cca0 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3d259c640 1 --2- 172.21.15.33:0/373570112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567340c0 0x55d756748510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/373570112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 0x55d756736dc0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc003d00 tx=0x7ff3bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe5443faa360110 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3bbfff640 1 -- 172.21.15.33:0/373570112 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc067070 con 0x55d75673cca0 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3bbfff640 1 -- 172.21.15.33:0/373570112 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff3bc0035f0 con 0x55d75673cca0 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.487+0000 7ff3bbfff640 1 -- 172.21.15.33:0/373570112 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc057040 con 0x55d75673cca0 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.488+0000 7ff3db578740 1 -- 172.21.15.33:0/373570112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 msgr2=0x55d756736dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.488+0000 7ff3db578740 1 --2- 172.21.15.33:0/373570112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 0x55d756736dc0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc003d00 tx=0x7ff3bc000bc0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.488+0000 7ff3db578740 1 -- 172.21.15.33:0/373570112 shutdown_connections 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.488+0000 7ff3db578740 1 --2- 172.21.15.33:0/373570112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d75673cca0 0x55d756736dc0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.488+0000 7ff3db578740 1 --2- 172.21.15.33:0/373570112 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567340c0 0x55d756748510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.488+0000 7ff3db578740 1 --2- 172.21.15.33:0/373570112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d7567337a0 0x55d75673c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.488+0000 7ff3db578740 1 -- 172.21.15.33:0/373570112 >> 172.21.15.33:0/373570112 conn(0x55d75672ded0 msgr2=0x55d756747e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.488+0000 7ff3db578740 1 -- 172.21.15.33:0/373570112 shutdown_connections 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.488+0000 7ff3db578740 1 -- 172.21.15.33:0/373570112 wait complete. 2024-04-02T13:45:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.488+0000 7ff3db578740 1 -- start start 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.488+0000 7ff3db578740 1 Processor -- start 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3db578740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567337a0 0x55d75674f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3db578740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567340c0 0x55d75674f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3db578740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d75673cca0 0x55d7567b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3db578740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41ad560 con 0x55d7567340c0 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3db578740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41ae9d0 con 0x55d75673cca0 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3db578740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41adc30 con 0x55d7567337a0 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3db578740 10 client.0 _ll_register_callbacks cb 0x7ff3c400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3d259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567340c0 0x55d75674f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3d259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567340c0 0x55d75674f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60240/0 (socket says 172.21.15.33:60240) 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3d259c640 1 -- 172.21.15.33:0/1183172007 learned_addr learned my addr 172.21.15.33:0/1183172007 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567337a0 0x55d75674f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d75673cca0 0x55d7567b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3d259c640 1 -- 172.21.15.33:0/1183172007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567337a0 msgr2=0x55d75674f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3d259c640 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567337a0 0x55d75674f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3d259c640 1 -- 172.21.15.33:0/1183172007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d75673cca0 msgr2=0x55d7567b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3d259c640 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d75673cca0 0x55d7567b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3d259c640 1 -- 172.21.15.33:0/1183172007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d756750090 con 0x55d7567340c0 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567337a0 0x55d75674f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3d259c640 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567340c0 0x55d75674f7c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4005d00 tx=0x7ff3c400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c07e7e825f8f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3b9ffb640 1 -- 172.21.15.33:0/1183172007 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c401d470 con 0x55d7567340c0 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3b9ffb640 1 -- 172.21.15.33:0/1183172007 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7ff3c401d610 con 0x55d7567340c0 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3db578740 1 -- 172.21.15.33:0/1183172007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3bc0035f0 con 0x55d7567340c0 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3b9ffb640 1 -- 172.21.15.33:0/1183172007 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff3c402c050 con 0x55d7567340c0 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3db578740 10 client.17139 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.489+0000 7ff3db578740 1 -- 172.21.15.33:0/1183172007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d7567b26a0 con 0x55d7567340c0 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.490+0000 7ff3b9ffb640 1 -- 172.21.15.33:0/1183172007 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff3bc0035f0 con 0x55d7567340c0 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.490+0000 7ff3b9ffb640 1 -- 172.21.15.33:0/1183172007 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7ff3c401c3d0 con 0x55d7567340c0 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.490+0000 7ff3db578740 10 client.17139 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.490+0000 7ff3db578740 10 client.17139 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.490+0000 7ff3db578740 20 client.17139 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.490+0000 7ff3db578740 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d7567b2e20 0x55d7567b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.490+0000 7ff3db578740 4 client.17139 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.490+0000 7ff3db578740 1 -- 172.21.15.33:0/1183172007 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d75669f040 con 0x55d7567b2e20 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.490+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d7567b2e20 0x55d7567b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3b9ffb640 10 client.17139 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3cbfff640 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d7567b2e20 0x55d7567b5240 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cc6b6c480f98d1 server_cookie=8c7b3094a57a6411 in_seq=0 out_seq=0 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3b9ffb640 1 -- 172.21.15.33:0/1183172007 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d75669f040 con 0x55d7567b2e20 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3b9ffb640 10 client.17139 handle_command_reply: tid=0 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3db578740 1 client.17139 shutdown 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3db578740 1 -- 172.21.15.33:0/1183172007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567340c0 msgr2=0x55d75674f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3db578740 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567340c0 0x55d75674f7c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4005d00 tx=0x7ff3c400be90 comp rx=0 tx=0).stop 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3db578740 1 -- 172.21.15.33:0/1183172007 shutdown_connections 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3db578740 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55d7567b2e20 0x55d7567b5240 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3db578740 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d75673cca0 0x55d7567b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3db578740 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7567340c0 0x55d75674f7c0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3db578740 1 --2- 172.21.15.33:0/1183172007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d7567337a0 0x55d75674f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3b9ffb640 0 client.17139 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:45:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3db578740 1 -- 172.21.15.33:0/1183172007 >> 172.21.15.33:0/1183172007 conn(0x55d75672ded0 msgr2=0x55d7567382e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3db578740 1 -- 172.21.15.33:0/1183172007 shutdown_connections 2024-04-02T13:45:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3db578740 1 -- 172.21.15.33:0/1183172007 wait complete. 2024-04-02T13:45:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.491+0000 7ff3db578740 20 client.17139 trim_cache size 0 max 16384 2024-04-02T13:45:54.502 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:45:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.492+0000 7ff3d88b8640 1 -- 172.21.15.33:0/4082829244 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff3c00a3d20 msgr2=0x7ff3c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.492+0000 7ff3d88b8640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff3c00a3d20 0x7ff3c00a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc098eb0 tx=0x7ff3cc085f90 comp rx=0 tx=0).stop 2024-04-02T13:45:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.492+0000 7ff3d88b8640 1 -- 172.21.15.33:0/4082829244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d419cd20 msgr2=0x7ff3d419d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.492+0000 7ff3d88b8640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d419cd20 0x7ff3d419d1d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00eaa0 tx=0x7ff3cc00ef60 comp rx=0 tx=0).stop 2024-04-02T13:45:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.492+0000 7ff3d88b8640 1 -- 172.21.15.33:0/4082829244 shutdown_connections 2024-04-02T13:45:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.492+0000 7ff3d88b8640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff3c00a3d20 0x7ff3c00a61e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.492+0000 7ff3d88b8640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3d419cd20 0x7ff3d419d1d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.492+0000 7ff3d88b8640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3d4106ef0 0x7ff3d41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.492+0000 7ff3d88b8640 1 --2- 172.21.15.33:0/4082829244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3d41002e0 0x7ff3d4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.492+0000 7ff3d88b8640 1 -- 172.21.15.33:0/4082829244 >> 172.21.15.33:0/4082829244 conn(0x7ff3d4077030 msgr2=0x7ff3d40fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.492+0000 7ff3d88b8640 1 -- 172.21.15.33:0/4082829244 shutdown_connections 2024-04-02T13:45:54.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.492+0000 7ff3d88b8640 1 -- 172.21.15.33:0/4082829244 wait complete. 2024-04-02T13:45:54.506 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:45:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.608+0000 7f3eb94e3640 1 -- 172.21.15.33:0/2280417799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106a30 msgr2=0x7f3eb4106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.608+0000 7f3eb94e3640 1 --2- 172.21.15.33:0/2280417799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106a30 0x7f3eb4106e10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009930 tx=0x7f3ea402bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.608+0000 7f3eb94e3640 1 -- 172.21.15.33:0/2280417799 shutdown_connections 2024-04-02T13:45:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.608+0000 7f3eb94e3640 1 --2- 172.21.15.33:0/2280417799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eb41013e0 0x7f3eb410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.608+0000 7f3eb94e3640 1 --2- 172.21.15.33:0/2280417799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb4100a20 0x7f3eb4100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.608+0000 7f3eb94e3640 1 --2- 172.21.15.33:0/2280417799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106a30 0x7f3eb4106e10 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.608+0000 7f3eb94e3640 1 -- 172.21.15.33:0/2280417799 >> 172.21.15.33:0/2280417799 conn(0x7f3eb4077b60 msgr2=0x7f3eb4077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.608+0000 7f3eb94e3640 1 -- 172.21.15.33:0/2280417799 shutdown_connections 2024-04-02T13:45:54.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.608+0000 7f3eb94e3640 1 -- 172.21.15.33:0/2280417799 wait complete. 2024-04-02T13:45:54.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb94e3640 1 Processor -- start 2024-04-02T13:45:54.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb94e3640 1 -- start start 2024-04-02T13:45:54.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb94e3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb4100a20 0x7f3eb4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb94e3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eb41013e0 0x7f3eb41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb94e3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106a30 0x7f3eb419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb94e3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4110de0 con 0x7f3eb4106a30 2024-04-02T13:45:54.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb94e3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4110c60 con 0x7f3eb4100a20 2024-04-02T13:45:54.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb94e3640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4110f60 con 0x7f3eb41013e0 2024-04-02T13:45:54.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106a30 0x7f3eb419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106a30 0x7f3eb419cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60258/0 (socket says 172.21.15.33:60258) 2024-04-02T13:45:54.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb37fe640 1 -- 172.21.15.33:0/1832308034 learned_addr learned my addr 172.21.15.33:0/1832308034 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb2ffd640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb4100a20 0x7f3eb4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eb41013e0 0x7f3eb41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb37fe640 1 -- 172.21.15.33:0/1832308034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eb41013e0 msgr2=0x7f3eb41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eb41013e0 0x7f3eb41988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb37fe640 1 -- 172.21.15.33:0/1832308034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb4100a20 msgr2=0x7f3eb4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb4100a20 0x7f3eb4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.609+0000 7f3eb37fe640 1 -- 172.21.15.33:0/1832308034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb419d310 con 0x7f3eb4106a30 2024-04-02T13:45:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.610+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106a30 0x7f3eb419cbe0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00cce0 tx=0x7f3eac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.610+0000 7f3eb0ff9640 1 -- 172.21.15.33:0/1832308034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac00f450 con 0x7f3eb4106a30 2024-04-02T13:45:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.610+0000 7f3eb94e3640 1 -- 172.21.15.33:0/1832308034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb419d600 con 0x7f3eb4106a30 2024-04-02T13:45:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.610+0000 7f3eb0ff9640 1 -- 172.21.15.33:0/1832308034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3eac00fac0 con 0x7f3eb4106a30 2024-04-02T13:45:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.610+0000 7f3eb0ff9640 1 -- 172.21.15.33:0/1832308034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac018be0 con 0x7f3eb4106a30 2024-04-02T13:45:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.610+0000 7f3eb94e3640 1 -- 172.21.15.33:0/1832308034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb41a4e90 con 0x7f3eb4106a30 2024-04-02T13:45:54.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3ebc19b740 1 Processor -- start 2024-04-02T13:45:54.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3eb0ff9640 1 -- 172.21.15.33:0/1832308034 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3eac00fc70 con 0x7f3eb4106a30 2024-04-02T13:45:54.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3ebc19b740 1 -- start start 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3ebc19b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 0x56202dad7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3ebc19b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 0x56202dae0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3ebc19b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 0x56202dae3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3ebc19b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56202da3b7b0 con 0x56202dae0ca0 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3ebc19b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56202d9da570 con 0x56202dad80c0 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3ebc19b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56202da37760 con 0x56202dad77a0 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3eaa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 0x56202dae3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3eaa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 0x56202dae3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60266/0 (socket says 172.21.15.33:60266) 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3eb0ff9640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3ea00a3d20 0x7f3ea00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3eaa7fc640 1 -- 172.21.15.33:0/2114266702 learned_addr learned my addr 172.21.15.33:0/2114266702 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.613+0000 7f3eb2ffd640 1 --2- 172.21.15.33:0/2114266702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 0x56202dae0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.614+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/2114266702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 0x56202dad7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.614+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3ea00a3d20 0x7f3ea00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.614+0000 7f3eb0ff9640 1 -- 172.21.15.33:0/1832308034 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f3eac0d12e0 con 0x7f3eb4106a30 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.614+0000 7f3eaa7fc640 1 -- 172.21.15.33:0/2114266702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 msgr2=0x56202dad7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.614+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/2114266702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 0x56202dad7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.614+0000 7f3eaa7fc640 1 -- 172.21.15.33:0/2114266702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 msgr2=0x56202dae0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.614+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/2114266702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 0x56202dae0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.614+0000 7f3eaa7fc640 1 -- 172.21.15.33:0/2114266702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56202d74f3f0 con 0x56202dae0ca0 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.614+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/2114266702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 0x56202dae3090 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c013500 tx=0x7f3e9c00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ddff2b63c1fa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.614+0000 7f3ea97fa640 1 -- 172.21.15.33:0/2114266702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c019030 con 0x56202dae0ca0 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.614+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3ea00a3d20 0x7f3ea00a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3eac09cbb0 tx=0x7f3eac092340 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.614+0000 7f3ea97fa640 1 -- 172.21.15.33:0/2114266702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3e9c011910 con 0x56202dae0ca0 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.614+0000 7f3ea97fa640 1 -- 172.21.15.33:0/2114266702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c01d820 con 0x56202dae0ca0 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.615+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2114266702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 msgr2=0x56202dae3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.615+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2114266702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 0x56202dae3090 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c013500 tx=0x7f3e9c00b690 comp rx=0 tx=0).stop 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.615+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2114266702 shutdown_connections 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.615+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2114266702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 0x56202dae3090 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.615+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2114266702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 0x56202dae0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.615+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2114266702 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 0x56202dad7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.615+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2114266702 >> 172.21.15.33:0/2114266702 conn(0x56202dad1ed0 msgr2=0x56202dad22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.615+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2114266702 shutdown_connections 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.615+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2114266702 wait complete. 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.615+0000 7f3ebc19b740 1 -- start start 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.615+0000 7f3ebc19b740 1 Processor -- start 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.615+0000 7f3ebc19b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 0x56202dae0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.615+0000 7f3ebc19b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 0x56202daddad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3ebc19b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 0x56202dade1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3ebc19b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56202dae3a90 con 0x56202dae0ca0 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3ebc19b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56202d9f6e80 con 0x56202dad80c0 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3ebc19b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56202dae3c10 con 0x56202dad77a0 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3ebc19b740 10 client.0 _ll_register_callbacks cb 0x56202da32aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3eaa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 0x56202dade1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3eaa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 0x56202dade1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60276/0 (socket says 172.21.15.33:60276) 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3eb2ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 0x56202daddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3eaa7fc640 1 -- 172.21.15.33:0/556803336 learned_addr learned my addr 172.21.15.33:0/556803336 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/556803336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 0x56202dae0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3eaa7fc640 1 -- 172.21.15.33:0/556803336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 msgr2=0x56202dae0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/556803336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 0x56202dae0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3eaa7fc640 1 -- 172.21.15.33:0/556803336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 msgr2=0x56202daddad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/556803336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 0x56202daddad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3eaa7fc640 1 -- 172.21.15.33:0/556803336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56202d8197d0 con 0x56202dae0ca0 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/556803336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 0x56202dade1c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c005fd0 tx=0x7f3e9c01d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf4f77f23794dee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3e9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/556803336 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c019040 con 0x56202dae0ca0 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/556803336 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f3e9c011d90 con 0x56202dae0ca0 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3ebc19b740 1 -- 172.21.15.33:0/556803336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e9c011910 con 0x56202dae0ca0 2024-04-02T13:45:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/556803336 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3e9c004030 con 0x56202dae0ca0 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3ebc19b740 10 client.17151 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.616+0000 7f3ebc19b740 1 -- 172.21.15.33:0/556803336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56202daefb50 con 0x56202dae0ca0 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.617+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/556803336 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3e9c011910 con 0x56202dae0ca0 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.617+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/556803336 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f3e9c0046c0 con 0x56202dae0ca0 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.617+0000 7f3ebc19b740 10 client.17151 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.617+0000 7f3ebc19b740 10 client.17151 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.617+0000 7f3ebc19b740 20 client.17151 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.617+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/556803336 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56202daf0fb0 0x56202daf33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.617+0000 7f3ebc19b740 4 client.17151 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.617+0000 7f3ebc19b740 1 -- 172.21.15.33:0/556803336 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56202da48cf0 con 0x56202daf0fb0 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.617+0000 7f3eb2ffd640 1 --2- 172.21.15.33:0/556803336 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56202daf0fb0 0x56202daf33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.617+0000 7f3e9a7fc640 10 client.17151 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.617+0000 7f3eb2ffd640 1 --2- 172.21.15.33:0/556803336 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56202daf0fb0 0x56202daf33a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fab8833ce484cc2 server_cookie=b73eb7130fd26da5 in_seq=0 out_seq=0 2024-04-02T13:45:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/556803336 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56202da48cf0 con 0x56202daf0fb0 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3e9a7fc640 10 client.17151 handle_command_reply: tid=0 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3ebc19b740 1 client.17151 shutdown 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3ebc19b740 1 -- 172.21.15.33:0/556803336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 msgr2=0x56202dade1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/556803336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 0x56202dade1c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c005fd0 tx=0x7f3e9c01d460 comp rx=0 tx=0).stop 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3ebc19b740 1 -- 172.21.15.33:0/556803336 shutdown_connections 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/556803336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dae0ca0 0x56202dade1c0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/556803336 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 0x56202daddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/556803336 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56202daf0fb0 0x56202daf33a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/556803336 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 0x56202dae0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3e9a7fc640 0 client.17151 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3ebc19b740 1 -- 172.21.15.33:0/556803336 >> 172.21.15.33:0/556803336 conn(0x56202dad1ed0 msgr2=0x56202dad3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3ebc19b740 1 -- 172.21.15.33:0/556803336 shutdown_connections 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.618+0000 7f3ebc19b740 1 -- 172.21.15.33:0/556803336 wait complete. 2024-04-02T13:45:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.619+0000 7f3ebc19b740 20 client.17151 trim_cache size 0 max 16384 2024-04-02T13:45:54.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.632+0000 7f3ebc19b740 1 Processor -- start 2024-04-02T13:45:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.632+0000 7f3ebc19b740 1 -- start start 2024-04-02T13:45:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.632+0000 7f3ebc19b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad77a0 0x56202daec980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.632+0000 7f3ebc19b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 0x56202daed2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.632+0000 7f3ebc19b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dae0ca0 0x56202dae0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.632+0000 7f3ebc19b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41b1840 con 0x56202dad77a0 2024-04-02T13:45:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.632+0000 7f3ebc19b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41af110 con 0x56202dad80c0 2024-04-02T13:45:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.632+0000 7f3ebc19b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41acf80 con 0x56202dae0ca0 2024-04-02T13:45:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.632+0000 7f3eaa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad77a0 0x56202daec980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.632+0000 7f3eaa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad77a0 0x56202daec980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60286/0 (socket says 172.21.15.33:60286) 2024-04-02T13:45:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.632+0000 7f3eaa7fc640 1 -- 172.21.15.33:0/2881802856 learned_addr learned my addr 172.21.15.33:0/2881802856 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.632+0000 7f3eb2ffd640 1 --2- 172.21.15.33:0/2881802856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 0x56202daed2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.633+0000 7f3eaa7fc640 1 -- 172.21.15.33:0/2881802856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dae0ca0 msgr2=0x56202dae0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.633+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/2881802856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dae0ca0 0x56202dae0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.633+0000 7f3eaa7fc640 1 -- 172.21.15.33:0/2881802856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 msgr2=0x56202daed2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.633+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/2881802856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 0x56202daed2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.633+0000 7f3eaa7fc640 1 -- 172.21.15.33:0/2881802856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56202dad7ef0 con 0x56202dad77a0 2024-04-02T13:45:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.633+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/2881802856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad77a0 0x56202daec980 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0059f0 tx=0x7f3e9c0134b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d88ddc2132b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.633+0000 7f3ea8ff9640 1 -- 172.21.15.33:0/2881802856 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c019070 con 0x56202dad77a0 2024-04-02T13:45:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.633+0000 7f3ea8ff9640 1 -- 172.21.15.33:0/2881802856 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3e9c00be60 con 0x56202dad77a0 2024-04-02T13:45:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.633+0000 7f3ea8ff9640 1 -- 172.21.15.33:0/2881802856 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c014e60 con 0x56202dad77a0 2024-04-02T13:45:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2881802856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad77a0 msgr2=0x56202daec980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2881802856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad77a0 0x56202daec980 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0059f0 tx=0x7f3e9c0134b0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2881802856 shutdown_connections 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2881802856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dae0ca0 0x56202dae0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2881802856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dad80c0 0x56202daed2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2881802856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad77a0 0x56202daec980 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2881802856 >> 172.21.15.33:0/2881802856 conn(0x56202dad1ed0 msgr2=0x56202dadc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2881802856 shutdown_connections 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2881802856 wait complete. 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 -- start start 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 Processor -- start 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 0x56202dae8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad80c0 0x56202dae93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dae0ca0 0x56202dada1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41ad5d0 con 0x56202dad80c0 2024-04-02T13:45:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41aea40 con 0x56202dae0ca0 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41adca0 con 0x56202dad77a0 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3ebc19b740 10 client.0 _ll_register_callbacks cb 0x7f3ea4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3eb2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad80c0 0x56202dae93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3eb2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad80c0 0x56202dae93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60302/0 (socket says 172.21.15.33:60302) 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3eb2ffd640 1 -- 172.21.15.33:0/2526229147 learned_addr learned my addr 172.21.15.33:0/2526229147 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.634+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dae0ca0 0x56202dada1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 0x56202dae8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3eb2ffd640 1 -- 172.21.15.33:0/2526229147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 msgr2=0x56202dae8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3eb2ffd640 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 0x56202dae8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3eb2ffd640 1 -- 172.21.15.33:0/2526229147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dae0ca0 msgr2=0x56202dada1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3eb2ffd640 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dae0ca0 0x56202dada1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3eb2ffd640 1 -- 172.21.15.33:0/2526229147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56202dada9e0 con 0x56202dad80c0 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dae0ca0 0x56202dada1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3eb2ffd640 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad80c0 0x56202dae93d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4005940 tx=0x7f3ea4005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f60ce2bfb7d74bd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3e9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/2526229147 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea404d070 con 0x56202dad80c0 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/2526229147 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f3ea402c5d0 con 0x56202dad80c0 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2526229147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e9c00be60 con 0x56202dad80c0 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/2526229147 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3ea4058050 con 0x56202dad80c0 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3ebc19b740 10 client.17157 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2526229147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56202db56d90 con 0x56202dad80c0 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.635+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/2526229147 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3e9c00be60 con 0x56202dad80c0 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/2526229147 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f3ea4004cc0 con 0x56202dad80c0 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3ebc19b740 10 client.17157 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3ebc19b740 10 client.17157 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3ebc19b740 20 client.17157 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56202db574f0 0x56202db59910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3ebc19b740 4 client.17157 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2526229147 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56202da43040 con 0x56202db574f0 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56202db574f0 0x56202db59910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3eaa7fc640 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56202db574f0 0x56202db59910 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17f848657ad13814 server_cookie=4313a2ebf00ce5f9 in_seq=0 out_seq=0 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3e9a7fc640 10 client.17157 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/2526229147 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56202da43040 con 0x56202db574f0 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3e9a7fc640 10 client.17157 handle_command_reply: tid=0 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3ebc19b740 1 client.17157 shutdown 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2526229147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad80c0 msgr2=0x56202dae93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad80c0 0x56202dae93d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4005940 tx=0x7f3ea4005130 comp rx=0 tx=0).stop 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2526229147 shutdown_connections 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56202dae0ca0 0x56202dada1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56202dad80c0 0x56202dae93d0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.636+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x56202db574f0 0x56202db59910 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3ebc19b740 1 --2- 172.21.15.33:0/2526229147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56202dad77a0 0x56202dae8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2526229147 >> 172.21.15.33:0/2526229147 conn(0x56202dad1ed0 msgr2=0x56202dad3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3e9a7fc640 0 client.17157 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:45:54.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2526229147 shutdown_connections 2024-04-02T13:45:54.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3ebc19b740 1 -- 172.21.15.33:0/2526229147 wait complete. 2024-04-02T13:45:54.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3ebc19b740 20 client.17157 trim_cache size 0 max 16384 2024-04-02T13:45:54.647 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:45:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3eb94e3640 1 -- 172.21.15.33:0/1832308034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3ea00a3d20 msgr2=0x7f3ea00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3eb94e3640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3ea00a3d20 0x7f3ea00a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3eac09cbb0 tx=0x7f3eac092340 comp rx=0 tx=0).stop 2024-04-02T13:45:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3eb94e3640 1 -- 172.21.15.33:0/1832308034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106a30 msgr2=0x7f3eb419cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3eb94e3640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106a30 0x7f3eb419cbe0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00cce0 tx=0x7f3eac00a620 comp rx=0 tx=0).stop 2024-04-02T13:45:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3eb94e3640 1 -- 172.21.15.33:0/1832308034 shutdown_connections 2024-04-02T13:45:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3eb94e3640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3ea00a3d20 0x7f3ea00a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3eb94e3640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106a30 0x7f3eb419cbe0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3eb94e3640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3eb41013e0 0x7f3eb41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3eb94e3640 1 --2- 172.21.15.33:0/1832308034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb4100a20 0x7f3eb4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3eb94e3640 1 -- 172.21.15.33:0/1832308034 >> 172.21.15.33:0/1832308034 conn(0x7f3eb4077b60 msgr2=0x7f3eb40ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3eb94e3640 1 -- 172.21.15.33:0/1832308034 shutdown_connections 2024-04-02T13:45:54.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.637+0000 7f3eb94e3640 1 -- 172.21.15.33:0/1832308034 wait complete. 2024-04-02T13:45:54.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:45:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.755+0000 7f3a7a47b640 1 -- 172.21.15.33:0/1526385649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a74102450 msgr2=0x7f3a7410e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.755+0000 7f3a7a47b640 1 --2- 172.21.15.33:0/1526385649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a74102450 0x7f3a7410e960 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009cd0 tx=0x7f3a6c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.755+0000 7f3a7a47b640 1 -- 172.21.15.33:0/1526385649 shutdown_connections 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.755+0000 7f3a7a47b640 1 --2- 172.21.15.33:0/1526385649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a74102450 0x7f3a7410e960 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.755+0000 7f3a7a47b640 1 --2- 172.21.15.33:0/1526385649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a74101a90 0x7f3a74101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.755+0000 7f3a7a47b640 1 --2- 172.21.15.33:0/1526385649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a74107aa0 0x7f3a74107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.755+0000 7f3a7a47b640 1 -- 172.21.15.33:0/1526385649 >> 172.21.15.33:0/1526385649 conn(0x7f3a740fd5a0 msgr2=0x7f3a740ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.755+0000 7f3a7a47b640 1 -- 172.21.15.33:0/1526385649 shutdown_connections 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.755+0000 7f3a7a47b640 1 -- 172.21.15.33:0/1526385649 wait complete. 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.756+0000 7f3a7a47b640 1 Processor -- start 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.756+0000 7f3a7a47b640 1 -- start start 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.756+0000 7f3a7a47b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a74101a90 0x7f3a74073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.756+0000 7f3a7a47b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a74102450 0x7f3a74074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.756+0000 7f3a7a47b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a74107aa0 0x7f3a74074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.756+0000 7f3a7a47b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74111e80 con 0x7f3a74107aa0 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.756+0000 7f3a7a47b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74111d00 con 0x7f3a74101a90 2024-04-02T13:45:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.756+0000 7f3a7a47b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74112000 con 0x7f3a74102450 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.756+0000 7f3a78a18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a74107aa0 0x7f3a74074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.756+0000 7f3a78a18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a74107aa0 0x7f3a74074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60324/0 (socket says 172.21.15.33:60324) 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.756+0000 7f3a78a18640 1 -- 172.21.15.33:0/1803250465 learned_addr learned my addr 172.21.15.33:0/1803250465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.757+0000 7f3a73fff640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a74101a90 0x7f3a74073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.757+0000 7f3a737fe640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a74102450 0x7f3a74074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.757+0000 7f3a78a18640 1 -- 172.21.15.33:0/1803250465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a74102450 msgr2=0x7f3a74074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.757+0000 7f3a78a18640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a74102450 0x7f3a74074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.757+0000 7f3a78a18640 1 -- 172.21.15.33:0/1803250465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a74101a90 msgr2=0x7f3a74073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.757+0000 7f3a78a18640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a74101a90 0x7f3a74073e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.757+0000 7f3a78a18640 1 -- 172.21.15.33:0/1803250465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a741a6860 con 0x7f3a74107aa0 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.757+0000 7f3a78a18640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a74107aa0 0x7f3a74074880 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c004af0 tx=0x7f3a6c034d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.757+0000 7f3a717fa640 1 -- 172.21.15.33:0/1803250465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c039050 con 0x7f3a74107aa0 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.757+0000 7f3a7a47b640 1 -- 172.21.15.33:0/1803250465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a741a6af0 con 0x7f3a74107aa0 2024-04-02T13:45:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.757+0000 7f3a717fa640 1 -- 172.21.15.33:0/1803250465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3a6c045070 con 0x7f3a74107aa0 2024-04-02T13:45:54.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.757+0000 7f3a717fa640 1 -- 172.21.15.33:0/1803250465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0046b0 con 0x7f3a74107aa0 2024-04-02T13:45:54.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.757+0000 7f3a7a47b640 1 -- 172.21.15.33:0/1803250465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a741a7030 con 0x7f3a74107aa0 2024-04-02T13:45:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.760+0000 7f3a7d067740 1 Processor -- start 2024-04-02T13:45:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.760+0000 7f3a717fa640 1 -- 172.21.15.33:0/1803250465 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3a6c003970 con 0x7f3a74107aa0 2024-04-02T13:45:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.760+0000 7f3a7d067740 1 -- start start 2024-04-02T13:45:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.760+0000 7f3a7d067740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896a7a0 0x55ad3896ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.760+0000 7f3a7d067740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 0x55ad38973760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.760+0000 7f3a7d067740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad38973ca0 0x55ad38976090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.760+0000 7f3a7d067740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ad388ce7b0 con 0x55ad3896b0c0 2024-04-02T13:45:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.760+0000 7f3a7d067740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ad3886d570 con 0x55ad3896a7a0 2024-04-02T13:45:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.760+0000 7f3a7d067740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ad388ca760 con 0x55ad38973ca0 2024-04-02T13:45:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.760+0000 7f3a73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 0x55ad38973760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 0x55ad38973760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60330/0 (socket says 172.21.15.33:60330) 2024-04-02T13:45:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a73fff640 1 -- 172.21.15.33:0/2371267438 learned_addr learned my addr 172.21.15.33:0/2371267438 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a717fa640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3a5c0a3d20 0x7f3a5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a737fe640 1 --2- 172.21.15.33:0/2371267438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896a7a0 0x55ad3896ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a737fe640 1 --2- 172.21.15.33:0/2371267438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad38973ca0 0x55ad38976090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a73fff640 1 -- 172.21.15.33:0/2371267438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad38973ca0 msgr2=0x55ad38976090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a73fff640 1 --2- 172.21.15.33:0/2371267438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad38973ca0 0x55ad38976090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a78a18640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3a5c0a3d20 0x7f3a5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a73fff640 1 -- 172.21.15.33:0/2371267438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896a7a0 msgr2=0x55ad3896ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a73fff640 1 --2- 172.21.15.33:0/2371267438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896a7a0 0x55ad3896ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a73fff640 1 -- 172.21.15.33:0/2371267438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad385e23f0 con 0x55ad3896b0c0 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a717fa640 1 -- 172.21.15.33:0/1803250465 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f3a6c0f3630 con 0x7f3a74107aa0 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a73fff640 1 --2- 172.21.15.33:0/2371267438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 0x55ad38973760 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400bc60 tx=0x7f3a6400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8360a66e70ea91 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a5a7fc640 1 -- 172.21.15.33:0/2371267438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64015070 con 0x55ad3896b0c0 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a78a18640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3a5c0a3d20 0x7f3a5c0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0f0b70 tx=0x7f3a6c0a8040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a5a7fc640 1 -- 172.21.15.33:0/2371267438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3a64011910 con 0x55ad3896b0c0 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.761+0000 7f3a5a7fc640 1 -- 172.21.15.33:0/2371267438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a640108b0 con 0x55ad3896b0c0 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.762+0000 7f3a7d067740 1 -- 172.21.15.33:0/2371267438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 msgr2=0x55ad38973760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.762+0000 7f3a7d067740 1 --2- 172.21.15.33:0/2371267438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 0x55ad38973760 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400bc60 tx=0x7f3a6400f730 comp rx=0 tx=0).stop 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.762+0000 7f3a7d067740 1 -- 172.21.15.33:0/2371267438 shutdown_connections 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.762+0000 7f3a7d067740 1 --2- 172.21.15.33:0/2371267438 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad38973ca0 0x55ad38976090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.762+0000 7f3a7d067740 1 --2- 172.21.15.33:0/2371267438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 0x55ad38973760 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.762+0000 7f3a7d067740 1 --2- 172.21.15.33:0/2371267438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896a7a0 0x55ad3896ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.762+0000 7f3a7d067740 1 -- 172.21.15.33:0/2371267438 >> 172.21.15.33:0/2371267438 conn(0x55ad38964ed0 msgr2=0x55ad389652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.762+0000 7f3a7d067740 1 -- 172.21.15.33:0/2371267438 shutdown_connections 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.762+0000 7f3a7d067740 1 -- 172.21.15.33:0/2371267438 wait complete. 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.762+0000 7f3a7d067740 1 -- start start 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.762+0000 7f3a7d067740 1 Processor -- start 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a7d067740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896a7a0 0x55ad38973700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a7d067740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad3896b0c0 0x55ad38970a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a7d067740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad38973ca0 0x55ad389711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a7d067740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ad38976a90 con 0x55ad3896a7a0 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a7d067740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ad38889e80 con 0x55ad38973ca0 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a7d067740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ad38976c10 con 0x55ad3896b0c0 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a7d067740 10 client.0 _ll_register_callbacks cb 0x55ad388c5aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896a7a0 0x55ad38973700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896a7a0 0x55ad38973700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60336/0 (socket says 172.21.15.33:60336) 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a737fe640 1 -- 172.21.15.33:0/4127692212 learned_addr learned my addr 172.21.15.33:0/4127692212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a73fff640 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad3896b0c0 0x55ad38970a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a737fe640 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad38973ca0 0x55ad389711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a737fe640 1 -- 172.21.15.33:0/4127692212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad3896b0c0 msgr2=0x55ad38970a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a737fe640 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad3896b0c0 0x55ad38970a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a737fe640 1 -- 172.21.15.33:0/4127692212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad38973ca0 msgr2=0x55ad389711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a737fe640 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad38973ca0 0x55ad389711b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a737fe640 1 -- 172.21.15.33:0/4127692212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad386ac7d0 con 0x55ad3896a7a0 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a73fff640 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad3896b0c0 0x55ad38970a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a737fe640 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896a7a0 0x55ad38973700 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c8c0 tx=0x7f3a68016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e97eb4bd57a296 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a537fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a537fe640 1 -- 172.21.15.33:0/4127692212 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800f440 con 0x55ad3896a7a0 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.763+0000 7f3a537fe640 1 -- 172.21.15.33:0/4127692212 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f3a6800f5e0 con 0x55ad3896a7a0 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.764+0000 7f3a7d067740 1 -- 172.21.15.33:0/4127692212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a64011910 con 0x55ad3896a7a0 2024-04-02T13:45:54.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.764+0000 7f3a537fe640 1 -- 172.21.15.33:0/4127692212 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3a68026050 con 0x55ad3896a7a0 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.764+0000 7f3a7d067740 10 client.17169 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.764+0000 7f3a7d067740 1 -- 172.21.15.33:0/4127692212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ad38982b90 con 0x55ad3896a7a0 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.764+0000 7f3a537fe640 1 -- 172.21.15.33:0/4127692212 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3a64011910 con 0x55ad3896a7a0 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.764+0000 7f3a537fe640 1 -- 172.21.15.33:0/4127692212 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f3a6800e890 con 0x55ad3896a7a0 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.764+0000 7f3a7d067740 10 client.17169 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.764+0000 7f3a7d067740 10 client.17169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.764+0000 7f3a7d067740 20 client.17169 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.764+0000 7f3a7d067740 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad38984050 0x55ad38986440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.764+0000 7f3a7d067740 4 client.17169 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.764+0000 7f3a7d067740 1 -- 172.21.15.33:0/4127692212 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad388dbcf0 con 0x55ad38984050 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.764+0000 7f3a73fff640 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad38984050 0x55ad38986440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.765+0000 7f3a73fff640 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad38984050 0x55ad38986440 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f6a98a500eeabc server_cookie=30cf20119948eeec in_seq=0 out_seq=0 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.765+0000 7f3a537fe640 10 client.17169 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.765+0000 7f3a537fe640 1 -- 172.21.15.33:0/4127692212 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad388dbcf0 con 0x55ad38984050 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a537fe640 10 client.17169 handle_command_reply: tid=0 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a7d067740 1 client.17169 shutdown 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a7d067740 1 -- 172.21.15.33:0/4127692212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896a7a0 msgr2=0x55ad38973700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a7d067740 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896a7a0 0x55ad38973700 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c8c0 tx=0x7f3a68016040 comp rx=0 tx=0).stop 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a7d067740 1 -- 172.21.15.33:0/4127692212 shutdown_connections 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a7d067740 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad38984050 0x55ad38986440 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a7d067740 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad38973ca0 0x55ad389711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a537fe640 0 client.17169 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a7d067740 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad3896b0c0 0x55ad38970a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a7d067740 1 --2- 172.21.15.33:0/4127692212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896a7a0 0x55ad38973700 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a7d067740 1 -- 172.21.15.33:0/4127692212 >> 172.21.15.33:0/4127692212 conn(0x55ad38964ed0 msgr2=0x55ad38966c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a7d067740 1 -- 172.21.15.33:0/4127692212 shutdown_connections 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a7d067740 1 -- 172.21.15.33:0/4127692212 wait complete. 2024-04-02T13:45:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.766+0000 7f3a7d067740 20 client.17169 trim_cache size 0 max 16384 2024-04-02T13:45:54.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.779+0000 7f3a7d067740 1 Processor -- start 2024-04-02T13:45:54.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.779+0000 7f3a7d067740 1 -- start start 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.779+0000 7f3a7d067740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad3896a7a0 0x55ad38970390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.779+0000 7f3a7d067740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896b0c0 0x55ad389652b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.779+0000 7f3a7d067740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad38973ca0 0x55ad3898bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.779+0000 7f3a7d067740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a741b3bc0 con 0x55ad38973ca0 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.779+0000 7f3a7d067740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3a741b1490 con 0x55ad3896b0c0 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.779+0000 7f3a7d067740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3a741af300 con 0x55ad3896a7a0 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad38973ca0 0x55ad3898bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad38973ca0 0x55ad3898bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60350/0 (socket says 172.21.15.33:60350) 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a737fe640 1 -- 172.21.15.33:0/3705154084 learned_addr learned my addr 172.21.15.33:0/3705154084 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a73fff640 1 --2- 172.21.15.33:0/3705154084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896b0c0 0x55ad389652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a737fe640 1 --2- 172.21.15.33:0/3705154084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad3896a7a0 0x55ad38970390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a737fe640 1 -- 172.21.15.33:0/3705154084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad3896a7a0 msgr2=0x55ad38970390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a737fe640 1 --2- 172.21.15.33:0/3705154084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad3896a7a0 0x55ad38970390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a737fe640 1 -- 172.21.15.33:0/3705154084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896b0c0 msgr2=0x55ad389652b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a737fe640 1 --2- 172.21.15.33:0/3705154084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896b0c0 0x55ad389652b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a737fe640 1 -- 172.21.15.33:0/3705154084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad38971520 con 0x55ad38973ca0 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a73fff640 1 --2- 172.21.15.33:0/3705154084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896b0c0 0x55ad389652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a737fe640 1 --2- 172.21.15.33:0/3705154084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad38973ca0 0x55ad3898bd50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3a68012a30 tx=0x7f3a6800cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0a1c9c1d91cf22 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a59ffb640 1 -- 172.21.15.33:0/3705154084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68005070 con 0x55ad38973ca0 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a59ffb640 1 -- 172.21.15.33:0/3705154084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3a6800e3e0 con 0x55ad38973ca0 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.780+0000 7f3a59ffb640 1 -- 172.21.15.33:0/3705154084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68004920 con 0x55ad38973ca0 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.781+0000 7f3a7d067740 1 -- 172.21.15.33:0/3705154084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad38973ca0 msgr2=0x55ad3898bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.781+0000 7f3a7d067740 1 --2- 172.21.15.33:0/3705154084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad38973ca0 0x55ad3898bd50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3a68012a30 tx=0x7f3a6800cfd0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.781+0000 7f3a7d067740 1 -- 172.21.15.33:0/3705154084 shutdown_connections 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.781+0000 7f3a7d067740 1 --2- 172.21.15.33:0/3705154084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad38973ca0 0x55ad3898bd50 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.781+0000 7f3a7d067740 1 --2- 172.21.15.33:0/3705154084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896b0c0 0x55ad389652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.781+0000 7f3a7d067740 1 --2- 172.21.15.33:0/3705154084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad3896a7a0 0x55ad38970390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.781+0000 7f3a7d067740 1 -- 172.21.15.33:0/3705154084 >> 172.21.15.33:0/3705154084 conn(0x55ad38964ed0 msgr2=0x55ad38963f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.781+0000 7f3a7d067740 1 -- 172.21.15.33:0/3705154084 shutdown_connections 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.781+0000 7f3a7d067740 1 -- 172.21.15.33:0/3705154084 wait complete. 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.781+0000 7f3a7d067740 1 -- start start 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.781+0000 7f3a7d067740 1 Processor -- start 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a7d067740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896a7a0 0x55ad3896cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a7d067740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 0x55ad3896d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a7d067740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad38973ca0 0x55ad3896d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a7d067740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a741af950 con 0x55ad3896b0c0 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a7d067740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3a741b0dc0 con 0x55ad3896a7a0 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a7d067740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3a741b0020 con 0x55ad38973ca0 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a7d067740 10 client.0 _ll_register_callbacks cb 0x7f3a6800c970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 0x55ad3896d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 0x55ad3896d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60364/0 (socket says 172.21.15.33:60364) 2024-04-02T13:45:54.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a73fff640 1 -- 172.21.15.33:0/860371968 learned_addr learned my addr 172.21.15.33:0/860371968 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a737fe640 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad38973ca0 0x55ad3896d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a737fe640 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896a7a0 0x55ad3896cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a73fff640 1 -- 172.21.15.33:0/860371968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad38973ca0 msgr2=0x55ad3896d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a73fff640 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad38973ca0 0x55ad3896d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a73fff640 1 -- 172.21.15.33:0/860371968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896a7a0 msgr2=0x55ad3896cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a73fff640 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896a7a0 0x55ad3896cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a73fff640 1 -- 172.21.15.33:0/860371968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad389867b0 con 0x55ad3896b0c0 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a737fe640 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad38973ca0 0x55ad3896d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a73fff640 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 0x55ad3896d2e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400bc60 tx=0x7f3a64005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ac59cc1fc8b53f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a537fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a537fe640 1 -- 172.21.15.33:0/860371968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6402e060 con 0x55ad3896b0c0 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a537fe640 1 -- 172.21.15.33:0/860371968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f3a64004440 con 0x55ad3896b0c0 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a7d067740 1 -- 172.21.15.33:0/860371968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a6800e3e0 con 0x55ad3896b0c0 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a537fe640 1 -- 172.21.15.33:0/860371968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3a64035050 con 0x55ad3896b0c0 2024-04-02T13:45:54.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a7d067740 10 client.17175 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.782+0000 7f3a7d067740 1 -- 172.21.15.33:0/860371968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ad3896aef0 con 0x55ad3896b0c0 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.783+0000 7f3a537fe640 1 -- 172.21.15.33:0/860371968 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3a6800e3e0 con 0x55ad3896b0c0 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.783+0000 7f3a537fe640 1 -- 172.21.15.33:0/860371968 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f3a64028600 con 0x55ad3896b0c0 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.783+0000 7f3a7d067740 10 client.17175 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.783+0000 7f3a7d067740 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.783+0000 7f3a7d067740 20 client.17175 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.783+0000 7f3a7d067740 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad3897c3f0 0x55ad389ec5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.783+0000 7f3a7d067740 4 client.17175 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.783+0000 7f3a7d067740 1 -- 172.21.15.33:0/860371968 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad388d6040 con 0x55ad3897c3f0 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.783+0000 7f3a737fe640 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad3897c3f0 0x55ad389ec5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.783+0000 7f3a537fe640 10 client.17175 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.783+0000 7f3a737fe640 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad3897c3f0 0x55ad389ec5d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcfb6eca4a637bab server_cookie=8b397fc12b010b24 in_seq=0 out_seq=0 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a537fe640 1 -- 172.21.15.33:0/860371968 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad388d6040 con 0x55ad3897c3f0 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a537fe640 10 client.17175 handle_command_reply: tid=0 2024-04-02T13:45:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7d067740 1 client.17175 shutdown 2024-04-02T13:45:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7d067740 1 -- 172.21.15.33:0/860371968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 msgr2=0x55ad3896d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7d067740 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 0x55ad3896d2e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400bc60 tx=0x7f3a64005850 comp rx=0 tx=0).stop 2024-04-02T13:45:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7d067740 1 -- 172.21.15.33:0/860371968 shutdown_connections 2024-04-02T13:45:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7d067740 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ad3897c3f0 0x55ad389ec5d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7d067740 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ad38973ca0 0x55ad3896d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7d067740 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad3896b0c0 0x55ad3896d2e0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a537fe640 0 client.17175 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7d067740 1 --2- 172.21.15.33:0/860371968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad3896a7a0 0x55ad3896cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7d067740 1 -- 172.21.15.33:0/860371968 >> 172.21.15.33:0/860371968 conn(0x55ad38964ed0 msgr2=0x55ad38963f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7d067740 1 -- 172.21.15.33:0/860371968 shutdown_connections 2024-04-02T13:45:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7d067740 1 -- 172.21.15.33:0/860371968 wait complete. 2024-04-02T13:45:54.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7d067740 20 client.17175 trim_cache size 0 max 16384 2024-04-02T13:45:54.794 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:45:54.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7a47b640 1 -- 172.21.15.33:0/1803250465 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3a5c0a3d20 msgr2=0x7f3a5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7a47b640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3a5c0a3d20 0x7f3a5c0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0f0b70 tx=0x7f3a6c0a8040 comp rx=0 tx=0).stop 2024-04-02T13:45:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7a47b640 1 -- 172.21.15.33:0/1803250465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a74107aa0 msgr2=0x7f3a74074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.784+0000 7f3a7a47b640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a74107aa0 0x7f3a74074880 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c004af0 tx=0x7f3a6c034d70 comp rx=0 tx=0).stop 2024-04-02T13:45:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.785+0000 7f3a7a47b640 1 -- 172.21.15.33:0/1803250465 shutdown_connections 2024-04-02T13:45:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.785+0000 7f3a7a47b640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3a5c0a3d20 0x7f3a5c0a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.785+0000 7f3a7a47b640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a74107aa0 0x7f3a74074880 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.785+0000 7f3a7a47b640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a74102450 0x7f3a74074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.785+0000 7f3a7a47b640 1 --2- 172.21.15.33:0/1803250465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a74101a90 0x7f3a74073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.785+0000 7f3a7a47b640 1 -- 172.21.15.33:0/1803250465 >> 172.21.15.33:0/1803250465 conn(0x7f3a740fd5a0 msgr2=0x7f3a740fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.785+0000 7f3a7a47b640 1 -- 172.21.15.33:0/1803250465 shutdown_connections 2024-04-02T13:45:54.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:54.785+0000 7f3a7a47b640 1 -- 172.21.15.33:0/1803250465 wait complete. 2024-04-02T13:45:55.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:55 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2376447356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:45:55.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:55 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2376447356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:45:55.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:55 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2376447356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:45:55.799 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:45:55.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:45:55.800 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:45:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.916+0000 7f422d02c640 1 -- 172.21.15.33:0/3602621127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4228106980 msgr2=0x7f4228106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.916+0000 7f422d02c640 1 --2- 172.21.15.33:0/3602621127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4228106980 0x7f4228106d60 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4220009cd0 tx=0x7f422002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.916+0000 7f422d02c640 1 -- 172.21.15.33:0/3602621127 shutdown_connections 2024-04-02T13:45:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.916+0000 7f422d02c640 1 --2- 172.21.15.33:0/3602621127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4228101330 0x7f422810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.916+0000 7f422d02c640 1 --2- 172.21.15.33:0/3602621127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4228100970 0x7f4228100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.916+0000 7f422d02c640 1 --2- 172.21.15.33:0/3602621127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4228106980 0x7f4228106d60 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.916+0000 7f422d02c640 1 -- 172.21.15.33:0/3602621127 >> 172.21.15.33:0/3602621127 conn(0x7f4228077b00 msgr2=0x7f4228077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.917+0000 7f422d02c640 1 -- 172.21.15.33:0/3602621127 shutdown_connections 2024-04-02T13:45:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.917+0000 7f422d02c640 1 -- 172.21.15.33:0/3602621127 wait complete. 2024-04-02T13:45:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.917+0000 7f422d02c640 1 Processor -- start 2024-04-02T13:45:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.917+0000 7f422d02c640 1 -- start start 2024-04-02T13:45:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422d02c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4228100970 0x7f4228197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422d02c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4228101330 0x7f42281977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422d02c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4228106980 0x7f422819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422d02c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4228110d60 con 0x7f4228106980 2024-04-02T13:45:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422d02c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4228110be0 con 0x7f4228100970 2024-04-02T13:45:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422d02c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4228110ee0 con 0x7f4228101330 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4228106980 0x7f422819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4228106980 0x7f422819baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60382/0 (socket says 172.21.15.33:60382) 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422759e640 1 -- 172.21.15.33:0/2317969992 learned_addr learned my addr 172.21.15.33:0/2317969992 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f4226d9d640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4228100970 0x7f4228197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422659c640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4228101330 0x7f42281977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422759e640 1 -- 172.21.15.33:0/2317969992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4228101330 msgr2=0x7f42281977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422759e640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4228101330 0x7f42281977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422759e640 1 -- 172.21.15.33:0/2317969992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4228100970 msgr2=0x7f4228197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422759e640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4228100970 0x7f4228197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422759e640 1 -- 172.21.15.33:0/2317969992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422819c220 con 0x7f4228106980 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.918+0000 7f422759e640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4228106980 0x7f422819baf0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f421800c970 tx=0x7f421800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.919+0000 7f421ffff640 1 -- 172.21.15.33:0/2317969992 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218007bb0 con 0x7f4228106980 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.919+0000 7f421ffff640 1 -- 172.21.15.33:0/2317969992 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4218007d50 con 0x7f4228106980 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.919+0000 7f422d02c640 1 -- 172.21.15.33:0/2317969992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422819c510 con 0x7f4228106980 2024-04-02T13:45:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.919+0000 7f421ffff640 1 -- 172.21.15.33:0/2317969992 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218018790 con 0x7f4228106980 2024-04-02T13:45:55.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.919+0000 7f422d02c640 1 -- 172.21.15.33:0/2317969992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42281a3da0 con 0x7f4228106980 2024-04-02T13:45:55.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.921+0000 7f421ffff640 1 -- 172.21.15.33:0/2317969992 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f421800fab0 con 0x7f4228106980 2024-04-02T13:45:55.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.921+0000 7f422fc0f740 1 Processor -- start 2024-04-02T13:45:55.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.921+0000 7f422fc0f740 1 -- start start 2024-04-02T13:45:55.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f422fc0f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce04d0 0x564de1ce08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f422fc0f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 0x564de1ce9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f422fc0f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce99d0 0x564de1cebdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f422fc0f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564de1c42770 con 0x564de1ce0df0 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f422fc0f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564de1bea960 con 0x564de1ce99d0 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f422fc0f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564de1c3e8c0 con 0x564de1ce04d0 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f421ffff640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f42140a3d20 0x7f42140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f4226d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 0x564de1ce9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f4226d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 0x564de1ce9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60390/0 (socket says 172.21.15.33:60390) 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f4226d9d640 1 -- 172.21.15.33:0/1222227344 learned_addr learned my addr 172.21.15.33:0/1222227344 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f422659c640 1 --2- 172.21.15.33:0/1222227344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce04d0 0x564de1ce08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f422659c640 1 --2- 172.21.15.33:0/1222227344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce99d0 0x564de1cebdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f422759e640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f42140a3d20 0x7f42140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f421ffff640 1 -- 172.21.15.33:0/2317969992 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f42180d0c90 con 0x7f4228106980 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f4226d9d640 1 -- 172.21.15.33:0/1222227344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce04d0 msgr2=0x564de1ce08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f4226d9d640 1 --2- 172.21.15.33:0/1222227344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce04d0 0x564de1ce08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f4226d9d640 1 -- 172.21.15.33:0/1222227344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce99d0 msgr2=0x564de1cebdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f4226d9d640 1 --2- 172.21.15.33:0/1222227344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce99d0 0x564de1cebdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f4226d9d640 1 -- 172.21.15.33:0/1222227344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de1a1e7e0 con 0x564de1ce0df0 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.922+0000 7f4226d9d640 1 --2- 172.21.15.33:0/1222227344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 0x564de1ce9490 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f4220000f80 tx=0x7f4220032380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad26a8a6d5808b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.923+0000 7f422759e640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f42140a3d20 0x7f42140a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f421809c500 tx=0x7f4218087040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.923+0000 7f421d7fa640 1 -- 172.21.15.33:0/1222227344 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220045020 con 0x564de1ce0df0 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.923+0000 7f421d7fa640 1 -- 172.21.15.33:0/1222227344 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f422003a040 con 0x564de1ce0df0 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.923+0000 7f421d7fa640 1 -- 172.21.15.33:0/1222227344 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220039040 con 0x564de1ce0df0 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.923+0000 7f422fc0f740 1 -- 172.21.15.33:0/1222227344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 msgr2=0x564de1ce9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.923+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1222227344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 0x564de1ce9490 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f4220000f80 tx=0x7f4220032380 comp rx=0 tx=0).stop 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.923+0000 7f422fc0f740 1 -- 172.21.15.33:0/1222227344 shutdown_connections 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.923+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1222227344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce99d0 0x564de1cebdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.923+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1222227344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 0x564de1ce9490 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.923+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1222227344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce04d0 0x564de1ce08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.923+0000 7f422fc0f740 1 -- 172.21.15.33:0/1222227344 >> 172.21.15.33:0/1222227344 conn(0x564de1cdac00 msgr2=0x564de1cdafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422fc0f740 1 -- 172.21.15.33:0/1222227344 shutdown_connections 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422fc0f740 1 -- 172.21.15.33:0/1222227344 wait complete. 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422fc0f740 1 -- start start 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422fc0f740 1 Processor -- start 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422fc0f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 0x564de1ce6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422fc0f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce0df0 0x564de1ce6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422fc0f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce99d0 0x564de1ce74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422fc0f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564de1cec760 con 0x564de1ce04d0 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422fc0f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564de1bfdd20 con 0x564de1ce0df0 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422fc0f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564de1cec8e0 con 0x564de1ce99d0 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422fc0f740 10 client.0 _ll_register_callbacks cb 0x564de1bcaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 0x564de1ce6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 0x564de1ce6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60400/0 (socket says 172.21.15.33:60400) 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f422659c640 1 -- 172.21.15.33:0/1031275778 learned_addr learned my addr 172.21.15.33:0/1031275778 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.924+0000 7f4226d9d640 1 --2- 172.21.15.33:0/1031275778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce0df0 0x564de1ce6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f422659c640 1 -- 172.21.15.33:0/1031275778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce99d0 msgr2=0x564de1ce74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f422659c640 1 --2- 172.21.15.33:0/1031275778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce99d0 0x564de1ce74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f422659c640 1 -- 172.21.15.33:0/1031275778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce0df0 msgr2=0x564de1ce6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f422659c640 1 --2- 172.21.15.33:0/1031275778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce0df0 0x564de1ce6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f422659c640 1 -- 172.21.15.33:0/1031275778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de1a7ef70 con 0x564de1ce04d0 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f422659c640 1 --2- 172.21.15.33:0/1031275778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 0x564de1ce6a20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f421000e880 tx=0x7f42100108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6382dc7ebaaa2662 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f420e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f420e7fc640 1 -- 172.21.15.33:0/1031275778 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210004020 con 0x564de1ce04d0 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f420e7fc640 1 -- 172.21.15.33:0/1031275778 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f42100041c0 con 0x564de1ce04d0 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f422fc0f740 1 -- 172.21.15.33:0/1031275778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f422003a040 con 0x564de1ce04d0 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f420e7fc640 1 -- 172.21.15.33:0/1031275778 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f42100043c0 con 0x564de1ce04d0 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f422fc0f740 10 client.17187 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f422fc0f740 1 -- 172.21.15.33:0/1031275778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564de1916170 con 0x564de1ce04d0 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.925+0000 7f420e7fc640 1 -- 172.21.15.33:0/1031275778 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f422003a040 con 0x564de1ce04d0 2024-04-02T13:45:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.926+0000 7f420e7fc640 1 -- 172.21.15.33:0/1031275778 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f421000f6f0 con 0x564de1ce04d0 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.926+0000 7f422fc0f740 10 client.17187 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.926+0000 7f422fc0f740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.926+0000 7f422fc0f740 20 client.17187 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.926+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1031275778 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564de1cf9da0 0x564de1cfc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.926+0000 7f422fc0f740 4 client.17187 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.926+0000 7f422fc0f740 1 -- 172.21.15.33:0/1031275778 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564de1c4fcb0 con 0x564de1cf9da0 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.926+0000 7f4226d9d640 1 --2- 172.21.15.33:0/1031275778 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564de1cf9da0 0x564de1cfc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.926+0000 7f420e7fc640 10 client.17187 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.926+0000 7f4226d9d640 1 --2- 172.21.15.33:0/1031275778 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564de1cf9da0 0x564de1cfc1f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3f1194f03dc67a server_cookie=143fbb99f76c5ef0 in_seq=0 out_seq=0 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f420e7fc640 1 -- 172.21.15.33:0/1031275778 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564de1c4fcb0 con 0x564de1cf9da0 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f420e7fc640 10 client.17187 handle_command_reply: tid=0 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f422fc0f740 1 client.17187 shutdown 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f422fc0f740 1 -- 172.21.15.33:0/1031275778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 msgr2=0x564de1ce6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1031275778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 0x564de1ce6a20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f421000e880 tx=0x7f42100108b0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f422fc0f740 1 -- 172.21.15.33:0/1031275778 shutdown_connections 2024-04-02T13:45:55.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1031275778 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564de1cf9da0 0x564de1cfc1f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1031275778 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce99d0 0x564de1ce74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1031275778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce0df0 0x564de1ce6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f420e7fc640 0 client.17187 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1031275778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 0x564de1ce6a20 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f422fc0f740 1 -- 172.21.15.33:0/1031275778 >> 172.21.15.33:0/1031275778 conn(0x564de1cdac00 msgr2=0x564de1cdc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f422fc0f740 1 -- 172.21.15.33:0/1031275778 shutdown_connections 2024-04-02T13:45:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f422fc0f740 1 -- 172.21.15.33:0/1031275778 wait complete. 2024-04-02T13:45:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.927+0000 7f422fc0f740 20 client.17187 trim_cache size 0 max 16384 2024-04-02T13:45:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.945+0000 7f422fc0f740 1 Processor -- start 2024-04-02T13:45:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.945+0000 7f422fc0f740 1 -- start start 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.945+0000 7f422fc0f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 0x564de1ce9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.945+0000 7f422fc0f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce0df0 0x564de1cf5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.945+0000 7f422fc0f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce99d0 0x564de1d01ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.945+0000 7f422fc0f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42281af640 con 0x564de1ce04d0 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.945+0000 7f422fc0f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f42281acf10 con 0x564de1ce99d0 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.945+0000 7f422fc0f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f42281aad80 con 0x564de1ce0df0 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f422659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 0x564de1ce9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f422659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 0x564de1ce9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60412/0 (socket says 172.21.15.33:60412) 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f422659c640 1 -- 172.21.15.33:0/1470498815 learned_addr learned my addr 172.21.15.33:0/1470498815 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f422659c640 1 --2- 172.21.15.33:0/1470498815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce99d0 0x564de1d01ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f4226d9d640 1 --2- 172.21.15.33:0/1470498815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce0df0 0x564de1cf5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f422659c640 1 -- 172.21.15.33:0/1470498815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce0df0 msgr2=0x564de1cf5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f422659c640 1 --2- 172.21.15.33:0/1470498815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce0df0 0x564de1cf5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f422659c640 1 -- 172.21.15.33:0/1470498815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce99d0 msgr2=0x564de1d01ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f422659c640 1 --2- 172.21.15.33:0/1470498815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce99d0 0x564de1d01ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f422659c640 1 -- 172.21.15.33:0/1470498815 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de1ce0c20 con 0x564de1ce04d0 2024-04-02T13:45:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f4226d9d640 1 --2- 172.21.15.33:0/1470498815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce0df0 0x564de1cf5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f422659c640 1 --2- 172.21.15.33:0/1470498815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 0x564de1ce9150 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4210010e30 tx=0x7f4210005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2133ca091d1bd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f421cff9640 1 -- 172.21.15.33:0/1470498815 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421000fe40 con 0x564de1ce04d0 2024-04-02T13:45:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f421cff9640 1 -- 172.21.15.33:0/1470498815 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4210005030 con 0x564de1ce04d0 2024-04-02T13:45:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.946+0000 7f421cff9640 1 -- 172.21.15.33:0/1470498815 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210004990 con 0x564de1ce04d0 2024-04-02T13:45:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.947+0000 7f422fc0f740 1 -- 172.21.15.33:0/1470498815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 msgr2=0x564de1ce9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.947+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1470498815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 0x564de1ce9150 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4210010e30 tx=0x7f4210005e20 comp rx=0 tx=0).stop 2024-04-02T13:45:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.947+0000 7f422fc0f740 1 -- 172.21.15.33:0/1470498815 shutdown_connections 2024-04-02T13:45:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.947+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1470498815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce99d0 0x564de1d01ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.947+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1470498815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce0df0 0x564de1cf5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.947+0000 7f422fc0f740 1 --2- 172.21.15.33:0/1470498815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce04d0 0x564de1ce9150 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.947+0000 7f422fc0f740 1 -- 172.21.15.33:0/1470498815 >> 172.21.15.33:0/1470498815 conn(0x564de1cdac00 msgr2=0x564de1cd7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.947+0000 7f422fc0f740 1 -- 172.21.15.33:0/1470498815 shutdown_connections 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.947+0000 7f422fc0f740 1 -- 172.21.15.33:0/1470498815 wait complete. 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.947+0000 7f422fc0f740 1 -- start start 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f422fc0f740 1 Processor -- start 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f422fc0f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce04d0 0x564de1ce37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f422fc0f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 0x564de1ce3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f422fc0f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce99d0 0x564de1ce4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f422fc0f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42281ab3d0 con 0x564de1ce0df0 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f422fc0f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f42281ac840 con 0x564de1ce04d0 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f422fc0f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f42281abaa0 con 0x564de1ce99d0 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f422fc0f740 10 client.0 _ll_register_callbacks cb 0x7f4210010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f4226d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 0x564de1ce3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f4226d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 0x564de1ce3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60428/0 (socket says 172.21.15.33:60428) 2024-04-02T13:45:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f4226d9d640 1 -- 172.21.15.33:0/2972716344 learned_addr learned my addr 172.21.15.33:0/2972716344 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f422659c640 1 --2- 172.21.15.33:0/2972716344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce99d0 0x564de1ce4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f422659c640 1 --2- 172.21.15.33:0/2972716344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce04d0 0x564de1ce37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f4226d9d640 1 -- 172.21.15.33:0/2972716344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce99d0 msgr2=0x564de1ce4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f4226d9d640 1 --2- 172.21.15.33:0/2972716344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce99d0 0x564de1ce4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f4226d9d640 1 -- 172.21.15.33:0/2972716344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce04d0 msgr2=0x564de1ce37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f4226d9d640 1 --2- 172.21.15.33:0/2972716344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce04d0 0x564de1ce37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f4226d9d640 1 -- 172.21.15.33:0/2972716344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de1d5f430 con 0x564de1ce0df0 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f4226d9d640 1 --2- 172.21.15.33:0/2972716344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 0x564de1ce3d00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4220005cc0 tx=0x7f422002c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8187b8abe4fa7cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.948+0000 7f420e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f420e7fc640 1 -- 172.21.15.33:0/2972716344 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220039040 con 0x564de1ce0df0 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f420e7fc640 1 -- 172.21.15.33:0/2972716344 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f42200325d0 con 0x564de1ce0df0 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f422fc0f740 1 -- 172.21.15.33:0/2972716344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4210005030 con 0x564de1ce0df0 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f420e7fc640 1 -- 172.21.15.33:0/2972716344 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4220045040 con 0x564de1ce0df0 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f422fc0f740 10 client.17193 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f422fc0f740 1 -- 172.21.15.33:0/2972716344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564de1d60710 con 0x564de1ce0df0 2024-04-02T13:45:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f420e7fc640 1 -- 172.21.15.33:0/2972716344 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4210005030 con 0x564de1ce0df0 2024-04-02T13:45:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f420e7fc640 1 -- 172.21.15.33:0/2972716344 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f42200040d0 con 0x564de1ce0df0 2024-04-02T13:45:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f422fc0f740 10 client.17193 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f422fc0f740 10 client.17193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f422fc0f740 20 client.17193 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f422fc0f740 1 --2- 172.21.15.33:0/2972716344 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564de1d60a80 0x564de1d62e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f422fc0f740 4 client.17193 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:45:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f422fc0f740 1 -- 172.21.15.33:0/2972716344 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564de1c4a000 con 0x564de1d60a80 2024-04-02T13:45:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.949+0000 7f422659c640 1 --2- 172.21.15.33:0/2972716344 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564de1d60a80 0x564de1d62e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.950+0000 7f422659c640 1 --2- 172.21.15.33:0/2972716344 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564de1d60a80 0x564de1d62e70 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c7600f39cd8554 server_cookie=f9ce5b35ce56d1fc in_seq=0 out_seq=0 2024-04-02T13:45:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.950+0000 7f420e7fc640 10 client.17193 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.951+0000 7f420e7fc640 1 -- 172.21.15.33:0/2972716344 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564de1c4a000 con 0x564de1d60a80 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.951+0000 7f420e7fc640 10 client.17193 handle_command_reply: tid=0 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.951+0000 7f422fc0f740 1 client.17193 shutdown 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.951+0000 7f422fc0f740 1 -- 172.21.15.33:0/2972716344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 msgr2=0x564de1ce3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.951+0000 7f422fc0f740 1 --2- 172.21.15.33:0/2972716344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 0x564de1ce3d00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4220005cc0 tx=0x7f422002c7f0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.951+0000 7f422fc0f740 1 -- 172.21.15.33:0/2972716344 shutdown_connections 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422fc0f740 1 --2- 172.21.15.33:0/2972716344 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x564de1d60a80 0x564de1d62e70 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422fc0f740 1 --2- 172.21.15.33:0/2972716344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564de1ce99d0 0x564de1ce4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422fc0f740 1 --2- 172.21.15.33:0/2972716344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564de1ce0df0 0x564de1ce3d00 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f420e7fc640 0 client.17193 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422fc0f740 1 --2- 172.21.15.33:0/2972716344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564de1ce04d0 0x564de1ce37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422fc0f740 1 -- 172.21.15.33:0/2972716344 >> 172.21.15.33:0/2972716344 conn(0x564de1cdac00 msgr2=0x564de1ce69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422fc0f740 1 -- 172.21.15.33:0/2972716344 shutdown_connections 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422fc0f740 1 -- 172.21.15.33:0/2972716344 wait complete. 2024-04-02T13:45:55.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422fc0f740 20 client.17193 trim_cache size 0 max 16384 2024-04-02T13:45:55.960 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:45:55.960 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:45:55.960 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "93ae72f7-8a1f-4ab2-862f-516f9b19d251", 2024-04-02T13:45:55.960 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:45:55.960 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:45:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422d02c640 1 -- 172.21.15.33:0/2317969992 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f42140a3d20 msgr2=0x7f42140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422d02c640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f42140a3d20 0x7f42140a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f421809c500 tx=0x7f4218087040 comp rx=0 tx=0).stop 2024-04-02T13:45:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422d02c640 1 -- 172.21.15.33:0/2317969992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4228106980 msgr2=0x7f422819baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422d02c640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4228106980 0x7f422819baf0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f421800c970 tx=0x7f421800ce30 comp rx=0 tx=0).stop 2024-04-02T13:45:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422d02c640 1 -- 172.21.15.33:0/2317969992 shutdown_connections 2024-04-02T13:45:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422d02c640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f42140a3d20 0x7f42140a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422d02c640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4228106980 0x7f422819baf0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422d02c640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4228101330 0x7f42281977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422d02c640 1 --2- 172.21.15.33:0/2317969992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4228100970 0x7f4228197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422d02c640 1 -- 172.21.15.33:0/2317969992 >> 172.21.15.33:0/2317969992 conn(0x7f4228077b00 msgr2=0x7f42281030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422d02c640 1 -- 172.21.15.33:0/2317969992 shutdown_connections 2024-04-02T13:45:55.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:55.952+0000 7f422d02c640 1 -- 172.21.15.33:0/2317969992 wait complete. 2024-04-02T13:45:55.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:45:55.966 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:45:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.069+0000 7fb100982640 1 -- 172.21.15.33:0/1513037560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc100970 msgr2=0x7fb0fc100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.069+0000 7fb100982640 1 --2- 172.21.15.33:0/1513037560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc100970 0x7fb0fc100df0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009930 tx=0x7fb0f002bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.069+0000 7fb100982640 1 -- 172.21.15.33:0/1513037560 shutdown_connections 2024-04-02T13:45:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.069+0000 7fb100982640 1 --2- 172.21.15.33:0/1513037560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0fc101330 0x7fb0fc10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.069+0000 7fb100982640 1 --2- 172.21.15.33:0/1513037560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc100970 0x7fb0fc100df0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.069+0000 7fb100982640 1 --2- 172.21.15.33:0/1513037560 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0fc106980 0x7fb0fc106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.069+0000 7fb100982640 1 -- 172.21.15.33:0/1513037560 >> 172.21.15.33:0/1513037560 conn(0x7fb0fc077b00 msgr2=0x7fb0fc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.069+0000 7fb100982640 1 -- 172.21.15.33:0/1513037560 shutdown_connections 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.069+0000 7fb100982640 1 -- 172.21.15.33:0/1513037560 wait complete. 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.070+0000 7fb100982640 1 Processor -- start 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.070+0000 7fb100982640 1 -- start start 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.070+0000 7fb100982640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc100970 0x7fb0fc19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.070+0000 7fb100982640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0fc101330 0x7fb0fc19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.070+0000 7fb100982640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0fc106980 0x7fb0fc19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.070+0000 7fb100982640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc110d60 con 0x7fb0fc100970 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.070+0000 7fb100982640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc110be0 con 0x7fb0fc101330 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.070+0000 7fb100982640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc110ee0 con 0x7fb0fc106980 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.070+0000 7fb0fa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc100970 0x7fb0fc19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.070+0000 7fb0fa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc100970 0x7fb0fc19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60460/0 (socket says 172.21.15.33:60460) 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.070+0000 7fb0fa59c640 1 -- 172.21.15.33:0/989847586 learned_addr learned my addr 172.21.15.33:0/989847586 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.070+0000 7fb0f9d9b640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0fc101330 0x7fb0fc19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.071+0000 7fb0fad9d640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0fc106980 0x7fb0fc19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.071+0000 7fb0fa59c640 1 -- 172.21.15.33:0/989847586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0fc106980 msgr2=0x7fb0fc19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.071+0000 7fb0fa59c640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0fc106980 0x7fb0fc19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.071+0000 7fb0fa59c640 1 -- 172.21.15.33:0/989847586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0fc101330 msgr2=0x7fb0fc19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.071+0000 7fb0fa59c640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0fc101330 0x7fb0fc19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.071+0000 7fb0fa59c640 1 -- 172.21.15.33:0/989847586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc19f540 con 0x7fb0fc100970 2024-04-02T13:45:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.071+0000 7fb0fa59c640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc100970 0x7fb0fc19a550 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400d770 tx=0x7fb0f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.071+0000 7fb0eb7fe640 1 -- 172.21.15.33:0/989847586 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4017070 con 0x7fb0fc100970 2024-04-02T13:45:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.071+0000 7fb0eb7fe640 1 -- 172.21.15.33:0/989847586 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb0f40054f0 con 0x7fb0fc100970 2024-04-02T13:45:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.071+0000 7fb100982640 1 -- 172.21.15.33:0/989847586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0fc19f830 con 0x7fb0fc100970 2024-04-02T13:45:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.071+0000 7fb0eb7fe640 1 -- 172.21.15.33:0/989847586 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4004d60 con 0x7fb0fc100970 2024-04-02T13:45:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.071+0000 7fb100982640 1 -- 172.21.15.33:0/989847586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0fc1a70c0 con 0x7fb0fc100970 2024-04-02T13:45:56.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.073+0000 7fb0eb7fe640 1 -- 172.21.15.33:0/989847586 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb0f4004f00 con 0x7fb0fc100970 2024-04-02T13:45:56.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.073+0000 7fb10356b740 1 Processor -- start 2024-04-02T13:45:56.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb10356b740 1 -- start start 2024-04-02T13:45:56.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb10356b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc24dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb10356b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc24e1f0 0x55c0bc256890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb10356b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 0x55c0bc2591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb10356b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc1b1830 con 0x55c0bc256dd0 2024-04-02T13:45:56.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb10356b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc159820 con 0x55c0bc24e1f0 2024-04-02T13:45:56.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb0eb7fe640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb0e40a3d20 0x7fb0e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb10356b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc1ad7e0 con 0x55c0bc24d8d0 2024-04-02T13:45:56.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb0f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 0x55c0bc2591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb0fad9d640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb0e40a3d20 0x7fb0e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb0f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 0x55c0bc2591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60464/0 (socket says 172.21.15.33:60464) 2024-04-02T13:45:56.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb0f9d9b640 1 -- 172.21.15.33:0/3910833326 learned_addr learned my addr 172.21.15.33:0/3910833326 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb0fad9d640 1 --2- 172.21.15.33:0/3910833326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc24dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb0eb7fe640 1 -- 172.21.15.33:0/989847586 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fb0f40d0620 con 0x7fb0fc100970 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb0fa59c640 1 --2- 172.21.15.33:0/3910833326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc24e1f0 0x55c0bc256890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb0f9d9b640 1 -- 172.21.15.33:0/3910833326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 msgr2=0x55c0bc24dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb0f9d9b640 1 --2- 172.21.15.33:0/3910833326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc24dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb0f9d9b640 1 -- 172.21.15.33:0/3910833326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc24e1f0 msgr2=0x55c0bc256890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb0f9d9b640 1 --2- 172.21.15.33:0/3910833326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc24e1f0 0x55c0bc256890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.074+0000 7fb0f9d9b640 1 -- 172.21.15.33:0/3910833326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0bbf8f7d0 con 0x55c0bc256dd0 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.075+0000 7fb0fad9d640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb0e40a3d20 0x7fb0e40a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009fd0 tx=0x7fb0ec009dc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.075+0000 7fb0f9d9b640 1 --2- 172.21.15.33:0/3910833326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 0x55c0bc2591c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009230 tx=0x7fb0f0009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3466af61316e4fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.075+0000 7fb0e8ff9640 1 -- 172.21.15.33:0/3910833326 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0045020 con 0x55c0bc256dd0 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.075+0000 7fb0e8ff9640 1 -- 172.21.15.33:0/3910833326 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb0f0005040 con 0x55c0bc256dd0 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.075+0000 7fb0e8ff9640 1 -- 172.21.15.33:0/3910833326 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f002cb30 con 0x55c0bc256dd0 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 -- 172.21.15.33:0/3910833326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 msgr2=0x55c0bc2591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:56.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 --2- 172.21.15.33:0/3910833326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 0x55c0bc2591c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009230 tx=0x7fb0f0009610 comp rx=0 tx=0).stop 2024-04-02T13:45:56.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 -- 172.21.15.33:0/3910833326 shutdown_connections 2024-04-02T13:45:56.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 --2- 172.21.15.33:0/3910833326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 0x55c0bc2591c0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 --2- 172.21.15.33:0/3910833326 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc24e1f0 0x55c0bc256890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 --2- 172.21.15.33:0/3910833326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc24dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 -- 172.21.15.33:0/3910833326 >> 172.21.15.33:0/3910833326 conn(0x55c0bc248000 msgr2=0x55c0bc2483e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:56.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 -- 172.21.15.33:0/3910833326 shutdown_connections 2024-04-02T13:45:56.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 -- 172.21.15.33:0/3910833326 wait complete. 2024-04-02T13:45:56.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 -- start start 2024-04-02T13:45:56.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 Processor -- start 2024-04-02T13:45:56.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc2565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc24e1f0 0x55c0bc253940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 0x55c0bc253e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc259b60 con 0x55c0bc256dd0 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc16cd30 con 0x55c0bc24e1f0 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc259ce0 con 0x55c0bc24d8d0 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb10356b740 10 client.0 _ll_register_callbacks cb 0x55c0bc16c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.076+0000 7fb0fad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 0x55c0bc253e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0fad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 0x55c0bc253e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60474/0 (socket says 172.21.15.33:60474) 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0fad9d640 1 -- 172.21.15.33:0/116106207 learned_addr learned my addr 172.21.15.33:0/116106207 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0f9d9b640 1 --2- 172.21.15.33:0/116106207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc24e1f0 0x55c0bc253940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0fa59c640 1 --2- 172.21.15.33:0/116106207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc2565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0fad9d640 1 -- 172.21.15.33:0/116106207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 msgr2=0x55c0bc2565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0fad9d640 1 --2- 172.21.15.33:0/116106207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc2565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0fad9d640 1 -- 172.21.15.33:0/116106207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc24e1f0 msgr2=0x55c0bc253940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0fad9d640 1 --2- 172.21.15.33:0/116106207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc24e1f0 0x55c0bc253940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0fad9d640 1 -- 172.21.15.33:0/116106207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0bbec53f0 con 0x55c0bc256dd0 2024-04-02T13:45:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0fad9d640 1 --2- 172.21.15.33:0/116106207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 0x55c0bc253e80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec011d00 tx=0x7fb0ec009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea6c95855424404 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:56.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:56.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0e1ffb640 1 -- 172.21.15.33:0/116106207 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec027070 con 0x55c0bc256dd0 2024-04-02T13:45:56.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0e1ffb640 1 -- 172.21.15.33:0/116106207 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fb0ec003b70 con 0x55c0bc256dd0 2024-04-02T13:45:56.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb10356b740 1 -- 172.21.15.33:0/116106207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0f0005040 con 0x55c0bc256dd0 2024-04-02T13:45:56.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0e1ffb640 1 -- 172.21.15.33:0/116106207 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb0ec032050 con 0x55c0bc256dd0 2024-04-02T13:45:56.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb10356b740 10 client.17205 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:56.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb10356b740 1 -- 172.21.15.33:0/116106207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c0bbe850f0 con 0x55c0bc256dd0 2024-04-02T13:45:56.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.077+0000 7fb0e1ffb640 1 -- 172.21.15.33:0/116106207 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb0f0005040 con 0x55c0bc256dd0 2024-04-02T13:45:56.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.078+0000 7fb0e1ffb640 1 -- 172.21.15.33:0/116106207 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fb0ec021480 con 0x55c0bc256dd0 2024-04-02T13:45:56.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.078+0000 7fb10356b740 10 client.17205 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:56.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.078+0000 7fb10356b740 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:56.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.078+0000 7fb10356b740 20 client.17205 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:56.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.078+0000 7fb10356b740 1 --2- 172.21.15.33:0/116106207 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c0bc266be0 0x55c0bc269030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.078+0000 7fb10356b740 4 client.17205 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:56.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.078+0000 7fb10356b740 1 -- 172.21.15.33:0/116106207 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0bc1bed70 con 0x55c0bc266be0 2024-04-02T13:45:56.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.078+0000 7fb0fa59c640 1 --2- 172.21.15.33:0/116106207 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c0bc266be0 0x55c0bc269030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.078+0000 7fb0e1ffb640 10 client.17205 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:56.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.078+0000 7fb0fa59c640 1 --2- 172.21.15.33:0/116106207 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c0bc266be0 0x55c0bc269030 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed66059bd4659b6 server_cookie=29810a378788086a in_seq=0 out_seq=0 2024-04-02T13:45:56.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb0e1ffb640 1 -- 172.21.15.33:0/116106207 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0bc1bed70 con 0x55c0bc266be0 2024-04-02T13:45:56.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb0e1ffb640 10 client.17205 handle_command_reply: tid=0 2024-04-02T13:45:56.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb10356b740 1 client.17205 shutdown 2024-04-02T13:45:56.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb10356b740 1 -- 172.21.15.33:0/116106207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 msgr2=0x55c0bc253e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:56.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb10356b740 1 --2- 172.21.15.33:0/116106207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 0x55c0bc253e80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec011d00 tx=0x7fb0ec009500 comp rx=0 tx=0).stop 2024-04-02T13:45:56.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb10356b740 1 -- 172.21.15.33:0/116106207 shutdown_connections 2024-04-02T13:45:56.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb10356b740 1 --2- 172.21.15.33:0/116106207 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c0bc266be0 0x55c0bc269030 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb10356b740 1 --2- 172.21.15.33:0/116106207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc256dd0 0x55c0bc253e80 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb10356b740 1 --2- 172.21.15.33:0/116106207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc24e1f0 0x55c0bc253940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb0e1ffb640 0 client.17205 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:56.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb10356b740 1 --2- 172.21.15.33:0/116106207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc2565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb10356b740 1 -- 172.21.15.33:0/116106207 >> 172.21.15.33:0/116106207 conn(0x55c0bc248000 msgr2=0x55c0bc249d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:56.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb10356b740 1 -- 172.21.15.33:0/116106207 shutdown_connections 2024-04-02T13:45:56.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.079+0000 7fb10356b740 1 -- 172.21.15.33:0/116106207 wait complete. 2024-04-02T13:45:56.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.080+0000 7fb10356b740 20 client.17205 trim_cache size 0 max 16384 2024-04-02T13:45:56.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb10356b740 1 Processor -- start 2024-04-02T13:45:56.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb10356b740 1 -- start start 2024-04-02T13:45:56.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb10356b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc262a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb10356b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 0x55c0bc256070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb10356b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc256dd0 0x55c0bc2565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb10356b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1b3a30 con 0x55c0bc24e1f0 2024-04-02T13:45:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb10356b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1b1300 con 0x55c0bc256dd0 2024-04-02T13:45:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb10356b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1af170 con 0x55c0bc24d8d0 2024-04-02T13:45:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb0f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 0x55c0bc256070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb0f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 0x55c0bc256070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60482/0 (socket says 172.21.15.33:60482) 2024-04-02T13:45:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb0f9d9b640 1 -- 172.21.15.33:0/2991498411 learned_addr learned my addr 172.21.15.33:0/2991498411 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb0fa59c640 1 --2- 172.21.15.33:0/2991498411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc262a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb0f9d9b640 1 -- 172.21.15.33:0/2991498411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 msgr2=0x55c0bc262a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb0f9d9b640 1 --2- 172.21.15.33:0/2991498411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc262a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb0fad9d640 1 --2- 172.21.15.33:0/2991498411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc256dd0 0x55c0bc2565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb0f9d9b640 1 -- 172.21.15.33:0/2991498411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc256dd0 msgr2=0x55c0bc2565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb0f9d9b640 1 --2- 172.21.15.33:0/2991498411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc256dd0 0x55c0bc2565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb0f9d9b640 1 -- 172.21.15.33:0/2991498411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0bc247e30 con 0x55c0bc24e1f0 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.097+0000 7fb0f9d9b640 1 --2- 172.21.15.33:0/2991498411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 0x55c0bc256070 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0005ab0 tx=0x7fb0f002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd07a4898d9d402 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.098+0000 7fb0e3fff640 1 -- 172.21.15.33:0/2991498411 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0045040 con 0x55c0bc24e1f0 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.098+0000 7fb0e3fff640 1 -- 172.21.15.33:0/2991498411 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb0f0005040 con 0x55c0bc24e1f0 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.098+0000 7fb0e3fff640 1 -- 172.21.15.33:0/2991498411 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0004a80 con 0x55c0bc24e1f0 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.098+0000 7fb10356b740 1 -- 172.21.15.33:0/2991498411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 msgr2=0x55c0bc256070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.098+0000 7fb10356b740 1 --2- 172.21.15.33:0/2991498411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 0x55c0bc256070 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0005ab0 tx=0x7fb0f002bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.098+0000 7fb10356b740 1 -- 172.21.15.33:0/2991498411 shutdown_connections 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.098+0000 7fb10356b740 1 --2- 172.21.15.33:0/2991498411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc256dd0 0x55c0bc2565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.098+0000 7fb10356b740 1 --2- 172.21.15.33:0/2991498411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 0x55c0bc256070 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.098+0000 7fb10356b740 1 --2- 172.21.15.33:0/2991498411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc262a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.098+0000 7fb10356b740 1 -- 172.21.15.33:0/2991498411 >> 172.21.15.33:0/2991498411 conn(0x55c0bc248000 msgr2=0x55c0bc24efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.098+0000 7fb10356b740 1 -- 172.21.15.33:0/2991498411 shutdown_connections 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.098+0000 7fb10356b740 1 -- 172.21.15.33:0/2991498411 wait complete. 2024-04-02T13:45:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.098+0000 7fb10356b740 1 -- start start 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb10356b740 1 Processor -- start 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb10356b740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc25aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb10356b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 0x55c0bc25afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb10356b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc256dd0 0x55c0bc25b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb10356b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1af7c0 con 0x55c0bc24e1f0 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb10356b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1b0c30 con 0x55c0bc256dd0 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb10356b740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1afe90 con 0x55c0bc24d8d0 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb10356b740 10 client.0 _ll_register_callbacks cb 0x7fb0ec014510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb0f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 0x55c0bc25afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb0f9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 0x55c0bc25afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60488/0 (socket says 172.21.15.33:60488) 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb0f9d9b640 1 -- 172.21.15.33:0/3335175829 learned_addr learned my addr 172.21.15.33:0/3335175829 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb0fa59c640 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc25aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb0fad9d640 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc256dd0 0x55c0bc25b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb0f9d9b640 1 -- 172.21.15.33:0/3335175829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 msgr2=0x55c0bc25aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb0f9d9b640 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc25aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb0f9d9b640 1 -- 172.21.15.33:0/3335175829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc256dd0 msgr2=0x55c0bc25b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb0f9d9b640 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc256dd0 0x55c0bc25b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb0f9d9b640 1 -- 172.21.15.33:0/3335175829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0bc24d070 con 0x55c0bc24e1f0 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb0fad9d640 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc256dd0 0x55c0bc25b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb0f9d9b640 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 0x55c0bc25afc0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0000c00 tx=0x7fb0f0005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c777106a15d459bb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.099+0000 7fb0e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb0e1ffb640 1 -- 172.21.15.33:0/3335175829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0045040 con 0x55c0bc24e1f0 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb0e1ffb640 1 -- 172.21.15.33:0/3335175829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fb0f002cb80 con 0x55c0bc24e1f0 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb10356b740 1 -- 172.21.15.33:0/3335175829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0f0005040 con 0x55c0bc24e1f0 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb0e1ffb640 1 -- 172.21.15.33:0/3335175829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb0f00047c0 con 0x55c0bc24e1f0 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb10356b740 10 client.17211 fetch_fsmap learned FSMap version 36 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb10356b740 1 -- 172.21.15.33:0/3335175829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c0bc2cd370 con 0x55c0bc24e1f0 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb0e1ffb640 1 -- 172.21.15.33:0/3335175829 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb0f0005040 con 0x55c0bc24e1f0 2024-04-02T13:45:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb0e1ffb640 1 -- 172.21.15.33:0/3335175829 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fb0f0003be0 con 0x55c0bc24e1f0 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb10356b740 10 client.17211 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb10356b740 10 client.17211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb10356b740 20 client.17211 populate_metadata read hostname 'smithi033' 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb10356b740 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c0bc2cd690 0x55c0bc2cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb10356b740 4 client.17211 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb10356b740 1 -- 172.21.15.33:0/3335175829 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c0bc2543c0 con 0x55c0bc2cd690 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.100+0000 7fb0fa59c640 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c0bc2cd690 0x55c0bc2cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb0e1ffb640 10 client.17211 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb0fa59c640 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c0bc2cd690 0x55c0bc2cfa80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97e8bede02f52d89 server_cookie=feff760fef462853 in_seq=0 out_seq=0 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb0e1ffb640 1 -- 172.21.15.33:0/3335175829 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c0bc2543c0 con 0x55c0bc2cd690 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb0e1ffb640 10 client.17211 handle_command_reply: tid=0 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb10356b740 1 client.17211 shutdown 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb10356b740 1 -- 172.21.15.33:0/3335175829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 msgr2=0x55c0bc25afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb10356b740 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 0x55c0bc25afc0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0000c00 tx=0x7fb0f0005570 comp rx=0 tx=0).stop 2024-04-02T13:45:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb10356b740 1 -- 172.21.15.33:0/3335175829 shutdown_connections 2024-04-02T13:45:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb10356b740 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c0bc2cd690 0x55c0bc2cfa80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb10356b740 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0bc256dd0 0x55c0bc25b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb0e1ffb640 0 client.17211 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:45:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb10356b740 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc24e1f0 0x55c0bc25afc0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb10356b740 1 --2- 172.21.15.33:0/3335175829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c0bc24d8d0 0x55c0bc25aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb10356b740 1 -- 172.21.15.33:0/3335175829 >> 172.21.15.33:0/3335175829 conn(0x55c0bc248000 msgr2=0x55c0bc24e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb10356b740 1 -- 172.21.15.33:0/3335175829 shutdown_connections 2024-04-02T13:45:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb10356b740 1 -- 172.21.15.33:0/3335175829 wait complete. 2024-04-02T13:45:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.101+0000 7fb10356b740 20 client.17211 trim_cache size 0 max 16384 2024-04-02T13:45:56.111 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:45:56.112 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:45:56.112 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:45:56.112 INFO:teuthology.orchestra.run.smithi033.stdout: "93ae72f7-8a1f-4ab2-862f-516f9b19d251": { 2024-04-02T13:45:56.112 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:45:56.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "93ae72f7-8a1f-4ab2-862f-516f9b19d251", 2024-04-02T13:45:56.112 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:45:56.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:45:56.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:45:56.112 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:45:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.102+0000 7fb100982640 1 -- 172.21.15.33:0/989847586 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb0e40a3d20 msgr2=0x7fb0e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.102+0000 7fb100982640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb0e40a3d20 0x7fb0e40a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009fd0 tx=0x7fb0ec009dc0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.102+0000 7fb100982640 1 -- 172.21.15.33:0/989847586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc100970 msgr2=0x7fb0fc19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.102+0000 7fb100982640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc100970 0x7fb0fc19a550 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400d770 tx=0x7fb0f400dc30 comp rx=0 tx=0).stop 2024-04-02T13:45:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.102+0000 7fb100982640 1 -- 172.21.15.33:0/989847586 shutdown_connections 2024-04-02T13:45:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.102+0000 7fb100982640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb0e40a3d20 0x7fb0e40a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.102+0000 7fb100982640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0fc106980 0x7fb0fc19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.102+0000 7fb100982640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0fc101330 0x7fb0fc19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.102+0000 7fb100982640 1 --2- 172.21.15.33:0/989847586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0fc100970 0x7fb0fc19a550 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:56.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.102+0000 7fb100982640 1 -- 172.21.15.33:0/989847586 >> 172.21.15.33:0/989847586 conn(0x7fb0fc077b00 msgr2=0x7fb0fc104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:56.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.102+0000 7fb100982640 1 -- 172.21.15.33:0/989847586 shutdown_connections 2024-04-02T13:45:56.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:45:56.102+0000 7fb100982640 1 -- 172.21.15.33:0/989847586 wait complete. 2024-04-02T13:45:56.116 INFO:tasks.cephfs.filesystem:scrub status for tag:93ae72f7-8a1f-4ab2-862f-516f9b19d251 - {'path': '/', 'tag': '93ae72f7-8a1f-4ab2-862f-516f9b19d251', 'options': 'recursive,force'} 2024-04-02T13:45:56.380 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:56 smithi145 ceph-mon[35646]: pgmap v219: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:56.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:56 smithi050 ceph-mon[35524]: pgmap v219: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:56.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:56 smithi033 ceph-mon[32390]: pgmap v219: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:57.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:57 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:45:57.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:57 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:57.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:57 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:45:57.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:57 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:57.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:57 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:45:57.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:57 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:57.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:57 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:45:57.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:57 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:57.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:57 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:45:57.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:57 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:57.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:57 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:45:57.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:57 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:58.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:58 smithi145 ceph-mon[35646]: pgmap v220: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:58.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:58 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:58.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:58 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:58.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:58 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:58.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:45:58 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:45:58.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:58 smithi050 ceph-mon[35524]: pgmap v220: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:58.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:58 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:58.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:58 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:58.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:58 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:58.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:45:58 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:45:58.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:58 smithi033 ceph-mon[32390]: pgmap v220: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:45:58.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:58 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:58.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:58 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:58.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:58 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:58.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:45:58 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:46:00.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:00 smithi145 ceph-mon[35646]: pgmap v221: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:46:00.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:00 smithi050 ceph-mon[35524]: pgmap v221: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:46:00.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:00 smithi033 ceph-mon[32390]: pgmap v221: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:46:01.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:01 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:46:01.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:01 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:46:01.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:01 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:46:02.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:02 smithi145 ceph-mon[35646]: pgmap v222: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:46:02.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:02 smithi050 ceph-mon[35524]: pgmap v222: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:46:02.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:02 smithi033 ceph-mon[32390]: pgmap v222: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:46:04.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:04 smithi145 ceph-mon[35646]: pgmap v223: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-02T13:46:04.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:04 smithi050 ceph-mon[35524]: pgmap v223: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-02T13:46:04.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:04 smithi033 ceph-mon[32390]: pgmap v223: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-02T13:46:06.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:06 smithi145 ceph-mon[35646]: pgmap v224: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-02T13:46:06.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:06 smithi050 ceph-mon[35524]: pgmap v224: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-02T13:46:06.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:06 smithi033 ceph-mon[32390]: pgmap v224: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-02T13:46:07.485 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:07 smithi145 ceph-mon[35646]: osdmap e98: 12 total, 12 up, 12 in 2024-04-02T13:46:07.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:07 smithi050 ceph-mon[35524]: osdmap e98: 12 total, 12 up, 12 in 2024-04-02T13:46:07.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:07 smithi033 ceph-mon[32390]: osdmap e98: 12 total, 12 up, 12 in 2024-04-02T13:46:08.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:08 smithi145 ceph-mon[35646]: pgmap v226: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:46:08.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:08 smithi050 ceph-mon[35524]: pgmap v226: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:46:08.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:08 smithi033 ceph-mon[32390]: pgmap v226: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:46:10.478 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:10 smithi145 ceph-mon[35646]: pgmap v227: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:46:10.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:10 smithi050 ceph-mon[35524]: pgmap v227: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:46:10.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:10 smithi033 ceph-mon[32390]: pgmap v227: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:46:12.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:12 smithi145 ceph-mon[35646]: pgmap v228: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:46:12.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:12 smithi145 ceph-mon[35646]: osdmap e99: 12 total, 12 up, 12 in 2024-04-02T13:46:12.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:12 smithi050 ceph-mon[35524]: pgmap v228: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:46:12.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:12 smithi050 ceph-mon[35524]: osdmap e99: 12 total, 12 up, 12 in 2024-04-02T13:46:13.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:12 smithi033 ceph-mon[32390]: pgmap v228: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:46:13.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:12 smithi033 ceph-mon[32390]: osdmap e99: 12 total, 12 up, 12 in 2024-04-02T13:46:14.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:14 smithi145 ceph-mon[35646]: pgmap v230: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:46:14.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:14 smithi050 ceph-mon[35524]: pgmap v230: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:46:15.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:14 smithi033 ceph-mon[32390]: pgmap v230: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:46:16.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:16 smithi145 ceph-mon[35646]: pgmap v231: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 231 B/s wr, 8 op/s 2024-04-02T13:46:16.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:16 smithi050 ceph-mon[35524]: pgmap v231: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 231 B/s wr, 8 op/s 2024-04-02T13:46:17.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:16 smithi033 ceph-mon[32390]: pgmap v231: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 231 B/s wr, 8 op/s 2024-04-02T13:46:18.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:18 smithi145 ceph-mon[35646]: pgmap v232: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:46:18.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:18 smithi050 ceph-mon[35524]: pgmap v232: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:46:19.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:18 smithi033 ceph-mon[32390]: pgmap v232: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:46:20.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:20 smithi145 ceph-mon[35646]: pgmap v233: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:46:20.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:20 smithi050 ceph-mon[35524]: pgmap v233: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:46:21.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:20 smithi033 ceph-mon[32390]: pgmap v233: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:46:22.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:22 smithi145 ceph-mon[35646]: pgmap v234: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:46:22.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:22 smithi050 ceph-mon[35524]: pgmap v234: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:46:23.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:22 smithi033 ceph-mon[32390]: pgmap v234: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:46:24.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:24 smithi145 ceph-mon[35646]: pgmap v235: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 89 B/s wr, 9 op/s 2024-04-02T13:46:24.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:24 smithi050 ceph-mon[35524]: pgmap v235: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 89 B/s wr, 9 op/s 2024-04-02T13:46:25.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:24 smithi033 ceph-mon[32390]: pgmap v235: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 89 B/s wr, 9 op/s 2024-04-02T13:46:26.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:46:26.117 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:46:26.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.232+0000 7f8a5e4fb640 1 -- 172.21.15.33:0/534698164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a581048c0 msgr2=0x7f8a58108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.232+0000 7f8a5e4fb640 1 --2- 172.21.15.33:0/534698164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a581048c0 0x7f8a58108900 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009900 tx=0x7f8a4c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:26.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.232+0000 7f8a5e4fb640 1 -- 172.21.15.33:0/534698164 shutdown_connections 2024-04-02T13:46:26.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.232+0000 7f8a5e4fb640 1 --2- 172.21.15.33:0/534698164 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a58108fe0 0x7f8a58110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.232+0000 7f8a5e4fb640 1 --2- 172.21.15.33:0/534698164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a581048c0 0x7f8a58108900 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.232+0000 7f8a5e4fb640 1 --2- 172.21.15.33:0/534698164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a58103f10 0x7f8a581042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.232+0000 7f8a5e4fb640 1 -- 172.21.15.33:0/534698164 >> 172.21.15.33:0/534698164 conn(0x7f8a580ff7b0 msgr2=0x7f8a58101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.232+0000 7f8a5e4fb640 1 -- 172.21.15.33:0/534698164 shutdown_connections 2024-04-02T13:46:26.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.232+0000 7f8a5e4fb640 1 -- 172.21.15.33:0/534698164 wait complete. 2024-04-02T13:46:26.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.233+0000 7f8a5e4fb640 1 Processor -- start 2024-04-02T13:46:26.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.233+0000 7f8a5e4fb640 1 -- start start 2024-04-02T13:46:26.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.233+0000 7f8a5e4fb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a58103f10 0x7f8a5819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.233+0000 7f8a5e4fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a58108fe0 0x7f8a5819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.233+0000 7f8a5e4fb640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5819ee40 0x7f8a5819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.233+0000 7f8a5e4fb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58114060 con 0x7f8a58108fe0 2024-04-02T13:46:26.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.233+0000 7f8a5e4fb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58113ee0 con 0x7f8a58103f10 2024-04-02T13:46:26.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.233+0000 7f8a5e4fb640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581141e0 con 0x7f8a5819ee40 2024-04-02T13:46:26.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.233+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a58108fe0 0x7f8a5819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.233+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a58108fe0 0x7f8a5819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57936/0 (socket says 172.21.15.33:57936) 2024-04-02T13:46:26.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.233+0000 7f8a577fe640 1 -- 172.21.15.33:0/1997717067 learned_addr learned my addr 172.21.15.33:0/1997717067 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.233+0000 7f8a57fff640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a58103f10 0x7f8a5819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.234+0000 7f8a5ca98640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5819ee40 0x7f8a5819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.234+0000 7f8a577fe640 1 -- 172.21.15.33:0/1997717067 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5819ee40 msgr2=0x7f8a5819f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.234+0000 7f8a577fe640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5819ee40 0x7f8a5819f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.234+0000 7f8a577fe640 1 -- 172.21.15.33:0/1997717067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a58103f10 msgr2=0x7f8a5819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.234+0000 7f8a577fe640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a58103f10 0x7f8a5819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.234+0000 7f8a577fe640 1 -- 172.21.15.33:0/1997717067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5819f930 con 0x7f8a58108fe0 2024-04-02T13:46:26.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.234+0000 7f8a57fff640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a58103f10 0x7f8a5819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:26.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.234+0000 7f8a577fe640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a58108fe0 0x7f8a5819aac0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c000c00 tx=0x7f8a4c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.234+0000 7f8a557fa640 1 -- 172.21.15.33:0/1997717067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c043020 con 0x7f8a58108fe0 2024-04-02T13:46:26.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.234+0000 7f8a557fa640 1 -- 172.21.15.33:0/1997717067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8a4c02caa0 con 0x7f8a58108fe0 2024-04-02T13:46:26.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.234+0000 7f8a557fa640 1 -- 172.21.15.33:0/1997717067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c003c20 con 0x7f8a58108fe0 2024-04-02T13:46:26.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.234+0000 7f8a5e4fb640 1 -- 172.21.15.33:0/1997717067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5819fbc0 con 0x7f8a58108fe0 2024-04-02T13:46:26.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.234+0000 7f8a5e4fb640 1 -- 172.21.15.33:0/1997717067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a581a73e0 con 0x7f8a58108fe0 2024-04-02T13:46:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.237+0000 7f8a557fa640 1 -- 172.21.15.33:0/1997717067 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a4c046020 con 0x7f8a58108fe0 2024-04-02T13:46:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.237+0000 7f8a611bd740 1 Processor -- start 2024-04-02T13:46:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.237+0000 7f8a611bd740 1 -- start start 2024-04-02T13:46:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.237+0000 7f8a611bd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 0x5636e0c2acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.237+0000 7f8a611bd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 0x5636e0c33890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.237+0000 7f8a611bd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c33dd0 0x5636e0c361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.237+0000 7f8a611bd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636e0b8e830 con 0x5636e0c2b1f0 2024-04-02T13:46:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.237+0000 7f8a611bd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5636e0b36820 con 0x5636e0c2a8d0 2024-04-02T13:46:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.237+0000 7f8a611bd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5636e0b8a7e0 con 0x5636e0c33dd0 2024-04-02T13:46:26.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.237+0000 7f8a57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 0x5636e0c33890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.237+0000 7f8a57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 0x5636e0c33890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57946/0 (socket says 172.21.15.33:57946) 2024-04-02T13:46:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.237+0000 7f8a57fff640 1 -- 172.21.15.33:0/1610011768 learned_addr learned my addr 172.21.15.33:0/1610011768 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a5ca98640 1 --2- 172.21.15.33:0/1610011768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 0x5636e0c2acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a557fa640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8a400a3d20 0x7f8a400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a577fe640 1 --2- 172.21.15.33:0/1610011768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c33dd0 0x5636e0c361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a57fff640 1 -- 172.21.15.33:0/1610011768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c33dd0 msgr2=0x5636e0c361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a5ca98640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8a400a3d20 0x7f8a400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a57fff640 1 --2- 172.21.15.33:0/1610011768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c33dd0 0x5636e0c361c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a57fff640 1 -- 172.21.15.33:0/1610011768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 msgr2=0x5636e0c2acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a57fff640 1 --2- 172.21.15.33:0/1610011768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 0x5636e0c2acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a57fff640 1 -- 172.21.15.33:0/1610011768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e096c7d0 con 0x5636e0c2b1f0 2024-04-02T13:46:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a577fe640 1 --2- 172.21.15.33:0/1610011768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c33dd0 0x5636e0c361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a557fa640 1 -- 172.21.15.33:0/1997717067 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f8a4c0f27d0 con 0x7f8a58108fe0 2024-04-02T13:46:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a57fff640 1 --2- 172.21.15.33:0/1610011768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 0x5636e0c33890 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8a48008c60 tx=0x7f8a4800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98135438d9b1eb03 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a5ca98640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8a400a3d20 0x7f8a400a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8a500147f0 tx=0x7f8a50004ab0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a3e7fc640 1 -- 172.21.15.33:0/1610011768 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48014070 con 0x5636e0c2b1f0 2024-04-02T13:46:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a3e7fc640 1 -- 172.21.15.33:0/1610011768 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8a480047a0 con 0x5636e0c2b1f0 2024-04-02T13:46:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.238+0000 7f8a3e7fc640 1 -- 172.21.15.33:0/1610011768 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48010760 con 0x5636e0c2b1f0 2024-04-02T13:46:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.239+0000 7f8a611bd740 1 -- 172.21.15.33:0/1610011768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 msgr2=0x5636e0c33890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.239+0000 7f8a611bd740 1 --2- 172.21.15.33:0/1610011768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 0x5636e0c33890 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8a48008c60 tx=0x7f8a4800e720 comp rx=0 tx=0).stop 2024-04-02T13:46:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.239+0000 7f8a611bd740 1 -- 172.21.15.33:0/1610011768 shutdown_connections 2024-04-02T13:46:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.239+0000 7f8a611bd740 1 --2- 172.21.15.33:0/1610011768 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c33dd0 0x5636e0c361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.239+0000 7f8a611bd740 1 --2- 172.21.15.33:0/1610011768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 0x5636e0c33890 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.239+0000 7f8a611bd740 1 --2- 172.21.15.33:0/1610011768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 0x5636e0c2acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.239+0000 7f8a611bd740 1 -- 172.21.15.33:0/1610011768 >> 172.21.15.33:0/1610011768 conn(0x5636e0c25000 msgr2=0x5636e0c253e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.239+0000 7f8a611bd740 1 -- 172.21.15.33:0/1610011768 shutdown_connections 2024-04-02T13:46:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.239+0000 7f8a611bd740 1 -- 172.21.15.33:0/1610011768 wait complete. 2024-04-02T13:46:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.239+0000 7f8a611bd740 1 -- start start 2024-04-02T13:46:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a611bd740 1 Processor -- start 2024-04-02T13:46:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a611bd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 0x5636e0c335f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a611bd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 0x5636e0c30980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a611bd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c33dd0 0x5636e0c30ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a611bd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636e0c36bc0 con 0x5636e0c2b1f0 2024-04-02T13:46:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a611bd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5636e0b49d30 con 0x5636e0c2a8d0 2024-04-02T13:46:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a611bd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5636e0c36d40 con 0x5636e0c33dd0 2024-04-02T13:46:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a611bd740 10 client.0 _ll_register_callbacks cb 0x5636e0b49720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 0x5636e0c30980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 0x5636e0c30980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57950/0 (socket says 172.21.15.33:57950) 2024-04-02T13:46:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a577fe640 1 -- 172.21.15.33:0/3072231172 learned_addr learned my addr 172.21.15.33:0/3072231172 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a57fff640 1 --2- 172.21.15.33:0/3072231172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 0x5636e0c335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a5ca98640 1 --2- 172.21.15.33:0/3072231172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c33dd0 0x5636e0c30ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a577fe640 1 -- 172.21.15.33:0/3072231172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c33dd0 msgr2=0x5636e0c30ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a577fe640 1 --2- 172.21.15.33:0/3072231172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c33dd0 0x5636e0c30ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a577fe640 1 -- 172.21.15.33:0/3072231172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 msgr2=0x5636e0c335f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a577fe640 1 --2- 172.21.15.33:0/3072231172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 0x5636e0c335f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.240+0000 7f8a577fe640 1 -- 172.21.15.33:0/3072231172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e08a23f0 con 0x5636e0c2b1f0 2024-04-02T13:46:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a577fe640 1 --2- 172.21.15.33:0/3072231172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 0x5636e0c30980 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c004650 tx=0x7f8a4c08dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591a7f2fd0bfb5df server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a377fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a377fe640 1 -- 172.21.15.33:0/3072231172 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c0bdff0 con 0x5636e0c2b1f0 2024-04-02T13:46:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a377fe640 1 -- 172.21.15.33:0/3072231172 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f8a4c004c20 con 0x5636e0c2b1f0 2024-04-02T13:46:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a611bd740 1 -- 172.21.15.33:0/3072231172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a480047a0 con 0x5636e0c2b1f0 2024-04-02T13:46:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a377fe640 1 -- 172.21.15.33:0/3072231172 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8a4c0bde10 con 0x5636e0c2b1f0 2024-04-02T13:46:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a611bd740 10 client.17223 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a611bd740 1 -- 172.21.15.33:0/3072231172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5636e08620f0 con 0x5636e0c2b1f0 2024-04-02T13:46:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a377fe640 1 -- 172.21.15.33:0/3072231172 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8a480047a0 con 0x5636e0c2b1f0 2024-04-02T13:46:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a377fe640 1 -- 172.21.15.33:0/3072231172 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f8a4c0bab20 con 0x5636e0c2b1f0 2024-04-02T13:46:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a611bd740 10 client.17223 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a611bd740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a611bd740 20 client.17223 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a611bd740 1 --2- 172.21.15.33:0/3072231172 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5636e0c43b60 0x5636e0c45fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a611bd740 4 client.17223 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.241+0000 7f8a611bd740 1 -- 172.21.15.33:0/3072231172 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636e0b9bd70 con 0x5636e0c43b60 2024-04-02T13:46:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.242+0000 7f8a57fff640 1 --2- 172.21.15.33:0/3072231172 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5636e0c43b60 0x5636e0c45fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.242+0000 7f8a57fff640 1 --2- 172.21.15.33:0/3072231172 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5636e0c43b60 0x5636e0c45fd0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d8cb3844a8a7ad4 server_cookie=83ed8c59317e6f98 in_seq=0 out_seq=0 2024-04-02T13:46:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.242+0000 7f8a377fe640 10 client.17223 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a377fe640 1 -- 172.21.15.33:0/3072231172 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636e0b9bd70 con 0x5636e0c43b60 2024-04-02T13:46:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a377fe640 10 client.17223 handle_command_reply: tid=0 2024-04-02T13:46:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a611bd740 1 client.17223 shutdown 2024-04-02T13:46:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a611bd740 1 -- 172.21.15.33:0/3072231172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 msgr2=0x5636e0c30980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a611bd740 1 --2- 172.21.15.33:0/3072231172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 0x5636e0c30980 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c004650 tx=0x7f8a4c08dd90 comp rx=0 tx=0).stop 2024-04-02T13:46:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a611bd740 1 -- 172.21.15.33:0/3072231172 shutdown_connections 2024-04-02T13:46:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a611bd740 1 --2- 172.21.15.33:0/3072231172 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5636e0c43b60 0x5636e0c45fd0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a611bd740 1 --2- 172.21.15.33:0/3072231172 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c33dd0 0x5636e0c30ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a377fe640 0 client.17223 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a611bd740 1 --2- 172.21.15.33:0/3072231172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c2b1f0 0x5636e0c30980 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a611bd740 1 --2- 172.21.15.33:0/3072231172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 0x5636e0c335f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a611bd740 1 -- 172.21.15.33:0/3072231172 >> 172.21.15.33:0/3072231172 conn(0x5636e0c25000 msgr2=0x5636e0c26d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a611bd740 1 -- 172.21.15.33:0/3072231172 shutdown_connections 2024-04-02T13:46:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a611bd740 1 -- 172.21.15.33:0/3072231172 wait complete. 2024-04-02T13:46:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.243+0000 7f8a611bd740 20 client.17223 trim_cache size 0 max 16384 2024-04-02T13:46:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.261+0000 7f8a611bd740 1 Processor -- start 2024-04-02T13:46:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.261+0000 7f8a611bd740 1 -- start start 2024-04-02T13:46:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.261+0000 7f8a611bd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 0x5636e0c2fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.261+0000 7f8a611bd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c2b1f0 0x5636e0c240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.261+0000 7f8a611bd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 0x5636e0c330b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.261+0000 7f8a611bd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581b3af0 con 0x5636e0c33dd0 2024-04-02T13:46:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.261+0000 7f8a611bd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581b13c0 con 0x5636e0c2a8d0 2024-04-02T13:46:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.261+0000 7f8a611bd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581af230 con 0x5636e0c2b1f0 2024-04-02T13:46:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.261+0000 7f8a5ca98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 0x5636e0c330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.261+0000 7f8a5ca98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 0x5636e0c330b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57952/0 (socket says 172.21.15.33:57952) 2024-04-02T13:46:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.261+0000 7f8a5ca98640 1 -- 172.21.15.33:0/271985714 learned_addr learned my addr 172.21.15.33:0/271985714 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.261+0000 7f8a57fff640 1 --2- 172.21.15.33:0/271985714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 0x5636e0c2fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.261+0000 7f8a577fe640 1 --2- 172.21.15.33:0/271985714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c2b1f0 0x5636e0c240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.262+0000 7f8a5ca98640 1 -- 172.21.15.33:0/271985714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c2b1f0 msgr2=0x5636e0c240c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.262+0000 7f8a5ca98640 1 --2- 172.21.15.33:0/271985714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c2b1f0 0x5636e0c240c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.262+0000 7f8a5ca98640 1 -- 172.21.15.33:0/271985714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 msgr2=0x5636e0c2fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.262+0000 7f8a5ca98640 1 --2- 172.21.15.33:0/271985714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 0x5636e0c2fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.262+0000 7f8a5ca98640 1 -- 172.21.15.33:0/271985714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e0c24e30 con 0x5636e0c33dd0 2024-04-02T13:46:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.262+0000 7f8a5ca98640 1 --2- 172.21.15.33:0/271985714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 0x5636e0c330b0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000b8b0 tx=0x7f8a5000b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceea51adb78ac962 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.262+0000 7f8a3dffb640 1 -- 172.21.15.33:0/271985714 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50024030 con 0x5636e0c33dd0 2024-04-02T13:46:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.262+0000 7f8a3dffb640 1 -- 172.21.15.33:0/271985714 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8a50018040 con 0x5636e0c33dd0 2024-04-02T13:46:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.262+0000 7f8a3dffb640 1 -- 172.21.15.33:0/271985714 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a500058c0 con 0x5636e0c33dd0 2024-04-02T13:46:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.262+0000 7f8a611bd740 1 -- 172.21.15.33:0/271985714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 msgr2=0x5636e0c330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.262+0000 7f8a611bd740 1 --2- 172.21.15.33:0/271985714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 0x5636e0c330b0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000b8b0 tx=0x7f8a5000b640 comp rx=0 tx=0).stop 2024-04-02T13:46:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 -- 172.21.15.33:0/271985714 shutdown_connections 2024-04-02T13:46:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 --2- 172.21.15.33:0/271985714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 0x5636e0c330b0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 --2- 172.21.15.33:0/271985714 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c2b1f0 0x5636e0c240c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 --2- 172.21.15.33:0/271985714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2a8d0 0x5636e0c2fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 -- 172.21.15.33:0/271985714 >> 172.21.15.33:0/271985714 conn(0x5636e0c25000 msgr2=0x5636e0c239b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 -- 172.21.15.33:0/271985714 shutdown_connections 2024-04-02T13:46:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 -- 172.21.15.33:0/271985714 wait complete. 2024-04-02T13:46:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 -- start start 2024-04-02T13:46:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 Processor -- start 2024-04-02T13:46:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c2a8d0 0x5636e0c379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2b1f0 0x5636e0c37f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 0x5636e0c38460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581af880 con 0x5636e0c33dd0 2024-04-02T13:46:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581b0cf0 con 0x5636e0c2b1f0 2024-04-02T13:46:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581aff50 con 0x5636e0c2a8d0 2024-04-02T13:46:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a611bd740 10 client.0 _ll_register_callbacks cb 0x7f8a4c0053b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a5ca98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 0x5636e0c38460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a5ca98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 0x5636e0c38460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57968/0 (socket says 172.21.15.33:57968) 2024-04-02T13:46:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.263+0000 7f8a5ca98640 1 -- 172.21.15.33:0/1177262390 learned_addr learned my addr 172.21.15.33:0/1177262390 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a577fe640 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2b1f0 0x5636e0c37f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a57fff640 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c2a8d0 0x5636e0c379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a5ca98640 1 -- 172.21.15.33:0/1177262390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c2a8d0 msgr2=0x5636e0c379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a5ca98640 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c2a8d0 0x5636e0c379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a5ca98640 1 -- 172.21.15.33:0/1177262390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2b1f0 msgr2=0x5636e0c37f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a5ca98640 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2b1f0 0x5636e0c37f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a5ca98640 1 -- 172.21.15.33:0/1177262390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e0ca90f0 con 0x5636e0c33dd0 2024-04-02T13:46:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a57fff640 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c2a8d0 0x5636e0c379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a5ca98640 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 0x5636e0c38460 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000c010 tx=0x7f8a50005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1f2af708002721 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a377fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a377fe640 1 -- 172.21.15.33:0/1177262390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50024040 con 0x5636e0c33dd0 2024-04-02T13:46:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a377fe640 1 -- 172.21.15.33:0/1177262390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f8a50004d20 con 0x5636e0c33dd0 2024-04-02T13:46:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a611bd740 1 -- 172.21.15.33:0/1177262390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a50018040 con 0x5636e0c33dd0 2024-04-02T13:46:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a377fe640 1 -- 172.21.15.33:0/1177262390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8a5000bca0 con 0x5636e0c33dd0 2024-04-02T13:46:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a611bd740 10 client.17229 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a611bd740 1 -- 172.21.15.33:0/1177262390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5636e0caa3a0 con 0x5636e0c33dd0 2024-04-02T13:46:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.264+0000 7f8a377fe640 1 -- 172.21.15.33:0/1177262390 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8a50018040 con 0x5636e0c33dd0 2024-04-02T13:46:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.265+0000 7f8a377fe640 1 -- 172.21.15.33:0/1177262390 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f8a5001d050 con 0x5636e0c33dd0 2024-04-02T13:46:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.265+0000 7f8a611bd740 10 client.17229 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.265+0000 7f8a611bd740 10 client.17229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.265+0000 7f8a611bd740 20 client.17229 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.265+0000 7f8a611bd740 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5636e0caaa90 0x5636e0caceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.265+0000 7f8a611bd740 4 client.17229 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:46:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.265+0000 7f8a611bd740 1 -- 172.21.15.33:0/1177262390 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636e0b960c0 con 0x5636e0caaa90 2024-04-02T13:46:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.265+0000 7f8a57fff640 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5636e0caaa90 0x5636e0caceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.265+0000 7f8a377fe640 10 client.17229 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.265+0000 7f8a57fff640 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5636e0caaa90 0x5636e0caceb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b74b7ea69dbf02a server_cookie=d879196d8b07d0b0 in_seq=0 out_seq=0 2024-04-02T13:46:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a377fe640 1 -- 172.21.15.33:0/1177262390 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5636e0b960c0 con 0x5636e0caaa90 2024-04-02T13:46:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a377fe640 10 client.17229 handle_command_reply: tid=0 2024-04-02T13:46:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a611bd740 1 client.17229 shutdown 2024-04-02T13:46:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a611bd740 1 -- 172.21.15.33:0/1177262390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 msgr2=0x5636e0c38460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a611bd740 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 0x5636e0c38460 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000c010 tx=0x7f8a50005870 comp rx=0 tx=0).stop 2024-04-02T13:46:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a611bd740 1 -- 172.21.15.33:0/1177262390 shutdown_connections 2024-04-02T13:46:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a611bd740 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5636e0caaa90 0x5636e0caceb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a611bd740 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636e0c33dd0 0x5636e0c38460 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a611bd740 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636e0c2b1f0 0x5636e0c37f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a377fe640 0 client.17229 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a611bd740 1 --2- 172.21.15.33:0/1177262390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5636e0c2a8d0 0x5636e0c379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a611bd740 1 -- 172.21.15.33:0/1177262390 >> 172.21.15.33:0/1177262390 conn(0x5636e0c25000 msgr2=0x5636e0c24240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a611bd740 1 -- 172.21.15.33:0/1177262390 shutdown_connections 2024-04-02T13:46:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a611bd740 1 -- 172.21.15.33:0/1177262390 wait complete. 2024-04-02T13:46:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a611bd740 20 client.17229 trim_cache size 0 max 16384 2024-04-02T13:46:26.279 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:46:26.279 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:46:26.279 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:46:26.279 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:46:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a5e4fb640 1 -- 172.21.15.33:0/1997717067 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8a400a3d20 msgr2=0x7f8a400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a5e4fb640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8a400a3d20 0x7f8a400a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8a500147f0 tx=0x7f8a50004ab0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a5e4fb640 1 -- 172.21.15.33:0/1997717067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a58108fe0 msgr2=0x7f8a5819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.266+0000 7f8a5e4fb640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a58108fe0 0x7f8a5819aac0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c000c00 tx=0x7f8a4c0045e0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.267+0000 7f8a5e4fb640 1 -- 172.21.15.33:0/1997717067 shutdown_connections 2024-04-02T13:46:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.267+0000 7f8a5e4fb640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8a400a3d20 0x7f8a400a61e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.267+0000 7f8a5e4fb640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a5819ee40 0x7f8a5819f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.267+0000 7f8a5e4fb640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a58108fe0 0x7f8a5819aac0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.267+0000 7f8a5e4fb640 1 --2- 172.21.15.33:0/1997717067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a58103f10 0x7f8a5819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.267+0000 7f8a5e4fb640 1 -- 172.21.15.33:0/1997717067 >> 172.21.15.33:0/1997717067 conn(0x7f8a580ff7b0 msgr2=0x7f8a58101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.267+0000 7f8a5e4fb640 1 -- 172.21.15.33:0/1997717067 shutdown_connections 2024-04-02T13:46:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.267+0000 7f8a5e4fb640 1 -- 172.21.15.33:0/1997717067 wait complete. 2024-04-02T13:46:26.282 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:46:26.282 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:46:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.382+0000 7f4ef11f0640 1 -- 172.21.15.33:0/977179870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eec101aa0 msgr2=0x7f4eec101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.382+0000 7f4ef11f0640 1 --2- 172.21.15.33:0/977179870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eec101aa0 0x7f4eec101f20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009930 tx=0x7f4ed402bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.382+0000 7f4ef11f0640 1 -- 172.21.15.33:0/977179870 shutdown_connections 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.382+0000 7f4ef11f0640 1 --2- 172.21.15.33:0/977179870 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4eec102460 0x7f4eec10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.382+0000 7f4ef11f0640 1 --2- 172.21.15.33:0/977179870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eec101aa0 0x7f4eec101f20 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.382+0000 7f4ef11f0640 1 --2- 172.21.15.33:0/977179870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4eec107ab0 0x7f4eec107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.382+0000 7f4ef11f0640 1 -- 172.21.15.33:0/977179870 >> 172.21.15.33:0/977179870 conn(0x7f4eec0fd590 msgr2=0x7f4eec0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.382+0000 7f4ef11f0640 1 -- 172.21.15.33:0/977179870 shutdown_connections 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.382+0000 7f4ef11f0640 1 -- 172.21.15.33:0/977179870 wait complete. 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.383+0000 7f4ef11f0640 1 Processor -- start 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.383+0000 7f4ef11f0640 1 -- start start 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.383+0000 7f4ef11f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eec101aa0 0x7f4eec198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.383+0000 7f4ef11f0640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4eec102460 0x7f4eec1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.383+0000 7f4ef11f0640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4eec107ab0 0x7f4eec19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.383+0000 7f4ef11f0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec111e90 con 0x7f4eec101aa0 2024-04-02T13:46:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.383+0000 7f4ef11f0640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec111d10 con 0x7f4eec102460 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.383+0000 7f4ef11f0640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec112010 con 0x7f4eec107ab0 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4eead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eec101aa0 0x7f4eec198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4eead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eec101aa0 0x7f4eec198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57976/0 (socket says 172.21.15.33:57976) 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4eead9d640 1 -- 172.21.15.33:0/1565833804 learned_addr learned my addr 172.21.15.33:0/1565833804 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4eea59c640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4eec102460 0x7f4eec1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4eeb59e640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4eec107ab0 0x7f4eec19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4eead9d640 1 -- 172.21.15.33:0/1565833804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4eec107ab0 msgr2=0x7f4eec19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4eead9d640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4eec107ab0 0x7f4eec19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4eead9d640 1 -- 172.21.15.33:0/1565833804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4eec102460 msgr2=0x7f4eec1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4eead9d640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4eec102460 0x7f4eec1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4eead9d640 1 -- 172.21.15.33:0/1565833804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec19d380 con 0x7f4eec101aa0 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4eead9d640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eec101aa0 0x7f4eec198390 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400d950 tx=0x7f4ee400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4ee3fff640 1 -- 172.21.15.33:0/1565833804 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4017030 con 0x7f4eec101aa0 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4ef11f0640 1 -- 172.21.15.33:0/1565833804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eec19d670 con 0x7f4eec101aa0 2024-04-02T13:46:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4ee3fff640 1 -- 172.21.15.33:0/1565833804 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4ee4004020 con 0x7f4eec101aa0 2024-04-02T13:46:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4ee3fff640 1 -- 172.21.15.33:0/1565833804 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40047a0 con 0x7f4eec101aa0 2024-04-02T13:46:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.384+0000 7f4ef11f0640 1 -- 172.21.15.33:0/1565833804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eec1a4f00 con 0x7f4eec101aa0 2024-04-02T13:46:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.385+0000 7f4ef11f0640 1 -- 172.21.15.33:0/1565833804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eec0688f0 con 0x7f4eec101aa0 2024-04-02T13:46:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.386+0000 7f4ee3fff640 1 -- 172.21.15.33:0/1565833804 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ee40041d0 con 0x7f4eec101aa0 2024-04-02T13:46:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.387+0000 7f4ee3fff640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4ed80a3d20 0x7f4ed80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.387+0000 7f4eea59c640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4ed80a3d20 0x7f4ed80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.387+0000 7f4ee3fff640 1 -- 172.21.15.33:0/1565833804 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f4ee40d0600 con 0x7f4eec101aa0 2024-04-02T13:46:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.388+0000 7f4eea59c640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4ed80a3d20 0x7f4ed80a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4000c00 tx=0x7f4ed4005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.392+0000 7f4ee3fff640 1 -- 172.21.15.33:0/1565833804 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4ee409c870 con 0x7f4eec101aa0 2024-04-02T13:46:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.619+0000 7f4ef11f0640 1 -- 172.21.15.33:0/1565833804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4eec02d370 con 0x7f4eec101aa0 2024-04-02T13:46:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.619+0000 7f4ee3fff640 1 -- 172.21.15.33:0/1565833804 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11873 (secure 0 0 0) 0x7f4ee409c170 con 0x7f4eec101aa0 2024-04-02T13:46:26.622 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:46:26.622 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:44:55.702357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:46:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 36 2024-04-02T13:46:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.621+0000 7f4ef11f0640 1 -- 172.21.15.33:0/1565833804 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4ed80a3d20 msgr2=0x7f4ed80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.621+0000 7f4ef11f0640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4ed80a3d20 0x7f4ed80a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4000c00 tx=0x7f4ed4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.621+0000 7f4ef11f0640 1 -- 172.21.15.33:0/1565833804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eec101aa0 msgr2=0x7f4eec198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.621+0000 7f4ef11f0640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eec101aa0 0x7f4eec198390 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400d950 tx=0x7f4ee400de10 comp rx=0 tx=0).stop 2024-04-02T13:46:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.621+0000 7f4ef11f0640 1 -- 172.21.15.33:0/1565833804 shutdown_connections 2024-04-02T13:46:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.621+0000 7f4ef11f0640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4ed80a3d20 0x7f4ed80a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.621+0000 7f4ef11f0640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4eec107ab0 0x7f4eec19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.621+0000 7f4ef11f0640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4eec102460 0x7f4eec1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.621+0000 7f4ef11f0640 1 --2- 172.21.15.33:0/1565833804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eec101aa0 0x7f4eec198390 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.621+0000 7f4ef11f0640 1 -- 172.21.15.33:0/1565833804 >> 172.21.15.33:0/1565833804 conn(0x7f4eec0fd590 msgr2=0x7f4eec0fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.621+0000 7f4ef11f0640 1 -- 172.21.15.33:0/1565833804 shutdown_connections 2024-04-02T13:46:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.621+0000 7f4ef11f0640 1 -- 172.21.15.33:0/1565833804 wait complete. 2024-04-02T13:46:26.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:46:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.739+0000 7fde10849640 1 -- 172.21.15.33:0/2265013028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c101370 msgr2=0x7fde0c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.739+0000 7fde10849640 1 --2- 172.21.15.33:0/2265013028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c101370 0x7fde0c10d880 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009cd0 tx=0x7fddfc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.739+0000 7fde10849640 1 -- 172.21.15.33:0/2265013028 shutdown_connections 2024-04-02T13:46:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.739+0000 7fde10849640 1 --2- 172.21.15.33:0/2265013028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c101370 0x7fde0c10d880 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.739+0000 7fde10849640 1 --2- 172.21.15.33:0/2265013028 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c1009b0 0x7fde0c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.739+0000 7fde10849640 1 --2- 172.21.15.33:0/2265013028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c1069c0 0x7fde0c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.739+0000 7fde10849640 1 -- 172.21.15.33:0/2265013028 >> 172.21.15.33:0/2265013028 conn(0x7fde0c076d80 msgr2=0x7fde0c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.740+0000 7fde10849640 1 -- 172.21.15.33:0/2265013028 shutdown_connections 2024-04-02T13:46:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.740+0000 7fde10849640 1 -- 172.21.15.33:0/2265013028 wait complete. 2024-04-02T13:46:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.740+0000 7fde10849640 1 Processor -- start 2024-04-02T13:46:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.740+0000 7fde10849640 1 -- start start 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.740+0000 7fde10849640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c1009b0 0x7fde0c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.740+0000 7fde10849640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c101370 0x7fde0c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.740+0000 7fde10849640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c1069c0 0x7fde0c19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.740+0000 7fde10849640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c110da0 con 0x7fde0c1009b0 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.740+0000 7fde10849640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c110c20 con 0x7fde0c101370 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.740+0000 7fde10849640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c110f20 con 0x7fde0c1069c0 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fde0a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c1009b0 0x7fde0c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fde0a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c1009b0 0x7fde0c197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57996/0 (socket says 172.21.15.33:57996) 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fde0a59c640 1 -- 172.21.15.33:0/1140468118 learned_addr learned my addr 172.21.15.33:0/1140468118 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fde09d9b640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c101370 0x7fde0c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fde0ad9d640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c1069c0 0x7fde0c19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fde0a59c640 1 -- 172.21.15.33:0/1140468118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c1069c0 msgr2=0x7fde0c19bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fde0a59c640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c1069c0 0x7fde0c19bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fde0a59c640 1 -- 172.21.15.33:0/1140468118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c101370 msgr2=0x7fde0c197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fde0a59c640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c101370 0x7fde0c197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fde0a59c640 1 -- 172.21.15.33:0/1140468118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c19c230 con 0x7fde0c1009b0 2024-04-02T13:46:26.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fde0a59c640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c1009b0 0x7fde0c197240 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fde0400d950 tx=0x7fde0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fddfb7fe640 1 -- 172.21.15.33:0/1140468118 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04017070 con 0x7fde0c1009b0 2024-04-02T13:46:26.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fde10849640 1 -- 172.21.15.33:0/1140468118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0c19c520 con 0x7fde0c1009b0 2024-04-02T13:46:26.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fddfb7fe640 1 -- 172.21.15.33:0/1140468118 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde04009c30 con 0x7fde0c1009b0 2024-04-02T13:46:26.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fddfb7fe640 1 -- 172.21.15.33:0/1140468118 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0400f6b0 con 0x7fde0c1009b0 2024-04-02T13:46:26.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.741+0000 7fde10849640 1 -- 172.21.15.33:0/1140468118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0c1a3db0 con 0x7fde0c1009b0 2024-04-02T13:46:26.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.744+0000 7fddfb7fe640 1 -- 172.21.15.33:0/1140468118 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fde04010a50 con 0x7fde0c1009b0 2024-04-02T13:46:26.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.744+0000 7fde1342a740 1 Processor -- start 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.744+0000 7fde1342a740 1 -- start start 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.744+0000 7fde1342a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95180 0x557c57b95560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.744+0000 7fde1342a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95b30 0x557c57ba0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.744+0000 7fde1342a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57ba0eb0 0x557c57ba32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.744+0000 7fde1342a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557c57af97b0 con 0x557c57b95180 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.744+0000 7fde1342a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557c57a98570 con 0x557c57b95b30 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.744+0000 7fde1342a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557c57af5760 con 0x557c57ba0eb0 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.744+0000 7fde0ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95180 0x557c57b95560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.744+0000 7fde0ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95180 0x557c57b95560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58008/0 (socket says 172.21.15.33:58008) 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.744+0000 7fde0ad9d640 1 -- 172.21.15.33:0/1007765160 learned_addr learned my addr 172.21.15.33:0/1007765160 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.744+0000 7fddfb7fe640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fddf40a3d20 0x7fddf40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fde09d9b640 1 --2- 172.21.15.33:0/1007765160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57ba0eb0 0x557c57ba32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fde0a59c640 1 --2- 172.21.15.33:0/1007765160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95b30 0x557c57ba0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fde0ad9d640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fddf40a3d20 0x7fddf40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fddfb7fe640 1 -- 172.21.15.33:0/1140468118 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fde040d0760 con 0x7fde0c1009b0 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fde0ad9d640 1 -- 172.21.15.33:0/1007765160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57ba0eb0 msgr2=0x557c57ba32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fde0ad9d640 1 --2- 172.21.15.33:0/1007765160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57ba0eb0 0x557c57ba32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fde0ad9d640 1 -- 172.21.15.33:0/1007765160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95b30 msgr2=0x557c57ba0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fde0ad9d640 1 --2- 172.21.15.33:0/1007765160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95b30 0x557c57ba0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fde0ad9d640 1 -- 172.21.15.33:0/1007765160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c5780d3f0 con 0x557c57b95180 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fde09d9b640 1 --2- 172.21.15.33:0/1007765160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57ba0eb0 0x557c57ba32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fde0ad9d640 1 --2- 172.21.15.33:0/1007765160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95180 0x557c57b95560 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fddfc005e30 tx=0x7fddfc032e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8860eb4104689b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fddf8ff9640 1 -- 172.21.15.33:0/1007765160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc039040 con 0x557c57b95180 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fddf8ff9640 1 -- 172.21.15.33:0/1007765160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fddfc032830 con 0x557c57b95180 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fddf8ff9640 1 -- 172.21.15.33:0/1007765160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc034640 con 0x557c57b95180 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.745+0000 7fde0ad9d640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fddf40a3d20 0x7fddf40a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fddfc005fa0 tx=0x7fddfc04b000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.746+0000 7fde1342a740 1 -- 172.21.15.33:0/1007765160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95180 msgr2=0x557c57b95560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.746+0000 7fde1342a740 1 --2- 172.21.15.33:0/1007765160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95180 0x557c57b95560 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fddfc005e30 tx=0x7fddfc032e80 comp rx=0 tx=0).stop 2024-04-02T13:46:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.746+0000 7fde1342a740 1 -- 172.21.15.33:0/1007765160 shutdown_connections 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.746+0000 7fde1342a740 1 --2- 172.21.15.33:0/1007765160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57ba0eb0 0x557c57ba32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.746+0000 7fde1342a740 1 --2- 172.21.15.33:0/1007765160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95b30 0x557c57ba0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.746+0000 7fde1342a740 1 --2- 172.21.15.33:0/1007765160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95180 0x557c57b95560 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.746+0000 7fde1342a740 1 -- 172.21.15.33:0/1007765160 >> 172.21.15.33:0/1007765160 conn(0x557c57b8fed0 msgr2=0x557c57b902b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.746+0000 7fde1342a740 1 -- 172.21.15.33:0/1007765160 shutdown_connections 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.746+0000 7fde1342a740 1 -- 172.21.15.33:0/1007765160 wait complete. 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.746+0000 7fde1342a740 1 -- start start 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.746+0000 7fde1342a740 1 Processor -- start 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde1342a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95180 0x557c57ba0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde1342a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 0x557c57b9dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde1342a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57ba0eb0 0x557c57b9e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde1342a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557c57ba3c70 con 0x557c57b95b30 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde1342a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557c57ab4e80 con 0x557c57b95180 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde1342a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x557c57ba3df0 con 0x557c57ba0eb0 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde1342a740 10 client.0 _ll_register_callbacks cb 0x557c57af0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde09d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 0x557c57b9dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde09d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 0x557c57b9dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58016/0 (socket says 172.21.15.33:58016) 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde09d9b640 1 -- 172.21.15.33:0/1266148878 learned_addr learned my addr 172.21.15.33:0/1266148878 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde0a59c640 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95180 0x557c57ba0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde0ad9d640 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57ba0eb0 0x557c57b9e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde09d9b640 1 -- 172.21.15.33:0/1266148878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57ba0eb0 msgr2=0x557c57b9e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde09d9b640 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57ba0eb0 0x557c57b9e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde09d9b640 1 -- 172.21.15.33:0/1266148878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95180 msgr2=0x557c57ba0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde09d9b640 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95180 0x557c57ba0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde09d9b640 1 -- 172.21.15.33:0/1266148878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c578d77d0 con 0x557c57b95b30 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde0ad9d640 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57ba0eb0 0x557c57b9e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde09d9b640 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 0x557c57b9dc90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fde00012700 tx=0x7fde00012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d98b932b4fe3d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fddf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fddf1ffb640 1 -- 172.21.15.33:0/1266148878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00004050 con 0x557c57b95b30 2024-04-02T13:46:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fddf1ffb640 1 -- 172.21.15.33:0/1266148878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fde000041f0 con 0x557c57b95b30 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde1342a740 1 -- 172.21.15.33:0/1266148878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddfc032830 con 0x557c57b95b30 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fddf1ffb640 1 -- 172.21.15.33:0/1266148878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fde00025050 con 0x557c57b95b30 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde1342a740 10 client.17247 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.747+0000 7fde1342a740 1 -- 172.21.15.33:0/1266148878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557c57bafe10 con 0x557c57b95b30 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.748+0000 7fddf1ffb640 1 -- 172.21.15.33:0/1266148878 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fddfc032830 con 0x557c57b95b30 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.748+0000 7fddf1ffb640 1 -- 172.21.15.33:0/1266148878 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fde00003820 con 0x557c57b95b30 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.748+0000 7fde1342a740 10 client.17247 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.748+0000 7fde1342a740 10 client.17247 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.748+0000 7fde1342a740 20 client.17247 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.748+0000 7fde1342a740 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x557c57bb1320 0x557c57bb3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.748+0000 7fde1342a740 4 client.17247 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.748+0000 7fde1342a740 1 -- 172.21.15.33:0/1266148878 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c57b06cf0 con 0x557c57bb1320 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.748+0000 7fde0a59c640 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x557c57bb1320 0x557c57bb3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fde0a59c640 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x557c57bb1320 0x557c57bb3730 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39d83a4d5e574189 server_cookie=bfe04b183d461097 in_seq=0 out_seq=0 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fddf1ffb640 10 client.17247 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:46:26.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fddf1ffb640 1 -- 172.21.15.33:0/1266148878 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c57b06cf0 con 0x557c57bb1320 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fddf1ffb640 10 client.17247 handle_command_reply: tid=0 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fde1342a740 1 client.17247 shutdown 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fde1342a740 1 -- 172.21.15.33:0/1266148878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 msgr2=0x557c57b9dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fde1342a740 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 0x557c57b9dc90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fde00012700 tx=0x7fde00012b00 comp rx=0 tx=0).stop 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fde1342a740 1 -- 172.21.15.33:0/1266148878 shutdown_connections 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fde1342a740 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x557c57bb1320 0x557c57bb3730 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fde1342a740 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57ba0eb0 0x557c57b9e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fde1342a740 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 0x557c57b9dc90 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fddf1ffb640 0 client.17247 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fde1342a740 1 --2- 172.21.15.33:0/1266148878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95180 0x557c57ba0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.749+0000 7fde1342a740 1 -- 172.21.15.33:0/1266148878 >> 172.21.15.33:0/1266148878 conn(0x557c57b8fed0 msgr2=0x557c57b91c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.750+0000 7fde1342a740 1 -- 172.21.15.33:0/1266148878 shutdown_connections 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.750+0000 7fde1342a740 1 -- 172.21.15.33:0/1266148878 wait complete. 2024-04-02T13:46:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.750+0000 7fde1342a740 20 client.17247 trim_cache size 0 max 16384 2024-04-02T13:46:26.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde1342a740 1 Processor -- start 2024-04-02T13:46:26.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde1342a740 1 -- start start 2024-04-02T13:46:26.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde1342a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57b95180 0x557c57bb3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde1342a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 0x557c57bacbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde1342a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57ba0eb0 0x557c57bb9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde1342a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1af710 con 0x557c57b95b30 2024-04-02T13:46:26.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde1342a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1acfe0 con 0x557c57ba0eb0 2024-04-02T13:46:26.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde1342a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1aae50 con 0x557c57b95180 2024-04-02T13:46:26.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde09d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 0x557c57bacbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde09d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 0x557c57bacbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58022/0 (socket says 172.21.15.33:58022) 2024-04-02T13:46:26.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde09d9b640 1 -- 172.21.15.33:0/3476416465 learned_addr learned my addr 172.21.15.33:0/3476416465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde0ad9d640 1 --2- 172.21.15.33:0/3476416465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57ba0eb0 0x557c57bb9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde0a59c640 1 --2- 172.21.15.33:0/3476416465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57b95180 0x557c57bb3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde09d9b640 1 -- 172.21.15.33:0/3476416465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57b95180 msgr2=0x557c57bb3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde09d9b640 1 --2- 172.21.15.33:0/3476416465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57b95180 0x557c57bb3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde09d9b640 1 -- 172.21.15.33:0/3476416465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57ba0eb0 msgr2=0x557c57bb9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde09d9b640 1 --2- 172.21.15.33:0/3476416465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57ba0eb0 0x557c57bb9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde09d9b640 1 -- 172.21.15.33:0/3476416465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c57b95890 con 0x557c57b95b30 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.763+0000 7fde09d9b640 1 --2- 172.21.15.33:0/3476416465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 0x557c57bacbd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fde00003c20 tx=0x7fde00009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101830cb87f68f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.764+0000 7fddf3fff640 1 -- 172.21.15.33:0/3476416465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0002e070 con 0x557c57b95b30 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.764+0000 7fddf3fff640 1 -- 172.21.15.33:0/3476416465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde000050c0 con 0x557c57b95b30 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.764+0000 7fddf3fff640 1 -- 172.21.15.33:0/3476416465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0000fbc0 con 0x557c57b95b30 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.764+0000 7fde1342a740 1 -- 172.21.15.33:0/3476416465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 msgr2=0x557c57bacbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.764+0000 7fde1342a740 1 --2- 172.21.15.33:0/3476416465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 0x557c57bacbd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fde00003c20 tx=0x7fde00009e90 comp rx=0 tx=0).stop 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.764+0000 7fde1342a740 1 -- 172.21.15.33:0/3476416465 shutdown_connections 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.764+0000 7fde1342a740 1 --2- 172.21.15.33:0/3476416465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57ba0eb0 0x557c57bb9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.764+0000 7fde1342a740 1 --2- 172.21.15.33:0/3476416465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57b95b30 0x557c57bacbd0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.764+0000 7fde1342a740 1 --2- 172.21.15.33:0/3476416465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57b95180 0x557c57bb3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.764+0000 7fde1342a740 1 -- 172.21.15.33:0/3476416465 >> 172.21.15.33:0/3476416465 conn(0x557c57b8fed0 msgr2=0x557c57b9c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.764+0000 7fde1342a740 1 -- 172.21.15.33:0/3476416465 shutdown_connections 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.764+0000 7fde1342a740 1 -- 172.21.15.33:0/3476416465 wait complete. 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.764+0000 7fde1342a740 1 -- start start 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde1342a740 1 Processor -- start 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde1342a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57b95180 0x557c57ba9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde1342a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95b30 0x557c57ba1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde1342a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57ba0eb0 0x557c57ba2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde1342a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1ab4a0 con 0x557c57ba0eb0 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde1342a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1ac910 con 0x557c57b95b30 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde1342a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1abb70 con 0x557c57b95180 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde1342a740 10 client.0 _ll_register_callbacks cb 0x7fde00009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde0ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57ba0eb0 0x557c57ba2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde0ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57ba0eb0 0x557c57ba2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58036/0 (socket says 172.21.15.33:58036) 2024-04-02T13:46:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde0ad9d640 1 -- 172.21.15.33:0/1212018799 learned_addr learned my addr 172.21.15.33:0/1212018799 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde09d9b640 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95b30 0x557c57ba1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde0a59c640 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57b95180 0x557c57ba9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde0ad9d640 1 -- 172.21.15.33:0/1212018799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57b95180 msgr2=0x557c57ba9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde0ad9d640 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57b95180 0x557c57ba9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde0ad9d640 1 -- 172.21.15.33:0/1212018799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95b30 msgr2=0x557c57ba1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde0ad9d640 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95b30 0x557c57ba1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde0ad9d640 1 -- 172.21.15.33:0/1212018799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c57ba2890 con 0x557c57ba0eb0 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde09d9b640 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95b30 0x557c57ba1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde0a59c640 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57b95180 0x557c57ba9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fde0ad9d640 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57ba0eb0 0x557c57ba2010 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fddfc02c130 tx=0x7fddfc009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3387a39ecdf0611e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fddf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.765+0000 7fddf1ffb640 1 -- 172.21.15.33:0/1212018799 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc05a030 con 0x557c57ba0eb0 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fddf1ffb640 1 -- 172.21.15.33:0/1212018799 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fddfc003e00 con 0x557c57ba0eb0 2024-04-02T13:46:26.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fde1342a740 1 -- 172.21.15.33:0/1212018799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde000050c0 con 0x557c57ba0eb0 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fddf1ffb640 1 -- 172.21.15.33:0/1212018799 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fddfc067050 con 0x557c57ba0eb0 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fde1342a740 10 client.17253 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fde1342a740 1 -- 172.21.15.33:0/1212018799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557c57c17970 con 0x557c57ba0eb0 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fddf1ffb640 1 -- 172.21.15.33:0/1212018799 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde000050c0 con 0x557c57ba0eb0 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fddf1ffb640 1 -- 172.21.15.33:0/1212018799 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fddfc0042d0 con 0x557c57ba0eb0 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fde1342a740 10 client.17253 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fde1342a740 10 client.17253 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fde1342a740 20 client.17253 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fde1342a740 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x557c57c17cd0 0x557c57c1a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fde1342a740 4 client.17253 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fde1342a740 1 -- 172.21.15.33:0/1212018799 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c57b01040 con 0x557c57c17cd0 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.766+0000 7fde0a59c640 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x557c57c17cd0 0x557c57c1a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.767+0000 7fddf1ffb640 10 client.17253 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.767+0000 7fde0a59c640 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x557c57c17cd0 0x557c57c1a0c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9948d7811557b710 server_cookie=89f1df40a52ae32f in_seq=0 out_seq=0 2024-04-02T13:46:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.767+0000 7fddf1ffb640 1 -- 172.21.15.33:0/1212018799 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c57b01040 con 0x557c57c17cd0 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.767+0000 7fddf1ffb640 10 client.17253 handle_command_reply: tid=0 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.767+0000 7fde1342a740 1 client.17253 shutdown 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.767+0000 7fde1342a740 1 -- 172.21.15.33:0/1212018799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57ba0eb0 msgr2=0x557c57ba2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.767+0000 7fde1342a740 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57ba0eb0 0x557c57ba2010 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fddfc02c130 tx=0x7fddfc009540 comp rx=0 tx=0).stop 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.767+0000 7fde1342a740 1 -- 172.21.15.33:0/1212018799 shutdown_connections 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.767+0000 7fde1342a740 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x557c57c17cd0 0x557c57c1a0c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde1342a740 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c57ba0eb0 0x557c57ba2010 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde1342a740 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557c57b95b30 0x557c57ba1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fddf1ffb640 0 client.17253 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde1342a740 1 --2- 172.21.15.33:0/1212018799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557c57b95180 0x557c57ba9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde1342a740 1 -- 172.21.15.33:0/1212018799 >> 172.21.15.33:0/1212018799 conn(0x557c57b8fed0 msgr2=0x557c57b8e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde1342a740 1 -- 172.21.15.33:0/1212018799 shutdown_connections 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde1342a740 1 -- 172.21.15.33:0/1212018799 wait complete. 2024-04-02T13:46:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde1342a740 20 client.17253 trim_cache size 0 max 16384 2024-04-02T13:46:26.777 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:46:26.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde10849640 1 -- 172.21.15.33:0/1140468118 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fddf40a3d20 msgr2=0x7fddf40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde10849640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fddf40a3d20 0x7fddf40a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fddfc005fa0 tx=0x7fddfc04b000 comp rx=0 tx=0).stop 2024-04-02T13:46:26.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde10849640 1 -- 172.21.15.33:0/1140468118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c1009b0 msgr2=0x7fde0c197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde10849640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c1009b0 0x7fde0c197240 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fde0400d950 tx=0x7fde0400de10 comp rx=0 tx=0).stop 2024-04-02T13:46:26.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde10849640 1 -- 172.21.15.33:0/1140468118 shutdown_connections 2024-04-02T13:46:26.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde10849640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fddf40a3d20 0x7fddf40a61e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde10849640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c1069c0 0x7fde0c19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde10849640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde0c101370 0x7fde0c197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde10849640 1 --2- 172.21.15.33:0/1140468118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde0c1009b0 0x7fde0c197240 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde10849640 1 -- 172.21.15.33:0/1140468118 >> 172.21.15.33:0/1140468118 conn(0x7fde0c076d80 msgr2=0x7fde0c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde10849640 1 -- 172.21.15.33:0/1140468118 shutdown_connections 2024-04-02T13:46:26.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.768+0000 7fde10849640 1 -- 172.21.15.33:0/1140468118 wait complete. 2024-04-02T13:46:26.781 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:46:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.884+0000 7f7dd6796640 1 -- 172.21.15.33:0/219080601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dd0101af0 msgr2=0x7f7dd010d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.884+0000 7f7dd6796640 1 --2- 172.21.15.33:0/219080601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dd0101af0 0x7f7dd010d820 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009ca0 tx=0x7f7dc802c140 comp rx=0 tx=0).stop 2024-04-02T13:46:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.884+0000 7f7dd6796640 1 -- 172.21.15.33:0/219080601 shutdown_connections 2024-04-02T13:46:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.884+0000 7f7dd6796640 1 --2- 172.21.15.33:0/219080601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dd0101af0 0x7f7dd010d820 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.884+0000 7f7dd6796640 1 --2- 172.21.15.33:0/219080601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dd0101150 0x7f7dd01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.884+0000 7f7dd6796640 1 --2- 172.21.15.33:0/219080601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd01059b0 0x7f7dd0105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.884+0000 7f7dd6796640 1 -- 172.21.15.33:0/219080601 >> 172.21.15.33:0/219080601 conn(0x7f7dd0076fc0 msgr2=0x7f7dd00fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.884+0000 7f7dd6796640 1 -- 172.21.15.33:0/219080601 shutdown_connections 2024-04-02T13:46:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.884+0000 7f7dd6796640 1 -- 172.21.15.33:0/219080601 wait complete. 2024-04-02T13:46:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.884+0000 7f7dd6796640 1 Processor -- start 2024-04-02T13:46:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.884+0000 7f7dd6796640 1 -- start start 2024-04-02T13:46:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dd6796640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dd0101150 0x7f7dd0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dd6796640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd0101af0 0x7f7dd01997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dd6796640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dd01059b0 0x7f7dd01a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dd6796640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd0110e80 con 0x7f7dd0101150 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dd6796640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd0110d00 con 0x7f7dd0101af0 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dd6796640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd0111000 con 0x7f7dd01059b0 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dcffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dd0101150 0x7f7dd0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dcffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dd0101150 0x7f7dd0199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58046/0 (socket says 172.21.15.33:58046) 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dcffff640 1 -- 172.21.15.33:0/2850268957 learned_addr learned my addr 172.21.15.33:0/2850268957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dd4d33640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dd01059b0 0x7f7dd01a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dcf7fe640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd0101af0 0x7f7dd01997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dcffff640 1 -- 172.21.15.33:0/2850268957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dd01059b0 msgr2=0x7f7dd01a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dcffff640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dd01059b0 0x7f7dd01a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dcffff640 1 -- 172.21.15.33:0/2850268957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd0101af0 msgr2=0x7f7dd01997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dcffff640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd0101af0 0x7f7dd01997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dcffff640 1 -- 172.21.15.33:0/2850268957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd01a4840 con 0x7f7dd0101150 2024-04-02T13:46:26.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dcf7fe640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd0101af0 0x7f7dd01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:26.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dcffff640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dd0101150 0x7f7dd0199280 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000e9f0 tx=0x7f7dc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dcd7fa640 1 -- 172.21.15.33:0/2850268957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc000cd80 con 0x7f7dd0101150 2024-04-02T13:46:26.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.885+0000 7f7dcd7fa640 1 -- 172.21.15.33:0/2850268957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7dc0010900 con 0x7f7dd0101150 2024-04-02T13:46:26.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.886+0000 7f7dd6796640 1 -- 172.21.15.33:0/2850268957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd01a4a10 con 0x7f7dd0101150 2024-04-02T13:46:26.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.886+0000 7f7dcd7fa640 1 -- 172.21.15.33:0/2850268957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00189e0 con 0x7f7dd0101150 2024-04-02T13:46:26.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.886+0000 7f7dd6796640 1 -- 172.21.15.33:0/2850268957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd01a4f30 con 0x7f7dd0101150 2024-04-02T13:46:26.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.888+0000 7f7dcd7fa640 1 -- 172.21.15.33:0/2850268957 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7dc0010ab0 con 0x7f7dd0101150 2024-04-02T13:46:26.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.888+0000 7f7dd937c740 1 Processor -- start 2024-04-02T13:46:26.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.888+0000 7f7dd937c740 1 -- start start 2024-04-02T13:46:26.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.888+0000 7f7dd937c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d47a0 0x563af94d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.888+0000 7f7dd937c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d50c0 0x563af94dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.888+0000 7f7dd937c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 0x563af94e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.888+0000 7f7dd937c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563af94387b0 con 0x563af94d47a0 2024-04-02T13:46:26.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.888+0000 7f7dd937c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563af93d7570 con 0x563af94d50c0 2024-04-02T13:46:26.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.888+0000 7f7dd937c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563af9434760 con 0x563af94ddca0 2024-04-02T13:46:26.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.888+0000 7f7dcd7fa640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7db80a3d20 0x7f7db80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.888+0000 7f7dd4d33640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d47a0 0x563af94d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.888+0000 7f7dd4d33640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d47a0 0x563af94d4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58052/0 (socket says 172.21.15.33:58052) 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.888+0000 7f7dd4d33640 1 -- 172.21.15.33:0/2031939550 learned_addr learned my addr 172.21.15.33:0/2031939550 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7dcffff640 1 --2- 172.21.15.33:0/2031939550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d50c0 0x563af94dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7dcf7fe640 1 --2- 172.21.15.33:0/2031939550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 0x563af94e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7dd4d33640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7db80a3d20 0x7f7db80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7dcd7fa640 1 -- 172.21.15.33:0/2850268957 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f7dc00d0f00 con 0x7f7dd0101150 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7dd4d33640 1 -- 172.21.15.33:0/2031939550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 msgr2=0x563af94e0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7dd4d33640 1 --2- 172.21.15.33:0/2031939550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 0x563af94e0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7dd4d33640 1 -- 172.21.15.33:0/2031939550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d50c0 msgr2=0x563af94dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7dd4d33640 1 --2- 172.21.15.33:0/2031939550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d50c0 0x563af94dd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7dd4d33640 1 -- 172.21.15.33:0/2031939550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af914c3f0 con 0x563af94d47a0 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7dcffff640 1 --2- 172.21.15.33:0/2031939550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d50c0 0x563af94dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7dd4d33640 1 --2- 172.21.15.33:0/2031939550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d47a0 0x563af94d4b80 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009430 tx=0x7f7dc8032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda7d8b3d66c2379 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7db67fc640 1 -- 172.21.15.33:0/2031939550 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8039040 con 0x563af94d47a0 2024-04-02T13:46:26.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7db67fc640 1 -- 172.21.15.33:0/2031939550 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7dc803a040 con 0x563af94d47a0 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.889+0000 7f7db67fc640 1 -- 172.21.15.33:0/2031939550 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc80047a0 con 0x563af94d47a0 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.890+0000 7f7dd937c740 1 -- 172.21.15.33:0/2031939550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d47a0 msgr2=0x563af94d4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.890+0000 7f7dd4d33640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7db80a3d20 0x7f7db80a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7dc804e000 tx=0x7f7dc804c000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.890+0000 7f7dd937c740 1 --2- 172.21.15.33:0/2031939550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d47a0 0x563af94d4b80 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009430 tx=0x7f7dc8032eb0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.890+0000 7f7dd937c740 1 -- 172.21.15.33:0/2031939550 shutdown_connections 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.890+0000 7f7dd937c740 1 --2- 172.21.15.33:0/2031939550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 0x563af94e0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.890+0000 7f7dd937c740 1 --2- 172.21.15.33:0/2031939550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d50c0 0x563af94dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.890+0000 7f7dd937c740 1 --2- 172.21.15.33:0/2031939550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d47a0 0x563af94d4b80 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.890+0000 7f7dd937c740 1 -- 172.21.15.33:0/2031939550 >> 172.21.15.33:0/2031939550 conn(0x563af94ceed0 msgr2=0x563af94cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.890+0000 7f7dd937c740 1 -- 172.21.15.33:0/2031939550 shutdown_connections 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.890+0000 7f7dd937c740 1 -- 172.21.15.33:0/2031939550 wait complete. 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.890+0000 7f7dd937c740 1 -- start start 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.890+0000 7f7dd937c740 1 Processor -- start 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dd937c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94dd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dd937c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 0x563af94daa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dd937c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 0x563af94db0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dd937c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563af94e0a90 con 0x563af94d50c0 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dd937c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563af93f3e80 con 0x563af94d47a0 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dd937c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563af94e0c10 con 0x563af94ddca0 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dd937c740 10 client.0 _ll_register_callbacks cb 0x563af942faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dcf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 0x563af94daa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dcf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 0x563af94daa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58060/0 (socket says 172.21.15.33:58060) 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dcf7fe640 1 -- 172.21.15.33:0/1819713618 learned_addr learned my addr 172.21.15.33:0/1819713618 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dd4d33640 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 0x563af94db0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dcffff640 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94dd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dcf7fe640 1 -- 172.21.15.33:0/1819713618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 msgr2=0x563af94db0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dcf7fe640 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 0x563af94db0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dcf7fe640 1 -- 172.21.15.33:0/1819713618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 msgr2=0x563af94dd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dcf7fe640 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94dd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dcf7fe640 1 -- 172.21.15.33:0/1819713618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af92167d0 con 0x563af94d50c0 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dd4d33640 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 0x563af94db0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dcffff640 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94dd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dcf7fe640 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 0x563af94daa70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400bec0 tx=0x7f7dc40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb4de91502f984a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7daf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7daf7fe640 1 -- 172.21.15.33:0/1819713618 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4005220 con 0x563af94d50c0 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7daf7fe640 1 -- 172.21.15.33:0/1819713618 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f7dc40053c0 con 0x563af94d50c0 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.891+0000 7f7dd937c740 1 -- 172.21.15.33:0/1819713618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dc803a040 con 0x563af94d50c0 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.892+0000 7f7daf7fe640 1 -- 172.21.15.33:0/1819713618 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7dc40055c0 con 0x563af94d50c0 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.892+0000 7f7dd937c740 10 client.17265 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.892+0000 7f7dd937c740 1 -- 172.21.15.33:0/1819713618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563af94ecb50 con 0x563af94d50c0 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.892+0000 7f7daf7fe640 1 -- 172.21.15.33:0/1819713618 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7dc803a040 con 0x563af94d50c0 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.892+0000 7f7daf7fe640 1 -- 172.21.15.33:0/1819713618 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f7dc401b9e0 con 0x563af94d50c0 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.892+0000 7f7dd937c740 10 client.17265 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.892+0000 7f7dd937c740 10 client.17265 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:46:26.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.892+0000 7f7dd937c740 20 client.17265 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.892+0000 7f7dd937c740 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x563af94ee010 0x563af94f0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.892+0000 7f7dd937c740 4 client.17265 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.892+0000 7f7dd937c740 1 -- 172.21.15.33:0/1819713618 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563af9445cf0 con 0x563af94ee010 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.892+0000 7f7dcffff640 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x563af94ee010 0x563af94f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.893+0000 7f7dcffff640 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x563af94ee010 0x563af94f0480 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ec4852554f204bc server_cookie=c79228247acda540 in_seq=0 out_seq=0 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.893+0000 7f7daf7fe640 10 client.17265 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.893+0000 7f7daf7fe640 1 -- 172.21.15.33:0/1819713618 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563af9445cf0 con 0x563af94ee010 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.893+0000 7f7daf7fe640 10 client.17265 handle_command_reply: tid=0 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.893+0000 7f7dd937c740 1 client.17265 shutdown 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.894+0000 7f7dd937c740 1 -- 172.21.15.33:0/1819713618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 msgr2=0x563af94daa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.894+0000 7f7dd937c740 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 0x563af94daa70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400bec0 tx=0x7f7dc40117b0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.894+0000 7f7dd937c740 1 -- 172.21.15.33:0/1819713618 shutdown_connections 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.894+0000 7f7dd937c740 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x563af94ee010 0x563af94f0480 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.894+0000 7f7dd937c740 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 0x563af94db0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.894+0000 7f7dd937c740 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 0x563af94daa70 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.894+0000 7f7daf7fe640 0 client.17265 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:46:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.894+0000 7f7dd937c740 1 --2- 172.21.15.33:0/1819713618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94dd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.894+0000 7f7dd937c740 1 -- 172.21.15.33:0/1819713618 >> 172.21.15.33:0/1819713618 conn(0x563af94ceed0 msgr2=0x563af94d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.894+0000 7f7dd937c740 1 -- 172.21.15.33:0/1819713618 shutdown_connections 2024-04-02T13:46:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.894+0000 7f7dd937c740 1 -- 172.21.15.33:0/1819713618 wait complete. 2024-04-02T13:46:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.894+0000 7f7dd937c740 20 client.17265 trim_cache size 0 max 16384 2024-04-02T13:46:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.907+0000 7f7dd937c740 1 Processor -- start 2024-04-02T13:46:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.907+0000 7f7dd937c740 1 -- start start 2024-04-02T13:46:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.907+0000 7f7dd937c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94e9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.907+0000 7f7dd937c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94d50c0 0x563af94f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.907+0000 7f7dd937c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94ddca0 0x563af94f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.907+0000 7f7dd937c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd01b18d0 con 0x563af94ddca0 2024-04-02T13:46:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.907+0000 7f7dd937c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd01af1a0 con 0x563af94d47a0 2024-04-02T13:46:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.907+0000 7f7dd937c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd01ad010 con 0x563af94d50c0 2024-04-02T13:46:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.907+0000 7f7dd4d33640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94ddca0 0x563af94f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.907+0000 7f7dd4d33640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94ddca0 0x563af94f62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58068/0 (socket says 172.21.15.33:58068) 2024-04-02T13:46:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.907+0000 7f7dd4d33640 1 -- 172.21.15.33:0/2357824699 learned_addr learned my addr 172.21.15.33:0/2357824699 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.908+0000 7f7dcffff640 1 --2- 172.21.15.33:0/2357824699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94e9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.908+0000 7f7dd4d33640 1 -- 172.21.15.33:0/2357824699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94d50c0 msgr2=0x563af94f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.908+0000 7f7dd4d33640 1 --2- 172.21.15.33:0/2357824699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94d50c0 0x563af94f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.908+0000 7f7dd4d33640 1 -- 172.21.15.33:0/2357824699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 msgr2=0x563af94e9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.908+0000 7f7dd4d33640 1 --2- 172.21.15.33:0/2357824699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94e9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.908+0000 7f7dd4d33640 1 -- 172.21.15.33:0/2357824699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af94db460 con 0x563af94ddca0 2024-04-02T13:46:26.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.908+0000 7f7dcffff640 1 --2- 172.21.15.33:0/2357824699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94e9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:26.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.908+0000 7f7dd4d33640 1 --2- 172.21.15.33:0/2357824699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94ddca0 0x563af94f62d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f7dc802c680 tx=0x7f7dc8053000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509ca51675c3954a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.908+0000 7f7db5ffb640 1 -- 172.21.15.33:0/2357824699 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc805b070 con 0x563af94ddca0 2024-04-02T13:46:26.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.908+0000 7f7db5ffb640 1 -- 172.21.15.33:0/2357824699 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7dc8003e40 con 0x563af94ddca0 2024-04-02T13:46:26.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.908+0000 7f7db5ffb640 1 -- 172.21.15.33:0/2357824699 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8039030 con 0x563af94ddca0 2024-04-02T13:46:26.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.908+0000 7f7dd937c740 1 -- 172.21.15.33:0/2357824699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94ddca0 msgr2=0x563af94f62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.908+0000 7f7dd937c740 1 --2- 172.21.15.33:0/2357824699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94ddca0 0x563af94f62d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f7dc802c680 tx=0x7f7dc8053000 comp rx=0 tx=0).stop 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 -- 172.21.15.33:0/2357824699 shutdown_connections 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 --2- 172.21.15.33:0/2357824699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94ddca0 0x563af94f62d0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 --2- 172.21.15.33:0/2357824699 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94d50c0 0x563af94f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 --2- 172.21.15.33:0/2357824699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94e9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 -- 172.21.15.33:0/2357824699 >> 172.21.15.33:0/2357824699 conn(0x563af94ceed0 msgr2=0x563af94d9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 -- 172.21.15.33:0/2357824699 shutdown_connections 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 -- 172.21.15.33:0/2357824699 wait complete. 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 -- start start 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 Processor -- start 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94d78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 0x563af94d7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 0x563af94d8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd01ad660 con 0x563af94d50c0 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd01aead0 con 0x563af94d47a0 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd01add30 con 0x563af94ddca0 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dd937c740 10 client.0 _ll_register_callbacks cb 0x7f7dc400b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dcf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 0x563af94d7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dcf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 0x563af94d7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58082/0 (socket says 172.21.15.33:58082) 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dcf7fe640 1 -- 172.21.15.33:0/3937777006 learned_addr learned my addr 172.21.15.33:0/3937777006 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.909+0000 7f7dcffff640 1 --2- 172.21.15.33:0/3937777006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94d78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7dd4d33640 1 --2- 172.21.15.33:0/3937777006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 0x563af94d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7dcf7fe640 1 -- 172.21.15.33:0/3937777006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 msgr2=0x563af94d8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7dcf7fe640 1 --2- 172.21.15.33:0/3937777006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 0x563af94d8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7dcf7fe640 1 -- 172.21.15.33:0/3937777006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 msgr2=0x563af94d78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7dcf7fe640 1 --2- 172.21.15.33:0/3937777006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94d78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7dcf7fe640 1 -- 172.21.15.33:0/3937777006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af94d4ef0 con 0x563af94d50c0 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7dcf7fe640 1 --2- 172.21.15.33:0/3937777006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 0x563af94d7df0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4006010 tx=0x7f7dc400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6573d1c997d6710e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:26.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7daf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7daf7fe640 1 -- 172.21.15.33:0/3937777006 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4012970 con 0x563af94d50c0 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7daf7fe640 1 -- 172.21.15.33:0/3937777006 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f7dc4012b10 con 0x563af94d50c0 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7dd937c740 1 -- 172.21.15.33:0/3937777006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563af9440040 con 0x563af94d50c0 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7daf7fe640 1 -- 172.21.15.33:0/3937777006 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7dc4012d30 con 0x563af94d50c0 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7dd937c740 10 client.17271 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7dd937c740 1 -- 172.21.15.33:0/3937777006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563af9554810 con 0x563af94d50c0 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.910+0000 7f7daf7fe640 1 -- 172.21.15.33:0/3937777006 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x563af9440040 con 0x563af94d50c0 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.911+0000 7f7daf7fe640 1 -- 172.21.15.33:0/3937777006 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f7dc402b840 con 0x563af94d50c0 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.911+0000 7f7dd937c740 10 client.17271 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.911+0000 7f7dd937c740 10 client.17271 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.911+0000 7f7dd937c740 20 client.17271 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.911+0000 7f7dd937c740 1 --2- 172.21.15.33:0/3937777006 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x563af9554fb0 0x563af95573d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.911+0000 7f7dd937c740 4 client.17271 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.911+0000 7f7dd937c740 1 -- 172.21.15.33:0/3937777006 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563af93bd6b0 con 0x563af9554fb0 2024-04-02T13:46:26.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.911+0000 7f7dcffff640 1 --2- 172.21.15.33:0/3937777006 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x563af9554fb0 0x563af95573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.911+0000 7f7daf7fe640 10 client.17271 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.911+0000 7f7dcffff640 1 --2- 172.21.15.33:0/3937777006 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x563af9554fb0 0x563af95573d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2678fbe307dea361 server_cookie=3a566bc0961f6648 in_seq=0 out_seq=0 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7daf7fe640 1 -- 172.21.15.33:0/3937777006 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563af93bd6b0 con 0x563af9554fb0 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7daf7fe640 10 client.17271 handle_command_reply: tid=0 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd937c740 1 client.17271 shutdown 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd937c740 1 -- 172.21.15.33:0/3937777006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 msgr2=0x563af94d7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd937c740 1 --2- 172.21.15.33:0/3937777006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 0x563af94d7df0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4006010 tx=0x7f7dc400b9c0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd937c740 1 -- 172.21.15.33:0/3937777006 shutdown_connections 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd937c740 1 --2- 172.21.15.33:0/3937777006 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x563af9554fb0 0x563af95573d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd937c740 1 --2- 172.21.15.33:0/3937777006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563af94ddca0 0x563af94d8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7daf7fe640 0 client.17271 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd937c740 1 --2- 172.21.15.33:0/3937777006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563af94d50c0 0x563af94d7df0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd937c740 1 --2- 172.21.15.33:0/3937777006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563af94d47a0 0x563af94d78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd937c740 1 -- 172.21.15.33:0/3937777006 >> 172.21.15.33:0/3937777006 conn(0x563af94ceed0 msgr2=0x563af94d9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd937c740 1 -- 172.21.15.33:0/3937777006 shutdown_connections 2024-04-02T13:46:26.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd937c740 1 -- 172.21.15.33:0/3937777006 wait complete. 2024-04-02T13:46:26.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd937c740 20 client.17271 trim_cache size 0 max 16384 2024-04-02T13:46:26.922 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:46:26.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd6796640 1 -- 172.21.15.33:0/2850268957 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7db80a3d20 msgr2=0x7f7db80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd6796640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7db80a3d20 0x7f7db80a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7dc804e000 tx=0x7f7dc804c000 comp rx=0 tx=0).stop 2024-04-02T13:46:26.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd6796640 1 -- 172.21.15.33:0/2850268957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dd0101150 msgr2=0x7f7dd0199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:26.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.912+0000 7f7dd6796640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dd0101150 0x7f7dd0199280 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000e9f0 tx=0x7f7dc000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.913+0000 7f7dd6796640 1 -- 172.21.15.33:0/2850268957 shutdown_connections 2024-04-02T13:46:26.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.913+0000 7f7dd6796640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7db80a3d20 0x7f7db80a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.913+0000 7f7dd6796640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7dd01059b0 0x7f7dd01a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.913+0000 7f7dd6796640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dd0101af0 0x7f7dd01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.913+0000 7f7dd6796640 1 --2- 172.21.15.33:0/2850268957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dd0101150 0x7f7dd0199280 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:26.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.913+0000 7f7dd6796640 1 -- 172.21.15.33:0/2850268957 >> 172.21.15.33:0/2850268957 conn(0x7f7dd0076fc0 msgr2=0x7f7dd0103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:26.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.913+0000 7f7dd6796640 1 -- 172.21.15.33:0/2850268957 shutdown_connections 2024-04-02T13:46:26.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:26.913+0000 7f7dd6796640 1 -- 172.21.15.33:0/2850268957 wait complete. 2024-04-02T13:46:26.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:26 smithi033 ceph-mon[32390]: pgmap v236: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:26.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:46:26.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:26 smithi145 ceph-mon[35646]: pgmap v236: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:26.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:26 smithi050 ceph-mon[35524]: pgmap v236: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.028+0000 7f30e73e9640 1 -- 172.21.15.33:0/3554053504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30e0100b80 msgr2=0x7f30e0100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.028+0000 7f30e73e9640 1 --2- 172.21.15.33:0/3554053504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30e0100b80 0x7f30e0100f60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009930 tx=0x7f30dc02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.028+0000 7f30e73e9640 1 -- 172.21.15.33:0/3554053504 shutdown_connections 2024-04-02T13:46:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.028+0000 7f30e73e9640 1 --2- 172.21.15.33:0/3554053504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30e0105f80 0x7f30e0106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.028+0000 7f30e73e9640 1 --2- 172.21.15.33:0/3554053504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e01014a0 0x7f30e0105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.028+0000 7f30e73e9640 1 --2- 172.21.15.33:0/3554053504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30e0100b80 0x7f30e0100f60 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.028+0000 7f30e73e9640 1 -- 172.21.15.33:0/3554053504 >> 172.21.15.33:0/3554053504 conn(0x7f30e0077b60 msgr2=0x7f30e0077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.028+0000 7f30e73e9640 1 -- 172.21.15.33:0/3554053504 shutdown_connections 2024-04-02T13:46:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.028+0000 7f30e73e9640 1 -- 172.21.15.33:0/3554053504 wait complete. 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e73e9640 1 Processor -- start 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e73e9640 1 -- start start 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e73e9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e0100b80 0x7f30e01a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e73e9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30e01014a0 0x7f30e01a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e73e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30e0105f80 0x7f30e019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e73e9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0118510 con 0x7f30e0105f80 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e73e9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0118390 con 0x7f30e01014a0 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e73e9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0118690 con 0x7f30e0100b80 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e5986640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30e0105f80 0x7f30e019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e5986640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30e0105f80 0x7f30e019e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58112/0 (socket says 172.21.15.33:58112) 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e5986640 1 -- 172.21.15.33:0/1404941746 learned_addr learned my addr 172.21.15.33:0/1404941746 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e4984640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30e01014a0 0x7f30e01a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e5185640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e0100b80 0x7f30e01a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e5986640 1 -- 172.21.15.33:0/1404941746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e0100b80 msgr2=0x7f30e01a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:27.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e5986640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e0100b80 0x7f30e01a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e5986640 1 -- 172.21.15.33:0/1404941746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30e01014a0 msgr2=0x7f30e01a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:27.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e5986640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30e01014a0 0x7f30e01a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.029+0000 7f30e5986640 1 -- 172.21.15.33:0/1404941746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e019f010 con 0x7f30e0105f80 2024-04-02T13:46:27.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.030+0000 7f30e5986640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30e0105f80 0x7f30e019e890 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f30d800eaa0 tx=0x7f30d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:27.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.030+0000 7f30d67fc640 1 -- 172.21.15.33:0/1404941746 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d800cdb0 con 0x7f30e0105f80 2024-04-02T13:46:27.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.030+0000 7f30d67fc640 1 -- 172.21.15.33:0/1404941746 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f30d8005020 con 0x7f30e0105f80 2024-04-02T13:46:27.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.030+0000 7f30e73e9640 1 -- 172.21.15.33:0/1404941746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e019f300 con 0x7f30e0105f80 2024-04-02T13:46:27.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.030+0000 7f30d67fc640 1 -- 172.21.15.33:0/1404941746 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d80116b0 con 0x7f30e0105f80 2024-04-02T13:46:27.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.030+0000 7f30e73e9640 1 -- 172.21.15.33:0/1404941746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e01ab5a0 con 0x7f30e0105f80 2024-04-02T13:46:27.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30e9fcd740 1 Processor -- start 2024-04-02T13:46:27.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30d67fc640 1 -- 172.21.15.33:0/1404941746 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f30d80051d0 con 0x7f30e0105f80 2024-04-02T13:46:27.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30e9fcd740 1 -- start start 2024-04-02T13:46:27.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30e9fcd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606022207a0 0x560602220b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30e9fcd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602229760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30e9fcd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560602229ca0 0x56060222c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30e9fcd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5606021847b0 con 0x5606022210c0 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30e9fcd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560602123570 con 0x560602229ca0 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30e9fcd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560602180760 con 0x5606022207a0 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30e5185640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602229760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30e5185640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602229760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58124/0 (socket says 172.21.15.33:58124) 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30e5185640 1 -- 172.21.15.33:0/3317787002 learned_addr learned my addr 172.21.15.33:0/3317787002 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30d67fc640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f30cc0a3d20 0x7f30cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.033+0000 7f30e4984640 1 --2- 172.21.15.33:0/3317787002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606022207a0 0x560602220b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30e5986640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f30cc0a3d20 0x7f30cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30e4984640 1 --2- 172.21.15.33:0/3317787002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560602229ca0 0x56060222c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30e5185640 1 -- 172.21.15.33:0/3317787002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606022207a0 msgr2=0x560602220b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30e5185640 1 --2- 172.21.15.33:0/3317787002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606022207a0 0x560602220b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30e5185640 1 -- 172.21.15.33:0/3317787002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560602229ca0 msgr2=0x56060222c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30e5185640 1 --2- 172.21.15.33:0/3317787002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560602229ca0 0x56060222c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30e5185640 1 -- 172.21.15.33:0/3317787002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560601e983f0 con 0x5606022210c0 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30d67fc640 1 -- 172.21.15.33:0/1404941746 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f30d80d0dd0 con 0x7f30e0105f80 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30e4984640 1 --2- 172.21.15.33:0/3317787002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560602229ca0 0x56060222c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30e5185640 1 --2- 172.21.15.33:0/3317787002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602229760 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f30dc000c00 tx=0x7f30dc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67155e893b8f2095 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30cb7fe640 1 -- 172.21.15.33:0/3317787002 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc044070 con 0x5606022210c0 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30e5986640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f30cc0a3d20 0x7f30cc0a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30d8015800 tx=0x7f30d8087040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30cb7fe640 1 -- 172.21.15.33:0/3317787002 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f30dc0043c0 con 0x5606022210c0 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.034+0000 7f30cb7fe640 1 -- 172.21.15.33:0/3317787002 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc003ca0 con 0x5606022210c0 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.035+0000 7f30e9fcd740 1 -- 172.21.15.33:0/3317787002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 msgr2=0x560602229760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.035+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/3317787002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602229760 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f30dc000c00 tx=0x7f30dc005e80 comp rx=0 tx=0).stop 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.035+0000 7f30e9fcd740 1 -- 172.21.15.33:0/3317787002 shutdown_connections 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.035+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/3317787002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560602229ca0 0x56060222c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.035+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/3317787002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602229760 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.035+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/3317787002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5606022207a0 0x560602220b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.035+0000 7f30e9fcd740 1 -- 172.21.15.33:0/3317787002 >> 172.21.15.33:0/3317787002 conn(0x56060221aed0 msgr2=0x56060221b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:27.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.035+0000 7f30e9fcd740 1 -- 172.21.15.33:0/3317787002 shutdown_connections 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.035+0000 7f30e9fcd740 1 -- 172.21.15.33:0/3317787002 wait complete. 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.035+0000 7f30e9fcd740 1 -- start start 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.035+0000 7f30e9fcd740 1 Processor -- start 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e9fcd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022207a0 0x560602226b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e9fcd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602227070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e9fcd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 0x5606022275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e9fcd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56060222ca30 con 0x5606022210c0 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e9fcd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56060213fe80 con 0x5606022207a0 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e9fcd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56060222cbb0 con 0x560602229ca0 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e9fcd740 10 client.0 _ll_register_callbacks cb 0x56060217baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e5185640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602227070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e5185640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602227070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58128/0 (socket says 172.21.15.33:58128) 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e5185640 1 -- 172.21.15.33:0/1445277408 learned_addr learned my addr 172.21.15.33:0/1445277408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e4984640 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022207a0 0x560602226b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e4984640 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 0x5606022275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e5185640 1 -- 172.21.15.33:0/1445277408 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 msgr2=0x5606022275b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e5185640 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 0x5606022275b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e5185640 1 -- 172.21.15.33:0/1445277408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022207a0 msgr2=0x560602226b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e5185640 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022207a0 0x560602226b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e5185640 1 -- 172.21.15.33:0/1445277408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560601f627d0 con 0x5606022210c0 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e4984640 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022207a0 0x560602226b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e5185640 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602227070 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f30dc005c40 tx=0x7f30dc003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211f92b2bb32645f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30c8ff9640 1 -- 172.21.15.33:0/1445277408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc04f070 con 0x5606022210c0 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30c8ff9640 1 -- 172.21.15.33:0/1445277408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f30dc005250 con 0x5606022210c0 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e9fcd740 1 -- 172.21.15.33:0/1445277408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30dc0043c0 con 0x5606022210c0 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30c8ff9640 1 -- 172.21.15.33:0/1445277408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f30dc05b050 con 0x5606022210c0 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e9fcd740 10 client.17283 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.036+0000 7f30e9fcd740 1 -- 172.21.15.33:0/1445277408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560602238ad0 con 0x5606022210c0 2024-04-02T13:46:27.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.037+0000 7f30c8ff9640 1 -- 172.21.15.33:0/1445277408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f30dc0043c0 con 0x5606022210c0 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.037+0000 7f30c8ff9640 1 -- 172.21.15.33:0/1445277408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f30dc004950 con 0x5606022210c0 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.037+0000 7f30e9fcd740 10 client.17283 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.037+0000 7f30e9fcd740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.037+0000 7f30e9fcd740 20 client.17283 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.037+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x560602239e20 0x56060223c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.037+0000 7f30e9fcd740 4 client.17283 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.037+0000 7f30e9fcd740 1 -- 172.21.15.33:0/1445277408 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560602191cf0 con 0x560602239e20 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.037+0000 7f30e4984640 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x560602239e20 0x56060223c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30c8ff9640 10 client.17283 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30e4984640 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x560602239e20 0x56060223c210 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71b6318a9a05ee6b server_cookie=202a749885f061e0 in_seq=0 out_seq=0 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30c8ff9640 1 -- 172.21.15.33:0/1445277408 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560602191cf0 con 0x560602239e20 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30c8ff9640 10 client.17283 handle_command_reply: tid=0 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30e9fcd740 1 client.17283 shutdown 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30e9fcd740 1 -- 172.21.15.33:0/1445277408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 msgr2=0x560602227070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:27.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602227070 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f30dc005c40 tx=0x7f30dc003cd0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30e9fcd740 1 -- 172.21.15.33:0/1445277408 shutdown_connections 2024-04-02T13:46:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x560602239e20 0x56060223c210 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 0x5606022275b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602227070 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30c8ff9640 0 client.17283 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/1445277408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022207a0 0x560602226b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30e9fcd740 1 -- 172.21.15.33:0/1445277408 >> 172.21.15.33:0/1445277408 conn(0x56060221aed0 msgr2=0x56060221cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.038+0000 7f30e9fcd740 1 -- 172.21.15.33:0/1445277408 shutdown_connections 2024-04-02T13:46:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.039+0000 7f30e9fcd740 1 -- 172.21.15.33:0/1445277408 wait complete. 2024-04-02T13:46:27.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.039+0000 7f30e9fcd740 20 client.17283 trim_cache size 0 max 16384 2024-04-02T13:46:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.052+0000 7f30e9fcd740 1 Processor -- start 2024-04-02T13:46:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.052+0000 7f30e9fcd740 1 -- start start 2024-04-02T13:46:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.052+0000 7f30e9fcd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022207a0 0x560602229260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.052+0000 7f30e9fcd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602235920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.052+0000 7f30e9fcd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 0x5606022417b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.052+0000 7f30e9fcd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30e01b7f00 con 0x5606022210c0 2024-04-02T13:46:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.052+0000 7f30e9fcd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f30e01b57d0 con 0x5606022207a0 2024-04-02T13:46:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.052+0000 7f30e9fcd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f30e01b3640 con 0x560602229ca0 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.052+0000 7f30e5185640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602235920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.052+0000 7f30e5185640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602235920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58144/0 (socket says 172.21.15.33:58144) 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.052+0000 7f30e5185640 1 -- 172.21.15.33:0/3720718559 learned_addr learned my addr 172.21.15.33:0/3720718559 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30e5185640 1 -- 172.21.15.33:0/3720718559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 msgr2=0x5606022417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30e5185640 1 --2- 172.21.15.33:0/3720718559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 0x5606022417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30e5185640 1 -- 172.21.15.33:0/3720718559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022207a0 msgr2=0x560602229260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30e5185640 1 --2- 172.21.15.33:0/3720718559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022207a0 0x560602229260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30e5185640 1 -- 172.21.15.33:0/3720718559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560602220ef0 con 0x5606022210c0 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30e5185640 1 --2- 172.21.15.33:0/3720718559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602235920 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0095d0 tx=0x7f30dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1789d93b5802977f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30caffd640 1 -- 172.21.15.33:0/3720718559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc04f040 con 0x5606022210c0 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30caffd640 1 -- 172.21.15.33:0/3720718559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f30dc0040c0 con 0x5606022210c0 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30caffd640 1 -- 172.21.15.33:0/3720718559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc044040 con 0x5606022210c0 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30e9fcd740 1 -- 172.21.15.33:0/3720718559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 msgr2=0x560602235920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/3720718559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602235920 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0095d0 tx=0x7f30dc000bc0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30e9fcd740 1 -- 172.21.15.33:0/3720718559 shutdown_connections 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/3720718559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 0x5606022417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/3720718559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022210c0 0x560602235920 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/3720718559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022207a0 0x560602229260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.053+0000 7f30e9fcd740 1 -- 172.21.15.33:0/3720718559 >> 172.21.15.33:0/3720718559 conn(0x56060221aed0 msgr2=0x560602227380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e9fcd740 1 -- 172.21.15.33:0/3720718559 shutdown_connections 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e9fcd740 1 -- 172.21.15.33:0/3720718559 wait complete. 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e9fcd740 1 -- start start 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e9fcd740 1 Processor -- start 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e9fcd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022207a0 0x56060222d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e9fcd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022210c0 0x56060222de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e9fcd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 0x56060222e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e9fcd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30e01b3c90 con 0x5606022207a0 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e9fcd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f30e01b5100 con 0x5606022210c0 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e9fcd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f30e01b4360 con 0x560602229ca0 2024-04-02T13:46:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e9fcd740 10 client.0 _ll_register_callbacks cb 0x7f30dc005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e4984640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022207a0 0x56060222d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e4984640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022207a0 0x56060222d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58158/0 (socket says 172.21.15.33:58158) 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e4984640 1 -- 172.21.15.33:0/651586881 learned_addr learned my addr 172.21.15.33:0/651586881 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.054+0000 7f30e5185640 1 --2- 172.21.15.33:0/651586881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022210c0 0x56060222de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30e4984640 1 --2- 172.21.15.33:0/651586881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 0x56060222e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30e4984640 1 -- 172.21.15.33:0/651586881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 msgr2=0x56060222e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30e4984640 1 --2- 172.21.15.33:0/651586881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 0x56060222e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30e4984640 1 -- 172.21.15.33:0/651586881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022210c0 msgr2=0x56060222de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30e4984640 1 --2- 172.21.15.33:0/651586881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022210c0 0x56060222de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30e4984640 1 -- 172.21.15.33:0/651586881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56060229f0d0 con 0x5606022207a0 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30e4984640 1 --2- 172.21.15.33:0/651586881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022207a0 0x56060222d8e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f30d000bf40 tx=0x7f30d000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabcb9ddedae1717 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30c8ff9640 1 -- 172.21.15.33:0/651586881 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0003a40 con 0x5606022207a0 2024-04-02T13:46:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30c8ff9640 1 -- 172.21.15.33:0/651586881 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f30d0003be0 con 0x5606022207a0 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30e9fcd740 1 -- 172.21.15.33:0/651586881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30dc0040c0 con 0x5606022207a0 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30c8ff9640 1 -- 172.21.15.33:0/651586881 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f30d0003de0 con 0x5606022207a0 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30e9fcd740 10 client.17289 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30e9fcd740 1 -- 172.21.15.33:0/651586881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5606022a03f0 con 0x5606022207a0 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30c8ff9640 1 -- 172.21.15.33:0/651586881 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f30dc0040c0 con 0x5606022207a0 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30c8ff9640 1 -- 172.21.15.33:0/651586881 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f30d0010550 con 0x5606022207a0 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30e9fcd740 10 client.17289 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30e9fcd740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.055+0000 7f30e9fcd740 20 client.17289 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.056+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/651586881 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5606022a0b70 0x5606022a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.056+0000 7f30e9fcd740 4 client.17289 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.056+0000 7f30e9fcd740 1 -- 172.21.15.33:0/651586881 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56060218c040 con 0x5606022a0b70 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.056+0000 7f30e5185640 1 --2- 172.21.15.33:0/651586881 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5606022a0b70 0x5606022a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.056+0000 7f30c8ff9640 10 client.17289 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.056+0000 7f30e5185640 1 --2- 172.21.15.33:0/651586881 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5606022a0b70 0x5606022a2f90 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a04b986584d8af74 server_cookie=176d7cc1cc7dafc3 in_seq=0 out_seq=0 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.056+0000 7f30c8ff9640 1 -- 172.21.15.33:0/651586881 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56060218c040 con 0x5606022a0b70 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.056+0000 7f30c8ff9640 10 client.17289 handle_command_reply: tid=0 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.056+0000 7f30e9fcd740 1 client.17289 shutdown 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e9fcd740 1 -- 172.21.15.33:0/651586881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022207a0 msgr2=0x56060222d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/651586881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022207a0 0x56060222d8e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f30d000bf40 tx=0x7f30d000ba50 comp rx=0 tx=0).stop 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e9fcd740 1 -- 172.21.15.33:0/651586881 shutdown_connections 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/651586881 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5606022a0b70 0x5606022a2f90 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/651586881 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560602229ca0 0x56060222e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/651586881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5606022210c0 0x56060222de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e9fcd740 1 --2- 172.21.15.33:0/651586881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606022207a0 0x56060222d8e0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30c8ff9640 0 client.17289 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e9fcd740 1 -- 172.21.15.33:0/651586881 >> 172.21.15.33:0/651586881 conn(0x56060221aed0 msgr2=0x5606022196a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e9fcd740 1 -- 172.21.15.33:0/651586881 shutdown_connections 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e9fcd740 1 -- 172.21.15.33:0/651586881 wait complete. 2024-04-02T13:46:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e9fcd740 20 client.17289 trim_cache size 0 max 16384 2024-04-02T13:46:27.066 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:46:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e73e9640 1 -- 172.21.15.33:0/1404941746 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f30cc0a3d20 msgr2=0x7f30cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e73e9640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f30cc0a3d20 0x7f30cc0a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30d8015800 tx=0x7f30d8087040 comp rx=0 tx=0).stop 2024-04-02T13:46:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e73e9640 1 -- 172.21.15.33:0/1404941746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30e0105f80 msgr2=0x7f30e019e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e73e9640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30e0105f80 0x7f30e019e890 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f30d800eaa0 tx=0x7f30d800ef60 comp rx=0 tx=0).stop 2024-04-02T13:46:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e73e9640 1 -- 172.21.15.33:0/1404941746 shutdown_connections 2024-04-02T13:46:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e73e9640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f30cc0a3d20 0x7f30cc0a61e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e73e9640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30e0105f80 0x7f30e019e890 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e73e9640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f30e01014a0 0x7f30e01a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e73e9640 1 --2- 172.21.15.33:0/1404941746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30e0100b80 0x7f30e01a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e73e9640 1 -- 172.21.15.33:0/1404941746 >> 172.21.15.33:0/1404941746 conn(0x7f30e0077b60 msgr2=0x7f30e0103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e73e9640 1 -- 172.21.15.33:0/1404941746 shutdown_connections 2024-04-02T13:46:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:27.057+0000 7f30e73e9640 1 -- 172.21.15.33:0/1404941746 wait complete. 2024-04-02T13:46:27.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:27 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1565833804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:46:27.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:27 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1565833804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:46:28.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:27 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1565833804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:46:28.071 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:46:28.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:46:28.071 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:46:28.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 -- 172.21.15.33:0/888525248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e4107b50 msgr2=0x7fb2e4107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 --2- 172.21.15.33:0/888525248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e4107b50 0x7fb2e4107f30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009930 tx=0x7fb2dc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 -- 172.21.15.33:0/888525248 shutdown_connections 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 --2- 172.21.15.33:0/888525248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e40696c0 0x7fb2e410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 --2- 172.21.15.33:0/888525248 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e4068d00 0x7fb2e4069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 --2- 172.21.15.33:0/888525248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e4107b50 0x7fb2e4107f30 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 -- 172.21.15.33:0/888525248 >> 172.21.15.33:0/888525248 conn(0x7fb2e4077b00 msgr2=0x7fb2e4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 -- 172.21.15.33:0/888525248 shutdown_connections 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 -- 172.21.15.33:0/888525248 wait complete. 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 Processor -- start 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 -- start start 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e4068d00 0x7fb2e41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e40696c0 0x7fb2e4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e4107b50 0x7fb2e419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.188+0000 7fb2ec6c5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4111e40 con 0x7fb2e4068d00 2024-04-02T13:46:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2ec6c5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4111cc0 con 0x7fb2e4107b50 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2ec6c5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4111fc0 con 0x7fb2e40696c0 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2ea461640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e4068d00 0x7fb2e41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2ea461640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e4068d00 0x7fb2e41982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58168/0 (socket says 172.21.15.33:58168) 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2ea461640 1 -- 172.21.15.33:0/3206222947 learned_addr learned my addr 172.21.15.33:0/3206222947 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2eac62640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e4107b50 0x7fb2e419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2e9c60640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e40696c0 0x7fb2e4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2ea461640 1 -- 172.21.15.33:0/3206222947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e40696c0 msgr2=0x7fb2e4198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2ea461640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e40696c0 0x7fb2e4198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2ea461640 1 -- 172.21.15.33:0/3206222947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e4107b50 msgr2=0x7fb2e419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2ea461640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e4107b50 0x7fb2e419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2ea461640 1 -- 172.21.15.33:0/3206222947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e419d2b0 con 0x7fb2e4068d00 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2e9c60640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e40696c0 0x7fb2e4198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2ea461640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e4068d00 0x7fb2e41982c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009900 tx=0x7fb2dc02ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2db7fe640 1 -- 172.21.15.33:0/3206222947 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc043020 con 0x7fb2e4068d00 2024-04-02T13:46:28.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2ec6c5640 1 -- 172.21.15.33:0/3206222947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e419d540 con 0x7fb2e4068d00 2024-04-02T13:46:28.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2db7fe640 1 -- 172.21.15.33:0/3206222947 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb2dc005430 con 0x7fb2e4068d00 2024-04-02T13:46:28.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2db7fe640 1 -- 172.21.15.33:0/3206222947 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc003ab0 con 0x7fb2e4068d00 2024-04-02T13:46:28.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.189+0000 7fb2ec6c5640 1 -- 172.21.15.33:0/3206222947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e41a4e30 con 0x7fb2e4068d00 2024-04-02T13:46:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.192+0000 7fb2db7fe640 1 -- 172.21.15.33:0/3206222947 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb2dc046020 con 0x7fb2e4068d00 2024-04-02T13:46:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.192+0000 7fb2ef387740 1 Processor -- start 2024-04-02T13:46:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.192+0000 7fb2ef387740 1 -- start start 2024-04-02T13:46:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.192+0000 7fb2ef387740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0b2c0 0x55b317c0b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.192+0000 7fb2ef387740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 0x55b317c14280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.192+0000 7fb2ef387740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c147c0 0x55b317c16bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.192+0000 7fb2ef387740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b317b6f770 con 0x55b317c0b2c0 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.192+0000 7fb2ef387740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b317b17960 con 0x55b317c0bbe0 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.192+0000 7fb2ef387740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b317b6b8c0 con 0x55b317c147c0 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2eac62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0b2c0 0x55b317c0b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2eac62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0b2c0 0x55b317c0b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58182/0 (socket says 172.21.15.33:58182) 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2db7fe640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb2d00a3d20 0x7fb2d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2eac62640 1 -- 172.21.15.33:0/1674893947 learned_addr learned my addr 172.21.15.33:0/1674893947 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2ea461640 1 --2- 172.21.15.33:0/1674893947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 0x55b317c14280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2e9c60640 1 --2- 172.21.15.33:0/1674893947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c147c0 0x55b317c16bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2eac62640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb2d00a3d20 0x7fb2d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2eac62640 1 -- 172.21.15.33:0/1674893947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c147c0 msgr2=0x55b317c16bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2eac62640 1 --2- 172.21.15.33:0/1674893947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c147c0 0x55b317c16bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2db7fe640 1 -- 172.21.15.33:0/3206222947 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fb2dc0f25a0 con 0x7fb2e4068d00 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2eac62640 1 -- 172.21.15.33:0/1674893947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 msgr2=0x55b317c14280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2eac62640 1 --2- 172.21.15.33:0/1674893947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 0x55b317c14280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2eac62640 1 -- 172.21.15.33:0/1674893947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31794b7e0 con 0x55b317c0b2c0 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2eac62640 1 --2- 172.21.15.33:0/1674893947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0b2c0 0x55b317c0b6a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0005c40 tx=0x7fb2e000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9584687879be413e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2d8ff9640 1 -- 172.21.15.33:0/1674893947 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0022020 con 0x55b317c0b2c0 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2d8ff9640 1 -- 172.21.15.33:0/1674893947 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb2e001e420 con 0x55b317c0b2c0 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.193+0000 7fb2d8ff9640 1 -- 172.21.15.33:0/1674893947 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e001ee60 con 0x55b317c0b2c0 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.194+0000 7fb2ef387740 1 -- 172.21.15.33:0/1674893947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0b2c0 msgr2=0x55b317c0b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.194+0000 7fb2eac62640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb2d00a3d20 0x7fb2d00a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0023000 tx=0x7fb2e0021000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.194+0000 7fb2ef387740 1 --2- 172.21.15.33:0/1674893947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0b2c0 0x55b317c0b6a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0005c40 tx=0x7fb2e000f3c0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.194+0000 7fb2ef387740 1 -- 172.21.15.33:0/1674893947 shutdown_connections 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.194+0000 7fb2ef387740 1 --2- 172.21.15.33:0/1674893947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c147c0 0x55b317c16bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.194+0000 7fb2ef387740 1 --2- 172.21.15.33:0/1674893947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 0x55b317c14280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.194+0000 7fb2ef387740 1 --2- 172.21.15.33:0/1674893947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0b2c0 0x55b317c0b6a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.194+0000 7fb2ef387740 1 -- 172.21.15.33:0/1674893947 >> 172.21.15.33:0/1674893947 conn(0x55b317c059f0 msgr2=0x55b317c05dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.194+0000 7fb2ef387740 1 -- 172.21.15.33:0/1674893947 shutdown_connections 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.194+0000 7fb2ef387740 1 -- 172.21.15.33:0/1674893947 wait complete. 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2ef387740 1 -- start start 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2ef387740 1 Processor -- start 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2ef387740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c0b2c0 0x55b317c11740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2ef387740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 0x55b317c11c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2ef387740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 0x55b317c121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2ef387740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b317c175b0 con 0x55b317c147c0 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2ef387740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b317b2ad20 con 0x55b317c0bbe0 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2ef387740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b317c17730 con 0x55b317c0b2c0 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2ef387740 10 client.0 _ll_register_callbacks cb 0x55b317af7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2eac62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 0x55b317c121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2eac62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 0x55b317c121c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58184/0 (socket says 172.21.15.33:58184) 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2eac62640 1 -- 172.21.15.33:0/330255854 learned_addr learned my addr 172.21.15.33:0/330255854 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2ea461640 1 --2- 172.21.15.33:0/330255854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c0b2c0 0x55b317c11740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2e9c60640 1 --2- 172.21.15.33:0/330255854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 0x55b317c11c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2eac62640 1 -- 172.21.15.33:0/330255854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c0b2c0 msgr2=0x55b317c11740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2eac62640 1 --2- 172.21.15.33:0/330255854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c0b2c0 0x55b317c11740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2eac62640 1 -- 172.21.15.33:0/330255854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 msgr2=0x55b317c11c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2eac62640 1 --2- 172.21.15.33:0/330255854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 0x55b317c11c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.195+0000 7fb2eac62640 1 -- 172.21.15.33:0/330255854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3179abf70 con 0x55b317c147c0 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2eac62640 1 --2- 172.21.15.33:0/330255854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 0x55b317c121c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb2e001ed60 tx=0x7fb2e001e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7410317bd6028124 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2cdffb640 1 -- 172.21.15.33:0/330255854 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0022040 con 0x55b317c147c0 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2cdffb640 1 -- 172.21.15.33:0/330255854 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fb2e0013170 con 0x55b317c147c0 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2ef387740 1 -- 172.21.15.33:0/330255854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e001e420 con 0x55b317c147c0 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2cdffb640 1 -- 172.21.15.33:0/330255854 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb2e003c050 con 0x55b317c147c0 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2ef387740 10 client.17301 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2ef387740 1 -- 172.21.15.33:0/330255854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b317843170 con 0x55b317c147c0 2024-04-02T13:46:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2cdffb640 1 -- 172.21.15.33:0/330255854 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb2e001e420 con 0x55b317c147c0 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2cdffb640 1 -- 172.21.15.33:0/330255854 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fb2e0005520 con 0x55b317c147c0 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2ef387740 10 client.17301 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2ef387740 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2ef387740 20 client.17301 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2ef387740 1 --2- 172.21.15.33:0/330255854 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b317c24970 0x55b317c26d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2ef387740 4 client.17301 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.196+0000 7fb2ef387740 1 -- 172.21.15.33:0/330255854 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b317b7ccb0 con 0x55b317c24970 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.197+0000 7fb2ea461640 1 --2- 172.21.15.33:0/330255854 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b317c24970 0x55b317c26d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.197+0000 7fb2cdffb640 10 client.17301 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.197+0000 7fb2ea461640 1 --2- 172.21.15.33:0/330255854 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b317c24970 0x55b317c26d60 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee20bae9151f08bf server_cookie=5c5eaa98a4ab0093 in_seq=0 out_seq=0 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2cdffb640 1 -- 172.21.15.33:0/330255854 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b317b7ccb0 con 0x55b317c24970 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2cdffb640 10 client.17301 handle_command_reply: tid=0 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2ef387740 1 client.17301 shutdown 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2ef387740 1 -- 172.21.15.33:0/330255854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 msgr2=0x55b317c121c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2ef387740 1 --2- 172.21.15.33:0/330255854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 0x55b317c121c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb2e001ed60 tx=0x7fb2e001e9f0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2ef387740 1 -- 172.21.15.33:0/330255854 shutdown_connections 2024-04-02T13:46:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2ef387740 1 --2- 172.21.15.33:0/330255854 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b317c24970 0x55b317c26d60 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2ef387740 1 --2- 172.21.15.33:0/330255854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 0x55b317c121c0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2ef387740 1 --2- 172.21.15.33:0/330255854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 0x55b317c11c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2cdffb640 0 client.17301 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2ef387740 1 --2- 172.21.15.33:0/330255854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c0b2c0 0x55b317c11740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2ef387740 1 -- 172.21.15.33:0/330255854 >> 172.21.15.33:0/330255854 conn(0x55b317c059f0 msgr2=0x55b317c07740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2ef387740 1 -- 172.21.15.33:0/330255854 shutdown_connections 2024-04-02T13:46:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2ef387740 1 -- 172.21.15.33:0/330255854 wait complete. 2024-04-02T13:46:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.198+0000 7fb2ef387740 20 client.17301 trim_cache size 0 max 16384 2024-04-02T13:46:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2ef387740 1 Processor -- start 2024-04-02T13:46:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2ef387740 1 -- start start 2024-04-02T13:46:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2ef387740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c0b2c0 0x55b317c13e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2ef387740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 0x55b317c20470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2ef387740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 0x55b317c2c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2ef387740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41b18b0 con 0x55b317c147c0 2024-04-02T13:46:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2ef387740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41af180 con 0x55b317c0bbe0 2024-04-02T13:46:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2ef387740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41acff0 con 0x55b317c0b2c0 2024-04-02T13:46:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2eac62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 0x55b317c2c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2eac62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 0x55b317c2c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58190/0 (socket says 172.21.15.33:58190) 2024-04-02T13:46:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2eac62640 1 -- 172.21.15.33:0/885649391 learned_addr learned my addr 172.21.15.33:0/885649391 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2e9c60640 1 --2- 172.21.15.33:0/885649391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 0x55b317c20470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2ea461640 1 --2- 172.21.15.33:0/885649391 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c0b2c0 0x55b317c13e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2eac62640 1 -- 172.21.15.33:0/885649391 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c0b2c0 msgr2=0x55b317c13e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2eac62640 1 --2- 172.21.15.33:0/885649391 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c0b2c0 0x55b317c13e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2eac62640 1 -- 172.21.15.33:0/885649391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 msgr2=0x55b317c20470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2eac62640 1 --2- 172.21.15.33:0/885649391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 0x55b317c20470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2eac62640 1 -- 172.21.15.33:0/885649391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b317c0ba10 con 0x55b317c147c0 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.216+0000 7fb2e9c60640 1 --2- 172.21.15.33:0/885649391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 0x55b317c20470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.217+0000 7fb2eac62640 1 --2- 172.21.15.33:0/885649391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 0x55b317c2c570 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb2e001ed30 tx=0x7fb2e0005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2d25c87d63dd45 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.217+0000 7fb2cffff640 1 -- 172.21.15.33:0/885649391 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0022040 con 0x55b317c147c0 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.217+0000 7fb2cffff640 1 -- 172.21.15.33:0/885649391 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb2e00136c0 con 0x55b317c147c0 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.217+0000 7fb2cffff640 1 -- 172.21.15.33:0/885649391 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00133d0 con 0x55b317c147c0 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.217+0000 7fb2ef387740 1 -- 172.21.15.33:0/885649391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 msgr2=0x55b317c2c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.217+0000 7fb2ef387740 1 --2- 172.21.15.33:0/885649391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 0x55b317c2c570 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb2e001ed30 tx=0x7fb2e0005f50 comp rx=0 tx=0).stop 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.217+0000 7fb2ef387740 1 -- 172.21.15.33:0/885649391 shutdown_connections 2024-04-02T13:46:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.217+0000 7fb2ef387740 1 --2- 172.21.15.33:0/885649391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c147c0 0x55b317c2c570 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.217+0000 7fb2ef387740 1 --2- 172.21.15.33:0/885649391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0bbe0 0x55b317c20470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.217+0000 7fb2ef387740 1 --2- 172.21.15.33:0/885649391 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c0b2c0 0x55b317c13e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.217+0000 7fb2ef387740 1 -- 172.21.15.33:0/885649391 >> 172.21.15.33:0/885649391 conn(0x55b317c059f0 msgr2=0x55b317c04ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.217+0000 7fb2ef387740 1 -- 172.21.15.33:0/885649391 shutdown_connections 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.217+0000 7fb2ef387740 1 -- 172.21.15.33:0/885649391 wait complete. 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2ef387740 1 -- start start 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2ef387740 1 Processor -- start 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2ef387740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0b2c0 0x55b317c0e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2ef387740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0bbe0 0x55b317c0ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2ef387740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c147c0 0x55b317c0f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2ef387740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41ad640 con 0x55b317c0bbe0 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2ef387740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41aeab0 con 0x55b317c0b2c0 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2ef387740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41add10 con 0x55b317c147c0 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2ef387740 10 client.0 _ll_register_callbacks cb 0x7fb2dc0be0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2e9c60640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0bbe0 0x55b317c0ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2e9c60640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0bbe0 0x55b317c0ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58202/0 (socket says 172.21.15.33:58202) 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2e9c60640 1 -- 172.21.15.33:0/4042287377 learned_addr learned my addr 172.21.15.33:0/4042287377 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2eac62640 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c147c0 0x55b317c0f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2ea461640 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0b2c0 0x55b317c0e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2e9c60640 1 -- 172.21.15.33:0/4042287377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c147c0 msgr2=0x55b317c0f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2e9c60640 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c147c0 0x55b317c0f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2e9c60640 1 -- 172.21.15.33:0/4042287377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0b2c0 msgr2=0x55b317c0e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2e9c60640 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0b2c0 0x55b317c0e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2e9c60640 1 -- 172.21.15.33:0/4042287377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b317c8a0b0 con 0x55b317c0bbe0 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2eac62640 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c147c0 0x55b317c0f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2e9c60640 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0bbe0 0x55b317c0ec50 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40fdaf0 tx=0x7fb2d400fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a333c77aac1f3c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.218+0000 7fb2cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2cdffb640 1 -- 172.21.15.33:0/4042287377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d400ecc0 con 0x55b317c0bbe0 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2cdffb640 1 -- 172.21.15.33:0/4042287377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fb2d4005020 con 0x55b317c0bbe0 2024-04-02T13:46:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2ef387740 1 -- 172.21.15.33:0/4042287377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e00136c0 con 0x55b317c0bbe0 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2cdffb640 1 -- 172.21.15.33:0/4042287377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb2d4025050 con 0x55b317c0bbe0 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2ef387740 10 client.17307 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2ef387740 1 -- 172.21.15.33:0/4042287377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b317c8b3c0 con 0x55b317c0bbe0 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2cdffb640 1 -- 172.21.15.33:0/4042287377 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb2e00136c0 con 0x55b317c0bbe0 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2cdffb640 1 -- 172.21.15.33:0/4042287377 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fb2d401a400 con 0x55b317c0bbe0 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2ef387740 10 client.17307 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2ef387740 10 client.17307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2ef387740 20 client.17307 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2ef387740 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b317c8bbf0 0x55b317c8e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2ef387740 4 client.17307 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2ef387740 1 -- 172.21.15.33:0/4042287377 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b317b77000 con 0x55b317c8bbf0 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.219+0000 7fb2ea461640 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b317c8bbf0 0x55b317c8e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.220+0000 7fb2cdffb640 10 client.17307 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.220+0000 7fb2ea461640 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b317c8bbf0 0x55b317c8e010 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc272b506ea32a2 server_cookie=1774d6b791a6515 in_seq=0 out_seq=0 2024-04-02T13:46:28.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2cdffb640 1 -- 172.21.15.33:0/4042287377 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b317b77000 con 0x55b317c8bbf0 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2cdffb640 10 client.17307 handle_command_reply: tid=0 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2ef387740 1 client.17307 shutdown 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2ef387740 1 -- 172.21.15.33:0/4042287377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0bbe0 msgr2=0x55b317c0ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2ef387740 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0bbe0 0x55b317c0ec50 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40fdaf0 tx=0x7fb2d400fc70 comp rx=0 tx=0).stop 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2ef387740 1 -- 172.21.15.33:0/4042287377 shutdown_connections 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2ef387740 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55b317c8bbf0 0x55b317c8e010 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2ef387740 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b317c147c0 0x55b317c0f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2ef387740 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b317c0bbe0 0x55b317c0ec50 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2ef387740 1 --2- 172.21.15.33:0/4042287377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b317c0b2c0 0x55b317c0e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2ef387740 1 -- 172.21.15.33:0/4042287377 >> 172.21.15.33:0/4042287377 conn(0x55b317c059f0 msgr2=0x55b317c06d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2cdffb640 0 client.17307 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2ef387740 1 -- 172.21.15.33:0/4042287377 shutdown_connections 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2ef387740 1 -- 172.21.15.33:0/4042287377 wait complete. 2024-04-02T13:46:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.221+0000 7fb2ef387740 20 client.17307 trim_cache size 0 max 16384 2024-04-02T13:46:28.230 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:46:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:46:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "5048311b-e895-4cdc-b531-fb4e7679b624", 2024-04-02T13:46:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:46:28.230 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:46:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.222+0000 7fb2ec6c5640 1 -- 172.21.15.33:0/3206222947 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb2d00a3d20 msgr2=0x7fb2d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.222+0000 7fb2ec6c5640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb2d00a3d20 0x7fb2d00a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0023000 tx=0x7fb2e0021000 comp rx=0 tx=0).stop 2024-04-02T13:46:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.222+0000 7fb2ec6c5640 1 -- 172.21.15.33:0/3206222947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e4068d00 msgr2=0x7fb2e41982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.222+0000 7fb2ec6c5640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e4068d00 0x7fb2e41982c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009900 tx=0x7fb2dc02ca40 comp rx=0 tx=0).stop 2024-04-02T13:46:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.222+0000 7fb2ec6c5640 1 -- 172.21.15.33:0/3206222947 shutdown_connections 2024-04-02T13:46:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.222+0000 7fb2ec6c5640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb2d00a3d20 0x7fb2d00a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.222+0000 7fb2ec6c5640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e4107b50 0x7fb2e419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.222+0000 7fb2ec6c5640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2e40696c0 0x7fb2e4198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.222+0000 7fb2ec6c5640 1 --2- 172.21.15.33:0/3206222947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e4068d00 0x7fb2e41982c0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.222+0000 7fb2ec6c5640 1 -- 172.21.15.33:0/3206222947 >> 172.21.15.33:0/3206222947 conn(0x7fb2e4077b00 msgr2=0x7fb2e40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.222+0000 7fb2ec6c5640 1 -- 172.21.15.33:0/3206222947 shutdown_connections 2024-04-02T13:46:28.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.222+0000 7fb2ec6c5640 1 -- 172.21.15.33:0/3206222947 wait complete. 2024-04-02T13:46:28.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:46:28.235 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:46:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.337+0000 7fa13b950640 1 -- 172.21.15.33:0/737716469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa134100970 msgr2=0x7fa134100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.337+0000 7fa13b950640 1 --2- 172.21.15.33:0/737716469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa134100970 0x7fa134100df0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa128009930 tx=0x7fa12802bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.337+0000 7fa13b950640 1 -- 172.21.15.33:0/737716469 shutdown_connections 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.337+0000 7fa13b950640 1 --2- 172.21.15.33:0/737716469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa134101330 0x7fa13410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.337+0000 7fa13b950640 1 --2- 172.21.15.33:0/737716469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa134100970 0x7fa134100df0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.337+0000 7fa13b950640 1 --2- 172.21.15.33:0/737716469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa134106980 0x7fa134106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.337+0000 7fa13b950640 1 -- 172.21.15.33:0/737716469 >> 172.21.15.33:0/737716469 conn(0x7fa134077b00 msgr2=0x7fa134077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa13b950640 1 -- 172.21.15.33:0/737716469 shutdown_connections 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa13b950640 1 -- 172.21.15.33:0/737716469 wait complete. 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa13b950640 1 Processor -- start 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa13b950640 1 -- start start 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa13b950640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa134100970 0x7fa134198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa13b950640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa134101330 0x7fa1341988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa13b950640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa134106980 0x7fa13419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa13b950640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa134110d30 con 0x7fa134100970 2024-04-02T13:46:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa13b950640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa134110bb0 con 0x7fa134101330 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa13b950640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa134110eb0 con 0x7fa134106980 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa1396ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa134100970 0x7fa134198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa1396ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa134100970 0x7fa134198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58212/0 (socket says 172.21.15.33:58212) 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa1396ec640 1 -- 172.21.15.33:0/1176028286 learned_addr learned my addr 172.21.15.33:0/1176028286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.338+0000 7fa138eeb640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa134101330 0x7fa1341988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.339+0000 7fa139eed640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa134106980 0x7fa13419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.339+0000 7fa1396ec640 1 -- 172.21.15.33:0/1176028286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa134106980 msgr2=0x7fa13419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.339+0000 7fa1396ec640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa134106980 0x7fa13419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.339+0000 7fa1396ec640 1 -- 172.21.15.33:0/1176028286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa134101330 msgr2=0x7fa1341988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.339+0000 7fa1396ec640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa134101330 0x7fa1341988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.339+0000 7fa1396ec640 1 -- 172.21.15.33:0/1176028286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13419d370 con 0x7fa134100970 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.339+0000 7fa1396ec640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa134100970 0x7fa134198380 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00c920 tx=0x7fa12c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.339+0000 7fa1327fc640 1 -- 172.21.15.33:0/1176028286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c007c70 con 0x7fa134100970 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.339+0000 7fa1327fc640 1 -- 172.21.15.33:0/1176028286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa12c007e10 con 0x7fa134100970 2024-04-02T13:46:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.339+0000 7fa13b950640 1 -- 172.21.15.33:0/1176028286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13419d660 con 0x7fa134100970 2024-04-02T13:46:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.339+0000 7fa1327fc640 1 -- 172.21.15.33:0/1176028286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c018850 con 0x7fa134100970 2024-04-02T13:46:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.339+0000 7fa13b950640 1 -- 172.21.15.33:0/1176028286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1341a4ef0 con 0x7fa134100970 2024-04-02T13:46:28.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.341+0000 7fa1327fc640 1 -- 172.21.15.33:0/1176028286 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa12c018a30 con 0x7fa134100970 2024-04-02T13:46:28.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa13e60f740 1 Processor -- start 2024-04-02T13:46:28.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa13e60f740 1 -- start start 2024-04-02T13:46:28.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa13e60f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 0x5617886d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa13e60f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d11f0 0x5617886d9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa13e60f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d9dd0 0x5617886dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa13e60f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561788634830 con 0x5617886d11f0 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa13e60f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5617885dc820 con 0x5617886d9dd0 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa13e60f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5617886307e0 con 0x5617886d08d0 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa1327fc640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa11c0a3d20 0x7fa11c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa1396ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d11f0 0x5617886d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa1396ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d11f0 0x5617886d9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58228/0 (socket says 172.21.15.33:58228) 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa1396ec640 1 -- 172.21.15.33:0/2319986648 learned_addr learned my addr 172.21.15.33:0/2319986648 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa139eed640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa11c0a3d20 0x7fa11c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa139eed640 1 --2- 172.21.15.33:0/2319986648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 0x5617886d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa138eeb640 1 --2- 172.21.15.33:0/2319986648 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d9dd0 0x5617886dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa1327fc640 1 -- 172.21.15.33:0/1176028286 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fa12c0d0cd0 con 0x7fa134100970 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa1396ec640 1 -- 172.21.15.33:0/2319986648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 msgr2=0x5617886d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa1396ec640 1 --2- 172.21.15.33:0/2319986648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 0x5617886d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa1396ec640 1 -- 172.21.15.33:0/2319986648 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d9dd0 msgr2=0x5617886dc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa1396ec640 1 --2- 172.21.15.33:0/2319986648 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d9dd0 0x5617886dc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.342+0000 7fa1396ec640 1 -- 172.21.15.33:0/2319986648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617884127d0 con 0x5617886d11f0 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa139eed640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa11c0a3d20 0x7fa11c0a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa124009270 tx=0x7fa124009e30 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa1396ec640 1 --2- 172.21.15.33:0/2319986648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d11f0 0x5617886d9890 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00fe30 tx=0x7fa12c09c200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725db068805c3543 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa11b7fe640 1 -- 172.21.15.33:0/2319986648 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c09cfa0 con 0x5617886d11f0 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa11b7fe640 1 -- 172.21.15.33:0/2319986648 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa12c0a5510 con 0x5617886d11f0 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa11b7fe640 1 -- 172.21.15.33:0/2319986648 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0ce2d0 con 0x5617886d11f0 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa13e60f740 1 -- 172.21.15.33:0/2319986648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d11f0 msgr2=0x5617886d9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa13e60f740 1 --2- 172.21.15.33:0/2319986648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d11f0 0x5617886d9890 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00fe30 tx=0x7fa12c09c200 comp rx=0 tx=0).stop 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa13e60f740 1 -- 172.21.15.33:0/2319986648 shutdown_connections 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa13e60f740 1 --2- 172.21.15.33:0/2319986648 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d9dd0 0x5617886dc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa13e60f740 1 --2- 172.21.15.33:0/2319986648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d11f0 0x5617886d9890 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa13e60f740 1 --2- 172.21.15.33:0/2319986648 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 0x5617886d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa13e60f740 1 -- 172.21.15.33:0/2319986648 >> 172.21.15.33:0/2319986648 conn(0x5617886cb000 msgr2=0x5617886cb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa13e60f740 1 -- 172.21.15.33:0/2319986648 shutdown_connections 2024-04-02T13:46:28.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.343+0000 7fa13e60f740 1 -- 172.21.15.33:0/2319986648 wait complete. 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.344+0000 7fa13e60f740 1 -- start start 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.344+0000 7fa13e60f740 1 Processor -- start 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.344+0000 7fa13e60f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 0x5617886d9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.344+0000 7fa13e60f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d11f0 0x5617886d69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.344+0000 7fa13e60f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.344+0000 7fa13e60f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5617886dcb60 con 0x5617886d9dd0 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.344+0000 7fa13e60f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5617885efd30 con 0x5617886d11f0 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.344+0000 7fa13e60f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5617886dcce0 con 0x5617886d08d0 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.344+0000 7fa13e60f740 10 client.0 _ll_register_callbacks cb 0x5617885ef720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.344+0000 7fa139eed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.344+0000 7fa139eed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58242/0 (socket says 172.21.15.33:58242) 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.344+0000 7fa139eed640 1 -- 172.21.15.33:0/2970827111 learned_addr learned my addr 172.21.15.33:0/2970827111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.344+0000 7fa138eeb640 1 --2- 172.21.15.33:0/2970827111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d11f0 0x5617886d69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa1396ec640 1 --2- 172.21.15.33:0/2970827111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 0x5617886d9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa139eed640 1 -- 172.21.15.33:0/2970827111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 msgr2=0x5617886d9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa139eed640 1 --2- 172.21.15.33:0/2970827111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 0x5617886d9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa139eed640 1 -- 172.21.15.33:0/2970827111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d11f0 msgr2=0x5617886d69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa139eed640 1 --2- 172.21.15.33:0/2970827111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d11f0 0x5617886d69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa139eed640 1 -- 172.21.15.33:0/2970827111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617883483f0 con 0x5617886d9dd0 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa139eed640 1 --2- 172.21.15.33:0/2970827111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d6f10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fa124012fd0 tx=0x7fa124009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f449f3ed7536d7de server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa118ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa118ff9640 1 -- 172.21.15.33:0/2970827111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1240044c0 con 0x5617886d9dd0 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa118ff9640 1 -- 172.21.15.33:0/2970827111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fa124004660 con 0x5617886d9dd0 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa13e60f740 1 -- 172.21.15.33:0/2970827111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa12c0a5510 con 0x5617886d9dd0 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa118ff9640 1 -- 172.21.15.33:0/2970827111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa124032050 con 0x5617886d9dd0 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa13e60f740 10 client.17319 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa13e60f740 1 -- 172.21.15.33:0/2970827111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5617883080f0 con 0x5617886d9dd0 2024-04-02T13:46:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.345+0000 7fa118ff9640 1 -- 172.21.15.33:0/2970827111 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa12c0a5510 con 0x5617886d9dd0 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.346+0000 7fa118ff9640 1 -- 172.21.15.33:0/2970827111 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fa12401f050 con 0x5617886d9dd0 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.346+0000 7fa13e60f740 10 client.17319 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.346+0000 7fa13e60f740 10 client.17319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.346+0000 7fa13e60f740 20 client.17319 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.346+0000 7fa13e60f740 1 --2- 172.21.15.33:0/2970827111 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5617886e9cf0 0x5617886ec150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.346+0000 7fa13e60f740 4 client.17319 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.346+0000 7fa13e60f740 1 -- 172.21.15.33:0/2970827111 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561788641d70 con 0x5617886e9cf0 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.346+0000 7fa1396ec640 1 --2- 172.21.15.33:0/2970827111 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5617886e9cf0 0x5617886ec150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.346+0000 7fa118ff9640 10 client.17319 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.346+0000 7fa1396ec640 1 --2- 172.21.15.33:0/2970827111 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5617886e9cf0 0x5617886ec150 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d9cf6050171c25 server_cookie=4f18eeb91f5f0435 in_seq=0 out_seq=0 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa118ff9640 1 -- 172.21.15.33:0/2970827111 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561788641d70 con 0x5617886e9cf0 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa118ff9640 10 client.17319 handle_command_reply: tid=0 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa13e60f740 1 client.17319 shutdown 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa13e60f740 1 -- 172.21.15.33:0/2970827111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 msgr2=0x5617886d6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa13e60f740 1 --2- 172.21.15.33:0/2970827111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d6f10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fa124012fd0 tx=0x7fa124009500 comp rx=0 tx=0).stop 2024-04-02T13:46:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa13e60f740 1 -- 172.21.15.33:0/2970827111 shutdown_connections 2024-04-02T13:46:28.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa13e60f740 1 --2- 172.21.15.33:0/2970827111 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5617886e9cf0 0x5617886ec150 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa13e60f740 1 --2- 172.21.15.33:0/2970827111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d6f10 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa13e60f740 1 --2- 172.21.15.33:0/2970827111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d11f0 0x5617886d69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa13e60f740 1 --2- 172.21.15.33:0/2970827111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 0x5617886d9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa118ff9640 0 client.17319 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:28.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa13e60f740 1 -- 172.21.15.33:0/2970827111 >> 172.21.15.33:0/2970827111 conn(0x5617886cb000 msgr2=0x5617886ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:28.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa13e60f740 1 -- 172.21.15.33:0/2970827111 shutdown_connections 2024-04-02T13:46:28.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa13e60f740 1 -- 172.21.15.33:0/2970827111 wait complete. 2024-04-02T13:46:28.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.347+0000 7fa13e60f740 20 client.17319 trim_cache size 0 max 16384 2024-04-02T13:46:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa13e60f740 1 Processor -- start 2024-04-02T13:46:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa13e60f740 1 -- start start 2024-04-02T13:46:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa13e60f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 0x5617886c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa13e60f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d11f0 0x5617886e2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa13e60f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa13e60f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341b1880 con 0x5617886d9dd0 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa13e60f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341af150 con 0x5617886d11f0 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa13e60f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341acfc0 con 0x5617886d08d0 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa139eed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa139eed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58248/0 (socket says 172.21.15.33:58248) 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa139eed640 1 -- 172.21.15.33:0/174453417 learned_addr learned my addr 172.21.15.33:0/174453417 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa1396ec640 1 --2- 172.21.15.33:0/174453417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 0x5617886c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa138eeb640 1 --2- 172.21.15.33:0/174453417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d11f0 0x5617886e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa139eed640 1 -- 172.21.15.33:0/174453417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 msgr2=0x5617886c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa139eed640 1 --2- 172.21.15.33:0/174453417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 0x5617886c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa139eed640 1 -- 172.21.15.33:0/174453417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d11f0 msgr2=0x5617886e2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa139eed640 1 --2- 172.21.15.33:0/174453417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d11f0 0x5617886e2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa139eed640 1 -- 172.21.15.33:0/174453417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617886ca860 con 0x5617886d9dd0 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.365+0000 7fa1396ec640 1 --2- 172.21.15.33:0/174453417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 0x5617886c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:28.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.366+0000 7fa139eed640 1 --2- 172.21.15.33:0/174453417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d9100 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa12402bff0 tx=0x7fa124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb29655d48d8f2b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.366+0000 7fa11affd640 1 -- 172.21.15.33:0/174453417 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124012800 con 0x5617886d9dd0 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.366+0000 7fa11affd640 1 -- 172.21.15.33:0/174453417 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa124024070 con 0x5617886d9dd0 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.366+0000 7fa11affd640 1 -- 172.21.15.33:0/174453417 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124011c30 con 0x5617886d9dd0 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.366+0000 7fa13e60f740 1 -- 172.21.15.33:0/174453417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 msgr2=0x5617886d9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.366+0000 7fa13e60f740 1 --2- 172.21.15.33:0/174453417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d9100 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa12402bff0 tx=0x7fa124004a40 comp rx=0 tx=0).stop 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.366+0000 7fa13e60f740 1 -- 172.21.15.33:0/174453417 shutdown_connections 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.366+0000 7fa13e60f740 1 --2- 172.21.15.33:0/174453417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d9100 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.366+0000 7fa13e60f740 1 --2- 172.21.15.33:0/174453417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d11f0 0x5617886e2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.366+0000 7fa13e60f740 1 --2- 172.21.15.33:0/174453417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d08d0 0x5617886c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.366+0000 7fa13e60f740 1 -- 172.21.15.33:0/174453417 >> 172.21.15.33:0/174453417 conn(0x5617886cb000 msgr2=0x5617886d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.366+0000 7fa13e60f740 1 -- 172.21.15.33:0/174453417 shutdown_connections 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.366+0000 7fa13e60f740 1 -- 172.21.15.33:0/174453417 wait complete. 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa13e60f740 1 -- start start 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa13e60f740 1 Processor -- start 2024-04-02T13:46:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa13e60f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d08d0 0x5617886d42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa13e60f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d11f0 0x5617886d2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa13e60f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa13e60f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341ad610 con 0x5617886d9dd0 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa13e60f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341aea80 con 0x5617886d08d0 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa13e60f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341adce0 con 0x5617886d11f0 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa13e60f740 10 client.0 _ll_register_callbacks cb 0x7fa124009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa139eed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa139eed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58262/0 (socket says 172.21.15.33:58262) 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa139eed640 1 -- 172.21.15.33:0/228125195 learned_addr learned my addr 172.21.15.33:0/228125195 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa138eeb640 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d11f0 0x5617886d2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa1396ec640 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d08d0 0x5617886d42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa139eed640 1 -- 172.21.15.33:0/228125195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d11f0 msgr2=0x5617886d2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa139eed640 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d11f0 0x5617886d2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa139eed640 1 -- 172.21.15.33:0/228125195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d08d0 msgr2=0x5617886d42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:28.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa139eed640 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d08d0 0x5617886d42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa139eed640 1 -- 172.21.15.33:0/228125195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617886d3740 con 0x5617886d9dd0 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa1396ec640 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d08d0 0x5617886d42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.367+0000 7fa138eeb640 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d11f0 0x5617886d2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa139eed640 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d3180 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa124010880 tx=0x7fa124009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf04f2e2bfd2442 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa118ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa118ff9640 1 -- 172.21.15.33:0/228125195 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124004410 con 0x5617886d9dd0 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa13e60f740 1 -- 172.21.15.33:0/228125195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa124024070 con 0x5617886d9dd0 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa118ff9640 1 -- 172.21.15.33:0/228125195 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fa1240045b0 con 0x5617886d9dd0 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa118ff9640 1 -- 172.21.15.33:0/228125195 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa124046050 con 0x5617886d9dd0 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa13e60f740 10 client.17325 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa13e60f740 1 -- 172.21.15.33:0/228125195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561788755090 con 0x5617886d9dd0 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa118ff9640 1 -- 172.21.15.33:0/228125195 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa124024070 con 0x5617886d9dd0 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa118ff9640 1 -- 172.21.15.33:0/228125195 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fa1240298c0 con 0x5617886d9dd0 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa13e60f740 10 client.17325 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:28.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa13e60f740 10 client.17325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa13e60f740 20 client.17325 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa13e60f740 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x561788755690 0x561788757a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa13e60f740 4 client.17325 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.368+0000 7fa13e60f740 1 -- 172.21.15.33:0/228125195 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56178863c0c0 con 0x561788755690 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa1396ec640 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x561788755690 0x561788757a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa118ff9640 10 client.17325 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa1396ec640 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x561788755690 0x561788757a80 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3836436e0468659 server_cookie=cff933fe23a7673d in_seq=0 out_seq=0 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa118ff9640 1 -- 172.21.15.33:0/228125195 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56178863c0c0 con 0x561788755690 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa118ff9640 10 client.17325 handle_command_reply: tid=0 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa13e60f740 1 client.17325 shutdown 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa13e60f740 1 -- 172.21.15.33:0/228125195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 msgr2=0x5617886d3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa13e60f740 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d3180 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa124010880 tx=0x7fa124009570 comp rx=0 tx=0).stop 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa13e60f740 1 -- 172.21.15.33:0/228125195 shutdown_connections 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa13e60f740 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x561788755690 0x561788757a80 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa13e60f740 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617886d9dd0 0x5617886d3180 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa118ff9640 0 client.17325 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa13e60f740 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5617886d11f0 0x5617886d2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa13e60f740 1 --2- 172.21.15.33:0/228125195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617886d08d0 0x5617886d42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa13e60f740 1 -- 172.21.15.33:0/228125195 >> 172.21.15.33:0/228125195 conn(0x5617886cb000 msgr2=0x5617886cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa13e60f740 1 -- 172.21.15.33:0/228125195 shutdown_connections 2024-04-02T13:46:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa13e60f740 1 -- 172.21.15.33:0/228125195 wait complete. 2024-04-02T13:46:28.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.369+0000 7fa13e60f740 20 client.17325 trim_cache size 0 max 16384 2024-04-02T13:46:28.379 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:46:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:46:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:46:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "5048311b-e895-4cdc-b531-fb4e7679b624": { 2024-04-02T13:46:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:46:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "5048311b-e895-4cdc-b531-fb4e7679b624", 2024-04-02T13:46:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:46:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:46:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:46:28.381 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:46:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.370+0000 7fa13b950640 1 -- 172.21.15.33:0/1176028286 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa11c0a3d20 msgr2=0x7fa11c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.370+0000 7fa13b950640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa11c0a3d20 0x7fa11c0a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa124009270 tx=0x7fa124009e30 comp rx=0 tx=0).stop 2024-04-02T13:46:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.370+0000 7fa13b950640 1 -- 172.21.15.33:0/1176028286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa134100970 msgr2=0x7fa134198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.370+0000 7fa13b950640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa134100970 0x7fa134198380 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00c920 tx=0x7fa12c00cde0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.370+0000 7fa13b950640 1 -- 172.21.15.33:0/1176028286 shutdown_connections 2024-04-02T13:46:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.370+0000 7fa13b950640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa11c0a3d20 0x7fa11c0a61e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.370+0000 7fa13b950640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa134106980 0x7fa13419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.370+0000 7fa13b950640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa134101330 0x7fa1341988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.370+0000 7fa13b950640 1 --2- 172.21.15.33:0/1176028286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa134100970 0x7fa134198380 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.370+0000 7fa13b950640 1 -- 172.21.15.33:0/1176028286 >> 172.21.15.33:0/1176028286 conn(0x7fa134077b00 msgr2=0x7fa134068f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.370+0000 7fa13b950640 1 -- 172.21.15.33:0/1176028286 shutdown_connections 2024-04-02T13:46:28.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:28.370+0000 7fa13b950640 1 -- 172.21.15.33:0/1176028286 wait complete. 2024-04-02T13:46:28.384 INFO:tasks.cephfs.filesystem:scrub status for tag:5048311b-e895-4cdc-b531-fb4e7679b624 - {'path': '/', 'tag': '5048311b-e895-4cdc-b531-fb4e7679b624', 'options': 'recursive,force'} 2024-04-02T13:46:28.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:28 smithi033 ceph-mon[32390]: pgmap v237: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:28.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:28 smithi145 ceph-mon[35646]: pgmap v237: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:28.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:28 smithi050 ceph-mon[35524]: pgmap v237: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:29.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:29 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:46:29.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:29 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:29.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:29 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:46:29.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:29 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:29.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:29 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:46:29.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:29 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:29.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:29 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:46:29.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:29 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:30.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:29 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:46:30.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:29 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:30.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:29 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:46:30.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:29 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:30.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:30 smithi050 ceph-mon[35524]: pgmap v238: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:46:30.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:30 smithi145 ceph-mon[35646]: pgmap v238: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:46:31.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:30 smithi033 ceph-mon[32390]: pgmap v238: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:46:31.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:31 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:46:31.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:31 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:46:32.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:31 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:46:32.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:32 smithi145 ceph-mon[35646]: pgmap v239: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:32 smithi050 ceph-mon[35524]: pgmap v239: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:33.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:32 smithi033 ceph-mon[32390]: pgmap v239: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:34.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:34 smithi145 ceph-mon[35646]: pgmap v240: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:46:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:34 smithi050 ceph-mon[35524]: pgmap v240: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:46:35.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:34 smithi033 ceph-mon[32390]: pgmap v240: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:46:36.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:36 smithi145 ceph-mon[35646]: pgmap v241: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:46:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:36 smithi050 ceph-mon[35524]: pgmap v241: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:46:37.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:36 smithi033 ceph-mon[32390]: pgmap v241: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:46:38.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:38 smithi145 ceph-mon[35646]: pgmap v242: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:46:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:38 smithi050 ceph-mon[35524]: pgmap v242: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:46:39.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:38 smithi033 ceph-mon[32390]: pgmap v242: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:46:40.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:40 smithi145 ceph-mon[35646]: pgmap v243: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:46:40.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:40 smithi050 ceph-mon[35524]: pgmap v243: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:46:41.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:40 smithi033 ceph-mon[32390]: pgmap v243: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:46:42.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:42 smithi145 ceph-mon[35646]: pgmap v244: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:46:42.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:42 smithi050 ceph-mon[35524]: pgmap v244: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:46:43.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:42 smithi033 ceph-mon[32390]: pgmap v244: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:46:44.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:44 smithi145 ceph-mon[35646]: pgmap v245: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:46:44.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:44 smithi050 ceph-mon[35524]: pgmap v245: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:46:45.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:44 smithi033 ceph-mon[32390]: pgmap v245: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:46:46.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:46 smithi145 ceph-mon[35646]: pgmap v246: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:46 smithi050 ceph-mon[35524]: pgmap v246: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:47.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:46 smithi033 ceph-mon[32390]: pgmap v246: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:48.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:48 smithi145 ceph-mon[35646]: pgmap v247: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:48.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:48 smithi050 ceph-mon[35524]: pgmap v247: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:49.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:48 smithi033 ceph-mon[32390]: pgmap v247: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:50.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:50 smithi145 ceph-mon[35646]: pgmap v248: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:46:50.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:50 smithi050 ceph-mon[35524]: pgmap v248: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:46:51.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:50 smithi033 ceph-mon[32390]: pgmap v248: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:46:52.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:52 smithi145 ceph-mon[35646]: pgmap v249: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:52.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:52 smithi050 ceph-mon[35524]: pgmap v249: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:53.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:52 smithi033 ceph-mon[32390]: pgmap v249: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:54.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:54 smithi145 ceph-mon[35646]: pgmap v250: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:46:54.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:54 smithi050 ceph-mon[35524]: pgmap v250: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:46:55.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:54 smithi033 ceph-mon[32390]: pgmap v250: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:46:56.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:56 smithi145 ceph-mon[35646]: pgmap v251: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:56.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:56 smithi050 ceph-mon[35524]: pgmap v251: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:57.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:56 smithi033 ceph-mon[32390]: pgmap v251: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:58.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:46:58.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:46:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.489+0000 7fecf2a77640 1 -- 172.21.15.33:0/517163530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecec068d00 msgr2=0x7fecec069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.489+0000 7fecf2a77640 1 --2- 172.21.15.33:0/517163530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecec068d00 0x7fecec069180 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009930 tx=0x7fecd802bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.489+0000 7fecf2a77640 1 -- 172.21.15.33:0/517163530 shutdown_connections 2024-04-02T13:46:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.489+0000 7fecf2a77640 1 --2- 172.21.15.33:0/517163530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecec0696c0 0x7fecec10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.489+0000 7fecf2a77640 1 --2- 172.21.15.33:0/517163530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecec068d00 0x7fecec069180 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.489+0000 7fecf2a77640 1 --2- 172.21.15.33:0/517163530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecec107b50 0x7fecec107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.489+0000 7fecf2a77640 1 -- 172.21.15.33:0/517163530 >> 172.21.15.33:0/517163530 conn(0x7fecec077b00 msgr2=0x7fecec077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.489+0000 7fecf2a77640 1 -- 172.21.15.33:0/517163530 shutdown_connections 2024-04-02T13:46:58.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.489+0000 7fecf2a77640 1 -- 172.21.15.33:0/517163530 wait complete. 2024-04-02T13:46:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.489+0000 7fecf2a77640 1 Processor -- start 2024-04-02T13:46:58.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.489+0000 7fecf2a77640 1 -- start start 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf2a77640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecec068d00 0x7fecec198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf2a77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecec0696c0 0x7fecec1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf2a77640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecec107b50 0x7fecec19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf2a77640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fecec111e70 con 0x7fecec0696c0 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf2a77640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fecec111cf0 con 0x7fecec068d00 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf2a77640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fecec111ff0 con 0x7fecec107b50 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecebfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecec0696c0 0x7fecec1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf0813640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecec068d00 0x7fecec198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecebfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecec0696c0 0x7fecec1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41156/0 (socket says 172.21.15.33:41156) 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecebfff640 1 -- 172.21.15.33:0/2056356062 learned_addr learned my addr 172.21.15.33:0/2056356062 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf1014640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecec107b50 0x7fecec19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf0813640 1 -- 172.21.15.33:0/2056356062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecec107b50 msgr2=0x7fecec19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf0813640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecec107b50 0x7fecec19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf0813640 1 -- 172.21.15.33:0/2056356062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecec0696c0 msgr2=0x7fecec1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf0813640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecec0696c0 0x7fecec1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf0813640 1 -- 172.21.15.33:0/2056356062 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec19d330 con 0x7fecec068d00 2024-04-02T13:46:58.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecebfff640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecec0696c0 0x7fecec1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:58.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.490+0000 7fecf0813640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecec068d00 0x7fecec198390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fece000e9f0 tx=0x7fece000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:58.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.491+0000 7fece9ffb640 1 -- 172.21.15.33:0/2056356062 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece000ccc0 con 0x7fecec068d00 2024-04-02T13:46:58.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.491+0000 7fecf2a77640 1 -- 172.21.15.33:0/2056356062 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecec19d620 con 0x7fecec068d00 2024-04-02T13:46:58.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.491+0000 7fece9ffb640 1 -- 172.21.15.33:0/2056356062 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fece0010900 con 0x7fecec068d00 2024-04-02T13:46:58.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.491+0000 7fece9ffb640 1 -- 172.21.15.33:0/2056356062 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0018960 con 0x7fecec068d00 2024-04-02T13:46:58.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.491+0000 7fecf2a77640 1 -- 172.21.15.33:0/2056356062 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecec1a4eb0 con 0x7fecec068d00 2024-04-02T13:46:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.493+0000 7fece9ffb640 1 -- 172.21.15.33:0/2056356062 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fece0010ab0 con 0x7fecec068d00 2024-04-02T13:46:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.493+0000 7fecf565a740 1 Processor -- start 2024-04-02T13:46:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.493+0000 7fecf565a740 1 -- start start 2024-04-02T13:46:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.493+0000 7fecf565a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4648d0 0x55889f464cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.493+0000 7fecf565a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f4651f0 0x55889f46d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.493+0000 7fecf565a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f46ddd0 0x55889f4701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.493+0000 7fecf565a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55889f3c8830 con 0x55889f4651f0 2024-04-02T13:46:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.493+0000 7fecf565a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55889f370820 con 0x55889f4648d0 2024-04-02T13:46:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.493+0000 7fecf565a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55889f3c47e0 con 0x55889f46ddd0 2024-04-02T13:46:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fece9ffb640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fecdc0a3d20 0x7fecdc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecf1014640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4648d0 0x55889f464cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecf0813640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f4651f0 0x55889f46d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecf1014640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fecdc0a3d20 0x7fecdc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecebfff640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f46ddd0 0x55889f4701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecf1014640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4648d0 0x55889f464cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:38766/0 (socket says 172.21.15.33:38766) 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecf1014640 1 -- 172.21.15.33:0/1702314540 learned_addr learned my addr 172.21.15.33:0/1702314540 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fece9ffb640 1 -- 172.21.15.33:0/2056356062 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fece00d0f00 con 0x7fecec068d00 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecf1014640 1 -- 172.21.15.33:0/1702314540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f46ddd0 msgr2=0x55889f4701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecf1014640 1 --2- 172.21.15.33:0/1702314540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f46ddd0 0x55889f4701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecf1014640 1 -- 172.21.15.33:0/1702314540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f4651f0 msgr2=0x55889f46d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecf1014640 1 --2- 172.21.15.33:0/1702314540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f4651f0 0x55889f46d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecf1014640 1 -- 172.21.15.33:0/1702314540 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889f1a67d0 con 0x55889f4648d0 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecf0813640 1 --2- 172.21.15.33:0/1702314540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f4651f0 0x55889f46d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecf1014640 1 --2- 172.21.15.33:0/1702314540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4648d0 0x55889f464cb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fece4012d90 tx=0x7fece400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2d7fc561894fc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecf1014640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fecdc0a3d20 0x7fecdc0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fece400fc30 tx=0x7fece4017000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecd6ffd640 1 -- 172.21.15.33:0/1702314540 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4022070 con 0x55889f4648d0 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecd6ffd640 1 -- 172.21.15.33:0/1702314540 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fece4012870 con 0x55889f4648d0 2024-04-02T13:46:58.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.494+0000 7fecd6ffd640 1 -- 172.21.15.33:0/1702314540 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4003ca0 con 0x55889f4648d0 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.495+0000 7fecf565a740 1 -- 172.21.15.33:0/1702314540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4648d0 msgr2=0x55889f464cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.495+0000 7fecf565a740 1 --2- 172.21.15.33:0/1702314540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4648d0 0x55889f464cb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fece4012d90 tx=0x7fece400f730 comp rx=0 tx=0).stop 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.495+0000 7fecf565a740 1 -- 172.21.15.33:0/1702314540 shutdown_connections 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.495+0000 7fecf565a740 1 --2- 172.21.15.33:0/1702314540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f46ddd0 0x55889f4701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.495+0000 7fecf565a740 1 --2- 172.21.15.33:0/1702314540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f4651f0 0x55889f46d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.495+0000 7fecf565a740 1 --2- 172.21.15.33:0/1702314540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4648d0 0x55889f464cb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.495+0000 7fecf565a740 1 -- 172.21.15.33:0/1702314540 >> 172.21.15.33:0/1702314540 conn(0x55889f45f000 msgr2=0x55889f45f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.495+0000 7fecf565a740 1 -- 172.21.15.33:0/1702314540 shutdown_connections 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.495+0000 7fecf565a740 1 -- 172.21.15.33:0/1702314540 wait complete. 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.495+0000 7fecf565a740 1 -- start start 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf565a740 1 Processor -- start 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf565a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4648d0 0x55889f46d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf565a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4651f0 0x55889f46aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf565a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f46afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf565a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55889f470b60 con 0x55889f46ddd0 2024-04-02T13:46:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf565a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55889f383d30 con 0x55889f4651f0 2024-04-02T13:46:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf565a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55889f470ce0 con 0x55889f4648d0 2024-04-02T13:46:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf565a740 10 client.0 _ll_register_callbacks cb 0x55889f383720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf1014640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f46afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecebfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4651f0 0x55889f46aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf0813640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4648d0 0x55889f46d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecebfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4651f0 0x55889f46aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:38778/0 (socket says 172.21.15.33:38778) 2024-04-02T13:46:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf1014640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f46afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41168/0 (socket says 172.21.15.33:41168) 2024-04-02T13:46:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecebfff640 1 -- 172.21.15.33:0/2592386022 learned_addr learned my addr 172.21.15.33:0/2592386022 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf0813640 1 -- 172.21.15.33:0/2592386022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4651f0 msgr2=0x55889f46aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf0813640 1 --2- 172.21.15.33:0/2592386022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4651f0 0x55889f46aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf0813640 1 -- 172.21.15.33:0/2592386022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 msgr2=0x55889f46afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf0813640 1 --2- 172.21.15.33:0/2592386022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f46afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.496+0000 7fecf0813640 1 -- 172.21.15.33:0/2592386022 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889f0dc3f0 con 0x55889f4648d0 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7fecf0813640 1 --2- 172.21.15.33:0/2592386022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4648d0 0x55889f46d710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fece00ce220 tx=0x7fece006c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb6d2d158ba4dcc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7feccbfff640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7feccbfff640 1 -- 172.21.15.33:0/2592386022 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece00ce6b0 con 0x55889f4648d0 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7feccbfff640 1 -- 172.21.15.33:0/2592386022 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fece009cf80 con 0x55889f4648d0 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7fecf565a740 1 -- 172.21.15.33:0/2592386022 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fece4012870 con 0x55889f4648d0 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7feccbfff640 1 -- 172.21.15.33:0/2592386022 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fece009cb20 con 0x55889f4648d0 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7fecf565a740 10 client.26347 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7fecf565a740 1 -- 172.21.15.33:0/2592386022 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55889f09c0f0 con 0x55889f4648d0 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7feccbfff640 1 -- 172.21.15.33:0/2592386022 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fece4012870 con 0x55889f4648d0 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7feccbfff640 1 -- 172.21.15.33:0/2592386022 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fece006e0b0 con 0x55889f4648d0 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7fecf565a740 10 client.26347 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7fecf565a740 10 client.26347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7fecf565a740 20 client.26347 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7fecf565a740 1 --2- 172.21.15.33:0/2592386022 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55889f47dda0 0x55889f4801d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7fecf565a740 4 client.26347 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.497+0000 7fecf565a740 1 -- 172.21.15.33:0/2592386022 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55889f3d5d70 con 0x55889f47dda0 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.498+0000 7fecebfff640 1 --2- 172.21.15.33:0/2592386022 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55889f47dda0 0x55889f4801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.498+0000 7feccbfff640 10 client.26347 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.498+0000 7fecebfff640 1 --2- 172.21.15.33:0/2592386022 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55889f47dda0 0x55889f4801d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdb36effc7ae37d7 server_cookie=247970ad2fc3c1b8 in_seq=0 out_seq=0 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7feccbfff640 1 -- 172.21.15.33:0/2592386022 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55889f3d5d70 con 0x55889f47dda0 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7feccbfff640 10 client.26347 handle_command_reply: tid=0 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7fecf565a740 1 client.26347 shutdown 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7fecf565a740 1 -- 172.21.15.33:0/2592386022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4648d0 msgr2=0x55889f46d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7fecf565a740 1 --2- 172.21.15.33:0/2592386022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4648d0 0x55889f46d710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fece00ce220 tx=0x7fece006c400 comp rx=0 tx=0).stop 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7fecf565a740 1 -- 172.21.15.33:0/2592386022 shutdown_connections 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7fecf565a740 1 --2- 172.21.15.33:0/2592386022 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55889f47dda0 0x55889f4801d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7fecf565a740 1 --2- 172.21.15.33:0/2592386022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f46afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7feccbfff640 0 client.26347 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7fecf565a740 1 --2- 172.21.15.33:0/2592386022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4651f0 0x55889f46aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7fecf565a740 1 --2- 172.21.15.33:0/2592386022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4648d0 0x55889f46d710 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7fecf565a740 1 -- 172.21.15.33:0/2592386022 >> 172.21.15.33:0/2592386022 conn(0x55889f45f000 msgr2=0x55889f460d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:58.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7fecf565a740 1 -- 172.21.15.33:0/2592386022 shutdown_connections 2024-04-02T13:46:58.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7fecf565a740 1 -- 172.21.15.33:0/2592386022 wait complete. 2024-04-02T13:46:58.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.499+0000 7fecf565a740 20 client.26347 trim_cache size 0 max 16384 2024-04-02T13:46:58.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.516+0000 7fecf565a740 1 Processor -- start 2024-04-02T13:46:58.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf565a740 1 -- start start 2024-04-02T13:46:58.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf565a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4648d0 0x55889f45e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf565a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4651f0 0x55889f46d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf565a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f46d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf565a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fecec1b18a0 con 0x55889f46ddd0 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf565a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fecec1af170 con 0x55889f4648d0 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf565a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fecec1acfc0 con 0x55889f4651f0 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf1014640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f46d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf1014640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f46d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41170/0 (socket says 172.21.15.33:41170) 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf1014640 1 -- 172.21.15.33:0/1924145036 learned_addr learned my addr 172.21.15.33:0/1924145036 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf0813640 1 --2- 172.21.15.33:0/1924145036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4648d0 0x55889f45e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecebfff640 1 --2- 172.21.15.33:0/1924145036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4651f0 0x55889f46d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf1014640 1 -- 172.21.15.33:0/1924145036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4651f0 msgr2=0x55889f46d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf1014640 1 --2- 172.21.15.33:0/1924145036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4651f0 0x55889f46d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf1014640 1 -- 172.21.15.33:0/1924145036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4648d0 msgr2=0x55889f45e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf1014640 1 --2- 172.21.15.33:0/1924145036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4648d0 0x55889f45e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf1014640 1 -- 172.21.15.33:0/1924145036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889f45ac50 con 0x55889f46ddd0 2024-04-02T13:46:58.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.517+0000 7fecf1014640 1 --2- 172.21.15.33:0/1924145036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f46d710 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fece4012c30 tx=0x7fece4009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f14c46bbc8cb1f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.518+0000 7fecd67fc640 1 -- 172.21.15.33:0/1924145036 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4022040 con 0x55889f46ddd0 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.518+0000 7fecd67fc640 1 -- 172.21.15.33:0/1924145036 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fece4003e40 con 0x55889f46ddd0 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.518+0000 7fecd67fc640 1 -- 172.21.15.33:0/1924145036 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece40052d0 con 0x55889f46ddd0 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.518+0000 7fecf565a740 1 -- 172.21.15.33:0/1924145036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 msgr2=0x55889f46d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.518+0000 7fecf565a740 1 --2- 172.21.15.33:0/1924145036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f46d710 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fece4012c30 tx=0x7fece4009d60 comp rx=0 tx=0).stop 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.518+0000 7fecf565a740 1 -- 172.21.15.33:0/1924145036 shutdown_connections 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.518+0000 7fecf565a740 1 --2- 172.21.15.33:0/1924145036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f46d710 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.518+0000 7fecf565a740 1 --2- 172.21.15.33:0/1924145036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4651f0 0x55889f46d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.518+0000 7fecf565a740 1 --2- 172.21.15.33:0/1924145036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4648d0 0x55889f45e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.518+0000 7fecf565a740 1 -- 172.21.15.33:0/1924145036 >> 172.21.15.33:0/1924145036 conn(0x55889f45f000 msgr2=0x55889f460300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.518+0000 7fecf565a740 1 -- 172.21.15.33:0/1924145036 shutdown_connections 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.518+0000 7fecf565a740 1 -- 172.21.15.33:0/1924145036 wait complete. 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecf565a740 1 -- start start 2024-04-02T13:46:58.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecf565a740 1 Processor -- start 2024-04-02T13:46:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecf565a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4648d0 0x55889f467810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecf565a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4651f0 0x55889f467d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecf565a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f468290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecf565a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fecec1ad630 con 0x55889f46ddd0 2024-04-02T13:46:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecf565a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fecec1aeaa0 con 0x55889f4651f0 2024-04-02T13:46:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecf565a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fecec1add00 con 0x55889f4648d0 2024-04-02T13:46:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecf565a740 10 client.0 _ll_register_callbacks cb 0x7fece009c660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecf1014640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f468290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecebfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4651f0 0x55889f467d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecf1014640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f468290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41178/0 (socket says 172.21.15.33:41178) 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecf0813640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4648d0 0x55889f467810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.519+0000 7fecf1014640 1 -- 172.21.15.33:0/1032270123 learned_addr learned my addr 172.21.15.33:0/1032270123 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7fecf1014640 1 -- 172.21.15.33:0/1032270123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4648d0 msgr2=0x55889f467810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7fecf1014640 1 --2- 172.21.15.33:0/1032270123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4648d0 0x55889f467810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7fecf1014640 1 -- 172.21.15.33:0/1032270123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4651f0 msgr2=0x55889f467d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7fecf1014640 1 --2- 172.21.15.33:0/1032270123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4651f0 0x55889f467d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7fecf1014640 1 -- 172.21.15.33:0/1032270123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889f4e3370 con 0x55889f46ddd0 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7fecebfff640 1 --2- 172.21.15.33:0/1032270123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4651f0 0x55889f467d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7fecf1014640 1 --2- 172.21.15.33:0/1032270123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f468290 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fece40092f0 tx=0x7fece400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbac59649193a5f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7feccbfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7feccbfff640 1 -- 172.21.15.33:0/1032270123 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4022040 con 0x55889f46ddd0 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7feccbfff640 1 -- 172.21.15.33:0/1032270123 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fece4012040 con 0x55889f46ddd0 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7fecf565a740 1 -- 172.21.15.33:0/1032270123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55889f3d00c0 con 0x55889f46ddd0 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7feccbfff640 1 -- 172.21.15.33:0/1032270123 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fece4012870 con 0x55889f46ddd0 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7fecf565a740 10 client.17343 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.520+0000 7fecf565a740 1 -- 172.21.15.33:0/1032270123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55889f4e4660 con 0x55889f46ddd0 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.521+0000 7feccbfff640 1 -- 172.21.15.33:0/1032270123 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55889f3d00c0 con 0x55889f46ddd0 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.521+0000 7feccbfff640 1 -- 172.21.15.33:0/1032270123 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fece400f040 con 0x55889f46ddd0 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.521+0000 7fecf565a740 10 client.17343 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.521+0000 7fecf565a740 10 client.17343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.521+0000 7fecf565a740 20 client.17343 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.521+0000 7fecf565a740 1 --2- 172.21.15.33:0/1032270123 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55889f4e4b80 0x55889f4e6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.521+0000 7fecf565a740 4 client.17343 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.521+0000 7fecf565a740 1 -- 172.21.15.33:0/1032270123 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55889f34d560 con 0x55889f4e4b80 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.521+0000 7fecf0813640 1 --2- 172.21.15.33:0/1032270123 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55889f4e4b80 0x55889f4e6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.521+0000 7feccbfff640 10 client.17343 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:58.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7fecf0813640 1 --2- 172.21.15.33:0/1032270123 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55889f4e4b80 0x55889f4e6f70 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c758fdc6a9994bd9 server_cookie=ea96d6d97f8f2a23 in_seq=0 out_seq=0 2024-04-02T13:46:58.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7feccbfff640 1 -- 172.21.15.33:0/1032270123 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55889f34d560 con 0x55889f4e4b80 2024-04-02T13:46:58.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7feccbfff640 10 client.17343 handle_command_reply: tid=0 2024-04-02T13:46:58.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7fecf565a740 1 client.17343 shutdown 2024-04-02T13:46:58.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7fecf565a740 1 -- 172.21.15.33:0/1032270123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 msgr2=0x55889f468290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7fecf565a740 1 --2- 172.21.15.33:0/1032270123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f468290 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fece40092f0 tx=0x7fece400edf0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7fecf565a740 1 -- 172.21.15.33:0/1032270123 shutdown_connections 2024-04-02T13:46:58.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7fecf565a740 1 --2- 172.21.15.33:0/1032270123 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55889f4e4b80 0x55889f4e6f70 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7fecf565a740 1 --2- 172.21.15.33:0/1032270123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55889f46ddd0 0x55889f468290 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7fecf565a740 1 --2- 172.21.15.33:0/1032270123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55889f4651f0 0x55889f467d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7fecf565a740 1 --2- 172.21.15.33:0/1032270123 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55889f4648d0 0x55889f467810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7fecf565a740 1 -- 172.21.15.33:0/1032270123 >> 172.21.15.33:0/1032270123 conn(0x55889f45f000 msgr2=0x55889f45e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:58.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7feccbfff640 0 client.17343 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:58.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7fecf565a740 1 -- 172.21.15.33:0/1032270123 shutdown_connections 2024-04-02T13:46:58.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7fecf565a740 1 -- 172.21.15.33:0/1032270123 wait complete. 2024-04-02T13:46:58.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.522+0000 7fecf565a740 20 client.17343 trim_cache size 0 max 16384 2024-04-02T13:46:58.531 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:46:58.531 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:46:58.531 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:46:58.531 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.523+0000 7fecf2a77640 1 -- 172.21.15.33:0/2056356062 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fecdc0a3d20 msgr2=0x7fecdc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.523+0000 7fecf2a77640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fecdc0a3d20 0x7fecdc0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fece400fc30 tx=0x7fece4017000 comp rx=0 tx=0).stop 2024-04-02T13:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.523+0000 7fecf2a77640 1 -- 172.21.15.33:0/2056356062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecec068d00 msgr2=0x7fecec198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.523+0000 7fecf2a77640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecec068d00 0x7fecec198390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fece000e9f0 tx=0x7fece000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.523+0000 7fecf2a77640 1 -- 172.21.15.33:0/2056356062 shutdown_connections 2024-04-02T13:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.523+0000 7fecf2a77640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fecdc0a3d20 0x7fecdc0a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.523+0000 7fecf2a77640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fecec107b50 0x7fecec19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.523+0000 7fecf2a77640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecec0696c0 0x7fecec1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.523+0000 7fecf2a77640 1 --2- 172.21.15.33:0/2056356062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecec068d00 0x7fecec198390 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.523+0000 7fecf2a77640 1 -- 172.21.15.33:0/2056356062 >> 172.21.15.33:0/2056356062 conn(0x7fecec077b00 msgr2=0x7fecec0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.523+0000 7fecf2a77640 1 -- 172.21.15.33:0/2056356062 shutdown_connections 2024-04-02T13:46:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.523+0000 7fecf2a77640 1 -- 172.21.15.33:0/2056356062 wait complete. 2024-04-02T13:46:58.537 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:46:58.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:46:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.640+0000 7f2cf54b7640 1 -- 172.21.15.33:0/1974973324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf0107ab0 msgr2=0x7f2cf0107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.640+0000 7f2cf54b7640 1 --2- 172.21.15.33:0/1974973324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf0107ab0 0x7f2cf0107e90 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009930 tx=0x7f2ce402bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.640+0000 7f2cf54b7640 1 -- 172.21.15.33:0/1974973324 shutdown_connections 2024-04-02T13:46:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.640+0000 7f2cf54b7640 1 --2- 172.21.15.33:0/1974973324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cf0102460 0x7f2cf010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.640+0000 7f2cf54b7640 1 --2- 172.21.15.33:0/1974973324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cf0101aa0 0x7f2cf0101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.640+0000 7f2cf54b7640 1 --2- 172.21.15.33:0/1974973324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf0107ab0 0x7f2cf0107e90 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.640+0000 7f2cf54b7640 1 -- 172.21.15.33:0/1974973324 >> 172.21.15.33:0/1974973324 conn(0x7f2cf00fd550 msgr2=0x7f2cf00ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.640+0000 7f2cf54b7640 1 -- 172.21.15.33:0/1974973324 shutdown_connections 2024-04-02T13:46:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.640+0000 7f2cf54b7640 1 -- 172.21.15.33:0/1974973324 wait complete. 2024-04-02T13:46:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.640+0000 7f2cf54b7640 1 Processor -- start 2024-04-02T13:46:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cf54b7640 1 -- start start 2024-04-02T13:46:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cf54b7640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cf0101aa0 0x7f2cf0073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cf54b7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cf0102460 0x7f2cf0074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cf54b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf0107ab0 0x7f2cf0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cf54b7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0111e60 con 0x7f2cf0107ab0 2024-04-02T13:46:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cf54b7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0111ce0 con 0x7f2cf0102460 2024-04-02T13:46:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cf54b7640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0111fe0 con 0x7f2cf0101aa0 2024-04-02T13:46:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf0107ab0 0x7f2cf0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf0107ab0 0x7f2cf0074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41192/0 (socket says 172.21.15.33:41192) 2024-04-02T13:46:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cef7fe640 1 -- 172.21.15.33:0/636525130 learned_addr learned my addr 172.21.15.33:0/636525130 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cee7fc640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cf0102460 0x7f2cf0074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2ceeffd640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cf0101aa0 0x7f2cf0073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cef7fe640 1 -- 172.21.15.33:0/636525130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cf0101aa0 msgr2=0x7f2cf0073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cef7fe640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cf0101aa0 0x7f2cf0073e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cef7fe640 1 -- 172.21.15.33:0/636525130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cf0102460 msgr2=0x7f2cf0074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cef7fe640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cf0102460 0x7f2cf0074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cef7fe640 1 -- 172.21.15.33:0/636525130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf01a68e0 con 0x7f2cf0107ab0 2024-04-02T13:46:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.641+0000 7f2cee7fc640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cf0102460 0x7f2cf0074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.642+0000 7f2cef7fe640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf0107ab0 0x7f2cf0074880 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009fd0 tx=0x7f2ce800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.642+0000 7f2cd7fff640 1 -- 172.21.15.33:0/636525130 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8017070 con 0x7f2cf0107ab0 2024-04-02T13:46:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.642+0000 7f2cf54b7640 1 -- 172.21.15.33:0/636525130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf01a6bd0 con 0x7f2cf0107ab0 2024-04-02T13:46:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.642+0000 7f2cd7fff640 1 -- 172.21.15.33:0/636525130 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f2ce8003e40 con 0x7f2cf0107ab0 2024-04-02T13:46:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.642+0000 7f2cd7fff640 1 -- 172.21.15.33:0/636525130 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce80106c0 con 0x7f2cf0107ab0 2024-04-02T13:46:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.642+0000 7f2cf54b7640 1 -- 172.21.15.33:0/636525130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf01a70d0 con 0x7f2cf0107ab0 2024-04-02T13:46:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.643+0000 7f2cf54b7640 1 -- 172.21.15.33:0/636525130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf0068d40 con 0x7f2cf0107ab0 2024-04-02T13:46:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.644+0000 7f2cd7fff640 1 -- 172.21.15.33:0/636525130 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2ce8003970 con 0x7f2cf0107ab0 2024-04-02T13:46:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.644+0000 7f2cd7fff640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2cd80a3d20 0x7f2cd80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.645+0000 7f2ceeffd640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2cd80a3d20 0x7f2cd80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:58.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.645+0000 7f2cd7fff640 1 -- 172.21.15.33:0/636525130 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f2ce80d0820 con 0x7f2cf0107ab0 2024-04-02T13:46:58.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.645+0000 7f2ceeffd640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2cd80a3d20 0x7f2cd80a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4000c00 tx=0x7f2ce4005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:58.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.649+0000 7f2cd7fff640 1 -- 172.21.15.33:0/636525130 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2ce8003c60 con 0x7f2cf0107ab0 2024-04-02T13:46:58.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.881+0000 7f2cf54b7640 1 -- 172.21.15.33:0/636525130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cf0075330 con 0x7f2cf0107ab0 2024-04-02T13:46:58.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.882+0000 7f2cd7fff640 1 -- 172.21.15.33:0/636525130 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11873 (secure 0 0 0) 0x7f2ce809c4a0 con 0x7f2cf0107ab0 2024-04-02T13:46:58.885 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:46:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:44:55.702357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:46:58.885 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 36 2024-04-02T13:46:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.884+0000 7f2cf54b7640 1 -- 172.21.15.33:0/636525130 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2cd80a3d20 msgr2=0x7f2cd80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.884+0000 7f2cf54b7640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2cd80a3d20 0x7f2cd80a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4000c00 tx=0x7f2ce4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.884+0000 7f2cf54b7640 1 -- 172.21.15.33:0/636525130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf0107ab0 msgr2=0x7f2cf0074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.884+0000 7f2cf54b7640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf0107ab0 0x7f2cf0074880 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009fd0 tx=0x7f2ce800b260 comp rx=0 tx=0).stop 2024-04-02T13:46:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.884+0000 7f2cf54b7640 1 -- 172.21.15.33:0/636525130 shutdown_connections 2024-04-02T13:46:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.884+0000 7f2cf54b7640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f2cd80a3d20 0x7f2cd80a61e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.884+0000 7f2cf54b7640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cf0107ab0 0x7f2cf0074880 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.884+0000 7f2cf54b7640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cf0102460 0x7f2cf0074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.884+0000 7f2cf54b7640 1 --2- 172.21.15.33:0/636525130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cf0101aa0 0x7f2cf0073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.884+0000 7f2cf54b7640 1 -- 172.21.15.33:0/636525130 >> 172.21.15.33:0/636525130 conn(0x7f2cf00fd550 msgr2=0x7f2cf00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.884+0000 7f2cf54b7640 1 -- 172.21.15.33:0/636525130 shutdown_connections 2024-04-02T13:46:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:58.884+0000 7f2cf54b7640 1 -- 172.21.15.33:0/636525130 wait complete. 2024-04-02T13:46:58.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:46:58.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:58 smithi145 ceph-mon[35646]: pgmap v252: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:58.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:58 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:58.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:58 smithi050 ceph-mon[35524]: pgmap v252: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:58.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:58 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.001+0000 7f9804ee4640 1 -- 172.21.15.33:0/3382352606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98001069c0 msgr2=0x7f9800106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.001+0000 7f9804ee4640 1 --2- 172.21.15.33:0/3382352606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98001069c0 0x7f9800106da0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009930 tx=0x7f97f002bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.001+0000 7f9804ee4640 1 -- 172.21.15.33:0/3382352606 shutdown_connections 2024-04-02T13:46:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.001+0000 7f9804ee4640 1 --2- 172.21.15.33:0/3382352606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9800101370 0x7f980010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.001+0000 7f9804ee4640 1 --2- 172.21.15.33:0/3382352606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98001009b0 0x7f9800100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.001+0000 7f9804ee4640 1 --2- 172.21.15.33:0/3382352606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98001069c0 0x7f9800106da0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.001+0000 7f9804ee4640 1 -- 172.21.15.33:0/3382352606 >> 172.21.15.33:0/3382352606 conn(0x7f9800076d80 msgr2=0x7f9800077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:58 smithi033 ceph-mon[32390]: pgmap v252: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:46:59.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:58 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:59.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.001+0000 7f9804ee4640 1 -- 172.21.15.33:0/3382352606 shutdown_connections 2024-04-02T13:46:59.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.001+0000 7f9804ee4640 1 -- 172.21.15.33:0/3382352606 wait complete. 2024-04-02T13:46:59.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f9804ee4640 1 Processor -- start 2024-04-02T13:46:59.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f9804ee4640 1 -- start start 2024-04-02T13:46:59.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f9804ee4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98001009b0 0x7f98000fe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f9804ee4640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9800101370 0x7f98000fe700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f9804ee4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98001069c0 0x7f98000fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f9804ee4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9800110d70 con 0x7f98001009b0 2024-04-02T13:46:59.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f9804ee4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9800110bf0 con 0x7f98001069c0 2024-04-02T13:46:59.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f9804ee4640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9800110ef0 con 0x7f9800101370 2024-04-02T13:46:59.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f97fe59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98001009b0 0x7f98000fe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f97fe59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98001009b0 0x7f98000fe1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41204/0 (socket says 172.21.15.33:41204) 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f97fe59c640 1 -- 172.21.15.33:0/3412346589 learned_addr learned my addr 172.21.15.33:0/3412346589 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f97fed9d640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98001069c0 0x7f98000fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f97f7fff640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9800101370 0x7f98000fe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f97fe59c640 1 -- 172.21.15.33:0/3412346589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9800101370 msgr2=0x7f98000fe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f97fe59c640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9800101370 0x7f98000fe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f97fe59c640 1 -- 172.21.15.33:0/3412346589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98001069c0 msgr2=0x7f98000fec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f97fe59c640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98001069c0 0x7f98000fec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f97fe59c640 1 -- 172.21.15.33:0/3412346589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98001a6900 con 0x7f98001009b0 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f97f7fff640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9800101370 0x7f98000fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.002+0000 7f97fed9d640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98001069c0 0x7f98000fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.003+0000 7f97fe59c640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98001009b0 0x7f98000fe1c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f97f0000c00 tx=0x7f97f00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.003+0000 7f97f77fe640 1 -- 172.21.15.33:0/3412346589 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0043020 con 0x7f98001009b0 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.003+0000 7f9804ee4640 1 -- 172.21.15.33:0/3412346589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98001a6b90 con 0x7f98001009b0 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.003+0000 7f97f77fe640 1 -- 172.21.15.33:0/3412346589 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f97f002caa0 con 0x7f98001009b0 2024-04-02T13:46:59.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.003+0000 7f97f77fe640 1 -- 172.21.15.33:0/3412346589 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0003bf0 con 0x7f98001009b0 2024-04-02T13:46:59.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.003+0000 7f9804ee4640 1 -- 172.21.15.33:0/3412346589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98001a70e0 con 0x7f98001009b0 2024-04-02T13:46:59.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.005+0000 7f97f77fe640 1 -- 172.21.15.33:0/3412346589 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f97f0046020 con 0x7f98001009b0 2024-04-02T13:46:59.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.005+0000 7f9807ba5740 1 Processor -- start 2024-04-02T13:46:59.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.005+0000 7f9807ba5740 1 -- start start 2024-04-02T13:46:59.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f9807ba5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7363497a0 0x55a736349b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97f77fe640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f97ec0a3d20 0x7f97ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f9807ba5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a73634a0c0 0x55a736352760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f9807ba5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a736352ca0 0x55a736355090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f9807ba5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a7362ad7b0 con 0x55a7363497a0 2024-04-02T13:46:59.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f9807ba5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a73624c570 con 0x55a736352ca0 2024-04-02T13:46:59.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f9807ba5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a7362a9760 con 0x55a73634a0c0 2024-04-02T13:46:59.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7363497a0 0x55a736349b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7363497a0 0x55a736349b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41214/0 (socket says 172.21.15.33:41214) 2024-04-02T13:46:59.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97fed9d640 1 -- 172.21.15.33:0/1542708900 learned_addr learned my addr 172.21.15.33:0/1542708900 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97fe59c640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f97ec0a3d20 0x7f97ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97f7fff640 1 --2- 172.21.15.33:0/1542708900 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a73634a0c0 0x55a736352760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97f77fe640 1 -- 172.21.15.33:0/3412346589 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f97f00f2630 con 0x7f98001009b0 2024-04-02T13:46:59.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97fed9d640 1 --2- 172.21.15.33:0/1542708900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a736352ca0 0x55a736355090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97fed9d640 1 -- 172.21.15.33:0/1542708900 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a73634a0c0 msgr2=0x55a736352760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97fed9d640 1 --2- 172.21.15.33:0/1542708900 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a73634a0c0 0x55a736352760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97fed9d640 1 -- 172.21.15.33:0/1542708900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a736352ca0 msgr2=0x55a736355090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97fed9d640 1 --2- 172.21.15.33:0/1542708900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a736352ca0 0x55a736355090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97fed9d640 1 -- 172.21.15.33:0/1542708900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a735fc13f0 con 0x55a7363497a0 2024-04-02T13:46:59.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97fed9d640 1 --2- 172.21.15.33:0/1542708900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7363497a0 0x55a736349b80 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f97f80138c0 tx=0x7f97f80099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1119bfd01e993551 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.006+0000 7f97fe59c640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f97ec0a3d20 0x7f97ec0a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f97f00be010 tx=0x7f97f00c1040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.007+0000 7f97f4ff9640 1 -- 172.21.15.33:0/1542708900 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8019030 con 0x55a7363497a0 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.007+0000 7f97f4ff9640 1 -- 172.21.15.33:0/1542708900 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f97f8003e40 con 0x55a7363497a0 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.007+0000 7f97f4ff9640 1 -- 172.21.15.33:0/1542708900 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8011ac0 con 0x55a7363497a0 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.007+0000 7f9807ba5740 1 -- 172.21.15.33:0/1542708900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7363497a0 msgr2=0x55a736349b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.007+0000 7f9807ba5740 1 --2- 172.21.15.33:0/1542708900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7363497a0 0x55a736349b80 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f97f80138c0 tx=0x7f97f80099c0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.007+0000 7f9807ba5740 1 -- 172.21.15.33:0/1542708900 shutdown_connections 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.007+0000 7f9807ba5740 1 --2- 172.21.15.33:0/1542708900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a736352ca0 0x55a736355090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.007+0000 7f9807ba5740 1 --2- 172.21.15.33:0/1542708900 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a73634a0c0 0x55a736352760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.007+0000 7f9807ba5740 1 --2- 172.21.15.33:0/1542708900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7363497a0 0x55a736349b80 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.007+0000 7f9807ba5740 1 -- 172.21.15.33:0/1542708900 >> 172.21.15.33:0/1542708900 conn(0x55a736343ed0 msgr2=0x55a7363442b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.007+0000 7f9807ba5740 1 -- 172.21.15.33:0/1542708900 shutdown_connections 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.007+0000 7f9807ba5740 1 -- 172.21.15.33:0/1542708900 wait complete. 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f9807ba5740 1 -- start start 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f9807ba5740 1 Processor -- start 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f9807ba5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 0x55a736352710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f9807ba5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 0x55a73634faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f9807ba5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a7363501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f9807ba5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a736355a90 con 0x55a736352ca0 2024-04-02T13:46:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f9807ba5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a736268e80 con 0x55a73634a0c0 2024-04-02T13:46:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f9807ba5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55a736355c10 con 0x55a7363497a0 2024-04-02T13:46:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f9807ba5740 10 client.0 _ll_register_callbacks cb 0x55a7362a4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a7363501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a7363501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41222/0 (socket says 172.21.15.33:41222) 2024-04-02T13:46:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f97fed9d640 1 -- 172.21.15.33:0/1543250364 learned_addr learned my addr 172.21.15.33:0/1543250364 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f97fed9d640 1 --2- 172.21.15.33:0/1543250364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 0x55a736352710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f97f7fff640 1 --2- 172.21.15.33:0/1543250364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 0x55a73634faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f97fed9d640 1 -- 172.21.15.33:0/1543250364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 msgr2=0x55a736352710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f97fed9d640 1 --2- 172.21.15.33:0/1543250364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 0x55a736352710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f97fed9d640 1 -- 172.21.15.33:0/1543250364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 msgr2=0x55a73634faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f97fed9d640 1 --2- 172.21.15.33:0/1543250364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 0x55a73634faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.008+0000 7f97fed9d640 1 -- 172.21.15.33:0/1543250364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73608b7d0 con 0x55a736352ca0 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f97fed9d640 1 --2- 172.21.15.33:0/1543250364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a7363501e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f97f80097f0 tx=0x7f97f8013340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b690ba38d535bb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f97e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f97e5ffb640 1 -- 172.21.15.33:0/1543250364 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8003b80 con 0x55a736352ca0 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f97e5ffb640 1 -- 172.21.15.33:0/1543250364 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f97f8011880 con 0x55a736352ca0 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f9807ba5740 1 -- 172.21.15.33:0/1543250364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a7362bacf0 con 0x55a736352ca0 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f97e5ffb640 1 -- 172.21.15.33:0/1543250364 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f97f8011a80 con 0x55a736352ca0 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f9807ba5740 10 client.17361 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f9807ba5740 1 -- 172.21.15.33:0/1543250364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a736361b80 con 0x55a736352ca0 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f97e5ffb640 1 -- 172.21.15.33:0/1543250364 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55a7362bacf0 con 0x55a736352ca0 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f97e5ffb640 1 -- 172.21.15.33:0/1543250364 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f97f801e430 con 0x55a736352ca0 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f9807ba5740 10 client.17361 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f9807ba5740 10 client.17361 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:46:59.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f9807ba5740 20 client.17361 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f9807ba5740 1 --2- 172.21.15.33:0/1543250364 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a736362fe0 0x55a7363653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f9807ba5740 4 client.17361 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.009+0000 7f9807ba5740 1 -- 172.21.15.33:0/1543250364 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7362b5040 con 0x55a736362fe0 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.010+0000 7f97f7fff640 1 --2- 172.21.15.33:0/1543250364 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a736362fe0 0x55a7363653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.010+0000 7f97f7fff640 1 --2- 172.21.15.33:0/1543250364 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a736362fe0 0x55a7363653d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5fd87a417b244458 server_cookie=b1df4a26ad7fe645 in_seq=0 out_seq=0 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.010+0000 7f97e5ffb640 10 client.17361 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f97e5ffb640 1 -- 172.21.15.33:0/1543250364 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7362b5040 con 0x55a736362fe0 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f97e5ffb640 10 client.17361 handle_command_reply: tid=0 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f9807ba5740 1 client.17361 shutdown 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f9807ba5740 1 -- 172.21.15.33:0/1543250364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 msgr2=0x55a7363501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f9807ba5740 1 --2- 172.21.15.33:0/1543250364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a7363501e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f97f80097f0 tx=0x7f97f8013340 comp rx=0 tx=0).stop 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f9807ba5740 1 -- 172.21.15.33:0/1543250364 shutdown_connections 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f9807ba5740 1 --2- 172.21.15.33:0/1543250364 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a736362fe0 0x55a7363653d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f9807ba5740 1 --2- 172.21.15.33:0/1543250364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a7363501e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f9807ba5740 1 --2- 172.21.15.33:0/1543250364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 0x55a73634faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f97e5ffb640 0 client.17361 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:46:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f9807ba5740 1 --2- 172.21.15.33:0/1543250364 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 0x55a736352710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f9807ba5740 1 -- 172.21.15.33:0/1543250364 >> 172.21.15.33:0/1543250364 conn(0x55a736343ed0 msgr2=0x55a736345c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f9807ba5740 1 -- 172.21.15.33:0/1543250364 shutdown_connections 2024-04-02T13:46:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f9807ba5740 1 -- 172.21.15.33:0/1543250364 wait complete. 2024-04-02T13:46:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.011+0000 7f9807ba5740 20 client.17361 trim_cache size 0 max 16384 2024-04-02T13:46:59.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.024+0000 7f9807ba5740 1 Processor -- start 2024-04-02T13:46:59.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.024+0000 7f9807ba5740 1 -- start start 2024-04-02T13:46:59.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.024+0000 7f9807ba5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 0x55a73634ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.024+0000 7f9807ba5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 0x55a73635e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.024+0000 7f9807ba5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a7363521d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.024+0000 7f9807ba5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98001b3aa0 con 0x55a736352ca0 2024-04-02T13:46:59.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.024+0000 7f9807ba5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f98001b1370 con 0x55a73634a0c0 2024-04-02T13:46:59.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.024+0000 7f9807ba5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98001af1e0 con 0x55a7363497a0 2024-04-02T13:46:59.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.025+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a7363521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.025+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a7363521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41238/0 (socket says 172.21.15.33:41238) 2024-04-02T13:46:59.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.025+0000 7f97fed9d640 1 -- 172.21.15.33:0/539032013 learned_addr learned my addr 172.21.15.33:0/539032013 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.025+0000 7f97f7fff640 1 --2- 172.21.15.33:0/539032013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 0x55a73635e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.025+0000 7f97fed9d640 1 -- 172.21.15.33:0/539032013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 msgr2=0x55a73634ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.025+0000 7f97fed9d640 1 --2- 172.21.15.33:0/539032013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 0x55a73634ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.025+0000 7f97fed9d640 1 -- 172.21.15.33:0/539032013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 msgr2=0x55a73635e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.025+0000 7f97fed9d640 1 --2- 172.21.15.33:0/539032013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 0x55a73635e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.025+0000 7f97fed9d640 1 -- 172.21.15.33:0/539032013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73634dff0 con 0x55a736352ca0 2024-04-02T13:46:59.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.025+0000 7f97fed9d640 1 --2- 172.21.15.33:0/539032013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a7363521d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f97f8005d70 tx=0x7f97f8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adf6358d46a026a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.025+0000 7f97e7fff640 1 -- 172.21.15.33:0/539032013 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8019040 con 0x55a736352ca0 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.025+0000 7f97e7fff640 1 -- 172.21.15.33:0/539032013 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f97f8003b80 con 0x55a736352ca0 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.025+0000 7f97e7fff640 1 -- 172.21.15.33:0/539032013 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f801ea70 con 0x55a736352ca0 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 -- 172.21.15.33:0/539032013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 msgr2=0x55a7363521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 --2- 172.21.15.33:0/539032013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a7363521d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f97f8005d70 tx=0x7f97f8009c30 comp rx=0 tx=0).stop 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 -- 172.21.15.33:0/539032013 shutdown_connections 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 --2- 172.21.15.33:0/539032013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a7363521d0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 --2- 172.21.15.33:0/539032013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 0x55a73635e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 --2- 172.21.15.33:0/539032013 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 0x55a73634ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 -- 172.21.15.33:0/539032013 >> 172.21.15.33:0/539032013 conn(0x55a736343ed0 msgr2=0x55a73635dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 -- 172.21.15.33:0/539032013 shutdown_connections 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 -- 172.21.15.33:0/539032013 wait complete. 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 -- start start 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 Processor -- start 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 0x55a736368750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 0x55a736363ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a736364000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98001af830 con 0x55a736352ca0 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f98001b0ca0 con 0x55a73634a0c0 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f98001aff00 con 0x55a7363497a0 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f9807ba5740 10 client.0 _ll_register_callbacks cb 0x7f97e800b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.026+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a736364000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a736364000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41244/0 (socket says 172.21.15.33:41244) 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97fed9d640 1 -- 172.21.15.33:0/741078785 learned_addr learned my addr 172.21.15.33:0/741078785 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97f7fff640 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 0x55a736363ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97fed9d640 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 0x55a736368750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97fed9d640 1 -- 172.21.15.33:0/741078785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 msgr2=0x55a736368750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97fed9d640 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 0x55a736368750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97fed9d640 1 -- 172.21.15.33:0/741078785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 msgr2=0x55a736363ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97fed9d640 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 0x55a736363ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97fed9d640 1 -- 172.21.15.33:0/741078785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a736364800 con 0x55a736352ca0 2024-04-02T13:46:59.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97f7fff640 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 0x55a736363ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:59.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97fed9d640 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a736364000 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f97f8006010 tx=0x7f97f8009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879c2c489dd03d1a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:59.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97e5ffb640 1 -- 172.21.15.33:0/741078785 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f801e430 con 0x55a736352ca0 2024-04-02T13:46:59.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97e5ffb640 1 -- 172.21.15.33:0/741078785 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f97f801e5d0 con 0x55a736352ca0 2024-04-02T13:46:59.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f9807ba5740 1 -- 172.21.15.33:0/741078785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f8003b80 con 0x55a736352ca0 2024-04-02T13:46:59.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f97e5ffb640 1 -- 172.21.15.33:0/741078785 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f97f8044050 con 0x55a736352ca0 2024-04-02T13:46:59.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f9807ba5740 10 client.17367 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:59.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.027+0000 7f9807ba5740 1 -- 172.21.15.33:0/741078785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a7363c9140 con 0x55a736352ca0 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.028+0000 7f97e5ffb640 1 -- 172.21.15.33:0/741078785 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f97f8003b80 con 0x55a736352ca0 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.028+0000 7f97e5ffb640 1 -- 172.21.15.33:0/741078785 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f97f8028040 con 0x55a736352ca0 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.028+0000 7f9807ba5740 10 client.17367 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.028+0000 7f9807ba5740 10 client.17367 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.028+0000 7f9807ba5740 20 client.17367 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.028+0000 7f9807ba5740 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a7363c96f0 0x55a7363cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.028+0000 7f9807ba5740 4 client.17367 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.028+0000 7f9807ba5740 1 -- 172.21.15.33:0/741078785 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7362326b0 con 0x55a7363c96f0 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.028+0000 7f97f7fff640 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a7363c96f0 0x55a7363cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.028+0000 7f97e5ffb640 10 client.17367 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.028+0000 7f97f7fff640 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a7363c96f0 0x55a7363cbae0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32f7659b548474a server_cookie=6e2a11475f05624 in_seq=0 out_seq=0 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f97e5ffb640 1 -- 172.21.15.33:0/741078785 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7362326b0 con 0x55a7363c96f0 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f97e5ffb640 10 client.17367 handle_command_reply: tid=0 2024-04-02T13:46:59.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9807ba5740 1 client.17367 shutdown 2024-04-02T13:46:59.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9807ba5740 1 -- 172.21.15.33:0/741078785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 msgr2=0x55a736364000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9807ba5740 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a736364000 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f97f8006010 tx=0x7f97f8009ba0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9807ba5740 1 -- 172.21.15.33:0/741078785 shutdown_connections 2024-04-02T13:46:59.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9807ba5740 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55a7363c96f0 0x55a7363cbae0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9807ba5740 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a736352ca0 0x55a736364000 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9807ba5740 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a73634a0c0 0x55a736363ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f97e5ffb640 0 client.17367 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:46:59.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9807ba5740 1 --2- 172.21.15.33:0/741078785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55a7363497a0 0x55a736368750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9807ba5740 1 -- 172.21.15.33:0/741078785 >> 172.21.15.33:0/741078785 conn(0x55a736343ed0 msgr2=0x55a73635e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9807ba5740 1 -- 172.21.15.33:0/741078785 shutdown_connections 2024-04-02T13:46:59.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9807ba5740 1 -- 172.21.15.33:0/741078785 wait complete. 2024-04-02T13:46:59.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9807ba5740 20 client.17367 trim_cache size 0 max 16384 2024-04-02T13:46:59.040 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:46:59.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9804ee4640 1 -- 172.21.15.33:0/3412346589 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f97ec0a3d20 msgr2=0x7f97ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9804ee4640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f97ec0a3d20 0x7f97ec0a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f97f00be010 tx=0x7f97f00c1040 comp rx=0 tx=0).stop 2024-04-02T13:46:59.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9804ee4640 1 -- 172.21.15.33:0/3412346589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98001009b0 msgr2=0x7f98000fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9804ee4640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98001009b0 0x7f98000fe1c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f97f0000c00 tx=0x7f97f00045e0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9804ee4640 1 -- 172.21.15.33:0/3412346589 shutdown_connections 2024-04-02T13:46:59.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9804ee4640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f97ec0a3d20 0x7f97ec0a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9804ee4640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98001069c0 0x7f98000fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9804ee4640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9800101370 0x7f98000fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9804ee4640 1 --2- 172.21.15.33:0/3412346589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98001009b0 0x7f98000fe1c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9804ee4640 1 -- 172.21.15.33:0/3412346589 >> 172.21.15.33:0/3412346589 conn(0x7f9800076d80 msgr2=0x7f980006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9804ee4640 1 -- 172.21.15.33:0/3412346589 shutdown_connections 2024-04-02T13:46:59.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.029+0000 7f9804ee4640 1 -- 172.21.15.33:0/3412346589 wait complete. 2024-04-02T13:46:59.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:46:59.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.145+0000 7f20ee6f2640 1 -- 172.21.15.33:0/193358334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20e8107aa0 msgr2=0x7f20e8107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.145+0000 7f20ee6f2640 1 --2- 172.21.15.33:0/193358334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20e8107aa0 0x7f20e8107e80 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009930 tx=0x7f20d802bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.145+0000 7f20ee6f2640 1 -- 172.21.15.33:0/193358334 shutdown_connections 2024-04-02T13:46:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.145+0000 7f20ee6f2640 1 --2- 172.21.15.33:0/193358334 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20e8102450 0x7f20e810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.145+0000 7f20ee6f2640 1 --2- 172.21.15.33:0/193358334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20e8101a90 0x7f20e8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.145+0000 7f20ee6f2640 1 --2- 172.21.15.33:0/193358334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20e8107aa0 0x7f20e8107e80 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.145+0000 7f20ee6f2640 1 -- 172.21.15.33:0/193358334 >> 172.21.15.33:0/193358334 conn(0x7f20e8078f00 msgr2=0x7f20e8079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.145+0000 7f20ee6f2640 1 -- 172.21.15.33:0/193358334 shutdown_connections 2024-04-02T13:46:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.145+0000 7f20ee6f2640 1 -- 172.21.15.33:0/193358334 wait complete. 2024-04-02T13:46:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.146+0000 7f20ee6f2640 1 Processor -- start 2024-04-02T13:46:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.146+0000 7f20ee6f2640 1 -- start start 2024-04-02T13:46:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.146+0000 7f20ee6f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20e8101a90 0x7f20e81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.146+0000 7f20ee6f2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20e8102450 0x7f20e81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.146+0000 7f20ee6f2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20e8107aa0 0x7f20e819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.146+0000 7f20ee6f2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8111e50 con 0x7f20e8101a90 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.146+0000 7f20ee6f2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8111cd0 con 0x7f20e8102450 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.146+0000 7f20ee6f2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8111fd0 con 0x7f20e8107aa0 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.146+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20e8101a90 0x7f20e81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.146+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20e8101a90 0x7f20e81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41256/0 (socket says 172.21.15.33:41256) 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.146+0000 7f20e7fff640 1 -- 172.21.15.33:0/1514215688 learned_addr learned my addr 172.21.15.33:0/1514215688 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20e77fe640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20e8102450 0x7f20e81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20e7fff640 1 -- 172.21.15.33:0/1514215688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20e8107aa0 msgr2=0x7f20e819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20e8107aa0 0x7f20e819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20e7fff640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20e8107aa0 0x7f20e819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20e7fff640 1 -- 172.21.15.33:0/1514215688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20e8102450 msgr2=0x7f20e81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20e7fff640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20e8102450 0x7f20e81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20e7fff640 1 -- 172.21.15.33:0/1514215688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e819d350 con 0x7f20e8101a90 2024-04-02T13:46:59.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20e8107aa0 0x7f20e819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20e77fe640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20e8102450 0x7f20e81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20e7fff640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20e8101a90 0x7f20e81983b0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f20d8000c00 tx=0x7f20d80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20e57fa640 1 -- 172.21.15.33:0/1514215688 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8043020 con 0x7f20e8101a90 2024-04-02T13:46:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20e57fa640 1 -- 172.21.15.33:0/1514215688 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f20d802caa0 con 0x7f20e8101a90 2024-04-02T13:46:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20e57fa640 1 -- 172.21.15.33:0/1514215688 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8003bf0 con 0x7f20e8101a90 2024-04-02T13:46:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20ee6f2640 1 -- 172.21.15.33:0/1514215688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e819d5e0 con 0x7f20e8101a90 2024-04-02T13:46:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.147+0000 7f20ee6f2640 1 -- 172.21.15.33:0/1514215688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e81a4ed0 con 0x7f20e8101a90 2024-04-02T13:46:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20e57fa640 1 -- 172.21.15.33:0/1514215688 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f20d8046020 con 0x7f20e8101a90 2024-04-02T13:46:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20f13b1740 1 Processor -- start 2024-04-02T13:46:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20f13b1740 1 -- start start 2024-04-02T13:46:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20f13b1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393c450 0x55d3d393c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20f13b1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393cd70 0x55d3d3949b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20f13b1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d394a0b0 0x55d3d394c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20f13b1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d3d38a07b0 con 0x55d3d393c450 2024-04-02T13:46:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20f13b1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d3d383f570 con 0x55d3d394a0b0 2024-04-02T13:46:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20f13b1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d3d389c760 con 0x55d3d393cd70 2024-04-02T13:46:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20ecc8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393c450 0x55d3d393c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20ecc8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393c450 0x55d3d393c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41268/0 (socket says 172.21.15.33:41268) 2024-04-02T13:46:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20ecc8f640 1 -- 172.21.15.33:0/111760613 learned_addr learned my addr 172.21.15.33:0/111760613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20e57fa640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f20d40a3d20 0x7f20d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20e7fff640 1 --2- 172.21.15.33:0/111760613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393cd70 0x55d3d3949b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20ecc8f640 1 -- 172.21.15.33:0/111760613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393cd70 msgr2=0x55d3d3949b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20e77fe640 1 --2- 172.21.15.33:0/111760613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d394a0b0 0x55d3d394c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/111760613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393cd70 0x55d3d3949b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20ecc8f640 1 -- 172.21.15.33:0/111760613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d394a0b0 msgr2=0x55d3d394c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/111760613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d394a0b0 0x55d3d394c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20ecc8f640 1 -- 172.21.15.33:0/111760613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3d35b43f0 con 0x55d3d393c450 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20e77fe640 1 --2- 172.21.15.33:0/111760613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d394a0b0 0x55d3d394c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20e7fff640 1 --2- 172.21.15.33:0/111760613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393cd70 0x55d3d3949b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20e57fa640 1 -- 172.21.15.33:0/1514215688 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f20d80f2630 con 0x7f20e8101a90 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.150+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f20d40a3d20 0x7f20d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/111760613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393c450 0x55d3d393c830 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009fa0 tx=0x7f20e000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b7219229213654 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20de7fc640 1 -- 172.21.15.33:0/111760613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e001a610 con 0x55d3d393c450 2024-04-02T13:46:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20de7fc640 1 -- 172.21.15.33:0/111760613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f20e001ac80 con 0x55d3d393c450 2024-04-02T13:46:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20de7fc640 1 -- 172.21.15.33:0/111760613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e000db30 con 0x55d3d393c450 2024-04-02T13:46:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20f13b1740 1 -- 172.21.15.33:0/111760613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393c450 msgr2=0x55d3d393c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20f13b1740 1 --2- 172.21.15.33:0/111760613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393c450 0x55d3d393c830 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009fa0 tx=0x7f20e000f7f0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f20d40a3d20 0x7f20d40a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f20e0020000 tx=0x7f20e001d000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20f13b1740 1 -- 172.21.15.33:0/111760613 shutdown_connections 2024-04-02T13:46:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20f13b1740 1 --2- 172.21.15.33:0/111760613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d394a0b0 0x55d3d394c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20f13b1740 1 --2- 172.21.15.33:0/111760613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393cd70 0x55d3d3949b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20f13b1740 1 --2- 172.21.15.33:0/111760613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393c450 0x55d3d393c830 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20f13b1740 1 -- 172.21.15.33:0/111760613 >> 172.21.15.33:0/111760613 conn(0x55d3d3936ed0 msgr2=0x55d3d39372b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20f13b1740 1 -- 172.21.15.33:0/111760613 shutdown_connections 2024-04-02T13:46:59.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.151+0000 7f20f13b1740 1 -- 172.21.15.33:0/111760613 wait complete. 2024-04-02T13:46:59.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.152+0000 7f20f13b1740 1 -- start start 2024-04-02T13:46:59.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.152+0000 7f20f13b1740 1 Processor -- start 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.152+0000 7f20f13b1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d393c450 0x55d3d3949aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.152+0000 7f20f13b1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393cd70 0x55d3d3946e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.152+0000 7f20f13b1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 0x55d3d3947440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.152+0000 7f20f13b1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d3d394cef0 con 0x55d3d394a0b0 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.152+0000 7f20f13b1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d3d385be80 con 0x55d3d393c450 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.152+0000 7f20f13b1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55d3d394d070 con 0x55d3d393cd70 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.152+0000 7f20f13b1740 10 client.0 _ll_register_callbacks cb 0x55d3d3897aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.152+0000 7f20ecc8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 0x55d3d3947440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.152+0000 7f20ecc8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 0x55d3d3947440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41284/0 (socket says 172.21.15.33:41284) 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.152+0000 7f20ecc8f640 1 -- 172.21.15.33:0/1047662964 learned_addr learned my addr 172.21.15.33:0/1047662964 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.152+0000 7f20e7fff640 1 --2- 172.21.15.33:0/1047662964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d393c450 0x55d3d3949aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20e77fe640 1 --2- 172.21.15.33:0/1047662964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393cd70 0x55d3d3946e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20ecc8f640 1 -- 172.21.15.33:0/1047662964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393cd70 msgr2=0x55d3d3946e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/1047662964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393cd70 0x55d3d3946e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20ecc8f640 1 -- 172.21.15.33:0/1047662964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d393c450 msgr2=0x55d3d3949aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/1047662964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d393c450 0x55d3d3949aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20ecc8f640 1 -- 172.21.15.33:0/1047662964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3d367e7d0 con 0x55d3d394a0b0 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/1047662964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 0x55d3d3947440 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f20e000dbe0 tx=0x7f20e000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa975dd6c4c89fad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20c77fe640 1 -- 172.21.15.33:0/1047662964 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0009af0 con 0x55d3d394a0b0 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20c77fe640 1 -- 172.21.15.33:0/1047662964 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f20e001a410 con 0x55d3d394a0b0 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20f13b1740 1 -- 172.21.15.33:0/1047662964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e001ac80 con 0x55d3d394a0b0 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20c77fe640 1 -- 172.21.15.33:0/1047662964 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f20e0039050 con 0x55d3d394a0b0 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20f13b1740 10 client.17379 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20f13b1740 1 -- 172.21.15.33:0/1047662964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d3d3958f20 con 0x55d3d394a0b0 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20c77fe640 1 -- 172.21.15.33:0/1047662964 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f20e001ac80 con 0x55d3d394a0b0 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20c77fe640 1 -- 172.21.15.33:0/1047662964 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f20e0025440 con 0x55d3d394a0b0 2024-04-02T13:46:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20f13b1740 10 client.17379 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.153+0000 7f20f13b1740 10 client.17379 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.154+0000 7f20f13b1740 20 client.17379 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.154+0000 7f20f13b1740 1 --2- 172.21.15.33:0/1047662964 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55d3d395a380 0x55d3d395c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.154+0000 7f20f13b1740 4 client.17379 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.154+0000 7f20f13b1740 1 -- 172.21.15.33:0/1047662964 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3d38adcf0 con 0x55d3d395a380 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.154+0000 7f20e7fff640 1 --2- 172.21.15.33:0/1047662964 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55d3d395a380 0x55d3d395c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.154+0000 7f20c77fe640 10 client.17379 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.154+0000 7f20e7fff640 1 --2- 172.21.15.33:0/1047662964 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55d3d395a380 0x55d3d395c770 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b66debb67b674e92 server_cookie=c70e6025655e5803 in_seq=0 out_seq=0 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20c77fe640 1 -- 172.21.15.33:0/1047662964 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d3d38adcf0 con 0x55d3d395a380 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20c77fe640 10 client.17379 handle_command_reply: tid=0 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20f13b1740 1 client.17379 shutdown 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20f13b1740 1 -- 172.21.15.33:0/1047662964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 msgr2=0x55d3d3947440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20f13b1740 1 --2- 172.21.15.33:0/1047662964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 0x55d3d3947440 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f20e000dbe0 tx=0x7f20e000dc10 comp rx=0 tx=0).stop 2024-04-02T13:46:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20f13b1740 1 -- 172.21.15.33:0/1047662964 shutdown_connections 2024-04-02T13:46:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20f13b1740 1 --2- 172.21.15.33:0/1047662964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 0x55d3d3947440 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20f13b1740 1 --2- 172.21.15.33:0/1047662964 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55d3d395a380 0x55d3d395c770 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20f13b1740 1 --2- 172.21.15.33:0/1047662964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393cd70 0x55d3d3946e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20f13b1740 1 --2- 172.21.15.33:0/1047662964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d393c450 0x55d3d3949aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20c77fe640 0 client.17379 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:46:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20f13b1740 1 -- 172.21.15.33:0/1047662964 >> 172.21.15.33:0/1047662964 conn(0x55d3d3936ed0 msgr2=0x55d3d3938c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20f13b1740 1 -- 172.21.15.33:0/1047662964 shutdown_connections 2024-04-02T13:46:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20f13b1740 1 -- 172.21.15.33:0/1047662964 wait complete. 2024-04-02T13:46:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.155+0000 7f20f13b1740 20 client.17379 trim_cache size 0 max 16384 2024-04-02T13:46:59.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.168+0000 7f20f13b1740 1 Processor -- start 2024-04-02T13:46:59.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.168+0000 7f20f13b1740 1 -- start start 2024-04-02T13:46:59.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20f13b1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393c450 0x55d3d3955e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20f13b1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393cd70 0x55d3d3949560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20f13b1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d394a0b0 0x55d3d3949aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20f13b1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81af720 con 0x55d3d393cd70 2024-04-02T13:46:59.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20f13b1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81acff0 con 0x55d3d394a0b0 2024-04-02T13:46:59.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20f13b1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81aae60 con 0x55d3d393c450 2024-04-02T13:46:59.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20e77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393cd70 0x55d3d3949560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20e77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393cd70 0x55d3d3949560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41294/0 (socket says 172.21.15.33:41294) 2024-04-02T13:46:59.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20e77fe640 1 -- 172.21.15.33:0/2000148167 learned_addr learned my addr 172.21.15.33:0/2000148167 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20e7fff640 1 --2- 172.21.15.33:0/2000148167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393c450 0x55d3d3955e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/2000148167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d394a0b0 0x55d3d3949aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20e77fe640 1 -- 172.21.15.33:0/2000148167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393c450 msgr2=0x55d3d3955e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20e77fe640 1 --2- 172.21.15.33:0/2000148167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393c450 0x55d3d3955e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20e77fe640 1 -- 172.21.15.33:0/2000148167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d394a0b0 msgr2=0x55d3d3949aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20e77fe640 1 --2- 172.21.15.33:0/2000148167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d394a0b0 0x55d3d3949aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20e77fe640 1 -- 172.21.15.33:0/2000148167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3d393d150 con 0x55d3d393cd70 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/2000148167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d394a0b0 0x55d3d3949aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20e77fe640 1 --2- 172.21.15.33:0/2000148167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393cd70 0x55d3d3949560 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f20e8103d50 tx=0x7f20d00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89fade5f428b73c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20ddffb640 1 -- 172.21.15.33:0/2000148167 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0004030 con 0x55d3d393cd70 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20ddffb640 1 -- 172.21.15.33:0/2000148167 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f20d00041d0 con 0x55d3d393cd70 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.169+0000 7f20ddffb640 1 -- 172.21.15.33:0/2000148167 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d001b740 con 0x55d3d393cd70 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.170+0000 7f20f13b1740 1 -- 172.21.15.33:0/2000148167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393cd70 msgr2=0x55d3d3949560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.170+0000 7f20f13b1740 1 --2- 172.21.15.33:0/2000148167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393cd70 0x55d3d3949560 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f20e8103d50 tx=0x7f20d00117b0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.170+0000 7f20f13b1740 1 -- 172.21.15.33:0/2000148167 shutdown_connections 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.170+0000 7f20f13b1740 1 --2- 172.21.15.33:0/2000148167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d394a0b0 0x55d3d3949aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.170+0000 7f20f13b1740 1 --2- 172.21.15.33:0/2000148167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d393cd70 0x55d3d3949560 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.170+0000 7f20f13b1740 1 --2- 172.21.15.33:0/2000148167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393c450 0x55d3d3955e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.170+0000 7f20f13b1740 1 -- 172.21.15.33:0/2000148167 >> 172.21.15.33:0/2000148167 conn(0x55d3d3936ed0 msgr2=0x55d3d3946df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.170+0000 7f20f13b1740 1 -- 172.21.15.33:0/2000148167 shutdown_connections 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.170+0000 7f20f13b1740 1 -- 172.21.15.33:0/2000148167 wait complete. 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.170+0000 7f20f13b1740 1 -- start start 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20f13b1740 1 Processor -- start 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20f13b1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393c450 0x55d3d3952600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20f13b1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d393cd70 0x55d3d394aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20f13b1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 0x55d3d394afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20f13b1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81ab4b0 con 0x55d3d394a0b0 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20f13b1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81ac920 con 0x55d3d393cd70 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20f13b1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81abb80 con 0x55d3d393c450 2024-04-02T13:46:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20f13b1740 10 client.0 _ll_register_callbacks cb 0x7f20e0009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20ecc8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 0x55d3d394afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20ecc8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 0x55d3d394afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41304/0 (socket says 172.21.15.33:41304) 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20ecc8f640 1 -- 172.21.15.33:0/2912192984 learned_addr learned my addr 172.21.15.33:0/2912192984 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20e7fff640 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393c450 0x55d3d3952600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20e77fe640 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d393cd70 0x55d3d394aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20ecc8f640 1 -- 172.21.15.33:0/2912192984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393c450 msgr2=0x55d3d3952600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393c450 0x55d3d3952600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20ecc8f640 1 -- 172.21.15.33:0/2912192984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d393cd70 msgr2=0x55d3d394aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d393cd70 0x55d3d394aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20ecc8f640 1 -- 172.21.15.33:0/2912192984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3d394b610 con 0x55d3d394a0b0 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20e77fe640 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d393cd70 0x55d3d394aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20ecc8f640 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 0x55d3d394afd0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f20e0025800 tx=0x7f20e0026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa6a2e199883c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20c77fe640 1 -- 172.21.15.33:0/2912192984 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0009af0 con 0x55d3d394a0b0 2024-04-02T13:46:59.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20c77fe640 1 -- 172.21.15.33:0/2912192984 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f20e000d3a0 con 0x55d3d394a0b0 2024-04-02T13:46:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.171+0000 7f20f13b1740 1 -- 172.21.15.33:0/2912192984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20d00041d0 con 0x55d3d394a0b0 2024-04-02T13:46:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.172+0000 7f20c77fe640 1 -- 172.21.15.33:0/2912192984 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f20e00094a0 con 0x55d3d394a0b0 2024-04-02T13:46:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.172+0000 7f20f13b1740 10 client.17385 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.172+0000 7f20f13b1740 1 -- 172.21.15.33:0/2912192984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d3d39c0750 con 0x55d3d394a0b0 2024-04-02T13:46:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.172+0000 7f20c77fe640 1 -- 172.21.15.33:0/2912192984 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f20d00041d0 con 0x55d3d394a0b0 2024-04-02T13:46:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.172+0000 7f20c77fe640 1 -- 172.21.15.33:0/2912192984 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f20e001a5a0 con 0x55d3d394a0b0 2024-04-02T13:46:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.172+0000 7f20f13b1740 10 client.17385 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.172+0000 7f20f13b1740 10 client.17385 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:46:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.172+0000 7f20f13b1740 20 client.17385 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.172+0000 7f20f13b1740 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55d3d39c0d40 0x55d3d39c3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.172+0000 7f20f13b1740 4 client.17385 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.172+0000 7f20f13b1740 1 -- 172.21.15.33:0/2912192984 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3d38a8040 con 0x55d3d39c0d40 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.172+0000 7f20e7fff640 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55d3d39c0d40 0x55d3d39c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20e7fff640 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55d3d39c0d40 0x55d3d39c3180 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34edcb3afbbca9a5 server_cookie=2895fd807c14b593 in_seq=0 out_seq=0 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20c77fe640 10 client.17385 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20c77fe640 1 -- 172.21.15.33:0/2912192984 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3d38a8040 con 0x55d3d39c0d40 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20c77fe640 10 client.17385 handle_command_reply: tid=0 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20f13b1740 1 client.17385 shutdown 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20f13b1740 1 -- 172.21.15.33:0/2912192984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 msgr2=0x55d3d394afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20f13b1740 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 0x55d3d394afd0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f20e0025800 tx=0x7f20e0026000 comp rx=0 tx=0).stop 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20f13b1740 1 -- 172.21.15.33:0/2912192984 shutdown_connections 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20f13b1740 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3d394a0b0 0x55d3d394afd0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20f13b1740 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d3d393cd70 0x55d3d394aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20f13b1740 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55d3d39c0d40 0x55d3d39c3180 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20f13b1740 1 --2- 172.21.15.33:0/2912192984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d3d393c450 0x55d3d3952600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20c77fe640 0 client.17385 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:46:59.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20f13b1740 1 -- 172.21.15.33:0/2912192984 >> 172.21.15.33:0/2912192984 conn(0x55d3d3936ed0 msgr2=0x55d3d39456c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20f13b1740 1 -- 172.21.15.33:0/2912192984 shutdown_connections 2024-04-02T13:46:59.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20f13b1740 1 -- 172.21.15.33:0/2912192984 wait complete. 2024-04-02T13:46:59.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.173+0000 7f20f13b1740 20 client.17385 trim_cache size 0 max 16384 2024-04-02T13:46:59.185 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:46:59.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.174+0000 7f20ee6f2640 1 -- 172.21.15.33:0/1514215688 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f20d40a3d20 msgr2=0x7f20d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.174+0000 7f20ee6f2640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f20d40a3d20 0x7f20d40a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f20e0020000 tx=0x7f20e001d000 comp rx=0 tx=0).stop 2024-04-02T13:46:59.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.174+0000 7f20ee6f2640 1 -- 172.21.15.33:0/1514215688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20e8101a90 msgr2=0x7f20e81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.174+0000 7f20ee6f2640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20e8101a90 0x7f20e81983b0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f20d8000c00 tx=0x7f20d80045e0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.174+0000 7f20ee6f2640 1 -- 172.21.15.33:0/1514215688 shutdown_connections 2024-04-02T13:46:59.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.174+0000 7f20ee6f2640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f20d40a3d20 0x7f20d40a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.174+0000 7f20ee6f2640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20e8107aa0 0x7f20e819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.174+0000 7f20ee6f2640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f20e8102450 0x7f20e81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.174+0000 7f20ee6f2640 1 --2- 172.21.15.33:0/1514215688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20e8101a90 0x7f20e81983b0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.174+0000 7f20ee6f2640 1 -- 172.21.15.33:0/1514215688 >> 172.21.15.33:0/1514215688 conn(0x7f20e8078f00 msgr2=0x7f20e8104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.174+0000 7f20ee6f2640 1 -- 172.21.15.33:0/1514215688 shutdown_connections 2024-04-02T13:46:59.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.174+0000 7f20ee6f2640 1 -- 172.21.15.33:0/1514215688 wait complete. 2024-04-02T13:46:59.188 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:46:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.290+0000 7ff6f0960640 1 -- 172.21.15.33:0/3193109960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec0696a0 msgr2=0x7ff6ec10e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.290+0000 7ff6f0960640 1 --2- 172.21.15.33:0/3193109960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec0696a0 0x7ff6ec10e940 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009cd0 tx=0x7ff6dc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.291+0000 7ff6f0960640 1 -- 172.21.15.33:0/3193109960 shutdown_connections 2024-04-02T13:46:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.291+0000 7ff6f0960640 1 --2- 172.21.15.33:0/3193109960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec0696a0 0x7ff6ec10e940 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.291+0000 7ff6f0960640 1 --2- 172.21.15.33:0/3193109960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6ec068ce0 0x7ff6ec069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.291+0000 7ff6f0960640 1 --2- 172.21.15.33:0/3193109960 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6ec107a80 0x7ff6ec107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.291+0000 7ff6f0960640 1 -- 172.21.15.33:0/3193109960 >> 172.21.15.33:0/3193109960 conn(0x7ff6ec077b00 msgr2=0x7ff6ec077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.291+0000 7ff6f0960640 1 -- 172.21.15.33:0/3193109960 shutdown_connections 2024-04-02T13:46:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.291+0000 7ff6f0960640 1 -- 172.21.15.33:0/3193109960 wait complete. 2024-04-02T13:46:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.291+0000 7ff6f0960640 1 Processor -- start 2024-04-02T13:46:59.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.291+0000 7ff6f0960640 1 -- start start 2024-04-02T13:46:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6f0960640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6ec068ce0 0x7ff6ec198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6f0960640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6ec0696a0 0x7ff6ec198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6f0960640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec107a80 0x7ff6ec19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6f0960640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec111e60 con 0x7ff6ec107a80 2024-04-02T13:46:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6f0960640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec111ce0 con 0x7ff6ec0696a0 2024-04-02T13:46:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6f0960640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec111fe0 con 0x7ff6ec068ce0 2024-04-02T13:46:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6ead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec107a80 0x7ff6ec19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6ead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec107a80 0x7ff6ec19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41328/0 (socket says 172.21.15.33:41328) 2024-04-02T13:46:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6ead9d640 1 -- 172.21.15.33:0/324213356 learned_addr learned my addr 172.21.15.33:0/324213356 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6e9d9b640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6ec0696a0 0x7ff6ec198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6ea59c640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6ec068ce0 0x7ff6ec198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6ead9d640 1 -- 172.21.15.33:0/324213356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6ec068ce0 msgr2=0x7ff6ec198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6ead9d640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6ec068ce0 0x7ff6ec198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6ead9d640 1 -- 172.21.15.33:0/324213356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6ec0696a0 msgr2=0x7ff6ec198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6ead9d640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6ec0696a0 0x7ff6ec198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6ead9d640 1 -- 172.21.15.33:0/324213356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ec19d330 con 0x7ff6ec107a80 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6ead9d640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec107a80 0x7ff6ec19cc00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc005cc0 tx=0x7ff6dc034ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6db7fe640 1 -- 172.21.15.33:0/324213356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc039050 con 0x7ff6ec107a80 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6db7fe640 1 -- 172.21.15.33:0/324213356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff6dc03a040 con 0x7ff6ec107a80 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6f0960640 1 -- 172.21.15.33:0/324213356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ec19d5c0 con 0x7ff6ec107a80 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6db7fe640 1 -- 172.21.15.33:0/324213356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc004570 con 0x7ff6ec107a80 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.292+0000 7ff6f0960640 1 -- 172.21.15.33:0/324213356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ec1a4eb0 con 0x7ff6ec107a80 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.295+0000 7ff6db7fe640 1 -- 172.21.15.33:0/324213356 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff6dc0327d0 con 0x7ff6ec107a80 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.295+0000 7ff6f3625740 1 Processor -- start 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.295+0000 7ff6f3625740 1 -- start start 2024-04-02T13:46:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.295+0000 7ff6f3625740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af67a0 0x55aaa8af6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.295+0000 7ff6f3625740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8af70c0 0x55aaa8aff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.295+0000 7ff6db7fe640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff6d40a3d20 0x7ff6d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.295+0000 7ff6f3625740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8affca0 0x55aaa8b02090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.295+0000 7ff6f3625740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aaa8a5a7b0 con 0x55aaa8affca0 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.295+0000 7ff6f3625740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55aaa89f9570 con 0x55aaa8af67a0 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.295+0000 7ff6f3625740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55aaa8a56760 con 0x55aaa8af70c0 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6ea59c640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff6d40a3d20 0x7ff6d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6db7fe640 1 -- 172.21.15.33:0/324213356 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7ff6dc0f3190 con 0x7ff6ec107a80 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6ead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8affca0 0x55aaa8b02090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6ead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8affca0 0x55aaa8b02090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41342/0 (socket says 172.21.15.33:41342) 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6ead9d640 1 -- 172.21.15.33:0/3263511946 learned_addr learned my addr 172.21.15.33:0/3263511946 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6e9d9b640 1 --2- 172.21.15.33:0/3263511946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af67a0 0x55aaa8af6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6e9d9b640 1 --2- 172.21.15.33:0/3263511946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8af70c0 0x55aaa8aff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6ead9d640 1 -- 172.21.15.33:0/3263511946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8af70c0 msgr2=0x55aaa8aff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6ead9d640 1 --2- 172.21.15.33:0/3263511946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8af70c0 0x55aaa8aff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6ead9d640 1 -- 172.21.15.33:0/3263511946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af67a0 msgr2=0x55aaa8af6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6ead9d640 1 --2- 172.21.15.33:0/3263511946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af67a0 0x55aaa8af6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6ead9d640 1 -- 172.21.15.33:0/3263511946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaa876e3f0 con 0x55aaa8affca0 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6e9d9b640 1 --2- 172.21.15.33:0/3263511946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af67a0 0x55aaa8af6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6ea59c640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff6d40a3d20 0x7ff6d40a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4005fd0 tx=0x7ff6e400b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6ead9d640 1 --2- 172.21.15.33:0/3263511946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8affca0 0x55aaa8b02090 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc02c630 tx=0x7ff6dc0bb1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b80ced2bd15a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6d8ff9640 1 -- 172.21.15.33:0/3263511946 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0be7d0 con 0x55aaa8affca0 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6d8ff9640 1 -- 172.21.15.33:0/3263511946 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff6dc0bf570 con 0x55aaa8affca0 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.296+0000 7ff6d8ff9640 1 -- 172.21.15.33:0/3263511946 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0d1f30 con 0x55aaa8affca0 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.297+0000 7ff6f3625740 1 -- 172.21.15.33:0/3263511946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8affca0 msgr2=0x55aaa8b02090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.297+0000 7ff6f3625740 1 --2- 172.21.15.33:0/3263511946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8affca0 0x55aaa8b02090 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc02c630 tx=0x7ff6dc0bb1b0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.297+0000 7ff6f3625740 1 -- 172.21.15.33:0/3263511946 shutdown_connections 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.297+0000 7ff6f3625740 1 --2- 172.21.15.33:0/3263511946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8affca0 0x55aaa8b02090 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.297+0000 7ff6f3625740 1 --2- 172.21.15.33:0/3263511946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8af70c0 0x55aaa8aff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.297+0000 7ff6f3625740 1 --2- 172.21.15.33:0/3263511946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af67a0 0x55aaa8af6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.297+0000 7ff6f3625740 1 -- 172.21.15.33:0/3263511946 >> 172.21.15.33:0/3263511946 conn(0x55aaa8af0ed0 msgr2=0x55aaa8af12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.297+0000 7ff6f3625740 1 -- 172.21.15.33:0/3263511946 shutdown_connections 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.297+0000 7ff6f3625740 1 -- 172.21.15.33:0/3263511946 wait complete. 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.297+0000 7ff6f3625740 1 -- start start 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6f3625740 1 Processor -- start 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6f3625740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8af67a0 0x55aaa8aff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6f3625740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 0x55aaa8afca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6f3625740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8affca0 0x55aaa8afd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6f3625740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aaa8b02850 con 0x55aaa8af70c0 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6f3625740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55aaa8a15e80 con 0x55aaa8affca0 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6f3625740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55aaa8b029d0 con 0x55aaa8af67a0 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6f3625740 10 client.0 _ll_register_callbacks cb 0x55aaa8a51aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6e9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 0x55aaa8afca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6e9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 0x55aaa8afca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41358/0 (socket says 172.21.15.33:41358) 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6e9d9b640 1 -- 172.21.15.33:0/3006106240 learned_addr learned my addr 172.21.15.33:0/3006106240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6ea59c640 1 --2- 172.21.15.33:0/3006106240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8af67a0 0x55aaa8aff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6ead9d640 1 --2- 172.21.15.33:0/3006106240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8affca0 0x55aaa8afd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6e9d9b640 1 -- 172.21.15.33:0/3006106240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8af67a0 msgr2=0x55aaa8aff6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6e9d9b640 1 --2- 172.21.15.33:0/3006106240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8af67a0 0x55aaa8aff6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6e9d9b640 1 -- 172.21.15.33:0/3006106240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8affca0 msgr2=0x55aaa8afd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6e9d9b640 1 --2- 172.21.15.33:0/3006106240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8affca0 0x55aaa8afd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6e9d9b640 1 -- 172.21.15.33:0/3006106240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaa88387d0 con 0x55aaa8af70c0 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6e9d9b640 1 --2- 172.21.15.33:0/3006106240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 0x55aaa8afca50 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00107d0 tx=0x7ff6e0010c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0111f1c47e1a798 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6d1ffb640 1 -- 172.21.15.33:0/3006106240 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000fd50 con 0x55aaa8af70c0 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.298+0000 7ff6d1ffb640 1 -- 172.21.15.33:0/3006106240 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7ff6e001c3e0 con 0x55aaa8af70c0 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.299+0000 7ff6f3625740 1 -- 172.21.15.33:0/3006106240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6dc0bf570 con 0x55aaa8af70c0 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.299+0000 7ff6d1ffb640 1 -- 172.21.15.33:0/3006106240 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff6e001c5e0 con 0x55aaa8af70c0 2024-04-02T13:46:59.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.299+0000 7ff6f3625740 10 client.17397 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:59.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.299+0000 7ff6f3625740 1 -- 172.21.15.33:0/3006106240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55aaa8b0e850 con 0x55aaa8af70c0 2024-04-02T13:46:59.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.299+0000 7ff6d1ffb640 1 -- 172.21.15.33:0/3006106240 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff6dc0bf570 con 0x55aaa8af70c0 2024-04-02T13:46:59.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.299+0000 7ff6d1ffb640 1 -- 172.21.15.33:0/3006106240 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7ff6e001b070 con 0x55aaa8af70c0 2024-04-02T13:46:59.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.299+0000 7ff6f3625740 10 client.17397 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:59.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.299+0000 7ff6f3625740 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:59.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.299+0000 7ff6f3625740 20 client.17397 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:59.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.299+0000 7ff6f3625740 1 --2- 172.21.15.33:0/3006106240 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aaa8b0ff10 0x55aaa8b12340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.299+0000 7ff6f3625740 4 client.17397 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:59.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.299+0000 7ff6f3625740 1 -- 172.21.15.33:0/3006106240 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaa8a67cf0 con 0x55aaa8b0ff10 2024-04-02T13:46:59.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.299+0000 7ff6ea59c640 1 --2- 172.21.15.33:0/3006106240 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aaa8b0ff10 0x55aaa8b12340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.300+0000 7ff6d1ffb640 10 client.17397 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.300+0000 7ff6ea59c640 1 --2- 172.21.15.33:0/3006106240 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aaa8b0ff10 0x55aaa8b12340 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a85b0cd341fb7c4 server_cookie=dce4b8986496d8a7 in_seq=0 out_seq=0 2024-04-02T13:46:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.300+0000 7ff6d1ffb640 1 -- 172.21.15.33:0/3006106240 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aaa8a67cf0 con 0x55aaa8b0ff10 2024-04-02T13:46:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.300+0000 7ff6d1ffb640 10 client.17397 handle_command_reply: tid=0 2024-04-02T13:46:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.300+0000 7ff6f3625740 1 client.17397 shutdown 2024-04-02T13:46:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.300+0000 7ff6f3625740 1 -- 172.21.15.33:0/3006106240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 msgr2=0x55aaa8afca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.300+0000 7ff6f3625740 1 --2- 172.21.15.33:0/3006106240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 0x55aaa8afca50 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00107d0 tx=0x7ff6e0010c90 comp rx=0 tx=0).stop 2024-04-02T13:46:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.301+0000 7ff6f3625740 1 -- 172.21.15.33:0/3006106240 shutdown_connections 2024-04-02T13:46:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.301+0000 7ff6f3625740 1 --2- 172.21.15.33:0/3006106240 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aaa8b0ff10 0x55aaa8b12340 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.301+0000 7ff6f3625740 1 --2- 172.21.15.33:0/3006106240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8affca0 0x55aaa8afd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.301+0000 7ff6f3625740 1 --2- 172.21.15.33:0/3006106240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 0x55aaa8afca50 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.301+0000 7ff6f3625740 1 --2- 172.21.15.33:0/3006106240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8af67a0 0x55aaa8aff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.301+0000 7ff6d1ffb640 0 client.17397 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.301+0000 7ff6f3625740 1 -- 172.21.15.33:0/3006106240 >> 172.21.15.33:0/3006106240 conn(0x55aaa8af0ed0 msgr2=0x55aaa8af2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.301+0000 7ff6f3625740 1 -- 172.21.15.33:0/3006106240 shutdown_connections 2024-04-02T13:46:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.301+0000 7ff6f3625740 1 -- 172.21.15.33:0/3006106240 wait complete. 2024-04-02T13:46:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.301+0000 7ff6f3625740 20 client.17397 trim_cache size 0 max 16384 2024-04-02T13:46:59.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.314+0000 7ff6f3625740 1 Processor -- start 2024-04-02T13:46:59.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.314+0000 7ff6f3625740 1 -- start start 2024-04-02T13:46:59.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.314+0000 7ff6f3625740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af67a0 0x55aaa8b0b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.314+0000 7ff6f3625740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af70c0 0x55aaa8aff180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.314+0000 7ff6f3625740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8affca0 0x55aaa8aff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.314+0000 7ff6f3625740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1b18b0 con 0x55aaa8af67a0 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.314+0000 7ff6f3625740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1af180 con 0x55aaa8af70c0 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.314+0000 7ff6f3625740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1acff0 con 0x55aaa8affca0 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.314+0000 7ff6ea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af67a0 0x55aaa8b0b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.314+0000 7ff6ea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af67a0 0x55aaa8b0b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41364/0 (socket says 172.21.15.33:41364) 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.314+0000 7ff6ea59c640 1 -- 172.21.15.33:0/1109152193 learned_addr learned my addr 172.21.15.33:0/1109152193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.314+0000 7ff6e9d9b640 1 --2- 172.21.15.33:0/1109152193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af70c0 0x55aaa8aff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6ead9d640 1 --2- 172.21.15.33:0/1109152193 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8affca0 0x55aaa8aff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6ea59c640 1 -- 172.21.15.33:0/1109152193 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8affca0 msgr2=0x55aaa8aff6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6ea59c640 1 --2- 172.21.15.33:0/1109152193 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8affca0 0x55aaa8aff6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6ea59c640 1 -- 172.21.15.33:0/1109152193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af70c0 msgr2=0x55aaa8aff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6ea59c640 1 --2- 172.21.15.33:0/1109152193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af70c0 0x55aaa8aff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6ea59c640 1 -- 172.21.15.33:0/1109152193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaa8b129c0 con 0x55aaa8af67a0 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6ead9d640 1 --2- 172.21.15.33:0/1109152193 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8affca0 0x55aaa8aff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6ea59c640 1 --2- 172.21.15.33:0/1109152193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af67a0 0x55aaa8b0b500 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400bef0 tx=0x7ff6e4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f32c81c41163875 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6d3fff640 1 -- 172.21.15.33:0/1109152193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4011940 con 0x55aaa8af67a0 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6d3fff640 1 -- 172.21.15.33:0/1109152193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff6e40283a0 con 0x55aaa8af67a0 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6d3fff640 1 -- 172.21.15.33:0/1109152193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4005600 con 0x55aaa8af67a0 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6f3625740 1 -- 172.21.15.33:0/1109152193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af67a0 msgr2=0x55aaa8b0b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6f3625740 1 --2- 172.21.15.33:0/1109152193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af67a0 0x55aaa8b0b500 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400bef0 tx=0x7ff6e4013000 comp rx=0 tx=0).stop 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6f3625740 1 -- 172.21.15.33:0/1109152193 shutdown_connections 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6f3625740 1 --2- 172.21.15.33:0/1109152193 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8affca0 0x55aaa8aff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6f3625740 1 --2- 172.21.15.33:0/1109152193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af70c0 0x55aaa8aff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.315+0000 7ff6f3625740 1 --2- 172.21.15.33:0/1109152193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af67a0 0x55aaa8b0b500 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6f3625740 1 -- 172.21.15.33:0/1109152193 >> 172.21.15.33:0/1109152193 conn(0x55aaa8af0ed0 msgr2=0x55aaa8af7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6f3625740 1 -- 172.21.15.33:0/1109152193 shutdown_connections 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6f3625740 1 -- 172.21.15.33:0/1109152193 wait complete. 2024-04-02T13:46:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6f3625740 1 -- start start 2024-04-02T13:46:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6f3625740 1 Processor -- start 2024-04-02T13:46:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6f3625740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af67a0 0x55aaa8b07cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6f3625740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 0x55aaa8b00680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6f3625740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8affca0 0x55aaa8b00bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6f3625740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1ad640 con 0x55aaa8af70c0 2024-04-02T13:46:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6f3625740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1aeab0 con 0x55aaa8af67a0 2024-04-02T13:46:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6f3625740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1add10 con 0x55aaa8affca0 2024-04-02T13:46:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6f3625740 10 client.0 _ll_register_callbacks cb 0x7ff6e0010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6e9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 0x55aaa8b00680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6e9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 0x55aaa8b00680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41374/0 (socket says 172.21.15.33:41374) 2024-04-02T13:46:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6e9d9b640 1 -- 172.21.15.33:0/2439450006 learned_addr learned my addr 172.21.15.33:0/2439450006 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:46:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6ea59c640 1 --2- 172.21.15.33:0/2439450006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af67a0 0x55aaa8b07cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6ead9d640 1 --2- 172.21.15.33:0/2439450006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8affca0 0x55aaa8b00bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6e9d9b640 1 -- 172.21.15.33:0/2439450006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8affca0 msgr2=0x55aaa8b00bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6e9d9b640 1 --2- 172.21.15.33:0/2439450006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8affca0 0x55aaa8b00bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6e9d9b640 1 -- 172.21.15.33:0/2439450006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af67a0 msgr2=0x55aaa8b07cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6e9d9b640 1 --2- 172.21.15.33:0/2439450006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af67a0 0x55aaa8b07cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.316+0000 7ff6e9d9b640 1 -- 172.21.15.33:0/2439450006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaa8b01300 con 0x55aaa8af70c0 2024-04-02T13:46:59.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6e9d9b640 1 --2- 172.21.15.33:0/2439450006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 0x55aaa8b00680 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0010540 tx=0x7ff6e000fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b673a965ef9df1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6d1ffb640 1 -- 172.21.15.33:0/2439450006 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e001d650 con 0x55aaa8af70c0 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6d1ffb640 1 -- 172.21.15.33:0/2439450006 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7ff6e001d7f0 con 0x55aaa8af70c0 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6f3625740 1 -- 172.21.15.33:0/2439450006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e40283a0 con 0x55aaa8af70c0 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6d1ffb640 1 -- 172.21.15.33:0/2439450006 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff6e00050b0 con 0x55aaa8af70c0 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6f3625740 10 client.17403 fetch_fsmap learned FSMap version 36 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6f3625740 1 -- 172.21.15.33:0/2439450006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55aaa8b764c0 con 0x55aaa8af70c0 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6d1ffb640 1 -- 172.21.15.33:0/2439450006 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff6e40283a0 con 0x55aaa8af70c0 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6d1ffb640 1 -- 172.21.15.33:0/2439450006 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7ff6e001b040 con 0x55aaa8af70c0 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6f3625740 10 client.17403 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6f3625740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6f3625740 20 client.17403 populate_metadata read hostname 'smithi033' 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6f3625740 1 --2- 172.21.15.33:0/2439450006 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aaa8b76aa0 0x55aaa8b78ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6f3625740 4 client.17403 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.317+0000 7ff6f3625740 1 -- 172.21.15.33:0/2439450006 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aaa8a62040 con 0x55aaa8b76aa0 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6ea59c640 1 --2- 172.21.15.33:0/2439450006 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aaa8b76aa0 0x55aaa8b78ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6d1ffb640 10 client.17403 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6ea59c640 1 --2- 172.21.15.33:0/2439450006 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aaa8b76aa0 0x55aaa8b78ec0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceaaa9c312373fd8 server_cookie=5969533fbe0d32d0 in_seq=0 out_seq=0 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6d1ffb640 1 -- 172.21.15.33:0/2439450006 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aaa8a62040 con 0x55aaa8b76aa0 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6d1ffb640 10 client.17403 handle_command_reply: tid=0 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6f3625740 1 client.17403 shutdown 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6f3625740 1 -- 172.21.15.33:0/2439450006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 msgr2=0x55aaa8b00680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6f3625740 1 --2- 172.21.15.33:0/2439450006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 0x55aaa8b00680 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0010540 tx=0x7ff6e000fe00 comp rx=0 tx=0).stop 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6f3625740 1 -- 172.21.15.33:0/2439450006 shutdown_connections 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6f3625740 1 --2- 172.21.15.33:0/2439450006 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55aaa8b76aa0 0x55aaa8b78ec0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6f3625740 1 --2- 172.21.15.33:0/2439450006 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55aaa8affca0 0x55aaa8b00bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6d1ffb640 0 client.17403 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:46:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6f3625740 1 --2- 172.21.15.33:0/2439450006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aaa8af70c0 0x55aaa8b00680 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6f3625740 1 --2- 172.21.15.33:0/2439450006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aaa8af67a0 0x55aaa8b07cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6f3625740 1 -- 172.21.15.33:0/2439450006 >> 172.21.15.33:0/2439450006 conn(0x55aaa8af0ed0 msgr2=0x55aaa8aef6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6f3625740 1 -- 172.21.15.33:0/2439450006 shutdown_connections 2024-04-02T13:46:59.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6f3625740 1 -- 172.21.15.33:0/2439450006 wait complete. 2024-04-02T13:46:59.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.318+0000 7ff6f3625740 20 client.17403 trim_cache size 0 max 16384 2024-04-02T13:46:59.333 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:46:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.319+0000 7ff6f0960640 1 -- 172.21.15.33:0/324213356 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff6d40a3d20 msgr2=0x7ff6d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.319+0000 7ff6f0960640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff6d40a3d20 0x7ff6d40a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4005fd0 tx=0x7ff6e400b390 comp rx=0 tx=0).stop 2024-04-02T13:46:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.319+0000 7ff6f0960640 1 -- 172.21.15.33:0/324213356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec107a80 msgr2=0x7ff6ec19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:59.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.319+0000 7ff6f0960640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec107a80 0x7ff6ec19cc00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc005cc0 tx=0x7ff6dc034ac0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.319+0000 7ff6f0960640 1 -- 172.21.15.33:0/324213356 shutdown_connections 2024-04-02T13:46:59.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.319+0000 7ff6f0960640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff6d40a3d20 0x7ff6d40a61e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.319+0000 7ff6f0960640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6ec107a80 0x7ff6ec19cc00 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.319+0000 7ff6f0960640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6ec0696a0 0x7ff6ec198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.319+0000 7ff6f0960640 1 --2- 172.21.15.33:0/324213356 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6ec068ce0 0x7ff6ec198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:59.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.319+0000 7ff6f0960640 1 -- 172.21.15.33:0/324213356 >> 172.21.15.33:0/324213356 conn(0x7ff6ec077b00 msgr2=0x7ff6ec0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:59.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.319+0000 7ff6f0960640 1 -- 172.21.15.33:0/324213356 shutdown_connections 2024-04-02T13:46:59.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:46:59.319+0000 7ff6f0960640 1 -- 172.21.15.33:0/324213356 wait complete. 2024-04-02T13:46:59.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:59 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:59 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:59 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:46:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:46:59 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/636525130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:46:59.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:59 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:59.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:59 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:59.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:59 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:46:59.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:46:59 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/636525130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:47:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:59 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:59 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:59 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:47:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:46:59 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/636525130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:47:00.337 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:47:00.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:47:00.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:47:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.445+0000 7fd521d8d640 1 -- 172.21.15.33:0/1521746223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd51c107aa0 msgr2=0x7fd51c107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.445+0000 7fd521d8d640 1 --2- 172.21.15.33:0/1521746223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd51c107aa0 0x7fd51c107e80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009930 tx=0x7fd50c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.445+0000 7fd521d8d640 1 -- 172.21.15.33:0/1521746223 shutdown_connections 2024-04-02T13:47:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.445+0000 7fd521d8d640 1 --2- 172.21.15.33:0/1521746223 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd51c102450 0x7fd51c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.445+0000 7fd521d8d640 1 --2- 172.21.15.33:0/1521746223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd51c101a90 0x7fd51c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.445+0000 7fd521d8d640 1 --2- 172.21.15.33:0/1521746223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd51c107aa0 0x7fd51c107e80 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.445+0000 7fd521d8d640 1 -- 172.21.15.33:0/1521746223 >> 172.21.15.33:0/1521746223 conn(0x7fd51c078f00 msgr2=0x7fd51c079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.445+0000 7fd521d8d640 1 -- 172.21.15.33:0/1521746223 shutdown_connections 2024-04-02T13:47:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.445+0000 7fd521d8d640 1 -- 172.21.15.33:0/1521746223 wait complete. 2024-04-02T13:47:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.446+0000 7fd521d8d640 1 Processor -- start 2024-04-02T13:47:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.446+0000 7fd521d8d640 1 -- start start 2024-04-02T13:47:00.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.446+0000 7fd521d8d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd51c101a90 0x7fd51c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.446+0000 7fd521d8d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd51c102450 0x7fd51c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.446+0000 7fd521d8d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd51c107aa0 0x7fd51c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.446+0000 7fd521d8d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c111e50 con 0x7fd51c101a90 2024-04-02T13:47:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.446+0000 7fd521d8d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c111cd0 con 0x7fd51c102450 2024-04-02T13:47:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.446+0000 7fd521d8d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c111fd0 con 0x7fd51c107aa0 2024-04-02T13:47:00.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.446+0000 7fd51b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd51c101a90 0x7fd51c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.446+0000 7fd51b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd51c101a90 0x7fd51c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41380/0 (socket says 172.21.15.33:41380) 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.446+0000 7fd51b7fe640 1 -- 172.21.15.33:0/657389200 learned_addr learned my addr 172.21.15.33:0/657389200 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.446+0000 7fd51bfff640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd51c107aa0 0x7fd51c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.447+0000 7fd51affd640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd51c102450 0x7fd51c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.447+0000 7fd51b7fe640 1 -- 172.21.15.33:0/657389200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd51c107aa0 msgr2=0x7fd51c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.447+0000 7fd51b7fe640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd51c107aa0 0x7fd51c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.447+0000 7fd51b7fe640 1 -- 172.21.15.33:0/657389200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd51c102450 msgr2=0x7fd51c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.447+0000 7fd51b7fe640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd51c102450 0x7fd51c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.447+0000 7fd51b7fe640 1 -- 172.21.15.33:0/657389200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c19d320 con 0x7fd51c101a90 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.447+0000 7fd51bfff640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd51c107aa0 0x7fd51c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.447+0000 7fd51b7fe640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd51c101a90 0x7fd51c198330 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fd50c000c00 tx=0x7fd50c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.447+0000 7fd518ff9640 1 -- 172.21.15.33:0/657389200 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c043020 con 0x7fd51c101a90 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.447+0000 7fd518ff9640 1 -- 172.21.15.33:0/657389200 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd50c02caa0 con 0x7fd51c101a90 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.447+0000 7fd521d8d640 1 -- 172.21.15.33:0/657389200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51c19d5b0 con 0x7fd51c101a90 2024-04-02T13:47:00.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.447+0000 7fd518ff9640 1 -- 172.21.15.33:0/657389200 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c003c20 con 0x7fd51c101a90 2024-04-02T13:47:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.447+0000 7fd521d8d640 1 -- 172.21.15.33:0/657389200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51c1a4ea0 con 0x7fd51c101a90 2024-04-02T13:47:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.449+0000 7fd518ff9640 1 -- 172.21.15.33:0/657389200 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd50c046020 con 0x7fd51c101a90 2024-04-02T13:47:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.449+0000 7fd524973740 1 Processor -- start 2024-04-02T13:47:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.449+0000 7fd524973740 1 -- start start 2024-04-02T13:47:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd524973740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 0x558ee34703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd524973740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee3470910 0x558ee347d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd524973740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee347dbd0 0x558ee347ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd524973740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558ee33d4770 con 0x558ee346fff0 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd524973740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558ee337c960 con 0x558ee347dbd0 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd524973740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558ee33d08c0 con 0x558ee3470910 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd51bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 0x558ee34703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd51bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 0x558ee34703d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41394/0 (socket says 172.21.15.33:41394) 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd51bfff640 1 -- 172.21.15.33:0/2208691165 learned_addr learned my addr 172.21.15.33:0/2208691165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd51affd640 1 --2- 172.21.15.33:0/2208691165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee347dbd0 0x558ee347ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd51b7fe640 1 --2- 172.21.15.33:0/2208691165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee3470910 0x558ee347d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd518ff9640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd5040a3d20 0x7fd5040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd51bfff640 1 -- 172.21.15.33:0/2208691165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee3470910 msgr2=0x558ee347d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd51bfff640 1 --2- 172.21.15.33:0/2208691165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee3470910 0x558ee347d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd51bfff640 1 -- 172.21.15.33:0/2208691165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee347dbd0 msgr2=0x558ee347ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd51bfff640 1 --2- 172.21.15.33:0/2208691165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee347dbd0 0x558ee347ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd51bfff640 1 -- 172.21.15.33:0/2208691165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee31b07e0 con 0x558ee346fff0 2024-04-02T13:47:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd51b7fe640 1 --2- 172.21.15.33:0/2208691165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee3470910 0x558ee347d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd518ff9640 1 -- 172.21.15.33:0/657389200 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fd50c0f2790 con 0x7fd51c101a90 2024-04-02T13:47:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd51bfff640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd5040a3d20 0x7fd5040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd51bfff640 1 --2- 172.21.15.33:0/2208691165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 0x558ee34703d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fd514007e60 tx=0x7fd51400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5948b58c03ec80a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.450+0000 7fd501ffb640 1 -- 172.21.15.33:0/2208691165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514020070 con 0x558ee346fff0 2024-04-02T13:47:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.451+0000 7fd501ffb640 1 -- 172.21.15.33:0/2208691165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd51401ae30 con 0x558ee346fff0 2024-04-02T13:47:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.451+0000 7fd501ffb640 1 -- 172.21.15.33:0/2208691165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514014070 con 0x558ee346fff0 2024-04-02T13:47:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.451+0000 7fd524973740 1 -- 172.21.15.33:0/2208691165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 msgr2=0x558ee34703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.451+0000 7fd51bfff640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd5040a3d20 0x7fd5040a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd51401d000 tx=0x7fd514020000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.451+0000 7fd524973740 1 --2- 172.21.15.33:0/2208691165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 0x558ee34703d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fd514007e60 tx=0x7fd51400e720 comp rx=0 tx=0).stop 2024-04-02T13:47:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.451+0000 7fd524973740 1 -- 172.21.15.33:0/2208691165 shutdown_connections 2024-04-02T13:47:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.451+0000 7fd524973740 1 --2- 172.21.15.33:0/2208691165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee347dbd0 0x558ee347ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.451+0000 7fd524973740 1 --2- 172.21.15.33:0/2208691165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee3470910 0x558ee347d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.451+0000 7fd524973740 1 --2- 172.21.15.33:0/2208691165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 0x558ee34703d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.451+0000 7fd524973740 1 -- 172.21.15.33:0/2208691165 >> 172.21.15.33:0/2208691165 conn(0x558ee346a9f0 msgr2=0x558ee346add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:00.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.451+0000 7fd524973740 1 -- 172.21.15.33:0/2208691165 shutdown_connections 2024-04-02T13:47:00.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.451+0000 7fd524973740 1 -- 172.21.15.33:0/2208691165 wait complete. 2024-04-02T13:47:00.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd524973740 1 -- start start 2024-04-02T13:47:00.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd524973740 1 Processor -- start 2024-04-02T13:47:00.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd524973740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 0x558ee347ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd524973740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee3470910 0x558ee347b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd524973740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee347dbd0 0x558ee347b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd524973740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558ee3480990 con 0x558ee346fff0 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd524973740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558ee338fd20 con 0x558ee3470910 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd524973740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558ee3480b10 con 0x558ee347dbd0 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd524973740 10 client.0 _ll_register_callbacks cb 0x558ee335ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd51b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 0x558ee347ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd51b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 0x558ee347ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41398/0 (socket says 172.21.15.33:41398) 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd51b7fe640 1 -- 172.21.15.33:0/2249467126 learned_addr learned my addr 172.21.15.33:0/2249467126 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd51affd640 1 --2- 172.21.15.33:0/2249467126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee3470910 0x558ee347b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd51bfff640 1 --2- 172.21.15.33:0/2249467126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee347dbd0 0x558ee347b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd51b7fe640 1 -- 172.21.15.33:0/2249467126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee347dbd0 msgr2=0x558ee347b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd51b7fe640 1 --2- 172.21.15.33:0/2249467126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee347dbd0 0x558ee347b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd51b7fe640 1 -- 172.21.15.33:0/2249467126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee3470910 msgr2=0x558ee347b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd51b7fe640 1 --2- 172.21.15.33:0/2249467126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee3470910 0x558ee347b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.452+0000 7fd51b7fe640 1 -- 172.21.15.33:0/2249467126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee3210f70 con 0x558ee346fff0 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd51b7fe640 1 --2- 172.21.15.33:0/2249467126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 0x558ee347ac60 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd50c005ab0 tx=0x7fd50c0be240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80eb982fd2fc9c03 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd4faffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd4faffd640 1 -- 172.21.15.33:0/2249467126 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c0beb70 con 0x558ee346fff0 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd4faffd640 1 -- 172.21.15.33:0/2249467126 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fd50c003dc0 con 0x558ee346fff0 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd524973740 1 -- 172.21.15.33:0/2249467126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd51401ae30 con 0x558ee346fff0 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd4faffd640 1 -- 172.21.15.33:0/2249467126 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd50c004c40 con 0x558ee346fff0 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd524973740 10 client.17415 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd524973740 1 -- 172.21.15.33:0/2249467126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558ee30a8170 con 0x558ee346fff0 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd4faffd640 1 -- 172.21.15.33:0/2249467126 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd51401ae30 con 0x558ee346fff0 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd4faffd640 1 -- 172.21.15.33:0/2249467126 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fd50c0baae0 con 0x558ee346fff0 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd524973740 10 client.17415 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd524973740 10 client.17415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd524973740 20 client.17415 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd524973740 1 --2- 172.21.15.33:0/2249467126 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558ee348e000 0x558ee34904a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd524973740 4 client.17415 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.453+0000 7fd524973740 1 -- 172.21.15.33:0/2249467126 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ee33e1cb0 con 0x558ee348e000 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.454+0000 7fd51affd640 1 --2- 172.21.15.33:0/2249467126 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558ee348e000 0x558ee34904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.454+0000 7fd51affd640 1 --2- 172.21.15.33:0/2249467126 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558ee348e000 0x558ee34904a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c7007168c04fc03 server_cookie=55f536b353e10928 in_seq=0 out_seq=0 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.454+0000 7fd4faffd640 10 client.17415 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd4faffd640 1 -- 172.21.15.33:0/2249467126 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ee33e1cb0 con 0x558ee348e000 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd4faffd640 10 client.17415 handle_command_reply: tid=0 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd524973740 1 client.17415 shutdown 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd524973740 1 -- 172.21.15.33:0/2249467126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 msgr2=0x558ee347ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd524973740 1 --2- 172.21.15.33:0/2249467126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 0x558ee347ac60 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd50c005ab0 tx=0x7fd50c0be240 comp rx=0 tx=0).stop 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd524973740 1 -- 172.21.15.33:0/2249467126 shutdown_connections 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd524973740 1 --2- 172.21.15.33:0/2249467126 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558ee348e000 0x558ee34904a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd524973740 1 --2- 172.21.15.33:0/2249467126 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee347dbd0 0x558ee347b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd524973740 1 --2- 172.21.15.33:0/2249467126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee3470910 0x558ee347b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd4faffd640 0 client.17415 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:00.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd524973740 1 --2- 172.21.15.33:0/2249467126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee346fff0 0x558ee347ac60 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd524973740 1 -- 172.21.15.33:0/2249467126 >> 172.21.15.33:0/2249467126 conn(0x558ee346a9f0 msgr2=0x558ee346c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:00.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd524973740 1 -- 172.21.15.33:0/2249467126 shutdown_connections 2024-04-02T13:47:00.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd524973740 1 -- 172.21.15.33:0/2249467126 wait complete. 2024-04-02T13:47:00.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.455+0000 7fd524973740 20 client.17415 trim_cache size 0 max 16384 2024-04-02T13:47:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd524973740 1 Processor -- start 2024-04-02T13:47:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd524973740 1 -- start start 2024-04-02T13:47:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd524973740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee346fff0 0x558ee347d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd524973740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee3470910 0x558ee34898f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd524973740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee347dbd0 0x558ee3495db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd524973740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c1af5f0 con 0x558ee347dbd0 2024-04-02T13:47:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd524973740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c1acec0 con 0x558ee3470910 2024-04-02T13:47:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd524973740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c1aad30 con 0x558ee346fff0 2024-04-02T13:47:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd51bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee347dbd0 0x558ee3495db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd51bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee347dbd0 0x558ee3495db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41406/0 (socket says 172.21.15.33:41406) 2024-04-02T13:47:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd51bfff640 1 -- 172.21.15.33:0/1996925504 learned_addr learned my addr 172.21.15.33:0/1996925504 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd51bfff640 1 -- 172.21.15.33:0/1996925504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee346fff0 msgr2=0x558ee347d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd51bfff640 1 --2- 172.21.15.33:0/1996925504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee346fff0 0x558ee347d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd51bfff640 1 -- 172.21.15.33:0/1996925504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee3470910 msgr2=0x558ee34898f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd51affd640 1 --2- 172.21.15.33:0/1996925504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee3470910 0x558ee34898f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd51bfff640 1 --2- 172.21.15.33:0/1996925504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee3470910 0x558ee34898f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd51bfff640 1 -- 172.21.15.33:0/1996925504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee3480870 con 0x558ee347dbd0 2024-04-02T13:47:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd51affd640 1 --2- 172.21.15.33:0/1996925504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee3470910 0x558ee34898f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.473+0000 7fd51bfff640 1 --2- 172.21.15.33:0/1996925504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee347dbd0 0x558ee3495db0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd51400f010 tx=0x7fd514023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d28bde986f72a9f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.474+0000 7fd5017fa640 1 -- 172.21.15.33:0/1996925504 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514014040 con 0x558ee347dbd0 2024-04-02T13:47:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.474+0000 7fd5017fa640 1 -- 172.21.15.33:0/1996925504 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd514007940 con 0x558ee347dbd0 2024-04-02T13:47:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.474+0000 7fd5017fa640 1 -- 172.21.15.33:0/1996925504 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51400f3c0 con 0x558ee347dbd0 2024-04-02T13:47:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.474+0000 7fd524973740 1 -- 172.21.15.33:0/1996925504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee347dbd0 msgr2=0x558ee3495db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.474+0000 7fd524973740 1 --2- 172.21.15.33:0/1996925504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee347dbd0 0x558ee3495db0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd51400f010 tx=0x7fd514023000 comp rx=0 tx=0).stop 2024-04-02T13:47:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.474+0000 7fd524973740 1 -- 172.21.15.33:0/1996925504 shutdown_connections 2024-04-02T13:47:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.474+0000 7fd524973740 1 --2- 172.21.15.33:0/1996925504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee347dbd0 0x558ee3495db0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.474+0000 7fd524973740 1 --2- 172.21.15.33:0/1996925504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee3470910 0x558ee34898f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.474+0000 7fd524973740 1 --2- 172.21.15.33:0/1996925504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee346fff0 0x558ee347d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.474+0000 7fd524973740 1 -- 172.21.15.33:0/1996925504 >> 172.21.15.33:0/1996925504 conn(0x558ee346a9f0 msgr2=0x558ee3469ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.474+0000 7fd524973740 1 -- 172.21.15.33:0/1996925504 shutdown_connections 2024-04-02T13:47:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.474+0000 7fd524973740 1 -- 172.21.15.33:0/1996925504 wait complete. 2024-04-02T13:47:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.474+0000 7fd524973740 1 -- start start 2024-04-02T13:47:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd524973740 1 Processor -- start 2024-04-02T13:47:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd524973740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee346fff0 0x558ee3482010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd524973740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee3470910 0x558ee3482550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd524973740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee347dbd0 0x558ee34f3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd524973740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c1ab380 con 0x558ee3470910 2024-04-02T13:47:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd524973740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c1ac7f0 con 0x558ee346fff0 2024-04-02T13:47:00.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd524973740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c1aba50 con 0x558ee347dbd0 2024-04-02T13:47:00.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd524973740 10 client.0 _ll_register_callbacks cb 0x7fd50c0ba0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:00.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd51affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee3470910 0x558ee3482550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd51affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee3470910 0x558ee3482550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41416/0 (socket says 172.21.15.33:41416) 2024-04-02T13:47:00.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd51affd640 1 -- 172.21.15.33:0/1106639729 learned_addr learned my addr 172.21.15.33:0/1106639729 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:00.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd51b7fe640 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee346fff0 0x558ee3482010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd51bfff640 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee347dbd0 0x558ee34f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd51affd640 1 -- 172.21.15.33:0/1106639729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee347dbd0 msgr2=0x558ee34f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd51affd640 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee347dbd0 0x558ee34f3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd51affd640 1 -- 172.21.15.33:0/1106639729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee346fff0 msgr2=0x558ee3482010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd51affd640 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee346fff0 0x558ee3482010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd51affd640 1 -- 172.21.15.33:0/1106639729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee34f3be0 con 0x558ee3470910 2024-04-02T13:47:00.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd51b7fe640 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee346fff0 0x558ee3482010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:00.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd51affd640 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee3470910 0x558ee3482550 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x558ee346bb80 tx=0x7fd51000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826e927d48eea077 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:00.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.475+0000 7fd4faffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:00.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd4faffd640 1 -- 172.21.15.33:0/1106639729 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51001d250 con 0x558ee3470910 2024-04-02T13:47:00.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd4faffd640 1 -- 172.21.15.33:0/1106639729 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fd51001d3f0 con 0x558ee3470910 2024-04-02T13:47:00.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd524973740 1 -- 172.21.15.33:0/1106639729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd514007940 con 0x558ee3470910 2024-04-02T13:47:00.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd4faffd640 1 -- 172.21.15.33:0/1106639729 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd51002d050 con 0x558ee3470910 2024-04-02T13:47:00.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd524973740 10 client.17421 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:00.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd524973740 1 -- 172.21.15.33:0/1106639729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558ee34f4f00 con 0x558ee3470910 2024-04-02T13:47:00.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd4faffd640 1 -- 172.21.15.33:0/1106639729 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd514007940 con 0x558ee3470910 2024-04-02T13:47:00.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd4faffd640 1 -- 172.21.15.33:0/1106639729 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fd510011cb0 con 0x558ee3470910 2024-04-02T13:47:00.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd524973740 10 client.17421 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:00.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd524973740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd524973740 20 client.17421 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd524973740 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558ee34f5660 0x558ee34f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd524973740 4 client.17421 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:47:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd524973740 1 -- 172.21.15.33:0/1106639729 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558ee33dc000 con 0x558ee34f5660 2024-04-02T13:47:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.476+0000 7fd51b7fe640 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558ee34f5660 0x558ee34f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.477+0000 7fd51b7fe640 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558ee34f5660 0x558ee34f7a80 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53e1fac2e2a5a202 server_cookie=4a4f1386f1db0cd8 in_seq=0 out_seq=0 2024-04-02T13:47:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.477+0000 7fd4faffd640 10 client.17421 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd4faffd640 1 -- 172.21.15.33:0/1106639729 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558ee33dc000 con 0x558ee34f5660 2024-04-02T13:47:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd4faffd640 10 client.17421 handle_command_reply: tid=0 2024-04-02T13:47:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd524973740 1 client.17421 shutdown 2024-04-02T13:47:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd524973740 1 -- 172.21.15.33:0/1106639729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee3470910 msgr2=0x558ee3482550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd524973740 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee3470910 0x558ee3482550 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x558ee346bb80 tx=0x7fd51000ef90 comp rx=0 tx=0).stop 2024-04-02T13:47:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd524973740 1 -- 172.21.15.33:0/1106639729 shutdown_connections 2024-04-02T13:47:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd524973740 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558ee34f5660 0x558ee34f7a80 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd524973740 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558ee347dbd0 0x558ee34f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd524973740 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ee3470910 0x558ee3482550 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd524973740 1 --2- 172.21.15.33:0/1106639729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ee346fff0 0x558ee3482010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd524973740 1 -- 172.21.15.33:0/1106639729 >> 172.21.15.33:0/1106639729 conn(0x558ee346a9f0 msgr2=0x558ee346bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd4faffd640 0 client.17421 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd524973740 1 -- 172.21.15.33:0/1106639729 shutdown_connections 2024-04-02T13:47:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd524973740 1 -- 172.21.15.33:0/1106639729 wait complete. 2024-04-02T13:47:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.478+0000 7fd524973740 20 client.17421 trim_cache size 0 max 16384 2024-04-02T13:47:00.492 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:47:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:47:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "3233ed55-57b1-4940-a4f7-659ab6b6b030", 2024-04-02T13:47:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:47:00.493 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:47:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.479+0000 7fd521d8d640 1 -- 172.21.15.33:0/657389200 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd5040a3d20 msgr2=0x7fd5040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.479+0000 7fd521d8d640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd5040a3d20 0x7fd5040a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd51401d000 tx=0x7fd514020000 comp rx=0 tx=0).stop 2024-04-02T13:47:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.479+0000 7fd521d8d640 1 -- 172.21.15.33:0/657389200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd51c101a90 msgr2=0x7fd51c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.479+0000 7fd521d8d640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd51c101a90 0x7fd51c198330 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fd50c000c00 tx=0x7fd50c0045e0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.479+0000 7fd521d8d640 1 -- 172.21.15.33:0/657389200 shutdown_connections 2024-04-02T13:47:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.479+0000 7fd521d8d640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd5040a3d20 0x7fd5040a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.479+0000 7fd521d8d640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd51c107aa0 0x7fd51c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.479+0000 7fd521d8d640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd51c102450 0x7fd51c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.479+0000 7fd521d8d640 1 --2- 172.21.15.33:0/657389200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd51c101a90 0x7fd51c198330 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.479+0000 7fd521d8d640 1 -- 172.21.15.33:0/657389200 >> 172.21.15.33:0/657389200 conn(0x7fd51c078f00 msgr2=0x7fd51c104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.479+0000 7fd521d8d640 1 -- 172.21.15.33:0/657389200 shutdown_connections 2024-04-02T13:47:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.479+0000 7fd521d8d640 1 -- 172.21.15.33:0/657389200 wait complete. 2024-04-02T13:47:00.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:47:00.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:47:00.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.599+0000 7f5413893640 1 -- 172.21.15.33:0/2942067440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f540c101a90 msgr2=0x7f540c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.599+0000 7f5413893640 1 --2- 172.21.15.33:0/2942067440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f540c101a90 0x7f540c101f10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5400009930 tx=0x7f540002bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:00.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 -- 172.21.15.33:0/2942067440 shutdown_connections 2024-04-02T13:47:00.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 --2- 172.21.15.33:0/2942067440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f540c102450 0x7f540c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 --2- 172.21.15.33:0/2942067440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f540c101a90 0x7f540c101f10 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 --2- 172.21.15.33:0/2942067440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f540c107aa0 0x7f540c107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 -- 172.21.15.33:0/2942067440 >> 172.21.15.33:0/2942067440 conn(0x7f540c0fd5a0 msgr2=0x7f540c0ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:00.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 -- 172.21.15.33:0/2942067440 shutdown_connections 2024-04-02T13:47:00.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 -- 172.21.15.33:0/2942067440 wait complete. 2024-04-02T13:47:00.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 Processor -- start 2024-04-02T13:47:00.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 -- start start 2024-04-02T13:47:00.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f540c101a90 0x7f540c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f540c102450 0x7f540c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f540c107aa0 0x7f540c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f540c111e80 con 0x7f540c101a90 2024-04-02T13:47:00.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f540c111d00 con 0x7f540c107aa0 2024-04-02T13:47:00.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.600+0000 7f5413893640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f540c112000 con 0x7f540c102450 2024-04-02T13:47:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f541162f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f540c101a90 0x7f540c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f541162f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f540c101a90 0x7f540c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41446/0 (socket says 172.21.15.33:41446) 2024-04-02T13:47:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f541162f640 1 -- 172.21.15.33:0/734959276 learned_addr learned my addr 172.21.15.33:0/734959276 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f5410e2e640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f540c102450 0x7f540c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f5411e30640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f540c107aa0 0x7f540c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f541162f640 1 -- 172.21.15.33:0/734959276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f540c102450 msgr2=0x7f540c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f541162f640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f540c102450 0x7f540c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f541162f640 1 -- 172.21.15.33:0/734959276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f540c107aa0 msgr2=0x7f540c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f541162f640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f540c107aa0 0x7f540c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f541162f640 1 -- 172.21.15.33:0/734959276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c19d3d0 con 0x7f540c101a90 2024-04-02T13:47:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f5411e30640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f540c107aa0 0x7f540c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f541162f640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f540c101a90 0x7f540c1983e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f540400ebc0 tx=0x7f540400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f540a7fc640 1 -- 172.21.15.33:0/734959276 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404010630 con 0x7f540c101a90 2024-04-02T13:47:00.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f5413893640 1 -- 172.21.15.33:0/734959276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c19d6c0 con 0x7f540c101a90 2024-04-02T13:47:00.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f540a7fc640 1 -- 172.21.15.33:0/734959276 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5404010ca0 con 0x7f540c101a90 2024-04-02T13:47:00.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f540a7fc640 1 -- 172.21.15.33:0/734959276 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404018c00 con 0x7f540c101a90 2024-04-02T13:47:00.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.601+0000 7f5413893640 1 -- 172.21.15.33:0/734959276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c1a4f50 con 0x7f540c101a90 2024-04-02T13:47:00.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.604+0000 7f5416484740 1 Processor -- start 2024-04-02T13:47:00.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.604+0000 7f540a7fc640 1 -- 172.21.15.33:0/734959276 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f54040107d0 con 0x7f540c101a90 2024-04-02T13:47:00.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.604+0000 7f5416484740 1 -- start start 2024-04-02T13:47:00.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.604+0000 7f5416484740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8d8d0 0x55f886a8dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.604+0000 7f5416484740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8e1f0 0x55f886a96890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.604+0000 7f5416484740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a96dd0 0x55f886a991c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.604+0000 7f5416484740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f8869f1830 con 0x55f886a8d8d0 2024-04-02T13:47:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.604+0000 7f5416484740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f886999820 con 0x55f886a96dd0 2024-04-02T13:47:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.604+0000 7f5416484740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f8869ed7e0 con 0x55f886a8e1f0 2024-04-02T13:47:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f5411e30640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8d8d0 0x55f886a8dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f5411e30640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8d8d0 0x55f886a8dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41452/0 (socket says 172.21.15.33:41452) 2024-04-02T13:47:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f5411e30640 1 -- 172.21.15.33:0/4252889726 learned_addr learned my addr 172.21.15.33:0/4252889726 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f5410e2e640 1 --2- 172.21.15.33:0/4252889726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a96dd0 0x55f886a991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f540a7fc640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f53f40a3d20 0x7f53f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f541162f640 1 --2- 172.21.15.33:0/4252889726 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8e1f0 0x55f886a96890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f5411e30640 1 -- 172.21.15.33:0/4252889726 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8e1f0 msgr2=0x55f886a96890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f5411e30640 1 --2- 172.21.15.33:0/4252889726 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8e1f0 0x55f886a96890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f5411e30640 1 -- 172.21.15.33:0/4252889726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a96dd0 msgr2=0x55f886a991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f5411e30640 1 --2- 172.21.15.33:0/4252889726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a96dd0 0x55f886a991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f5411e30640 1 -- 172.21.15.33:0/4252889726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8867cf7d0 con 0x55f886a8d8d0 2024-04-02T13:47:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f541162f640 1 --2- 172.21.15.33:0/4252889726 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8e1f0 0x55f886a96890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f5411e30640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f53f40a3d20 0x7f53f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f540a7fc640 1 -- 172.21.15.33:0/734959276 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f54040d1310 con 0x7f540c101a90 2024-04-02T13:47:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f5411e30640 1 --2- 172.21.15.33:0/4252889726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8d8d0 0x55f886a8dcb0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009d30 tx=0x7f53fc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eeaccdf16b805a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f53f37fe640 1 -- 172.21.15.33:0/4252889726 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc003ae0 con 0x55f886a8d8d0 2024-04-02T13:47:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f53f37fe640 1 -- 172.21.15.33:0/4252889726 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f53fc003c80 con 0x55f886a8d8d0 2024-04-02T13:47:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f53f37fe640 1 -- 172.21.15.33:0/4252889726 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0113e0 con 0x55f886a8d8d0 2024-04-02T13:47:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.605+0000 7f5411e30640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f53f40a3d20 0x7f53f40a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f53fc021000 tx=0x7f53fc01e000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.606+0000 7f5416484740 1 -- 172.21.15.33:0/4252889726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8d8d0 msgr2=0x55f886a8dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.606+0000 7f5416484740 1 --2- 172.21.15.33:0/4252889726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8d8d0 0x55f886a8dcb0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009d30 tx=0x7f53fc010740 comp rx=0 tx=0).stop 2024-04-02T13:47:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.606+0000 7f5416484740 1 -- 172.21.15.33:0/4252889726 shutdown_connections 2024-04-02T13:47:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.606+0000 7f5416484740 1 --2- 172.21.15.33:0/4252889726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a96dd0 0x55f886a991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.606+0000 7f5416484740 1 --2- 172.21.15.33:0/4252889726 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8e1f0 0x55f886a96890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.606+0000 7f5416484740 1 --2- 172.21.15.33:0/4252889726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8d8d0 0x55f886a8dcb0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.606+0000 7f5416484740 1 -- 172.21.15.33:0/4252889726 >> 172.21.15.33:0/4252889726 conn(0x55f886a88000 msgr2=0x55f886a883e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.606+0000 7f5416484740 1 -- 172.21.15.33:0/4252889726 shutdown_connections 2024-04-02T13:47:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.606+0000 7f5416484740 1 -- 172.21.15.33:0/4252889726 wait complete. 2024-04-02T13:47:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.606+0000 7f5416484740 1 -- start start 2024-04-02T13:47:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5416484740 1 Processor -- start 2024-04-02T13:47:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5416484740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 0x55f886a965d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5416484740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8e1f0 0x55f886a93960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5416484740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a96dd0 0x55f886a93ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5416484740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f886a99bc0 con 0x55f886a8e1f0 2024-04-02T13:47:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5416484740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f8869acd30 con 0x55f886a96dd0 2024-04-02T13:47:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5416484740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f886a99d40 con 0x55f886a8d8d0 2024-04-02T13:47:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5416484740 10 client.0 _ll_register_callbacks cb 0x55f8869ac720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5410e2e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8e1f0 0x55f886a93960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5410e2e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8e1f0 0x55f886a93960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41454/0 (socket says 172.21.15.33:41454) 2024-04-02T13:47:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5411e30640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a96dd0 0x55f886a93ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5410e2e640 1 -- 172.21.15.33:0/778501568 learned_addr learned my addr 172.21.15.33:0/778501568 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f541162f640 1 --2- 172.21.15.33:0/778501568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 0x55f886a965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5410e2e640 1 -- 172.21.15.33:0/778501568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 msgr2=0x55f886a965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5410e2e640 1 --2- 172.21.15.33:0/778501568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 0x55f886a965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5410e2e640 1 -- 172.21.15.33:0/778501568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a96dd0 msgr2=0x55f886a93ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5410e2e640 1 --2- 172.21.15.33:0/778501568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a96dd0 0x55f886a93ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5410e2e640 1 -- 172.21.15.33:0/778501568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8867053f0 con 0x55f886a8e1f0 2024-04-02T13:47:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f541162f640 1 --2- 172.21.15.33:0/778501568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 0x55f886a965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f5410e2e640 1 --2- 172.21.15.33:0/778501568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8e1f0 0x55f886a93960 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f5400005940 tx=0x7f5400005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621575475538e860 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.607+0000 7f53f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f53f0ff9640 1 -- 172.21.15.33:0/778501568 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400045070 con 0x55f886a8e1f0 2024-04-02T13:47:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f53f0ff9640 1 -- 172.21.15.33:0/778501568 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f540002c7d0 con 0x55f886a8e1f0 2024-04-02T13:47:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f5416484740 1 -- 172.21.15.33:0/778501568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53fc003c80 con 0x55f886a8e1f0 2024-04-02T13:47:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f53f0ff9640 1 -- 172.21.15.33:0/778501568 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5400051050 con 0x55f886a8e1f0 2024-04-02T13:47:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f5416484740 10 client.17433 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f5416484740 1 -- 172.21.15.33:0/778501568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f8866c50f0 con 0x55f886a8e1f0 2024-04-02T13:47:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f53f0ff9640 1 -- 172.21.15.33:0/778501568 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f53fc003c80 con 0x55f886a8e1f0 2024-04-02T13:47:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f53f0ff9640 1 -- 172.21.15.33:0/778501568 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f5400004cd0 con 0x55f886a8e1f0 2024-04-02T13:47:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f5416484740 10 client.17433 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f5416484740 10 client.17433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f5416484740 20 client.17433 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f5416484740 1 --2- 172.21.15.33:0/778501568 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f886aa6be0 0x55f886aa9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f5416484740 4 client.17433 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.608+0000 7f5416484740 1 -- 172.21.15.33:0/778501568 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8869fed70 con 0x55f886aa6be0 2024-04-02T13:47:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.609+0000 7f541162f640 1 --2- 172.21.15.33:0/778501568 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f886aa6be0 0x55f886aa9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.609+0000 7f541162f640 1 --2- 172.21.15.33:0/778501568 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f886aa6be0 0x55f886aa9030 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa5429636d85b42c server_cookie=6d3c8b2b0eb8f3b9 in_seq=0 out_seq=0 2024-04-02T13:47:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.609+0000 7f53f0ff9640 10 client.17433 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f53f0ff9640 1 -- 172.21.15.33:0/778501568 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f8869fed70 con 0x55f886aa6be0 2024-04-02T13:47:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f53f0ff9640 10 client.17433 handle_command_reply: tid=0 2024-04-02T13:47:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f5416484740 1 client.17433 shutdown 2024-04-02T13:47:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f5416484740 1 -- 172.21.15.33:0/778501568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8e1f0 msgr2=0x55f886a93960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f5416484740 1 --2- 172.21.15.33:0/778501568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8e1f0 0x55f886a93960 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f5400005940 tx=0x7f5400005270 comp rx=0 tx=0).stop 2024-04-02T13:47:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f5416484740 1 -- 172.21.15.33:0/778501568 shutdown_connections 2024-04-02T13:47:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f5416484740 1 --2- 172.21.15.33:0/778501568 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f886aa6be0 0x55f886aa9030 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f5416484740 1 --2- 172.21.15.33:0/778501568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a96dd0 0x55f886a93ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f5416484740 1 --2- 172.21.15.33:0/778501568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a8e1f0 0x55f886a93960 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f5416484740 1 --2- 172.21.15.33:0/778501568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 0x55f886a965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f53f0ff9640 0 client.17433 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f5416484740 1 -- 172.21.15.33:0/778501568 >> 172.21.15.33:0/778501568 conn(0x55f886a88000 msgr2=0x55f886a89d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f5416484740 1 -- 172.21.15.33:0/778501568 shutdown_connections 2024-04-02T13:47:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f5416484740 1 -- 172.21.15.33:0/778501568 wait complete. 2024-04-02T13:47:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.610+0000 7f5416484740 20 client.17433 trim_cache size 0 max 16384 2024-04-02T13:47:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.627+0000 7f5416484740 1 Processor -- start 2024-04-02T13:47:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.627+0000 7f5416484740 1 -- start start 2024-04-02T13:47:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5416484740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 0x55f886a8efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5416484740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a8e1f0 0x55f886a870c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5416484740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 0x55f886a96090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5416484740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1b1950 con 0x55f886a96dd0 2024-04-02T13:47:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5416484740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1af220 con 0x55f886a8e1f0 2024-04-02T13:47:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5416484740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1ad090 con 0x55f886a8d8d0 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5411e30640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 0x55f886a96090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5411e30640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 0x55f886a96090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41466/0 (socket says 172.21.15.33:41466) 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5411e30640 1 -- 172.21.15.33:0/527552301 learned_addr learned my addr 172.21.15.33:0/527552301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5410e2e640 1 --2- 172.21.15.33:0/527552301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a8e1f0 0x55f886a870c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5411e30640 1 -- 172.21.15.33:0/527552301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 msgr2=0x55f886a8efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5411e30640 1 --2- 172.21.15.33:0/527552301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 0x55f886a8efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5411e30640 1 -- 172.21.15.33:0/527552301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a8e1f0 msgr2=0x55f886a870c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5411e30640 1 --2- 172.21.15.33:0/527552301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a8e1f0 0x55f886a870c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5411e30640 1 -- 172.21.15.33:0/527552301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f886a8e020 con 0x55f886a96dd0 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f5411e30640 1 --2- 172.21.15.33:0/527552301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 0x55f886a96090 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f53fc010c40 tx=0x7f53fc010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7dfa7bb39a47e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f53f2ffd640 1 -- 172.21.15.33:0/527552301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc003a60 con 0x55f886a96dd0 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f53f2ffd640 1 -- 172.21.15.33:0/527552301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f53fc003c00 con 0x55f886a96dd0 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.628+0000 7f53f2ffd640 1 -- 172.21.15.33:0/527552301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc011ab0 con 0x55f886a96dd0 2024-04-02T13:47:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.629+0000 7f5416484740 1 -- 172.21.15.33:0/527552301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 msgr2=0x55f886a96090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.629+0000 7f5416484740 1 --2- 172.21.15.33:0/527552301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 0x55f886a96090 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f53fc010c40 tx=0x7f53fc010d70 comp rx=0 tx=0).stop 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.629+0000 7f5416484740 1 -- 172.21.15.33:0/527552301 shutdown_connections 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.629+0000 7f5416484740 1 --2- 172.21.15.33:0/527552301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 0x55f886a96090 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.629+0000 7f5416484740 1 --2- 172.21.15.33:0/527552301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a8e1f0 0x55f886a870c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.629+0000 7f5416484740 1 --2- 172.21.15.33:0/527552301 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 0x55f886a8efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.629+0000 7f5416484740 1 -- 172.21.15.33:0/527552301 >> 172.21.15.33:0/527552301 conn(0x55f886a88000 msgr2=0x55f886a869b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.629+0000 7f5416484740 1 -- 172.21.15.33:0/527552301 shutdown_connections 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.629+0000 7f5416484740 1 -- 172.21.15.33:0/527552301 wait complete. 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.629+0000 7f5416484740 1 -- start start 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.629+0000 7f5416484740 1 Processor -- start 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5416484740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 0x55f886a9b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5416484740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a8e1f0 0x55f886a9b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5416484740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 0x55f886b06140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5416484740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1ad6e0 con 0x55f886a96dd0 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5416484740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1aeb50 con 0x55f886a8e1f0 2024-04-02T13:47:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5416484740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1addb0 con 0x55f886a8d8d0 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5416484740 10 client.0 _ll_register_callbacks cb 0x7f5400009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5411e30640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 0x55f886b06140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5411e30640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 0x55f886b06140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41478/0 (socket says 172.21.15.33:41478) 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5411e30640 1 -- 172.21.15.33:0/81081075 learned_addr learned my addr 172.21.15.33:0/81081075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f541162f640 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 0x55f886a9b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5410e2e640 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a8e1f0 0x55f886a9b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5411e30640 1 -- 172.21.15.33:0/81081075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 msgr2=0x55f886a9b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5411e30640 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 0x55f886a9b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5411e30640 1 -- 172.21.15.33:0/81081075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a8e1f0 msgr2=0x55f886a9b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5411e30640 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a8e1f0 0x55f886a9b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5411e30640 1 -- 172.21.15.33:0/81081075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f886b06730 con 0x55f886a96dd0 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5410e2e640 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a8e1f0 0x55f886a9b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5411e30640 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 0x55f886b06140 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f53fc006010 tx=0x7f53fc005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcfd36c3343bb4a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f53f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f53f0ff9640 1 -- 172.21.15.33:0/81081075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc011890 con 0x55f886a96dd0 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f53f0ff9640 1 -- 172.21.15.33:0/81081075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f53fc02d040 con 0x55f886a96dd0 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.630+0000 7f5416484740 1 -- 172.21.15.33:0/81081075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53fc003c00 con 0x55f886a96dd0 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.631+0000 7f53f0ff9640 1 -- 172.21.15.33:0/81081075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f53fc041050 con 0x55f886a96dd0 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.631+0000 7f5416484740 10 client.17439 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.631+0000 7f5416484740 1 -- 172.21.15.33:0/81081075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f886b0dac0 con 0x55f886a96dd0 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.631+0000 7f53f0ff9640 1 -- 172.21.15.33:0/81081075 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f53fc003c00 con 0x55f886a96dd0 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.631+0000 7f53f0ff9640 1 -- 172.21.15.33:0/81081075 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f53fc00f040 con 0x55f886a96dd0 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.631+0000 7f5416484740 10 client.17439 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.631+0000 7f5416484740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.631+0000 7f5416484740 20 client.17439 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.631+0000 7f5416484740 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f886b0e010 0x55f886b10430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.631+0000 7f5416484740 4 client.17439 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.631+0000 7f5416484740 1 -- 172.21.15.33:0/81081075 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f8869f90c0 con 0x55f886b0e010 2024-04-02T13:47:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.631+0000 7f541162f640 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f886b0e010 0x55f886b10430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f53f0ff9640 10 client.17439 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f541162f640 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f886b0e010 0x55f886b10430 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=345efb6c3e1debe2 server_cookie=25e53404bde000f8 in_seq=0 out_seq=0 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f53f0ff9640 1 -- 172.21.15.33:0/81081075 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f8869f90c0 con 0x55f886b0e010 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f53f0ff9640 10 client.17439 handle_command_reply: tid=0 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f5416484740 1 client.17439 shutdown 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f5416484740 1 -- 172.21.15.33:0/81081075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 msgr2=0x55f886b06140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f5416484740 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 0x55f886b06140 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f53fc006010 tx=0x7f53fc005650 comp rx=0 tx=0).stop 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f5416484740 1 -- 172.21.15.33:0/81081075 shutdown_connections 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f5416484740 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55f886b0e010 0x55f886b10430 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f5416484740 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f886a96dd0 0x55f886b06140 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f53f0ff9640 0 client.17439 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f5416484740 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f886a8e1f0 0x55f886a9b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f5416484740 1 --2- 172.21.15.33:0/81081075 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f886a8d8d0 0x55f886a9b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f5416484740 1 -- 172.21.15.33:0/81081075 >> 172.21.15.33:0/81081075 conn(0x55f886a88000 msgr2=0x55f886a87240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f5416484740 1 -- 172.21.15.33:0/81081075 shutdown_connections 2024-04-02T13:47:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f5416484740 1 -- 172.21.15.33:0/81081075 wait complete. 2024-04-02T13:47:00.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.632+0000 7f5416484740 20 client.17439 trim_cache size 0 max 16384 2024-04-02T13:47:00.644 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:47:00.644 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:47:00.644 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:47:00.644 INFO:teuthology.orchestra.run.smithi033.stdout: "3233ed55-57b1-4940-a4f7-659ab6b6b030": { 2024-04-02T13:47:00.644 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:47:00.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "3233ed55-57b1-4940-a4f7-659ab6b6b030", 2024-04-02T13:47:00.645 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:47:00.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:47:00.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:47:00.645 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:47:00.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.633+0000 7f5413893640 1 -- 172.21.15.33:0/734959276 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f53f40a3d20 msgr2=0x7f53f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.633+0000 7f5413893640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f53f40a3d20 0x7f53f40a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f53fc021000 tx=0x7f53fc01e000 comp rx=0 tx=0).stop 2024-04-02T13:47:00.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.633+0000 7f5413893640 1 -- 172.21.15.33:0/734959276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f540c101a90 msgr2=0x7f540c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:00.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.633+0000 7f5413893640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f540c101a90 0x7f540c1983e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f540400ebc0 tx=0x7f540400c6a0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.633+0000 7f5413893640 1 -- 172.21.15.33:0/734959276 shutdown_connections 2024-04-02T13:47:00.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.633+0000 7f5413893640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f53f40a3d20 0x7f53f40a61e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.633+0000 7f5413893640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f540c107aa0 0x7f540c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.633+0000 7f5413893640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f540c102450 0x7f540c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.633+0000 7f5413893640 1 --2- 172.21.15.33:0/734959276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f540c101a90 0x7f540c1983e0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:00.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.633+0000 7f5413893640 1 -- 172.21.15.33:0/734959276 >> 172.21.15.33:0/734959276 conn(0x7f540c0fd5a0 msgr2=0x7f540c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:00.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.633+0000 7f5413893640 1 -- 172.21.15.33:0/734959276 shutdown_connections 2024-04-02T13:47:00.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:00.633+0000 7f5413893640 1 -- 172.21.15.33:0/734959276 wait complete. 2024-04-02T13:47:00.648 INFO:tasks.cephfs.filesystem:scrub status for tag:3233ed55-57b1-4940-a4f7-659ab6b6b030 - {'path': '/', 'tag': '3233ed55-57b1-4940-a4f7-659ab6b6b030', 'options': 'recursive,force'} 2024-04-02T13:47:00.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:00 smithi145 ceph-mon[35646]: pgmap v253: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:47:00.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:00 smithi050 ceph-mon[35524]: pgmap v253: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:47:01.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:00 smithi033 ceph-mon[32390]: pgmap v253: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:47:01.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:01 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:47:01.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:01 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:01.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:01 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:47:01.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:01 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:01.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:01 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:47:01.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:01 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:47:01.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:01 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:01.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:01 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:47:01.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:01 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:01.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:01 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:47:02.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:01 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:47:02.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:01 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:02.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:01 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:47:02.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:01 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:02.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:01 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:47:02.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:02 smithi145 ceph-mon[35646]: pgmap v254: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:02.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:02 smithi050 ceph-mon[35524]: pgmap v254: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:03.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:02 smithi033 ceph-mon[32390]: pgmap v254: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:04.982 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:04 smithi145 ceph-mon[35646]: pgmap v255: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-02T13:47:04.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:04 smithi050 ceph-mon[35524]: pgmap v255: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-02T13:47:05.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:04 smithi033 ceph-mon[32390]: pgmap v255: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-02T13:47:06.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:06 smithi145 ceph-mon[35646]: pgmap v256: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-02T13:47:06.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:06 smithi145 ceph-mon[35646]: osdmap e100: 12 total, 12 up, 12 in 2024-04-02T13:47:06.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:06 smithi050 ceph-mon[35524]: pgmap v256: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-02T13:47:06.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:06 smithi050 ceph-mon[35524]: osdmap e100: 12 total, 12 up, 12 in 2024-04-02T13:47:07.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:06 smithi033 ceph-mon[32390]: pgmap v256: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-02T13:47:07.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:06 smithi033 ceph-mon[32390]: osdmap e100: 12 total, 12 up, 12 in 2024-04-02T13:47:08.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:08 smithi145 ceph-mon[35646]: pgmap v258: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:47:08.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:08 smithi050 ceph-mon[35524]: pgmap v258: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:47:09.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:08 smithi033 ceph-mon[32390]: pgmap v258: 129 pgs: 129 active+clean; 265 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:47:10.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:09 smithi033 ceph-mon[32390]: pgmap v259: 129 pgs: 129 active+clean; 145 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-02T13:47:10.228 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:09 smithi145 ceph-mon[35646]: pgmap v259: 129 pgs: 129 active+clean; 145 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-02T13:47:10.244 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:09 smithi050 ceph-mon[35524]: pgmap v259: 129 pgs: 129 active+clean; 145 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-02T13:47:12.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:12 smithi145 ceph-mon[35646]: pgmap v260: 129 pgs: 129 active+clean; 145 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-02T13:47:12.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:12 smithi145 ceph-mon[35646]: osdmap e101: 12 total, 12 up, 12 in 2024-04-02T13:47:12.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:12 smithi050 ceph-mon[35524]: pgmap v260: 129 pgs: 129 active+clean; 145 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-02T13:47:12.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:12 smithi050 ceph-mon[35524]: osdmap e101: 12 total, 12 up, 12 in 2024-04-02T13:47:13.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:12 smithi033 ceph-mon[32390]: pgmap v260: 129 pgs: 129 active+clean; 145 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-02T13:47:13.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:12 smithi033 ceph-mon[32390]: osdmap e101: 12 total, 12 up, 12 in 2024-04-02T13:47:14.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:14 smithi145 ceph-mon[35646]: pgmap v262: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:47:14.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:14 smithi050 ceph-mon[35524]: pgmap v262: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:47:15.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:14 smithi033 ceph-mon[32390]: pgmap v262: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:47:16.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:16 smithi145 ceph-mon[35646]: pgmap v263: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 219 B/s wr, 7 op/s 2024-04-02T13:47:16.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:16 smithi050 ceph-mon[35524]: pgmap v263: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 219 B/s wr, 7 op/s 2024-04-02T13:47:17.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:16 smithi033 ceph-mon[32390]: pgmap v263: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 219 B/s wr, 7 op/s 2024-04-02T13:47:18.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:18 smithi145 ceph-mon[35646]: pgmap v264: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:47:18.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:18 smithi050 ceph-mon[35524]: pgmap v264: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:47:19.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:18 smithi033 ceph-mon[32390]: pgmap v264: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:47:20.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:20 smithi145 ceph-mon[35646]: pgmap v265: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:47:20.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:20 smithi050 ceph-mon[35524]: pgmap v265: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:47:21.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:20 smithi033 ceph-mon[32390]: pgmap v265: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:47:22.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:22 smithi145 ceph-mon[35646]: pgmap v266: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:47:22.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:22 smithi050 ceph-mon[35524]: pgmap v266: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:47:23.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:22 smithi033 ceph-mon[32390]: pgmap v266: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:47:24.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:24 smithi145 ceph-mon[35646]: pgmap v267: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 89 B/s wr, 9 op/s 2024-04-02T13:47:24.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:24 smithi050 ceph-mon[35524]: pgmap v267: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 89 B/s wr, 9 op/s 2024-04-02T13:47:25.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:24 smithi033 ceph-mon[32390]: pgmap v267: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 89 B/s wr, 9 op/s 2024-04-02T13:47:26.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:26 smithi145 ceph-mon[35646]: pgmap v268: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:26.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:26 smithi050 ceph-mon[35524]: pgmap v268: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:27.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:26 smithi033 ceph-mon[32390]: pgmap v268: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:28.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:28 smithi145 ceph-mon[35646]: pgmap v269: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:28.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:28 smithi050 ceph-mon[35524]: pgmap v269: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:29.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:28 smithi033 ceph-mon[32390]: pgmap v269: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:30.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:47:30.648 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:47:30.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.765+0000 7f760ed7b640 1 -- 172.21.15.33:0/727242567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7608101370 msgr2=0x7f760810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.765+0000 7f760ed7b640 1 --2- 172.21.15.33:0/727242567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7608101370 0x7f760810d880 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7600009cd0 tx=0x7f760002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.765+0000 7f760ed7b640 1 -- 172.21.15.33:0/727242567 shutdown_connections 2024-04-02T13:47:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.765+0000 7f760ed7b640 1 --2- 172.21.15.33:0/727242567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7608101370 0x7f760810d880 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.765+0000 7f760ed7b640 1 --2- 172.21.15.33:0/727242567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76081009b0 0x7f7608100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.765+0000 7f760ed7b640 1 --2- 172.21.15.33:0/727242567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76081069c0 0x7f7608106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.765+0000 7f760ed7b640 1 -- 172.21.15.33:0/727242567 >> 172.21.15.33:0/727242567 conn(0x7f7608076d80 msgr2=0x7f7608077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.765+0000 7f760ed7b640 1 -- 172.21.15.33:0/727242567 shutdown_connections 2024-04-02T13:47:30.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.765+0000 7f760ed7b640 1 -- 172.21.15.33:0/727242567 wait complete. 2024-04-02T13:47:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.765+0000 7f760ed7b640 1 Processor -- start 2024-04-02T13:47:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.765+0000 7f760ed7b640 1 -- start start 2024-04-02T13:47:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.765+0000 7f760ed7b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76081009b0 0x7f7608198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760ed7b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7608101370 0x7f76081988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760ed7b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76081069c0 0x7f760819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760ed7b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7608110da0 con 0x7f76081069c0 2024-04-02T13:47:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760ed7b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7608110c20 con 0x7f7608101370 2024-04-02T13:47:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760ed7b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7608110f20 con 0x7f76081009b0 2024-04-02T13:47:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760d318640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76081069c0 0x7f760819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760d318640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76081069c0 0x7f760819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43852/0 (socket says 172.21.15.33:43852) 2024-04-02T13:47:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760d318640 1 -- 172.21.15.33:0/706755875 learned_addr learned my addr 172.21.15.33:0/706755875 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760cb17640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76081009b0 0x7f7608198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f7607fff640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7608101370 0x7f76081988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760d318640 1 -- 172.21.15.33:0/706755875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76081009b0 msgr2=0x7f7608198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:30.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760d318640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76081009b0 0x7f7608198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760d318640 1 -- 172.21.15.33:0/706755875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7608101370 msgr2=0x7f76081988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:30.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760d318640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7608101370 0x7f76081988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760d318640 1 -- 172.21.15.33:0/706755875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760819d350 con 0x7f76081069c0 2024-04-02T13:47:30.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760d318640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76081069c0 0x7f760819cc20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7600005cc0 tx=0x7f7600034b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:30.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f7605ffb640 1 -- 172.21.15.33:0/706755875 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600039050 con 0x7f76081069c0 2024-04-02T13:47:30.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f7605ffb640 1 -- 172.21.15.33:0/706755875 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7600045070 con 0x7f76081069c0 2024-04-02T13:47:30.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f760ed7b640 1 -- 172.21.15.33:0/706755875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760819d5e0 con 0x7f76081069c0 2024-04-02T13:47:30.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.766+0000 7f7605ffb640 1 -- 172.21.15.33:0/706755875 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600004610 con 0x7f76081069c0 2024-04-02T13:47:30.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.767+0000 7f760ed7b640 1 -- 172.21.15.33:0/706755875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76081a4ed0 con 0x7f76081069c0 2024-04-02T13:47:30.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.769+0000 7f7605ffb640 1 -- 172.21.15.33:0/706755875 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7600003970 con 0x7f76081069c0 2024-04-02T13:47:30.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.769+0000 7f7611963740 1 Processor -- start 2024-04-02T13:47:30.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.769+0000 7f7611963740 1 -- start start 2024-04-02T13:47:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.769+0000 7f7605ffb640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f75f80a3d20 0x7f75f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.769+0000 7f7611963740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b88d0 0x55c68f4b8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.769+0000 7f7611963740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b91f0 0x55c68f4c1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.769+0000 7f7611963740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.769+0000 7f7611963740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c68f41c830 con 0x55c68f4c1dd0 2024-04-02T13:47:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.769+0000 7f7611963740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c68f3c4820 con 0x55c68f4b88d0 2024-04-02T13:47:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.769+0000 7f7611963740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c68f4187e0 con 0x55c68f4b91f0 2024-04-02T13:47:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f7607fff640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f75f80a3d20 0x7f75f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f760d318640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f7605ffb640 1 -- 172.21.15.33:0/706755875 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f76000f33d0 con 0x7f76081069c0 2024-04-02T13:47:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f760d318640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43864/0 (socket says 172.21.15.33:43864) 2024-04-02T13:47:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f760d318640 1 -- 172.21.15.33:0/1424418080 learned_addr learned my addr 172.21.15.33:0/1424418080 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f760cb17640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b88d0 0x55c68f4b8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f7607fff640 1 --2- 172.21.15.33:0/1424418080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b91f0 0x55c68f4c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f760d318640 1 -- 172.21.15.33:0/1424418080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b91f0 msgr2=0x55c68f4c1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f760d318640 1 --2- 172.21.15.33:0/1424418080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b91f0 0x55c68f4c1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f760d318640 1 -- 172.21.15.33:0/1424418080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b88d0 msgr2=0x55c68f4b8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f760d318640 1 --2- 172.21.15.33:0/1424418080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b88d0 0x55c68f4b8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f760d318640 1 -- 172.21.15.33:0/1424418080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c68f1fa7d0 con 0x55c68f4c1dd0 2024-04-02T13:47:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f760cb17640 1 --2- 172.21.15.33:0/1424418080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b88d0 0x55c68f4b8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f7607fff640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f75f80a3d20 0x7f75f80a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f76081050e0 tx=0x7f75f4013880 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f760d318640 1 --2- 172.21.15.33:0/1424418080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4c41c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f76000bf040 tx=0x7f76000099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a709e4d05c78e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f75f2ffd640 1 -- 172.21.15.33:0/1424418080 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76000bebf0 con 0x55c68f4c1dd0 2024-04-02T13:47:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f75f2ffd640 1 -- 172.21.15.33:0/1424418080 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f760003a040 con 0x55c68f4c1dd0 2024-04-02T13:47:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.770+0000 7f75f2ffd640 1 -- 172.21.15.33:0/1424418080 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76000047b0 con 0x55c68f4c1dd0 2024-04-02T13:47:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.771+0000 7f7611963740 1 -- 172.21.15.33:0/1424418080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 msgr2=0x55c68f4c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.771+0000 7f7611963740 1 --2- 172.21.15.33:0/1424418080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4c41c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f76000bf040 tx=0x7f76000099c0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.771+0000 7f7611963740 1 -- 172.21.15.33:0/1424418080 shutdown_connections 2024-04-02T13:47:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.771+0000 7f7611963740 1 --2- 172.21.15.33:0/1424418080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4c41c0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.771+0000 7f7611963740 1 --2- 172.21.15.33:0/1424418080 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b91f0 0x55c68f4c1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.771+0000 7f7611963740 1 --2- 172.21.15.33:0/1424418080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b88d0 0x55c68f4b8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.771+0000 7f7611963740 1 -- 172.21.15.33:0/1424418080 >> 172.21.15.33:0/1424418080 conn(0x55c68f4b3000 msgr2=0x55c68f4b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.771+0000 7f7611963740 1 -- 172.21.15.33:0/1424418080 shutdown_connections 2024-04-02T13:47:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.771+0000 7f7611963740 1 -- 172.21.15.33:0/1424418080 wait complete. 2024-04-02T13:47:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.771+0000 7f7611963740 1 -- start start 2024-04-02T13:47:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f7611963740 1 Processor -- start 2024-04-02T13:47:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f7611963740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b88d0 0x55c68f4c1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f7611963740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b91f0 0x55c68f4be890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f7611963740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4bedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f7611963740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c68f4c4980 con 0x55c68f4c1dd0 2024-04-02T13:47:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f7611963740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c68f3d7d30 con 0x55c68f4b88d0 2024-04-02T13:47:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f7611963740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c68f4c4b00 con 0x55c68f4b91f0 2024-04-02T13:47:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f7611963740 10 client.0 _ll_register_callbacks cb 0x55c68f3d7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f760d318640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4bedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f760d318640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4bedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43874/0 (socket says 172.21.15.33:43874) 2024-04-02T13:47:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f760d318640 1 -- 172.21.15.33:0/3651486940 learned_addr learned my addr 172.21.15.33:0/3651486940 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:30.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f760cb17640 1 --2- 172.21.15.33:0/3651486940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b88d0 0x55c68f4c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f760d318640 1 -- 172.21.15.33:0/3651486940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b91f0 msgr2=0x55c68f4be890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f760d318640 1 --2- 172.21.15.33:0/3651486940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b91f0 0x55c68f4be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f760d318640 1 -- 172.21.15.33:0/3651486940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b88d0 msgr2=0x55c68f4c1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f760d318640 1 --2- 172.21.15.33:0/3651486940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b88d0 0x55c68f4c1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f760d318640 1 -- 172.21.15.33:0/3651486940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c68f1303f0 con 0x55c68f4c1dd0 2024-04-02T13:47:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.772+0000 7f760cb17640 1 --2- 172.21.15.33:0/3651486940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b88d0 0x55c68f4c1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f760d318640 1 --2- 172.21.15.33:0/3651486940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4bedd0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f7600004160 tx=0x7f76000c3480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba51961b9ba5097 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f75e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f75e7fff640 1 -- 172.21.15.33:0/3651486940 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600096e90 con 0x55c68f4c1dd0 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f75e7fff640 1 -- 172.21.15.33:0/3651486940 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f76000c30e0 con 0x55c68f4c1dd0 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f7611963740 1 -- 172.21.15.33:0/3651486940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f760003a040 con 0x55c68f4c1dd0 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f75e7fff640 1 -- 172.21.15.33:0/3651486940 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f76000bea10 con 0x55c68f4c1dd0 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f7611963740 10 client.17451 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f7611963740 1 -- 172.21.15.33:0/3651486940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c68f0f00f0 con 0x55c68f4c1dd0 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f75e7fff640 1 -- 172.21.15.33:0/3651486940 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f760003a040 con 0x55c68f4c1dd0 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f75e7fff640 1 -- 172.21.15.33:0/3651486940 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f7600003c60 con 0x55c68f4c1dd0 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f7611963740 10 client.17451 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f7611963740 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f7611963740 20 client.17451 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f7611963740 1 --2- 172.21.15.33:0/3651486940 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c68f4d1790 0x55c68f4d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f7611963740 4 client.17451 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.773+0000 7f7611963740 1 -- 172.21.15.33:0/3651486940 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c68f429d70 con 0x55c68f4d1790 2024-04-02T13:47:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.774+0000 7f760cb17640 1 --2- 172.21.15.33:0/3651486940 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c68f4d1790 0x55c68f4d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.774+0000 7f760cb17640 1 --2- 172.21.15.33:0/3651486940 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c68f4d1790 0x55c68f4d3b80 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43bf6a040895fd3f server_cookie=55d254fd704ba868 in_seq=0 out_seq=0 2024-04-02T13:47:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.774+0000 7f75e7fff640 10 client.17451 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f75e7fff640 1 -- 172.21.15.33:0/3651486940 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c68f429d70 con 0x55c68f4d1790 2024-04-02T13:47:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f75e7fff640 10 client.17451 handle_command_reply: tid=0 2024-04-02T13:47:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f7611963740 1 client.17451 shutdown 2024-04-02T13:47:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f7611963740 1 -- 172.21.15.33:0/3651486940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 msgr2=0x55c68f4bedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f7611963740 1 --2- 172.21.15.33:0/3651486940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4bedd0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f7600004160 tx=0x7f76000c3480 comp rx=0 tx=0).stop 2024-04-02T13:47:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f7611963740 1 -- 172.21.15.33:0/3651486940 shutdown_connections 2024-04-02T13:47:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f7611963740 1 --2- 172.21.15.33:0/3651486940 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c68f4d1790 0x55c68f4d3b80 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f7611963740 1 --2- 172.21.15.33:0/3651486940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4bedd0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f7611963740 1 --2- 172.21.15.33:0/3651486940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b91f0 0x55c68f4be890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f75e7fff640 0 client.17451 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f7611963740 1 --2- 172.21.15.33:0/3651486940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b88d0 0x55c68f4c1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f7611963740 1 -- 172.21.15.33:0/3651486940 >> 172.21.15.33:0/3651486940 conn(0x55c68f4b3000 msgr2=0x55c68f4b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f7611963740 1 -- 172.21.15.33:0/3651486940 shutdown_connections 2024-04-02T13:47:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f7611963740 1 -- 172.21.15.33:0/3651486940 wait complete. 2024-04-02T13:47:30.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.775+0000 7f7611963740 20 client.17451 trim_cache size 0 max 16384 2024-04-02T13:47:30.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f7611963740 1 Processor -- start 2024-04-02T13:47:30.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f7611963740 1 -- start start 2024-04-02T13:47:30.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f7611963740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b88d0 0x55c68f4b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f7611963740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b91f0 0x55c68f4d9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f7611963740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4d9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f7611963740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76081b18a0 con 0x55c68f4c1dd0 2024-04-02T13:47:30.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f7611963740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f76081af170 con 0x55c68f4b91f0 2024-04-02T13:47:30.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f7611963740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f76081acfe0 con 0x55c68f4b88d0 2024-04-02T13:47:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f760d318640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4d9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f760d318640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4d9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43880/0 (socket says 172.21.15.33:43880) 2024-04-02T13:47:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f760d318640 1 -- 172.21.15.33:0/1551325218 learned_addr learned my addr 172.21.15.33:0/1551325218 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f760cb17640 1 --2- 172.21.15.33:0/1551325218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b88d0 0x55c68f4b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f760d318640 1 -- 172.21.15.33:0/1551325218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b88d0 msgr2=0x55c68f4b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f760d318640 1 --2- 172.21.15.33:0/1551325218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b88d0 0x55c68f4b4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f760d318640 1 -- 172.21.15.33:0/1551325218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b91f0 msgr2=0x55c68f4d9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f7607fff640 1 --2- 172.21.15.33:0/1551325218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b91f0 0x55c68f4d9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f760d318640 1 --2- 172.21.15.33:0/1551325218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b91f0 0x55c68f4d9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.793+0000 7f760d318640 1 -- 172.21.15.33:0/1551325218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c68f4d3ef0 con 0x55c68f4c1dd0 2024-04-02T13:47:30.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.794+0000 7f760d318640 1 --2- 172.21.15.33:0/1551325218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4d9950 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f76000bb4c0 tx=0x7f76000bd5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d224933ff5efb88b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.794+0000 7f75f27fc640 1 -- 172.21.15.33:0/1551325218 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76000bebf0 con 0x55c68f4c1dd0 2024-04-02T13:47:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.794+0000 7f75f27fc640 1 -- 172.21.15.33:0/1551325218 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f76000bf7b0 con 0x55c68f4c1dd0 2024-04-02T13:47:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.794+0000 7f75f27fc640 1 -- 172.21.15.33:0/1551325218 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600003c60 con 0x55c68f4c1dd0 2024-04-02T13:47:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.794+0000 7f7611963740 1 -- 172.21.15.33:0/1551325218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 msgr2=0x55c68f4d9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.794+0000 7f7611963740 1 --2- 172.21.15.33:0/1551325218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4d9950 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f76000bb4c0 tx=0x7f76000bd5c0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.794+0000 7f7611963740 1 -- 172.21.15.33:0/1551325218 shutdown_connections 2024-04-02T13:47:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.794+0000 7f7611963740 1 --2- 172.21.15.33:0/1551325218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4c1dd0 0x55c68f4d9950 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.794+0000 7f7611963740 1 --2- 172.21.15.33:0/1551325218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b91f0 0x55c68f4d9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.794+0000 7f7611963740 1 --2- 172.21.15.33:0/1551325218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4b88d0 0x55c68f4b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.794+0000 7f7611963740 1 -- 172.21.15.33:0/1551325218 >> 172.21.15.33:0/1551325218 conn(0x55c68f4b3000 msgr2=0x55c68f4b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.794+0000 7f7611963740 1 -- 172.21.15.33:0/1551325218 shutdown_connections 2024-04-02T13:47:30.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.794+0000 7f7611963740 1 -- 172.21.15.33:0/1551325218 wait complete. 2024-04-02T13:47:30.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.795+0000 7f7611963740 1 -- start start 2024-04-02T13:47:30.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.795+0000 7f7611963740 1 Processor -- start 2024-04-02T13:47:30.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.795+0000 7f7611963740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4b88d0 0x55c68f4ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.795+0000 7f7611963740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b91f0 0x55c68f4c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.795+0000 7f7611963740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4c1dd0 0x55c68f4c3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.795+0000 7f7611963740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76081ad630 con 0x55c68f4b88d0 2024-04-02T13:47:30.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.795+0000 7f7611963740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f76081aeaa0 con 0x55c68f4b91f0 2024-04-02T13:47:30.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.795+0000 7f7611963740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f76081add00 con 0x55c68f4c1dd0 2024-04-02T13:47:30.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.795+0000 7f7611963740 10 client.0 _ll_register_callbacks cb 0x55c68f3d7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:30.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.795+0000 7f760cb17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4b88d0 0x55c68f4ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.795+0000 7f760cb17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4b88d0 0x55c68f4ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43894/0 (socket says 172.21.15.33:43894) 2024-04-02T13:47:30.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.795+0000 7f760cb17640 1 -- 172.21.15.33:0/3527323988 learned_addr learned my addr 172.21.15.33:0/3527323988 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:30.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.795+0000 7f760d318640 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4c1dd0 0x55c68f4c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f7607fff640 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b91f0 0x55c68f4c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f760cb17640 1 -- 172.21.15.33:0/3527323988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4c1dd0 msgr2=0x55c68f4c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:30.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f760cb17640 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4c1dd0 0x55c68f4c3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f760cb17640 1 -- 172.21.15.33:0/3527323988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b91f0 msgr2=0x55c68f4c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:30.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f760cb17640 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b91f0 0x55c68f4c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f760cb17640 1 -- 172.21.15.33:0/3527323988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c68f4c7ce0 con 0x55c68f4b88d0 2024-04-02T13:47:30.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f760d318640 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4c1dd0 0x55c68f4c3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:30.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f7607fff640 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b91f0 0x55c68f4c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:30.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f760cb17640 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4b88d0 0x55c68f4ca1d0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00b600 tx=0x7f75fc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cd70a51c07a3a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:30.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f75e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:30.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f75e7fff640 1 -- 172.21.15.33:0/3527323988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc01f070 con 0x55c68f4b88d0 2024-04-02T13:47:30.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f75e7fff640 1 -- 172.21.15.33:0/3527323988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f75fc01f210 con 0x55c68f4b88d0 2024-04-02T13:47:30.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f7611963740 1 -- 172.21.15.33:0/3527323988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76000bf7b0 con 0x55c68f4b88d0 2024-04-02T13:47:30.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f75e7fff640 1 -- 172.21.15.33:0/3527323988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f75fc030050 con 0x55c68f4b88d0 2024-04-02T13:47:30.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f7611963740 10 client.17457 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:30.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f7611963740 1 -- 172.21.15.33:0/3527323988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c68f537ef0 con 0x55c68f4b88d0 2024-04-02T13:47:30.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f75e7fff640 1 -- 172.21.15.33:0/3527323988 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f76000bf7b0 con 0x55c68f4b88d0 2024-04-02T13:47:30.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f75e7fff640 1 -- 172.21.15.33:0/3527323988 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f75fc011ab0 con 0x55c68f4b88d0 2024-04-02T13:47:30.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f7611963740 10 client.17457 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:30.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f7611963740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:30.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.796+0000 7f7611963740 20 client.17457 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:30.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.797+0000 7f7611963740 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c68f538640 0x55c68f53aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.797+0000 7f7611963740 4 client.17457 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:47:30.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.797+0000 7f7611963740 1 -- 172.21.15.33:0/3527323988 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c68f4240c0 con 0x55c68f538640 2024-04-02T13:47:30.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.797+0000 7f7607fff640 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c68f538640 0x55c68f53aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.797+0000 7f7607fff640 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c68f538640 0x55c68f53aa60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71c05c36d0cc89d3 server_cookie=23e31cec1773c854 in_seq=0 out_seq=0 2024-04-02T13:47:30.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.797+0000 7f75e7fff640 10 client.17457 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:30.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.797+0000 7f75e7fff640 1 -- 172.21.15.33:0/3527323988 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c68f4240c0 con 0x55c68f538640 2024-04-02T13:47:30.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.797+0000 7f75e7fff640 10 client.17457 handle_command_reply: tid=0 2024-04-02T13:47:30.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.797+0000 7f7611963740 1 client.17457 shutdown 2024-04-02T13:47:30.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f7611963740 1 -- 172.21.15.33:0/3527323988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4b88d0 msgr2=0x55c68f4ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:30.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f7611963740 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4b88d0 0x55c68f4ca1d0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00b600 tx=0x7f75fc00be60 comp rx=0 tx=0).stop 2024-04-02T13:47:30.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f7611963740 1 -- 172.21.15.33:0/3527323988 shutdown_connections 2024-04-02T13:47:30.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f7611963740 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55c68f538640 0x55c68f53aa60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f7611963740 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c68f4c1dd0 0x55c68f4c3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f7611963740 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c68f4b91f0 0x55c68f4c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f75e7fff640 0 client.17457 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:30.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f7611963740 1 --2- 172.21.15.33:0/3527323988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c68f4b88d0 0x55c68f4ca1d0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f7611963740 1 -- 172.21.15.33:0/3527323988 >> 172.21.15.33:0/3527323988 conn(0x55c68f4b3000 msgr2=0x55c68f4b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:30.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f7611963740 1 -- 172.21.15.33:0/3527323988 shutdown_connections 2024-04-02T13:47:30.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f7611963740 1 -- 172.21.15.33:0/3527323988 wait complete. 2024-04-02T13:47:30.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f7611963740 20 client.17457 trim_cache size 0 max 16384 2024-04-02T13:47:30.811 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:47:30.811 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:47:30.811 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:47:30.811 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:47:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f760ed7b640 1 -- 172.21.15.33:0/706755875 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f75f80a3d20 msgr2=0x7f75f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f760ed7b640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f75f80a3d20 0x7f75f80a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f76081050e0 tx=0x7f75f4013880 comp rx=0 tx=0).stop 2024-04-02T13:47:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f760ed7b640 1 -- 172.21.15.33:0/706755875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76081069c0 msgr2=0x7f760819cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f760ed7b640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76081069c0 0x7f760819cc20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7600005cc0 tx=0x7f7600034b40 comp rx=0 tx=0).stop 2024-04-02T13:47:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f760ed7b640 1 -- 172.21.15.33:0/706755875 shutdown_connections 2024-04-02T13:47:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f760ed7b640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f75f80a3d20 0x7f75f80a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f760ed7b640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76081069c0 0x7f760819cc20 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f760ed7b640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7608101370 0x7f76081988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f760ed7b640 1 --2- 172.21.15.33:0/706755875 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f76081009b0 0x7f7608198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.798+0000 7f760ed7b640 1 -- 172.21.15.33:0/706755875 >> 172.21.15.33:0/706755875 conn(0x7f7608076d80 msgr2=0x7f760806a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.799+0000 7f760ed7b640 1 -- 172.21.15.33:0/706755875 shutdown_connections 2024-04-02T13:47:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.799+0000 7f760ed7b640 1 -- 172.21.15.33:0/706755875 wait complete. 2024-04-02T13:47:30.814 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:47:30.814 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:47:30.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.915+0000 7f737939f640 1 -- 172.21.15.33:0/4173945854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7374100660 msgr2=0x7f7374100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:30.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.915+0000 7f737939f640 1 --2- 172.21.15.33:0/4173945854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7374100660 0x7f7374100a40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f7364009900 tx=0x7f736402bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:30.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.915+0000 7f737939f640 1 -- 172.21.15.33:0/4173945854 shutdown_connections 2024-04-02T13:47:30.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.915+0000 7f737939f640 1 --2- 172.21.15.33:0/4173945854 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f737410e210 0x7f7374110600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.915+0000 7f737939f640 1 --2- 172.21.15.33:0/4173945854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7374101010 0x7f737410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.915+0000 7f737939f640 1 --2- 172.21.15.33:0/4173945854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7374100660 0x7f7374100a40 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.915+0000 7f737939f640 1 -- 172.21.15.33:0/4173945854 >> 172.21.15.33:0/4173945854 conn(0x7f73740fc2a0 msgr2=0x7f73740fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:30.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.915+0000 7f737939f640 1 -- 172.21.15.33:0/4173945854 shutdown_connections 2024-04-02T13:47:30.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.915+0000 7f737939f640 1 -- 172.21.15.33:0/4173945854 wait complete. 2024-04-02T13:47:30.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.915+0000 7f737939f640 1 Processor -- start 2024-04-02T13:47:30.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.915+0000 7f737939f640 1 -- start start 2024-04-02T13:47:30.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f737939f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7374100660 0x7f737419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f737939f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7374101010 0x7f737419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f737939f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f737410e210 0x7f737419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f737939f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7374113ee0 con 0x7f737410e210 2024-04-02T13:47:30.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f737939f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7374113d60 con 0x7f7374100660 2024-04-02T13:47:30.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f737939f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7374114060 con 0x7f7374101010 2024-04-02T13:47:30.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f7378b9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f737410e210 0x7f737419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f7378b9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f737410e210 0x7f737419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43912/0 (socket says 172.21.15.33:43912) 2024-04-02T13:47:30.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f7378b9e640 1 -- 172.21.15.33:0/1747776445 learned_addr learned my addr 172.21.15.33:0/1747776445 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:30.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f73737fe640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7374101010 0x7f737419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f7373fff640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7374100660 0x7f737419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f7378b9e640 1 -- 172.21.15.33:0/1747776445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7374101010 msgr2=0x7f737419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:30.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f7378b9e640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7374101010 0x7f737419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f7378b9e640 1 -- 172.21.15.33:0/1747776445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7374100660 msgr2=0x7f737419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:30.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f7378b9e640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7374100660 0x7f737419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:30.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f7378b9e640 1 -- 172.21.15.33:0/1747776445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737419f4b0 con 0x7f737410e210 2024-04-02T13:47:30.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.916+0000 7f7378b9e640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f737410e210 0x7f737419ed80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f736c00cab0 tx=0x7f736c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:30.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.917+0000 7f73717fa640 1 -- 172.21.15.33:0/1747776445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c00ae20 con 0x7f737410e210 2024-04-02T13:47:30.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.917+0000 7f737939f640 1 -- 172.21.15.33:0/1747776445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737419f7a0 con 0x7f737410e210 2024-04-02T13:47:30.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.917+0000 7f73717fa640 1 -- 172.21.15.33:0/1747776445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f736c00f920 con 0x7f737410e210 2024-04-02T13:47:30.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.917+0000 7f73717fa640 1 -- 172.21.15.33:0/1747776445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c018ad0 con 0x7f737410e210 2024-04-02T13:47:30.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.917+0000 7f737939f640 1 -- 172.21.15.33:0/1747776445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73741a7030 con 0x7f737410e210 2024-04-02T13:47:30.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.918+0000 7f737939f640 1 -- 172.21.15.33:0/1747776445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7374068c60 con 0x7f737410e210 2024-04-02T13:47:30.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.919+0000 7f73717fa640 1 -- 172.21.15.33:0/1747776445 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f736c00fad0 con 0x7f737410e210 2024-04-02T13:47:30.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.919+0000 7f73717fa640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f73600a3d20 0x7f73600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:30.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.920+0000 7f7373fff640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f73600a3d20 0x7f73600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:30.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.920+0000 7f73717fa640 1 -- 172.21.15.33:0/1747776445 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f736c0d11b0 con 0x7f737410e210 2024-04-02T13:47:30.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.920+0000 7f7373fff640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f73600a3d20 0x7f73600a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7364005ab0 tx=0x7f7364005a20 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:30.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:30.924+0000 7f73717fa640 1 -- 172.21.15.33:0/1747776445 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f736c014030 con 0x7f737410e210 2024-04-02T13:47:30.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:30 smithi145 ceph-mon[35646]: pgmap v270: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:47:30.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:30 smithi050 ceph-mon[35524]: pgmap v270: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:47:31.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:30 smithi033 ceph-mon[32390]: pgmap v270: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:47:31.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.150+0000 7f737939f640 1 -- 172.21.15.33:0/1747776445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73741093a0 con 0x7f737410e210 2024-04-02T13:47:31.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.151+0000 7f73717fa640 1 -- 172.21.15.33:0/1747776445 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11873 (secure 0 0 0) 0x7f736c09d420 con 0x7f737410e210 2024-04-02T13:47:31.154 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:47:31.154 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:44:55.702357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:47:31.154 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 36 2024-04-02T13:47:31.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.153+0000 7f737939f640 1 -- 172.21.15.33:0/1747776445 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f73600a3d20 msgr2=0x7f73600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.153+0000 7f737939f640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f73600a3d20 0x7f73600a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7364005ab0 tx=0x7f7364005a20 comp rx=0 tx=0).stop 2024-04-02T13:47:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.153+0000 7f737939f640 1 -- 172.21.15.33:0/1747776445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f737410e210 msgr2=0x7f737419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.153+0000 7f737939f640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f737410e210 0x7f737419ed80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f736c00cab0 tx=0x7f736c00cf70 comp rx=0 tx=0).stop 2024-04-02T13:47:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.153+0000 7f737939f640 1 -- 172.21.15.33:0/1747776445 shutdown_connections 2024-04-02T13:47:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.153+0000 7f737939f640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f73600a3d20 0x7f73600a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.153+0000 7f737939f640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f737410e210 0x7f737419ed80 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.153+0000 7f737939f640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7374101010 0x7f737419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.153+0000 7f737939f640 1 --2- 172.21.15.33:0/1747776445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7374100660 0x7f737419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.153+0000 7f737939f640 1 -- 172.21.15.33:0/1747776445 >> 172.21.15.33:0/1747776445 conn(0x7f73740fc2a0 msgr2=0x7f73740fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.153+0000 7f737939f640 1 -- 172.21.15.33:0/1747776445 shutdown_connections 2024-04-02T13:47:31.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.153+0000 7f737939f640 1 -- 172.21.15.33:0/1747776445 wait complete. 2024-04-02T13:47:31.166 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:47:31.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.269+0000 7f6983406640 1 -- 172.21.15.33:0/4138026633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c1002e0 msgr2=0x7f697c1006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.269+0000 7f6983406640 1 --2- 172.21.15.33:0/4138026633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c1002e0 0x7f697c1006c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6978009900 tx=0x7f697802bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:31.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.269+0000 7f6983406640 1 -- 172.21.15.33:0/4138026633 shutdown_connections 2024-04-02T13:47:31.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.269+0000 7f6983406640 1 --2- 172.21.15.33:0/4138026633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f697c106ef0 0x7f697c10ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.269+0000 7f6983406640 1 --2- 172.21.15.33:0/4138026633 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f697c100c90 0x7f697c1067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.269+0000 7f6983406640 1 --2- 172.21.15.33:0/4138026633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c1002e0 0x7f697c1006c0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.269+0000 7f6983406640 1 -- 172.21.15.33:0/4138026633 >> 172.21.15.33:0/4138026633 conn(0x7f697c077030 msgr2=0x7f697c0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.269+0000 7f6983406640 1 -- 172.21.15.33:0/4138026633 shutdown_connections 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.269+0000 7f6983406640 1 -- 172.21.15.33:0/4138026633 wait complete. 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.270+0000 7f6983406640 1 Processor -- start 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.270+0000 7f6983406640 1 -- start start 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.270+0000 7f6983406640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f697c1002e0 0x7f697c101250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.270+0000 7f6983406640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f697c106ef0 0x7f697c101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.270+0000 7f6983406640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101cd0 0x7f697c1a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.270+0000 7f6983406640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f697c111f40 con 0x7f697c101cd0 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.270+0000 7f6983406640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f697c111dc0 con 0x7f697c1002e0 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.270+0000 7f6983406640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1120c0 con 0x7f697c106ef0 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.270+0000 7f69819a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101cd0 0x7f697c1a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.270+0000 7f69819a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101cd0 0x7f697c1a6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43928/0 (socket says 172.21.15.33:43928) 2024-04-02T13:47:31.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.270+0000 7f69819a3640 1 -- 172.21.15.33:0/2699756806 learned_addr learned my addr 172.21.15.33:0/2699756806 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.270+0000 7f69811a2640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f697c1002e0 0x7f697c101250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f69819a3640 1 -- 172.21.15.33:0/2699756806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f697c106ef0 msgr2=0x7f697c101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f69809a1640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f697c106ef0 0x7f697c101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f69819a3640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f697c106ef0 0x7f697c101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f69819a3640 1 -- 172.21.15.33:0/2699756806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f697c1002e0 msgr2=0x7f697c101250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f69819a3640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f697c1002e0 0x7f697c101250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f69819a3640 1 -- 172.21.15.33:0/2699756806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c1a6d70 con 0x7f697c101cd0 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f69811a2640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f697c1002e0 0x7f697c101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f69809a1640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f697c106ef0 0x7f697c101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f69819a3640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101cd0 0x7f697c1a6670 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6974009fd0 tx=0x7f697400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f69727fc640 1 -- 172.21.15.33:0/2699756806 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974017070 con 0x7f697c101cd0 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f6983406640 1 -- 172.21.15.33:0/2699756806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c1a7060 con 0x7f697c101cd0 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f69727fc640 1 -- 172.21.15.33:0/2699756806 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6974003e40 con 0x7f697c101cd0 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f69727fc640 1 -- 172.21.15.33:0/2699756806 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740106c0 con 0x7f697c101cd0 2024-04-02T13:47:31.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.271+0000 7f6983406640 1 -- 172.21.15.33:0/2699756806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c1a7560 con 0x7f697c101cd0 2024-04-02T13:47:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.273+0000 7f69727fc640 1 -- 172.21.15.33:0/2699756806 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6974003970 con 0x7f697c101cd0 2024-04-02T13:47:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.273+0000 7f6985fed740 1 Processor -- start 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f6985fed740 1 -- start start 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f6985fed740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f2277a0 0x55645f227b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f6985fed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f230760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f6985fed740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f230ca0 0x55645f233090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f6985fed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55645f18b7b0 con 0x55645f2280c0 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f6985fed740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55645f12a570 con 0x55645f2277a0 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f6985fed740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55645f187760 con 0x55645f230ca0 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69811a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f230760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69811a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f230760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43936/0 (socket says 172.21.15.33:43936) 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69811a2640 1 -- 172.21.15.33:0/1806617009 learned_addr learned my addr 172.21.15.33:0/1806617009 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69727fc640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f69680a3d50 0x7f69680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69809a1640 1 --2- 172.21.15.33:0/1806617009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f2277a0 0x55645f227b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69809a1640 1 --2- 172.21.15.33:0/1806617009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f230ca0 0x55645f233090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69811a2640 1 -- 172.21.15.33:0/1806617009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f230ca0 msgr2=0x55645f233090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69811a2640 1 --2- 172.21.15.33:0/1806617009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f230ca0 0x55645f233090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69819a3640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f69680a3d50 0x7f69680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69811a2640 1 -- 172.21.15.33:0/1806617009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f2277a0 msgr2=0x55645f227b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69811a2640 1 --2- 172.21.15.33:0/1806617009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f2277a0 0x55645f227b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69811a2640 1 -- 172.21.15.33:0/1806617009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55645ee9f3f0 con 0x55645f2280c0 2024-04-02T13:47:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69809a1640 1 --2- 172.21.15.33:0/1806617009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f230ca0 0x55645f233090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69727fc640 1 -- 172.21.15.33:0/2699756806 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f69740d0990 con 0x7f697c101cd0 2024-04-02T13:47:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.274+0000 7f69811a2640 1 --2- 172.21.15.33:0/1806617009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f230760 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f6978009900 tx=0x7f69780040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992e96789e79d2a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.275+0000 7f69677fe640 1 -- 172.21.15.33:0/1806617009 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69780044b0 con 0x55645f2280c0 2024-04-02T13:47:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.275+0000 7f69819a3640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f69680a3d50 0x7f69680a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6974098f30 tx=0x7f697409f040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.275+0000 7f69677fe640 1 -- 172.21.15.33:0/1806617009 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6978005500 con 0x55645f2280c0 2024-04-02T13:47:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.275+0000 7f69677fe640 1 -- 172.21.15.33:0/1806617009 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978004920 con 0x55645f2280c0 2024-04-02T13:47:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.275+0000 7f6985fed740 1 -- 172.21.15.33:0/1806617009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 msgr2=0x55645f230760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.275+0000 7f6985fed740 1 --2- 172.21.15.33:0/1806617009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f230760 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f6978009900 tx=0x7f69780040c0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.275+0000 7f6985fed740 1 -- 172.21.15.33:0/1806617009 shutdown_connections 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.275+0000 7f6985fed740 1 --2- 172.21.15.33:0/1806617009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f230ca0 0x55645f233090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.275+0000 7f6985fed740 1 --2- 172.21.15.33:0/1806617009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f230760 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.275+0000 7f6985fed740 1 --2- 172.21.15.33:0/1806617009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f2277a0 0x55645f227b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.275+0000 7f6985fed740 1 -- 172.21.15.33:0/1806617009 >> 172.21.15.33:0/1806617009 conn(0x55645f221ed0 msgr2=0x55645f2222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.275+0000 7f6985fed740 1 -- 172.21.15.33:0/1806617009 shutdown_connections 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.275+0000 7f6985fed740 1 -- 172.21.15.33:0/1806617009 wait complete. 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f6985fed740 1 -- start start 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f6985fed740 1 Processor -- start 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f6985fed740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f2277a0 0x55645f22db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f6985fed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f22e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f6985fed740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f230ca0 0x55645f22e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f6985fed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55645f233a30 con 0x55645f2280c0 2024-04-02T13:47:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f6985fed740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55645f146e80 con 0x55645f230ca0 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f6985fed740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55645f233bb0 con 0x55645f2277a0 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f6985fed740 10 client.0 _ll_register_callbacks cb 0x55645f182aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f69811a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f22e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f69811a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f22e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43948/0 (socket says 172.21.15.33:43948) 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f69811a2640 1 -- 172.21.15.33:0/2614336719 learned_addr learned my addr 172.21.15.33:0/2614336719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f69809a1640 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f230ca0 0x55645f22e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.276+0000 7f69809a1640 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f2277a0 0x55645f22db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f69811a2640 1 -- 172.21.15.33:0/2614336719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f2277a0 msgr2=0x55645f22db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f69811a2640 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f2277a0 0x55645f22db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f69811a2640 1 -- 172.21.15.33:0/2614336719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f230ca0 msgr2=0x55645f22e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f69811a2640 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f230ca0 0x55645f22e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f69811a2640 1 -- 172.21.15.33:0/2614336719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55645ef697d0 con 0x55645f2280c0 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f69809a1640 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f2277a0 0x55645f22db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f69811a2640 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f22e040 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6978005ab0 tx=0x7f697802ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f00fb3076d2cef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f6964ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f6964ff9640 1 -- 172.21.15.33:0/2614336719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978003a80 con 0x55645f2280c0 2024-04-02T13:47:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f6964ff9640 1 -- 172.21.15.33:0/2614336719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f6978003c20 con 0x55645f2280c0 2024-04-02T13:47:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f6985fed740 1 -- 172.21.15.33:0/2614336719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6978005500 con 0x55645f2280c0 2024-04-02T13:47:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f6964ff9640 1 -- 172.21.15.33:0/2614336719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6978003e20 con 0x55645f2280c0 2024-04-02T13:47:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f6985fed740 10 client.17475 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f6985fed740 1 -- 172.21.15.33:0/2614336719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55645f23fa70 con 0x55645f2280c0 2024-04-02T13:47:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f6964ff9640 1 -- 172.21.15.33:0/2614336719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f6978005500 con 0x55645f2280c0 2024-04-02T13:47:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.277+0000 7f6964ff9640 1 -- 172.21.15.33:0/2614336719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f6978050660 con 0x55645f2280c0 2024-04-02T13:47:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.278+0000 7f6985fed740 10 client.17475 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.278+0000 7f6985fed740 10 client.17475 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:47:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.278+0000 7f6985fed740 20 client.17475 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.278+0000 7f6985fed740 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55645f241040 0x55645f243460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.278+0000 7f6985fed740 4 client.17475 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.278+0000 7f6985fed740 1 -- 172.21.15.33:0/2614336719 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55645f198cf0 con 0x55645f241040 2024-04-02T13:47:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.278+0000 7f69809a1640 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55645f241040 0x55645f243460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.278+0000 7f6964ff9640 10 client.17475 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:47:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.278+0000 7f69809a1640 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55645f241040 0x55645f243460 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=223a4c41b9e035d7 server_cookie=258b80ea3d4bab73 in_seq=0 out_seq=0 2024-04-02T13:47:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6964ff9640 1 -- 172.21.15.33:0/2614336719 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55645f198cf0 con 0x55645f241040 2024-04-02T13:47:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6964ff9640 10 client.17475 handle_command_reply: tid=0 2024-04-02T13:47:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6985fed740 1 client.17475 shutdown 2024-04-02T13:47:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6985fed740 1 -- 172.21.15.33:0/2614336719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 msgr2=0x55645f22e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6985fed740 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f22e040 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6978005ab0 tx=0x7f697802ca90 comp rx=0 tx=0).stop 2024-04-02T13:47:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6985fed740 1 -- 172.21.15.33:0/2614336719 shutdown_connections 2024-04-02T13:47:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6985fed740 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55645f241040 0x55645f243460 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6985fed740 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f230ca0 0x55645f22e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6985fed740 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f22e040 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6964ff9640 0 client.17475 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:47:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6985fed740 1 --2- 172.21.15.33:0/2614336719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f2277a0 0x55645f22db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6985fed740 1 -- 172.21.15.33:0/2614336719 >> 172.21.15.33:0/2614336719 conn(0x55645f221ed0 msgr2=0x55645f223c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6985fed740 1 -- 172.21.15.33:0/2614336719 shutdown_connections 2024-04-02T13:47:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6985fed740 1 -- 172.21.15.33:0/2614336719 wait complete. 2024-04-02T13:47:31.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.279+0000 7f6985fed740 20 client.17475 trim_cache size 0 max 16384 2024-04-02T13:47:31.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f6985fed740 1 Processor -- start 2024-04-02T13:47:31.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f6985fed740 1 -- start start 2024-04-02T13:47:31.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f6985fed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2277a0 0x55645f23c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f6985fed740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f2280c0 0x55645f22cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f6985fed740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f230ca0 0x55645f22e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f6985fed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1b3a90 con 0x55645f2277a0 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f6985fed740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1b1360 con 0x55645f2280c0 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f6985fed740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1af1d0 con 0x55645f230ca0 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f69809a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2277a0 0x55645f23c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f69809a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2277a0 0x55645f23c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43952/0 (socket says 172.21.15.33:43952) 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f69809a1640 1 -- 172.21.15.33:0/3333369673 learned_addr learned my addr 172.21.15.33:0/3333369673 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f69811a2640 1 --2- 172.21.15.33:0/3333369673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f2280c0 0x55645f22cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f69809a1640 1 -- 172.21.15.33:0/3333369673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f230ca0 msgr2=0x55645f22e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f69809a1640 1 --2- 172.21.15.33:0/3333369673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f230ca0 0x55645f22e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f69809a1640 1 -- 172.21.15.33:0/3333369673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f2280c0 msgr2=0x55645f22cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f69809a1640 1 --2- 172.21.15.33:0/3333369673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f2280c0 0x55645f22cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f69809a1640 1 -- 172.21.15.33:0/3333369673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55645f22c050 con 0x55645f2277a0 2024-04-02T13:47:31.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.293+0000 7f69809a1640 1 --2- 172.21.15.33:0/3333369673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2277a0 0x55645f23c8f0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f696c013560 tx=0x7f696c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b989cad924a4bae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6966ffd640 1 -- 172.21.15.33:0/3333369673 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c022070 con 0x55645f2277a0 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6966ffd640 1 -- 172.21.15.33:0/3333369673 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f696c01d7d0 con 0x55645f2277a0 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6966ffd640 1 -- 172.21.15.33:0/3333369673 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c012450 con 0x55645f2277a0 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6985fed740 1 -- 172.21.15.33:0/3333369673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2277a0 msgr2=0x55645f23c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6985fed740 1 --2- 172.21.15.33:0/3333369673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2277a0 0x55645f23c8f0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f696c013560 tx=0x7f696c005b50 comp rx=0 tx=0).stop 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6985fed740 1 -- 172.21.15.33:0/3333369673 shutdown_connections 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6985fed740 1 --2- 172.21.15.33:0/3333369673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f230ca0 0x55645f22e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6985fed740 1 --2- 172.21.15.33:0/3333369673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f2280c0 0x55645f22cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6985fed740 1 --2- 172.21.15.33:0/3333369673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2277a0 0x55645f23c8f0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6985fed740 1 -- 172.21.15.33:0/3333369673 >> 172.21.15.33:0/3333369673 conn(0x55645f221ed0 msgr2=0x55645f223280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6985fed740 1 -- 172.21.15.33:0/3333369673 shutdown_connections 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6985fed740 1 -- 172.21.15.33:0/3333369673 wait complete. 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6985fed740 1 -- start start 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.294+0000 7f6985fed740 1 Processor -- start 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f6985fed740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f2277a0 0x55645f233dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f6985fed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f234300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f6985fed740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f230ca0 0x55645f234840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f6985fed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1af820 con 0x55645f2280c0 2024-04-02T13:47:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f6985fed740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1b0c90 con 0x55645f230ca0 2024-04-02T13:47:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f6985fed740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1afef0 con 0x55645f2277a0 2024-04-02T13:47:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f6985fed740 10 client.0 _ll_register_callbacks cb 0x7f6978005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f69811a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f234300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f69811a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f234300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43964/0 (socket says 172.21.15.33:43964) 2024-04-02T13:47:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f69811a2640 1 -- 172.21.15.33:0/3211565212 learned_addr learned my addr 172.21.15.33:0/3211565212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f69809a1640 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f230ca0 0x55645f234840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f69809a1640 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f2277a0 0x55645f233dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f69811a2640 1 -- 172.21.15.33:0/3211565212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f2277a0 msgr2=0x55645f233dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f69811a2640 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f2277a0 0x55645f233dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f69811a2640 1 -- 172.21.15.33:0/3211565212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f230ca0 msgr2=0x55645f234840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f69811a2640 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f230ca0 0x55645f234840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f69811a2640 1 -- 172.21.15.33:0/3211565212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55645f2a5750 con 0x55645f2280c0 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f69809a1640 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f2277a0 0x55645f233dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f69811a2640 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f234300 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x55645f22f0d0 tx=0x7f69780041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af168f65283f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.295+0000 7f6964ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6964ff9640 1 -- 172.21.15.33:0/3211565212 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978050430 con 0x55645f2280c0 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6964ff9640 1 -- 172.21.15.33:0/3211565212 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f69780505d0 con 0x55645f2280c0 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6985fed740 1 -- 172.21.15.33:0/3211565212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f696c01d7d0 con 0x55645f2280c0 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6964ff9640 1 -- 172.21.15.33:0/3211565212 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f69780507d0 con 0x55645f2280c0 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6985fed740 10 client.17481 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6985fed740 1 -- 172.21.15.33:0/3211565212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55645f2a6a60 con 0x55645f2280c0 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6964ff9640 1 -- 172.21.15.33:0/3211565212 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f696c01d7d0 con 0x55645f2280c0 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6964ff9640 1 -- 172.21.15.33:0/3211565212 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f6978004b10 con 0x55645f2280c0 2024-04-02T13:47:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6985fed740 10 client.17481 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6985fed740 10 client.17481 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6985fed740 20 client.17481 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6985fed740 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55645f2a7000 0x55645f2a9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6985fed740 4 client.17481 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f6985fed740 1 -- 172.21.15.33:0/3211565212 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55645f193040 con 0x55645f2a7000 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.296+0000 7f69809a1640 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55645f2a7000 0x55645f2a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f69809a1640 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55645f2a7000 0x55645f2a9420 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=faa1dd6cc71c9bbb server_cookie=f239ac0187b44788 in_seq=0 out_seq=0 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6964ff9640 10 client.17481 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6964ff9640 1 -- 172.21.15.33:0/3211565212 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55645f193040 con 0x55645f2a7000 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6964ff9640 10 client.17481 handle_command_reply: tid=0 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6985fed740 1 client.17481 shutdown 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6985fed740 1 -- 172.21.15.33:0/3211565212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 msgr2=0x55645f234300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6985fed740 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f234300 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x55645f22f0d0 tx=0x7f69780041f0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6985fed740 1 -- 172.21.15.33:0/3211565212 shutdown_connections 2024-04-02T13:47:31.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6985fed740 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x55645f2a7000 0x55645f2a9420 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6985fed740 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55645f230ca0 0x55645f234840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6985fed740 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55645f2280c0 0x55645f234300 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6985fed740 1 --2- 172.21.15.33:0/3211565212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55645f2277a0 0x55645f233dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6964ff9640 0 client.17481 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:47:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6985fed740 1 -- 172.21.15.33:0/3211565212 >> 172.21.15.33:0/3211565212 conn(0x55645f221ed0 msgr2=0x55645f22cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6985fed740 1 -- 172.21.15.33:0/3211565212 shutdown_connections 2024-04-02T13:47:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6985fed740 1 -- 172.21.15.33:0/3211565212 wait complete. 2024-04-02T13:47:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.297+0000 7f6985fed740 20 client.17481 trim_cache size 0 max 16384 2024-04-02T13:47:31.313 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:47:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.298+0000 7f6983406640 1 -- 172.21.15.33:0/2699756806 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f69680a3d50 msgr2=0x7f69680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.298+0000 7f6983406640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f69680a3d50 0x7f69680a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6974098f30 tx=0x7f697409f040 comp rx=0 tx=0).stop 2024-04-02T13:47:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.298+0000 7f6983406640 1 -- 172.21.15.33:0/2699756806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101cd0 msgr2=0x7f697c1a6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.298+0000 7f6983406640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101cd0 0x7f697c1a6670 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6974009fd0 tx=0x7f697400b260 comp rx=0 tx=0).stop 2024-04-02T13:47:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.298+0000 7f6983406640 1 -- 172.21.15.33:0/2699756806 shutdown_connections 2024-04-02T13:47:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.298+0000 7f6983406640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f69680a3d50 0x7f69680a6210 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.298+0000 7f6983406640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c101cd0 0x7f697c1a6670 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.298+0000 7f6983406640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f697c106ef0 0x7f697c101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.298+0000 7f6983406640 1 --2- 172.21.15.33:0/2699756806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f697c1002e0 0x7f697c101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.298+0000 7f6983406640 1 -- 172.21.15.33:0/2699756806 >> 172.21.15.33:0/2699756806 conn(0x7f697c077030 msgr2=0x7f697c0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.298+0000 7f6983406640 1 -- 172.21.15.33:0/2699756806 shutdown_connections 2024-04-02T13:47:31.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.298+0000 7f6983406640 1 -- 172.21.15.33:0/2699756806 wait complete. 2024-04-02T13:47:31.316 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:47:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.417+0000 7fa349047640 1 -- 172.21.15.33:0/4184698102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34410c140 msgr2=0x7fa34410e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.417+0000 7fa349047640 1 --2- 172.21.15.33:0/4184698102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34410c140 0x7fa34410e5b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009ca0 tx=0x7fa33c02c180 comp rx=0 tx=0).stop 2024-04-02T13:47:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.417+0000 7fa349047640 1 -- 172.21.15.33:0/4184698102 shutdown_connections 2024-04-02T13:47:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.417+0000 7fa349047640 1 --2- 172.21.15.33:0/4184698102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34410c140 0x7fa34410e5b0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.417+0000 7fa349047640 1 --2- 172.21.15.33:0/4184698102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa344103690 0x7fa34410bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.417+0000 7fa349047640 1 --2- 172.21.15.33:0/4184698102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa344102d70 0x7fa344103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.417+0000 7fa349047640 1 -- 172.21.15.33:0/4184698102 >> 172.21.15.33:0/4184698102 conn(0x7fa344077b60 msgr2=0x7fa344077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.417+0000 7fa349047640 1 -- 172.21.15.33:0/4184698102 shutdown_connections 2024-04-02T13:47:31.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.417+0000 7fa349047640 1 -- 172.21.15.33:0/4184698102 wait complete. 2024-04-02T13:47:31.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.418+0000 7fa349047640 1 Processor -- start 2024-04-02T13:47:31.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.418+0000 7fa349047640 1 -- start start 2024-04-02T13:47:31.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.418+0000 7fa349047640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa344102d70 0x7fa3441983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.418+0000 7fa349047640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa344103690 0x7fa3441988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.418+0000 7fa349047640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34410c140 0x7fa34419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.418+0000 7fa349047640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa344111ee0 con 0x7fa34410c140 2024-04-02T13:47:31.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.418+0000 7fa349047640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa344111d60 con 0x7fa344102d70 2024-04-02T13:47:31.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.418+0000 7fa349047640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa344112060 con 0x7fa344103690 2024-04-02T13:47:31.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.418+0000 7fa34359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34410c140 0x7fa34419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.418+0000 7fa34359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34410c140 0x7fa34419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43988/0 (socket says 172.21.15.33:43988) 2024-04-02T13:47:31.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.418+0000 7fa34359e640 1 -- 172.21.15.33:0/1349119522 learned_addr learned my addr 172.21.15.33:0/1349119522 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.419+0000 7fa33bfff640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa344103690 0x7fa3441988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.419+0000 7fa342d9d640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa344102d70 0x7fa3441983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.419+0000 7fa34359e640 1 -- 172.21.15.33:0/1349119522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa344103690 msgr2=0x7fa3441988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.419+0000 7fa34359e640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa344103690 0x7fa3441988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.419+0000 7fa34359e640 1 -- 172.21.15.33:0/1349119522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa344102d70 msgr2=0x7fa3441983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.419+0000 7fa34359e640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa344102d70 0x7fa3441983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.419+0000 7fa34359e640 1 -- 172.21.15.33:0/1349119522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34419d340 con 0x7fa34410c140 2024-04-02T13:47:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.419+0000 7fa34359e640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34410c140 0x7fa34419cc10 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa33c004af0 tx=0x7fa33c034ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.419+0000 7fa340d99640 1 -- 172.21.15.33:0/1349119522 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c039050 con 0x7fa34410c140 2024-04-02T13:47:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.419+0000 7fa349047640 1 -- 172.21.15.33:0/1349119522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34419d5d0 con 0x7fa34410c140 2024-04-02T13:47:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.419+0000 7fa340d99640 1 -- 172.21.15.33:0/1349119522 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa33c045070 con 0x7fa34410c140 2024-04-02T13:47:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.419+0000 7fa340d99640 1 -- 172.21.15.33:0/1349119522 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c032b30 con 0x7fa34410c140 2024-04-02T13:47:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.419+0000 7fa349047640 1 -- 172.21.15.33:0/1349119522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3441a4ec0 con 0x7fa34410c140 2024-04-02T13:47:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.421+0000 7fa340d99640 1 -- 172.21.15.33:0/1349119522 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa33c0046f0 con 0x7fa34410c140 2024-04-02T13:47:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa34bc2d740 1 Processor -- start 2024-04-02T13:47:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa34bc2d740 1 -- start start 2024-04-02T13:47:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa34bc2d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df260d7a0 0x564df260db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa34bc2d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df260e0c0 0x564df2616760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa34bc2d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ca0 0x564df2619090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa34bc2d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564df25717b0 con 0x564df2616ca0 2024-04-02T13:47:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa34bc2d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564df2510570 con 0x564df260d7a0 2024-04-02T13:47:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa34bc2d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564df256d760 con 0x564df260e0c0 2024-04-02T13:47:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ca0 0x564df2619090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa340d99640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3300a3d20 0x7fa3300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ca0 0x564df2619090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44000/0 (socket says 172.21.15.33:44000) 2024-04-02T13:47:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa33bfff640 1 -- 172.21.15.33:0/311562244 learned_addr learned my addr 172.21.15.33:0/311562244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa342d9d640 1 --2- 172.21.15.33:0/311562244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df260e0c0 0x564df2616760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa33bfff640 1 --2- 172.21.15.33:0/311562244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df260d7a0 0x564df260db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa34359e640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3300a3d20 0x7fa3300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa33bfff640 1 -- 172.21.15.33:0/311562244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df260e0c0 msgr2=0x564df2616760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa33bfff640 1 --2- 172.21.15.33:0/311562244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df260e0c0 0x564df2616760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa33bfff640 1 -- 172.21.15.33:0/311562244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df260d7a0 msgr2=0x564df260db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.422+0000 7fa33bfff640 1 --2- 172.21.15.33:0/311562244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df260d7a0 0x564df260db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.423+0000 7fa33bfff640 1 -- 172.21.15.33:0/311562244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564df22853f0 con 0x564df2616ca0 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.423+0000 7fa340d99640 1 -- 172.21.15.33:0/1349119522 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fa33c0f35e0 con 0x7fa34410c140 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.423+0000 7fa33bfff640 1 --2- 172.21.15.33:0/311562244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ca0 0x564df2619090 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00cf90 tx=0x7fa32c00c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f2cef12216e6eb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.423+0000 7fa34359e640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3300a3d20 0x7fa3300a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009430 tx=0x7fa33c08dc10 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.423+0000 7fa3397fa640 1 -- 172.21.15.33:0/311562244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c019030 con 0x564df2616ca0 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.423+0000 7fa3397fa640 1 -- 172.21.15.33:0/311562244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa32c0119f0 con 0x564df2616ca0 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.423+0000 7fa3397fa640 1 -- 172.21.15.33:0/311562244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c00f980 con 0x564df2616ca0 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 -- 172.21.15.33:0/311562244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ca0 msgr2=0x564df2619090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/311562244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ca0 0x564df2619090 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00cf90 tx=0x7fa32c00c630 comp rx=0 tx=0).stop 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 -- 172.21.15.33:0/311562244 shutdown_connections 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/311562244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df2616ca0 0x564df2619090 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/311562244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df260e0c0 0x564df2616760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/311562244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df260d7a0 0x564df260db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 -- 172.21.15.33:0/311562244 >> 172.21.15.33:0/311562244 conn(0x564df2607ed0 msgr2=0x564df26082b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 -- 172.21.15.33:0/311562244 shutdown_connections 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 -- 172.21.15.33:0/311562244 wait complete. 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 -- start start 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 Processor -- start 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df260d7a0 0x564df2616750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df2613ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ca0 0x564df2614220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564df2619a90 con 0x564df260e0c0 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564df252ce80 con 0x564df2616ca0 2024-04-02T13:47:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x564df2619c10 con 0x564df260d7a0 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.424+0000 7fa34bc2d740 10 client.0 _ll_register_callbacks cb 0x564df2568aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa342d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df2613ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa342d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df2613ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44006/0 (socket says 172.21.15.33:44006) 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa342d9d640 1 -- 172.21.15.33:0/2588521682 learned_addr learned my addr 172.21.15.33:0/2588521682 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa33bfff640 1 --2- 172.21.15.33:0/2588521682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ca0 0x564df2614220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa342d9d640 1 -- 172.21.15.33:0/2588521682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df260d7a0 msgr2=0x564df2616750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa342d9d640 1 --2- 172.21.15.33:0/2588521682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df260d7a0 0x564df2616750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa342d9d640 1 -- 172.21.15.33:0/2588521682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ca0 msgr2=0x564df2614220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa342d9d640 1 --2- 172.21.15.33:0/2588521682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ca0 0x564df2614220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa342d9d640 1 -- 172.21.15.33:0/2588521682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564df234f7d0 con 0x564df260e0c0 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa33bfff640 1 --2- 172.21.15.33:0/2588521682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ca0 0x564df2614220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa342d9d640 1 --2- 172.21.15.33:0/2588521682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df2613ae0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa33400bf60 tx=0x7fa33400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f2c9adc962c0ed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa32a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa32a7fc640 1 -- 172.21.15.33:0/2588521682 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334004230 con 0x564df260e0c0 2024-04-02T13:47:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa32a7fc640 1 -- 172.21.15.33:0/2588521682 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fa3340043d0 con 0x564df260e0c0 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa34bc2d740 1 -- 172.21.15.33:0/2588521682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa32c0119f0 con 0x564df260e0c0 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa32a7fc640 1 -- 172.21.15.33:0/2588521682 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa3340106a0 con 0x564df260e0c0 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa34bc2d740 10 client.17493 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa34bc2d740 1 -- 172.21.15.33:0/2588521682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564df2625bd0 con 0x564df260e0c0 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.425+0000 7fa32a7fc640 1 -- 172.21.15.33:0/2588521682 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa32c0119f0 con 0x564df260e0c0 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.426+0000 7fa32a7fc640 1 -- 172.21.15.33:0/2588521682 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fa334019070 con 0x564df260e0c0 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.426+0000 7fa34bc2d740 10 client.17493 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.426+0000 7fa34bc2d740 10 client.17493 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.426+0000 7fa34bc2d740 20 client.17493 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.426+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/2588521682 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564df2627130 0x564df26295b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.426+0000 7fa34bc2d740 4 client.17493 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.426+0000 7fa34bc2d740 1 -- 172.21.15.33:0/2588521682 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564df257ecf0 con 0x564df2627130 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.426+0000 7fa33bfff640 1 --2- 172.21.15.33:0/2588521682 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564df2627130 0x564df26295b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.426+0000 7fa32a7fc640 10 client.17493 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:47:31.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.426+0000 7fa33bfff640 1 --2- 172.21.15.33:0/2588521682 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564df2627130 0x564df26295b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10ce5161ef48ff38 server_cookie=55cb4087cec79dea in_seq=0 out_seq=0 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa32a7fc640 1 -- 172.21.15.33:0/2588521682 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564df257ecf0 con 0x564df2627130 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa32a7fc640 10 client.17493 handle_command_reply: tid=0 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa34bc2d740 1 client.17493 shutdown 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa34bc2d740 1 -- 172.21.15.33:0/2588521682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 msgr2=0x564df2613ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/2588521682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df2613ae0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa33400bf60 tx=0x7fa33400bf90 comp rx=0 tx=0).stop 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa34bc2d740 1 -- 172.21.15.33:0/2588521682 shutdown_connections 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/2588521682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ca0 0x564df2614220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/2588521682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df2613ae0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/2588521682 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564df2627130 0x564df26295b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/2588521682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df260d7a0 0x564df2616750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa34bc2d740 1 -- 172.21.15.33:0/2588521682 >> 172.21.15.33:0/2588521682 conn(0x564df2607ed0 msgr2=0x564df2609c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa32a7fc640 0 client.17493 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa34bc2d740 1 -- 172.21.15.33:0/2588521682 shutdown_connections 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa34bc2d740 1 -- 172.21.15.33:0/2588521682 wait complete. 2024-04-02T13:47:31.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.427+0000 7fa34bc2d740 20 client.17493 trim_cache size 0 max 16384 2024-04-02T13:47:31.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.441+0000 7fa34bc2d740 1 Processor -- start 2024-04-02T13:47:31.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.441+0000 7fa34bc2d740 1 -- start start 2024-04-02T13:47:31.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.441+0000 7fa34bc2d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df260d7a0 0x564df2622980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.441+0000 7fa34bc2d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df26232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.441+0000 7fa34bc2d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ca0 0x564df2612ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.441+0000 7fa34bc2d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441b1880 con 0x564df260e0c0 2024-04-02T13:47:31.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.441+0000 7fa34bc2d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441af150 con 0x564df2616ca0 2024-04-02T13:47:31.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.441+0000 7fa34bc2d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441acfc0 con 0x564df260d7a0 2024-04-02T13:47:31.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.441+0000 7fa342d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df26232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.441+0000 7fa342d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df26232c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44012/0 (socket says 172.21.15.33:44012) 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.441+0000 7fa342d9d640 1 -- 172.21.15.33:0/2678346091 learned_addr learned my addr 172.21.15.33:0/2678346091 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.441+0000 7fa33bfff640 1 --2- 172.21.15.33:0/2678346091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ca0 0x564df2612ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.442+0000 7fa342d9d640 1 -- 172.21.15.33:0/2678346091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df260d7a0 msgr2=0x564df2622980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.442+0000 7fa342d9d640 1 --2- 172.21.15.33:0/2678346091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df260d7a0 0x564df2622980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.442+0000 7fa342d9d640 1 -- 172.21.15.33:0/2678346091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ca0 msgr2=0x564df2612ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.442+0000 7fa342d9d640 1 --2- 172.21.15.33:0/2678346091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ca0 0x564df2612ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.442+0000 7fa342d9d640 1 -- 172.21.15.33:0/2678346091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564df2614590 con 0x564df260e0c0 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.442+0000 7fa33bfff640 1 --2- 172.21.15.33:0/2678346091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ca0 0x564df2612ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.442+0000 7fa342d9d640 1 --2- 172.21.15.33:0/2678346091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df26232c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa334004630 tx=0x7fa334005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4269171afb831d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.442+0000 7fa338ff9640 1 -- 172.21.15.33:0/2678346091 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33402d020 con 0x564df260e0c0 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.442+0000 7fa338ff9640 1 -- 172.21.15.33:0/2678346091 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa334003ac0 con 0x564df260e0c0 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.442+0000 7fa338ff9640 1 -- 172.21.15.33:0/2678346091 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33401b4c0 con 0x564df260e0c0 2024-04-02T13:47:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.442+0000 7fa34bc2d740 1 -- 172.21.15.33:0/2678346091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 msgr2=0x564df26232c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.442+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/2678346091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df26232c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa334004630 tx=0x7fa334005710 comp rx=0 tx=0).stop 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 -- 172.21.15.33:0/2678346091 shutdown_connections 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/2678346091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df2616ca0 0x564df2612ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/2678346091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df26232c0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/2678346091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df260d7a0 0x564df2622980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 -- 172.21.15.33:0/2678346091 >> 172.21.15.33:0/2678346091 conn(0x564df2607ed0 msgr2=0x564df26123b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 -- 172.21.15.33:0/2678346091 shutdown_connections 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 -- 172.21.15.33:0/2678346091 wait complete. 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 -- start start 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 Processor -- start 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df260d7a0 0x564df2610180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df26106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ca0 0x564df261de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441ad610 con 0x564df260e0c0 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441aea80 con 0x564df260d7a0 2024-04-02T13:47:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa3441adce0 con 0x564df2616ca0 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa34bc2d740 10 client.0 _ll_register_callbacks cb 0x7fa33400b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa342d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df26106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa342d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df26106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44020/0 (socket says 172.21.15.33:44020) 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa342d9d640 1 -- 172.21.15.33:0/3759439696 learned_addr learned my addr 172.21.15.33:0/3759439696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.443+0000 7fa33bfff640 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df260d7a0 0x564df2610180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa33bfff640 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ca0 0x564df261de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa342d9d640 1 -- 172.21.15.33:0/3759439696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ca0 msgr2=0x564df261de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa342d9d640 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ca0 0x564df261de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa342d9d640 1 -- 172.21.15.33:0/3759439696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df260d7a0 msgr2=0x564df2610180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa342d9d640 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df260d7a0 0x564df2610180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa342d9d640 1 -- 172.21.15.33:0/3759439696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564df261e3d0 con 0x564df260e0c0 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa33bfff640 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ca0 0x564df261de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa342d9d640 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df26106c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa334005fa0 tx=0x7fa334004240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed27d2eeee770b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa32a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa32a7fc640 1 -- 172.21.15.33:0/3759439696 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33402d030 con 0x564df260e0c0 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa32a7fc640 1 -- 172.21.15.33:0/3759439696 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fa334010a50 con 0x564df260e0c0 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa34bc2d740 1 -- 172.21.15.33:0/3759439696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa334003ac0 con 0x564df260e0c0 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa32a7fc640 1 -- 172.21.15.33:0/3759439696 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa334043050 con 0x564df260e0c0 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa34bc2d740 10 client.17499 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa34bc2d740 1 -- 172.21.15.33:0/3759439696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564df261f6d0 con 0x564df260e0c0 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa32a7fc640 1 -- 172.21.15.33:0/3759439696 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa334003ac0 con 0x564df260e0c0 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa32a7fc640 1 -- 172.21.15.33:0/3759439696 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fa334019040 con 0x564df260e0c0 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa34bc2d740 10 client.17499 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa34bc2d740 10 client.17499 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa34bc2d740 20 client.17499 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564df2691d10 0x564df2694130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa34bc2d740 4 client.17499 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.444+0000 7fa34bc2d740 1 -- 172.21.15.33:0/3759439696 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564df2579040 con 0x564df2691d10 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.445+0000 7fa33bfff640 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564df2691d10 0x564df2694130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.445+0000 7fa33bfff640 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564df2691d10 0x564df2694130 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ad853203515ffb server_cookie=ec1ff9cb9e151baa in_seq=0 out_seq=0 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.445+0000 7fa32a7fc640 10 client.17499 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.445+0000 7fa32a7fc640 1 -- 172.21.15.33:0/3759439696 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564df2579040 con 0x564df2691d10 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.445+0000 7fa32a7fc640 10 client.17499 handle_command_reply: tid=0 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.445+0000 7fa34bc2d740 1 client.17499 shutdown 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa34bc2d740 1 -- 172.21.15.33:0/3759439696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 msgr2=0x564df26106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df26106c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa334005fa0 tx=0x7fa334004240 comp rx=0 tx=0).stop 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa34bc2d740 1 -- 172.21.15.33:0/3759439696 shutdown_connections 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x564df2691d10 0x564df2694130 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564df2616ca0 0x564df261de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa32a7fc640 0 client.17499 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564df260e0c0 0x564df26106c0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa34bc2d740 1 --2- 172.21.15.33:0/3759439696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564df260d7a0 0x564df2610180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa34bc2d740 1 -- 172.21.15.33:0/3759439696 >> 172.21.15.33:0/3759439696 conn(0x564df2607ed0 msgr2=0x564df2606f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa34bc2d740 1 -- 172.21.15.33:0/3759439696 shutdown_connections 2024-04-02T13:47:31.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa34bc2d740 1 -- 172.21.15.33:0/3759439696 wait complete. 2024-04-02T13:47:31.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa34bc2d740 20 client.17499 trim_cache size 0 max 16384 2024-04-02T13:47:31.455 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:47:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa349047640 1 -- 172.21.15.33:0/1349119522 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3300a3d20 msgr2=0x7fa3300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa349047640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3300a3d20 0x7fa3300a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009430 tx=0x7fa33c08dc10 comp rx=0 tx=0).stop 2024-04-02T13:47:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa349047640 1 -- 172.21.15.33:0/1349119522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34410c140 msgr2=0x7fa34419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa349047640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34410c140 0x7fa34419cc10 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa33c004af0 tx=0x7fa33c034ed0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa349047640 1 -- 172.21.15.33:0/1349119522 shutdown_connections 2024-04-02T13:47:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa349047640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3300a3d20 0x7fa3300a61e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa349047640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa34410c140 0x7fa34419cc10 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa349047640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa344103690 0x7fa3441988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa349047640 1 --2- 172.21.15.33:0/1349119522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa344102d70 0x7fa3441983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa349047640 1 -- 172.21.15.33:0/1349119522 >> 172.21.15.33:0/1349119522 conn(0x7fa344077b60 msgr2=0x7fa3440fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa349047640 1 -- 172.21.15.33:0/1349119522 shutdown_connections 2024-04-02T13:47:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.446+0000 7fa349047640 1 -- 172.21.15.33:0/1349119522 wait complete. 2024-04-02T13:47:31.460 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:47:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 -- 172.21.15.33:0/267156374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbbc101330 msgr2=0x7fdbbc10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 --2- 172.21.15.33:0/267156374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbbc101330 0x7fdbbc10d840 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009cd0 tx=0x7fdbb402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 -- 172.21.15.33:0/267156374 shutdown_connections 2024-04-02T13:47:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 --2- 172.21.15.33:0/267156374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbbc101330 0x7fdbbc10d840 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 --2- 172.21.15.33:0/267156374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbbc100970 0x7fdbbc100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 --2- 172.21.15.33:0/267156374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbbc106980 0x7fdbbc106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 -- 172.21.15.33:0/267156374 >> 172.21.15.33:0/267156374 conn(0x7fdbbc077b00 msgr2=0x7fdbbc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 -- 172.21.15.33:0/267156374 shutdown_connections 2024-04-02T13:47:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 -- 172.21.15.33:0/267156374 wait complete. 2024-04-02T13:47:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 Processor -- start 2024-04-02T13:47:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 -- start start 2024-04-02T13:47:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbbc100970 0x7fdbbc1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbbc101330 0x7fdbbc1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbbc106980 0x7fdbbc19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc110d60 con 0x7fdbbc106980 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc110be0 con 0x7fdbbc100970 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.562+0000 7fdbc3242640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc110ee0 con 0x7fdbbc101330 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbc17df640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbbc106980 0x7fdbbc19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbc17df640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbbc106980 0x7fdbbc19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44040/0 (socket says 172.21.15.33:44040) 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbc17df640 1 -- 172.21.15.33:0/1162897086 learned_addr learned my addr 172.21.15.33:0/1162897086 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbc0fde640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbbc100970 0x7fdbbc1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbbc101330 0x7fdbbc1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbc17df640 1 -- 172.21.15.33:0/1162897086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbbc101330 msgr2=0x7fdbbc1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbc17df640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbbc101330 0x7fdbbc1976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbc17df640 1 -- 172.21.15.33:0/1162897086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbbc100970 msgr2=0x7fdbbc1971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbc17df640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbbc100970 0x7fdbbc1971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbc17df640 1 -- 172.21.15.33:0/1162897086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbbc19c190 con 0x7fdbbc106980 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbc17df640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbbc106980 0x7fdbbc19ba60 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4004af0 tx=0x7fdbb4034d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbb9ffb640 1 -- 172.21.15.33:0/1162897086 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4039050 con 0x7fdbbc106980 2024-04-02T13:47:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbb9ffb640 1 -- 172.21.15.33:0/1162897086 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdbb4045070 con 0x7fdbbc106980 2024-04-02T13:47:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbb9ffb640 1 -- 172.21.15.33:0/1162897086 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb40046b0 con 0x7fdbbc106980 2024-04-02T13:47:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.563+0000 7fdbc3242640 1 -- 172.21.15.33:0/1162897086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbbc19c420 con 0x7fdbbc106980 2024-04-02T13:47:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.564+0000 7fdbc3242640 1 -- 172.21.15.33:0/1162897086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbbc1a3d10 con 0x7fdbbc106980 2024-04-02T13:47:31.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbb9ffb640 1 -- 172.21.15.33:0/1162897086 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdbb4003970 con 0x7fdbbc106980 2024-04-02T13:47:31.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbc5e20740 1 Processor -- start 2024-04-02T13:47:31.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbc5e20740 1 -- start start 2024-04-02T13:47:31.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbc5e20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 0x55ef23f4ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbc5e20740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f4f2d0 0x55ef23f57970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbc5e20740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f57eb0 0x55ef23f5a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbc5e20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ef23eb07b0 con 0x55ef23f4e9b0 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbc5e20740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ef23e4f570 con 0x55ef23f4f2d0 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbc5e20740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ef23eac760 con 0x55ef23f57eb0 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbbbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 0x55ef23f4ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbb9ffb640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fdbac0a3d20 0x7fdbac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbbbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 0x55ef23f4ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44054/0 (socket says 172.21.15.33:44054) 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbbbfff640 1 -- 172.21.15.33:0/1845344835 learned_addr learned my addr 172.21.15.33:0/1845344835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbc0fde640 1 --2- 172.21.15.33:0/1845344835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f4f2d0 0x55ef23f57970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.566+0000 7fdbc17df640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fdbac0a3d20 0x7fdbac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.567+0000 7fdbbbfff640 1 -- 172.21.15.33:0/1845344835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f57eb0 msgr2=0x55ef23f5a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.567+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/1845344835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f57eb0 0x55ef23f5a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.567+0000 7fdbbbfff640 1 -- 172.21.15.33:0/1845344835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f4f2d0 msgr2=0x55ef23f57970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.567+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/1845344835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f4f2d0 0x55ef23f57970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.567+0000 7fdbbbfff640 1 -- 172.21.15.33:0/1845344835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef23bc43f0 con 0x55ef23f4e9b0 2024-04-02T13:47:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.567+0000 7fdbb9ffb640 1 -- 172.21.15.33:0/1162897086 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fdbb40f3490 con 0x7fdbbc106980 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.567+0000 7fdbc0fde640 1 --2- 172.21.15.33:0/1845344835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f4f2d0 0x55ef23f57970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.567+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/1845344835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 0x55ef23f4ed90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fdba800ccd0 tx=0x7fdba8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c345a1b913fd5af server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.567+0000 7fdbc17df640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fdbac0a3d20 0x7fdbac0a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40f0a70 tx=0x7fdbb4090040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.567+0000 7fdba6ffd640 1 -- 172.21.15.33:0/1845344835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8016070 con 0x55ef23f4e9b0 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.567+0000 7fdba6ffd640 1 -- 172.21.15.33:0/1845344835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdba8012b50 con 0x55ef23f4e9b0 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.567+0000 7fdba6ffd640 1 -- 172.21.15.33:0/1845344835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8011c10 con 0x55ef23f4e9b0 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.568+0000 7fdbc5e20740 1 -- 172.21.15.33:0/1845344835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 msgr2=0x55ef23f4ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.568+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/1845344835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 0x55ef23f4ed90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fdba800ccd0 tx=0x7fdba8010740 comp rx=0 tx=0).stop 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.568+0000 7fdbc5e20740 1 -- 172.21.15.33:0/1845344835 shutdown_connections 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.568+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/1845344835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f57eb0 0x55ef23f5a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.568+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/1845344835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f4f2d0 0x55ef23f57970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.568+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/1845344835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 0x55ef23f4ed90 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.568+0000 7fdbc5e20740 1 -- 172.21.15.33:0/1845344835 >> 172.21.15.33:0/1845344835 conn(0x55ef23f490e0 msgr2=0x55ef23f494c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.568+0000 7fdbc5e20740 1 -- 172.21.15.33:0/1845344835 shutdown_connections 2024-04-02T13:47:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.568+0000 7fdbc5e20740 1 -- 172.21.15.33:0/1845344835 wait complete. 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.568+0000 7fdbc5e20740 1 -- start start 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.568+0000 7fdbc5e20740 1 Processor -- start 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc5e20740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f4e9b0 0x55ef23f57970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc5e20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 0x55ef23f54d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc5e20740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f57eb0 0x55ef23f554b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc5e20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ef23f5aca0 con 0x55ef23f4f2d0 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc5e20740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ef23e6be80 con 0x55ef23f57eb0 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc5e20740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55ef23f5ae20 con 0x55ef23f4e9b0 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc5e20740 10 client.0 _ll_register_callbacks cb 0x55ef23ea7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc0fde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 0x55ef23f54d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc0fde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 0x55ef23f54d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44070/0 (socket says 172.21.15.33:44070) 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc0fde640 1 -- 172.21.15.33:0/691228127 learned_addr learned my addr 172.21.15.33:0/691228127 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f57eb0 0x55ef23f554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f4e9b0 0x55ef23f57970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc0fde640 1 -- 172.21.15.33:0/691228127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f4e9b0 msgr2=0x55ef23f57970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc0fde640 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f4e9b0 0x55ef23f57970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc0fde640 1 -- 172.21.15.33:0/691228127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f57eb0 msgr2=0x55ef23f554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc0fde640 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f57eb0 0x55ef23f554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc0fde640 1 -- 172.21.15.33:0/691228127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef23c8e7d0 con 0x55ef23f4f2d0 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f57eb0 0x55ef23f554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc0fde640 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 0x55ef23f54d70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000bf10 tx=0x7fdbb0012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f0bca5b4223fc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdb9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdb9bfff640 1 -- 172.21.15.33:0/691228127 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0010690 con 0x55ef23f4f2d0 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdb9bfff640 1 -- 172.21.15.33:0/691228127 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fdbb0010830 con 0x55ef23f4f2d0 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdbc5e20740 1 -- 172.21.15.33:0/691228127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdba8012b50 con 0x55ef23f4f2d0 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.569+0000 7fdb9bfff640 1 -- 172.21.15.33:0/691228127 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdbb0010a30 con 0x55ef23f4f2d0 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.570+0000 7fdbc5e20740 10 client.17511 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.570+0000 7fdbc5e20740 1 -- 172.21.15.33:0/691228127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ef23f66e50 con 0x55ef23f4f2d0 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.570+0000 7fdb9bfff640 1 -- 172.21.15.33:0/691228127 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdba8012b50 con 0x55ef23f4f2d0 2024-04-02T13:47:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.570+0000 7fdb9bfff640 1 -- 172.21.15.33:0/691228127 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fdbb001c3d0 con 0x55ef23f4f2d0 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.570+0000 7fdbc5e20740 10 client.17511 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.570+0000 7fdbc5e20740 10 client.17511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.570+0000 7fdbc5e20740 20 client.17511 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.570+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ef23f683b0 0x55ef23f6a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.570+0000 7fdbc5e20740 4 client.17511 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.570+0000 7fdbc5e20740 1 -- 172.21.15.33:0/691228127 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef23ebdcf0 con 0x55ef23f683b0 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.570+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ef23f683b0 0x55ef23f6a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdb9bfff640 10 client.17511 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ef23f683b0 0x55ef23f6a7d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0650c40f27e340a server_cookie=2b7b95bf4f5ab61c in_seq=0 out_seq=0 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdb9bfff640 1 -- 172.21.15.33:0/691228127 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef23ebdcf0 con 0x55ef23f683b0 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdb9bfff640 10 client.17511 handle_command_reply: tid=0 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdbc5e20740 1 client.17511 shutdown 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdbc5e20740 1 -- 172.21.15.33:0/691228127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 msgr2=0x55ef23f54d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 0x55ef23f54d70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000bf10 tx=0x7fdbb0012820 comp rx=0 tx=0).stop 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdbc5e20740 1 -- 172.21.15.33:0/691228127 shutdown_connections 2024-04-02T13:47:31.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ef23f683b0 0x55ef23f6a7d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f57eb0 0x55ef23f554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 0x55ef23f54d70 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdb9bfff640 0 client.17511 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:31.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/691228127 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f4e9b0 0x55ef23f57970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.571+0000 7fdbc5e20740 1 -- 172.21.15.33:0/691228127 >> 172.21.15.33:0/691228127 conn(0x55ef23f490e0 msgr2=0x55ef23f4ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.572+0000 7fdbc5e20740 1 -- 172.21.15.33:0/691228127 shutdown_connections 2024-04-02T13:47:31.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.572+0000 7fdbc5e20740 1 -- 172.21.15.33:0/691228127 wait complete. 2024-04-02T13:47:31.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.572+0000 7fdbc5e20740 20 client.17511 trim_cache size 0 max 16384 2024-04-02T13:47:31.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc5e20740 1 Processor -- start 2024-04-02T13:47:31.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc5e20740 1 -- start start 2024-04-02T13:47:31.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc5e20740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f4e9b0 0x55ef23f63b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc5e20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 0x55ef23f45db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc5e20740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f57eb0 0x55ef23f70080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc5e20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1af5e0 con 0x55ef23f4f2d0 2024-04-02T13:47:31.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc5e20740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1aceb0 con 0x55ef23f57eb0 2024-04-02T13:47:31.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc5e20740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1aad20 con 0x55ef23f4e9b0 2024-04-02T13:47:31.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc0fde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 0x55ef23f45db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc0fde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 0x55ef23f45db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44086/0 (socket says 172.21.15.33:44086) 2024-04-02T13:47:31.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc0fde640 1 -- 172.21.15.33:0/396504380 learned_addr learned my addr 172.21.15.33:0/396504380 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/396504380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f4e9b0 0x55ef23f63b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/396504380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f57eb0 0x55ef23f70080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc0fde640 1 -- 172.21.15.33:0/396504380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f4e9b0 msgr2=0x55ef23f63b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc0fde640 1 --2- 172.21.15.33:0/396504380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f4e9b0 0x55ef23f63b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc0fde640 1 -- 172.21.15.33:0/396504380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f57eb0 msgr2=0x55ef23f70080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc0fde640 1 --2- 172.21.15.33:0/396504380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f57eb0 0x55ef23f70080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbc0fde640 1 -- 172.21.15.33:0/396504380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef23f4f100 con 0x55ef23f4f2d0 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.585+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/396504380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f4e9b0 0x55ef23f63b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.586+0000 7fdbc0fde640 1 --2- 172.21.15.33:0/396504380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 0x55ef23f45db0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0001eb0 tx=0x7fdbb001ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801e74e91ea35550 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.586+0000 7fdba67fc640 1 -- 172.21.15.33:0/396504380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb002f020 con 0x55ef23f4f2d0 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.586+0000 7fdba67fc640 1 -- 172.21.15.33:0/396504380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdbb0005850 con 0x55ef23f4f2d0 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.586+0000 7fdba67fc640 1 -- 172.21.15.33:0/396504380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0000e60 con 0x55ef23f4f2d0 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.586+0000 7fdbc5e20740 1 -- 172.21.15.33:0/396504380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 msgr2=0x55ef23f45db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.586+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/396504380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 0x55ef23f45db0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0001eb0 tx=0x7fdbb001ce80 comp rx=0 tx=0).stop 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.586+0000 7fdbc5e20740 1 -- 172.21.15.33:0/396504380 shutdown_connections 2024-04-02T13:47:31.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.586+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/396504380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f57eb0 0x55ef23f70080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.586+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/396504380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4f2d0 0x55ef23f45db0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.586+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/396504380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f4e9b0 0x55ef23f63b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.586+0000 7fdbc5e20740 1 -- 172.21.15.33:0/396504380 >> 172.21.15.33:0/396504380 conn(0x55ef23f490e0 msgr2=0x55ef23f456a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.586+0000 7fdbc5e20740 1 -- 172.21.15.33:0/396504380 shutdown_connections 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.586+0000 7fdbc5e20740 1 -- 172.21.15.33:0/396504380 wait complete. 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbc5e20740 1 -- start start 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbc5e20740 1 Processor -- start 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbc5e20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 0x55ef23f58850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbc5e20740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f4f2d0 0x55ef23f58d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbc5e20740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f57eb0 0x55ef23f5f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbc5e20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1ab370 con 0x55ef23f4e9b0 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbc5e20740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1ac7e0 con 0x55ef23f4f2d0 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbc5e20740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1aba40 con 0x55ef23f57eb0 2024-04-02T13:47:31.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbc5e20740 10 client.0 _ll_register_callbacks cb 0x7fdbb000baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbbbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 0x55ef23f58850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbbbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 0x55ef23f58850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44094/0 (socket says 172.21.15.33:44094) 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbbbfff640 1 -- 172.21.15.33:0/479248302 learned_addr learned my addr 172.21.15.33:0/479248302 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbbbfff640 1 -- 172.21.15.33:0/479248302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f57eb0 msgr2=0x55ef23f5f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/479248302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f57eb0 0x55ef23f5f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbbbfff640 1 -- 172.21.15.33:0/479248302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f4f2d0 msgr2=0x55ef23f58d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/479248302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f4f2d0 0x55ef23f58d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.587+0000 7fdbbbfff640 1 -- 172.21.15.33:0/479248302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef23f5f5e0 con 0x55ef23f4e9b0 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdbbbfff640 1 --2- 172.21.15.33:0/479248302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 0x55ef23f58850 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fdba800cfd0 tx=0x7fdba800c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faaa76462fd01723 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdb9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdb9bfff640 1 -- 172.21.15.33:0/479248302 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8016040 con 0x55ef23f4e9b0 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdb9bfff640 1 -- 172.21.15.33:0/479248302 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fdba80052d0 con 0x55ef23f4e9b0 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdbc5e20740 1 -- 172.21.15.33:0/479248302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbb0005850 con 0x55ef23f4e9b0 2024-04-02T13:47:31.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdb9bfff640 1 -- 172.21.15.33:0/479248302 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdba8037050 con 0x55ef23f4e9b0 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdbc5e20740 10 client.17517 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdbc5e20740 1 -- 172.21.15.33:0/479248302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ef23f608e0 con 0x55ef23f4e9b0 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdb9bfff640 1 -- 172.21.15.33:0/479248302 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fdbb0005850 con 0x55ef23f4e9b0 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdb9bfff640 1 -- 172.21.15.33:0/479248302 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fdba8032030 con 0x55ef23f4e9b0 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdbc5e20740 10 client.17517 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdbc5e20740 10 client.17517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdbc5e20740 20 client.17517 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/479248302 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ef23fcead0 0x55ef23fd0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdbc5e20740 4 client.17517 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.588+0000 7fdbc5e20740 1 -- 172.21.15.33:0/479248302 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef23eb8040 con 0x55ef23fcead0 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdbc0fde640 1 --2- 172.21.15.33:0/479248302 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ef23fcead0 0x55ef23fd0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdbc0fde640 1 --2- 172.21.15.33:0/479248302 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ef23fcead0 0x55ef23fd0ef0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63e32b281eec739 server_cookie=917a05737e120ea7 in_seq=0 out_seq=0 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdb9bfff640 10 client.17517 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdb9bfff640 1 -- 172.21.15.33:0/479248302 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef23eb8040 con 0x55ef23fcead0 2024-04-02T13:47:31.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdb9bfff640 10 client.17517 handle_command_reply: tid=0 2024-04-02T13:47:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdbc5e20740 1 client.17517 shutdown 2024-04-02T13:47:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdbc5e20740 1 -- 172.21.15.33:0/479248302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 msgr2=0x55ef23f58850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/479248302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 0x55ef23f58850 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fdba800cfd0 tx=0x7fdba800c460 comp rx=0 tx=0).stop 2024-04-02T13:47:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdbc5e20740 1 -- 172.21.15.33:0/479248302 shutdown_connections 2024-04-02T13:47:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/479248302 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55ef23fcead0 0x55ef23fd0ef0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/479248302 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55ef23f57eb0 0x55ef23f5f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/479248302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef23f4f2d0 0x55ef23f58d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdb9bfff640 0 client.17517 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdbc5e20740 1 --2- 172.21.15.33:0/479248302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ef23f4e9b0 0x55ef23f58850 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdbc5e20740 1 -- 172.21.15.33:0/479248302 >> 172.21.15.33:0/479248302 conn(0x55ef23f490e0 msgr2=0x55ef23f45f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdbc5e20740 1 -- 172.21.15.33:0/479248302 shutdown_connections 2024-04-02T13:47:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.589+0000 7fdbc5e20740 1 -- 172.21.15.33:0/479248302 wait complete. 2024-04-02T13:47:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.590+0000 7fdbc5e20740 20 client.17517 trim_cache size 0 max 16384 2024-04-02T13:47:31.599 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:47:31.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.590+0000 7fdbc3242640 1 -- 172.21.15.33:0/1162897086 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fdbac0a3d20 msgr2=0x7fdbac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.590+0000 7fdbc3242640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fdbac0a3d20 0x7fdbac0a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40f0a70 tx=0x7fdbb4090040 comp rx=0 tx=0).stop 2024-04-02T13:47:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.590+0000 7fdbc3242640 1 -- 172.21.15.33:0/1162897086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbbc106980 msgr2=0x7fdbbc19ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.590+0000 7fdbc3242640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbbc106980 0x7fdbbc19ba60 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4004af0 tx=0x7fdbb4034d70 comp rx=0 tx=0).stop 2024-04-02T13:47:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.590+0000 7fdbc3242640 1 -- 172.21.15.33:0/1162897086 shutdown_connections 2024-04-02T13:47:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.590+0000 7fdbc3242640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fdbac0a3d20 0x7fdbac0a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.590+0000 7fdbc3242640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbbc106980 0x7fdbbc19ba60 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.590+0000 7fdbc3242640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdbbc101330 0x7fdbbc1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.590+0000 7fdbc3242640 1 --2- 172.21.15.33:0/1162897086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbbc100970 0x7fdbbc1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.590+0000 7fdbc3242640 1 -- 172.21.15.33:0/1162897086 >> 172.21.15.33:0/1162897086 conn(0x7fdbbc077b00 msgr2=0x7fdbbc103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.590+0000 7fdbc3242640 1 -- 172.21.15.33:0/1162897086 shutdown_connections 2024-04-02T13:47:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:31.590+0000 7fdbc3242640 1 -- 172.21.15.33:0/1162897086 wait complete. 2024-04-02T13:47:31.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:31 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1747776445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:47:31.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:31 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1747776445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:47:31.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:31 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1747776445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:47:32.604 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:47:32.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:47:32.604 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:47:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.723+0000 7fb3dd7c8640 1 -- 172.21.15.33:0/3127474340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d8103690 msgr2=0x7fb3d810bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.723+0000 7fb3dd7c8640 1 --2- 172.21.15.33:0/3127474340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d8103690 0x7fb3d810bb70 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009900 tx=0x7fb3c002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.723+0000 7fb3dd7c8640 1 -- 172.21.15.33:0/3127474340 shutdown_connections 2024-04-02T13:47:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.723+0000 7fb3dd7c8640 1 --2- 172.21.15.33:0/3127474340 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3d810c140 0x7fb3d810e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.723+0000 7fb3dd7c8640 1 --2- 172.21.15.33:0/3127474340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d8103690 0x7fb3d810bb70 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.723+0000 7fb3dd7c8640 1 --2- 172.21.15.33:0/3127474340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3d8102d70 0x7fb3d8103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.723+0000 7fb3dd7c8640 1 -- 172.21.15.33:0/3127474340 >> 172.21.15.33:0/3127474340 conn(0x7fb3d8077b60 msgr2=0x7fb3d8077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.723+0000 7fb3dd7c8640 1 -- 172.21.15.33:0/3127474340 shutdown_connections 2024-04-02T13:47:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.723+0000 7fb3dd7c8640 1 -- 172.21.15.33:0/3127474340 wait complete. 2024-04-02T13:47:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3dd7c8640 1 Processor -- start 2024-04-02T13:47:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3dd7c8640 1 -- start start 2024-04-02T13:47:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3dd7c8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3d8102d70 0x7fb3d81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3dd7c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d8103690 0x7fb3d81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3dd7c8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3d810c140 0x7fb3d819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3dd7c8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8111ee0 con 0x7fb3d8103690 2024-04-02T13:47:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3dd7c8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8111d60 con 0x7fb3d8102d70 2024-04-02T13:47:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3dd7c8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8112060 con 0x7fb3d810c140 2024-04-02T13:47:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d8103690 0x7fb3d81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d8103690 0x7fb3d81989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44106/0 (socket says 172.21.15.33:44106) 2024-04-02T13:47:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3cffff640 1 -- 172.21.15.33:0/2143873872 learned_addr learned my addr 172.21.15.33:0/2143873872 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3d6ffd640 1 --2- 172.21.15.33:0/2143873872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3d8102d70 0x7fb3d81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3cffff640 1 -- 172.21.15.33:0/2143873872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3d810c140 msgr2=0x7fb3d819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3cffff640 1 --2- 172.21.15.33:0/2143873872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3d810c140 0x7fb3d819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3cffff640 1 -- 172.21.15.33:0/2143873872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3d8102d70 msgr2=0x7fb3d81984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3cffff640 1 --2- 172.21.15.33:0/2143873872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3d8102d70 0x7fb3d81984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.724+0000 7fb3cffff640 1 -- 172.21.15.33:0/2143873872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d819d490 con 0x7fb3d8103690 2024-04-02T13:47:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.725+0000 7fb3cffff640 1 --2- 172.21.15.33:0/2143873872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d8103690 0x7fb3d81989e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009230 tx=0x7fb3c00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.725+0000 7fb3d4ff9640 1 -- 172.21.15.33:0/2143873872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0043020 con 0x7fb3d8103690 2024-04-02T13:47:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.725+0000 7fb3d4ff9640 1 -- 172.21.15.33:0/2143873872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb3c002cb40 con 0x7fb3d8103690 2024-04-02T13:47:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.725+0000 7fb3dd7c8640 1 -- 172.21.15.33:0/2143873872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d819d720 con 0x7fb3d8103690 2024-04-02T13:47:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.725+0000 7fb3d4ff9640 1 -- 172.21.15.33:0/2143873872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0003bc0 con 0x7fb3d8103690 2024-04-02T13:47:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.725+0000 7fb3dd7c8640 1 -- 172.21.15.33:0/2143873872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d81a5010 con 0x7fb3d8103690 2024-04-02T13:47:32.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.727+0000 7fb3d4ff9640 1 -- 172.21.15.33:0/2143873872 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb3c0046020 con 0x7fb3d8103690 2024-04-02T13:47:32.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.727+0000 7fb3e03ae740 1 Processor -- start 2024-04-02T13:47:32.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.727+0000 7fb3e03ae740 1 -- start start 2024-04-02T13:47:32.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3e03ae740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 0x55557498d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3e03ae740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498dbe0 0x555574996280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3d4ff9640 1 --2- 172.21.15.33:0/2143873872 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb3c40a3da0 0x7fb3c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3e03ae740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574998bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3e03ae740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5555748f1770 con 0x5555749967c0 2024-04-02T13:47:32.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3e03ae740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555574899960 con 0x55557498d2c0 2024-04-02T13:47:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3e03ae740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5555748ed8c0 con 0x55557498dbe0 2024-04-02T13:47:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574998bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3cffff640 1 --2- 172.21.15.33:0/2143873872 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb3c40a3da0 0x7fb3c40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574998bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44108/0 (socket says 172.21.15.33:44108) 2024-04-02T13:47:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3d77fe640 1 -- 172.21.15.33:0/38086060 learned_addr learned my addr 172.21.15.33:0/38086060 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/38086060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 0x55557498d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3d4ff9640 1 -- 172.21.15.33:0/2143873872 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fb3c00f2c20 con 0x7fb3d8103690 2024-04-02T13:47:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3d6ffd640 1 --2- 172.21.15.33:0/38086060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498dbe0 0x555574996280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3d77fe640 1 -- 172.21.15.33:0/38086060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498dbe0 msgr2=0x555574996280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/38086060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498dbe0 0x555574996280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3d77fe640 1 -- 172.21.15.33:0/38086060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 msgr2=0x55557498d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/38086060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 0x55557498d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3d77fe640 1 -- 172.21.15.33:0/38086060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555746cd7e0 con 0x5555749967c0 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.728+0000 7fb3d6ffd640 1 --2- 172.21.15.33:0/38086060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498dbe0 0x555574996280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.729+0000 7fb3cffff640 1 --2- 172.21.15.33:0/2143873872 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb3c40a3da0 0x7fb3c40a6260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00f05a0 tx=0x7fb3c00a9040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.729+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/38086060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574998bb0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00129b0 tx=0x7fb3d0009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c33daadf011a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.729+0000 7fb3cd7fa640 1 -- 172.21.15.33:0/38086060 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0004d20 con 0x5555749967c0 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.729+0000 7fb3cd7fa640 1 -- 172.21.15.33:0/38086060 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb3d000dbb0 con 0x5555749967c0 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.729+0000 7fb3cd7fa640 1 -- 172.21.15.33:0/38086060 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d00056a0 con 0x5555749967c0 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.729+0000 7fb3e03ae740 1 -- 172.21.15.33:0/38086060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 msgr2=0x555574998bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.729+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/38086060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574998bb0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00129b0 tx=0x7fb3d0009630 comp rx=0 tx=0).stop 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 -- 172.21.15.33:0/38086060 shutdown_connections 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/38086060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574998bb0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/38086060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498dbe0 0x555574996280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/38086060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 0x55557498d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 -- 172.21.15.33:0/38086060 >> 172.21.15.33:0/38086060 conn(0x5555749879f0 msgr2=0x555574987dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 -- 172.21.15.33:0/38086060 shutdown_connections 2024-04-02T13:47:32.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 -- 172.21.15.33:0/38086060 wait complete. 2024-04-02T13:47:32.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 -- start start 2024-04-02T13:47:32.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 Processor -- start 2024-04-02T13:47:32.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 0x5555749937f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498dbe0 0x555574993d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574994270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5555749995b0 con 0x5555749967c0 2024-04-02T13:47:32.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5555748acd20 con 0x55557498d2c0 2024-04-02T13:47:32.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555574999730 con 0x55557498dbe0 2024-04-02T13:47:32.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3e03ae740 10 client.0 _ll_register_callbacks cb 0x555574879a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:32.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574994270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574994270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44114/0 (socket says 172.21.15.33:44114) 2024-04-02T13:47:32.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.730+0000 7fb3d77fe640 1 -- 172.21.15.33:0/2322259077 learned_addr learned my addr 172.21.15.33:0/2322259077 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:32.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3d6ffd640 1 --2- 172.21.15.33:0/2322259077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498dbe0 0x555574993d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/2322259077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 0x5555749937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3d77fe640 1 -- 172.21.15.33:0/2322259077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498dbe0 msgr2=0x555574993d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/2322259077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498dbe0 0x555574993d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3d77fe640 1 -- 172.21.15.33:0/2322259077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 msgr2=0x5555749937f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/2322259077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 0x5555749937f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3d77fe640 1 -- 172.21.15.33:0/2322259077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55557472df70 con 0x5555749967c0 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/2322259077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574994270 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00095d0 tx=0x7fb3d0005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a5fa3ef10c6152 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3be7fc640 1 -- 172.21.15.33:0/2322259077 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0027040 con 0x5555749967c0 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3be7fc640 1 -- 172.21.15.33:0/2322259077 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fb3d000d6e0 con 0x5555749967c0 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3e03ae740 1 -- 172.21.15.33:0/2322259077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3d000dbb0 con 0x5555749967c0 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3be7fc640 1 -- 172.21.15.33:0/2322259077 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb3d0035050 con 0x5555749967c0 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3e03ae740 10 client.17529 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.731+0000 7fb3e03ae740 1 -- 172.21.15.33:0/2322259077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5555745c5170 con 0x5555749967c0 2024-04-02T13:47:32.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.732+0000 7fb3be7fc640 1 -- 172.21.15.33:0/2322259077 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb3d000dbb0 con 0x5555749967c0 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.732+0000 7fb3be7fc640 1 -- 172.21.15.33:0/2322259077 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fb3d001d430 con 0x5555749967c0 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.732+0000 7fb3e03ae740 10 client.17529 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.732+0000 7fb3e03ae740 10 client.17529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.732+0000 7fb3e03ae740 20 client.17529 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.732+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/2322259077 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5555749a6a60 0x5555749a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.732+0000 7fb3e03ae740 4 client.17529 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.732+0000 7fb3e03ae740 1 -- 172.21.15.33:0/2322259077 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555748fecb0 con 0x5555749a6a60 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.732+0000 7fb3d6ffd640 1 --2- 172.21.15.33:0/2322259077 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5555749a6a60 0x5555749a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3d6ffd640 1 --2- 172.21.15.33:0/2322259077 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5555749a6a60 0x5555749a8ea0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd120de7a4593051 server_cookie=c217dad94fe5a54e in_seq=0 out_seq=0 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3be7fc640 10 client.17529 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3be7fc640 1 -- 172.21.15.33:0/2322259077 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555748fecb0 con 0x5555749a6a60 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3be7fc640 10 client.17529 handle_command_reply: tid=0 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3e03ae740 1 client.17529 shutdown 2024-04-02T13:47:32.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3e03ae740 1 -- 172.21.15.33:0/2322259077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 msgr2=0x555574994270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/2322259077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574994270 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00095d0 tx=0x7fb3d0005280 comp rx=0 tx=0).stop 2024-04-02T13:47:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3e03ae740 1 -- 172.21.15.33:0/2322259077 shutdown_connections 2024-04-02T13:47:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/2322259077 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5555749a6a60 0x5555749a8ea0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/2322259077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574994270 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/2322259077 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498dbe0 0x555574993d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3be7fc640 0 client.17529 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/2322259077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 0x5555749937f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.733+0000 7fb3e03ae740 1 -- 172.21.15.33:0/2322259077 >> 172.21.15.33:0/2322259077 conn(0x5555749879f0 msgr2=0x555574989740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.734+0000 7fb3e03ae740 1 -- 172.21.15.33:0/2322259077 shutdown_connections 2024-04-02T13:47:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.734+0000 7fb3e03ae740 1 -- 172.21.15.33:0/2322259077 wait complete. 2024-04-02T13:47:32.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.734+0000 7fb3e03ae740 20 client.17529 trim_cache size 0 max 16384 2024-04-02T13:47:32.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.751+0000 7fb3e03ae740 1 Processor -- start 2024-04-02T13:47:32.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.751+0000 7fb3e03ae740 1 -- start start 2024-04-02T13:47:32.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3e03ae740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 0x555574995f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3e03ae740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55557498dbe0 0x555574989910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3e03ae740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555749967c0 0x5555749a2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3e03ae740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81b18a0 con 0x55557498dbe0 2024-04-02T13:47:32.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3e03ae740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81af170 con 0x55557498d2c0 2024-04-02T13:47:32.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3e03ae740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81acfe0 con 0x5555749967c0 2024-04-02T13:47:32.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3d6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55557498dbe0 0x555574989910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3d6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55557498dbe0 0x555574989910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44116/0 (socket says 172.21.15.33:44116) 2024-04-02T13:47:32.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3d6ffd640 1 -- 172.21.15.33:0/3260848620 learned_addr learned my addr 172.21.15.33:0/3260848620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:32.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/3260848620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 0x555574995f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/3260848620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555749967c0 0x5555749a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3d6ffd640 1 -- 172.21.15.33:0/3260848620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555749967c0 msgr2=0x5555749a2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3d6ffd640 1 --2- 172.21.15.33:0/3260848620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555749967c0 0x5555749a2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3d6ffd640 1 -- 172.21.15.33:0/3260848620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 msgr2=0x555574995f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3d6ffd640 1 --2- 172.21.15.33:0/3260848620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 0x555574995f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3d6ffd640 1 -- 172.21.15.33:0/3260848620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55557498da10 con 0x55557498dbe0 2024-04-02T13:47:32.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/3260848620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 0x555574995f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:32.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.752+0000 7fb3d6ffd640 1 --2- 172.21.15.33:0/3260848620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55557498dbe0 0x555574989910 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80058c0 tx=0x7fb3c8009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37bdb0a22e72a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:32.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.753+0000 7fb3ccff9640 1 -- 172.21.15.33:0/3260848620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c801f020 con 0x55557498dbe0 2024-04-02T13:47:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.753+0000 7fb3ccff9640 1 -- 172.21.15.33:0/3260848620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb3c8005500 con 0x55557498dbe0 2024-04-02T13:47:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.753+0000 7fb3ccff9640 1 -- 172.21.15.33:0/3260848620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8004990 con 0x55557498dbe0 2024-04-02T13:47:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.753+0000 7fb3e03ae740 1 -- 172.21.15.33:0/3260848620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55557498dbe0 msgr2=0x555574989910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.753+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/3260848620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55557498dbe0 0x555574989910 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80058c0 tx=0x7fb3c8009430 comp rx=0 tx=0).stop 2024-04-02T13:47:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.753+0000 7fb3e03ae740 1 -- 172.21.15.33:0/3260848620 shutdown_connections 2024-04-02T13:47:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.753+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/3260848620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5555749967c0 0x5555749a2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.753+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/3260848620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55557498dbe0 0x555574989910 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.753+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/3260848620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498d2c0 0x555574995f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.753+0000 7fb3e03ae740 1 -- 172.21.15.33:0/3260848620 >> 172.21.15.33:0/3260848620 conn(0x5555749879f0 msgr2=0x555574994040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.753+0000 7fb3e03ae740 1 -- 172.21.15.33:0/3260848620 shutdown_connections 2024-04-02T13:47:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.753+0000 7fb3e03ae740 1 -- 172.21.15.33:0/3260848620 wait complete. 2024-04-02T13:47:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3e03ae740 1 -- start start 2024-04-02T13:47:32.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3e03ae740 1 Processor -- start 2024-04-02T13:47:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3e03ae740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498d2c0 0x555574a04fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3e03ae740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498dbe0 0x555574a054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3e03ae740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574a05a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3e03ae740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81ad630 con 0x5555749967c0 2024-04-02T13:47:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3e03ae740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81aeaa0 con 0x55557498dbe0 2024-04-02T13:47:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3e03ae740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81add00 con 0x55557498d2c0 2024-04-02T13:47:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3e03ae740 10 client.0 _ll_register_callbacks cb 0x7fb3d00123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574a05a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574a05a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44120/0 (socket says 172.21.15.33:44120) 2024-04-02T13:47:32.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3d77fe640 1 -- 172.21.15.33:0/1884445908 learned_addr learned my addr 172.21.15.33:0/1884445908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:32.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3d6ffd640 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498dbe0 0x555574a054e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498d2c0 0x555574a04fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3d77fe640 1 -- 172.21.15.33:0/1884445908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498d2c0 msgr2=0x555574a04fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498d2c0 0x555574a04fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3d77fe640 1 -- 172.21.15.33:0/1884445908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498dbe0 msgr2=0x555574a054e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498dbe0 0x555574a054e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3d77fe640 1 -- 172.21.15.33:0/1884445908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555574a10120 con 0x5555749967c0 2024-04-02T13:47:32.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3d6ffd640 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498dbe0 0x555574a054e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:32.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3d77fe640 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574a05a20 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00053f0 tx=0x7fb3d0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988a28594a97de62 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:32.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.754+0000 7fb3be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:32.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3be7fc640 1 -- 172.21.15.33:0/1884445908 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0042020 con 0x5555749967c0 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3be7fc640 1 -- 172.21.15.33:0/1884445908 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fb3d001d430 con 0x5555749967c0 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3e03ae740 1 -- 172.21.15.33:0/1884445908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3c8005500 con 0x5555749967c0 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3be7fc640 1 -- 172.21.15.33:0/1884445908 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb3d004a050 con 0x5555749967c0 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3e03ae740 10 client.17535 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3e03ae740 1 -- 172.21.15.33:0/1884445908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555574a11460 con 0x5555749967c0 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3be7fc640 1 -- 172.21.15.33:0/1884445908 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb3c8005500 con 0x5555749967c0 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3be7fc640 1 -- 172.21.15.33:0/1884445908 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fb3d0005710 con 0x5555749967c0 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3e03ae740 10 client.17535 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3e03ae740 10 client.17535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3e03ae740 20 client.17535 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x555574a11ac0 0x555574a13eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3e03ae740 4 client.17535 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3e03ae740 1 -- 172.21.15.33:0/1884445908 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5555748f9000 con 0x555574a11ac0 2024-04-02T13:47:32.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.755+0000 7fb3d6ffd640 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x555574a11ac0 0x555574a13eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.756+0000 7fb3d6ffd640 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x555574a11ac0 0x555574a13eb0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df22ecacdf386661 server_cookie=54a01686e1cef56d in_seq=0 out_seq=0 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.756+0000 7fb3be7fc640 10 client.17535 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3be7fc640 1 -- 172.21.15.33:0/1884445908 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5555748f9000 con 0x555574a11ac0 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3be7fc640 10 client.17535 handle_command_reply: tid=0 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3e03ae740 1 client.17535 shutdown 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3e03ae740 1 -- 172.21.15.33:0/1884445908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 msgr2=0x555574a05a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574a05a20 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00053f0 tx=0x7fb3d0014040 comp rx=0 tx=0).stop 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3e03ae740 1 -- 172.21.15.33:0/1884445908 shutdown_connections 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x555574a11ac0 0x555574a13eb0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555749967c0 0x555574a05a20 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55557498dbe0 0x555574a054e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3e03ae740 1 --2- 172.21.15.33:0/1884445908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55557498d2c0 0x555574a04fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3be7fc640 0 client.17535 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3e03ae740 1 -- 172.21.15.33:0/1884445908 >> 172.21.15.33:0/1884445908 conn(0x5555749879f0 msgr2=0x555574986220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:32.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3e03ae740 1 -- 172.21.15.33:0/1884445908 shutdown_connections 2024-04-02T13:47:32.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3e03ae740 1 -- 172.21.15.33:0/1884445908 wait complete. 2024-04-02T13:47:32.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.757+0000 7fb3e03ae740 20 client.17535 trim_cache size 0 max 16384 2024-04-02T13:47:32.768 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:47:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:47:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "d171e14b-ec7e-4048-839c-46a168efb8e6", 2024-04-02T13:47:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:47:32.768 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:47:32.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.758+0000 7fb3dd7c8640 1 -- 172.21.15.33:0/2143873872 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb3c40a3da0 msgr2=0x7fb3c40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.758+0000 7fb3dd7c8640 1 --2- 172.21.15.33:0/2143873872 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb3c40a3da0 0x7fb3c40a6260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00f05a0 tx=0x7fb3c00a9040 comp rx=0 tx=0).stop 2024-04-02T13:47:32.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.758+0000 7fb3dd7c8640 1 -- 172.21.15.33:0/2143873872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d8103690 msgr2=0x7fb3d81989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.758+0000 7fb3dd7c8640 1 --2- 172.21.15.33:0/2143873872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d8103690 0x7fb3d81989e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009230 tx=0x7fb3c00045e0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.758+0000 7fb3dd7c8640 1 -- 172.21.15.33:0/2143873872 shutdown_connections 2024-04-02T13:47:32.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.758+0000 7fb3dd7c8640 1 --2- 172.21.15.33:0/2143873872 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb3c40a3da0 0x7fb3c40a6260 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.758+0000 7fb3dd7c8640 1 --2- 172.21.15.33:0/2143873872 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3d810c140 0x7fb3d819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.758+0000 7fb3dd7c8640 1 --2- 172.21.15.33:0/2143873872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3d8103690 0x7fb3d81989e0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.758+0000 7fb3dd7c8640 1 --2- 172.21.15.33:0/2143873872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3d8102d70 0x7fb3d81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.758+0000 7fb3dd7c8640 1 -- 172.21.15.33:0/2143873872 >> 172.21.15.33:0/2143873872 conn(0x7fb3d8077b60 msgr2=0x7fb3d80fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:32.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.758+0000 7fb3dd7c8640 1 -- 172.21.15.33:0/2143873872 shutdown_connections 2024-04-02T13:47:32.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.758+0000 7fb3dd7c8640 1 -- 172.21.15.33:0/2143873872 wait complete. 2024-04-02T13:47:32.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:47:32.771 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:47:32.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.875+0000 7fd7973db640 1 -- 172.21.15.33:0/4186128700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790100990 msgr2=0x7fd790100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.875+0000 7fd7973db640 1 --2- 172.21.15.33:0/4186128700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790100990 0x7fd790100e10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd780009930 tx=0x7fd78002bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.875+0000 7fd7973db640 1 -- 172.21.15.33:0/4186128700 shutdown_connections 2024-04-02T13:47:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.875+0000 7fd7973db640 1 --2- 172.21.15.33:0/4186128700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd790101350 0x7fd79010d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.875+0000 7fd7973db640 1 --2- 172.21.15.33:0/4186128700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790100990 0x7fd790100e10 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.875+0000 7fd7973db640 1 --2- 172.21.15.33:0/4186128700 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7901069a0 0x7fd790106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.875+0000 7fd7973db640 1 -- 172.21.15.33:0/4186128700 >> 172.21.15.33:0/4186128700 conn(0x7fd790077ad0 msgr2=0x7fd790077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.875+0000 7fd7973db640 1 -- 172.21.15.33:0/4186128700 shutdown_connections 2024-04-02T13:47:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.875+0000 7fd7973db640 1 -- 172.21.15.33:0/4186128700 wait complete. 2024-04-02T13:47:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.876+0000 7fd7973db640 1 Processor -- start 2024-04-02T13:47:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.876+0000 7fd7973db640 1 -- start start 2024-04-02T13:47:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.876+0000 7fd7973db640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790100990 0x7fd79010e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.876+0000 7fd7973db640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd790101350 0x7fd79010e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.876+0000 7fd7973db640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7901069a0 0x7fd79010ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.876+0000 7fd7973db640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd790111ea0 con 0x7fd790100990 2024-04-02T13:47:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.876+0000 7fd7973db640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd790111d20 con 0x7fd7901069a0 2024-04-02T13:47:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.876+0000 7fd7973db640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd790112020 con 0x7fd790101350 2024-04-02T13:47:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.876+0000 7fd795177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790100990 0x7fd79010e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.876+0000 7fd795177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790100990 0x7fd79010e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44150/0 (socket says 172.21.15.33:44150) 2024-04-02T13:47:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.876+0000 7fd795177640 1 -- 172.21.15.33:0/1817972961 learned_addr learned my addr 172.21.15.33:0/1817972961 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.876+0000 7fd795978640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7901069a0 0x7fd79010ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.877+0000 7fd794976640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd790101350 0x7fd79010e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.877+0000 7fd795177640 1 -- 172.21.15.33:0/1817972961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd790101350 msgr2=0x7fd79010e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.877+0000 7fd795177640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd790101350 0x7fd79010e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.877+0000 7fd795177640 1 -- 172.21.15.33:0/1817972961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7901069a0 msgr2=0x7fd79010ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.877+0000 7fd795177640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7901069a0 0x7fd79010ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.877+0000 7fd795177640 1 -- 172.21.15.33:0/1817972961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7901a6940 con 0x7fd790100990 2024-04-02T13:47:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.877+0000 7fd795177640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790100990 0x7fd79010e2f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00b4a0 tx=0x7fd78c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.877+0000 7fd7867fc640 1 -- 172.21.15.33:0/1817972961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c014070 con 0x7fd790100990 2024-04-02T13:47:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.877+0000 7fd7867fc640 1 -- 172.21.15.33:0/1817972961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd78c010ae0 con 0x7fd790100990 2024-04-02T13:47:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.877+0000 7fd7973db640 1 -- 172.21.15.33:0/1817972961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7901a6c30 con 0x7fd790100990 2024-04-02T13:47:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.877+0000 7fd7867fc640 1 -- 172.21.15.33:0/1817972961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c017c70 con 0x7fd790100990 2024-04-02T13:47:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.877+0000 7fd7973db640 1 -- 172.21.15.33:0/1817972961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7901a7180 con 0x7fd790100990 2024-04-02T13:47:32.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.879+0000 7fd7867fc640 1 -- 172.21.15.33:0/1817972961 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd78c017070 con 0x7fd790100990 2024-04-02T13:47:32.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.879+0000 7fd799fc4740 1 Processor -- start 2024-04-02T13:47:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.879+0000 7fd799fc4740 1 -- start start 2024-04-02T13:47:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd799fc4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ff58d0 0x5641b9ff5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd799fc4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 0x5641b9ffe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd799fc4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ffedd0 0x5641ba0011c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd799fc4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5641b9f59830 con 0x5641b9ff61f0 2024-04-02T13:47:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd799fc4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5641b9f01820 con 0x5641b9ff58d0 2024-04-02T13:47:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd799fc4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5641b9f557e0 con 0x5641b9ffedd0 2024-04-02T13:47:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd7867fc640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd77c0a3d20 0x7fd77c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd795177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 0x5641b9ffe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd795978640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd77c0a3d20 0x7fd77c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd794976640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ffedd0 0x5641ba0011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd795177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 0x5641b9ffe890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44158/0 (socket says 172.21.15.33:44158) 2024-04-02T13:47:32.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd794976640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ffedd0 0x5641ba0011c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.33:48882/0 (socket says 172.21.15.33:48882) 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd795177640 1 -- 172.21.15.33:0/1573144600 learned_addr learned my addr 172.21.15.33:0/1573144600 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd795978640 1 --2- 172.21.15.33:0/1573144600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ff58d0 0x5641b9ff5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd7867fc640 1 -- 172.21.15.33:0/1817972961 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fd78c0cfea0 con 0x7fd790100990 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd795177640 1 -- 172.21.15.33:0/1573144600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ffedd0 msgr2=0x5641ba0011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd795177640 1 --2- 172.21.15.33:0/1573144600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ffedd0 0x5641ba0011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd795177640 1 -- 172.21.15.33:0/1573144600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ff58d0 msgr2=0x5641b9ff5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd795177640 1 --2- 172.21.15.33:0/1573144600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ff58d0 0x5641b9ff5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd795177640 1 -- 172.21.15.33:0/1573144600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b9d377d0 con 0x5641b9ff61f0 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd795978640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd77c0a3d20 0x7fd77c0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd788005fd0 tx=0x7fd788009f00 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.880+0000 7fd795177640 1 --2- 172.21.15.33:0/1573144600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 0x5641b9ffe890 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0a4260 tx=0x7fd78c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106e655c3dc7b319 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.881+0000 7fd77b7fe640 1 -- 172.21.15.33:0/1573144600 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c01e070 con 0x5641b9ff61f0 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.881+0000 7fd77b7fe640 1 -- 172.21.15.33:0/1573144600 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd78c09ce50 con 0x5641b9ff61f0 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.881+0000 7fd77b7fe640 1 -- 172.21.15.33:0/1573144600 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c017360 con 0x5641b9ff61f0 2024-04-02T13:47:32.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.881+0000 7fd799fc4740 1 -- 172.21.15.33:0/1573144600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 msgr2=0x5641b9ffe890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.881+0000 7fd799fc4740 1 --2- 172.21.15.33:0/1573144600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 0x5641b9ffe890 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0a4260 tx=0x7fd78c0046c0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.881+0000 7fd799fc4740 1 -- 172.21.15.33:0/1573144600 shutdown_connections 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.881+0000 7fd799fc4740 1 --2- 172.21.15.33:0/1573144600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ffedd0 0x5641ba0011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.881+0000 7fd799fc4740 1 --2- 172.21.15.33:0/1573144600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 0x5641b9ffe890 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.881+0000 7fd799fc4740 1 --2- 172.21.15.33:0/1573144600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ff58d0 0x5641b9ff5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.881+0000 7fd799fc4740 1 -- 172.21.15.33:0/1573144600 >> 172.21.15.33:0/1573144600 conn(0x5641b9ff0000 msgr2=0x5641b9ff03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.881+0000 7fd799fc4740 1 -- 172.21.15.33:0/1573144600 shutdown_connections 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.881+0000 7fd799fc4740 1 -- 172.21.15.33:0/1573144600 wait complete. 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd799fc4740 1 -- start start 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd799fc4740 1 Processor -- start 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd799fc4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 0x5641b9ffe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd799fc4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ff61f0 0x5641b9ffba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd799fc4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ffedd0 0x5641b9ffbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd799fc4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5641ba001bc0 con 0x5641b9ff58d0 2024-04-02T13:47:32.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd799fc4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5641b9f14d30 con 0x5641b9ffedd0 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd799fc4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5641ba001d40 con 0x5641b9ff61f0 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd799fc4740 10 client.0 _ll_register_callbacks cb 0x5641b9f14720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd795177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 0x5641b9ffe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd795177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 0x5641b9ffe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44166/0 (socket says 172.21.15.33:44166) 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd795177640 1 -- 172.21.15.33:0/3899309421 learned_addr learned my addr 172.21.15.33:0/3899309421 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd794976640 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ff61f0 0x5641b9ffba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd795978640 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ffedd0 0x5641b9ffbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd795177640 1 -- 172.21.15.33:0/3899309421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ff61f0 msgr2=0x5641b9ffba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd795177640 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ff61f0 0x5641b9ffba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd795177640 1 -- 172.21.15.33:0/3899309421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ffedd0 msgr2=0x5641b9ffbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.882+0000 7fd795177640 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ffedd0 0x5641b9ffbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd795177640 1 -- 172.21.15.33:0/3899309421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b9c6d3f0 con 0x5641b9ff58d0 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd795978640 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ffedd0 0x5641b9ffbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:32.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd795177640 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 0x5641b9ffe680 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00be60 tx=0x7fd78c017590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a53fcb216967e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd778ff9640 1 -- 172.21.15.33:0/3899309421 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0b5070 con 0x5641b9ff58d0 2024-04-02T13:47:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd778ff9640 1 -- 172.21.15.33:0/3899309421 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fd78c098190 con 0x5641b9ff58d0 2024-04-02T13:47:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd799fc4740 1 -- 172.21.15.33:0/3899309421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd78c09ce50 con 0x5641b9ff58d0 2024-04-02T13:47:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd778ff9640 1 -- 172.21.15.33:0/3899309421 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd78c01e040 con 0x5641b9ff58d0 2024-04-02T13:47:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd799fc4740 10 client.17547 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd799fc4740 1 -- 172.21.15.33:0/3899309421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5641b9c2d0f0 con 0x5641b9ff58d0 2024-04-02T13:47:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd778ff9640 1 -- 172.21.15.33:0/3899309421 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd78c09ce50 con 0x5641b9ff58d0 2024-04-02T13:47:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd778ff9640 1 -- 172.21.15.33:0/3899309421 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fd78c0bace0 con 0x5641b9ff58d0 2024-04-02T13:47:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd799fc4740 10 client.17547 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd799fc4740 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd799fc4740 20 client.17547 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:32.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd799fc4740 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5641ba00ebf0 0x5641ba010fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd799fc4740 4 client.17547 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:32.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.883+0000 7fd799fc4740 1 -- 172.21.15.33:0/3899309421 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641b9f66d70 con 0x5641ba00ebf0 2024-04-02T13:47:32.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.884+0000 7fd794976640 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5641ba00ebf0 0x5641ba010fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.884+0000 7fd778ff9640 10 client.17547 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:32.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.884+0000 7fd794976640 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5641ba00ebf0 0x5641ba010fe0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=633b51e08a0d3a6d server_cookie=86b7a62a6b201d84 in_seq=0 out_seq=0 2024-04-02T13:47:32.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd778ff9640 1 -- 172.21.15.33:0/3899309421 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641b9f66d70 con 0x5641ba00ebf0 2024-04-02T13:47:32.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd778ff9640 10 client.17547 handle_command_reply: tid=0 2024-04-02T13:47:32.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd799fc4740 1 client.17547 shutdown 2024-04-02T13:47:32.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd799fc4740 1 -- 172.21.15.33:0/3899309421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 msgr2=0x5641b9ffe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd799fc4740 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 0x5641b9ffe680 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00be60 tx=0x7fd78c017590 comp rx=0 tx=0).stop 2024-04-02T13:47:32.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd799fc4740 1 -- 172.21.15.33:0/3899309421 shutdown_connections 2024-04-02T13:47:32.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd799fc4740 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5641ba00ebf0 0x5641ba010fe0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd799fc4740 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ffedd0 0x5641b9ffbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd799fc4740 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ff61f0 0x5641b9ffba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd778ff9640 0 client.17547 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:32.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd799fc4740 1 --2- 172.21.15.33:0/3899309421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 0x5641b9ffe680 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd799fc4740 1 -- 172.21.15.33:0/3899309421 >> 172.21.15.33:0/3899309421 conn(0x5641b9ff0000 msgr2=0x5641b9ff1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:32.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd799fc4740 1 -- 172.21.15.33:0/3899309421 shutdown_connections 2024-04-02T13:47:32.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd799fc4740 1 -- 172.21.15.33:0/3899309421 wait complete. 2024-04-02T13:47:32.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.885+0000 7fd799fc4740 20 client.17547 trim_cache size 0 max 16384 2024-04-02T13:47:32.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd799fc4740 1 Processor -- start 2024-04-02T13:47:32.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd799fc4740 1 -- start start 2024-04-02T13:47:32.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd799fc4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 0x5641b9fee9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd799fc4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ff61f0 0x5641b9ffe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd799fc4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ffedd0 0x5641b9ffe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd799fc4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901b3c80 con 0x5641b9ff58d0 2024-04-02T13:47:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd799fc4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901b1550 con 0x5641b9ffedd0 2024-04-02T13:47:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd799fc4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901af3c0 con 0x5641b9ff61f0 2024-04-02T13:47:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd795177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 0x5641b9fee9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd795177640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 0x5641b9fee9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44174/0 (socket says 172.21.15.33:44174) 2024-04-02T13:47:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd795177640 1 -- 172.21.15.33:0/2817593982 learned_addr learned my addr 172.21.15.33:0/2817593982 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd795177640 1 -- 172.21.15.33:0/2817593982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ff61f0 msgr2=0x5641b9ffe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd795978640 1 --2- 172.21.15.33:0/2817593982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ffedd0 0x5641b9ffe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.903+0000 7fd794976640 1 --2- 172.21.15.33:0/2817593982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ff61f0 0x5641b9ffe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd795177640 1 --2- 172.21.15.33:0/2817593982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ff61f0 0x5641b9ffe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd795177640 1 -- 172.21.15.33:0/2817593982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ffedd0 msgr2=0x5641b9ffe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd795177640 1 --2- 172.21.15.33:0/2817593982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ffedd0 0x5641b9ffe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd795177640 1 -- 172.21.15.33:0/2817593982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b9fef860 con 0x5641b9ff58d0 2024-04-02T13:47:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd794976640 1 --2- 172.21.15.33:0/2817593982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ff61f0 0x5641b9ffe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd795978640 1 --2- 172.21.15.33:0/2817593982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ffedd0 0x5641b9ffe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd795177640 1 --2- 172.21.15.33:0/2817593982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 0x5641b9fee9b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fd78c097f80 tx=0x7fd78c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c309ac4c578f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd77affd640 1 -- 172.21.15.33:0/2817593982 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0b5040 con 0x5641b9ff58d0 2024-04-02T13:47:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd77affd640 1 -- 172.21.15.33:0/2817593982 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd78c09ce50 con 0x5641b9ff58d0 2024-04-02T13:47:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd77affd640 1 -- 172.21.15.33:0/2817593982 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0a43a0 con 0x5641b9ff58d0 2024-04-02T13:47:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd799fc4740 1 -- 172.21.15.33:0/2817593982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 msgr2=0x5641b9fee9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd799fc4740 1 --2- 172.21.15.33:0/2817593982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 0x5641b9fee9b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fd78c097f80 tx=0x7fd78c0046c0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd799fc4740 1 -- 172.21.15.33:0/2817593982 shutdown_connections 2024-04-02T13:47:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.904+0000 7fd799fc4740 1 --2- 172.21.15.33:0/2817593982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ffedd0 0x5641b9ffe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 1 --2- 172.21.15.33:0/2817593982 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ff61f0 0x5641b9ffe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 1 --2- 172.21.15.33:0/2817593982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff58d0 0x5641b9fee9b0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 1 -- 172.21.15.33:0/2817593982 >> 172.21.15.33:0/2817593982 conn(0x5641b9ff0000 msgr2=0x5641ba00a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 1 -- 172.21.15.33:0/2817593982 shutdown_connections 2024-04-02T13:47:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 1 -- 172.21.15.33:0/2817593982 wait complete. 2024-04-02T13:47:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 1 -- start start 2024-04-02T13:47:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 1 Processor -- start 2024-04-02T13:47:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ff58d0 0x5641ba0143c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 0x5641ba011750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ffedd0 0x5641ba011c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901afa10 con 0x5641b9ff61f0 2024-04-02T13:47:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901b0e80 con 0x5641b9ff58d0 2024-04-02T13:47:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd7901b00e0 con 0x5641b9ffedd0 2024-04-02T13:47:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd799fc4740 10 client.0 _ll_register_callbacks cb 0x7fd78c09b660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd794976640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 0x5641ba011750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd794976640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 0x5641ba011750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44178/0 (socket says 172.21.15.33:44178) 2024-04-02T13:47:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd794976640 1 -- 172.21.15.33:0/680925586 learned_addr learned my addr 172.21.15.33:0/680925586 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:47:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd795177640 1 --2- 172.21.15.33:0/680925586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ff58d0 0x5641ba0143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.905+0000 7fd795978640 1 --2- 172.21.15.33:0/680925586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ffedd0 0x5641ba011c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd794976640 1 -- 172.21.15.33:0/680925586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ffedd0 msgr2=0x5641ba011c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd794976640 1 --2- 172.21.15.33:0/680925586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ffedd0 0x5641ba011c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd794976640 1 -- 172.21.15.33:0/680925586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ff58d0 msgr2=0x5641ba0143c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd794976640 1 --2- 172.21.15.33:0/680925586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ff58d0 0x5641ba0143c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd794976640 1 -- 172.21.15.33:0/680925586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b9ff6020 con 0x5641b9ff61f0 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd794976640 1 --2- 172.21.15.33:0/680925586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 0x5641ba011750 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fd780009900 tx=0x7fd7800044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5028a51bf83e66ea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd778ff9640 1 -- 172.21.15.33:0/680925586 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78004b060 con 0x5641b9ff61f0 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd778ff9640 1 -- 172.21.15.33:0/680925586 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fd780046b50 con 0x5641b9ff61f0 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd799fc4740 1 -- 172.21.15.33:0/680925586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd78c09ce50 con 0x5641b9ff61f0 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd778ff9640 1 -- 172.21.15.33:0/680925586 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd780052050 con 0x5641b9ff61f0 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd799fc4740 10 client.17553 fetch_fsmap learned FSMap version 36 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd799fc4740 1 -- 172.21.15.33:0/680925586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5641ba074be0 con 0x5641b9ff61f0 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.906+0000 7fd778ff9640 1 -- 172.21.15.33:0/680925586 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fd78c09ce50 con 0x5641b9ff61f0 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd778ff9640 1 -- 172.21.15.33:0/680925586 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fd780046790 con 0x5641b9ff61f0 2024-04-02T13:47:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd799fc4740 10 client.17553 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd799fc4740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd799fc4740 20 client.17553 populate_metadata read hostname 'smithi033' 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd799fc4740 1 --2- 172.21.15.33:0/680925586 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5641ba075330 0x5641ba077750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd799fc4740 4 client.17553 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd799fc4740 1 -- 172.21.15.33:0/680925586 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5641b9f610c0 con 0x5641ba075330 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd795177640 1 --2- 172.21.15.33:0/680925586 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5641ba075330 0x5641ba077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd778ff9640 10 client.17553 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd795177640 1 --2- 172.21.15.33:0/680925586 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5641ba075330 0x5641ba077750 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d88cd0e4a7b7ab4 server_cookie=5ffd21ba4771641 in_seq=0 out_seq=0 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd778ff9640 1 -- 172.21.15.33:0/680925586 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5641b9f610c0 con 0x5641ba075330 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd778ff9640 10 client.17553 handle_command_reply: tid=0 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd799fc4740 1 client.17553 shutdown 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd799fc4740 1 -- 172.21.15.33:0/680925586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 msgr2=0x5641ba011750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.907+0000 7fd799fc4740 1 --2- 172.21.15.33:0/680925586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 0x5641ba011750 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fd780009900 tx=0x7fd7800044b0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd799fc4740 1 -- 172.21.15.33:0/680925586 shutdown_connections 2024-04-02T13:47:32.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd799fc4740 1 --2- 172.21.15.33:0/680925586 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5641ba075330 0x5641ba077750 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd799fc4740 1 --2- 172.21.15.33:0/680925586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5641b9ffedd0 0x5641ba011c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd799fc4740 1 --2- 172.21.15.33:0/680925586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5641b9ff61f0 0x5641ba011750 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd799fc4740 1 --2- 172.21.15.33:0/680925586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641b9ff58d0 0x5641ba0143c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd778ff9640 0 client.17553 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:47:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd799fc4740 1 -- 172.21.15.33:0/680925586 >> 172.21.15.33:0/680925586 conn(0x5641b9ff0000 msgr2=0x5641b9ff6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd799fc4740 1 -- 172.21.15.33:0/680925586 shutdown_connections 2024-04-02T13:47:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd799fc4740 1 -- 172.21.15.33:0/680925586 wait complete. 2024-04-02T13:47:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd799fc4740 20 client.17553 trim_cache size 0 max 16384 2024-04-02T13:47:32.918 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:47:32.919 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:47:32.919 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:47:32.919 INFO:teuthology.orchestra.run.smithi033.stdout: "d171e14b-ec7e-4048-839c-46a168efb8e6": { 2024-04-02T13:47:32.919 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:47:32.919 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "d171e14b-ec7e-4048-839c-46a168efb8e6", 2024-04-02T13:47:32.919 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:47:32.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:47:32.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:47:32.919 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:47:32.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd7973db640 1 -- 172.21.15.33:0/1817972961 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd77c0a3d20 msgr2=0x7fd77c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd7973db640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd77c0a3d20 0x7fd77c0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd788005fd0 tx=0x7fd788009f00 comp rx=0 tx=0).stop 2024-04-02T13:47:32.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd7973db640 1 -- 172.21.15.33:0/1817972961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790100990 msgr2=0x7fd79010e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:32.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd7973db640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790100990 0x7fd79010e2f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00b4a0 tx=0x7fd78c00b960 comp rx=0 tx=0).stop 2024-04-02T13:47:32.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd7973db640 1 -- 172.21.15.33:0/1817972961 shutdown_connections 2024-04-02T13:47:32.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd7973db640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fd77c0a3d20 0x7fd77c0a61e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd7973db640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd7901069a0 0x7fd79010ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd7973db640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd790101350 0x7fd79010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd7973db640 1 --2- 172.21.15.33:0/1817972961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790100990 0x7fd79010e2f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:32.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd7973db640 1 -- 172.21.15.33:0/1817972961 >> 172.21.15.33:0/1817972961 conn(0x7fd790077ad0 msgr2=0x7fd790103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:32.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd7973db640 1 -- 172.21.15.33:0/1817972961 shutdown_connections 2024-04-02T13:47:32.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:47:32.908+0000 7fd7973db640 1 -- 172.21.15.33:0/1817972961 wait complete. 2024-04-02T13:47:32.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:32 smithi033 ceph-mon[32390]: pgmap v271: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:32.922 INFO:tasks.cephfs.filesystem:scrub status for tag:d171e14b-ec7e-4048-839c-46a168efb8e6 - {'path': '/', 'tag': 'd171e14b-ec7e-4048-839c-46a168efb8e6', 'options': 'recursive,force'} 2024-04-02T13:47:32.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:32 smithi145 ceph-mon[35646]: pgmap v271: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:32 smithi050 ceph-mon[35524]: pgmap v271: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:34.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:34 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:47:34.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:34 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:34.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:34 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:47:34.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:34 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:34.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:34 smithi145 ceph-mon[35646]: pgmap v272: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:47:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:34 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:47:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:34 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:34 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:47:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:34 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:34 smithi050 ceph-mon[35524]: pgmap v272: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:47:35.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:34 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:47:35.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:34 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:35.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:34 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:47:35.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:34 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:35.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:34 smithi033 ceph-mon[32390]: pgmap v272: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:47:36.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:36 smithi145 ceph-mon[35646]: pgmap v273: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:47:36.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:36 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:47:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:36 smithi050 ceph-mon[35524]: pgmap v273: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:47:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:36 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:47:37.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:36 smithi033 ceph-mon[32390]: pgmap v273: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:47:37.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:36 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:47:38.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:38 smithi145 ceph-mon[35646]: pgmap v274: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:47:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:38 smithi050 ceph-mon[35524]: pgmap v274: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:47:39.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:38 smithi033 ceph-mon[32390]: pgmap v274: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:47:40.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:40 smithi145 ceph-mon[35646]: pgmap v275: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:47:40.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:40 smithi050 ceph-mon[35524]: pgmap v275: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:47:41.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:40 smithi033 ceph-mon[32390]: pgmap v275: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:47:42.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:42 smithi145 ceph-mon[35646]: pgmap v276: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:47:42.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:42 smithi050 ceph-mon[35524]: pgmap v276: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:47:43.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:42 smithi033 ceph-mon[32390]: pgmap v276: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:47:44.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:44 smithi145 ceph-mon[35646]: pgmap v277: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:47:44.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:44 smithi050 ceph-mon[35524]: pgmap v277: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:47:45.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:44 smithi033 ceph-mon[32390]: pgmap v277: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:47:46.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:46 smithi145 ceph-mon[35646]: pgmap v278: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:46 smithi050 ceph-mon[35524]: pgmap v278: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:47.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:46 smithi033 ceph-mon[32390]: pgmap v278: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:48.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:48 smithi145 ceph-mon[35646]: pgmap v279: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:48.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:48 smithi050 ceph-mon[35524]: pgmap v279: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:49.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:48 smithi033 ceph-mon[32390]: pgmap v279: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:50.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:50 smithi145 ceph-mon[35646]: pgmap v280: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:47:50.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:50 smithi050 ceph-mon[35524]: pgmap v280: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:47:51.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:50 smithi033 ceph-mon[32390]: pgmap v280: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:47:52.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:52 smithi145 ceph-mon[35646]: pgmap v281: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:52.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:52 smithi050 ceph-mon[35524]: pgmap v281: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:53.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:52 smithi033 ceph-mon[32390]: pgmap v281: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:54.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:54 smithi145 ceph-mon[35646]: pgmap v282: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:47:54.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:54 smithi050 ceph-mon[35524]: pgmap v282: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:47:55.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:54 smithi033 ceph-mon[32390]: pgmap v282: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:47:56.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:56 smithi145 ceph-mon[35646]: pgmap v283: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:56.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:56 smithi050 ceph-mon[35524]: pgmap v283: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:57.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:56 smithi033 ceph-mon[32390]: pgmap v283: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:58.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:58 smithi145 ceph-mon[35646]: pgmap v284: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:58.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:58 smithi050 ceph-mon[35524]: pgmap v284: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:59.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:58 smithi033 ceph-mon[32390]: pgmap v284: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:47:59.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:59 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:59 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:59 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:47:59 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:47:59.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:59 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:59.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:59 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:59.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:59 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:59.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:47:59 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:48:00.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:59 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:59 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:59 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:00.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:47:59 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:48:00.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:00 smithi145 ceph-mon[35646]: pgmap v285: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:48:00.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:00 smithi050 ceph-mon[35524]: pgmap v285: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:48:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:00 smithi033 ceph-mon[32390]: pgmap v285: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:48:02.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:48:02.924 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:48:02.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:02 smithi033 ceph-mon[32390]: pgmap v286: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:02.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:02 smithi145 ceph-mon[35646]: pgmap v286: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:02.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:02 smithi050 ceph-mon[35524]: pgmap v286: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.039+0000 7f75e644a640 1 -- 172.21.15.33:0/474069189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75e010c140 msgr2=0x7f75e010e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.039+0000 7f75e644a640 1 --2- 172.21.15.33:0/474069189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75e010c140 0x7f75e010e5b0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009ca0 tx=0x7f75d802c180 comp rx=0 tx=0).stop 2024-04-02T13:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.039+0000 7f75e644a640 1 -- 172.21.15.33:0/474069189 shutdown_connections 2024-04-02T13:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.039+0000 7f75e644a640 1 --2- 172.21.15.33:0/474069189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75e010c140 0x7f75e010e5b0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.039+0000 7f75e644a640 1 --2- 172.21.15.33:0/474069189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75e0103690 0x7f75e010bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.039+0000 7f75e644a640 1 --2- 172.21.15.33:0/474069189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e0102d70 0x7f75e0103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.039+0000 7f75e644a640 1 -- 172.21.15.33:0/474069189 >> 172.21.15.33:0/474069189 conn(0x7f75e0077b60 msgr2=0x7f75e0077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.039+0000 7f75e644a640 1 -- 172.21.15.33:0/474069189 shutdown_connections 2024-04-02T13:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.039+0000 7f75e644a640 1 -- 172.21.15.33:0/474069189 wait complete. 2024-04-02T13:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.040+0000 7f75e644a640 1 Processor -- start 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.040+0000 7f75e644a640 1 -- start start 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.040+0000 7f75e644a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75e0102d70 0x7f75e0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.040+0000 7f75e644a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75e0103690 0x7f75e01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.040+0000 7f75e644a640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e010c140 0x7f75e019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.040+0000 7f75e644a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75e0111ee0 con 0x7f75e0103690 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.040+0000 7f75e644a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f75e0111d60 con 0x7f75e0102d70 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.040+0000 7f75e644a640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75e0112060 con 0x7f75e010c140 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75e0103690 0x7f75e01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75e0103690 0x7f75e01988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50182/0 (socket says 172.21.15.33:50182) 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75df7fe640 1 -- 172.21.15.33:0/3835131901 learned_addr learned my addr 172.21.15.33:0/3835131901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75dffff640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75e0102d70 0x7f75e0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75e49e7640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e010c140 0x7f75e019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75df7fe640 1 -- 172.21.15.33:0/3835131901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e010c140 msgr2=0x7f75e019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75df7fe640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e010c140 0x7f75e019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75df7fe640 1 -- 172.21.15.33:0/3835131901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75e0102d70 msgr2=0x7f75e0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75df7fe640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75e0102d70 0x7f75e0198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75df7fe640 1 -- 172.21.15.33:0/3835131901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e019d370 con 0x7f75e0103690 2024-04-02T13:48:03.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75dffff640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75e0102d70 0x7f75e0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:03.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75e49e7640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e010c140 0x7f75e019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:03.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75df7fe640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75e0103690 0x7f75e01988c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f75d000d950 tx=0x7f75d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75dd7fa640 1 -- 172.21.15.33:0/3835131901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0017070 con 0x7f75e0103690 2024-04-02T13:48:03.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75dd7fa640 1 -- 172.21.15.33:0/3835131901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f75d0009c30 con 0x7f75e0103690 2024-04-02T13:48:03.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75e644a640 1 -- 172.21.15.33:0/3835131901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e019d660 con 0x7f75e0103690 2024-04-02T13:48:03.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75dd7fa640 1 -- 172.21.15.33:0/3835131901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d000f6b0 con 0x7f75e0103690 2024-04-02T13:48:03.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.041+0000 7f75e644a640 1 -- 172.21.15.33:0/3835131901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e01a4ef0 con 0x7f75e0103690 2024-04-02T13:48:03.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.043+0000 7f75dd7fa640 1 -- 172.21.15.33:0/3835131901 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f75d0010a50 con 0x7f75e0103690 2024-04-02T13:48:03.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75e9035740 1 Processor -- start 2024-04-02T13:48:03.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75dd7fa640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f75c80a3d20 0x7f75c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75e9035740 1 -- start start 2024-04-02T13:48:03.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75dd7fa640 1 -- 172.21.15.33:0/3835131901 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f75d00d0740 con 0x7f75e0103690 2024-04-02T13:48:03.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75e49e7640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f75c80a3d20 0x7f75c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75e9035740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56e8d0 0x5562ef56ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75e9035740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56f1f0 0x5562ef577890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75e9035740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 0x5562ef57a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75e9035740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5562ef4d2830 con 0x5562ef56f1f0 2024-04-02T13:48:03.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75e9035740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5562ef47a820 con 0x5562ef56e8d0 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75e9035740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5562ef4ce7e0 con 0x5562ef577dd0 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56f1f0 0x5562ef577890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56f1f0 0x5562ef577890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50186/0 (socket says 172.21.15.33:50186) 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.044+0000 7f75df7fe640 1 -- 172.21.15.33:0/4063085293 learned_addr learned my addr 172.21.15.33:0/4063085293 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.045+0000 7f75dffff640 1 --2- 172.21.15.33:0/4063085293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56e8d0 0x5562ef56ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.045+0000 7f75e49e7640 1 --2- 172.21.15.33:0/4063085293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 0x5562ef57a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.045+0000 7f75df7fe640 1 -- 172.21.15.33:0/4063085293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 msgr2=0x5562ef57a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.045+0000 7f75df7fe640 1 --2- 172.21.15.33:0/4063085293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 0x5562ef57a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.045+0000 7f75df7fe640 1 -- 172.21.15.33:0/4063085293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56e8d0 msgr2=0x5562ef56ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.045+0000 7f75df7fe640 1 --2- 172.21.15.33:0/4063085293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56e8d0 0x5562ef56ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.045+0000 7f75df7fe640 1 -- 172.21.15.33:0/4063085293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562ef2b07d0 con 0x5562ef56f1f0 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.045+0000 7f75e49e7640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f75c80a3d20 0x7f75c80a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f75d8005e30 tx=0x7f75d8032480 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.045+0000 7f75df7fe640 1 --2- 172.21.15.33:0/4063085293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56f1f0 0x5562ef577890 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f75d00a4a50 tx=0x7f75d009ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7022a438a40e1d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.045+0000 7f75c67fc640 1 -- 172.21.15.33:0/4063085293 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0020040 con 0x5562ef56f1f0 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.045+0000 7f75c67fc640 1 -- 172.21.15.33:0/4063085293 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f75d00a06c0 con 0x5562ef56f1f0 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.045+0000 7f75c67fc640 1 -- 172.21.15.33:0/4063085293 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d00d0570 con 0x5562ef56f1f0 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.046+0000 7f75e9035740 1 -- 172.21.15.33:0/4063085293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56f1f0 msgr2=0x5562ef577890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.046+0000 7f75e9035740 1 --2- 172.21.15.33:0/4063085293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56f1f0 0x5562ef577890 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f75d00a4a50 tx=0x7f75d009ca10 comp rx=0 tx=0).stop 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.046+0000 7f75e9035740 1 -- 172.21.15.33:0/4063085293 shutdown_connections 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.046+0000 7f75e9035740 1 --2- 172.21.15.33:0/4063085293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 0x5562ef57a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.046+0000 7f75e9035740 1 --2- 172.21.15.33:0/4063085293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56f1f0 0x5562ef577890 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.046+0000 7f75e9035740 1 --2- 172.21.15.33:0/4063085293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56e8d0 0x5562ef56ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.046+0000 7f75e9035740 1 -- 172.21.15.33:0/4063085293 >> 172.21.15.33:0/4063085293 conn(0x5562ef569000 msgr2=0x5562ef5693e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.046+0000 7f75e9035740 1 -- 172.21.15.33:0/4063085293 shutdown_connections 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.046+0000 7f75e9035740 1 -- 172.21.15.33:0/4063085293 wait complete. 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.046+0000 7f75e9035740 1 -- start start 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75e9035740 1 Processor -- start 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75e9035740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 0x5562ef5776f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75e9035740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 0x5562ef574a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75e9035740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 0x5562ef574fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75e9035740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5562ef57ab60 con 0x5562ef56e8d0 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75e9035740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5562ef48dd30 con 0x5562ef56f1f0 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75e9035740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5562ef57ace0 con 0x5562ef577dd0 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75e9035740 10 client.0 _ll_register_callbacks cb 0x5562ef48d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75dffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 0x5562ef5776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75dffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 0x5562ef5776f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50202/0 (socket says 172.21.15.33:50202) 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75dffff640 1 -- 172.21.15.33:0/3282251145 learned_addr learned my addr 172.21.15.33:0/3282251145 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75e49e7640 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 0x5562ef574fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75df7fe640 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 0x5562ef574a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75dffff640 1 -- 172.21.15.33:0/3282251145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 msgr2=0x5562ef574fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75dffff640 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 0x5562ef574fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75dffff640 1 -- 172.21.15.33:0/3282251145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 msgr2=0x5562ef574a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75dffff640 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 0x5562ef574a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75dffff640 1 -- 172.21.15.33:0/3282251145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562ef1e63f0 con 0x5562ef56e8d0 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75e49e7640 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 0x5562ef574fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75df7fe640 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 0x5562ef574a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75dffff640 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 0x5562ef5776f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f75d4010f10 tx=0x7f75d400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b589202ca51cd9c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.047+0000 7f75bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75bf7fe640 1 -- 172.21.15.33:0/3282251145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d4005020 con 0x5562ef56e8d0 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75e9035740 1 -- 172.21.15.33:0/3282251145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75d00a06c0 con 0x5562ef56e8d0 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75bf7fe640 1 -- 172.21.15.33:0/3282251145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f75d40051c0 con 0x5562ef56e8d0 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75bf7fe640 1 -- 172.21.15.33:0/3282251145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f75d4024050 con 0x5562ef56e8d0 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75e9035740 10 client.17565 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75e9035740 1 -- 172.21.15.33:0/3282251145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5562ef1a60f0 con 0x5562ef56e8d0 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75bf7fe640 1 -- 172.21.15.33:0/3282251145 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f75d00a06c0 con 0x5562ef56e8d0 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75bf7fe640 1 -- 172.21.15.33:0/3282251145 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f75d40194a0 con 0x5562ef56e8d0 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75e9035740 10 client.17565 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75e9035740 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:03.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75e9035740 20 client.17565 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75e9035740 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5562ef587dc0 0x5562ef58a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75e9035740 4 client.17565 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75e9035740 1 -- 172.21.15.33:0/3282251145 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562ef4dfd70 con 0x5562ef587dc0 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.048+0000 7f75df7fe640 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5562ef587dc0 0x5562ef58a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.049+0000 7f75bf7fe640 10 client.17565 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.049+0000 7f75df7fe640 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5562ef587dc0 0x5562ef58a280 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee9cbe1c2e9b0efc server_cookie=95a8f25a36dcc2b8 in_seq=0 out_seq=0 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75bf7fe640 1 -- 172.21.15.33:0/3282251145 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5562ef4dfd70 con 0x5562ef587dc0 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75bf7fe640 10 client.17565 handle_command_reply: tid=0 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75e9035740 1 client.17565 shutdown 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75e9035740 1 -- 172.21.15.33:0/3282251145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 msgr2=0x5562ef5776f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75e9035740 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 0x5562ef5776f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f75d4010f10 tx=0x7f75d400f7f0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75e9035740 1 -- 172.21.15.33:0/3282251145 shutdown_connections 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75e9035740 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5562ef587dc0 0x5562ef58a280 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75e9035740 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 0x5562ef574fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75bf7fe640 0 client.17565 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:03.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75e9035740 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 0x5562ef574a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75e9035740 1 --2- 172.21.15.33:0/3282251145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 0x5562ef5776f0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75e9035740 1 -- 172.21.15.33:0/3282251145 >> 172.21.15.33:0/3282251145 conn(0x5562ef569000 msgr2=0x5562ef56ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75e9035740 1 -- 172.21.15.33:0/3282251145 shutdown_connections 2024-04-02T13:48:03.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75e9035740 1 -- 172.21.15.33:0/3282251145 wait complete. 2024-04-02T13:48:03.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.050+0000 7f75e9035740 20 client.17565 trim_cache size 0 max 16384 2024-04-02T13:48:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75e9035740 1 Processor -- start 2024-04-02T13:48:03.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75e9035740 1 -- start start 2024-04-02T13:48:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75e9035740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 0x5562ef56f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75e9035740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 0x5562ef574360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75e9035740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 0x5562ef5771b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75e9035740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01b1930 con 0x5562ef56e8d0 2024-04-02T13:48:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75e9035740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01af200 con 0x5562ef56f1f0 2024-04-02T13:48:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75e9035740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01ad070 con 0x5562ef577dd0 2024-04-02T13:48:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75dffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 0x5562ef56f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75dffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 0x5562ef56f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50218/0 (socket says 172.21.15.33:50218) 2024-04-02T13:48:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75dffff640 1 -- 172.21.15.33:0/1615855531 learned_addr learned my addr 172.21.15.33:0/1615855531 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75df7fe640 1 --2- 172.21.15.33:0/1615855531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 0x5562ef574360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75e49e7640 1 --2- 172.21.15.33:0/1615855531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 0x5562ef5771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75dffff640 1 -- 172.21.15.33:0/1615855531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 msgr2=0x5562ef5771b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75dffff640 1 --2- 172.21.15.33:0/1615855531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 0x5562ef5771b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75dffff640 1 -- 172.21.15.33:0/1615855531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 msgr2=0x5562ef574360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75dffff640 1 --2- 172.21.15.33:0/1615855531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 0x5562ef574360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.068+0000 7f75dffff640 1 -- 172.21.15.33:0/1615855531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562ef5869c0 con 0x5562ef56e8d0 2024-04-02T13:48:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.069+0000 7f75dffff640 1 --2- 172.21.15.33:0/1615855531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 0x5562ef56f830 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f75d4000c00 tx=0x7f75d4019d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf1bbf8111475d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.069+0000 7f75c5ffb640 1 -- 172.21.15.33:0/1615855531 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d402d070 con 0x5562ef56e8d0 2024-04-02T13:48:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.069+0000 7f75c5ffb640 1 -- 172.21.15.33:0/1615855531 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f75d4005540 con 0x5562ef56e8d0 2024-04-02T13:48:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.069+0000 7f75c5ffb640 1 -- 172.21.15.33:0/1615855531 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d4004ac0 con 0x5562ef56e8d0 2024-04-02T13:48:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.069+0000 7f75e9035740 1 -- 172.21.15.33:0/1615855531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 msgr2=0x5562ef56f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.069+0000 7f75e9035740 1 --2- 172.21.15.33:0/1615855531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 0x5562ef56f830 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f75d4000c00 tx=0x7f75d4019d40 comp rx=0 tx=0).stop 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.069+0000 7f75e9035740 1 -- 172.21.15.33:0/1615855531 shutdown_connections 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.069+0000 7f75e9035740 1 --2- 172.21.15.33:0/1615855531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef577dd0 0x5562ef5771b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.069+0000 7f75e9035740 1 --2- 172.21.15.33:0/1615855531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 0x5562ef574360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.069+0000 7f75e9035740 1 --2- 172.21.15.33:0/1615855531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef56e8d0 0x5562ef56f830 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.069+0000 7f75e9035740 1 -- 172.21.15.33:0/1615855531 >> 172.21.15.33:0/1615855531 conn(0x5562ef569000 msgr2=0x5562ef568240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.069+0000 7f75e9035740 1 -- 172.21.15.33:0/1615855531 shutdown_connections 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.069+0000 7f75e9035740 1 -- 172.21.15.33:0/1615855531 wait complete. 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75e9035740 1 -- start start 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75e9035740 1 Processor -- start 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75e9035740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef56e8d0 0x5562ef580200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75e9035740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 0x5562ef571390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75e9035740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef577dd0 0x5562ef5718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75e9035740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01ad6c0 con 0x5562ef577dd0 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75e9035740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01aeb30 con 0x5562ef56f1f0 2024-04-02T13:48:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75e9035740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01add90 con 0x5562ef56e8d0 2024-04-02T13:48:03.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75e9035740 10 client.0 _ll_register_callbacks cb 0x7f75d4010b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:03.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75e49e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef577dd0 0x5562ef5718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75e49e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef577dd0 0x5562ef5718d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50222/0 (socket says 172.21.15.33:50222) 2024-04-02T13:48:03.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75e49e7640 1 -- 172.21.15.33:0/3887424213 learned_addr learned my addr 172.21.15.33:0/3887424213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75df7fe640 1 --2- 172.21.15.33:0/3887424213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 0x5562ef571390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.070+0000 7f75dffff640 1 --2- 172.21.15.33:0/3887424213 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef56e8d0 0x5562ef580200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75e49e7640 1 -- 172.21.15.33:0/3887424213 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef56e8d0 msgr2=0x5562ef580200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75e49e7640 1 --2- 172.21.15.33:0/3887424213 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef56e8d0 0x5562ef580200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75e49e7640 1 -- 172.21.15.33:0/3887424213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 msgr2=0x5562ef571390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75e49e7640 1 --2- 172.21.15.33:0/3887424213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 0x5562ef571390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75e49e7640 1 -- 172.21.15.33:0/3887424213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562ef57a1c0 con 0x5562ef577dd0 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75e49e7640 1 --2- 172.21.15.33:0/3887424213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef577dd0 0x5562ef5718d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f75d802c790 tx=0x7f75d802c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264bda0fe283a18f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75bf7fe640 1 -- 172.21.15.33:0/3887424213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8050070 con 0x5562ef577dd0 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75bf7fe640 1 -- 172.21.15.33:0/3887424213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f75d8034850 con 0x5562ef577dd0 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75e9035740 1 -- 172.21.15.33:0/3887424213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75d4005540 con 0x5562ef577dd0 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75bf7fe640 1 -- 172.21.15.33:0/3887424213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f75d8047070 con 0x5562ef577dd0 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75e9035740 10 client.17571 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75e9035740 1 -- 172.21.15.33:0/3887424213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5562ef5ee0e0 con 0x5562ef577dd0 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75bf7fe640 1 -- 172.21.15.33:0/3887424213 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f75d4005540 con 0x5562ef577dd0 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.071+0000 7f75bf7fe640 1 -- 172.21.15.33:0/3887424213 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f75d803f040 con 0x5562ef577dd0 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.072+0000 7f75e9035740 10 client.17571 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.072+0000 7f75e9035740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.072+0000 7f75e9035740 20 client.17571 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:03.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.072+0000 7f75e9035740 1 --2- 172.21.15.33:0/3887424213 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5562ef5ee860 0x5562ef5f0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.072+0000 7f75e9035740 4 client.17571 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.072+0000 7f75e9035740 1 -- 172.21.15.33:0/3887424213 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5562ef4da0c0 con 0x5562ef5ee860 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.072+0000 7f75dffff640 1 --2- 172.21.15.33:0/3887424213 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5562ef5ee860 0x5562ef5f0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.072+0000 7f75dffff640 1 --2- 172.21.15.33:0/3887424213 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5562ef5ee860 0x5562ef5f0c80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46d17318a6721dd2 server_cookie=b0c627b57f9c1d7b in_seq=0 out_seq=0 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.072+0000 7f75bf7fe640 10 client.17571 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.072+0000 7f75bf7fe640 1 -- 172.21.15.33:0/3887424213 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5562ef4da0c0 con 0x5562ef5ee860 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.072+0000 7f75bf7fe640 10 client.17571 handle_command_reply: tid=0 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.072+0000 7f75e9035740 1 client.17571 shutdown 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e9035740 1 -- 172.21.15.33:0/3887424213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef577dd0 msgr2=0x5562ef5718d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e9035740 1 --2- 172.21.15.33:0/3887424213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef577dd0 0x5562ef5718d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f75d802c790 tx=0x7f75d802c180 comp rx=0 tx=0).stop 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e9035740 1 -- 172.21.15.33:0/3887424213 shutdown_connections 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e9035740 1 --2- 172.21.15.33:0/3887424213 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5562ef5ee860 0x5562ef5f0c80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e9035740 1 --2- 172.21.15.33:0/3887424213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562ef577dd0 0x5562ef5718d0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e9035740 1 --2- 172.21.15.33:0/3887424213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562ef56f1f0 0x5562ef571390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e9035740 1 --2- 172.21.15.33:0/3887424213 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5562ef56e8d0 0x5562ef580200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75bf7fe640 0 client.17571 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:03.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e9035740 1 -- 172.21.15.33:0/3887424213 >> 172.21.15.33:0/3887424213 conn(0x5562ef569000 msgr2=0x5562ef568240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e9035740 1 -- 172.21.15.33:0/3887424213 shutdown_connections 2024-04-02T13:48:03.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e9035740 1 -- 172.21.15.33:0/3887424213 wait complete. 2024-04-02T13:48:03.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e9035740 20 client.17571 trim_cache size 0 max 16384 2024-04-02T13:48:03.084 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:48:03.084 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:48:03.084 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:48:03.085 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:48:03.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e644a640 1 -- 172.21.15.33:0/3835131901 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f75c80a3d20 msgr2=0x7f75c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e644a640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f75c80a3d20 0x7f75c80a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f75d8005e30 tx=0x7f75d8032480 comp rx=0 tx=0).stop 2024-04-02T13:48:03.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e644a640 1 -- 172.21.15.33:0/3835131901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75e0103690 msgr2=0x7f75e01988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e644a640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75e0103690 0x7f75e01988c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f75d000d950 tx=0x7f75d000de10 comp rx=0 tx=0).stop 2024-04-02T13:48:03.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e644a640 1 -- 172.21.15.33:0/3835131901 shutdown_connections 2024-04-02T13:48:03.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e644a640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f75c80a3d20 0x7f75c80a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e644a640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75e010c140 0x7f75e019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e644a640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75e0103690 0x7f75e01988c0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e644a640 1 --2- 172.21.15.33:0/3835131901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f75e0102d70 0x7f75e0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e644a640 1 -- 172.21.15.33:0/3835131901 >> 172.21.15.33:0/3835131901 conn(0x7f75e0077b60 msgr2=0x7f75e00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e644a640 1 -- 172.21.15.33:0/3835131901 shutdown_connections 2024-04-02T13:48:03.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.073+0000 7f75e644a640 1 -- 172.21.15.33:0/3835131901 wait complete. 2024-04-02T13:48:03.087 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:48:03.087 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:48:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.229+0000 7f18c8f8f640 1 -- 172.21.15.33:0/801577543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18c41069c0 msgr2=0x7f18c4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.229+0000 7f18c8f8f640 1 --2- 172.21.15.33:0/801577543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18c41069c0 0x7f18c4106da0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009930 tx=0x7f18b802bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.229+0000 7f18c8f8f640 1 -- 172.21.15.33:0/801577543 shutdown_connections 2024-04-02T13:48:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.229+0000 7f18c8f8f640 1 --2- 172.21.15.33:0/801577543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18c41013c0 0x7f18c410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.229+0000 7f18c8f8f640 1 --2- 172.21.15.33:0/801577543 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18c4100a00 0x7f18c4100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.229+0000 7f18c8f8f640 1 --2- 172.21.15.33:0/801577543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18c41069c0 0x7f18c4106da0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.229+0000 7f18c8f8f640 1 -- 172.21.15.33:0/801577543 >> 172.21.15.33:0/801577543 conn(0x7f18c4077b60 msgr2=0x7f18c4077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.229+0000 7f18c8f8f640 1 -- 172.21.15.33:0/801577543 shutdown_connections 2024-04-02T13:48:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.229+0000 7f18c8f8f640 1 -- 172.21.15.33:0/801577543 wait complete. 2024-04-02T13:48:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c8f8f640 1 Processor -- start 2024-04-02T13:48:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c8f8f640 1 -- start start 2024-04-02T13:48:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c8f8f640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18c4100a00 0x7f18c4197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c8f8f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18c41013c0 0x7f18c4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c8f8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18c41069c0 0x7f18c419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c8f8f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4110d70 con 0x7f18c41069c0 2024-04-02T13:48:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c8f8f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4110bf0 con 0x7f18c41013c0 2024-04-02T13:48:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c8f8f640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4110ef0 con 0x7f18c4100a00 2024-04-02T13:48:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18c41069c0 0x7f18c419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18c41069c0 0x7f18c419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50238/0 (socket says 172.21.15.33:50238) 2024-04-02T13:48:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c2d9d640 1 -- 172.21.15.33:0/3063114035 learned_addr learned my addr 172.21.15.33:0/3063114035 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c1d9b640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18c41013c0 0x7f18c4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c259c640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18c4100a00 0x7f18c4197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c2d9d640 1 -- 172.21.15.33:0/3063114035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18c4100a00 msgr2=0x7f18c4197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c2d9d640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18c4100a00 0x7f18c4197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c2d9d640 1 -- 172.21.15.33:0/3063114035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18c41013c0 msgr2=0x7f18c4197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c2d9d640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18c41013c0 0x7f18c4197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c2d9d640 1 -- 172.21.15.33:0/3063114035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c419c300 con 0x7f18c41069c0 2024-04-02T13:48:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c259c640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18c4100a00 0x7f18c4197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.230+0000 7f18c2d9d640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18c41069c0 0x7f18c419bbd0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00eaa0 tx=0x7f18bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.231+0000 7f18b37fe640 1 -- 172.21.15.33:0/3063114035 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc00cdb0 con 0x7f18c41069c0 2024-04-02T13:48:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.231+0000 7f18b37fe640 1 -- 172.21.15.33:0/3063114035 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f18bc005020 con 0x7f18c41069c0 2024-04-02T13:48:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.231+0000 7f18c8f8f640 1 -- 172.21.15.33:0/3063114035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c419c5f0 con 0x7f18c41069c0 2024-04-02T13:48:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.231+0000 7f18b37fe640 1 -- 172.21.15.33:0/3063114035 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0116b0 con 0x7f18c41069c0 2024-04-02T13:48:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.231+0000 7f18c8f8f640 1 -- 172.21.15.33:0/3063114035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c41a3e80 con 0x7f18c41069c0 2024-04-02T13:48:03.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.232+0000 7f18c8f8f640 1 -- 172.21.15.33:0/3063114035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c4068d40 con 0x7f18c41069c0 2024-04-02T13:48:03.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.233+0000 7f18b37fe640 1 -- 172.21.15.33:0/3063114035 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18bc0051d0 con 0x7f18c41069c0 2024-04-02T13:48:03.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.233+0000 7f18b37fe640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f18ac0a3d20 0x7f18ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.233+0000 7f18b37fe640 1 -- 172.21.15.33:0/3063114035 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f18bc0d1170 con 0x7f18c41069c0 2024-04-02T13:48:03.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.233+0000 7f18c259c640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f18ac0a3d20 0x7f18ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.234+0000 7f18c259c640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f18ac0a3d20 0x7f18ac0a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f18b8036040 tx=0x7f18b8005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.237+0000 7f18b37fe640 1 -- 172.21.15.33:0/3063114035 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f18bc016030 con 0x7f18c41069c0 2024-04-02T13:48:03.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.462+0000 7f18c8f8f640 1 -- 172.21.15.33:0/3063114035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18c4108f80 con 0x7f18c41069c0 2024-04-02T13:48:03.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.462+0000 7f18b37fe640 1 -- 172.21.15.33:0/3063114035 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11873 (secure 0 0 0) 0x7f18bc09d3e0 con 0x7f18c41069c0 2024-04-02T13:48:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:48:03.465 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:44:55.702357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:48:03.466 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 36 2024-04-02T13:48:03.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.464+0000 7f18c8f8f640 1 -- 172.21.15.33:0/3063114035 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f18ac0a3d20 msgr2=0x7f18ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.464+0000 7f18c8f8f640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f18ac0a3d20 0x7f18ac0a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f18b8036040 tx=0x7f18b8005fd0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.464+0000 7f18c8f8f640 1 -- 172.21.15.33:0/3063114035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18c41069c0 msgr2=0x7f18c419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.464+0000 7f18c8f8f640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18c41069c0 0x7f18c419bbd0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00eaa0 tx=0x7f18bc00ef60 comp rx=0 tx=0).stop 2024-04-02T13:48:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.464+0000 7f18c8f8f640 1 -- 172.21.15.33:0/3063114035 shutdown_connections 2024-04-02T13:48:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.464+0000 7f18c8f8f640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f18ac0a3d20 0x7f18ac0a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.464+0000 7f18c8f8f640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18c41069c0 0x7f18c419bbd0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.464+0000 7f18c8f8f640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f18c41013c0 0x7f18c4197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.464+0000 7f18c8f8f640 1 --2- 172.21.15.33:0/3063114035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18c4100a00 0x7f18c4197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.464+0000 7f18c8f8f640 1 -- 172.21.15.33:0/3063114035 >> 172.21.15.33:0/3063114035 conn(0x7f18c4077b60 msgr2=0x7f18c4103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.465+0000 7f18c8f8f640 1 -- 172.21.15.33:0/3063114035 shutdown_connections 2024-04-02T13:48:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.465+0000 7f18c8f8f640 1 -- 172.21.15.33:0/3063114035 wait complete. 2024-04-02T13:48:03.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:48:03.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.579+0000 7fc4b110e640 1 -- 172.21.15.33:0/3391051746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac106980 msgr2=0x7fc4ac106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.579+0000 7fc4b110e640 1 --2- 172.21.15.33:0/3391051746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac106980 0x7fc4ac106d60 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009930 tx=0x7fc4a402bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.579+0000 7fc4b110e640 1 -- 172.21.15.33:0/3391051746 shutdown_connections 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.579+0000 7fc4b110e640 1 --2- 172.21.15.33:0/3391051746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ac101330 0x7fc4ac10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.579+0000 7fc4b110e640 1 --2- 172.21.15.33:0/3391051746 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ac100970 0x7fc4ac100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.579+0000 7fc4b110e640 1 --2- 172.21.15.33:0/3391051746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac106980 0x7fc4ac106d60 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.579+0000 7fc4b110e640 1 -- 172.21.15.33:0/3391051746 >> 172.21.15.33:0/3391051746 conn(0x7fc4ac077b00 msgr2=0x7fc4ac077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.579+0000 7fc4b110e640 1 -- 172.21.15.33:0/3391051746 shutdown_connections 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.579+0000 7fc4b110e640 1 -- 172.21.15.33:0/3391051746 wait complete. 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.580+0000 7fc4b110e640 1 Processor -- start 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.580+0000 7fc4b110e640 1 -- start start 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.580+0000 7fc4b110e640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ac100970 0x7fc4ac198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.580+0000 7fc4b110e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ac101330 0x7fc4ac1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.580+0000 7fc4b110e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac106980 0x7fc4ac19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4b110e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac111e60 con 0x7fc4ac106980 2024-04-02T13:48:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4b110e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac111ce0 con 0x7fc4ac101330 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4b110e640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac111fe0 con 0x7fc4ac100970 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4ab59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac106980 0x7fc4ac19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4ab59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac106980 0x7fc4ac19cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37748/0 (socket says 172.21.15.33:37748) 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4ab59e640 1 -- 172.21.15.33:0/105433101 learned_addr learned my addr 172.21.15.33:0/105433101 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ac101330 0x7fc4ac1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ac100970 0x7fc4ac198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4ab59e640 1 -- 172.21.15.33:0/105433101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ac100970 msgr2=0x7fc4ac198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4ab59e640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ac100970 0x7fc4ac198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4ab59e640 1 -- 172.21.15.33:0/105433101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ac101330 msgr2=0x7fc4ac1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4ab59e640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ac101330 0x7fc4ac1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4ab59e640 1 -- 172.21.15.33:0/105433101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ac19d320 con 0x7fc4ac106980 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4ab59e640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac106980 0x7fc4ac19cbf0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00cce0 tx=0x7fc49c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4a3fff640 1 -- 172.21.15.33:0/105433101 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00f450 con 0x7fc4ac106980 2024-04-02T13:48:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4a3fff640 1 -- 172.21.15.33:0/105433101 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc49c00fac0 con 0x7fc4ac106980 2024-04-02T13:48:03.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4b110e640 1 -- 172.21.15.33:0/105433101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ac19d610 con 0x7fc4ac106980 2024-04-02T13:48:03.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4a3fff640 1 -- 172.21.15.33:0/105433101 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c018be0 con 0x7fc4ac106980 2024-04-02T13:48:03.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.581+0000 7fc4b110e640 1 -- 172.21.15.33:0/105433101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ac1a4ea0 con 0x7fc4ac106980 2024-04-02T13:48:03.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.583+0000 7fc4a3fff640 1 -- 172.21.15.33:0/105433101 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc49c00fc70 con 0x7fc4ac106980 2024-04-02T13:48:03.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4b3dd4740 1 Processor -- start 2024-04-02T13:48:03.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4b3dd4740 1 -- start start 2024-04-02T13:48:03.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4b3dd4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 0x556d89f2eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4b3dd4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2f0c0 0x556d89f37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4b3dd4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f37ca0 0x556d89f3a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4b3dd4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d89e927b0 con 0x556d89f2e7a0 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4b3dd4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556d89e31570 con 0x556d89f37ca0 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4b3dd4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556d89e8e760 con 0x556d89f2f0c0 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4a3fff640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc4980a3d20 0x7fc4980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 0x556d89f2eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 0x556d89f2eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37754/0 (socket says 172.21.15.33:37754) 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4aa59c640 1 -- 172.21.15.33:0/1265516167 learned_addr learned my addr 172.21.15.33:0/1265516167 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4ab59e640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc4980a3d20 0x7fc4980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4a3fff640 1 -- 172.21.15.33:0/105433101 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fc49c0d1260 con 0x7fc4ac106980 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/1265516167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2f0c0 0x556d89f37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.584+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/1265516167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f37ca0 0x556d89f3a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.585+0000 7fc4aa59c640 1 -- 172.21.15.33:0/1265516167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2f0c0 msgr2=0x556d89f37760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.585+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/1265516167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2f0c0 0x556d89f37760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.585+0000 7fc4aa59c640 1 -- 172.21.15.33:0/1265516167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f37ca0 msgr2=0x556d89f3a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.585+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/1265516167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f37ca0 0x556d89f3a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.585+0000 7fc4aa59c640 1 -- 172.21.15.33:0/1265516167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d89ba63f0 con 0x556d89f2e7a0 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.585+0000 7fc4ab59e640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc4980a3d20 0x7fc4980a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0cebe0 tx=0x7fc49c01a810 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.585+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/1265516167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 0x556d89f2eb80 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc49400bc20 tx=0x7fc494004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b229eb16f36d7d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.585+0000 7fc4a17fa640 1 -- 172.21.15.33:0/1265516167 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494005020 con 0x556d89f2e7a0 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.585+0000 7fc4a17fa640 1 -- 172.21.15.33:0/1265516167 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc494005690 con 0x556d89f2e7a0 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.585+0000 7fc4a17fa640 1 -- 172.21.15.33:0/1265516167 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940106c0 con 0x556d89f2e7a0 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/1265516167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 msgr2=0x556d89f2eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/1265516167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 0x556d89f2eb80 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc49400bc20 tx=0x7fc494004250 comp rx=0 tx=0).stop 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/1265516167 shutdown_connections 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/1265516167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f37ca0 0x556d89f3a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/1265516167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2f0c0 0x556d89f37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/1265516167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 0x556d89f2eb80 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/1265516167 >> 172.21.15.33:0/1265516167 conn(0x556d89f28ed0 msgr2=0x556d89f292b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/1265516167 shutdown_connections 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/1265516167 wait complete. 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 -- start start 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 Processor -- start 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 0x556d89f37670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2f0c0 0x556d89f34a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f37ca0 0x556d89f35010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d89f3aa30 con 0x556d89f2e7a0 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556d89e4de80 con 0x556d89f37ca0 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x556d89f3abb0 con 0x556d89f2f0c0 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4b3dd4740 10 client.0 _ll_register_callbacks cb 0x556d89e89aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.586+0000 7fc4aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 0x556d89f37670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 0x556d89f37670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37758/0 (socket says 172.21.15.33:37758) 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4aa59c640 1 -- 172.21.15.33:0/759034882 learned_addr learned my addr 172.21.15.33:0/759034882 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/759034882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2f0c0 0x556d89f34a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/759034882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f37ca0 0x556d89f35010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4aa59c640 1 -- 172.21.15.33:0/759034882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2f0c0 msgr2=0x556d89f34a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/759034882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2f0c0 0x556d89f34a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4aa59c640 1 -- 172.21.15.33:0/759034882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f37ca0 msgr2=0x556d89f35010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/759034882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f37ca0 0x556d89f35010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4aa59c640 1 -- 172.21.15.33:0/759034882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d89c707d0 con 0x556d89f2e7a0 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/759034882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 0x556d89f37670 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc49400b3f0 tx=0x7fc49400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f52ea52dbf8e30 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4927fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4927fc640 1 -- 172.21.15.33:0/759034882 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494005130 con 0x556d89f2e7a0 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4927fc640 1 -- 172.21.15.33:0/759034882 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fc4940052d0 con 0x556d89f2e7a0 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/759034882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d89e9fcf0 con 0x556d89f2e7a0 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4927fc640 1 -- 172.21.15.33:0/759034882 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc4940054d0 con 0x556d89f2e7a0 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4b3dd4740 10 client.17589 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/759034882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556d89f46a00 con 0x556d89f2e7a0 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.587+0000 7fc4927fc640 1 -- 172.21.15.33:0/759034882 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x556d89e9fcf0 con 0x556d89f2e7a0 2024-04-02T13:48:03.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.588+0000 7fc4927fc640 1 -- 172.21.15.33:0/759034882 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fc494021520 con 0x556d89f2e7a0 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.588+0000 7fc4b3dd4740 10 client.17589 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.588+0000 7fc4b3dd4740 10 client.17589 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.588+0000 7fc4b3dd4740 20 client.17589 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.588+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/759034882 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x556d89f47e00 0x556d89f4a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.588+0000 7fc4b3dd4740 4 client.17589 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.588+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/759034882 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d89e9a040 con 0x556d89f47e00 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.588+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/759034882 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x556d89f47e00 0x556d89f4a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.588+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/759034882 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x556d89f47e00 0x556d89f4a1f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d3a1c3aaf3fc048 server_cookie=45a615489ee78e7f in_seq=0 out_seq=0 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.588+0000 7fc4927fc640 10 client.17589 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4927fc640 1 -- 172.21.15.33:0/759034882 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d89e9a040 con 0x556d89f47e00 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4927fc640 10 client.17589 handle_command_reply: tid=0 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4b3dd4740 1 client.17589 shutdown 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/759034882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 msgr2=0x556d89f37670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/759034882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 0x556d89f37670 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc49400b3f0 tx=0x7fc49400b5b0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/759034882 shutdown_connections 2024-04-02T13:48:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/759034882 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x556d89f47e00 0x556d89f4a1f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/759034882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f37ca0 0x556d89f35010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/759034882 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2f0c0 0x556d89f34a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/759034882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2e7a0 0x556d89f37670 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4927fc640 0 client.17589 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:48:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/759034882 >> 172.21.15.33:0/759034882 conn(0x556d89f28ed0 msgr2=0x556d89f2ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/759034882 shutdown_connections 2024-04-02T13:48:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/759034882 wait complete. 2024-04-02T13:48:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.589+0000 7fc4b3dd4740 20 client.17589 trim_cache size 0 max 16384 2024-04-02T13:48:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4b3dd4740 1 Processor -- start 2024-04-02T13:48:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4b3dd4740 1 -- start start 2024-04-02T13:48:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4b3dd4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f2e7a0 0x556d89f342e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4b3dd4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2f0c0 0x556d89f37130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4b3dd4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f37ca0 0x556d89f37670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4b3dd4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac1b18c0 con 0x556d89f2f0c0 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4b3dd4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac1af190 con 0x556d89f2e7a0 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4b3dd4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac1ad000 con 0x556d89f37ca0 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4aad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2f0c0 0x556d89f37130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4aad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2f0c0 0x556d89f37130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37770/0 (socket says 172.21.15.33:37770) 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4aad9d640 1 -- 172.21.15.33:0/2131172475 learned_addr learned my addr 172.21.15.33:0/2131172475 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/2131172475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f2e7a0 0x556d89f342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4aad9d640 1 -- 172.21.15.33:0/2131172475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f37ca0 msgr2=0x556d89f37670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/2131172475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f37ca0 0x556d89f37670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4aad9d640 1 -- 172.21.15.33:0/2131172475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f2e7a0 msgr2=0x556d89f342e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/2131172475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f2e7a0 0x556d89f342e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.603+0000 7fc4aad9d640 1 -- 172.21.15.33:0/2131172475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d89f35380 con 0x556d89f2f0c0 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.604+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/2131172475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2f0c0 0x556d89f37130 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009930 tx=0x7fc4a4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb49b5dfdb6406a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.604+0000 7fc4a0ff9640 1 -- 172.21.15.33:0/2131172475 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4005240 con 0x556d89f2f0c0 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.604+0000 7fc4a0ff9640 1 -- 172.21.15.33:0/2131172475 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc4a4004490 con 0x556d89f2f0c0 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.604+0000 7fc4a0ff9640 1 -- 172.21.15.33:0/2131172475 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4004a40 con 0x556d89f2f0c0 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.604+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/2131172475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2f0c0 msgr2=0x556d89f37130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.604+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/2131172475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2f0c0 0x556d89f37130 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009930 tx=0x7fc4a4009260 comp rx=0 tx=0).stop 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.604+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/2131172475 shutdown_connections 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.604+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/2131172475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f37ca0 0x556d89f37670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.604+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/2131172475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f2f0c0 0x556d89f37130 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.604+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/2131172475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f2e7a0 0x556d89f342e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.604+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/2131172475 >> 172.21.15.33:0/2131172475 conn(0x556d89f28ed0 msgr2=0x556d89f276a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.604+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/2131172475 shutdown_connections 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.604+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/2131172475 wait complete. 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4b3dd4740 1 -- start start 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4b3dd4740 1 Processor -- start 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4b3dd4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2e7a0 0x556d89f38260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4b3dd4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f2f0c0 0x556d89f387a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4b3dd4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f37ca0 0x556d89f38ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4b3dd4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac1ad650 con 0x556d89f37ca0 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4b3dd4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac1aeac0 con 0x556d89f2f0c0 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4b3dd4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac1add20 con 0x556d89f2e7a0 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4b3dd4740 10 client.0 _ll_register_callbacks cb 0x7fc49400ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f37ca0 0x556d89f38ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f37ca0 0x556d89f38ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37786/0 (socket says 172.21.15.33:37786) 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4aa59c640 1 -- 172.21.15.33:0/197417030 learned_addr learned my addr 172.21.15.33:0/197417030 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/197417030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f2f0c0 0x556d89f387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4aa59c640 1 -- 172.21.15.33:0/197417030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2e7a0 msgr2=0x556d89f38260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/197417030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2e7a0 0x556d89f38260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4aa59c640 1 -- 172.21.15.33:0/197417030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f2f0c0 msgr2=0x556d89f387a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/197417030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f2f0c0 0x556d89f387a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4aa59c640 1 -- 172.21.15.33:0/197417030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d89f3f080 con 0x556d89f37ca0 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/197417030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f2f0c0 0x556d89f387a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.605+0000 7fc4aa59c640 1 --2- 172.21.15.33:0/197417030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f37ca0 0x556d89f38ce0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc4940219c0 tx=0x7fc49400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae7b9b37b0544f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4927fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4927fc640 1 -- 172.21.15.33:0/197417030 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494004940 con 0x556d89f37ca0 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4927fc640 1 -- 172.21.15.33:0/197417030 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fc494021400 con 0x556d89f37ca0 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/197417030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a4004490 con 0x556d89f37ca0 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4927fc640 1 -- 172.21.15.33:0/197417030 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc49403e050 con 0x556d89f37ca0 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4b3dd4740 10 client.17595 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/197417030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556d89f40380 con 0x556d89f37ca0 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4927fc640 1 -- 172.21.15.33:0/197417030 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc4a4004490 con 0x556d89f37ca0 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4927fc640 1 -- 172.21.15.33:0/197417030 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fc494010460 con 0x556d89f37ca0 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4b3dd4740 10 client.17595 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4b3dd4740 10 client.17595 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4b3dd4740 20 client.17595 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/197417030 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x556d89fae380 0x556d89fb0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4b3dd4740 4 client.17595 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/197417030 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d89e176b0 con 0x556d89fae380 2024-04-02T13:48:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.606+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/197417030 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x556d89fae380 0x556d89fb0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4aad9d640 1 --2- 172.21.15.33:0/197417030 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x556d89fae380 0x556d89fb0770 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c9fc0ad85c6d9aa server_cookie=fcf81b9695da1445 in_seq=0 out_seq=0 2024-04-02T13:48:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4927fc640 10 client.17595 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:48:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4927fc640 1 -- 172.21.15.33:0/197417030 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d89e176b0 con 0x556d89fae380 2024-04-02T13:48:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4927fc640 10 client.17595 handle_command_reply: tid=0 2024-04-02T13:48:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4b3dd4740 1 client.17595 shutdown 2024-04-02T13:48:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/197417030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f37ca0 msgr2=0x556d89f38ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/197417030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f37ca0 0x556d89f38ce0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc4940219c0 tx=0x7fc49400bf40 comp rx=0 tx=0).stop 2024-04-02T13:48:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/197417030 shutdown_connections 2024-04-02T13:48:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/197417030 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x556d89fae380 0x556d89fb0770 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/197417030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d89f37ca0 0x556d89f38ce0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/197417030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d89f2f0c0 0x556d89f387a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4927fc640 0 client.17595 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:48:03.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4b3dd4740 1 --2- 172.21.15.33:0/197417030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556d89f2e7a0 0x556d89f38260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/197417030 >> 172.21.15.33:0/197417030 conn(0x556d89f28ed0 msgr2=0x556d89f2a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/197417030 shutdown_connections 2024-04-02T13:48:03.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.607+0000 7fc4b3dd4740 1 -- 172.21.15.33:0/197417030 wait complete. 2024-04-02T13:48:03.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.608+0000 7fc4b3dd4740 20 client.17595 trim_cache size 0 max 16384 2024-04-02T13:48:03.617 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:48:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.608+0000 7fc4b110e640 1 -- 172.21.15.33:0/105433101 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc4980a3d20 msgr2=0x7fc4980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.608+0000 7fc4b110e640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc4980a3d20 0x7fc4980a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0cebe0 tx=0x7fc49c01a810 comp rx=0 tx=0).stop 2024-04-02T13:48:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.608+0000 7fc4b110e640 1 -- 172.21.15.33:0/105433101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac106980 msgr2=0x7fc4ac19cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.608+0000 7fc4b110e640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac106980 0x7fc4ac19cbf0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00cce0 tx=0x7fc49c00a620 comp rx=0 tx=0).stop 2024-04-02T13:48:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.608+0000 7fc4b110e640 1 -- 172.21.15.33:0/105433101 shutdown_connections 2024-04-02T13:48:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.608+0000 7fc4b110e640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc4980a3d20 0x7fc4980a61e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.608+0000 7fc4b110e640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac106980 0x7fc4ac19cbf0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.608+0000 7fc4b110e640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4ac101330 0x7fc4ac1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.608+0000 7fc4b110e640 1 --2- 172.21.15.33:0/105433101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc4ac100970 0x7fc4ac198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.608+0000 7fc4b110e640 1 -- 172.21.15.33:0/105433101 >> 172.21.15.33:0/105433101 conn(0x7fc4ac077b00 msgr2=0x7fc4ac1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.608+0000 7fc4b110e640 1 -- 172.21.15.33:0/105433101 shutdown_connections 2024-04-02T13:48:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.608+0000 7fc4b110e640 1 -- 172.21.15.33:0/105433101 wait complete. 2024-04-02T13:48:03.621 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:48:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.724+0000 7f99d6300640 1 -- 172.21.15.33:0/1104416595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99d0106e80 msgr2=0x7f99d010ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.724+0000 7f99d6300640 1 --2- 172.21.15.33:0/1104416595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99d0106e80 0x7f99d010ea00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f99c8009ca0 tx=0x7f99c802c100 comp rx=0 tx=0).stop 2024-04-02T13:48:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.724+0000 7f99d6300640 1 -- 172.21.15.33:0/1104416595 shutdown_connections 2024-04-02T13:48:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.724+0000 7f99d6300640 1 --2- 172.21.15.33:0/1104416595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99d0106e80 0x7f99d010ea00 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.724+0000 7f99d6300640 1 --2- 172.21.15.33:0/1104416595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99d0100c20 0x7f99d0106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.724+0000 7f99d6300640 1 --2- 172.21.15.33:0/1104416595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99d0100270 0x7f99d0100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.724+0000 7f99d6300640 1 -- 172.21.15.33:0/1104416595 >> 172.21.15.33:0/1104416595 conn(0x7f99d0076fc0 msgr2=0x7f99d00fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.724+0000 7f99d6300640 1 -- 172.21.15.33:0/1104416595 shutdown_connections 2024-04-02T13:48:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.724+0000 7f99d6300640 1 -- 172.21.15.33:0/1104416595 wait complete. 2024-04-02T13:48:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d6300640 1 Processor -- start 2024-04-02T13:48:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d6300640 1 -- start start 2024-04-02T13:48:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d6300640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99d0100270 0x7f99d0198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d6300640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99d0106e80 0x7f99d0198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d6300640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99d019cac0 0x7f99d019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d6300640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0111eb0 con 0x7f99d019cac0 2024-04-02T13:48:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d6300640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0111d30 con 0x7f99d0106e80 2024-04-02T13:48:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d6300640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0112030 con 0x7f99d0100270 2024-04-02T13:48:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d489d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99d019cac0 0x7f99d019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d489d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99d019cac0 0x7f99d019cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37806/0 (socket says 172.21.15.33:37806) 2024-04-02T13:48:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d489d640 1 -- 172.21.15.33:0/1443115098 learned_addr learned my addr 172.21.15.33:0/1443115098 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99cffff640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99d0100270 0x7f99d0198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99d0106e80 0x7f99d0198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d489d640 1 -- 172.21.15.33:0/1443115098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99d0100270 msgr2=0x7f99d0198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d489d640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99d0100270 0x7f99d0198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d489d640 1 -- 172.21.15.33:0/1443115098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99d0106e80 msgr2=0x7f99d0198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d489d640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99d0106e80 0x7f99d0198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99d489d640 1 -- 172.21.15.33:0/1443115098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d019d5b0 con 0x7f99d019cac0 2024-04-02T13:48:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.725+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99d0106e80 0x7f99d0198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.726+0000 7f99d489d640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99d019cac0 0x7f99d019cf70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f99c8005cc0 tx=0x7f99c8034d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.726+0000 7f99cd7fa640 1 -- 172.21.15.33:0/1443115098 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8039050 con 0x7f99d019cac0 2024-04-02T13:48:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.726+0000 7f99cd7fa640 1 -- 172.21.15.33:0/1443115098 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f99c803a040 con 0x7f99d019cac0 2024-04-02T13:48:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.726+0000 7f99d6300640 1 -- 172.21.15.33:0/1443115098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d019d840 con 0x7f99d019cac0 2024-04-02T13:48:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.726+0000 7f99cd7fa640 1 -- 172.21.15.33:0/1443115098 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c80046c0 con 0x7f99d019cac0 2024-04-02T13:48:03.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.726+0000 7f99d6300640 1 -- 172.21.15.33:0/1443115098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d01a5000 con 0x7f99d019cac0 2024-04-02T13:48:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.728+0000 7f99cd7fa640 1 -- 172.21.15.33:0/1443115098 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f99c8003970 con 0x7f99d019cac0 2024-04-02T13:48:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.728+0000 7f99d8fc1740 1 Processor -- start 2024-04-02T13:48:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.728+0000 7f99d8fc1740 1 -- start start 2024-04-02T13:48:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99d8fc1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc84d0 0x55f500bc88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99d8fc1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc8df0 0x55f500bd5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99d8fc1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 0x55f500bd84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99d8fc1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f500b2c7b0 con 0x55f500bc8df0 2024-04-02T13:48:03.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99d8fc1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f500acb570 con 0x55f500bd60b0 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99d8fc1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f500b28760 con 0x55f500bc84d0 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cd7fa640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f99b80a3d20 0x7f99b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc8df0 0x55f500bd5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc8df0 0x55f500bd5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37810/0 (socket says 172.21.15.33:37810) 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cffff640 1 -- 172.21.15.33:0/3979984790 learned_addr learned my addr 172.21.15.33:0/3979984790 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/3979984790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 0x55f500bd84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99d489d640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f99b80a3d20 0x7f99b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/3979984790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc84d0 0x55f500bc88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cd7fa640 1 -- 172.21.15.33:0/1443115098 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f99c80f3480 con 0x7f99d019cac0 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cffff640 1 -- 172.21.15.33:0/3979984790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc84d0 msgr2=0x55f500bc88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cffff640 1 --2- 172.21.15.33:0/3979984790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc84d0 0x55f500bc88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cffff640 1 -- 172.21.15.33:0/3979984790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 msgr2=0x55f500bd84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cffff640 1 --2- 172.21.15.33:0/3979984790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 0x55f500bd84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cffff640 1 -- 172.21.15.33:0/3979984790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5008403f0 con 0x55f500bc8df0 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/3979984790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc84d0 0x55f500bc88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99cffff640 1 --2- 172.21.15.33:0/3979984790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc8df0 0x55f500bd5b70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f99c000ad00 tx=0x7f99c000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f42c21932b9b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99d489d640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f99b80a3d20 0x7f99b80a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f99c80bee00 tx=0x7f99c80095c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99b67fc640 1 -- 172.21.15.33:0/3979984790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c001d070 con 0x55f500bc8df0 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99b67fc640 1 -- 172.21.15.33:0/3979984790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f99c0018b10 con 0x55f500bc8df0 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.729+0000 7f99b67fc640 1 -- 172.21.15.33:0/3979984790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0010b70 con 0x55f500bc8df0 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.730+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3979984790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc8df0 msgr2=0x55f500bd5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.730+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3979984790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc8df0 0x55f500bd5b70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f99c000ad00 tx=0x7f99c000f790 comp rx=0 tx=0).stop 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.730+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3979984790 shutdown_connections 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.730+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3979984790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 0x55f500bd84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.730+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3979984790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc8df0 0x55f500bd5b70 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.730+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3979984790 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc84d0 0x55f500bc88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.730+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3979984790 >> 172.21.15.33:0/3979984790 conn(0x55f500bc2ed0 msgr2=0x55f500bc32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.730+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3979984790 shutdown_connections 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.730+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3979984790 wait complete. 2024-04-02T13:48:03.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.730+0000 7f99d8fc1740 1 -- start start 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99d8fc1740 1 Processor -- start 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99d8fc1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99d8fc1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc8df0 0x55f500bd2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99d8fc1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 0x55f500bd3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99d8fc1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f500bd8e70 con 0x55f500bc84d0 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99d8fc1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f500ae7e80 con 0x55f500bd60b0 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99d8fc1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55f500bd8ff0 con 0x55f500bc8df0 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99d8fc1740 10 client.0 _ll_register_callbacks cb 0x55f500b23aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37826/0 (socket says 172.21.15.33:37826) 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99cf7fe640 1 -- 172.21.15.33:0/3343332195 learned_addr learned my addr 172.21.15.33:0/3343332195 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99cffff640 1 --2- 172.21.15.33:0/3343332195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc8df0 0x55f500bd2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/3343332195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 0x55f500bd3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99cf7fe640 1 -- 172.21.15.33:0/3343332195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc8df0 msgr2=0x55f500bd2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/3343332195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc8df0 0x55f500bd2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99cf7fe640 1 -- 172.21.15.33:0/3343332195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 msgr2=0x55f500bd3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/3343332195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 0x55f500bd3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.731+0000 7f99cf7fe640 1 -- 172.21.15.33:0/3343332195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50090a7d0 con 0x55f500bc84d0 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/3343332195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd5af0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f99d0069a60 tx=0x7f99c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf33806ddfc970e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99af7fe640 1 -- 172.21.15.33:0/3343332195 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4005220 con 0x55f500bc84d0 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99af7fe640 1 -- 172.21.15.33:0/3343332195 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f99c40053c0 con 0x55f500bc84d0 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3343332195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c0018b10 con 0x55f500bc84d0 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99af7fe640 1 -- 172.21.15.33:0/3343332195 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f99c4026050 con 0x55f500bc84d0 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99d8fc1740 10 client.17607 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3343332195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f500be4f20 con 0x55f500bc84d0 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99af7fe640 1 -- 172.21.15.33:0/3343332195 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f99c0018b10 con 0x55f500bc84d0 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99af7fe640 1 -- 172.21.15.33:0/3343332195 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f99c400f720 con 0x55f500bc84d0 2024-04-02T13:48:03.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99d8fc1740 10 client.17607 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99d8fc1740 10 client.17607 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99d8fc1740 20 client.17607 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3343332195 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f500be6330 0x55f500be8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99d8fc1740 4 client.17607 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.732+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3343332195 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f500b39cf0 con 0x55f500be6330 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.733+0000 7f99cffff640 1 --2- 172.21.15.33:0/3343332195 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f500be6330 0x55f500be8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.733+0000 7f99af7fe640 10 client.17607 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.733+0000 7f99cffff640 1 --2- 172.21.15.33:0/3343332195 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f500be6330 0x55f500be8790 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44643a510ed19b2a server_cookie=9c4fe1920f889e6 in_seq=0 out_seq=0 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99af7fe640 1 -- 172.21.15.33:0/3343332195 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f500b39cf0 con 0x55f500be6330 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99af7fe640 10 client.17607 handle_command_reply: tid=0 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99d8fc1740 1 client.17607 shutdown 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3343332195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 msgr2=0x55f500bd5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3343332195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd5af0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f99d0069a60 tx=0x7f99c4004730 comp rx=0 tx=0).stop 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3343332195 shutdown_connections 2024-04-02T13:48:03.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3343332195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 0x55f500bd3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3343332195 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f500be6330 0x55f500be8790 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3343332195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc8df0 0x55f500bd2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3343332195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd5af0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3343332195 >> 172.21.15.33:0/3343332195 conn(0x55f500bc2ed0 msgr2=0x55f500bc4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99af7fe640 0 client.17607 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:48:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3343332195 shutdown_connections 2024-04-02T13:48:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3343332195 wait complete. 2024-04-02T13:48:03.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.734+0000 7f99d8fc1740 20 client.17607 trim_cache size 0 max 16384 2024-04-02T13:48:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.747+0000 7f99d8fc1740 1 Processor -- start 2024-04-02T13:48:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.747+0000 7f99d8fc1740 1 -- start start 2024-04-02T13:48:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.747+0000 7f99d8fc1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99d8fc1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bc8df0 0x55f500bd5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99d8fc1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bd60b0 0x55f500bd5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99d8fc1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99d01af580 con 0x55f500bc84d0 2024-04-02T13:48:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99d8fc1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f99d01ace50 con 0x55f500bc8df0 2024-04-02T13:48:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99d8fc1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99d01aacc0 con 0x55f500bd60b0 2024-04-02T13:48:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37830/0 (socket says 172.21.15.33:37830) 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99cf7fe640 1 -- 172.21.15.33:0/2879214632 learned_addr learned my addr 172.21.15.33:0/2879214632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99cffff640 1 --2- 172.21.15.33:0/2879214632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bc8df0 0x55f500bd5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99cf7fe640 1 -- 172.21.15.33:0/2879214632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bd60b0 msgr2=0x55f500bd5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/2879214632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bd60b0 0x55f500bd5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99cf7fe640 1 -- 172.21.15.33:0/2879214632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bc8df0 msgr2=0x55f500bd5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/2879214632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bc8df0 0x55f500bd5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99cf7fe640 1 -- 172.21.15.33:0/2879214632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f500bc26f0 con 0x55f500bc84d0 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/2879214632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd27d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f99c4001eb0 tx=0x7f99c400d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19686e6dc8df720 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99b5ffb640 1 -- 172.21.15.33:0/2879214632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c402f070 con 0x55f500bc84d0 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99b5ffb640 1 -- 172.21.15.33:0/2879214632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f99c4023030 con 0x55f500bc84d0 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.748+0000 7f99b5ffb640 1 -- 172.21.15.33:0/2879214632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4004a00 con 0x55f500bc84d0 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.749+0000 7f99d8fc1740 1 -- 172.21.15.33:0/2879214632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 msgr2=0x55f500bd27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.749+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/2879214632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd27d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f99c4001eb0 tx=0x7f99c400d970 comp rx=0 tx=0).stop 2024-04-02T13:48:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.749+0000 7f99d8fc1740 1 -- 172.21.15.33:0/2879214632 shutdown_connections 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.749+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/2879214632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bd60b0 0x55f500bd5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.749+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/2879214632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bc8df0 0x55f500bd5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.749+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/2879214632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd27d0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.749+0000 7f99d8fc1740 1 -- 172.21.15.33:0/2879214632 >> 172.21.15.33:0/2879214632 conn(0x55f500bc2ed0 msgr2=0x55f500bc16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.749+0000 7f99d8fc1740 1 -- 172.21.15.33:0/2879214632 shutdown_connections 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.749+0000 7f99d8fc1740 1 -- 172.21.15.33:0/2879214632 wait complete. 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.749+0000 7f99d8fc1740 1 -- start start 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.749+0000 7f99d8fc1740 1 Processor -- start 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99d8fc1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99d8fc1740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc8df0 0x55f500bd71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99d8fc1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 0x55f500bdd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99d8fc1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99d01ab310 con 0x55f500bc84d0 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99d8fc1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f99d01ac780 con 0x55f500bd60b0 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99d8fc1740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99d01ab9e0 con 0x55f500bc8df0 2024-04-02T13:48:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99d8fc1740 10 client.0 _ll_register_callbacks cb 0x7f99c4005f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37842/0 (socket says 172.21.15.33:37842) 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99cf7fe640 1 -- 172.21.15.33:0/3036967212 learned_addr learned my addr 172.21.15.33:0/3036967212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 0x55f500bdd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99cffff640 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc8df0 0x55f500bd71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99cf7fe640 1 -- 172.21.15.33:0/3036967212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc8df0 msgr2=0x55f500bd71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc8df0 0x55f500bd71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99cf7fe640 1 -- 172.21.15.33:0/3036967212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 msgr2=0x55f500bdd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 0x55f500bdd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99cf7fe640 1 -- 172.21.15.33:0/3036967212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f500bdd840 con 0x55f500bc84d0 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99cffff640 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc8df0 0x55f500bd71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99cf7fe640 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd6c80 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f99c400faf0 tx=0x7f99c4013b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e1f5ac420c1296 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99af7fe640 1 -- 172.21.15.33:0/3036967212 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c402f070 con 0x55f500bc84d0 2024-04-02T13:48:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99af7fe640 1 -- 172.21.15.33:0/3036967212 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f99c4005230 con 0x55f500bc84d0 2024-04-02T13:48:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3036967212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c4023030 con 0x55f500bc84d0 2024-04-02T13:48:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99af7fe640 1 -- 172.21.15.33:0/3036967212 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f99c403d050 con 0x55f500bc84d0 2024-04-02T13:48:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99d8fc1740 10 client.17613 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.750+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3036967212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f500bdeb00 con 0x55f500bc84d0 2024-04-02T13:48:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.751+0000 7f99af7fe640 1 -- 172.21.15.33:0/3036967212 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f99c4023030 con 0x55f500bc84d0 2024-04-02T13:48:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.751+0000 7f99af7fe640 1 -- 172.21.15.33:0/3036967212 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f99c4004ad0 con 0x55f500bc84d0 2024-04-02T13:48:03.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:03 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3063114035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.751+0000 7f99d8fc1740 10 client.17613 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.751+0000 7f99d8fc1740 10 client.17613 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.751+0000 7f99d8fc1740 20 client.17613 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.751+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f500c4ccc0 0x55f500c4f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.751+0000 7f99d8fc1740 4 client.17613 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.751+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3036967212 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f500b34040 con 0x55f500c4ccc0 2024-04-02T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.751+0000 7f99cffff640 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f500c4ccc0 0x55f500c4f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.751+0000 7f99af7fe640 10 client.17613 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.751+0000 7f99cffff640 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f500c4ccc0 0x55f500c4f0e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=552783bcd5d72994 server_cookie=5e50c9021a37c991 in_seq=0 out_seq=0 2024-04-02T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99af7fe640 1 -- 172.21.15.33:0/3036967212 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f500b34040 con 0x55f500c4ccc0 2024-04-02T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99af7fe640 10 client.17613 handle_command_reply: tid=0 2024-04-02T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d8fc1740 1 client.17613 shutdown 2024-04-02T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3036967212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 msgr2=0x55f500bd6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd6c80 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f99c400faf0 tx=0x7f99c4013b20 comp rx=0 tx=0).stop 2024-04-02T13:48:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3036967212 shutdown_connections 2024-04-02T13:48:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f500bd60b0 0x55f500bdd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55f500c4ccc0 0x55f500c4f0e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f500bc8df0 0x55f500bd71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d8fc1740 1 --2- 172.21.15.33:0/3036967212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f500bc84d0 0x55f500bd6c80 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99af7fe640 0 client.17613 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:48:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3036967212 >> 172.21.15.33:0/3036967212 conn(0x55f500bc2ed0 msgr2=0x55f500bc4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3036967212 shutdown_connections 2024-04-02T13:48:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d8fc1740 1 -- 172.21.15.33:0/3036967212 wait complete. 2024-04-02T13:48:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d8fc1740 20 client.17613 trim_cache size 0 max 16384 2024-04-02T13:48:03.761 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:48:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d6300640 1 -- 172.21.15.33:0/1443115098 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f99b80a3d20 msgr2=0x7f99b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d6300640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f99b80a3d20 0x7f99b80a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f99c80bee00 tx=0x7f99c80095c0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d6300640 1 -- 172.21.15.33:0/1443115098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99d019cac0 msgr2=0x7f99d019cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.752+0000 7f99d6300640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99d019cac0 0x7f99d019cf70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f99c8005cc0 tx=0x7f99c8034d70 comp rx=0 tx=0).stop 2024-04-02T13:48:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.753+0000 7f99d6300640 1 -- 172.21.15.33:0/1443115098 shutdown_connections 2024-04-02T13:48:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.753+0000 7f99d6300640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f99b80a3d20 0x7f99b80a61e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.753+0000 7f99d6300640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99d019cac0 0x7f99d019cf70 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.753+0000 7f99d6300640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99d0106e80 0x7f99d0198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.753+0000 7f99d6300640 1 --2- 172.21.15.33:0/1443115098 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99d0100270 0x7f99d0198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.753+0000 7f99d6300640 1 -- 172.21.15.33:0/1443115098 >> 172.21.15.33:0/1443115098 conn(0x7f99d0076fc0 msgr2=0x7f99d0104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.753+0000 7f99d6300640 1 -- 172.21.15.33:0/1443115098 shutdown_connections 2024-04-02T13:48:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.753+0000 7f99d6300640 1 -- 172.21.15.33:0/1443115098 wait complete. 2024-04-02T13:48:03.766 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:48:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.868+0000 7fde4833d640 1 -- 172.21.15.33:0/2592812862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde40100670 msgr2=0x7fde40100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.868+0000 7fde4833d640 1 --2- 172.21.15.33:0/2592812862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde40100670 0x7fde40100a50 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009900 tx=0x7fde3c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.868+0000 7fde4833d640 1 -- 172.21.15.33:0/2592812862 shutdown_connections 2024-04-02T13:48:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.868+0000 7fde4833d640 1 --2- 172.21.15.33:0/2592812862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde40106e30 0x7fde4010ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.868+0000 7fde4833d640 1 --2- 172.21.15.33:0/2592812862 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde40101020 0x7fde40106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.868+0000 7fde4833d640 1 --2- 172.21.15.33:0/2592812862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde40100670 0x7fde40100a50 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.868+0000 7fde4833d640 1 -- 172.21.15.33:0/2592812862 >> 172.21.15.33:0/2592812862 conn(0x7fde40075c60 msgr2=0x7fde40076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.868+0000 7fde4833d640 1 -- 172.21.15.33:0/2592812862 shutdown_connections 2024-04-02T13:48:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.868+0000 7fde4833d640 1 -- 172.21.15.33:0/2592812862 wait complete. 2024-04-02T13:48:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde4833d640 1 Processor -- start 2024-04-02T13:48:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde4833d640 1 -- start start 2024-04-02T13:48:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde4833d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde40100670 0x7fde40198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde4833d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde40106e30 0x7fde401988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde4833d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde4019cc50 0x7fde4019d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde4833d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde40111ef0 con 0x7fde40100670 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde4833d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde40111d70 con 0x7fde40106e30 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde4833d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fde40112070 con 0x7fde4019cc50 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde460d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde40100670 0x7fde40198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde460d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde40100670 0x7fde40198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37872/0 (socket says 172.21.15.33:37872) 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde460d9640 1 -- 172.21.15.33:0/34742512 learned_addr learned my addr 172.21.15.33:0/34742512 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde468da640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde4019cc50 0x7fde4019d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde458d8640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde40106e30 0x7fde401988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde460d9640 1 -- 172.21.15.33:0/34742512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde4019cc50 msgr2=0x7fde4019d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde460d9640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde4019cc50 0x7fde4019d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde460d9640 1 -- 172.21.15.33:0/34742512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde40106e30 msgr2=0x7fde401988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde460d9640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde40106e30 0x7fde401988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde460d9640 1 -- 172.21.15.33:0/34742512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4019d740 con 0x7fde40100670 2024-04-02T13:48:03.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.869+0000 7fde460d9640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde40100670 0x7fde40198390 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005940 tx=0x7fde3c004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.870+0000 7fde377fe640 1 -- 172.21.15.33:0/34742512 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c043020 con 0x7fde40100670 2024-04-02T13:48:03.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.870+0000 7fde377fe640 1 -- 172.21.15.33:0/34742512 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde3c02c5d0 con 0x7fde40100670 2024-04-02T13:48:03.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.870+0000 7fde4833d640 1 -- 172.21.15.33:0/34742512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4019d9d0 con 0x7fde40100670 2024-04-02T13:48:03.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.870+0000 7fde377fe640 1 -- 172.21.15.33:0/34742512 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c003950 con 0x7fde40100670 2024-04-02T13:48:03.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.870+0000 7fde4833d640 1 -- 172.21.15.33:0/34742512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde401a51a0 con 0x7fde40100670 2024-04-02T13:48:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.872+0000 7fde377fe640 1 -- 172.21.15.33:0/34742512 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fde3c046020 con 0x7fde40100670 2024-04-02T13:48:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.872+0000 7fde4affe740 1 Processor -- start 2024-04-02T13:48:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.872+0000 7fde4affe740 1 -- start start 2024-04-02T13:48:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde4affe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 0x561d2d757b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde4affe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d7580c0 0x561d2d760760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde4affe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 0x561d2d763090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde377fe640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fde2c0a3d20 0x7fde2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde4affe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561d2d6bb7b0 con 0x561d2d760ca0 2024-04-02T13:48:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde4affe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561d2d65a570 con 0x561d2d7580c0 2024-04-02T13:48:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde4affe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561d2d6b7760 con 0x561d2d7577a0 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde458d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 0x561d2d763090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde468da640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fde2c0a3d20 0x7fde2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde458d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 0x561d2d763090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37874/0 (socket says 172.21.15.33:37874) 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde458d8640 1 -- 172.21.15.33:0/1308658194 learned_addr learned my addr 172.21.15.33:0/1308658194 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde377fe640 1 -- 172.21.15.33:0/34742512 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fde3c0f24c0 con 0x7fde40100670 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde460d9640 1 --2- 172.21.15.33:0/1308658194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d7580c0 0x561d2d760760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde468da640 1 --2- 172.21.15.33:0/1308658194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 0x561d2d757b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde458d8640 1 -- 172.21.15.33:0/1308658194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 msgr2=0x561d2d757b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde458d8640 1 --2- 172.21.15.33:0/1308658194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 0x561d2d757b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde458d8640 1 -- 172.21.15.33:0/1308658194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d7580c0 msgr2=0x561d2d760760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde458d8640 1 --2- 172.21.15.33:0/1308658194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d7580c0 0x561d2d760760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde458d8640 1 -- 172.21.15.33:0/1308658194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2d3cf3f0 con 0x561d2d760ca0 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde468da640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fde2c0a3d20 0x7fde2c0a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fde38005fd0 tx=0x7fde38011b40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde458d8640 1 --2- 172.21.15.33:0/1308658194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 0x561d2d763090 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fde3000ac70 tx=0x7fde3000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf056cfa3e8fc45 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.873+0000 7fde34bfe640 1 -- 172.21.15.33:0/1308658194 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3001d070 con 0x561d2d760ca0 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.874+0000 7fde34bfe640 1 -- 172.21.15.33:0/1308658194 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde30018b10 con 0x561d2d760ca0 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.874+0000 7fde34bfe640 1 -- 172.21.15.33:0/1308658194 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30010ae0 con 0x561d2d760ca0 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.874+0000 7fde4affe740 1 -- 172.21.15.33:0/1308658194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 msgr2=0x561d2d763090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.874+0000 7fde4affe740 1 --2- 172.21.15.33:0/1308658194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 0x561d2d763090 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fde3000ac70 tx=0x7fde3000f790 comp rx=0 tx=0).stop 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.874+0000 7fde4affe740 1 -- 172.21.15.33:0/1308658194 shutdown_connections 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.874+0000 7fde4affe740 1 --2- 172.21.15.33:0/1308658194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 0x561d2d763090 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.874+0000 7fde4affe740 1 --2- 172.21.15.33:0/1308658194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d7580c0 0x561d2d760760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.874+0000 7fde4affe740 1 --2- 172.21.15.33:0/1308658194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 0x561d2d757b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.874+0000 7fde4affe740 1 -- 172.21.15.33:0/1308658194 >> 172.21.15.33:0/1308658194 conn(0x561d2d751ed0 msgr2=0x561d2d7522b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.874+0000 7fde4affe740 1 -- 172.21.15.33:0/1308658194 shutdown_connections 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.874+0000 7fde4affe740 1 -- 172.21.15.33:0/1308658194 wait complete. 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde4affe740 1 -- start start 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde4affe740 1 Processor -- start 2024-04-02T13:48:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde4affe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d7577a0 0x561d2d7606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde4affe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7580c0 0x561d2d75da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde4affe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 0x561d2d75e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde4affe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561d2d763a90 con 0x561d2d760ca0 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde4affe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561d2d676e80 con 0x561d2d7577a0 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde4affe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x561d2d763c10 con 0x561d2d7580c0 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde4affe740 10 client.0 _ll_register_callbacks cb 0x561d2d6b2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde468da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 0x561d2d75e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde468da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 0x561d2d75e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37888/0 (socket says 172.21.15.33:37888) 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde468da640 1 -- 172.21.15.33:0/1738022200 learned_addr learned my addr 172.21.15.33:0/1738022200 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde458d8640 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7580c0 0x561d2d75da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde460d9640 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d7577a0 0x561d2d7606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde468da640 1 -- 172.21.15.33:0/1738022200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7580c0 msgr2=0x561d2d75da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde468da640 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7580c0 0x561d2d75da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde468da640 1 -- 172.21.15.33:0/1738022200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d7577a0 msgr2=0x561d2d7606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde468da640 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d7577a0 0x561d2d7606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde468da640 1 -- 172.21.15.33:0/1738022200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2d4997d0 con 0x561d2d760ca0 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde460d9640 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d7577a0 0x561d2d7606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.875+0000 7fde458d8640 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7580c0 0x561d2d75da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde468da640 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 0x561d2d75e1a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fde3800fcc0 tx=0x7fde38009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d68adeef15f124 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde29ffb640 1 -- 172.21.15.33:0/1738022200 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3802c060 con 0x561d2d760ca0 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde29ffb640 1 -- 172.21.15.33:0/1738022200 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fde380045b0 con 0x561d2d760ca0 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde4affe740 1 -- 172.21.15.33:0/1738022200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde30018b10 con 0x561d2d760ca0 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde29ffb640 1 -- 172.21.15.33:0/1738022200 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fde380047b0 con 0x561d2d760ca0 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde4affe740 10 client.17625 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde4affe740 1 -- 172.21.15.33:0/1738022200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561d2d76fb00 con 0x561d2d760ca0 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde29ffb640 1 -- 172.21.15.33:0/1738022200 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde30018b10 con 0x561d2d760ca0 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde29ffb640 1 -- 172.21.15.33:0/1738022200 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fde38026030 con 0x561d2d760ca0 2024-04-02T13:48:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde4affe740 10 client.17625 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde4affe740 10 client.17625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde4affe740 20 client.17625 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde4affe740 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x561d2d770fc0 0x561d2d773440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde4affe740 4 client.17625 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.876+0000 7fde4affe740 1 -- 172.21.15.33:0/1738022200 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d2d6c8cf0 con 0x561d2d770fc0 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.877+0000 7fde460d9640 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x561d2d770fc0 0x561d2d773440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.877+0000 7fde460d9640 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x561d2d770fc0 0x561d2d773440 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8f6280e2daa6ebb server_cookie=e350f02ba04f6a61 in_seq=0 out_seq=0 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.877+0000 7fde29ffb640 10 client.17625 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde29ffb640 1 -- 172.21.15.33:0/1738022200 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d2d6c8cf0 con 0x561d2d770fc0 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde29ffb640 10 client.17625 handle_command_reply: tid=0 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde4affe740 1 client.17625 shutdown 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde4affe740 1 -- 172.21.15.33:0/1738022200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 msgr2=0x561d2d75e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde4affe740 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 0x561d2d75e1a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fde3800fcc0 tx=0x7fde38009500 comp rx=0 tx=0).stop 2024-04-02T13:48:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde4affe740 1 -- 172.21.15.33:0/1738022200 shutdown_connections 2024-04-02T13:48:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde4affe740 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x561d2d770fc0 0x561d2d773440 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde4affe740 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d760ca0 0x561d2d75e1a0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde29ffb640 0 client.17625 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde4affe740 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7580c0 0x561d2d75da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde4affe740 1 --2- 172.21.15.33:0/1738022200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d7577a0 0x561d2d7606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde4affe740 1 -- 172.21.15.33:0/1738022200 >> 172.21.15.33:0/1738022200 conn(0x561d2d751ed0 msgr2=0x561d2d753c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde4affe740 1 -- 172.21.15.33:0/1738022200 shutdown_connections 2024-04-02T13:48:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde4affe740 1 -- 172.21.15.33:0/1738022200 wait complete. 2024-04-02T13:48:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.878+0000 7fde4affe740 20 client.17625 trim_cache size 0 max 16384 2024-04-02T13:48:03.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.891+0000 7fde4affe740 1 Processor -- start 2024-04-02T13:48:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.891+0000 7fde4affe740 1 -- start start 2024-04-02T13:48:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde4affe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 0x561d2d76c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde4affe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 0x561d2d778d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde4affe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d760ca0 0x561d2d779250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde4affe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde401b1780 con 0x561d2d7580c0 2024-04-02T13:48:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde4affe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde401af050 con 0x561d2d760ca0 2024-04-02T13:48:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde4affe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fde401acec0 con 0x561d2d7577a0 2024-04-02T13:48:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde458d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 0x561d2d778d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde458d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 0x561d2d778d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37902/0 (socket says 172.21.15.33:37902) 2024-04-02T13:48:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde458d8640 1 -- 172.21.15.33:0/2124905407 learned_addr learned my addr 172.21.15.33:0/2124905407 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde468da640 1 --2- 172.21.15.33:0/2124905407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d760ca0 0x561d2d779250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde458d8640 1 -- 172.21.15.33:0/2124905407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 msgr2=0x561d2d76c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde458d8640 1 --2- 172.21.15.33:0/2124905407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 0x561d2d76c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde458d8640 1 -- 172.21.15.33:0/2124905407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d760ca0 msgr2=0x561d2d779250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde458d8640 1 --2- 172.21.15.33:0/2124905407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d760ca0 0x561d2d779250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde458d8640 1 -- 172.21.15.33:0/2124905407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2d757ef0 con 0x561d2d7580c0 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde468da640 1 --2- 172.21.15.33:0/2124905407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d760ca0 0x561d2d779250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde458d8640 1 --2- 172.21.15.33:0/2124905407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 0x561d2d778d10 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fde3000ab90 tx=0x7fde30005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34badfc4debe9b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde2bfff640 1 -- 172.21.15.33:0/2124905407 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30027020 con 0x561d2d7580c0 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde2bfff640 1 -- 172.21.15.33:0/2124905407 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde30005510 con 0x561d2d7580c0 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.892+0000 7fde2bfff640 1 -- 172.21.15.33:0/2124905407 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3001d040 con 0x561d2d7580c0 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 -- 172.21.15.33:0/2124905407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 msgr2=0x561d2d778d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 --2- 172.21.15.33:0/2124905407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 0x561d2d778d10 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fde3000ab90 tx=0x7fde30005e60 comp rx=0 tx=0).stop 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 -- 172.21.15.33:0/2124905407 shutdown_connections 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 --2- 172.21.15.33:0/2124905407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d760ca0 0x561d2d779250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 --2- 172.21.15.33:0/2124905407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 0x561d2d778d10 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 --2- 172.21.15.33:0/2124905407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 0x561d2d76c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 -- 172.21.15.33:0/2124905407 >> 172.21.15.33:0/2124905407 conn(0x561d2d751ed0 msgr2=0x561d2d75c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 -- 172.21.15.33:0/2124905407 shutdown_connections 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 -- 172.21.15.33:0/2124905407 wait complete. 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 -- start start 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 Processor -- start 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 0x561d2d7691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 0x561d2d761780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d760ca0 0x561d2d761cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde401ad510 con 0x561d2d7580c0 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde401ae980 con 0x561d2d760ca0 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fde401adbe0 con 0x561d2d7577a0 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.893+0000 7fde4affe740 10 client.0 _ll_register_callbacks cb 0x7fde3800f9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde458d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 0x561d2d761780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde458d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 0x561d2d761780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37910/0 (socket says 172.21.15.33:37910) 2024-04-02T13:48:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde458d8640 1 -- 172.21.15.33:0/2912252792 learned_addr learned my addr 172.21.15.33:0/2912252792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde468da640 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d760ca0 0x561d2d761cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde460d9640 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 0x561d2d7691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde458d8640 1 -- 172.21.15.33:0/2912252792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 msgr2=0x561d2d7691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde458d8640 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 0x561d2d7691f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde458d8640 1 -- 172.21.15.33:0/2912252792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d760ca0 msgr2=0x561d2d761cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde458d8640 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d760ca0 0x561d2d761cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde458d8640 1 -- 172.21.15.33:0/2912252792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2d7624d0 con 0x561d2d7580c0 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde460d9640 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 0x561d2d7691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde468da640 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d760ca0 0x561d2d761cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde458d8640 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 0x561d2d761780 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fde3000f7d0 tx=0x7fde3000a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed707843b01113b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde29ffb640 1 -- 172.21.15.33:0/2912252792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30030020 con 0x561d2d7580c0 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde4affe740 1 -- 172.21.15.33:0/2912252792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde30005510 con 0x561d2d7580c0 2024-04-02T13:48:03.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde29ffb640 1 -- 172.21.15.33:0/2912252792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fde30004d30 con 0x561d2d7580c0 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde29ffb640 1 -- 172.21.15.33:0/2912252792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fde3003b050 con 0x561d2d7580c0 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde4affe740 10 client.17631 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.894+0000 7fde4affe740 1 -- 172.21.15.33:0/2912252792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561d2d7d7580 con 0x561d2d7580c0 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.895+0000 7fde29ffb640 1 -- 172.21.15.33:0/2912252792 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde30005510 con 0x561d2d7580c0 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.895+0000 7fde29ffb640 1 -- 172.21.15.33:0/2912252792 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fde3002ca40 con 0x561d2d7580c0 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.895+0000 7fde4affe740 10 client.17631 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.895+0000 7fde4affe740 10 client.17631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.895+0000 7fde4affe740 20 client.17631 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.895+0000 7fde4affe740 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x561d2d7d7b30 0x561d2d7d9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.895+0000 7fde4affe740 4 client.17631 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.895+0000 7fde4affe740 1 -- 172.21.15.33:0/2912252792 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d2d6c3040 con 0x561d2d7d7b30 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.895+0000 7fde460d9640 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x561d2d7d7b30 0x561d2d7d9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.895+0000 7fde460d9640 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x561d2d7d7b30 0x561d2d7d9f50 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5758579f748ba2d server_cookie=fc66a5c17c8246d in_seq=0 out_seq=0 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.895+0000 7fde29ffb640 10 client.17631 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde29ffb640 1 -- 172.21.15.33:0/2912252792 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d2d6c3040 con 0x561d2d7d7b30 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde29ffb640 10 client.17631 handle_command_reply: tid=0 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4affe740 1 client.17631 shutdown 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4affe740 1 -- 172.21.15.33:0/2912252792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 msgr2=0x561d2d761780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4affe740 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 0x561d2d761780 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fde3000f7d0 tx=0x7fde3000a6b0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4affe740 1 -- 172.21.15.33:0/2912252792 shutdown_connections 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4affe740 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x561d2d7d7b30 0x561d2d7d9f50 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4affe740 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561d2d760ca0 0x561d2d761cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4affe740 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d2d7580c0 0x561d2d761780 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4affe740 1 --2- 172.21.15.33:0/2912252792 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x561d2d7577a0 0x561d2d7691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde29ffb640 0 client.17631 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4affe740 1 -- 172.21.15.33:0/2912252792 >> 172.21.15.33:0/2912252792 conn(0x561d2d751ed0 msgr2=0x561d2d750f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4affe740 1 -- 172.21.15.33:0/2912252792 shutdown_connections 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4affe740 1 -- 172.21.15.33:0/2912252792 wait complete. 2024-04-02T13:48:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4affe740 20 client.17631 trim_cache size 0 max 16384 2024-04-02T13:48:03.906 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:48:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4833d640 1 -- 172.21.15.33:0/34742512 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fde2c0a3d20 msgr2=0x7fde2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4833d640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fde2c0a3d20 0x7fde2c0a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fde38005fd0 tx=0x7fde38011b40 comp rx=0 tx=0).stop 2024-04-02T13:48:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4833d640 1 -- 172.21.15.33:0/34742512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde40100670 msgr2=0x7fde40198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4833d640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde40100670 0x7fde40198390 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005940 tx=0x7fde3c004bd0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4833d640 1 -- 172.21.15.33:0/34742512 shutdown_connections 2024-04-02T13:48:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4833d640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fde2c0a3d20 0x7fde2c0a61e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4833d640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde4019cc50 0x7fde4019d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4833d640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde40106e30 0x7fde401988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4833d640 1 --2- 172.21.15.33:0/34742512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde40100670 0x7fde40198390 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.896+0000 7fde4833d640 1 -- 172.21.15.33:0/34742512 >> 172.21.15.33:0/34742512 conn(0x7fde40075c60 msgr2=0x7fde400fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.897+0000 7fde4833d640 1 -- 172.21.15.33:0/34742512 shutdown_connections 2024-04-02T13:48:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:03.897+0000 7fde4833d640 1 -- 172.21.15.33:0/34742512 wait complete. 2024-04-02T13:48:03.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:03 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3063114035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:48:03.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:03 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3063114035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:48:04.910 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:48:04.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:48:04.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:48:04.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:04 smithi145 ceph-mon[35646]: pgmap v287: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T13:48:04.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:04 smithi050 ceph-mon[35524]: pgmap v287: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T13:48:05.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:04 smithi033 ceph-mon[32390]: pgmap v287: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T13:48:05.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.028+0000 7fba156d3640 1 -- 172.21.15.33:0/1847032940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba101069c0 msgr2=0x7fba10106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.028+0000 7fba156d3640 1 --2- 172.21.15.33:0/1847032940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba101069c0 0x7fba10106da0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fba00009930 tx=0x7fba0002bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:05.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.028+0000 7fba156d3640 1 -- 172.21.15.33:0/1847032940 shutdown_connections 2024-04-02T13:48:05.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.028+0000 7fba156d3640 1 --2- 172.21.15.33:0/1847032940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba10101370 0x7fba1010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.028+0000 7fba156d3640 1 --2- 172.21.15.33:0/1847032940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba101009b0 0x7fba10100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.028+0000 7fba156d3640 1 --2- 172.21.15.33:0/1847032940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba101069c0 0x7fba10106da0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.028+0000 7fba156d3640 1 -- 172.21.15.33:0/1847032940 >> 172.21.15.33:0/1847032940 conn(0x7fba10076d80 msgr2=0x7fba10077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:05.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.028+0000 7fba156d3640 1 -- 172.21.15.33:0/1847032940 shutdown_connections 2024-04-02T13:48:05.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.028+0000 7fba156d3640 1 -- 172.21.15.33:0/1847032940 wait complete. 2024-04-02T13:48:05.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.028+0000 7fba156d3640 1 Processor -- start 2024-04-02T13:48:05.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba156d3640 1 -- start start 2024-04-02T13:48:05.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba156d3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba101009b0 0x7fba10199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba156d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba10101370 0x7fba10199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba156d3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba101069c0 0x7fba101a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba156d3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba10110dc0 con 0x7fba10101370 2024-04-02T13:48:05.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba156d3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fba10110c40 con 0x7fba101009b0 2024-04-02T13:48:05.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba156d3640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fba10110f40 con 0x7fba101069c0 2024-04-02T13:48:05.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba0e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba10101370 0x7fba10199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba0e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba10101370 0x7fba10199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37920/0 (socket says 172.21.15.33:37920) 2024-04-02T13:48:05.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba0e7fc640 1 -- 172.21.15.33:0/332999488 learned_addr learned my addr 172.21.15.33:0/332999488 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:05.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba0f7fe640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba101069c0 0x7fba101a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba0effd640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba101009b0 0x7fba10199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba0e7fc640 1 -- 172.21.15.33:0/332999488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba101069c0 msgr2=0x7fba101a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba0e7fc640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba101069c0 0x7fba101a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba0e7fc640 1 -- 172.21.15.33:0/332999488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba101009b0 msgr2=0x7fba10199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba0e7fc640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba101009b0 0x7fba10199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba0e7fc640 1 -- 172.21.15.33:0/332999488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba101a4700 con 0x7fba10101370 2024-04-02T13:48:05.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.029+0000 7fba0f7fe640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba101069c0 0x7fba101a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:05.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.030+0000 7fba0e7fc640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba10101370 0x7fba10199680 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c920 tx=0x7fba0400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:05.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.030+0000 7fb9f7fff640 1 -- 172.21.15.33:0/332999488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04007c70 con 0x7fba10101370 2024-04-02T13:48:05.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.030+0000 7fb9f7fff640 1 -- 172.21.15.33:0/332999488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fba04007e10 con 0x7fba10101370 2024-04-02T13:48:05.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.030+0000 7fba156d3640 1 -- 172.21.15.33:0/332999488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba101a4930 con 0x7fba10101370 2024-04-02T13:48:05.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.030+0000 7fb9f7fff640 1 -- 172.21.15.33:0/332999488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04018850 con 0x7fba10101370 2024-04-02T13:48:05.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.030+0000 7fba156d3640 1 -- 172.21.15.33:0/332999488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba101a4e50 con 0x7fba10101370 2024-04-02T13:48:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fb9f7fff640 1 -- 172.21.15.33:0/332999488 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba04018a30 con 0x7fba10101370 2024-04-02T13:48:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fba1838e740 1 Processor -- start 2024-04-02T13:48:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fba1838e740 1 -- start start 2024-04-02T13:48:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fba1838e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e82c0 0x5603ae4e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fba1838e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e8be0 0x5603ae4f1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fba1838e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4f17c0 0x5603ae4f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fba1838e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5603ae44c770 con 0x5603ae4e8be0 2024-04-02T13:48:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fba1838e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5603ae3f4960 con 0x5603ae4e82c0 2024-04-02T13:48:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fba1838e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5603ae4488c0 con 0x5603ae4f17c0 2024-04-02T13:48:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fba0effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e8be0 0x5603ae4f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fba0effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e8be0 0x5603ae4f1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37934/0 (socket says 172.21.15.33:37934) 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fba0effd640 1 -- 172.21.15.33:0/1115537695 learned_addr learned my addr 172.21.15.33:0/1115537695 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fb9f7fff640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb9f80a3d20 0x7fb9f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.033+0000 7fba0f7fe640 1 --2- 172.21.15.33:0/1115537695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e82c0 0x5603ae4e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.034+0000 7fba0f7fe640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb9f80a3d20 0x7fb9f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.034+0000 7fb9f7fff640 1 -- 172.21.15.33:0/332999488 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fba040d0dd0 con 0x7fba10101370 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.034+0000 7fba0e7fc640 1 --2- 172.21.15.33:0/1115537695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4f17c0 0x5603ae4f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.034+0000 7fba0effd640 1 -- 172.21.15.33:0/1115537695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4f17c0 msgr2=0x5603ae4f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.034+0000 7fba0effd640 1 --2- 172.21.15.33:0/1115537695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4f17c0 0x5603ae4f3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.034+0000 7fba0effd640 1 -- 172.21.15.33:0/1115537695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e82c0 msgr2=0x5603ae4e86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.034+0000 7fba0effd640 1 --2- 172.21.15.33:0/1115537695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e82c0 0x5603ae4e86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.034+0000 7fba0effd640 1 -- 172.21.15.33:0/1115537695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ae2287e0 con 0x5603ae4e8be0 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.034+0000 7fba0effd640 1 --2- 172.21.15.33:0/1115537695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e8be0 0x5603ae4f1280 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fba00009900 tx=0x7fba00004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a82a86005ef2f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:05.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.034+0000 7fba0f7fe640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb9f80a3d20 0x7fb9f80a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba08009fd0 tx=0x7fba08009500 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.035+0000 7fb9f57fa640 1 -- 172.21.15.33:0/1115537695 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00044020 con 0x5603ae4e8be0 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.035+0000 7fb9f57fa640 1 -- 172.21.15.33:0/1115537695 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fba00004430 con 0x5603ae4e8be0 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.035+0000 7fb9f57fa640 1 -- 172.21.15.33:0/1115537695 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00004920 con 0x5603ae4e8be0 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.035+0000 7fba1838e740 1 -- 172.21.15.33:0/1115537695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e8be0 msgr2=0x5603ae4f1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.035+0000 7fba1838e740 1 --2- 172.21.15.33:0/1115537695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e8be0 0x5603ae4f1280 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fba00009900 tx=0x7fba00004040 comp rx=0 tx=0).stop 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.035+0000 7fba1838e740 1 -- 172.21.15.33:0/1115537695 shutdown_connections 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.035+0000 7fba1838e740 1 --2- 172.21.15.33:0/1115537695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4f17c0 0x5603ae4f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.035+0000 7fba1838e740 1 --2- 172.21.15.33:0/1115537695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e8be0 0x5603ae4f1280 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.035+0000 7fba1838e740 1 --2- 172.21.15.33:0/1115537695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e82c0 0x5603ae4e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.035+0000 7fba1838e740 1 -- 172.21.15.33:0/1115537695 >> 172.21.15.33:0/1115537695 conn(0x5603ae4e29f0 msgr2=0x5603ae4e2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.035+0000 7fba1838e740 1 -- 172.21.15.33:0/1115537695 shutdown_connections 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.035+0000 7fba1838e740 1 -- 172.21.15.33:0/1115537695 wait complete. 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba1838e740 1 -- start start 2024-04-02T13:48:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba1838e740 1 Processor -- start 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba1838e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e82c0 0x5603ae4ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba1838e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4e8be0 0x5603ae4eed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba1838e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4f17c0 0x5603ae4ef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba1838e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5603ae4f4550 con 0x5603ae4f17c0 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba1838e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5603ae407d20 con 0x5603ae4e82c0 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba1838e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5603ae4f46d0 con 0x5603ae4e8be0 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba1838e740 10 client.0 _ll_register_callbacks cb 0x5603ae3d4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba0f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4f17c0 0x5603ae4ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba0f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4f17c0 0x5603ae4ef240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37946/0 (socket says 172.21.15.33:37946) 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba0f7fe640 1 -- 172.21.15.33:0/5970993 learned_addr learned my addr 172.21.15.33:0/5970993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba0effd640 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e82c0 0x5603ae4ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.036+0000 7fba0e7fc640 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4e8be0 0x5603ae4eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fba0f7fe640 1 -- 172.21.15.33:0/5970993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4e8be0 msgr2=0x5603ae4eed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fba0f7fe640 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4e8be0 0x5603ae4eed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fba0f7fe640 1 -- 172.21.15.33:0/5970993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e82c0 msgr2=0x5603ae4ee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fba0f7fe640 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e82c0 0x5603ae4ee7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fba0f7fe640 1 -- 172.21.15.33:0/5970993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ae288f70 con 0x5603ae4f17c0 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fba0e7fc640 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4e8be0 0x5603ae4eed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fba0f7fe640 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4f17c0 0x5603ae4ef240 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fba08012580 tx=0x7fba08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4423dd530ca0bf7c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fb9f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fb9f2ff5640 1 -- 172.21.15.33:0/5970993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0800fc10 con 0x5603ae4f17c0 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fb9f2ff5640 1 -- 172.21.15.33:0/5970993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fba0800fdb0 con 0x5603ae4f17c0 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fba1838e740 1 -- 172.21.15.33:0/5970993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba00004430 con 0x5603ae4f17c0 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fb9f2ff5640 1 -- 172.21.15.33:0/5970993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fba08004b20 con 0x5603ae4f17c0 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fba1838e740 10 client.17643 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fba1838e740 1 -- 172.21.15.33:0/5970993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5603ae120170 con 0x5603ae4f17c0 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fb9f2ff5640 1 -- 172.21.15.33:0/5970993 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fba00004430 con 0x5603ae4f17c0 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fb9f2ff5640 1 -- 172.21.15.33:0/5970993 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fba08020550 con 0x5603ae4f17c0 2024-04-02T13:48:05.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fba1838e740 10 client.17643 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:05.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.037+0000 7fba1838e740 10 client.17643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:05.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.038+0000 7fba1838e740 20 client.17643 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:05.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.038+0000 7fba1838e740 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5603ae501b60 0x5603ae503fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.038+0000 7fba1838e740 4 client.17643 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:05.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.038+0000 7fba1838e740 1 -- 172.21.15.33:0/5970993 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603ae459cb0 con 0x5603ae501b60 2024-04-02T13:48:05.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.038+0000 7fba0effd640 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5603ae501b60 0x5603ae503fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.038+0000 7fb9f2ff5640 10 client.17643 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:05.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.038+0000 7fba0effd640 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5603ae501b60 0x5603ae503fe0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeaa41291c9297cc server_cookie=32b14125db93fe48 in_seq=0 out_seq=0 2024-04-02T13:48:05.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fb9f2ff5640 1 -- 172.21.15.33:0/5970993 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603ae459cb0 con 0x5603ae501b60 2024-04-02T13:48:05.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fb9f2ff5640 10 client.17643 handle_command_reply: tid=0 2024-04-02T13:48:05.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fba1838e740 1 client.17643 shutdown 2024-04-02T13:48:05.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fba1838e740 1 -- 172.21.15.33:0/5970993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4f17c0 msgr2=0x5603ae4ef240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fba1838e740 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4f17c0 0x5603ae4ef240 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fba08012580 tx=0x7fba08004a40 comp rx=0 tx=0).stop 2024-04-02T13:48:05.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fba1838e740 1 -- 172.21.15.33:0/5970993 shutdown_connections 2024-04-02T13:48:05.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fba1838e740 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5603ae501b60 0x5603ae503fe0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fba1838e740 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4f17c0 0x5603ae4ef240 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fba1838e740 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4e8be0 0x5603ae4eed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fb9f2ff5640 0 client.17643 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:05.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fba1838e740 1 --2- 172.21.15.33:0/5970993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e82c0 0x5603ae4ee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fba1838e740 1 -- 172.21.15.33:0/5970993 >> 172.21.15.33:0/5970993 conn(0x5603ae4e29f0 msgr2=0x5603ae4e4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:05.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fba1838e740 1 -- 172.21.15.33:0/5970993 shutdown_connections 2024-04-02T13:48:05.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fba1838e740 1 -- 172.21.15.33:0/5970993 wait complete. 2024-04-02T13:48:05.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.039+0000 7fba1838e740 20 client.17643 trim_cache size 0 max 16384 2024-04-02T13:48:05.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.057+0000 7fba1838e740 1 Processor -- start 2024-04-02T13:48:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.057+0000 7fba1838e740 1 -- start start 2024-04-02T13:48:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.057+0000 7fba1838e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 0x5603ae4fd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.057+0000 7fba1838e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e8be0 0x5603ae4e4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.057+0000 7fba1838e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4f17c0 0x5603ae5098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.057+0000 7fba1838e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba101b18b0 con 0x5603ae4e82c0 2024-04-02T13:48:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.057+0000 7fba1838e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fba101af180 con 0x5603ae4e8be0 2024-04-02T13:48:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.057+0000 7fba1838e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fba101acff0 con 0x5603ae4f17c0 2024-04-02T13:48:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.057+0000 7fba0effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 0x5603ae4fd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fba0effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 0x5603ae4fd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37952/0 (socket says 172.21.15.33:37952) 2024-04-02T13:48:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fba0effd640 1 -- 172.21.15.33:0/1396534957 learned_addr learned my addr 172.21.15.33:0/1396534957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fba0e7fc640 1 --2- 172.21.15.33:0/1396534957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e8be0 0x5603ae4e4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fba0f7fe640 1 --2- 172.21.15.33:0/1396534957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4f17c0 0x5603ae5098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fba0effd640 1 -- 172.21.15.33:0/1396534957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4f17c0 msgr2=0x5603ae5098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fba0effd640 1 --2- 172.21.15.33:0/1396534957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4f17c0 0x5603ae5098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fba0effd640 1 -- 172.21.15.33:0/1396534957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e8be0 msgr2=0x5603ae4e4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fba0effd640 1 --2- 172.21.15.33:0/1396534957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e8be0 0x5603ae4e4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fba0effd640 1 -- 172.21.15.33:0/1396534957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ae4e8a10 con 0x5603ae4e82c0 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fba0f7fe640 1 --2- 172.21.15.33:0/1396534957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4f17c0 0x5603ae5098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fba0effd640 1 --2- 172.21.15.33:0/1396534957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 0x5603ae4fd470 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fba00004650 tx=0x7fba0002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70986fbc8d3f7dba server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fb9f4ff9640 1 -- 172.21.15.33:0/1396534957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00044040 con 0x5603ae4e82c0 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fb9f4ff9640 1 -- 172.21.15.33:0/1396534957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fba00004410 con 0x5603ae4e82c0 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.058+0000 7fb9f4ff9640 1 -- 172.21.15.33:0/1396534957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00004920 con 0x5603ae4e82c0 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 -- 172.21.15.33:0/1396534957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 msgr2=0x5603ae4fd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 --2- 172.21.15.33:0/1396534957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 0x5603ae4fd470 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fba00004650 tx=0x7fba0002bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 -- 172.21.15.33:0/1396534957 shutdown_connections 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 --2- 172.21.15.33:0/1396534957 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4f17c0 0x5603ae5098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 --2- 172.21.15.33:0/1396534957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4e8be0 0x5603ae4e4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 --2- 172.21.15.33:0/1396534957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 0x5603ae4fd470 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 -- 172.21.15.33:0/1396534957 >> 172.21.15.33:0/1396534957 conn(0x5603ae4e29f0 msgr2=0x5603ae4e3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 -- 172.21.15.33:0/1396534957 shutdown_connections 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 -- 172.21.15.33:0/1396534957 wait complete. 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 -- start start 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 Processor -- start 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 0x5603ae4f6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4e8be0 0x5603ae4f6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4f17c0 0x5603ae4f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba101ad640 con 0x5603ae4e82c0 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fba101aeab0 con 0x5603ae4f17c0 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fba101add10 con 0x5603ae4e8be0 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba1838e740 10 client.0 _ll_register_callbacks cb 0x7fba08009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba0effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 0x5603ae4f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba0effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 0x5603ae4f6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37968/0 (socket says 172.21.15.33:37968) 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.059+0000 7fba0effd640 1 -- 172.21.15.33:0/2926129583 learned_addr learned my addr 172.21.15.33:0/2926129583 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fba0f7fe640 1 --2- 172.21.15.33:0/2926129583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4f17c0 0x5603ae4f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fba0effd640 1 -- 172.21.15.33:0/2926129583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4e8be0 msgr2=0x5603ae4f6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fba0effd640 1 --2- 172.21.15.33:0/2926129583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4e8be0 0x5603ae4f6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fba0effd640 1 -- 172.21.15.33:0/2926129583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4f17c0 msgr2=0x5603ae4f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fba0effd640 1 --2- 172.21.15.33:0/2926129583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4f17c0 0x5603ae4f6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fba0effd640 1 -- 172.21.15.33:0/2926129583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ae56c540 con 0x5603ae4e82c0 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fba0effd640 1 --2- 172.21.15.33:0/2926129583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 0x5603ae4f6310 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fba000055c0 tx=0x7fba000055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21615341f3bd3af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fb9f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fb9f2ff5640 1 -- 172.21.15.33:0/2926129583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0005b060 con 0x5603ae4e82c0 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fb9f2ff5640 1 -- 172.21.15.33:0/2926129583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fba0002c5d0 con 0x5603ae4e82c0 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fba1838e740 1 -- 172.21.15.33:0/2926129583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba00004410 con 0x5603ae4e82c0 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fb9f2ff5640 1 -- 172.21.15.33:0/2926129583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fba00062050 con 0x5603ae4e82c0 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fba1838e740 10 client.17649 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fba1838e740 1 -- 172.21.15.33:0/2926129583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5603ae56d730 con 0x5603ae4e82c0 2024-04-02T13:48:05.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.060+0000 7fb9f2ff5640 1 -- 172.21.15.33:0/2926129583 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fba00004410 con 0x5603ae4e82c0 2024-04-02T13:48:05.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.061+0000 7fb9f2ff5640 1 -- 172.21.15.33:0/2926129583 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fba0004e180 con 0x5603ae4e82c0 2024-04-02T13:48:05.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.061+0000 7fba1838e740 10 client.17649 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:05.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.061+0000 7fba1838e740 10 client.17649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:05.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.061+0000 7fba1838e740 20 client.17649 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:05.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.061+0000 7fba1838e740 1 --2- 172.21.15.33:0/2926129583 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5603ae56da10 0x5603ae56fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.061+0000 7fba1838e740 4 client.17649 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:48:05.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.061+0000 7fba1838e740 1 -- 172.21.15.33:0/2926129583 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5603ae454000 con 0x5603ae56da10 2024-04-02T13:48:05.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.061+0000 7fba0e7fc640 1 --2- 172.21.15.33:0/2926129583 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5603ae56da10 0x5603ae56fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.061+0000 7fb9f2ff5640 10 client.17649 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:05.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.061+0000 7fba0e7fc640 1 --2- 172.21.15.33:0/2926129583 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5603ae56da10 0x5603ae56fe00 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c7173658705eda server_cookie=69351cc897169f9e in_seq=0 out_seq=0 2024-04-02T13:48:05.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.062+0000 7fb9f2ff5640 1 -- 172.21.15.33:0/2926129583 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5603ae454000 con 0x5603ae56da10 2024-04-02T13:48:05.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.062+0000 7fb9f2ff5640 10 client.17649 handle_command_reply: tid=0 2024-04-02T13:48:05.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.062+0000 7fba1838e740 1 client.17649 shutdown 2024-04-02T13:48:05.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.062+0000 7fba1838e740 1 -- 172.21.15.33:0/2926129583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 msgr2=0x5603ae4f6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.062+0000 7fba1838e740 1 --2- 172.21.15.33:0/2926129583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 0x5603ae4f6310 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fba000055c0 tx=0x7fba000055f0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba1838e740 1 -- 172.21.15.33:0/2926129583 shutdown_connections 2024-04-02T13:48:05.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba1838e740 1 --2- 172.21.15.33:0/2926129583 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5603ae56da10 0x5603ae56fe00 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba1838e740 1 --2- 172.21.15.33:0/2926129583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5603ae4f17c0 0x5603ae4f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fb9f2ff5640 0 client.17649 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:05.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba1838e740 1 --2- 172.21.15.33:0/2926129583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5603ae4e8be0 0x5603ae4f6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba1838e740 1 --2- 172.21.15.33:0/2926129583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5603ae4e82c0 0x5603ae4f6310 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba1838e740 1 -- 172.21.15.33:0/2926129583 >> 172.21.15.33:0/2926129583 conn(0x5603ae4e29f0 msgr2=0x5603ae4e46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:05.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba1838e740 1 -- 172.21.15.33:0/2926129583 shutdown_connections 2024-04-02T13:48:05.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba1838e740 1 -- 172.21.15.33:0/2926129583 wait complete. 2024-04-02T13:48:05.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba1838e740 20 client.17649 trim_cache size 0 max 16384 2024-04-02T13:48:05.072 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:48:05.072 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:48:05.072 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "0f2363cd-3e66-4424-ada4-c377f3a65f0a", 2024-04-02T13:48:05.072 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:48:05.072 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:48:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba156d3640 1 -- 172.21.15.33:0/332999488 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb9f80a3d20 msgr2=0x7fb9f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba156d3640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb9f80a3d20 0x7fb9f80a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba08009fd0 tx=0x7fba08009500 comp rx=0 tx=0).stop 2024-04-02T13:48:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba156d3640 1 -- 172.21.15.33:0/332999488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba10101370 msgr2=0x7fba10199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba156d3640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba10101370 0x7fba10199680 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c920 tx=0x7fba0400cde0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba156d3640 1 -- 172.21.15.33:0/332999488 shutdown_connections 2024-04-02T13:48:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba156d3640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb9f80a3d20 0x7fb9f80a61e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba156d3640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba101069c0 0x7fba101a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba156d3640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba10101370 0x7fba10199680 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba156d3640 1 --2- 172.21.15.33:0/332999488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba101009b0 0x7fba10199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba156d3640 1 -- 172.21.15.33:0/332999488 >> 172.21.15.33:0/332999488 conn(0x7fba10076d80 msgr2=0x7fba100fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:05.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.063+0000 7fba156d3640 1 -- 172.21.15.33:0/332999488 shutdown_connections 2024-04-02T13:48:05.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.064+0000 7fba156d3640 1 -- 172.21.15.33:0/332999488 wait complete. 2024-04-02T13:48:05.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:48:05.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:48:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.179+0000 7f8ae5722640 1 -- 172.21.15.33:0/1150231165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ae01009b0 msgr2=0x7f8ae0100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.179+0000 7f8ae5722640 1 --2- 172.21.15.33:0/1150231165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ae01009b0 0x7f8ae0100e30 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009930 tx=0x7f8ac802bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.179+0000 7f8ae5722640 1 -- 172.21.15.33:0/1150231165 shutdown_connections 2024-04-02T13:48:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.179+0000 7f8ae5722640 1 --2- 172.21.15.33:0/1150231165 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae0101370 0x7f8ae010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.179+0000 7f8ae5722640 1 --2- 172.21.15.33:0/1150231165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ae01009b0 0x7f8ae0100e30 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.179+0000 7f8ae5722640 1 --2- 172.21.15.33:0/1150231165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ae01069c0 0x7f8ae0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.179+0000 7f8ae5722640 1 -- 172.21.15.33:0/1150231165 >> 172.21.15.33:0/1150231165 conn(0x7f8ae0076d80 msgr2=0x7f8ae0077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.179+0000 7f8ae5722640 1 -- 172.21.15.33:0/1150231165 shutdown_connections 2024-04-02T13:48:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.179+0000 7f8ae5722640 1 -- 172.21.15.33:0/1150231165 wait complete. 2024-04-02T13:48:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.179+0000 7f8ae5722640 1 Processor -- start 2024-04-02T13:48:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.179+0000 7f8ae5722640 1 -- start start 2024-04-02T13:48:05.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8ae5722640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ae01009b0 0x7f8ae0197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8ae5722640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae0101370 0x7f8ae0197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8ae5722640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ae01069c0 0x7f8ae0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8ae5722640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae0110da0 con 0x7f8ae01069c0 2024-04-02T13:48:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8ae5722640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae0110c20 con 0x7f8ae01009b0 2024-04-02T13:48:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8ae5722640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae0110f20 con 0x7f8ae0101370 2024-04-02T13:48:05.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8adf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ae01069c0 0x7f8ae0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8adf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ae01069c0 0x7f8ae0197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37990/0 (socket says 172.21.15.33:37990) 2024-04-02T13:48:05.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8adf7fe640 1 -- 172.21.15.33:0/3162256015 learned_addr learned my addr 172.21.15.33:0/3162256015 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:05.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae0101370 0x7f8ae0197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8adeffd640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ae01009b0 0x7f8ae0197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8adf7fe640 1 -- 172.21.15.33:0/3162256015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae0101370 msgr2=0x7f8ae0197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8adf7fe640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae0101370 0x7f8ae0197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8adf7fe640 1 -- 172.21.15.33:0/3162256015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ae01009b0 msgr2=0x7f8ae0197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8adf7fe640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ae01009b0 0x7f8ae0197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8adf7fe640 1 -- 172.21.15.33:0/3162256015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae01a6830 con 0x7f8ae01069c0 2024-04-02T13:48:05.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8adf7fe640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ae01069c0 0x7f8ae0197ad0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800eb40 tx=0x7f8ad800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:05.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8adcff9640 1 -- 172.21.15.33:0/3162256015 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8004b20 con 0x7f8ae01069c0 2024-04-02T13:48:05.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8ae5722640 1 -- 172.21.15.33:0/3162256015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae01a6b20 con 0x7f8ae01069c0 2024-04-02T13:48:05.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8adcff9640 1 -- 172.21.15.33:0/3162256015 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8ad8004cc0 con 0x7f8ae01069c0 2024-04-02T13:48:05.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8adcff9640 1 -- 172.21.15.33:0/3162256015 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8010690 con 0x7f8ae01069c0 2024-04-02T13:48:05.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.180+0000 7f8ae5722640 1 -- 172.21.15.33:0/3162256015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae01a7040 con 0x7f8ae01069c0 2024-04-02T13:48:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8ae83df740 1 Processor -- start 2024-04-02T13:48:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8adcff9640 1 -- 172.21.15.33:0/3162256015 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8ad8005580 con 0x7f8ae01069c0 2024-04-02T13:48:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8ae83df740 1 -- start start 2024-04-02T13:48:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8ae83df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdcfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8ae83df740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 0x5620afdd8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8ae83df740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 0x5620afddb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8ae83df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5620afd33830 con 0x5620afdcf8d0 2024-04-02T13:48:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8ae83df740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5620afcdb820 con 0x5620afdd01f0 2024-04-02T13:48:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8ae83df740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5620afd2f7e0 con 0x5620afdd8dd0 2024-04-02T13:48:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdcfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdcfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37994/0 (socket says 172.21.15.33:37994) 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8ad7fff640 1 -- 172.21.15.33:0/2708847607 learned_addr learned my addr 172.21.15.33:0/2708847607 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8adcff9640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8acc0a3d20 0x7f8acc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8adeffd640 1 --2- 172.21.15.33:0/2708847607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 0x5620afdd8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.183+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/2708847607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 0x5620afddb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.184+0000 7f8adf7fe640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8acc0a3d20 0x7f8acc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.184+0000 7f8ad7fff640 1 -- 172.21.15.33:0/2708847607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 msgr2=0x5620afddb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.184+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/2708847607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 0x5620afddb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.184+0000 7f8ad7fff640 1 -- 172.21.15.33:0/2708847607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 msgr2=0x5620afdd8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.184+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/2708847607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 0x5620afdd8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.184+0000 7f8ad7fff640 1 -- 172.21.15.33:0/2708847607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620afb117d0 con 0x5620afdcf8d0 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.184+0000 7f8adcff9640 1 -- 172.21.15.33:0/3162256015 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f8ad80d1150 con 0x7f8ae01069c0 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.184+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/2708847607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdcfcb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8005ab0 tx=0x7f8ac8005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f26d14f9e76ce server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.184+0000 7f8ad57fa640 1 -- 172.21.15.33:0/2708847607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8047020 con 0x5620afdcf8d0 2024-04-02T13:48:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.184+0000 7f8adf7fe640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8acc0a3d20 0x7f8acc0a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ad809ca20 tx=0x7f8ad806e760 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.184+0000 7f8ad57fa640 1 -- 172.21.15.33:0/2708847607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8ac8005510 con 0x5620afdcf8d0 2024-04-02T13:48:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.184+0000 7f8ad57fa640 1 -- 172.21.15.33:0/2708847607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac802cc90 con 0x5620afdcf8d0 2024-04-02T13:48:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 -- 172.21.15.33:0/2708847607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 msgr2=0x5620afdcfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 --2- 172.21.15.33:0/2708847607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdcfcb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8005ab0 tx=0x7f8ac8005df0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 -- 172.21.15.33:0/2708847607 shutdown_connections 2024-04-02T13:48:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 --2- 172.21.15.33:0/2708847607 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 0x5620afddb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 --2- 172.21.15.33:0/2708847607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 0x5620afdd8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 --2- 172.21.15.33:0/2708847607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdcfcb0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 -- 172.21.15.33:0/2708847607 >> 172.21.15.33:0/2708847607 conn(0x5620afdca000 msgr2=0x5620afdca3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 -- 172.21.15.33:0/2708847607 shutdown_connections 2024-04-02T13:48:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 -- 172.21.15.33:0/2708847607 wait complete. 2024-04-02T13:48:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 -- start start 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 Processor -- start 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdcf8d0 0x5620afdd8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd01f0 0x5620afdd59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdd8dd0 0x5620afdd5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5620afddbb60 con 0x5620afdd8dd0 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5620afceed30 con 0x5620afdcf8d0 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.185+0000 7f8ae83df740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5620afddbce0 con 0x5620afdd01f0 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ae83df740 10 client.0 _ll_register_callbacks cb 0x5620afcee720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdd8dd0 0x5620afdd5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdd8dd0 0x5620afdd5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37998/0 (socket says 172.21.15.33:37998) 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ad7fff640 1 -- 172.21.15.33:0/1235323849 learned_addr learned my addr 172.21.15.33:0/1235323849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/1235323849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdcf8d0 0x5620afdd8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8adeffd640 1 --2- 172.21.15.33:0/1235323849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd01f0 0x5620afdd59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ad7fff640 1 -- 172.21.15.33:0/1235323849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd01f0 msgr2=0x5620afdd59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/1235323849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd01f0 0x5620afdd59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ad7fff640 1 -- 172.21.15.33:0/1235323849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdcf8d0 msgr2=0x5620afdd8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/1235323849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdcf8d0 0x5620afdd8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ad7fff640 1 -- 172.21.15.33:0/1235323849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620afa473f0 con 0x5620afdd8dd0 2024-04-02T13:48:05.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/1235323849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdd8dd0 0x5620afdd5ee0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8000c00 tx=0x7f8ac8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b7984df0024773 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:05.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ac67fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:05.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ac67fc640 1 -- 172.21.15.33:0/1235323849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8047070 con 0x5620afdd8dd0 2024-04-02T13:48:05.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ac67fc640 1 -- 172.21.15.33:0/1235323849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f8ac8005250 con 0x5620afdd8dd0 2024-04-02T13:48:05.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ae83df740 1 -- 172.21.15.33:0/1235323849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac8005510 con 0x5620afdd8dd0 2024-04-02T13:48:05.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ac67fc640 1 -- 172.21.15.33:0/1235323849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8ac80046c0 con 0x5620afdd8dd0 2024-04-02T13:48:05.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ae83df740 10 client.17661 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:05.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.186+0000 7f8ae83df740 1 -- 172.21.15.33:0/1235323849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5620afa070f0 con 0x5620afdd8dd0 2024-04-02T13:48:05.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.187+0000 7f8ac67fc640 1 -- 172.21.15.33:0/1235323849 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8ac8005510 con 0x5620afdd8dd0 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.187+0000 7f8ac67fc640 1 -- 172.21.15.33:0/1235323849 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f8ac8040040 con 0x5620afdd8dd0 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.187+0000 7f8ae83df740 10 client.17661 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.187+0000 7f8ae83df740 10 client.17661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.187+0000 7f8ae83df740 20 client.17661 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.187+0000 7f8ae83df740 1 --2- 172.21.15.33:0/1235323849 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5620afde8b40 0x5620afdeaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.187+0000 7f8ae83df740 4 client.17661 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.187+0000 7f8ae83df740 1 -- 172.21.15.33:0/1235323849 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620afd40d70 con 0x5620afde8b40 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.187+0000 7f8adeffd640 1 --2- 172.21.15.33:0/1235323849 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5620afde8b40 0x5620afdeaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ac67fc640 10 client.17661 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8adeffd640 1 --2- 172.21.15.33:0/1235323849 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5620afde8b40 0x5620afdeaf30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c00e98bc7c63afd5 server_cookie=a1f999a78bc9d441 in_seq=0 out_seq=0 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ac67fc640 1 -- 172.21.15.33:0/1235323849 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620afd40d70 con 0x5620afde8b40 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ac67fc640 10 client.17661 handle_command_reply: tid=0 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ae83df740 1 client.17661 shutdown 2024-04-02T13:48:05.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ae83df740 1 -- 172.21.15.33:0/1235323849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdd8dd0 msgr2=0x5620afdd5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ae83df740 1 --2- 172.21.15.33:0/1235323849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdd8dd0 0x5620afdd5ee0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8000c00 tx=0x7f8ac8005b50 comp rx=0 tx=0).stop 2024-04-02T13:48:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ae83df740 1 -- 172.21.15.33:0/1235323849 shutdown_connections 2024-04-02T13:48:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ae83df740 1 --2- 172.21.15.33:0/1235323849 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5620afde8b40 0x5620afdeaf30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ae83df740 1 --2- 172.21.15.33:0/1235323849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdd8dd0 0x5620afdd5ee0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ac67fc640 0 client.17661 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ae83df740 1 --2- 172.21.15.33:0/1235323849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd01f0 0x5620afdd59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ae83df740 1 --2- 172.21.15.33:0/1235323849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdcf8d0 0x5620afdd8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ae83df740 1 -- 172.21.15.33:0/1235323849 >> 172.21.15.33:0/1235323849 conn(0x5620afdca000 msgr2=0x5620afdcbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.188+0000 7f8ae83df740 1 -- 172.21.15.33:0/1235323849 shutdown_connections 2024-04-02T13:48:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.189+0000 7f8ae83df740 1 -- 172.21.15.33:0/1235323849 wait complete. 2024-04-02T13:48:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.189+0000 7f8ae83df740 20 client.17661 trim_cache size 0 max 16384 2024-04-02T13:48:05.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ae83df740 1 Processor -- start 2024-04-02T13:48:05.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ae83df740 1 -- start start 2024-04-02T13:48:05.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ae83df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdd0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ae83df740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 0x5620afdd80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ae83df740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 0x5620afdd8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ae83df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae01b3a90 con 0x5620afdcf8d0 2024-04-02T13:48:05.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ae83df740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae01b1360 con 0x5620afdd01f0 2024-04-02T13:48:05.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ae83df740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae01af1d0 con 0x5620afdd8dd0 2024-04-02T13:48:05.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdd0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdd0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38012/0 (socket says 172.21.15.33:38012) 2024-04-02T13:48:05.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ad7fff640 1 -- 172.21.15.33:0/2969541156 learned_addr learned my addr 172.21.15.33:0/2969541156 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:05.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8adeffd640 1 --2- 172.21.15.33:0/2969541156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 0x5620afdd80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ad7fff640 1 -- 172.21.15.33:0/2969541156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 msgr2=0x5620afdd8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/2969541156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 0x5620afdd8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ad7fff640 1 -- 172.21.15.33:0/2969541156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 msgr2=0x5620afdd80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/2969541156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 0x5620afdd80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8ad7fff640 1 -- 172.21.15.33:0/2969541156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620afdd0020 con 0x5620afdcf8d0 2024-04-02T13:48:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.206+0000 7f8adeffd640 1 --2- 172.21.15.33:0/2969541156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 0x5620afdd80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.207+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/2969541156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdd0fa0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8ac802c170 tx=0x7f8ac8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73821ba5f7c3aeb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.207+0000 7f8ad4ff9640 1 -- 172.21.15.33:0/2969541156 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8047030 con 0x5620afdcf8d0 2024-04-02T13:48:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.207+0000 7f8ad4ff9640 1 -- 172.21.15.33:0/2969541156 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8ac8052040 con 0x5620afdcf8d0 2024-04-02T13:48:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.207+0000 7f8ad4ff9640 1 -- 172.21.15.33:0/2969541156 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8004860 con 0x5620afdcf8d0 2024-04-02T13:48:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.207+0000 7f8ae83df740 1 -- 172.21.15.33:0/2969541156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 msgr2=0x5620afdd0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.207+0000 7f8ae83df740 1 --2- 172.21.15.33:0/2969541156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdd0fa0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8ac802c170 tx=0x7f8ac8005850 comp rx=0 tx=0).stop 2024-04-02T13:48:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.207+0000 7f8ae83df740 1 -- 172.21.15.33:0/2969541156 shutdown_connections 2024-04-02T13:48:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.207+0000 7f8ae83df740 1 --2- 172.21.15.33:0/2969541156 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 0x5620afdd8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.207+0000 7f8ae83df740 1 --2- 172.21.15.33:0/2969541156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 0x5620afdd80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.207+0000 7f8ae83df740 1 --2- 172.21.15.33:0/2969541156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdd0fa0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.207+0000 7f8ae83df740 1 -- 172.21.15.33:0/2969541156 >> 172.21.15.33:0/2969541156 conn(0x5620afdca000 msgr2=0x5620afdd48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ae83df740 1 -- 172.21.15.33:0/2969541156 shutdown_connections 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ae83df740 1 -- 172.21.15.33:0/2969541156 wait complete. 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ae83df740 1 -- start start 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ae83df740 1 Processor -- start 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ae83df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ae83df740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 0x5620afdeb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ae83df740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 0x5620afdeba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ae83df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae01af820 con 0x5620afdcf8d0 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ae83df740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae01b0c90 con 0x5620afdd01f0 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ae83df740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae01afef0 con 0x5620afdd8dd0 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ae83df740 10 client.0 _ll_register_callbacks cb 0x7f8ac802c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdee0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38028/0 (socket says 172.21.15.33:38028) 2024-04-02T13:48:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ad7fff640 1 -- 172.21.15.33:0/3220475360 learned_addr learned my addr 172.21.15.33:0/3220475360 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/3220475360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 0x5620afdeba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8adeffd640 1 --2- 172.21.15.33:0/3220475360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 0x5620afdeb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ad7fff640 1 -- 172.21.15.33:0/3220475360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 msgr2=0x5620afdeba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/3220475360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 0x5620afdeba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ad7fff640 1 -- 172.21.15.33:0/3220475360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 msgr2=0x5620afdeb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/3220475360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 0x5620afdeb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.208+0000 7f8ad7fff640 1 -- 172.21.15.33:0/3220475360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620afdec050 con 0x5620afdcf8d0 2024-04-02T13:48:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ad7fff640 1 --2- 172.21.15.33:0/3220475360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdee0d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8005bd0 tx=0x7f8ac802c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9534f809bc3f357c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ac67fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ac67fc640 1 -- 172.21.15.33:0/3220475360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8047040 con 0x5620afdcf8d0 2024-04-02T13:48:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ac67fc640 1 -- 172.21.15.33:0/3220475360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f8ac8003dc0 con 0x5620afdcf8d0 2024-04-02T13:48:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ae83df740 1 -- 172.21.15.33:0/3220475360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac8052040 con 0x5620afdcf8d0 2024-04-02T13:48:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ac67fc640 1 -- 172.21.15.33:0/3220475360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8ac80035f0 con 0x5620afdcf8d0 2024-04-02T13:48:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ae83df740 10 client.17667 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ae83df740 1 -- 172.21.15.33:0/3220475360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5620afe4ea70 con 0x5620afdcf8d0 2024-04-02T13:48:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ac67fc640 1 -- 172.21.15.33:0/3220475360 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8ac8052040 con 0x5620afdcf8d0 2024-04-02T13:48:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ac67fc640 1 -- 172.21.15.33:0/3220475360 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f8ac805b480 con 0x5620afdcf8d0 2024-04-02T13:48:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ae83df740 10 client.17667 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ae83df740 10 client.17667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ae83df740 20 client.17667 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:05.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ae83df740 1 --2- 172.21.15.33:0/3220475360 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5620afe4efb0 0x5620afe513d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:05.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ae83df740 4 client.17667 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:48:05.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.209+0000 7f8ae83df740 1 -- 172.21.15.33:0/3220475360 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620afd3b0c0 con 0x5620afe4efb0 2024-04-02T13:48:05.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8adeffd640 1 --2- 172.21.15.33:0/3220475360 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5620afe4efb0 0x5620afe513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:05.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ac67fc640 10 client.17667 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:05.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8adeffd640 1 --2- 172.21.15.33:0/3220475360 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5620afe4efb0 0x5620afe513d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ddee316b0a2c9f1 server_cookie=23ce9d33942b736a in_seq=0 out_seq=0 2024-04-02T13:48:05.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ac67fc640 1 -- 172.21.15.33:0/3220475360 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5620afd3b0c0 con 0x5620afe4efb0 2024-04-02T13:48:05.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ac67fc640 10 client.17667 handle_command_reply: tid=0 2024-04-02T13:48:05.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ae83df740 1 client.17667 shutdown 2024-04-02T13:48:05.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ae83df740 1 -- 172.21.15.33:0/3220475360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 msgr2=0x5620afdee0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ae83df740 1 --2- 172.21.15.33:0/3220475360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdee0d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8005bd0 tx=0x7f8ac802c750 comp rx=0 tx=0).stop 2024-04-02T13:48:05.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ae83df740 1 -- 172.21.15.33:0/3220475360 shutdown_connections 2024-04-02T13:48:05.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ae83df740 1 --2- 172.21.15.33:0/3220475360 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5620afe4efb0 0x5620afe513d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ae83df740 1 --2- 172.21.15.33:0/3220475360 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5620afdd8dd0 0x5620afdeba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ae83df740 1 --2- 172.21.15.33:0/3220475360 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620afdd01f0 0x5620afdeb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ac67fc640 0 client.17667 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:05.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ae83df740 1 --2- 172.21.15.33:0/3220475360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620afdcf8d0 0x5620afdee0d0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ae83df740 1 -- 172.21.15.33:0/3220475360 >> 172.21.15.33:0/3220475360 conn(0x5620afdca000 msgr2=0x5620afdcbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:05.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ae83df740 1 -- 172.21.15.33:0/3220475360 shutdown_connections 2024-04-02T13:48:05.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.210+0000 7f8ae83df740 1 -- 172.21.15.33:0/3220475360 wait complete. 2024-04-02T13:48:05.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.211+0000 7f8ae83df740 20 client.17667 trim_cache size 0 max 16384 2024-04-02T13:48:05.224 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:48:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:48:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:48:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "0f2363cd-3e66-4424-ada4-c377f3a65f0a": { 2024-04-02T13:48:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:48:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "0f2363cd-3e66-4424-ada4-c377f3a65f0a", 2024-04-02T13:48:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:48:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:48:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:48:05.225 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:48:05.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.211+0000 7f8ae5722640 1 -- 172.21.15.33:0/3162256015 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8acc0a3d20 msgr2=0x7f8acc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.211+0000 7f8ae5722640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8acc0a3d20 0x7f8acc0a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ad809ca20 tx=0x7f8ad806e760 comp rx=0 tx=0).stop 2024-04-02T13:48:05.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.211+0000 7f8ae5722640 1 -- 172.21.15.33:0/3162256015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ae01069c0 msgr2=0x7f8ae0197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.211+0000 7f8ae5722640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ae01069c0 0x7f8ae0197ad0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800eb40 tx=0x7f8ad800c2d0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.211+0000 7f8ae5722640 1 -- 172.21.15.33:0/3162256015 shutdown_connections 2024-04-02T13:48:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.211+0000 7f8ae5722640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8acc0a3d20 0x7f8acc0a61e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.211+0000 7f8ae5722640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ae01069c0 0x7f8ae0197ad0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.211+0000 7f8ae5722640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae0101370 0x7f8ae0197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.211+0000 7f8ae5722640 1 --2- 172.21.15.33:0/3162256015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ae01009b0 0x7f8ae0197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.211+0000 7f8ae5722640 1 -- 172.21.15.33:0/3162256015 >> 172.21.15.33:0/3162256015 conn(0x7f8ae0076d80 msgr2=0x7f8ae00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.211+0000 7f8ae5722640 1 -- 172.21.15.33:0/3162256015 shutdown_connections 2024-04-02T13:48:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:05.211+0000 7f8ae5722640 1 -- 172.21.15.33:0/3162256015 wait complete. 2024-04-02T13:48:05.227 INFO:tasks.cephfs.filesystem:scrub status for tag:0f2363cd-3e66-4424-ada4-c377f3a65f0a - {'path': '/', 'tag': '0f2363cd-3e66-4424-ada4-c377f3a65f0a', 'options': 'recursive,force'} 2024-04-02T13:48:06.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:06 smithi145 ceph-mon[35646]: pgmap v288: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-02T13:48:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:06 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:48:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:06 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:48:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:06 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:06 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:48:06.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:06 smithi145 ceph-mon[35646]: osdmap e102: 12 total, 12 up, 12 in 2024-04-02T13:48:06.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:06 smithi050 ceph-mon[35524]: pgmap v288: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-02T13:48:06.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:06 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:48:06.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:06.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:06 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:48:06.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:06 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:06.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:06 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:48:06.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:06 smithi050 ceph-mon[35524]: osdmap e102: 12 total, 12 up, 12 in 2024-04-02T13:48:07.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:06 smithi033 ceph-mon[32390]: pgmap v288: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-02T13:48:07.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:06 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:48:07.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:07.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:06 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:48:07.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:06 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:07.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:06 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:48:07.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:06 smithi033 ceph-mon[32390]: osdmap e102: 12 total, 12 up, 12 in 2024-04-02T13:48:08.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:08 smithi145 ceph-mon[35646]: pgmap v290: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:48:08.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:08 smithi050 ceph-mon[35524]: pgmap v290: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:48:09.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:08 smithi033 ceph-mon[32390]: pgmap v290: 129 pgs: 129 active+clean; 145 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:48:10.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:10 smithi145 ceph-mon[35646]: pgmap v291: 129 pgs: 129 active+clean; 126 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-02T13:48:10.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:10 smithi050 ceph-mon[35524]: pgmap v291: 129 pgs: 129 active+clean; 126 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-02T13:48:11.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:10 smithi033 ceph-mon[32390]: pgmap v291: 129 pgs: 129 active+clean; 126 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-02T13:48:12.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:12 smithi145 ceph-mon[35646]: pgmap v292: 129 pgs: 129 active+clean; 126 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-02T13:48:12.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:12 smithi145 ceph-mon[35646]: osdmap e103: 12 total, 12 up, 12 in 2024-04-02T13:48:12.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:12 smithi050 ceph-mon[35524]: pgmap v292: 129 pgs: 129 active+clean; 126 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-02T13:48:12.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:12 smithi050 ceph-mon[35524]: osdmap e103: 12 total, 12 up, 12 in 2024-04-02T13:48:13.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:12 smithi033 ceph-mon[32390]: pgmap v292: 129 pgs: 129 active+clean; 126 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-02T13:48:13.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:12 smithi033 ceph-mon[32390]: osdmap e103: 12 total, 12 up, 12 in 2024-04-02T13:48:14.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:14 smithi145 ceph-mon[35646]: pgmap v294: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:48:14.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:14 smithi050 ceph-mon[35524]: pgmap v294: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:48:15.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:14 smithi033 ceph-mon[32390]: pgmap v294: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:48:16.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:16 smithi145 ceph-mon[35646]: pgmap v295: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 219 B/s wr, 8 op/s 2024-04-02T13:48:16.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:16 smithi050 ceph-mon[35524]: pgmap v295: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 219 B/s wr, 8 op/s 2024-04-02T13:48:17.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:16 smithi033 ceph-mon[32390]: pgmap v295: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 219 B/s wr, 8 op/s 2024-04-02T13:48:18.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:18 smithi145 ceph-mon[35646]: pgmap v296: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:48:18.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:18 smithi050 ceph-mon[35524]: pgmap v296: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:48:19.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:18 smithi033 ceph-mon[32390]: pgmap v296: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:48:20.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:20 smithi145 ceph-mon[35646]: pgmap v297: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:48:20.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:20 smithi050 ceph-mon[35524]: pgmap v297: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:48:21.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:20 smithi033 ceph-mon[32390]: pgmap v297: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:48:22.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:22 smithi145 ceph-mon[35646]: pgmap v298: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:48:22.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:22 smithi050 ceph-mon[35524]: pgmap v298: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:48:23.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:22 smithi033 ceph-mon[32390]: pgmap v298: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:48:24.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:24 smithi145 ceph-mon[35646]: pgmap v299: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 89 B/s wr, 9 op/s 2024-04-02T13:48:24.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:24 smithi050 ceph-mon[35524]: pgmap v299: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 89 B/s wr, 9 op/s 2024-04-02T13:48:25.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:24 smithi033 ceph-mon[32390]: pgmap v299: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 89 B/s wr, 9 op/s 2024-04-02T13:48:26.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:26 smithi145 ceph-mon[35646]: pgmap v300: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:26.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:26 smithi050 ceph-mon[35524]: pgmap v300: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:27.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:26 smithi033 ceph-mon[32390]: pgmap v300: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:28.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:28 smithi145 ceph-mon[35646]: pgmap v301: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:28.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:28 smithi050 ceph-mon[35524]: pgmap v301: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:29.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:28 smithi033 ceph-mon[32390]: pgmap v301: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:30.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:30 smithi145 ceph-mon[35646]: pgmap v302: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:48:30.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:30 smithi050 ceph-mon[35524]: pgmap v302: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:48:31.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:30 smithi033 ceph-mon[32390]: pgmap v302: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:48:32.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:32 smithi145 ceph-mon[35646]: pgmap v303: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:32 smithi050 ceph-mon[35524]: pgmap v303: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:33.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:32 smithi033 ceph-mon[32390]: pgmap v303: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:34.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:34 smithi145 ceph-mon[35646]: pgmap v304: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:48:34.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:34 smithi050 ceph-mon[35524]: pgmap v304: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:48:35.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:34 smithi033 ceph-mon[32390]: pgmap v304: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:48:35.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:48:35.229 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:48:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.344+0000 7fde9d642640 1 -- 172.21.15.33:0/2205846280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde98100970 msgr2=0x7fde98100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.344+0000 7fde9d642640 1 --2- 172.21.15.33:0/2205846280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde98100970 0x7fde98100df0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009930 tx=0x7fde8c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.344+0000 7fde9d642640 1 -- 172.21.15.33:0/2205846280 shutdown_connections 2024-04-02T13:48:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.344+0000 7fde9d642640 1 --2- 172.21.15.33:0/2205846280 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde98101330 0x7fde9810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.344+0000 7fde9d642640 1 --2- 172.21.15.33:0/2205846280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde98100970 0x7fde98100df0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.344+0000 7fde9d642640 1 --2- 172.21.15.33:0/2205846280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde98106980 0x7fde98106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.344+0000 7fde9d642640 1 -- 172.21.15.33:0/2205846280 >> 172.21.15.33:0/2205846280 conn(0x7fde98077b00 msgr2=0x7fde98077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.344+0000 7fde9d642640 1 -- 172.21.15.33:0/2205846280 shutdown_connections 2024-04-02T13:48:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.344+0000 7fde9d642640 1 -- 172.21.15.33:0/2205846280 wait complete. 2024-04-02T13:48:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.344+0000 7fde9d642640 1 Processor -- start 2024-04-02T13:48:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde9d642640 1 -- start start 2024-04-02T13:48:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde9d642640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde98100970 0x7fde98198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde9d642640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde98101330 0x7fde981988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde9d642640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde98106980 0x7fde9819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde9d642640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde98110d60 con 0x7fde98106980 2024-04-02T13:48:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde9d642640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde98110be0 con 0x7fde98101330 2024-04-02T13:48:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde9d642640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fde98110ee0 con 0x7fde98100970 2024-04-02T13:48:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde98106980 0x7fde9819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde98106980 0x7fde9819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53170/0 (socket says 172.21.15.33:53170) 2024-04-02T13:48:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde977fe640 1 -- 172.21.15.33:0/3293468906 learned_addr learned my addr 172.21.15.33:0/3293468906 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde96ffd640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde98100970 0x7fde98198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde967fc640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde98101330 0x7fde981988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde977fe640 1 -- 172.21.15.33:0/3293468906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde98100970 msgr2=0x7fde98198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde977fe640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde98100970 0x7fde98198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde977fe640 1 -- 172.21.15.33:0/3293468906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde98101330 msgr2=0x7fde981988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde977fe640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde98101330 0x7fde981988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde977fe640 1 -- 172.21.15.33:0/3293468906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9819d360 con 0x7fde98106980 2024-04-02T13:48:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde967fc640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde98101330 0x7fde981988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.345+0000 7fde977fe640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde98106980 0x7fde9819cc30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fde9000eaa0 tx=0x7fde9000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.346+0000 7fde7ffff640 1 -- 172.21.15.33:0/3293468906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9000cdb0 con 0x7fde98106980 2024-04-02T13:48:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.346+0000 7fde7ffff640 1 -- 172.21.15.33:0/3293468906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde90005020 con 0x7fde98106980 2024-04-02T13:48:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.346+0000 7fde9d642640 1 -- 172.21.15.33:0/3293468906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9819d650 con 0x7fde98106980 2024-04-02T13:48:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.346+0000 7fde7ffff640 1 -- 172.21.15.33:0/3293468906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde900116b0 con 0x7fde98106980 2024-04-02T13:48:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.346+0000 7fde9d642640 1 -- 172.21.15.33:0/3293468906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde981a4ee0 con 0x7fde98106980 2024-04-02T13:48:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.348+0000 7fde7ffff640 1 -- 172.21.15.33:0/3293468906 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fde900051d0 con 0x7fde98106980 2024-04-02T13:48:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.348+0000 7fdea021f740 1 Processor -- start 2024-04-02T13:48:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fdea021f740 1 -- start start 2024-04-02T13:48:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fdea021f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d068d0 0x558c93d06cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fdea021f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 0x558c93d0f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fdea021f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d0fdd0 0x558c93d121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde7ffff640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fde800a3d20 0x7fde800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fdea021f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558c93c6a830 con 0x558c93d071f0 2024-04-02T13:48:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fdea021f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558c93c12820 con 0x558c93d068d0 2024-04-02T13:48:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fdea021f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558c93c667e0 con 0x558c93d0fdd0 2024-04-02T13:48:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 0x558c93d0f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 0x558c93d0f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53180/0 (socket says 172.21.15.33:53180) 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde967fc640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fde800a3d20 0x7fde800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde96ffd640 1 -- 172.21.15.33:0/516042928 learned_addr learned my addr 172.21.15.33:0/516042928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde7ffff640 1 -- 172.21.15.33:0/3293468906 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fde900d0ea0 con 0x7fde98106980 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde977fe640 1 --2- 172.21.15.33:0/516042928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d0fdd0 0x558c93d121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde967fc640 1 --2- 172.21.15.33:0/516042928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d068d0 0x558c93d06cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde96ffd640 1 -- 172.21.15.33:0/516042928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d0fdd0 msgr2=0x558c93d121c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde96ffd640 1 --2- 172.21.15.33:0/516042928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d0fdd0 0x558c93d121c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde96ffd640 1 -- 172.21.15.33:0/516042928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d068d0 msgr2=0x558c93d06cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde96ffd640 1 --2- 172.21.15.33:0/516042928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d068d0 0x558c93d06cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde96ffd640 1 -- 172.21.15.33:0/516042928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c93a487d0 con 0x558c93d071f0 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.349+0000 7fde977fe640 1 --2- 172.21.15.33:0/516042928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d0fdd0 0x558c93d121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.350+0000 7fde967fc640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fde800a3d20 0x7fde800a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fde8c006010 tx=0x7fde8c0040c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.350+0000 7fde96ffd640 1 --2- 172.21.15.33:0/516042928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 0x558c93d0f890 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fde88009c70 tx=0x7fde88011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93db9c431e73d793 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.350+0000 7fde7d7fa640 1 -- 172.21.15.33:0/516042928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88004230 con 0x558c93d071f0 2024-04-02T13:48:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.350+0000 7fde7d7fa640 1 -- 172.21.15.33:0/516042928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde880043d0 con 0x558c93d071f0 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.350+0000 7fde7d7fa640 1 -- 172.21.15.33:0/516042928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde880196d0 con 0x558c93d071f0 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.350+0000 7fdea021f740 1 -- 172.21.15.33:0/516042928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 msgr2=0x558c93d0f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.350+0000 7fdea021f740 1 --2- 172.21.15.33:0/516042928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 0x558c93d0f890 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fde88009c70 tx=0x7fde88011810 comp rx=0 tx=0).stop 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 -- 172.21.15.33:0/516042928 shutdown_connections 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 --2- 172.21.15.33:0/516042928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d0fdd0 0x558c93d121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 --2- 172.21.15.33:0/516042928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 0x558c93d0f890 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 --2- 172.21.15.33:0/516042928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d068d0 0x558c93d06cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 -- 172.21.15.33:0/516042928 >> 172.21.15.33:0/516042928 conn(0x558c93d01000 msgr2=0x558c93d013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 -- 172.21.15.33:0/516042928 shutdown_connections 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 -- 172.21.15.33:0/516042928 wait complete. 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 -- start start 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 Processor -- start 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d068d0 0x558c93d0f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 0x558c93d0c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d0cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558c93d12bc0 con 0x558c93d071f0 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558c93c25d30 con 0x558c93d0fdd0 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558c93d12d40 con 0x558c93d068d0 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fdea021f740 10 client.0 _ll_register_callbacks cb 0x558c93c25720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fde967fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 0x558c93d0c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fde967fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 0x558c93d0c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53190/0 (socket says 172.21.15.33:53190) 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fde967fc640 1 -- 172.21.15.33:0/643029252 learned_addr learned my addr 172.21.15.33:0/643029252 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.351+0000 7fde977fe640 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d0cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fde96ffd640 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d068d0 0x558c93d0f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fde967fc640 1 -- 172.21.15.33:0/643029252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d068d0 msgr2=0x558c93d0f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fde967fc640 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d068d0 0x558c93d0f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fde967fc640 1 -- 172.21.15.33:0/643029252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 msgr2=0x558c93d0cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fde967fc640 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d0cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fde967fc640 1 -- 172.21.15.33:0/643029252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c9397e3f0 con 0x558c93d071f0 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fde977fe640 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d0cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fde967fc640 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 0x558c93d0c9c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fde8c035000 tx=0x7fde8c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1445a6f180ba3b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fde7aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fde7aff5640 1 -- 172.21.15.33:0/643029252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c055060 con 0x558c93d071f0 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fde7aff5640 1 -- 172.21.15.33:0/643029252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fde8c004ad0 con 0x558c93d071f0 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fdea021f740 1 -- 172.21.15.33:0/643029252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde880043d0 con 0x558c93d071f0 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fde7aff5640 1 -- 172.21.15.33:0/643029252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fde8c05c050 con 0x558c93d071f0 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fdea021f740 10 client.17679 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fdea021f740 1 -- 172.21.15.33:0/643029252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558c9393e0f0 con 0x558c93d071f0 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.352+0000 7fde7aff5640 1 -- 172.21.15.33:0/643029252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde880043d0 con 0x558c93d071f0 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.353+0000 7fde7aff5640 1 -- 172.21.15.33:0/643029252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fde8c043450 con 0x558c93d071f0 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.353+0000 7fdea021f740 10 client.17679 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.353+0000 7fdea021f740 10 client.17679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.353+0000 7fdea021f740 20 client.17679 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.353+0000 7fdea021f740 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558c93d1fd10 0x558c93d22150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.353+0000 7fdea021f740 4 client.17679 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.353+0000 7fdea021f740 1 -- 172.21.15.33:0/643029252 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c93c77d70 con 0x558c93d1fd10 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.353+0000 7fde96ffd640 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558c93d1fd10 0x558c93d22150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.353+0000 7fde7aff5640 10 client.17679 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.353+0000 7fde96ffd640 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558c93d1fd10 0x558c93d22150 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78453697aef3538d server_cookie=fd6c88134b052639 in_seq=0 out_seq=0 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.354+0000 7fde7aff5640 1 -- 172.21.15.33:0/643029252 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c93c77d70 con 0x558c93d1fd10 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.354+0000 7fde7aff5640 10 client.17679 handle_command_reply: tid=0 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.354+0000 7fdea021f740 1 client.17679 shutdown 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.354+0000 7fdea021f740 1 -- 172.21.15.33:0/643029252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 msgr2=0x558c93d0c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.354+0000 7fdea021f740 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 0x558c93d0c9c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fde8c035000 tx=0x7fde8c000bc0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.355+0000 7fdea021f740 1 -- 172.21.15.33:0/643029252 shutdown_connections 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.355+0000 7fdea021f740 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558c93d1fd10 0x558c93d22150 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.355+0000 7fdea021f740 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d0cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.355+0000 7fdea021f740 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d071f0 0x558c93d0c9c0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.355+0000 7fde7aff5640 0 client.17679 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:35.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.355+0000 7fdea021f740 1 --2- 172.21.15.33:0/643029252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d068d0 0x558c93d0f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.355+0000 7fdea021f740 1 -- 172.21.15.33:0/643029252 >> 172.21.15.33:0/643029252 conn(0x558c93d01000 msgr2=0x558c93d02d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.355+0000 7fdea021f740 1 -- 172.21.15.33:0/643029252 shutdown_connections 2024-04-02T13:48:35.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.355+0000 7fdea021f740 1 -- 172.21.15.33:0/643029252 wait complete. 2024-04-02T13:48:35.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.355+0000 7fdea021f740 20 client.17679 trim_cache size 0 max 16384 2024-04-02T13:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.372+0000 7fdea021f740 1 Processor -- start 2024-04-02T13:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.372+0000 7fdea021f740 1 -- start start 2024-04-02T13:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fdea021f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 0x558c93d07fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fdea021f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d071f0 0x558c93d000c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fdea021f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d0f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fdea021f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde981b19c0 con 0x558c93d068d0 2024-04-02T13:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fdea021f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde981af290 con 0x558c93d0fdd0 2024-04-02T13:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fdea021f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fde981ad100 con 0x558c93d071f0 2024-04-02T13:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 0x558c93d07fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 0x558c93d07fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53194/0 (socket says 172.21.15.33:53194) 2024-04-02T13:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde96ffd640 1 -- 172.21.15.33:0/1538464725 learned_addr learned my addr 172.21.15.33:0/1538464725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde977fe640 1 --2- 172.21.15.33:0/1538464725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d0f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde967fc640 1 --2- 172.21.15.33:0/1538464725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d071f0 0x558c93d000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde96ffd640 1 -- 172.21.15.33:0/1538464725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d071f0 msgr2=0x558c93d000c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde96ffd640 1 --2- 172.21.15.33:0/1538464725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d071f0 0x558c93d000c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde96ffd640 1 -- 172.21.15.33:0/1538464725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 msgr2=0x558c93d0f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde96ffd640 1 --2- 172.21.15.33:0/1538464725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d0f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde96ffd640 1 -- 172.21.15.33:0/1538464725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c93d00e30 con 0x558c93d068d0 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde967fc640 1 --2- 172.21.15.33:0/1538464725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d071f0 0x558c93d000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde96ffd640 1 --2- 172.21.15.33:0/1538464725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 0x558c93d07fa0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fde880058c0 tx=0x7fde88009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e86ce74ee9e3fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde7cff9640 1 -- 172.21.15.33:0/1538464725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88005050 con 0x558c93d068d0 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde7cff9640 1 -- 172.21.15.33:0/1538464725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde88003ee0 con 0x558c93d068d0 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.373+0000 7fde7cff9640 1 -- 172.21.15.33:0/1538464725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde880039f0 con 0x558c93d068d0 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.374+0000 7fdea021f740 1 -- 172.21.15.33:0/1538464725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 msgr2=0x558c93d07fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.374+0000 7fdea021f740 1 --2- 172.21.15.33:0/1538464725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 0x558c93d07fa0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fde880058c0 tx=0x7fde88009ba0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.374+0000 7fdea021f740 1 -- 172.21.15.33:0/1538464725 shutdown_connections 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.374+0000 7fdea021f740 1 --2- 172.21.15.33:0/1538464725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d0f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.374+0000 7fdea021f740 1 --2- 172.21.15.33:0/1538464725 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d071f0 0x558c93d000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.374+0000 7fdea021f740 1 --2- 172.21.15.33:0/1538464725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 0x558c93d07fa0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.374+0000 7fdea021f740 1 -- 172.21.15.33:0/1538464725 >> 172.21.15.33:0/1538464725 conn(0x558c93d01000 msgr2=0x558c93cff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.374+0000 7fdea021f740 1 -- 172.21.15.33:0/1538464725 shutdown_connections 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.374+0000 7fdea021f740 1 -- 172.21.15.33:0/1538464725 wait complete. 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.374+0000 7fdea021f740 1 -- start start 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.374+0000 7fdea021f740 1 Processor -- start 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fdea021f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 0x558c93d13b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fdea021f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d071f0 0x558c93d14060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fdea021f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d145a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fdea021f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde981ad750 con 0x558c93d068d0 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fdea021f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde981aebc0 con 0x558c93d0fdd0 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fdea021f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fde981ade20 con 0x558c93d071f0 2024-04-02T13:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fdea021f740 10 client.0 _ll_register_callbacks cb 0x7fde8c0053a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 0x558c93d13b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 0x558c93d13b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53208/0 (socket says 172.21.15.33:53208) 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde96ffd640 1 -- 172.21.15.33:0/3523684977 learned_addr learned my addr 172.21.15.33:0/3523684977 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde967fc640 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d071f0 0x558c93d14060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde977fe640 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde96ffd640 1 -- 172.21.15.33:0/3523684977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d071f0 msgr2=0x558c93d14060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde96ffd640 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d071f0 0x558c93d14060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde96ffd640 1 -- 172.21.15.33:0/3523684977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 msgr2=0x558c93d145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde96ffd640 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d145a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde96ffd640 1 -- 172.21.15.33:0/3523684977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c93d85440 con 0x558c93d068d0 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde977fe640 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde96ffd640 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 0x558c93d13b20 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fde88003630 tx=0x7fde880098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a26d297ecfe0b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde7aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde7aff5640 1 -- 172.21.15.33:0/3523684977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8803a060 con 0x558c93d068d0 2024-04-02T13:48:35.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fde7aff5640 1 -- 172.21.15.33:0/3523684977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fde88005190 con 0x558c93d068d0 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.375+0000 7fdea021f740 1 -- 172.21.15.33:0/3523684977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde88003ee0 con 0x558c93d068d0 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.376+0000 7fde7aff5640 1 -- 172.21.15.33:0/3523684977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fde88041050 con 0x558c93d068d0 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.376+0000 7fdea021f740 10 client.17685 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.376+0000 7fdea021f740 1 -- 172.21.15.33:0/3523684977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558c93d86710 con 0x558c93d068d0 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.376+0000 7fde7aff5640 1 -- 172.21.15.33:0/3523684977 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fde88003ee0 con 0x558c93d068d0 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.376+0000 7fde7aff5640 1 -- 172.21.15.33:0/3523684977 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fde8c0042d0 con 0x558c93d068d0 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.376+0000 7fdea021f740 10 client.17685 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.376+0000 7fdea021f740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.376+0000 7fdea021f740 20 client.17685 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.376+0000 7fdea021f740 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558c93d86c30 0x558c93d89020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.376+0000 7fdea021f740 4 client.17685 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.376+0000 7fdea021f740 1 -- 172.21.15.33:0/3523684977 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c93c720c0 con 0x558c93d86c30 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.376+0000 7fde967fc640 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558c93d86c30 0x558c93d89020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fde7aff5640 10 client.17685 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fde967fc640 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558c93d86c30 0x558c93d89020 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06131a778847727 server_cookie=a5cc36a1ecff6816 in_seq=0 out_seq=0 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fde7aff5640 1 -- 172.21.15.33:0/3523684977 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558c93c720c0 con 0x558c93d86c30 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fde7aff5640 10 client.17685 handle_command_reply: tid=0 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fdea021f740 1 client.17685 shutdown 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fdea021f740 1 -- 172.21.15.33:0/3523684977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 msgr2=0x558c93d13b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fdea021f740 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 0x558c93d13b20 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fde88003630 tx=0x7fde880098a0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fdea021f740 1 -- 172.21.15.33:0/3523684977 shutdown_connections 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fdea021f740 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x558c93d86c30 0x558c93d89020 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fdea021f740 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c93d0fdd0 0x558c93d145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fde7aff5640 0 client.17685 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fdea021f740 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558c93d071f0 0x558c93d14060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fdea021f740 1 --2- 172.21.15.33:0/3523684977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c93d068d0 0x558c93d13b20 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fdea021f740 1 -- 172.21.15.33:0/3523684977 >> 172.21.15.33:0/3523684977 conn(0x558c93d01000 msgr2=0x558c93d07e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fdea021f740 1 -- 172.21.15.33:0/3523684977 shutdown_connections 2024-04-02T13:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fdea021f740 1 -- 172.21.15.33:0/3523684977 wait complete. 2024-04-02T13:48:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fdea021f740 20 client.17685 trim_cache size 0 max 16384 2024-04-02T13:48:35.387 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:48:35.387 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:48:35.387 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:48:35.387 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:48:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fde9d642640 1 -- 172.21.15.33:0/3293468906 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fde800a3d20 msgr2=0x7fde800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fde9d642640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fde800a3d20 0x7fde800a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fde8c006010 tx=0x7fde8c0040c0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fde9d642640 1 -- 172.21.15.33:0/3293468906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde98106980 msgr2=0x7fde9819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.377+0000 7fde9d642640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde98106980 0x7fde9819cc30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fde9000eaa0 tx=0x7fde9000ef60 comp rx=0 tx=0).stop 2024-04-02T13:48:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.378+0000 7fde9d642640 1 -- 172.21.15.33:0/3293468906 shutdown_connections 2024-04-02T13:48:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.378+0000 7fde9d642640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fde800a3d20 0x7fde800a61e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.378+0000 7fde9d642640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde98106980 0x7fde9819cc30 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.378+0000 7fde9d642640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde98101330 0x7fde981988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.378+0000 7fde9d642640 1 --2- 172.21.15.33:0/3293468906 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde98100970 0x7fde98198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.378+0000 7fde9d642640 1 -- 172.21.15.33:0/3293468906 >> 172.21.15.33:0/3293468906 conn(0x7fde98077b00 msgr2=0x7fde98068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.378+0000 7fde9d642640 1 -- 172.21.15.33:0/3293468906 shutdown_connections 2024-04-02T13:48:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.378+0000 7fde9d642640 1 -- 172.21.15.33:0/3293468906 wait complete. 2024-04-02T13:48:35.391 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:48:35.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:48:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.491+0000 7f4f2a682640 1 -- 172.21.15.33:0/4214824399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f241013c0 msgr2=0x7f4f2410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.491+0000 7f4f2a682640 1 --2- 172.21.15.33:0/4214824399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f241013c0 0x7f4f2410d8d0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009cd0 tx=0x7f4f1c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.492+0000 7f4f2a682640 1 -- 172.21.15.33:0/4214824399 shutdown_connections 2024-04-02T13:48:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.492+0000 7f4f2a682640 1 --2- 172.21.15.33:0/4214824399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f241013c0 0x7f4f2410d8d0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.492+0000 7f4f2a682640 1 --2- 172.21.15.33:0/4214824399 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f24100a00 0x7f4f24100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.492+0000 7f4f2a682640 1 --2- 172.21.15.33:0/4214824399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f24106a10 0x7f4f24106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.492+0000 7f4f2a682640 1 -- 172.21.15.33:0/4214824399 >> 172.21.15.33:0/4214824399 conn(0x7f4f24077b00 msgr2=0x7f4f24077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.492+0000 7f4f2a682640 1 -- 172.21.15.33:0/4214824399 shutdown_connections 2024-04-02T13:48:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.492+0000 7f4f2a682640 1 -- 172.21.15.33:0/4214824399 wait complete. 2024-04-02T13:48:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.492+0000 7f4f2a682640 1 Processor -- start 2024-04-02T13:48:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.492+0000 7f4f2a682640 1 -- start start 2024-04-02T13:48:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f2a682640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f24100a00 0x7f4f241982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f2a682640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f241013c0 0x7f4f241987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f2a682640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f24106a10 0x7f4f2419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f2a682640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24110df0 con 0x7f4f241013c0 2024-04-02T13:48:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f2a682640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24110c70 con 0x7f4f24106a10 2024-04-02T13:48:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f2a682640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24110f70 con 0x7f4f24100a00 2024-04-02T13:48:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f241013c0 0x7f4f241987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f241013c0 0x7f4f241987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53214/0 (socket says 172.21.15.33:53214) 2024-04-02T13:48:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f237fe640 1 -- 172.21.15.33:0/4202710770 learned_addr learned my addr 172.21.15.33:0/4202710770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f28c1f640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f24106a10 0x7f4f2419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f23fff640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f24100a00 0x7f4f241982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f237fe640 1 -- 172.21.15.33:0/4202710770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f24100a00 msgr2=0x7f4f241982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f237fe640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f24100a00 0x7f4f241982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f237fe640 1 -- 172.21.15.33:0/4202710770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f24106a10 msgr2=0x7f4f2419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f237fe640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f24106a10 0x7f4f2419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f237fe640 1 -- 172.21.15.33:0/4202710770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2419d290 con 0x7f4f241013c0 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f28c1f640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f24106a10 0x7f4f2419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f237fe640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f241013c0 0x7f4f241987e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00c920 tx=0x7f4f0c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f217fa640 1 -- 172.21.15.33:0/4202710770 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c007c70 con 0x7f4f241013c0 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f2a682640 1 -- 172.21.15.33:0/4202710770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f2419d580 con 0x7f4f241013c0 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f217fa640 1 -- 172.21.15.33:0/4202710770 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4f0c007e10 con 0x7f4f241013c0 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f217fa640 1 -- 172.21.15.33:0/4202710770 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c018800 con 0x7f4f241013c0 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.493+0000 7f4f2a682640 1 -- 172.21.15.33:0/4202710770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f241a4e10 con 0x7f4f241013c0 2024-04-02T13:48:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.494+0000 7f4f2a682640 1 -- 172.21.15.33:0/4202710770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f240688f0 con 0x7f4f241013c0 2024-04-02T13:48:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.495+0000 7f4f217fa640 1 -- 172.21.15.33:0/4202710770 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4f0c018a00 con 0x7f4f241013c0 2024-04-02T13:48:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.496+0000 7f4f217fa640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4f100a3d20 0x7f4f100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.496+0000 7f4f217fa640 1 -- 172.21.15.33:0/4202710770 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f4f0c0d0da0 con 0x7f4f241013c0 2024-04-02T13:48:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.496+0000 7f4f23fff640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4f100a3d20 0x7f4f100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.496+0000 7f4f23fff640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4f100a3d20 0x7f4f100a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f14005fd0 tx=0x7f4f14005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.501+0000 7f4f217fa640 1 -- 172.21.15.33:0/4202710770 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4f0c014030 con 0x7f4f241013c0 2024-04-02T13:48:35.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.728+0000 7f4f2a682640 1 -- 172.21.15.33:0/4202710770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f24109fe0 con 0x7f4f241013c0 2024-04-02T13:48:35.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.728+0000 7f4f217fa640 1 -- 172.21.15.33:0/4202710770 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11873 (secure 0 0 0) 0x7f4f0c00de20 con 0x7f4f241013c0 2024-04-02T13:48:35.731 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:48:35.731 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:44:55.702357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:48:35.732 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 36 2024-04-02T13:48:35.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.730+0000 7f4f2a682640 1 -- 172.21.15.33:0/4202710770 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4f100a3d20 msgr2=0x7f4f100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.730+0000 7f4f2a682640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4f100a3d20 0x7f4f100a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f14005fd0 tx=0x7f4f14005850 comp rx=0 tx=0).stop 2024-04-02T13:48:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.730+0000 7f4f2a682640 1 -- 172.21.15.33:0/4202710770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f241013c0 msgr2=0x7f4f241987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.730+0000 7f4f2a682640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f241013c0 0x7f4f241987e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00c920 tx=0x7f4f0c00cde0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.731+0000 7f4f2a682640 1 -- 172.21.15.33:0/4202710770 shutdown_connections 2024-04-02T13:48:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.731+0000 7f4f2a682640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4f100a3d20 0x7f4f100a61e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.731+0000 7f4f2a682640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f24106a10 0x7f4f2419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.731+0000 7f4f2a682640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f241013c0 0x7f4f241987e0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.731+0000 7f4f2a682640 1 --2- 172.21.15.33:0/4202710770 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4f24100a00 0x7f4f241982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.731+0000 7f4f2a682640 1 -- 172.21.15.33:0/4202710770 >> 172.21.15.33:0/4202710770 conn(0x7f4f24077b00 msgr2=0x7f4f24068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.731+0000 7f4f2a682640 1 -- 172.21.15.33:0/4202710770 shutdown_connections 2024-04-02T13:48:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.731+0000 7f4f2a682640 1 -- 172.21.15.33:0/4202710770 wait complete. 2024-04-02T13:48:35.744 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:48:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.846+0000 7f16d1edc640 1 -- 172.21.15.33:0/2594527860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc068ce0 msgr2=0x7f16cc069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.846+0000 7f16d1edc640 1 --2- 172.21.15.33:0/2594527860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc068ce0 0x7f16cc069160 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009930 tx=0x7f16c002bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.846+0000 7f16d1edc640 1 -- 172.21.15.33:0/2594527860 shutdown_connections 2024-04-02T13:48:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.846+0000 7f16d1edc640 1 --2- 172.21.15.33:0/2594527860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16cc0696a0 0x7f16cc10e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.846+0000 7f16d1edc640 1 --2- 172.21.15.33:0/2594527860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc068ce0 0x7f16cc069160 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.846+0000 7f16d1edc640 1 --2- 172.21.15.33:0/2594527860 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16cc107a80 0x7f16cc107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.846+0000 7f16d1edc640 1 -- 172.21.15.33:0/2594527860 >> 172.21.15.33:0/2594527860 conn(0x7f16cc077b00 msgr2=0x7f16cc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.846+0000 7f16d1edc640 1 -- 172.21.15.33:0/2594527860 shutdown_connections 2024-04-02T13:48:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.846+0000 7f16d1edc640 1 -- 172.21.15.33:0/2594527860 wait complete. 2024-04-02T13:48:35.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.847+0000 7f16d1edc640 1 Processor -- start 2024-04-02T13:48:35.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.847+0000 7f16d1edc640 1 -- start start 2024-04-02T13:48:35.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.847+0000 7f16d1edc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc068ce0 0x7f16cc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.847+0000 7f16d1edc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16cc0696a0 0x7f16cc198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.847+0000 7f16d1edc640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16cc107a80 0x7f16cc19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.847+0000 7f16d1edc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc111e30 con 0x7f16cc068ce0 2024-04-02T13:48:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.847+0000 7f16d1edc640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc111cb0 con 0x7f16cc0696a0 2024-04-02T13:48:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.847+0000 7f16d1edc640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc111fb0 con 0x7f16cc107a80 2024-04-02T13:48:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16cb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc068ce0 0x7f16cc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16cb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc068ce0 0x7f16cc198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53226/0 (socket says 172.21.15.33:53226) 2024-04-02T13:48:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16cb7fe640 1 -- 172.21.15.33:0/1356644200 learned_addr learned my addr 172.21.15.33:0/1356644200 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16caffd640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16cc0696a0 0x7f16cc198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16cb7fe640 1 -- 172.21.15.33:0/1356644200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16cc107a80 msgr2=0x7f16cc19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16cb7fe640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16cc107a80 0x7f16cc19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16cb7fe640 1 -- 172.21.15.33:0/1356644200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16cc0696a0 msgr2=0x7f16cc198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16cb7fe640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16cc0696a0 0x7f16cc198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16cb7fe640 1 -- 172.21.15.33:0/1356644200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc19d340 con 0x7f16cc068ce0 2024-04-02T13:48:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16caffd640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16cc0696a0 0x7f16cc198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16cb7fe640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc068ce0 0x7f16cc198350 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00c920 tx=0x7f16bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16c8ff9640 1 -- 172.21.15.33:0/1356644200 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc007c70 con 0x7f16cc068ce0 2024-04-02T13:48:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16c8ff9640 1 -- 172.21.15.33:0/1356644200 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f16bc007e10 con 0x7f16cc068ce0 2024-04-02T13:48:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16d1edc640 1 -- 172.21.15.33:0/1356644200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16cc19d630 con 0x7f16cc068ce0 2024-04-02T13:48:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16c8ff9640 1 -- 172.21.15.33:0/1356644200 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc018850 con 0x7f16cc068ce0 2024-04-02T13:48:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.848+0000 7f16d1edc640 1 -- 172.21.15.33:0/1356644200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16cc1a4ec0 con 0x7f16cc068ce0 2024-04-02T13:48:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.850+0000 7f16c8ff9640 1 -- 172.21.15.33:0/1356644200 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f16bc018a30 con 0x7f16cc068ce0 2024-04-02T13:48:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16d4b95740 1 Processor -- start 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16d4b95740 1 -- start start 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16d4b95740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0537a0 0x560d1a053b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16c8ff9640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f16b40a3da0 0x7f16b40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16d4b95740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0540c0 0x560d1a05c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16d4b95740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a05cca0 0x560d1a05f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16d4b95740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560d19fb77b0 con 0x560d1a0540c0 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16d4b95740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560d19f56570 con 0x560d1a0537a0 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16d4b95740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560d19fb3760 con 0x560d1a05cca0 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16caffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0540c0 0x560d1a05c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16caffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0540c0 0x560d1a05c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53232/0 (socket says 172.21.15.33:53232) 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16caffd640 1 -- 172.21.15.33:0/3387186767 learned_addr learned my addr 172.21.15.33:0/3387186767 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16cb7fe640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f16b40a3da0 0x7f16b40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16c8ff9640 1 -- 172.21.15.33:0/1356644200 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f16bc0d0d30 con 0x7f16cc068ce0 2024-04-02T13:48:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16cbfff640 1 --2- 172.21.15.33:0/3387186767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a05cca0 0x560d1a05f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16cbfff640 1 --2- 172.21.15.33:0/3387186767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0537a0 0x560d1a053b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16caffd640 1 -- 172.21.15.33:0/3387186767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a05cca0 msgr2=0x560d1a05f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16caffd640 1 --2- 172.21.15.33:0/3387186767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a05cca0 0x560d1a05f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16caffd640 1 -- 172.21.15.33:0/3387186767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0537a0 msgr2=0x560d1a053b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16caffd640 1 --2- 172.21.15.33:0/3387186767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0537a0 0x560d1a053b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16caffd640 1 -- 172.21.15.33:0/3387186767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d19ccb3f0 con 0x560d1a0540c0 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.851+0000 7f16cbfff640 1 --2- 172.21.15.33:0/3387186767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a05cca0 0x560d1a05f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.852+0000 7f16cb7fe640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f16b40a3da0 0x7f16b40a6260 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f16bc09c5a0 tx=0x7f16bc087040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.852+0000 7f16caffd640 1 --2- 172.21.15.33:0/3387186767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0540c0 0x560d1a05c760 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f16c002c170 tx=0x7f16c0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67895aa5b71896c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.852+0000 7f16b1ffb640 1 -- 172.21.15.33:0/3387186767 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c00043c0 con 0x560d1a0540c0 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.852+0000 7f16b1ffb640 1 -- 172.21.15.33:0/3387186767 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f16c0005500 con 0x560d1a0540c0 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.852+0000 7f16b1ffb640 1 -- 172.21.15.33:0/3387186767 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c0004920 con 0x560d1a0540c0 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.852+0000 7f16d4b95740 1 -- 172.21.15.33:0/3387186767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0540c0 msgr2=0x560d1a05c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.852+0000 7f16d4b95740 1 --2- 172.21.15.33:0/3387186767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0540c0 0x560d1a05c760 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f16c002c170 tx=0x7f16c0005e80 comp rx=0 tx=0).stop 2024-04-02T13:48:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.852+0000 7f16d4b95740 1 -- 172.21.15.33:0/3387186767 shutdown_connections 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.852+0000 7f16d4b95740 1 --2- 172.21.15.33:0/3387186767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a05cca0 0x560d1a05f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.852+0000 7f16d4b95740 1 --2- 172.21.15.33:0/3387186767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0540c0 0x560d1a05c760 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.852+0000 7f16d4b95740 1 --2- 172.21.15.33:0/3387186767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0537a0 0x560d1a053b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.852+0000 7f16d4b95740 1 -- 172.21.15.33:0/3387186767 >> 172.21.15.33:0/3387186767 conn(0x560d1a04ded0 msgr2=0x560d1a04e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.852+0000 7f16d4b95740 1 -- 172.21.15.33:0/3387186767 shutdown_connections 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16d4b95740 1 -- 172.21.15.33:0/3387186767 wait complete. 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16d4b95740 1 -- start start 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16d4b95740 1 Processor -- start 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16d4b95740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0537a0 0x560d1a05c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16d4b95740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0540c0 0x560d1a059a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16d4b95740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 0x560d1a05a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16d4b95740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560d1a05fa30 con 0x560d1a05cca0 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16d4b95740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560d19f72e80 con 0x560d1a0537a0 2024-04-02T13:48:35.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16d4b95740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x560d1a05fbb0 con 0x560d1a0540c0 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16d4b95740 10 client.0 _ll_register_callbacks cb 0x560d19faeaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 0x560d1a05a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 0x560d1a05a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53240/0 (socket says 172.21.15.33:53240) 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16cbfff640 1 -- 172.21.15.33:0/532949318 learned_addr learned my addr 172.21.15.33:0/532949318 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.853+0000 7f16cbfff640 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0537a0 0x560d1a05c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16caffd640 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0540c0 0x560d1a059a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16cbfff640 1 -- 172.21.15.33:0/532949318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0540c0 msgr2=0x560d1a059a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16cbfff640 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0540c0 0x560d1a059a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16cbfff640 1 -- 172.21.15.33:0/532949318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0537a0 msgr2=0x560d1a05c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16cbfff640 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0537a0 0x560d1a05c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16cbfff640 1 -- 172.21.15.33:0/532949318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d19d957d0 con 0x560d1a05cca0 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16caffd640 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0540c0 0x560d1a059a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16cbfff640 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 0x560d1a05a020 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009a70 tx=0x7f16c40099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8e5a7be373fbd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16aaffd640 1 -- 172.21.15.33:0/532949318 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4004b20 con 0x560d1a05cca0 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16aaffd640 1 -- 172.21.15.33:0/532949318 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f16c4004cc0 con 0x560d1a05cca0 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16d4b95740 1 -- 172.21.15.33:0/532949318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c0005500 con 0x560d1a05cca0 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16aaffd640 1 -- 172.21.15.33:0/532949318 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f16c4027050 con 0x560d1a05cca0 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16d4b95740 10 client.17703 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16d4b95740 1 -- 172.21.15.33:0/532949318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560d1a06ba80 con 0x560d1a05cca0 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.854+0000 7f16aaffd640 1 -- 172.21.15.33:0/532949318 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f16c0005500 con 0x560d1a05cca0 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.855+0000 7f16aaffd640 1 -- 172.21.15.33:0/532949318 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f16c40214e0 con 0x560d1a05cca0 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.855+0000 7f16d4b95740 10 client.17703 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.855+0000 7f16d4b95740 10 client.17703 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.855+0000 7f16d4b95740 20 client.17703 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.855+0000 7f16d4b95740 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560d1a06cf30 0x560d1a06f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.855+0000 7f16d4b95740 4 client.17703 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.855+0000 7f16d4b95740 1 -- 172.21.15.33:0/532949318 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d19fc4cf0 con 0x560d1a06cf30 2024-04-02T13:48:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.855+0000 7f16caffd640 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560d1a06cf30 0x560d1a06f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.855+0000 7f16caffd640 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560d1a06cf30 0x560d1a06f320 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8aa4e8ee771a321 server_cookie=8ce122ac6dd4a76a in_seq=0 out_seq=0 2024-04-02T13:48:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.855+0000 7f16aaffd640 10 client.17703 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:48:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16aaffd640 1 -- 172.21.15.33:0/532949318 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d19fc4cf0 con 0x560d1a06cf30 2024-04-02T13:48:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16aaffd640 10 client.17703 handle_command_reply: tid=0 2024-04-02T13:48:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16d4b95740 1 client.17703 shutdown 2024-04-02T13:48:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16d4b95740 1 -- 172.21.15.33:0/532949318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 msgr2=0x560d1a05a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16d4b95740 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 0x560d1a05a020 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009a70 tx=0x7f16c40099e0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16d4b95740 1 -- 172.21.15.33:0/532949318 shutdown_connections 2024-04-02T13:48:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16d4b95740 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560d1a06cf30 0x560d1a06f320 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16d4b95740 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 0x560d1a05a020 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16d4b95740 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0540c0 0x560d1a059a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16aaffd640 0 client.17703 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:48:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16d4b95740 1 --2- 172.21.15.33:0/532949318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0537a0 0x560d1a05c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16d4b95740 1 -- 172.21.15.33:0/532949318 >> 172.21.15.33:0/532949318 conn(0x560d1a04ded0 msgr2=0x560d1a04fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16d4b95740 1 -- 172.21.15.33:0/532949318 shutdown_connections 2024-04-02T13:48:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.856+0000 7f16d4b95740 1 -- 172.21.15.33:0/532949318 wait complete. 2024-04-02T13:48:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.857+0000 7f16d4b95740 20 client.17703 trim_cache size 0 max 16384 2024-04-02T13:48:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16d4b95740 1 Processor -- start 2024-04-02T13:48:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16d4b95740 1 -- start start 2024-04-02T13:48:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16d4b95740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0537a0 0x560d1a068950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16d4b95740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0540c0 0x560d1a0589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16d4b95740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a05cca0 0x560d1a05c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16d4b95740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1b1980 con 0x560d1a0537a0 2024-04-02T13:48:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16d4b95740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1af250 con 0x560d1a05cca0 2024-04-02T13:48:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16d4b95740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1ad0c0 con 0x560d1a0540c0 2024-04-02T13:48:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0537a0 0x560d1a068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0537a0 0x560d1a068950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53254/0 (socket says 172.21.15.33:53254) 2024-04-02T13:48:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16cbfff640 1 -- 172.21.15.33:0/683209121 learned_addr learned my addr 172.21.15.33:0/683209121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16caffd640 1 --2- 172.21.15.33:0/683209121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0540c0 0x560d1a0589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16cbfff640 1 -- 172.21.15.33:0/683209121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0540c0 msgr2=0x560d1a0589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16cbfff640 1 --2- 172.21.15.33:0/683209121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0540c0 0x560d1a0589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16cbfff640 1 -- 172.21.15.33:0/683209121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a05cca0 msgr2=0x560d1a05c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16cbfff640 1 --2- 172.21.15.33:0/683209121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a05cca0 0x560d1a05c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16cbfff640 1 -- 172.21.15.33:0/683209121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1a053ef0 con 0x560d1a0537a0 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.870+0000 7f16cbfff640 1 --2- 172.21.15.33:0/683209121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0537a0 0x560d1a068950 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f16c4000f80 tx=0x7f16c4010fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdb2fcc20d2ecdf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.871+0000 7f16b17fa640 1 -- 172.21.15.33:0/683209121 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4021cd0 con 0x560d1a0537a0 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.871+0000 7f16b17fa640 1 -- 172.21.15.33:0/683209121 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f16c4009520 con 0x560d1a0537a0 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.871+0000 7f16b17fa640 1 -- 172.21.15.33:0/683209121 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4021670 con 0x560d1a0537a0 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.871+0000 7f16d4b95740 1 -- 172.21.15.33:0/683209121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0537a0 msgr2=0x560d1a068950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.871+0000 7f16d4b95740 1 --2- 172.21.15.33:0/683209121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0537a0 0x560d1a068950 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f16c4000f80 tx=0x7f16c4010fb0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.871+0000 7f16d4b95740 1 -- 172.21.15.33:0/683209121 shutdown_connections 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.871+0000 7f16d4b95740 1 --2- 172.21.15.33:0/683209121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a05cca0 0x560d1a05c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.871+0000 7f16d4b95740 1 --2- 172.21.15.33:0/683209121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0540c0 0x560d1a0589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.871+0000 7f16d4b95740 1 --2- 172.21.15.33:0/683209121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a0537a0 0x560d1a068950 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.871+0000 7f16d4b95740 1 -- 172.21.15.33:0/683209121 >> 172.21.15.33:0/683209121 conn(0x560d1a04ded0 msgr2=0x560d1a0582e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.871+0000 7f16d4b95740 1 -- 172.21.15.33:0/683209121 shutdown_connections 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.871+0000 7f16d4b95740 1 -- 172.21.15.33:0/683209121 wait complete. 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16d4b95740 1 -- start start 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16d4b95740 1 Processor -- start 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16d4b95740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0537a0 0x560d1a065050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16d4b95740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0540c0 0x560d1a056c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16d4b95740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 0x560d1a0571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16d4b95740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1ad710 con 0x560d1a05cca0 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16d4b95740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1aeb80 con 0x560d1a0540c0 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16d4b95740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc1adde0 con 0x560d1a0537a0 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16d4b95740 10 client.0 _ll_register_callbacks cb 0x7f16c4009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 0x560d1a0571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 0x560d1a0571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53256/0 (socket says 172.21.15.33:53256) 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16cbfff640 1 -- 172.21.15.33:0/2527419159 learned_addr learned my addr 172.21.15.33:0/2527419159 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16cbfff640 1 --2- 172.21.15.33:0/2527419159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0537a0 0x560d1a065050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16caffd640 1 --2- 172.21.15.33:0/2527419159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0540c0 0x560d1a056c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16cbfff640 1 -- 172.21.15.33:0/2527419159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0537a0 msgr2=0x560d1a065050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16cbfff640 1 --2- 172.21.15.33:0/2527419159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0537a0 0x560d1a065050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16cbfff640 1 -- 172.21.15.33:0/2527419159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0540c0 msgr2=0x560d1a056c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16cbfff640 1 --2- 172.21.15.33:0/2527419159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0540c0 0x560d1a056c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.872+0000 7f16cbfff640 1 -- 172.21.15.33:0/2527419159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1a0577c0 con 0x560d1a05cca0 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16cbfff640 1 --2- 172.21.15.33:0/2527419159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 0x560d1a0571d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f16c4021310 tx=0x7f16c4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9ff91776b8de9c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16aaffd640 1 -- 172.21.15.33:0/2527419159 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4021610 con 0x560d1a05cca0 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16aaffd640 1 -- 172.21.15.33:0/2527419159 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f16c4021ad0 con 0x560d1a05cca0 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16d4b95740 1 -- 172.21.15.33:0/2527419159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c4009520 con 0x560d1a05cca0 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16aaffd640 1 -- 172.21.15.33:0/2527419159 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f16c4021cd0 con 0x560d1a05cca0 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16d4b95740 10 client.17709 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16d4b95740 1 -- 172.21.15.33:0/2527419159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560d1a05e4d0 con 0x560d1a05cca0 2024-04-02T13:48:35.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16aaffd640 1 -- 172.21.15.33:0/2527419159 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f16c4009520 con 0x560d1a05cca0 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16aaffd640 1 -- 172.21.15.33:0/2527419159 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f16c4004670 con 0x560d1a05cca0 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16d4b95740 10 client.17709 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16d4b95740 10 client.17709 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16d4b95740 20 client.17709 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16d4b95740 1 --2- 172.21.15.33:0/2527419159 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560d1a0d38c0 0x560d1a0d5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16d4b95740 4 client.17709 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16d4b95740 1 -- 172.21.15.33:0/2527419159 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d19fbf040 con 0x560d1a0d38c0 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.873+0000 7f16caffd640 1 --2- 172.21.15.33:0/2527419159 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560d1a0d38c0 0x560d1a0d5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16aaffd640 10 client.17709 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16caffd640 1 --2- 172.21.15.33:0/2527419159 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560d1a0d38c0 0x560d1a0d5ce0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d096769b4a38214 server_cookie=68ddbff34954890d in_seq=0 out_seq=0 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16aaffd640 1 -- 172.21.15.33:0/2527419159 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d19fbf040 con 0x560d1a0d38c0 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16aaffd640 10 client.17709 handle_command_reply: tid=0 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16d4b95740 1 client.17709 shutdown 2024-04-02T13:48:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16d4b95740 1 -- 172.21.15.33:0/2527419159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 msgr2=0x560d1a0571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16d4b95740 1 --2- 172.21.15.33:0/2527419159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 0x560d1a0571d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f16c4021310 tx=0x7f16c4014040 comp rx=0 tx=0).stop 2024-04-02T13:48:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16d4b95740 1 -- 172.21.15.33:0/2527419159 shutdown_connections 2024-04-02T13:48:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16d4b95740 1 --2- 172.21.15.33:0/2527419159 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x560d1a0d38c0 0x560d1a0d5ce0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16d4b95740 1 --2- 172.21.15.33:0/2527419159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d1a05cca0 0x560d1a0571d0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16d4b95740 1 --2- 172.21.15.33:0/2527419159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1a0540c0 0x560d1a056c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16aaffd640 0 client.17709 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:48:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16d4b95740 1 --2- 172.21.15.33:0/2527419159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560d1a0537a0 0x560d1a065050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16d4b95740 1 -- 172.21.15.33:0/2527419159 >> 172.21.15.33:0/2527419159 conn(0x560d1a04ded0 msgr2=0x560d1a04c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16d4b95740 1 -- 172.21.15.33:0/2527419159 shutdown_connections 2024-04-02T13:48:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16d4b95740 1 -- 172.21.15.33:0/2527419159 wait complete. 2024-04-02T13:48:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.874+0000 7f16d4b95740 20 client.17709 trim_cache size 0 max 16384 2024-04-02T13:48:35.884 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:48:35.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.875+0000 7f16d1edc640 1 -- 172.21.15.33:0/1356644200 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f16b40a3da0 msgr2=0x7f16b40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.875+0000 7f16d1edc640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f16b40a3da0 0x7f16b40a6260 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f16bc09c5a0 tx=0x7f16bc087040 comp rx=0 tx=0).stop 2024-04-02T13:48:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.875+0000 7f16d1edc640 1 -- 172.21.15.33:0/1356644200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc068ce0 msgr2=0x7f16cc198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.875+0000 7f16d1edc640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc068ce0 0x7f16cc198350 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00c920 tx=0x7f16bc00cde0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.875+0000 7f16d1edc640 1 -- 172.21.15.33:0/1356644200 shutdown_connections 2024-04-02T13:48:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.875+0000 7f16d1edc640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f16b40a3da0 0x7f16b40a6260 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.875+0000 7f16d1edc640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f16cc107a80 0x7f16cc19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.875+0000 7f16d1edc640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16cc0696a0 0x7f16cc198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.875+0000 7f16d1edc640 1 --2- 172.21.15.33:0/1356644200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16cc068ce0 0x7f16cc198350 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.875+0000 7f16d1edc640 1 -- 172.21.15.33:0/1356644200 >> 172.21.15.33:0/1356644200 conn(0x7f16cc077b00 msgr2=0x7f16cc0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.875+0000 7f16d1edc640 1 -- 172.21.15.33:0/1356644200 shutdown_connections 2024-04-02T13:48:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.875+0000 7f16d1edc640 1 -- 172.21.15.33:0/1356644200 wait complete. 2024-04-02T13:48:35.888 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:48:35.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.990+0000 7fbd793cd640 1 -- 172.21.15.33:0/1124279147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741013b0 msgr2=0x7fbd7410d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.990+0000 7fbd793cd640 1 --2- 172.21.15.33:0/1124279147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741013b0 0x7fbd7410d870 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009cd0 tx=0x7fbd6402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 -- 172.21.15.33:0/1124279147 shutdown_connections 2024-04-02T13:48:35.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 --2- 172.21.15.33:0/1124279147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741013b0 0x7fbd7410d870 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 --2- 172.21.15.33:0/1124279147 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd741009f0 0x7fbd74100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 --2- 172.21.15.33:0/1124279147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd741069b0 0x7fbd74106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 -- 172.21.15.33:0/1124279147 >> 172.21.15.33:0/1124279147 conn(0x7fbd74077b60 msgr2=0x7fbd74077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:35.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 -- 172.21.15.33:0/1124279147 shutdown_connections 2024-04-02T13:48:35.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 -- 172.21.15.33:0/1124279147 wait complete. 2024-04-02T13:48:35.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 Processor -- start 2024-04-02T13:48:35.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 -- start start 2024-04-02T13:48:35.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd741009f0 0x7fbd74198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd741013b0 0x7fbd74198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741069b0 0x7fbd7419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:35.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74110d90 con 0x7fbd741069b0 2024-04-02T13:48:35.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74110c10 con 0x7fbd741013b0 2024-04-02T13:48:35.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.991+0000 7fbd793cd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74110f10 con 0x7fbd741009f0 2024-04-02T13:48:35.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741069b0 0x7fbd7419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741069b0 0x7fbd7419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53270/0 (socket says 172.21.15.33:53270) 2024-04-02T13:48:35.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd737fe640 1 -- 172.21.15.33:0/65920612 learned_addr learned my addr 172.21.15.33:0/65920612 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:35.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd72ffd640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd741009f0 0x7fbd74198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd727fc640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd741013b0 0x7fbd74198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:35.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd737fe640 1 -- 172.21.15.33:0/65920612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd741009f0 msgr2=0x7fbd74198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd737fe640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd741009f0 0x7fbd74198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd737fe640 1 -- 172.21.15.33:0/65920612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd741013b0 msgr2=0x7fbd74198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:35.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd737fe640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd741013b0 0x7fbd74198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:35.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd737fe640 1 -- 172.21.15.33:0/65920612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7419d410 con 0x7fbd741069b0 2024-04-02T13:48:35.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd737fe640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741069b0 0x7fbd7419cce0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fbd64004af0 tx=0x7fbd64034d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:35.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd5bfff640 1 -- 172.21.15.33:0/65920612 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64039050 con 0x7fbd741069b0 2024-04-02T13:48:35.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd5bfff640 1 -- 172.21.15.33:0/65920612 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd64045070 con 0x7fbd741069b0 2024-04-02T13:48:35.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd793cd640 1 -- 172.21.15.33:0/65920612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7419d6a0 con 0x7fbd741069b0 2024-04-02T13:48:35.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd5bfff640 1 -- 172.21.15.33:0/65920612 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd640046b0 con 0x7fbd741069b0 2024-04-02T13:48:35.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.992+0000 7fbd793cd640 1 -- 172.21.15.33:0/65920612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd741a4f90 con 0x7fbd741069b0 2024-04-02T13:48:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd5bfff640 1 -- 172.21.15.33:0/65920612 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbd64003970 con 0x7fbd741069b0 2024-04-02T13:48:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd7bfbb740 1 Processor -- start 2024-04-02T13:48:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd7bfbb740 1 -- start start 2024-04-02T13:48:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd7bfbb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b04883a7a0 0x55b04883ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd7bfbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 0x55b048843760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd7bfbb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b048843ca0 0x55b048846090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd7bfbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b04879e7b0 con 0x55b04883b0c0 2024-04-02T13:48:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd5bfff640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbd5c0a3d20 0x7fbd5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd7bfbb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b04873d570 con 0x55b04883a7a0 2024-04-02T13:48:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd7bfbb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b04879a760 con 0x55b048843ca0 2024-04-02T13:48:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd72ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 0x55b048843760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd72ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 0x55b048843760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53280/0 (socket says 172.21.15.33:53280) 2024-04-02T13:48:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd72ffd640 1 -- 172.21.15.33:0/3583645570 learned_addr learned my addr 172.21.15.33:0/3583645570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.995+0000 7fbd737fe640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbd5c0a3d20 0x7fbd5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.996+0000 7fbd5bfff640 1 -- 172.21.15.33:0/65920612 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fbd640f3490 con 0x7fbd741069b0 2024-04-02T13:48:36.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.996+0000 7fbd72ffd640 1 -- 172.21.15.33:0/3583645570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b048843ca0 msgr2=0x55b048846090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.996+0000 7fbd72ffd640 1 --2- 172.21.15.33:0/3583645570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b048843ca0 0x55b048846090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.996+0000 7fbd72ffd640 1 -- 172.21.15.33:0/3583645570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b04883a7a0 msgr2=0x55b04883ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.996+0000 7fbd72ffd640 1 --2- 172.21.15.33:0/3583645570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b04883a7a0 0x55b04883ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.996+0000 7fbd72ffd640 1 -- 172.21.15.33:0/3583645570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0484b23f0 con 0x55b04883b0c0 2024-04-02T13:48:36.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.996+0000 7fbd737fe640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbd5c0a3d20 0x7fbd5c0a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbd640f0a70 tx=0x7fbd640bad90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:36.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.996+0000 7fbd72ffd640 1 --2- 172.21.15.33:0/3583645570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 0x55b048843760 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00bc50 tx=0x7fbd6c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9518451b04632f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:36.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.996+0000 7fbd597fa640 1 -- 172.21.15.33:0/3583645570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c015850 con 0x55b04883b0c0 2024-04-02T13:48:36.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.996+0000 7fbd597fa640 1 -- 172.21.15.33:0/3583645570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd6c020480 con 0x55b04883b0c0 2024-04-02T13:48:36.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.996+0000 7fbd597fa640 1 -- 172.21.15.33:0/3583645570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c01e5a0 con 0x55b04883b0c0 2024-04-02T13:48:36.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.997+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/3583645570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 msgr2=0x55b048843760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.997+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/3583645570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 0x55b048843760 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00bc50 tx=0x7fbd6c0107a0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.997+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/3583645570 shutdown_connections 2024-04-02T13:48:36.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.997+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/3583645570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b048843ca0 0x55b048846090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.997+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/3583645570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 0x55b048843760 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.997+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/3583645570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b04883a7a0 0x55b04883ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.997+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/3583645570 >> 172.21.15.33:0/3583645570 conn(0x55b048834ed0 msgr2=0x55b0488352b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.997+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/3583645570 shutdown_connections 2024-04-02T13:48:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.997+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/3583645570 wait complete. 2024-04-02T13:48:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.997+0000 7fbd7bfbb740 1 -- start start 2024-04-02T13:48:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.997+0000 7fbd7bfbb740 1 Processor -- start 2024-04-02T13:48:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd7bfbb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883a7a0 0x55b0488436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd7bfbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 0x55b048840a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd7bfbb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b048843ca0 0x55b0488411c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd7bfbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b048846a90 con 0x55b04883b0c0 2024-04-02T13:48:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd7bfbb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b048759e80 con 0x55b048843ca0 2024-04-02T13:48:36.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd7bfbb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b048846c10 con 0x55b04883a7a0 2024-04-02T13:48:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd7bfbb740 10 client.0 _ll_register_callbacks cb 0x55b048795aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd72ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 0x55b048840a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd72ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 0x55b048840a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53286/0 (socket says 172.21.15.33:53286) 2024-04-02T13:48:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd72ffd640 1 -- 172.21.15.33:0/2761315088 learned_addr learned my addr 172.21.15.33:0/2761315088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd727fc640 1 --2- 172.21.15.33:0/2761315088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b048843ca0 0x55b0488411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd727fc640 1 --2- 172.21.15.33:0/2761315088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883a7a0 0x55b0488436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd72ffd640 1 -- 172.21.15.33:0/2761315088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883a7a0 msgr2=0x55b0488436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd72ffd640 1 --2- 172.21.15.33:0/2761315088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883a7a0 0x55b0488436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd72ffd640 1 -- 172.21.15.33:0/2761315088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b048843ca0 msgr2=0x55b0488411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd72ffd640 1 --2- 172.21.15.33:0/2761315088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b048843ca0 0x55b0488411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd72ffd640 1 -- 172.21.15.33:0/2761315088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04857c7d0 con 0x55b04883b0c0 2024-04-02T13:48:36.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd72ffd640 1 --2- 172.21.15.33:0/2761315088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 0x55b048840a80 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c020ec0 tx=0x7fbd6c00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81023e3f70828b15 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.998+0000 7fbd56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd56ff5640 1 -- 172.21.15.33:0/2761315088 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c02f060 con 0x55b04883b0c0 2024-04-02T13:48:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd56ff5640 1 -- 172.21.15.33:0/2761315088 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fbd6c020840 con 0x55b04883b0c0 2024-04-02T13:48:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/2761315088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd6c020480 con 0x55b04883b0c0 2024-04-02T13:48:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd56ff5640 1 -- 172.21.15.33:0/2761315088 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbd6c036050 con 0x55b04883b0c0 2024-04-02T13:48:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd7bfbb740 10 client.17721 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/2761315088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b048852b10 con 0x55b04883b0c0 2024-04-02T13:48:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd56ff5640 1 -- 172.21.15.33:0/2761315088 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd6c020480 con 0x55b04883b0c0 2024-04-02T13:48:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd56ff5640 1 -- 172.21.15.33:0/2761315088 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fbd6c02aa10 con 0x55b04883b0c0 2024-04-02T13:48:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd7bfbb740 10 client.17721 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd7bfbb740 10 client.17721 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:48:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd7bfbb740 20 client.17721 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:36.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/2761315088 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b048853f70 0x55b048856360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd7bfbb740 4 client.17721 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/2761315088 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0487abcf0 con 0x55b048853f70 2024-04-02T13:48:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:35.999+0000 7fbd727fc640 1 --2- 172.21.15.33:0/2761315088 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b048853f70 0x55b048856360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.000+0000 7fbd727fc640 1 --2- 172.21.15.33:0/2761315088 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b048853f70 0x55b048856360 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a4aba8ad70f69cd server_cookie=42d458b57ac7d01d in_seq=0 out_seq=0 2024-04-02T13:48:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.000+0000 7fbd56ff5640 10 client.17721 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:48:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.000+0000 7fbd56ff5640 1 -- 172.21.15.33:0/2761315088 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0487abcf0 con 0x55b048853f70 2024-04-02T13:48:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.000+0000 7fbd56ff5640 10 client.17721 handle_command_reply: tid=0 2024-04-02T13:48:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.000+0000 7fbd7bfbb740 1 client.17721 shutdown 2024-04-02T13:48:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.001+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/2761315088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 msgr2=0x55b048840a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.001+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/2761315088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 0x55b048840a80 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c020ec0 tx=0x7fbd6c00b850 comp rx=0 tx=0).stop 2024-04-02T13:48:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.001+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/2761315088 shutdown_connections 2024-04-02T13:48:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.001+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/2761315088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b048843ca0 0x55b0488411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.001+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/2761315088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883b0c0 0x55b048840a80 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.001+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/2761315088 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b048853f70 0x55b048856360 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.001+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/2761315088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883a7a0 0x55b0488436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.001+0000 7fbd56ff5640 0 client.17721 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:48:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.001+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/2761315088 >> 172.21.15.33:0/2761315088 conn(0x55b048834ed0 msgr2=0x55b048836c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.001+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/2761315088 shutdown_connections 2024-04-02T13:48:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.001+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/2761315088 wait complete. 2024-04-02T13:48:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.001+0000 7fbd7bfbb740 20 client.17721 trim_cache size 0 max 16384 2024-04-02T13:48:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.014+0000 7fbd7bfbb740 1 Processor -- start 2024-04-02T13:48:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.014+0000 7fbd7bfbb740 1 -- start start 2024-04-02T13:48:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.014+0000 7fbd7bfbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883a7a0 0x55b04884f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.014+0000 7fbd7bfbb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883b0c0 0x55b04883fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.014+0000 7fbd7bfbb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b048843ca0 0x55b048839f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.014+0000 7fbd7bfbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741af6a0 con 0x55b04883a7a0 2024-04-02T13:48:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.014+0000 7fbd7bfbb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741acf70 con 0x55b048843ca0 2024-04-02T13:48:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.014+0000 7fbd7bfbb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741aade0 con 0x55b04883b0c0 2024-04-02T13:48:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.014+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883a7a0 0x55b04884f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.014+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883a7a0 0x55b04884f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53296/0 (socket says 172.21.15.33:53296) 2024-04-02T13:48:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.014+0000 7fbd727fc640 1 -- 172.21.15.33:0/2572387630 learned_addr learned my addr 172.21.15.33:0/2572387630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.015+0000 7fbd727fc640 1 --2- 172.21.15.33:0/2572387630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b048843ca0 0x55b048839f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.015+0000 7fbd72ffd640 1 --2- 172.21.15.33:0/2572387630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883b0c0 0x55b04883fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.015+0000 7fbd727fc640 1 -- 172.21.15.33:0/2572387630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883b0c0 msgr2=0x55b04883fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.015+0000 7fbd727fc640 1 --2- 172.21.15.33:0/2572387630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883b0c0 0x55b04883fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.015+0000 7fbd727fc640 1 -- 172.21.15.33:0/2572387630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b048843ca0 msgr2=0x55b048839f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.015+0000 7fbd727fc640 1 --2- 172.21.15.33:0/2572387630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b048843ca0 0x55b048839f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.015+0000 7fbd727fc640 1 -- 172.21.15.33:0/2572387630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04883efd0 con 0x55b04883a7a0 2024-04-02T13:48:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.015+0000 7fbd727fc640 1 --2- 172.21.15.33:0/2572387630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883a7a0 0x55b04884f980 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fbd68012510 tx=0x7fbd68005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54434938c9ecb282 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.015+0000 7fbd58ff9640 1 -- 172.21.15.33:0/2572387630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6801c040 con 0x55b04883a7a0 2024-04-02T13:48:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.015+0000 7fbd58ff9640 1 -- 172.21.15.33:0/2572387630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd6801cc90 con 0x55b04883a7a0 2024-04-02T13:48:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.015+0000 7fbd58ff9640 1 -- 172.21.15.33:0/2572387630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68005280 con 0x55b04883a7a0 2024-04-02T13:48:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/2572387630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883a7a0 msgr2=0x55b04884f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/2572387630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883a7a0 0x55b04884f980 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fbd68012510 tx=0x7fbd68005cc0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/2572387630 shutdown_connections 2024-04-02T13:48:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/2572387630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b048843ca0 0x55b048839f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/2572387630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883b0c0 0x55b04883fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/2572387630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b04883a7a0 0x55b04884f980 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/2572387630 >> 172.21.15.33:0/2572387630 conn(0x55b048834ed0 msgr2=0x55b0488352b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/2572387630 shutdown_connections 2024-04-02T13:48:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/2572387630 wait complete. 2024-04-02T13:48:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 -- start start 2024-04-02T13:48:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 Processor -- start 2024-04-02T13:48:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b04883a7a0 0x55b048844490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883b0c0 0x55b0488449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b048843ca0 0x55b04884ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741ab430 con 0x55b048843ca0 2024-04-02T13:48:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741ac8a0 con 0x55b04883a7a0 2024-04-02T13:48:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741abb00 con 0x55b04883b0c0 2024-04-02T13:48:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd7bfbb740 10 client.0 _ll_register_callbacks cb 0x7fbd6c00baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b048843ca0 0x55b04884ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b048843ca0 0x55b04884ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53304/0 (socket says 172.21.15.33:53304) 2024-04-02T13:48:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd727fc640 1 -- 172.21.15.33:0/4148563345 learned_addr learned my addr 172.21.15.33:0/4148563345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd727fc640 1 --2- 172.21.15.33:0/4148563345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b04883a7a0 0x55b048844490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.016+0000 7fbd72ffd640 1 --2- 172.21.15.33:0/4148563345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883b0c0 0x55b0488449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd727fc640 1 -- 172.21.15.33:0/4148563345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883b0c0 msgr2=0x55b0488449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd727fc640 1 --2- 172.21.15.33:0/4148563345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883b0c0 0x55b0488449d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd727fc640 1 -- 172.21.15.33:0/4148563345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b04883a7a0 msgr2=0x55b048844490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd727fc640 1 --2- 172.21.15.33:0/4148563345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b04883a7a0 0x55b048844490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd727fc640 1 -- 172.21.15.33:0/4148563345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0488452d0 con 0x55b048843ca0 2024-04-02T13:48:36.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd727fc640 1 --2- 172.21.15.33:0/4148563345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b048843ca0 0x55b04884ae90 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fbd6801c2f0 tx=0x7fbd68020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec2af4a03f9ea98 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:36.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:36.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd56ff5640 1 -- 172.21.15.33:0/4148563345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68032060 con 0x55b048843ca0 2024-04-02T13:48:36.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd56ff5640 1 -- 172.21.15.33:0/4148563345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fbd6801e0f0 con 0x55b048843ca0 2024-04-02T13:48:36.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/4148563345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd6801cc90 con 0x55b048843ca0 2024-04-02T13:48:36.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd56ff5640 1 -- 172.21.15.33:0/4148563345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbd6800e6b0 con 0x55b048843ca0 2024-04-02T13:48:36.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd7bfbb740 10 client.17727 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:36.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/4148563345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b04884c500 con 0x55b048843ca0 2024-04-02T13:48:36.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd56ff5640 1 -- 172.21.15.33:0/4148563345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fbd6801cc90 con 0x55b048843ca0 2024-04-02T13:48:36.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd56ff5640 1 -- 172.21.15.33:0/4148563345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fbd6802a040 con 0x55b048843ca0 2024-04-02T13:48:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd7bfbb740 10 client.17727 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd7bfbb740 10 client.17727 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:48:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd7bfbb740 20 client.17727 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/4148563345 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b0488ba400 0x55b0488bc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd7bfbb740 4 client.17727 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.017+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/4148563345 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0487a6040 con 0x55b0488ba400 2024-04-02T13:48:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd72ffd640 1 --2- 172.21.15.33:0/4148563345 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b0488ba400 0x55b0488bc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd72ffd640 1 --2- 172.21.15.33:0/4148563345 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b0488ba400 0x55b0488bc820 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=771af42538bb85c2 server_cookie=ff4625e32fae363f in_seq=0 out_seq=0 2024-04-02T13:48:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd56ff5640 10 client.17727 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:48:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd56ff5640 1 -- 172.21.15.33:0/4148563345 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0487a6040 con 0x55b0488ba400 2024-04-02T13:48:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd56ff5640 10 client.17727 handle_command_reply: tid=0 2024-04-02T13:48:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd7bfbb740 1 client.17727 shutdown 2024-04-02T13:48:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/4148563345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b048843ca0 msgr2=0x55b04884ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/4148563345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b048843ca0 0x55b04884ae90 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fbd6801c2f0 tx=0x7fbd68020040 comp rx=0 tx=0).stop 2024-04-02T13:48:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/4148563345 shutdown_connections 2024-04-02T13:48:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/4148563345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b048843ca0 0x55b04884ae90 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/4148563345 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x55b0488ba400 0x55b0488bc820 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/4148563345 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b04883b0c0 0x55b0488449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd7bfbb740 1 --2- 172.21.15.33:0/4148563345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b04883a7a0 0x55b048844490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/4148563345 >> 172.21.15.33:0/4148563345 conn(0x55b048834ed0 msgr2=0x55b048839f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd56ff5640 0 client.17727 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:48:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/4148563345 shutdown_connections 2024-04-02T13:48:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.018+0000 7fbd7bfbb740 1 -- 172.21.15.33:0/4148563345 wait complete. 2024-04-02T13:48:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.019+0000 7fbd7bfbb740 20 client.17727 trim_cache size 0 max 16384 2024-04-02T13:48:36.031 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:48:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.019+0000 7fbd793cd640 1 -- 172.21.15.33:0/65920612 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbd5c0a3d20 msgr2=0x7fbd5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.019+0000 7fbd793cd640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbd5c0a3d20 0x7fbd5c0a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbd640f0a70 tx=0x7fbd640bad90 comp rx=0 tx=0).stop 2024-04-02T13:48:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.019+0000 7fbd793cd640 1 -- 172.21.15.33:0/65920612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741069b0 msgr2=0x7fbd7419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.019+0000 7fbd793cd640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741069b0 0x7fbd7419cce0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fbd64004af0 tx=0x7fbd64034d70 comp rx=0 tx=0).stop 2024-04-02T13:48:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.019+0000 7fbd793cd640 1 -- 172.21.15.33:0/65920612 shutdown_connections 2024-04-02T13:48:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.019+0000 7fbd793cd640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fbd5c0a3d20 0x7fbd5c0a61e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.019+0000 7fbd793cd640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741069b0 0x7fbd7419cce0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.019+0000 7fbd793cd640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd741013b0 0x7fbd74198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.019+0000 7fbd793cd640 1 --2- 172.21.15.33:0/65920612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbd741009f0 0x7fbd74198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.019+0000 7fbd793cd640 1 -- 172.21.15.33:0/65920612 >> 172.21.15.33:0/65920612 conn(0x7fbd74077b60 msgr2=0x7fbd740ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.019+0000 7fbd793cd640 1 -- 172.21.15.33:0/65920612 shutdown_connections 2024-04-02T13:48:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.019+0000 7fbd793cd640 1 -- 172.21.15.33:0/65920612 wait complete. 2024-04-02T13:48:36.033 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:48:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.135+0000 7fe38331c640 1 -- 172.21.15.33:0/3693824387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c106980 msgr2=0x7fe37c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.135+0000 7fe38331c640 1 --2- 172.21.15.33:0/3693824387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c106980 0x7fe37c106d60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe374009930 tx=0x7fe37402bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.135+0000 7fe38331c640 1 -- 172.21.15.33:0/3693824387 shutdown_connections 2024-04-02T13:48:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.135+0000 7fe38331c640 1 --2- 172.21.15.33:0/3693824387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c101330 0x7fe37c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.135+0000 7fe38331c640 1 --2- 172.21.15.33:0/3693824387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe37c100970 0x7fe37c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.135+0000 7fe38331c640 1 --2- 172.21.15.33:0/3693824387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c106980 0x7fe37c106d60 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.136+0000 7fe38331c640 1 -- 172.21.15.33:0/3693824387 >> 172.21.15.33:0/3693824387 conn(0x7fe37c077b00 msgr2=0x7fe37c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.136+0000 7fe38331c640 1 -- 172.21.15.33:0/3693824387 shutdown_connections 2024-04-02T13:48:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.136+0000 7fe38331c640 1 -- 172.21.15.33:0/3693824387 wait complete. 2024-04-02T13:48:36.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.136+0000 7fe38331c640 1 Processor -- start 2024-04-02T13:48:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.136+0000 7fe38331c640 1 -- start start 2024-04-02T13:48:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.136+0000 7fe38331c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c100970 0x7fe37c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.136+0000 7fe38331c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c101330 0x7fe37c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.136+0000 7fe38331c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe37c106980 0x7fe37c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.136+0000 7fe38331c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c110d30 con 0x7fe37c101330 2024-04-02T13:48:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.136+0000 7fe38331c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c110bb0 con 0x7fe37c100970 2024-04-02T13:48:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.136+0000 7fe38331c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c110eb0 con 0x7fe37c106980 2024-04-02T13:48:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe3808b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c101330 0x7fe37c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe3810b8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c100970 0x7fe37c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe3808b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c101330 0x7fe37c197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53316/0 (socket says 172.21.15.33:53316) 2024-04-02T13:48:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe3808b7640 1 -- 172.21.15.33:0/3874468821 learned_addr learned my addr 172.21.15.33:0/3874468821 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:36.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe3818b9640 1 --2- 172.21.15.33:0/3874468821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe37c106980 0x7fe37c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe3808b7640 1 -- 172.21.15.33:0/3874468821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe37c106980 msgr2=0x7fe37c19bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe3808b7640 1 --2- 172.21.15.33:0/3874468821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe37c106980 0x7fe37c19bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe3808b7640 1 -- 172.21.15.33:0/3874468821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c100970 msgr2=0x7fe37c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe3808b7640 1 --2- 172.21.15.33:0/3874468821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c100970 0x7fe37c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe3808b7640 1 -- 172.21.15.33:0/3874468821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c19c240 con 0x7fe37c101330 2024-04-02T13:48:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe3808b7640 1 --2- 172.21.15.33:0/3874468821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c101330 0x7fe37c197790 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe37000c920 tx=0x7fe37000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe37a7fc640 1 -- 172.21.15.33:0/3874468821 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370007c70 con 0x7fe37c101330 2024-04-02T13:48:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe37a7fc640 1 -- 172.21.15.33:0/3874468821 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe370007e10 con 0x7fe37c101330 2024-04-02T13:48:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe38331c640 1 -- 172.21.15.33:0/3874468821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c19c530 con 0x7fe37c101330 2024-04-02T13:48:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe37a7fc640 1 -- 172.21.15.33:0/3874468821 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370018850 con 0x7fe37c101330 2024-04-02T13:48:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.137+0000 7fe38331c640 1 -- 172.21.15.33:0/3874468821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1a3dc0 con 0x7fe37c101330 2024-04-02T13:48:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.139+0000 7fe37a7fc640 1 -- 172.21.15.33:0/3874468821 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe370018a30 con 0x7fe37c101330 2024-04-02T13:48:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe385fe5740 1 Processor -- start 2024-04-02T13:48:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe385fe5740 1 -- start start 2024-04-02T13:48:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe385fe5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23549b0 0x55afb2354d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe385fe5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23552d0 0x55afb235d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe385fe5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb23602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe385fe5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55afb22b67b0 con 0x55afb235deb0 2024-04-02T13:48:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe385fe5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55afb2255570 con 0x55afb23552d0 2024-04-02T13:48:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe385fe5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55afb22b2760 con 0x55afb23549b0 2024-04-02T13:48:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe3808b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb23602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe37a7fc640 1 --2- 172.21.15.33:0/3874468821 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe3640a3d20 0x7fe3640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe3808b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb23602a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53318/0 (socket says 172.21.15.33:53318) 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe3808b7640 1 -- 172.21.15.33:0/3177286053 learned_addr learned my addr 172.21.15.33:0/3177286053 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe3818b9640 1 --2- 172.21.15.33:0/3177286053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23549b0 0x55afb2354d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe3810b8640 1 --2- 172.21.15.33:0/3177286053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23552d0 0x55afb235d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe3818b9640 1 --2- 172.21.15.33:0/3874468821 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe3640a3d20 0x7fe3640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe37a7fc640 1 -- 172.21.15.33:0/3874468821 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe3700d0d00 con 0x7fe37c101330 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.140+0000 7fe3808b7640 1 -- 172.21.15.33:0/3177286053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23549b0 msgr2=0x55afb2354d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.141+0000 7fe3808b7640 1 --2- 172.21.15.33:0/3177286053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23549b0 0x55afb2354d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.141+0000 7fe3808b7640 1 -- 172.21.15.33:0/3177286053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23552d0 msgr2=0x55afb235d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.141+0000 7fe3808b7640 1 --2- 172.21.15.33:0/3177286053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23552d0 0x55afb235d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.141+0000 7fe3808b7640 1 -- 172.21.15.33:0/3177286053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afb1fca3f0 con 0x55afb235deb0 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.141+0000 7fe3818b9640 1 --2- 172.21.15.33:0/3874468821 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe3640a3d20 0x7fe3640a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005c40 tx=0x7fe36c009500 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.141+0000 7fe3808b7640 1 --2- 172.21.15.33:0/3177286053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb23602a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe3700a5010 tx=0x7fe370000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23326d7d7a9b862e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.141+0000 7fe3637fe640 1 -- 172.21.15.33:0/3177286053 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37000fb90 con 0x55afb235deb0 2024-04-02T13:48:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.141+0000 7fe3637fe640 1 -- 172.21.15.33:0/3177286053 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe37009c8d0 con 0x55afb235deb0 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.141+0000 7fe3637fe640 1 -- 172.21.15.33:0/3177286053 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3700ce300 con 0x55afb235deb0 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 -- 172.21.15.33:0/3177286053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 msgr2=0x55afb23602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 --2- 172.21.15.33:0/3177286053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb23602a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe3700a5010 tx=0x7fe370000bc0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 -- 172.21.15.33:0/3177286053 shutdown_connections 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 --2- 172.21.15.33:0/3177286053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb23602a0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 --2- 172.21.15.33:0/3177286053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23552d0 0x55afb235d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 --2- 172.21.15.33:0/3177286053 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23549b0 0x55afb2354d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 -- 172.21.15.33:0/3177286053 >> 172.21.15.33:0/3177286053 conn(0x55afb234f0e0 msgr2=0x55afb234f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 -- 172.21.15.33:0/3177286053 shutdown_connections 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 -- 172.21.15.33:0/3177286053 wait complete. 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 -- start start 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 Processor -- start 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb235d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23552d0 0x55afb235ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb235b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55afb2360ca0 con 0x55afb235deb0 2024-04-02T13:48:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55afb2271e80 con 0x55afb23549b0 2024-04-02T13:48:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.142+0000 7fe385fe5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55afb2360e20 con 0x55afb23552d0 2024-04-02T13:48:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe385fe5740 10 client.0 _ll_register_callbacks cb 0x55afb22adaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe3818b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb235b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe3818b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb235b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53324/0 (socket says 172.21.15.33:53324) 2024-04-02T13:48:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe3818b9640 1 -- 172.21.15.33:0/2634526934 learned_addr learned my addr 172.21.15.33:0/2634526934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe3810b8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb235d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe3808b7640 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23552d0 0x55afb235ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe3818b9640 1 -- 172.21.15.33:0/2634526934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23552d0 msgr2=0x55afb235ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe3818b9640 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23552d0 0x55afb235ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe3818b9640 1 -- 172.21.15.33:0/2634526934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 msgr2=0x55afb235d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe3818b9640 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb235d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe3818b9640 1 -- 172.21.15.33:0/2634526934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afb20947d0 con 0x55afb235deb0 2024-04-02T13:48:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe3810b8640 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb235d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe3808b7640 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23552d0 0x55afb235ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe3818b9640 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb235b3d0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00dbe0 tx=0x7fe36c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1194b5ba2e36464 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe360ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe360ff9640 1 -- 172.21.15.33:0/2634526934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c025070 con 0x55afb235deb0 2024-04-02T13:48:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe385fe5740 1 -- 172.21.15.33:0/2634526934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55afb22c3cf0 con 0x55afb235deb0 2024-04-02T13:48:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe360ff9640 1 -- 172.21.15.33:0/2634526934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe36c009040 con 0x55afb235deb0 2024-04-02T13:48:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe360ff9640 1 -- 172.21.15.33:0/2634526934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe36c030050 con 0x55afb235deb0 2024-04-02T13:48:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe385fe5740 10 client.17739 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:36.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.143+0000 7fe385fe5740 1 -- 172.21.15.33:0/2634526934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55afb236cd50 con 0x55afb235deb0 2024-04-02T13:48:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.144+0000 7fe360ff9640 1 -- 172.21.15.33:0/2634526934 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x55afb22c3cf0 con 0x55afb235deb0 2024-04-02T13:48:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.144+0000 7fe360ff9640 1 -- 172.21.15.33:0/2634526934 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fe36c0054c0 con 0x55afb235deb0 2024-04-02T13:48:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.144+0000 7fe385fe5740 10 client.17739 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.144+0000 7fe385fe5740 10 client.17739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.144+0000 7fe385fe5740 20 client.17739 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.144+0000 7fe385fe5740 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55afb236e1b0 0x55afb2370610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.144+0000 7fe385fe5740 4 client.17739 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.144+0000 7fe385fe5740 1 -- 172.21.15.33:0/2634526934 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afb22be040 con 0x55afb236e1b0 2024-04-02T13:48:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.144+0000 7fe3810b8640 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55afb236e1b0 0x55afb2370610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe360ff9640 10 client.17739 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe3810b8640 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55afb236e1b0 0x55afb2370610 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af000ada9b93b35b server_cookie=52df4d31390e006a in_seq=0 out_seq=0 2024-04-02T13:48:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe360ff9640 1 -- 172.21.15.33:0/2634526934 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afb22be040 con 0x55afb236e1b0 2024-04-02T13:48:36.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe360ff9640 10 client.17739 handle_command_reply: tid=0 2024-04-02T13:48:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe385fe5740 1 client.17739 shutdown 2024-04-02T13:48:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe385fe5740 1 -- 172.21.15.33:0/2634526934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 msgr2=0x55afb235b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe385fe5740 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb235b3d0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00dbe0 tx=0x7fe36c004a40 comp rx=0 tx=0).stop 2024-04-02T13:48:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe385fe5740 1 -- 172.21.15.33:0/2634526934 shutdown_connections 2024-04-02T13:48:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe385fe5740 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55afb236e1b0 0x55afb2370610 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe385fe5740 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb235b3d0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe385fe5740 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23552d0 0x55afb235ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe360ff9640 0 client.17739 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe385fe5740 1 --2- 172.21.15.33:0/2634526934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb235d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.145+0000 7fe385fe5740 1 -- 172.21.15.33:0/2634526934 >> 172.21.15.33:0/2634526934 conn(0x55afb234f0e0 msgr2=0x55afb2350e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.146+0000 7fe385fe5740 1 -- 172.21.15.33:0/2634526934 shutdown_connections 2024-04-02T13:48:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.146+0000 7fe385fe5740 1 -- 172.21.15.33:0/2634526934 wait complete. 2024-04-02T13:48:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.146+0000 7fe385fe5740 20 client.17739 trim_cache size 0 max 16384 2024-04-02T13:48:36.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe385fe5740 1 Processor -- start 2024-04-02T13:48:36.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe385fe5740 1 -- start start 2024-04-02T13:48:36.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe385fe5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb2369b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe385fe5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb23552d0 0x55afb2375ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe385fe5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb235deb0 0x55afb2376420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe385fe5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1af710 con 0x55afb23552d0 2024-04-02T13:48:36.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe385fe5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1acfe0 con 0x55afb23549b0 2024-04-02T13:48:36.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe385fe5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1aae50 con 0x55afb235deb0 2024-04-02T13:48:36.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe3808b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb23552d0 0x55afb2375ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe3808b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb23552d0 0x55afb2375ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53328/0 (socket says 172.21.15.33:53328) 2024-04-02T13:48:36.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe3808b7640 1 -- 172.21.15.33:0/4275331722 learned_addr learned my addr 172.21.15.33:0/4275331722 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:36.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe3810b8640 1 --2- 172.21.15.33:0/4275331722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb2369b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe3808b7640 1 -- 172.21.15.33:0/4275331722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb235deb0 msgr2=0x55afb2376420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe3818b9640 1 --2- 172.21.15.33:0/4275331722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb235deb0 0x55afb2376420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe3808b7640 1 --2- 172.21.15.33:0/4275331722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb235deb0 0x55afb2376420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe3808b7640 1 -- 172.21.15.33:0/4275331722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 msgr2=0x55afb2369b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe3808b7640 1 --2- 172.21.15.33:0/4275331722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb2369b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe3808b7640 1 -- 172.21.15.33:0/4275331722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afb236a7d0 con 0x55afb23552d0 2024-04-02T13:48:36.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.159+0000 7fe3810b8640 1 --2- 172.21.15.33:0/4275331722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb2369b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:36.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.160+0000 7fe3808b7640 1 --2- 172.21.15.33:0/4275331722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb23552d0 0x55afb2375ee0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe37009ca90 tx=0x7fe370097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4f2e4a91952b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:36.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.160+0000 7fe362ffd640 1 -- 172.21.15.33:0/4275331722 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37009cfd0 con 0x55afb23552d0 2024-04-02T13:48:36.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.160+0000 7fe362ffd640 1 -- 172.21.15.33:0/4275331722 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe3700a0c80 con 0x55afb23552d0 2024-04-02T13:48:36.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.160+0000 7fe362ffd640 1 -- 172.21.15.33:0/4275331722 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370098f40 con 0x55afb23552d0 2024-04-02T13:48:36.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.160+0000 7fe385fe5740 1 -- 172.21.15.33:0/4275331722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb23552d0 msgr2=0x55afb2375ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.160+0000 7fe385fe5740 1 --2- 172.21.15.33:0/4275331722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb23552d0 0x55afb2375ee0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe37009ca90 tx=0x7fe370097040 comp rx=0 tx=0).stop 2024-04-02T13:48:36.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.160+0000 7fe385fe5740 1 -- 172.21.15.33:0/4275331722 shutdown_connections 2024-04-02T13:48:36.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.160+0000 7fe385fe5740 1 --2- 172.21.15.33:0/4275331722 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb235deb0 0x55afb2376420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.160+0000 7fe385fe5740 1 --2- 172.21.15.33:0/4275331722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb23552d0 0x55afb2375ee0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.160+0000 7fe385fe5740 1 --2- 172.21.15.33:0/4275331722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb2369b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.160+0000 7fe385fe5740 1 -- 172.21.15.33:0/4275331722 >> 172.21.15.33:0/4275331722 conn(0x55afb234f0e0 msgr2=0x55afb2355f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:36.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.160+0000 7fe385fe5740 1 -- 172.21.15.33:0/4275331722 shutdown_connections 2024-04-02T13:48:36.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.160+0000 7fe385fe5740 1 -- 172.21.15.33:0/4275331722 wait complete. 2024-04-02T13:48:36.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe385fe5740 1 -- start start 2024-04-02T13:48:36.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe385fe5740 1 Processor -- start 2024-04-02T13:48:36.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe385fe5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb23665f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe385fe5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23552d0 0x55afb235ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe385fe5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb235f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe385fe5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1ab4a0 con 0x55afb235deb0 2024-04-02T13:48:36.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe385fe5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1ac910 con 0x55afb23549b0 2024-04-02T13:48:36.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe385fe5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1abb70 con 0x55afb23552d0 2024-04-02T13:48:36.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe385fe5740 10 client.0 _ll_register_callbacks cb 0x7fe36c003d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:36.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe3818b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb235f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe3818b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb235f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53332/0 (socket says 172.21.15.33:53332) 2024-04-02T13:48:36.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe3818b9640 1 -- 172.21.15.33:0/579895620 learned_addr learned my addr 172.21.15.33:0/579895620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:36.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe3810b8640 1 --2- 172.21.15.33:0/579895620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb23665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe3808b7640 1 --2- 172.21.15.33:0/579895620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23552d0 0x55afb235ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe3818b9640 1 -- 172.21.15.33:0/579895620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23552d0 msgr2=0x55afb235ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.161+0000 7fe3818b9640 1 --2- 172.21.15.33:0/579895620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23552d0 0x55afb235ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe3818b9640 1 -- 172.21.15.33:0/579895620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 msgr2=0x55afb23665f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:36.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe3818b9640 1 --2- 172.21.15.33:0/579895620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb23665f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe3818b9640 1 -- 172.21.15.33:0/579895620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afb23640d0 con 0x55afb235deb0 2024-04-02T13:48:36.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe3818b9640 1 --2- 172.21.15.33:0/579895620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb235f2e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00dbb0 tx=0x7fe36c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c47a4c3f6069d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:36.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe360ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:36.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe360ff9640 1 -- 172.21.15.33:0/579895620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c025040 con 0x55afb235deb0 2024-04-02T13:48:36.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe360ff9640 1 -- 172.21.15.33:0/579895620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe36c012930 con 0x55afb235deb0 2024-04-02T13:48:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe385fe5740 1 -- 172.21.15.33:0/579895620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3700a0c80 con 0x55afb235deb0 2024-04-02T13:48:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe360ff9640 1 -- 172.21.15.33:0/579895620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe36c004400 con 0x55afb235deb0 2024-04-02T13:48:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe385fe5740 10 client.17745 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe385fe5740 1 -- 172.21.15.33:0/579895620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55afb23d4840 con 0x55afb235deb0 2024-04-02T13:48:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe360ff9640 1 -- 172.21.15.33:0/579895620 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe3700a0c80 con 0x55afb235deb0 2024-04-02T13:48:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe360ff9640 1 -- 172.21.15.33:0/579895620 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fe36c029840 con 0x55afb235deb0 2024-04-02T13:48:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe385fe5740 10 client.17745 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe385fe5740 10 client.17745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe385fe5740 20 client.17745 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe385fe5740 1 --2- 172.21.15.33:0/579895620 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55afb23d4f50 0x55afb23d7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe385fe5740 4 client.17745 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.162+0000 7fe385fe5740 1 -- 172.21.15.33:0/579895620 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55afb223b6b0 con 0x55afb23d4f50 2024-04-02T13:48:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe3810b8640 1 --2- 172.21.15.33:0/579895620 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55afb23d4f50 0x55afb23d7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe360ff9640 10 client.17745 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe3810b8640 1 --2- 172.21.15.33:0/579895620 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55afb23d4f50 0x55afb23d7370 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=579f5b1fdfb317ba server_cookie=c595efb157023753 in_seq=0 out_seq=0 2024-04-02T13:48:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe360ff9640 1 -- 172.21.15.33:0/579895620 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55afb223b6b0 con 0x55afb23d4f50 2024-04-02T13:48:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe360ff9640 10 client.17745 handle_command_reply: tid=0 2024-04-02T13:48:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe385fe5740 1 client.17745 shutdown 2024-04-02T13:48:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe385fe5740 1 -- 172.21.15.33:0/579895620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 msgr2=0x55afb235f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe385fe5740 1 --2- 172.21.15.33:0/579895620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb235f2e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00dbb0 tx=0x7fe36c0049d0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe385fe5740 1 -- 172.21.15.33:0/579895620 shutdown_connections 2024-04-02T13:48:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe385fe5740 1 --2- 172.21.15.33:0/579895620 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55afb23d4f50 0x55afb23d7370 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe385fe5740 1 --2- 172.21.15.33:0/579895620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afb235deb0 0x55afb235f2e0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe360ff9640 0 client.17745 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe385fe5740 1 --2- 172.21.15.33:0/579895620 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55afb23552d0 0x55afb235ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe385fe5740 1 --2- 172.21.15.33:0/579895620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afb23549b0 0x55afb23665f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe385fe5740 1 -- 172.21.15.33:0/579895620 >> 172.21.15.33:0/579895620 conn(0x55afb234f0e0 msgr2=0x55afb234b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.163+0000 7fe385fe5740 1 -- 172.21.15.33:0/579895620 shutdown_connections 2024-04-02T13:48:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe385fe5740 1 -- 172.21.15.33:0/579895620 wait complete. 2024-04-02T13:48:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe385fe5740 20 client.17745 trim_cache size 0 max 16384 2024-04-02T13:48:36.175 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:48:36.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe38331c640 1 -- 172.21.15.33:0/3874468821 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe3640a3d20 msgr2=0x7fe3640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe38331c640 1 --2- 172.21.15.33:0/3874468821 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe3640a3d20 0x7fe3640a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005c40 tx=0x7fe36c009500 comp rx=0 tx=0).stop 2024-04-02T13:48:36.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe38331c640 1 -- 172.21.15.33:0/3874468821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c101330 msgr2=0x7fe37c197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:36.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe38331c640 1 --2- 172.21.15.33:0/3874468821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c101330 0x7fe37c197790 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe37000c920 tx=0x7fe37000cde0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe38331c640 1 -- 172.21.15.33:0/3874468821 shutdown_connections 2024-04-02T13:48:36.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe38331c640 1 --2- 172.21.15.33:0/3874468821 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe3640a3d20 0x7fe3640a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe38331c640 1 --2- 172.21.15.33:0/3874468821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe37c106980 0x7fe37c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe38331c640 1 --2- 172.21.15.33:0/3874468821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c101330 0x7fe37c197790 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe38331c640 1 --2- 172.21.15.33:0/3874468821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c100970 0x7fe37c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:36.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe38331c640 1 -- 172.21.15.33:0/3874468821 >> 172.21.15.33:0/3874468821 conn(0x7fe37c077b00 msgr2=0x7fe37c103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:36.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe38331c640 1 -- 172.21.15.33:0/3874468821 shutdown_connections 2024-04-02T13:48:36.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:36.164+0000 7fe38331c640 1 -- 172.21.15.33:0/3874468821 wait complete. 2024-04-02T13:48:36.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:36 smithi145 ceph-mon[35646]: pgmap v305: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:36.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:36 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/4202710770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:48:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:36 smithi050 ceph-mon[35524]: pgmap v305: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:36 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/4202710770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:48:37.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:36 smithi033 ceph-mon[32390]: pgmap v305: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:37.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:36 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/4202710770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:48:37.179 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:48:37.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:48:37.179 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:48:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.296+0000 7f84a7fe9640 1 -- 172.21.15.33:0/4084652983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a0112aa0 msgr2=0x7f84a0114e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.296+0000 7f84a7fe9640 1 --2- 172.21.15.33:0/4084652983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a0112aa0 0x7f84a0114e90 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f8498009900 tx=0x7f849802bcd0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.296+0000 7f84a7fe9640 1 -- 172.21.15.33:0/4084652983 shutdown_connections 2024-04-02T13:48:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.296+0000 7f84a7fe9640 1 --2- 172.21.15.33:0/4084652983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a0112aa0 0x7f84a0114e90 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.296+0000 7f84a7fe9640 1 --2- 172.21.15.33:0/4084652983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84a00747c0 0x7f84a0112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.296+0000 7f84a7fe9640 1 --2- 172.21.15.33:0/4084652983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84a0073ea0 0x7f84a0074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.296+0000 7f84a7fe9640 1 -- 172.21.15.33:0/4084652983 >> 172.21.15.33:0/4084652983 conn(0x7f84a00fd5a0 msgr2=0x7f84a00ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.296+0000 7f84a7fe9640 1 -- 172.21.15.33:0/4084652983 shutdown_connections 2024-04-02T13:48:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.296+0000 7f84a7fe9640 1 -- 172.21.15.33:0/4084652983 wait complete. 2024-04-02T13:48:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.296+0000 7f84a7fe9640 1 Processor -- start 2024-04-02T13:48:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a7fe9640 1 -- start start 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a7fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a0073ea0 0x7f84a01a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a7fe9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84a00747c0 0x7f84a01a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a7fe9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84a0112aa0 0x7f84a019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a7fe9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01184c0 con 0x7f84a0073ea0 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a7fe9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0118340 con 0x7f84a00747c0 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a7fe9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0118640 con 0x7f84a0112aa0 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a5d85640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a0073ea0 0x7f84a01a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a5d85640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a0073ea0 0x7f84a01a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53356/0 (socket says 172.21.15.33:53356) 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a5d85640 1 -- 172.21.15.33:0/16755992 learned_addr learned my addr 172.21.15.33:0/16755992 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a6586640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84a0112aa0 0x7f84a019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a5584640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84a00747c0 0x7f84a01a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a5d85640 1 -- 172.21.15.33:0/16755992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84a0112aa0 msgr2=0x7f84a019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a5d85640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84a0112aa0 0x7f84a019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a5d85640 1 -- 172.21.15.33:0/16755992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84a00747c0 msgr2=0x7f84a01a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a5d85640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84a00747c0 0x7f84a01a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a5d85640 1 -- 172.21.15.33:0/16755992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a019f010 con 0x7f84a0073ea0 2024-04-02T13:48:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f84a5d85640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a0073ea0 0x7f84a01a4680 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f849c00db80 tx=0x7f849c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f8496ffd640 1 -- 172.21.15.33:0/16755992 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c017070 con 0x7f84a0073ea0 2024-04-02T13:48:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.297+0000 7f8496ffd640 1 -- 172.21.15.33:0/16755992 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f849c003b70 con 0x7f84a0073ea0 2024-04-02T13:48:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.298+0000 7f84a7fe9640 1 -- 172.21.15.33:0/16755992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a019f300 con 0x7f84a0073ea0 2024-04-02T13:48:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.298+0000 7f8496ffd640 1 -- 172.21.15.33:0/16755992 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c0106c0 con 0x7f84a0073ea0 2024-04-02T13:48:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.298+0000 7f84a7fe9640 1 -- 172.21.15.33:0/16755992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a01ab4d0 con 0x7f84a0073ea0 2024-04-02T13:48:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.300+0000 7f8496ffd640 1 -- 172.21.15.33:0/16755992 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f849c004020 con 0x7f84a0073ea0 2024-04-02T13:48:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.300+0000 7f84aabd0740 1 Processor -- start 2024-04-02T13:48:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.300+0000 7f84aabd0740 1 -- start start 2024-04-02T13:48:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f8496ffd640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f848c0a3da0 0x7f848c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84aabd0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0a9f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84aabd0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aa8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84aabd0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 0x55e1c0aaabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84aabd0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e1c0a03770 con 0x55e1c0aa87c0 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84aabd0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e1c09ab960 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84aabd0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e1c09ff8c0 con 0x55e1c0a9f2c0 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f8496ffd640 1 -- 172.21.15.33:0/16755992 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f849c0d0cb0 con 0x7f84a0073ea0 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a6586640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f848c0a3da0 0x7f848c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a5584640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aa8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a6586640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 0x55e1c0aaabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a5584640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aa8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:40534/0 (socket says 172.21.15.33:40534) 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a6586640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 0x55e1c0aaabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53358/0 (socket says 172.21.15.33:53358) 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a6586640 1 -- 172.21.15.33:0/2387282682 learned_addr learned my addr 172.21.15.33:0/2387282682 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a5d85640 1 --2- 172.21.15.33:0/2387282682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0a9f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a6586640 1 -- 172.21.15.33:0/2387282682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0a9f2c0 msgr2=0x55e1c0a9f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a6586640 1 --2- 172.21.15.33:0/2387282682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0a9f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a6586640 1 -- 172.21.15.33:0/2387282682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9fbe0 msgr2=0x55e1c0aa8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a6586640 1 --2- 172.21.15.33:0/2387282682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aa8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a6586640 1 -- 172.21.15.33:0/2387282682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c07df7e0 con 0x55e1c0aa87c0 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a6586640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f848c0a3da0 0x7f848c0a6260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f84a006b1d0 tx=0x7f8498004130 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.301+0000 7f84a6586640 1 --2- 172.21.15.33:0/2387282682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 0x55e1c0aaabb0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8498009900 tx=0x7f8498047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ee323c9c923fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.302+0000 7f848bfff640 1 -- 172.21.15.33:0/2387282682 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498051070 con 0x55e1c0aa87c0 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.302+0000 7f848bfff640 1 -- 172.21.15.33:0/2387282682 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8498005610 con 0x55e1c0aa87c0 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.302+0000 7f848bfff640 1 -- 172.21.15.33:0/2387282682 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849802ce60 con 0x55e1c0aa87c0 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.302+0000 7f84aabd0740 1 -- 172.21.15.33:0/2387282682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 msgr2=0x55e1c0aaabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.302+0000 7f84aabd0740 1 --2- 172.21.15.33:0/2387282682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 0x55e1c0aaabb0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8498009900 tx=0x7f8498047000 comp rx=0 tx=0).stop 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.302+0000 7f84aabd0740 1 -- 172.21.15.33:0/2387282682 shutdown_connections 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.302+0000 7f84aabd0740 1 --2- 172.21.15.33:0/2387282682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 0x55e1c0aaabb0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.302+0000 7f84aabd0740 1 --2- 172.21.15.33:0/2387282682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aa8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.302+0000 7f84aabd0740 1 --2- 172.21.15.33:0/2387282682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0a9f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.302+0000 7f84aabd0740 1 -- 172.21.15.33:0/2387282682 >> 172.21.15.33:0/2387282682 conn(0x55e1c0a999f0 msgr2=0x55e1c0a99dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.302+0000 7f84aabd0740 1 -- 172.21.15.33:0/2387282682 shutdown_connections 2024-04-02T13:48:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.302+0000 7f84aabd0740 1 -- 172.21.15.33:0/2387282682 wait complete. 2024-04-02T13:48:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.303+0000 7f84aabd0740 1 -- start start 2024-04-02T13:48:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.303+0000 7f84aabd0740 1 Processor -- start 2024-04-02T13:48:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.303+0000 7f84aabd0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0aa5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.303+0000 7f84aabd0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aa5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.303+0000 7f84aabd0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0aa87c0 0x55e1c0aa62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.303+0000 7f84aabd0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e1c0aab550 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.303+0000 7f84aabd0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e1c09bed20 con 0x55e1c0a9f2c0 2024-04-02T13:48:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.303+0000 7f84aabd0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55e1c0aab6d0 con 0x55e1c0aa87c0 2024-04-02T13:48:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.303+0000 7f84aabd0740 10 client.0 _ll_register_callbacks cb 0x55e1c098ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.303+0000 7f84a5584640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aa5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.303+0000 7f84a5584640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aa5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53374/0 (socket says 172.21.15.33:53374) 2024-04-02T13:48:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.303+0000 7f84a5584640 1 -- 172.21.15.33:0/3895526320 learned_addr learned my addr 172.21.15.33:0/3895526320 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84a5d85640 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0aa5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84a6586640 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0aa87c0 0x55e1c0aa62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84a5584640 1 -- 172.21.15.33:0/3895526320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0aa87c0 msgr2=0x55e1c0aa62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84a5584640 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0aa87c0 0x55e1c0aa62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84a5584640 1 -- 172.21.15.33:0/3895526320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9f2c0 msgr2=0x55e1c0aa5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84a5584640 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0aa5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84a5584640 1 -- 172.21.15.33:0/3895526320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c083ff70 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84a5d85640 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0aa5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84a5584640 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aa5d60 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f849000be80 tx=0x7f8490012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd6d12c1925bec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84897fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84897fa640 1 -- 172.21.15.33:0/3895526320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490005220 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84897fa640 1 -- 172.21.15.33:0/3895526320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f84900053c0 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84aabd0740 1 -- 172.21.15.33:0/3895526320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8498005610 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84897fa640 1 -- 172.21.15.33:0/3895526320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8490025050 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84aabd0740 10 client.17757 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84aabd0740 1 -- 172.21.15.33:0/3895526320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e1c06d7170 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.304+0000 7f84897fa640 1 -- 172.21.15.33:0/3895526320 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8498005610 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.305+0000 7f84897fa640 1 -- 172.21.15.33:0/3895526320 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f8490019030 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.305+0000 7f84aabd0740 10 client.17757 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.305+0000 7f84aabd0740 10 client.17757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.305+0000 7f84aabd0740 20 client.17757 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.305+0000 7f84aabd0740 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e1c0ab8b70 0x55e1c0abb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.305+0000 7f84aabd0740 4 client.17757 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.305+0000 7f84aabd0740 1 -- 172.21.15.33:0/3895526320 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1c0a10cb0 con 0x55e1c0ab8b70 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.305+0000 7f84a5d85640 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e1c0ab8b70 0x55e1c0abb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.305+0000 7f84897fa640 10 client.17757 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.305+0000 7f84a5d85640 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e1c0ab8b70 0x55e1c0abb010 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd4f0b3d4b9d1b5d server_cookie=698c74396214b9a2 in_seq=0 out_seq=0 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84897fa640 1 -- 172.21.15.33:0/3895526320 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1c0a10cb0 con 0x55e1c0ab8b70 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84897fa640 10 client.17757 handle_command_reply: tid=0 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84aabd0740 1 client.17757 shutdown 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84aabd0740 1 -- 172.21.15.33:0/3895526320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 msgr2=0x55e1c0aa5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84aabd0740 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aa5d60 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f849000be80 tx=0x7f8490012820 comp rx=0 tx=0).stop 2024-04-02T13:48:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84aabd0740 1 -- 172.21.15.33:0/3895526320 shutdown_connections 2024-04-02T13:48:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84aabd0740 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e1c0ab8b70 0x55e1c0abb010 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84aabd0740 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0aa87c0 0x55e1c0aa62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84aabd0740 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aa5d60 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84aabd0740 1 --2- 172.21.15.33:0/3895526320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0aa5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84897fa640 0 client.17757 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84aabd0740 1 -- 172.21.15.33:0/3895526320 >> 172.21.15.33:0/3895526320 conn(0x55e1c0a999f0 msgr2=0x55e1c0a9b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84aabd0740 1 -- 172.21.15.33:0/3895526320 shutdown_connections 2024-04-02T13:48:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84aabd0740 1 -- 172.21.15.33:0/3895526320 wait complete. 2024-04-02T13:48:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.306+0000 7f84aabd0740 20 client.17757 trim_cache size 0 max 16384 2024-04-02T13:48:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.324+0000 7f84aabd0740 1 Processor -- start 2024-04-02T13:48:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.324+0000 7f84aabd0740 1 -- start start 2024-04-02T13:48:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.324+0000 7f84aabd0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0aa7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.324+0000 7f84aabd0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0ab4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.324+0000 7f84aabd0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0aa87c0 0x55e1c0ac08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.324+0000 7f84aabd0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01b8000 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.324+0000 7f84aabd0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01b58d0 con 0x55e1c0a9f2c0 2024-04-02T13:48:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.324+0000 7f84aabd0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01b3740 con 0x55e1c0aa87c0 2024-04-02T13:48:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.325+0000 7f84a5584640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0ab4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.325+0000 7f84a5584640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0ab4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53386/0 (socket says 172.21.15.33:53386) 2024-04-02T13:48:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.325+0000 7f84a5584640 1 -- 172.21.15.33:0/2261615869 learned_addr learned my addr 172.21.15.33:0/2261615869 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.325+0000 7f84a5d85640 1 --2- 172.21.15.33:0/2261615869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0aa7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.325+0000 7f84a5584640 1 -- 172.21.15.33:0/2261615869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0aa87c0 msgr2=0x55e1c0ac08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.325+0000 7f84a5584640 1 --2- 172.21.15.33:0/2261615869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0aa87c0 0x55e1c0ac08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.325+0000 7f84a5584640 1 -- 172.21.15.33:0/2261615869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9f2c0 msgr2=0x55e1c0aa7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.325+0000 7f84a5584640 1 --2- 172.21.15.33:0/2261615869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0aa7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.325+0000 7f84a5584640 1 -- 172.21.15.33:0/2261615869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c0aa47d0 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.325+0000 7f84a5584640 1 --2- 172.21.15.33:0/2261615869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0ab4410 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f849000b5b0 tx=0x7f8490005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c4349a13a2b9e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.325+0000 7f848b7fe640 1 -- 172.21.15.33:0/2261615869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849002d070 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.325+0000 7f848b7fe640 1 -- 172.21.15.33:0/2261615869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f84900106a0 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.325+0000 7f848b7fe640 1 -- 172.21.15.33:0/2261615869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849001a650 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.326+0000 7f84aabd0740 1 -- 172.21.15.33:0/2261615869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 msgr2=0x55e1c0ab4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.326+0000 7f84aabd0740 1 --2- 172.21.15.33:0/2261615869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0ab4410 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f849000b5b0 tx=0x7f8490005650 comp rx=0 tx=0).stop 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.326+0000 7f84aabd0740 1 -- 172.21.15.33:0/2261615869 shutdown_connections 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.326+0000 7f84aabd0740 1 --2- 172.21.15.33:0/2261615869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0aa87c0 0x55e1c0ac08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.326+0000 7f84aabd0740 1 --2- 172.21.15.33:0/2261615869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0ab4410 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.326+0000 7f84aabd0740 1 --2- 172.21.15.33:0/2261615869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0aa7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.326+0000 7f84aabd0740 1 -- 172.21.15.33:0/2261615869 >> 172.21.15.33:0/2261615869 conn(0x55e1c0a999f0 msgr2=0x55e1c0a9b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.326+0000 7f84aabd0740 1 -- 172.21.15.33:0/2261615869 shutdown_connections 2024-04-02T13:48:37.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.326+0000 7f84aabd0740 1 -- 172.21.15.33:0/2261615869 wait complete. 2024-04-02T13:48:37.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.326+0000 7f84aabd0740 1 -- start start 2024-04-02T13:48:37.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.326+0000 7f84aabd0740 1 Processor -- start 2024-04-02T13:48:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.326+0000 7f84aabd0740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0aacbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84aabd0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84aabd0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 0x55e1c0b18190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84aabd0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01b3d90 con 0x55e1c0aa87c0 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84aabd0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01b5200 con 0x55e1c0a9fbe0 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84aabd0740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01b4460 con 0x55e1c0a9f2c0 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84aabd0740 10 client.0 _ll_register_callbacks cb 0x7f849000b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84a6586640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 0x55e1c0b18190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84a6586640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 0x55e1c0b18190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53396/0 (socket says 172.21.15.33:53396) 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84a6586640 1 -- 172.21.15.33:0/1811430655 learned_addr learned my addr 172.21.15.33:0/1811430655 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84a5584640 1 --2- 172.21.15.33:0/1811430655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84a5d85640 1 --2- 172.21.15.33:0/1811430655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0aacbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84a6586640 1 -- 172.21.15.33:0/1811430655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0a9f2c0 msgr2=0x55e1c0aacbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84a6586640 1 --2- 172.21.15.33:0/1811430655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0aacbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84a6586640 1 -- 172.21.15.33:0/1811430655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9fbe0 msgr2=0x55e1c0aad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84a6586640 1 --2- 172.21.15.33:0/1811430655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84a6586640 1 -- 172.21.15.33:0/1811430655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c0b18750 con 0x55e1c0aa87c0 2024-04-02T13:48:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84a6586640 1 --2- 172.21.15.33:0/1811430655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 0x55e1c0b18190 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f849802c2e0 tx=0x7f8498004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392df61b0c3bc00 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84897fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84897fa640 1 -- 172.21.15.33:0/1811430655 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498059020 con 0x55e1c0aa87c0 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84897fa640 1 -- 172.21.15.33:0/1811430655 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f849802c7b0 con 0x55e1c0aa87c0 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84aabd0740 1 -- 172.21.15.33:0/1811430655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84900106a0 con 0x55e1c0aa87c0 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84897fa640 1 -- 172.21.15.33:0/1811430655 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8498064050 con 0x55e1c0aa87c0 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84aabd0740 10 client.17763 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.327+0000 7f84aabd0740 1 -- 172.21.15.33:0/1811430655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e1c0b1fb40 con 0x55e1c0aa87c0 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.328+0000 7f84897fa640 1 -- 172.21.15.33:0/1811430655 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f84900106a0 con 0x55e1c0aa87c0 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.328+0000 7f84897fa640 1 -- 172.21.15.33:0/1811430655 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f8498054030 con 0x55e1c0aa87c0 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.328+0000 7f84aabd0740 10 client.17763 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.328+0000 7f84aabd0740 10 client.17763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.328+0000 7f84aabd0740 20 client.17763 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.328+0000 7f84aabd0740 1 --2- 172.21.15.33:0/1811430655 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e1c0b1fe20 0x55e1c0b22210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.328+0000 7f84aabd0740 4 client.17763 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:48:37.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.328+0000 7f84aabd0740 1 -- 172.21.15.33:0/1811430655 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e1c0a0b000 con 0x55e1c0b1fe20 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.328+0000 7f84a5d85640 1 --2- 172.21.15.33:0/1811430655 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e1c0b1fe20 0x55e1c0b22210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.329+0000 7f84897fa640 10 client.17763 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.329+0000 7f84a5d85640 1 --2- 172.21.15.33:0/1811430655 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e1c0b1fe20 0x55e1c0b22210 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bdab60998602b87 server_cookie=fccba357c77714e1 in_seq=0 out_seq=0 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84897fa640 1 -- 172.21.15.33:0/1811430655 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e1c0a0b000 con 0x55e1c0b1fe20 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84897fa640 10 client.17763 handle_command_reply: tid=0 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84aabd0740 1 client.17763 shutdown 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84aabd0740 1 -- 172.21.15.33:0/1811430655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 msgr2=0x55e1c0b18190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84aabd0740 1 --2- 172.21.15.33:0/1811430655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 0x55e1c0b18190 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f849802c2e0 tx=0x7f8498004d40 comp rx=0 tx=0).stop 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84aabd0740 1 -- 172.21.15.33:0/1811430655 shutdown_connections 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84aabd0740 1 --2- 172.21.15.33:0/1811430655 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x55e1c0b1fe20 0x55e1c0b22210 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84aabd0740 1 --2- 172.21.15.33:0/1811430655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1c0aa87c0 0x55e1c0b18190 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84aabd0740 1 --2- 172.21.15.33:0/1811430655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e1c0a9fbe0 0x55e1c0aad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84897fa640 0 client.17763 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84aabd0740 1 --2- 172.21.15.33:0/1811430655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e1c0a9f2c0 0x55e1c0aacbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84aabd0740 1 -- 172.21.15.33:0/1811430655 >> 172.21.15.33:0/1811430655 conn(0x55e1c0a999f0 msgr2=0x55e1c0a9ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:37.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84aabd0740 1 -- 172.21.15.33:0/1811430655 shutdown_connections 2024-04-02T13:48:37.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84aabd0740 1 -- 172.21.15.33:0/1811430655 wait complete. 2024-04-02T13:48:37.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.330+0000 7f84aabd0740 20 client.17763 trim_cache size 0 max 16384 2024-04-02T13:48:37.338 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:48:37.338 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-04-02T13:48:37.338 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "0b2c851e-a0a3-46cc-a21f-f53d5dea5dc8", 2024-04-02T13:48:37.338 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-04-02T13:48:37.338 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:48:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.331+0000 7f84a7fe9640 1 -- 172.21.15.33:0/16755992 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f848c0a3da0 msgr2=0x7f848c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.331+0000 7f84a7fe9640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f848c0a3da0 0x7f848c0a6260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f84a006b1d0 tx=0x7f8498004130 comp rx=0 tx=0).stop 2024-04-02T13:48:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.331+0000 7f84a7fe9640 1 -- 172.21.15.33:0/16755992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a0073ea0 msgr2=0x7f84a01a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.331+0000 7f84a7fe9640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a0073ea0 0x7f84a01a4680 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f849c00db80 tx=0x7f849c00b630 comp rx=0 tx=0).stop 2024-04-02T13:48:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.331+0000 7f84a7fe9640 1 -- 172.21.15.33:0/16755992 shutdown_connections 2024-04-02T13:48:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.331+0000 7f84a7fe9640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f848c0a3da0 0x7f848c0a6260 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.331+0000 7f84a7fe9640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f84a0112aa0 0x7f84a019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.331+0000 7f84a7fe9640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84a00747c0 0x7f84a01a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.331+0000 7f84a7fe9640 1 --2- 172.21.15.33:0/16755992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a0073ea0 0x7f84a01a4680 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.331+0000 7f84a7fe9640 1 -- 172.21.15.33:0/16755992 >> 172.21.15.33:0/16755992 conn(0x7f84a00fd5a0 msgr2=0x7f84a00feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.331+0000 7f84a7fe9640 1 -- 172.21.15.33:0/16755992 shutdown_connections 2024-04-02T13:48:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.331+0000 7f84a7fe9640 1 -- 172.21.15.33:0/16755992 wait complete. 2024-04-02T13:48:37.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:48:37.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:48:37.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.446+0000 7f45a152b640 1 -- 172.21.15.33:0/1890921764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f459c101370 msgr2=0x7f459c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.446+0000 7f45a152b640 1 --2- 172.21.15.33:0/1890921764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f459c101370 0x7f459c10d880 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f4594009cd0 tx=0x7f459402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.446+0000 7f45a152b640 1 -- 172.21.15.33:0/1890921764 shutdown_connections 2024-04-02T13:48:37.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.446+0000 7f45a152b640 1 --2- 172.21.15.33:0/1890921764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f459c101370 0x7f459c10d880 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.446+0000 7f45a152b640 1 --2- 172.21.15.33:0/1890921764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c1009b0 0x7f459c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.446+0000 7f45a152b640 1 --2- 172.21.15.33:0/1890921764 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f459c1069c0 0x7f459c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.446+0000 7f45a152b640 1 -- 172.21.15.33:0/1890921764 >> 172.21.15.33:0/1890921764 conn(0x7f459c076d80 msgr2=0x7f459c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.446+0000 7f45a152b640 1 -- 172.21.15.33:0/1890921764 shutdown_connections 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.446+0000 7f45a152b640 1 -- 172.21.15.33:0/1890921764 wait complete. 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f45a152b640 1 Processor -- start 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f45a152b640 1 -- start start 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f45a152b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f459c1009b0 0x7f459c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f45a152b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c101370 0x7f459c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f45a152b640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f459c1069c0 0x7f459c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f45a152b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f459c111ea0 con 0x7f459c1009b0 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f45a152b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f459c111d20 con 0x7f459c101370 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f45a152b640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f459c112020 con 0x7f459c1069c0 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f459affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f459c1009b0 0x7f459c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f459affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f459c1009b0 0x7f459c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53406/0 (socket says 172.21.15.33:53406) 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f459affd640 1 -- 172.21.15.33:0/331455666 learned_addr learned my addr 172.21.15.33:0/331455666 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f459b7fe640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f459c1069c0 0x7f459c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f45927fc640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c101370 0x7f459c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f459affd640 1 -- 172.21.15.33:0/331455666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f459c1069c0 msgr2=0x7f459c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f459affd640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f459c1069c0 0x7f459c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.447+0000 7f459affd640 1 -- 172.21.15.33:0/331455666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c101370 msgr2=0x7f459c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.448+0000 7f459affd640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c101370 0x7f459c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.448+0000 7f459affd640 1 -- 172.21.15.33:0/331455666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c19d350 con 0x7f459c1009b0 2024-04-02T13:48:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.448+0000 7f45927fc640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c101370 0x7f459c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.448+0000 7f459affd640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f459c1009b0 0x7f459c198360 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f458c00d770 tx=0x7f458c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.448+0000 7f4598ff9640 1 -- 172.21.15.33:0/331455666 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c017070 con 0x7f459c1009b0 2024-04-02T13:48:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.448+0000 7f4598ff9640 1 -- 172.21.15.33:0/331455666 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f458c0054f0 con 0x7f459c1009b0 2024-04-02T13:48:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.448+0000 7f45a152b640 1 -- 172.21.15.33:0/331455666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459c19d640 con 0x7f459c1009b0 2024-04-02T13:48:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.448+0000 7f4598ff9640 1 -- 172.21.15.33:0/331455666 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c004d60 con 0x7f459c1009b0 2024-04-02T13:48:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.448+0000 7f45a152b640 1 -- 172.21.15.33:0/331455666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f459c1a4ed0 con 0x7f459c1009b0 2024-04-02T13:48:37.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.450+0000 7f4598ff9640 1 -- 172.21.15.33:0/331455666 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f458c004f00 con 0x7f459c1009b0 2024-04-02T13:48:37.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.450+0000 7f45a41ef740 1 Processor -- start 2024-04-02T13:48:37.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.450+0000 7f45a41ef740 1 -- start start 2024-04-02T13:48:37.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f45a41ef740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3600 0x562446df39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f45a41ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3f20 0x562446e00ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f4598ff9640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f45880a3d20 0x7f45880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f45a41ef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446e035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f45a41ef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562446d57830 con 0x562446e011e0 2024-04-02T13:48:37.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f45a41ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562446cff820 con 0x562446df3f20 2024-04-02T13:48:37.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f45a41ef740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562446d537e0 con 0x562446df3600 2024-04-02T13:48:37.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f45927fc640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f45880a3d20 0x7f45880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f459b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446e035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f459b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446e035d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53420/0 (socket says 172.21.15.33:53420) 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f459b7fe640 1 -- 172.21.15.33:0/1529572835 learned_addr learned my addr 172.21.15.33:0/1529572835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f4598ff9640 1 -- 172.21.15.33:0/331455666 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f458c0d0670 con 0x7f459c1009b0 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f459b7fe640 1 --2- 172.21.15.33:0/1529572835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3600 0x562446df39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f459affd640 1 --2- 172.21.15.33:0/1529572835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3f20 0x562446e00ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f459b7fe640 1 -- 172.21.15.33:0/1529572835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3600 msgr2=0x562446df39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f459b7fe640 1 --2- 172.21.15.33:0/1529572835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3600 0x562446df39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f459b7fe640 1 -- 172.21.15.33:0/1529572835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3f20 msgr2=0x562446e00ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f459b7fe640 1 --2- 172.21.15.33:0/1529572835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3f20 0x562446e00ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f459b7fe640 1 -- 172.21.15.33:0/1529572835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562446b357d0 con 0x562446e011e0 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f459affd640 1 --2- 172.21.15.33:0/1529572835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3f20 0x562446e00ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f45927fc640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f45880a3d20 0x7f45880a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4584005d00 tx=0x7f4584009550 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.451+0000 7f459b7fe640 1 --2- 172.21.15.33:0/1529572835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446e035d0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4594004af0 tx=0x7f4594004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c0bc8d183559d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.452+0000 7f45917fa640 1 -- 172.21.15.33:0/1529572835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594049070 con 0x562446e011e0 2024-04-02T13:48:37.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.452+0000 7f45917fa640 1 -- 172.21.15.33:0/1529572835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4594003e40 con 0x562446e011e0 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.452+0000 7f45917fa640 1 -- 172.21.15.33:0/1529572835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594039070 con 0x562446e011e0 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.452+0000 7f45a41ef740 1 -- 172.21.15.33:0/1529572835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 msgr2=0x562446e035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.452+0000 7f45a41ef740 1 --2- 172.21.15.33:0/1529572835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446e035d0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4594004af0 tx=0x7f4594004a60 comp rx=0 tx=0).stop 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.452+0000 7f45a41ef740 1 -- 172.21.15.33:0/1529572835 shutdown_connections 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.452+0000 7f45a41ef740 1 --2- 172.21.15.33:0/1529572835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446e035d0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.452+0000 7f45a41ef740 1 --2- 172.21.15.33:0/1529572835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3f20 0x562446e00ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.452+0000 7f45a41ef740 1 --2- 172.21.15.33:0/1529572835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3600 0x562446df39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.452+0000 7f45a41ef740 1 -- 172.21.15.33:0/1529572835 >> 172.21.15.33:0/1529572835 conn(0x562446dee000 msgr2=0x562446dee3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.452+0000 7f45a41ef740 1 -- 172.21.15.33:0/1529572835 shutdown_connections 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.452+0000 7f45a41ef740 1 -- 172.21.15.33:0/1529572835 wait complete. 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f45a41ef740 1 -- start start 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f45a41ef740 1 Processor -- start 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f45a41ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3600 0x562446e00b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f45a41ef740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3f20 0x562446dfdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f45a41ef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446dfe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f45a41ef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562446e03f40 con 0x562446e011e0 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f45a41ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562446d12d30 con 0x562446df3600 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f45a41ef740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562446e040c0 con 0x562446df3f20 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f45a41ef740 10 client.0 _ll_register_callbacks cb 0x562446d12720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f459b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446dfe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f459b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446dfe3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53432/0 (socket says 172.21.15.33:53432) 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f459b7fe640 1 -- 172.21.15.33:0/3219694012 learned_addr learned my addr 172.21.15.33:0/3219694012 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f459affd640 1 --2- 172.21.15.33:0/3219694012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3600 0x562446e00b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f459b7fe640 1 -- 172.21.15.33:0/3219694012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3f20 msgr2=0x562446dfdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f459b7fe640 1 --2- 172.21.15.33:0/3219694012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3f20 0x562446dfdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f459b7fe640 1 -- 172.21.15.33:0/3219694012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3600 msgr2=0x562446e00b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f459b7fe640 1 --2- 172.21.15.33:0/3219694012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3600 0x562446e00b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f459b7fe640 1 -- 172.21.15.33:0/3219694012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562446a6b3f0 con 0x562446e011e0 2024-04-02T13:48:37.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.453+0000 7f459affd640 1 --2- 172.21.15.33:0/3219694012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3600 0x562446e00b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f459b7fe640 1 --2- 172.21.15.33:0/3219694012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446dfe3e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f459402c130 tx=0x7f4594003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bf904f483ad6f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45827fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45827fc640 1 -- 172.21.15.33:0/3219694012 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594049070 con 0x562446e011e0 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45827fc640 1 -- 172.21.15.33:0/3219694012 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f4594004210 con 0x562446e011e0 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45a41ef740 1 -- 172.21.15.33:0/3219694012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562446d64d70 con 0x562446e011e0 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45827fc640 1 -- 172.21.15.33:0/3219694012 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4594039040 con 0x562446e011e0 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45a41ef740 10 client.17775 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45a41ef740 1 -- 172.21.15.33:0/3219694012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562446a2b0f0 con 0x562446e011e0 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45827fc640 1 -- 172.21.15.33:0/3219694012 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x562446d64d70 con 0x562446e011e0 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45827fc640 1 -- 172.21.15.33:0/3219694012 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f4594034770 con 0x562446e011e0 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45a41ef740 10 client.17775 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45a41ef740 10 client.17775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45a41ef740 20 client.17775 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45a41ef740 1 --2- 172.21.15.33:0/3219694012 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562446e10fc0 0x562446e133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45a41ef740 4 client.17775 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:37.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.454+0000 7f45a41ef740 1 -- 172.21.15.33:0/3219694012 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562446d5f0c0 con 0x562446e10fc0 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.455+0000 7f459affd640 1 --2- 172.21.15.33:0/3219694012 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562446e10fc0 0x562446e133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.455+0000 7f45827fc640 10 client.17775 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.455+0000 7f459affd640 1 --2- 172.21.15.33:0/3219694012 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562446e10fc0 0x562446e133b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbf2f8b429841640 server_cookie=f13a47438d53ed96 in_seq=0 out_seq=0 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45827fc640 1 -- 172.21.15.33:0/3219694012 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562446d5f0c0 con 0x562446e10fc0 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45827fc640 10 client.17775 handle_command_reply: tid=0 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45a41ef740 1 client.17775 shutdown 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45a41ef740 1 -- 172.21.15.33:0/3219694012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 msgr2=0x562446dfe3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45a41ef740 1 --2- 172.21.15.33:0/3219694012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446dfe3e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f459402c130 tx=0x7f4594003c20 comp rx=0 tx=0).stop 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45a41ef740 1 -- 172.21.15.33:0/3219694012 shutdown_connections 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45a41ef740 1 --2- 172.21.15.33:0/3219694012 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562446e10fc0 0x562446e133b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45a41ef740 1 --2- 172.21.15.33:0/3219694012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446dfe3e0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45827fc640 0 client.17775 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45a41ef740 1 --2- 172.21.15.33:0/3219694012 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3f20 0x562446dfdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45a41ef740 1 --2- 172.21.15.33:0/3219694012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3600 0x562446e00b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45a41ef740 1 -- 172.21.15.33:0/3219694012 >> 172.21.15.33:0/3219694012 conn(0x562446dee000 msgr2=0x562446defd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:37.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45a41ef740 1 -- 172.21.15.33:0/3219694012 shutdown_connections 2024-04-02T13:48:37.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45a41ef740 1 -- 172.21.15.33:0/3219694012 wait complete. 2024-04-02T13:48:37.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.456+0000 7f45a41ef740 20 client.17775 trim_cache size 0 max 16384 2024-04-02T13:48:37.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.473+0000 7f45a41ef740 1 Processor -- start 2024-04-02T13:48:37.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.473+0000 7f45a41ef740 1 -- start start 2024-04-02T13:48:37.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f45a41ef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446df3600 0x562446e0ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f45a41ef740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3f20 0x562446dfcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f45a41ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446e011e0 0x562446e005d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f45a41ef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f459c1af7c0 con 0x562446df3600 2024-04-02T13:48:37.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f45a41ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f459c1ad090 con 0x562446e011e0 2024-04-02T13:48:37.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f45a41ef740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f459c1aaf00 con 0x562446df3f20 2024-04-02T13:48:37.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f459affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446df3600 0x562446e0ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f459affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446df3600 0x562446e0ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53446/0 (socket says 172.21.15.33:53446) 2024-04-02T13:48:37.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f459affd640 1 -- 172.21.15.33:0/4236124847 learned_addr learned my addr 172.21.15.33:0/4236124847 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:37.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f45927fc640 1 --2- 172.21.15.33:0/4236124847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3f20 0x562446dfcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f459affd640 1 -- 172.21.15.33:0/4236124847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3f20 msgr2=0x562446dfcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f459affd640 1 --2- 172.21.15.33:0/4236124847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3f20 0x562446dfcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f459affd640 1 -- 172.21.15.33:0/4236124847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446e011e0 msgr2=0x562446e005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f459affd640 1 --2- 172.21.15.33:0/4236124847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446e011e0 0x562446e005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f459affd640 1 -- 172.21.15.33:0/4236124847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562446dede30 con 0x562446df3600 2024-04-02T13:48:37.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f45927fc640 1 --2- 172.21.15.33:0/4236124847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3f20 0x562446dfcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:37.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f459affd640 1 --2- 172.21.15.33:0/4236124847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446df3600 0x562446e0ce40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f458c0986a0 tx=0x7f458c00de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12759b9384d5390 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:37.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f4590ff9640 1 -- 172.21.15.33:0/4236124847 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c020040 con 0x562446df3600 2024-04-02T13:48:37.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f4590ff9640 1 -- 172.21.15.33:0/4236124847 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f458c0a05f0 con 0x562446df3600 2024-04-02T13:48:37.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.474+0000 7f4590ff9640 1 -- 172.21.15.33:0/4236124847 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c0cdc70 con 0x562446df3600 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.475+0000 7f45a41ef740 1 -- 172.21.15.33:0/4236124847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446df3600 msgr2=0x562446e0ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.475+0000 7f45a41ef740 1 --2- 172.21.15.33:0/4236124847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446df3600 0x562446e0ce40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f458c0986a0 tx=0x7f458c00de80 comp rx=0 tx=0).stop 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.475+0000 7f45a41ef740 1 -- 172.21.15.33:0/4236124847 shutdown_connections 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.475+0000 7f45a41ef740 1 --2- 172.21.15.33:0/4236124847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446e011e0 0x562446e005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.475+0000 7f45a41ef740 1 --2- 172.21.15.33:0/4236124847 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3f20 0x562446dfcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.475+0000 7f45a41ef740 1 --2- 172.21.15.33:0/4236124847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446df3600 0x562446e0ce40 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.475+0000 7f45a41ef740 1 -- 172.21.15.33:0/4236124847 >> 172.21.15.33:0/4236124847 conn(0x562446dee000 msgr2=0x562446df94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.475+0000 7f45a41ef740 1 -- 172.21.15.33:0/4236124847 shutdown_connections 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.475+0000 7f45a41ef740 1 -- 172.21.15.33:0/4236124847 wait complete. 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.475+0000 7f45a41ef740 1 -- start start 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f45a41ef740 1 Processor -- start 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f45a41ef740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3600 0x562446e09450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f45a41ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3f20 0x562446e04930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f45a41ef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446e04e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f45a41ef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f459c1ab550 con 0x562446e011e0 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f45a41ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f459c1ac9c0 con 0x562446df3f20 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f45a41ef740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f459c1abc20 con 0x562446df3600 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f45a41ef740 10 client.0 _ll_register_callbacks cb 0x7f4594032f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f459b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446e04e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f459b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446e04e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53458/0 (socket says 172.21.15.33:53458) 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f459b7fe640 1 -- 172.21.15.33:0/1736362867 learned_addr learned my addr 172.21.15.33:0/1736362867 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f45927fc640 1 --2- 172.21.15.33:0/1736362867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3f20 0x562446e04930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f459affd640 1 --2- 172.21.15.33:0/1736362867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3600 0x562446e09450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f459b7fe640 1 -- 172.21.15.33:0/1736362867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3600 msgr2=0x562446e09450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f459b7fe640 1 --2- 172.21.15.33:0/1736362867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3600 0x562446e09450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f459b7fe640 1 -- 172.21.15.33:0/1736362867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3f20 msgr2=0x562446e04930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f459b7fe640 1 --2- 172.21.15.33:0/1736362867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3f20 0x562446e04930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f459b7fe640 1 -- 172.21.15.33:0/1736362867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562446e035d0 con 0x562446e011e0 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f459b7fe640 1 --2- 172.21.15.33:0/1736362867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446e04e70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4594034cb0 tx=0x7f4594003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ebffd92e25a228 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f45827fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f45827fc640 1 -- 172.21.15.33:0/1736362867 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594049070 con 0x562446e011e0 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f45827fc640 1 -- 172.21.15.33:0/1736362867 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f4594004d50 con 0x562446e011e0 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.476+0000 7f45a41ef740 1 -- 172.21.15.33:0/1736362867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f458c0a05f0 con 0x562446e011e0 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.477+0000 7f45827fc640 1 -- 172.21.15.33:0/1736362867 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4594064050 con 0x562446e011e0 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.477+0000 7f45a41ef740 10 client.17781 fetch_fsmap learned FSMap version 36 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.477+0000 7f45a41ef740 1 -- 172.21.15.33:0/1736362867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562446e76ce0 con 0x562446e011e0 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.477+0000 7f45827fc640 1 -- 172.21.15.33:0/1736362867 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f458c0a05f0 con 0x562446e011e0 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.477+0000 7f45827fc640 1 -- 172.21.15.33:0/1736362867 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f4594004560 con 0x562446e011e0 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.477+0000 7f45a41ef740 10 client.17781 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.477+0000 7f45a41ef740 10 client.17781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.477+0000 7f45a41ef740 20 client.17781 populate_metadata read hostname 'smithi033' 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.477+0000 7f45a41ef740 1 --2- 172.21.15.33:0/1736362867 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562446e77430 0x562446e79850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:37.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.477+0000 7f45a41ef740 4 client.17781 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.477+0000 7f45a41ef740 1 -- 172.21.15.33:0/1736362867 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562446cdc560 con 0x562446e77430 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.477+0000 7f459affd640 1 --2- 172.21.15.33:0/1736362867 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562446e77430 0x562446e79850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f459affd640 1 --2- 172.21.15.33:0/1736362867 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562446e77430 0x562446e79850 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf52a10e9683d9d server_cookie=6dc05567254e1d98 in_seq=0 out_seq=0 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45827fc640 10 client.17781 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45827fc640 1 -- 172.21.15.33:0/1736362867 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562446cdc560 con 0x562446e77430 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45827fc640 10 client.17781 handle_command_reply: tid=0 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45a41ef740 1 client.17781 shutdown 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45a41ef740 1 -- 172.21.15.33:0/1736362867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 msgr2=0x562446e04e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45a41ef740 1 --2- 172.21.15.33:0/1736362867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446e04e70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4594034cb0 tx=0x7f4594003db0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45a41ef740 1 -- 172.21.15.33:0/1736362867 shutdown_connections 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45a41ef740 1 --2- 172.21.15.33:0/1736362867 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x562446e77430 0x562446e79850 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45a41ef740 1 --2- 172.21.15.33:0/1736362867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562446e011e0 0x562446e04e70 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45a41ef740 1 --2- 172.21.15.33:0/1736362867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562446df3f20 0x562446e04930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45827fc640 0 client.17781 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:48:37.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45a41ef740 1 --2- 172.21.15.33:0/1736362867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562446df3600 0x562446e09450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45a41ef740 1 -- 172.21.15.33:0/1736362867 >> 172.21.15.33:0/1736362867 conn(0x562446dee000 msgr2=0x562446df94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:37.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45a41ef740 1 -- 172.21.15.33:0/1736362867 shutdown_connections 2024-04-02T13:48:37.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45a41ef740 1 -- 172.21.15.33:0/1736362867 wait complete. 2024-04-02T13:48:37.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.478+0000 7f45a41ef740 20 client.17781 trim_cache size 0 max 16384 2024-04-02T13:48:37.488 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:48:37.488 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:48:37.488 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-04-02T13:48:37.488 INFO:teuthology.orchestra.run.smithi033.stdout: "0b2c851e-a0a3-46cc-a21f-f53d5dea5dc8": { 2024-04-02T13:48:37.488 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-04-02T13:48:37.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "0b2c851e-a0a3-46cc-a21f-f53d5dea5dc8", 2024-04-02T13:48:37.489 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-04-02T13:48:37.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:48:37.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:48:37.489 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:48:37.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.479+0000 7f45a152b640 1 -- 172.21.15.33:0/331455666 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f45880a3d20 msgr2=0x7f45880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.479+0000 7f45a152b640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f45880a3d20 0x7f45880a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4584005d00 tx=0x7f4584009550 comp rx=0 tx=0).stop 2024-04-02T13:48:37.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.479+0000 7f45a152b640 1 -- 172.21.15.33:0/331455666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f459c1009b0 msgr2=0x7f459c198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:37.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.479+0000 7f45a152b640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f459c1009b0 0x7f459c198360 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f458c00d770 tx=0x7f458c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:48:37.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.479+0000 7f45a152b640 1 -- 172.21.15.33:0/331455666 shutdown_connections 2024-04-02T13:48:37.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.479+0000 7f45a152b640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f45880a3d20 0x7f45880a61e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.479+0000 7f45a152b640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f459c1069c0 0x7f459c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.479+0000 7f45a152b640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f459c101370 0x7f459c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.479+0000 7f45a152b640 1 --2- 172.21.15.33:0/331455666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f459c1009b0 0x7f459c198360 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:37.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.479+0000 7f45a152b640 1 -- 172.21.15.33:0/331455666 >> 172.21.15.33:0/331455666 conn(0x7f459c076d80 msgr2=0x7f459c0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:37.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.479+0000 7f45a152b640 1 -- 172.21.15.33:0/331455666 shutdown_connections 2024-04-02T13:48:37.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:37.479+0000 7f45a152b640 1 -- 172.21.15.33:0/331455666 wait complete. 2024-04-02T13:48:37.492 INFO:tasks.cephfs.filesystem:scrub status for tag:0b2c851e-a0a3-46cc-a21f-f53d5dea5dc8 - {'path': '/', 'tag': '0b2c851e-a0a3-46cc-a21f-f53d5dea5dc8', 'options': 'recursive,force'} 2024-04-02T13:48:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:38 smithi145 ceph-mon[35646]: pgmap v306: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:38 smithi145 ceph-mon[35646]: scrub queued for path: / 2024-04-02T13:48:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:38 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:38 smithi145 ceph-mon[35646]: scrub summary: active paths [/] 2024-04-02T13:48:38.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:38 smithi145 ceph-mon[35646]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:38 smithi050 ceph-mon[35524]: pgmap v306: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:38 smithi050 ceph-mon[35524]: scrub queued for path: / 2024-04-02T13:48:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:38 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:38 smithi050 ceph-mon[35524]: scrub summary: active paths [/] 2024-04-02T13:48:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:38 smithi050 ceph-mon[35524]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:39.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:38 smithi033 ceph-mon[32390]: pgmap v306: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:38 smithi033 ceph-mon[32390]: scrub queued for path: / 2024-04-02T13:48:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:38 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:38 smithi033 ceph-mon[32390]: scrub summary: active paths [/] 2024-04-02T13:48:39.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:38 smithi033 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:40.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:40 smithi145 ceph-mon[35646]: pgmap v307: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:48:40.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:40 smithi050 ceph-mon[35524]: pgmap v307: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:48:41.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:40 smithi033 ceph-mon[32390]: pgmap v307: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:48:41.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:41 smithi145 ceph-mon[35646]: scrub summary: idle 2024-04-02T13:48:41.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:41 smithi050 ceph-mon[35524]: scrub summary: idle 2024-04-02T13:48:42.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:41 smithi033 ceph-mon[32390]: scrub summary: idle 2024-04-02T13:48:42.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:42 smithi145 ceph-mon[35646]: pgmap v308: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:48:42.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:42 smithi050 ceph-mon[35524]: pgmap v308: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:48:43.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:42 smithi033 ceph-mon[32390]: pgmap v308: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:48:44.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:44 smithi145 ceph-mon[35646]: pgmap v309: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:48:44.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:44 smithi050 ceph-mon[35524]: pgmap v309: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:48:45.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:44 smithi033 ceph-mon[32390]: pgmap v309: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:48:46.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:46 smithi145 ceph-mon[35646]: pgmap v310: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:48:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:46 smithi050 ceph-mon[35524]: pgmap v310: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:48:47.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:46 smithi033 ceph-mon[32390]: pgmap v310: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:48:48.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:48 smithi145 ceph-mon[35646]: pgmap v311: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:48:48.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:48 smithi050 ceph-mon[35524]: pgmap v311: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:48:49.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:48 smithi033 ceph-mon[32390]: pgmap v311: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-02T13:48:50.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:50 smithi145 ceph-mon[35646]: pgmap v312: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:48:50.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:50 smithi050 ceph-mon[35524]: pgmap v312: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:48:51.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:50 smithi033 ceph-mon[32390]: pgmap v312: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-02T13:48:52.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:52 smithi145 ceph-mon[35646]: pgmap v313: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:52.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:52 smithi050 ceph-mon[35524]: pgmap v313: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:53.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:52 smithi033 ceph-mon[32390]: pgmap v313: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:48:54.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:54 smithi145 ceph-mon[35646]: pgmap v314: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:48:54.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:54 smithi050 ceph-mon[35524]: pgmap v314: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:48:55.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:54 smithi033 ceph-mon[32390]: pgmap v314: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:48:56.351 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-02T13:48:56.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.463+0000 7fb5cc488640 1 -- 172.21.15.33:0/40361707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4100c50 msgr2=0x7fb5c41010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:56.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.463+0000 7fb5cc488640 1 --2- 172.21.15.33:0/40361707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4100c50 0x7fb5c41010d0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009900 tx=0x7fb5b402bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:56.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.463+0000 7fb5cc488640 1 -- 172.21.15.33:0/40361707 shutdown_connections 2024-04-02T13:48:56.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.463+0000 7fb5cc488640 1 --2- 172.21.15.33:0/40361707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5c4101610 0x7fb5c410d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.463+0000 7fb5cc488640 1 --2- 172.21.15.33:0/40361707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4100c50 0x7fb5c41010d0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.463+0000 7fb5cc488640 1 --2- 172.21.15.33:0/40361707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5c41059a0 0x7fb5c4105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.463+0000 7fb5cc488640 1 -- 172.21.15.33:0/40361707 >> 172.21.15.33:0/40361707 conn(0x7fb5c4077850 msgr2=0x7fb5c4077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:56.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.463+0000 7fb5cc488640 1 -- 172.21.15.33:0/40361707 shutdown_connections 2024-04-02T13:48:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.463+0000 7fb5cc488640 1 -- 172.21.15.33:0/40361707 wait complete. 2024-04-02T13:48:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.463+0000 7fb5cc488640 1 Processor -- start 2024-04-02T13:48:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.463+0000 7fb5cc488640 1 -- start start 2024-04-02T13:48:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5cc488640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4100c50 0x7fb5c4196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5cc488640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5c4101610 0x7fb5c4197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5cc488640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5c41059a0 0x7fb5c4197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5cc488640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4110e90 con 0x7fb5c4100c50 2024-04-02T13:48:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5cc488640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4110d10 con 0x7fb5c41059a0 2024-04-02T13:48:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5cc488640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4111010 con 0x7fb5c4101610 2024-04-02T13:48:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5ca224640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4100c50 0x7fb5c4196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5ca224640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4100c50 0x7fb5c4196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41818/0 (socket says 172.21.15.33:41818) 2024-04-02T13:48:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5ca224640 1 -- 172.21.15.33:0/2597715046 learned_addr learned my addr 172.21.15.33:0/2597715046 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5c9a23640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5c4101610 0x7fb5c4197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5caa25640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5c41059a0 0x7fb5c4197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5ca224640 1 -- 172.21.15.33:0/2597715046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5c4101610 msgr2=0x7fb5c4197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5ca224640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5c4101610 0x7fb5c4197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5ca224640 1 -- 172.21.15.33:0/2597715046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5c41059a0 msgr2=0x7fb5c4197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5ca224640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5c41059a0 0x7fb5c4197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5ca224640 1 -- 172.21.15.33:0/2597715046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c41a6830 con 0x7fb5c4100c50 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5c9a23640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5c4101610 0x7fb5c4197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5caa25640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5c41059a0 0x7fb5c4197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5ca224640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4100c50 0x7fb5c4196fc0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00d950 tx=0x7fb5bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.464+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/2597715046 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc017030 con 0x7fb5c4100c50 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.465+0000 7fb5cc488640 1 -- 172.21.15.33:0/2597715046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c41a6b20 con 0x7fb5c4100c50 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.465+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/2597715046 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fb5bc004020 con 0x7fb5c4100c50 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.465+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/2597715046 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0047a0 con 0x7fb5c4100c50 2024-04-02T13:48:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.465+0000 7fb5cc488640 1 -- 172.21.15.33:0/2597715046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c41a7070 con 0x7fb5c4100c50 2024-04-02T13:48:56.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.466+0000 7fb5cc488640 1 -- 172.21.15.33:0/2597715046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c404f860 con 0x7fb5c4100c50 2024-04-02T13:48:56.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.467+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/2597715046 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb5bc0041d0 con 0x7fb5c4100c50 2024-04-02T13:48:56.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.467+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb5b00a3dc0 0x7fb5b00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:56.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.467+0000 7fb5c9a23640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb5b00a3dc0 0x7fb5b00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:56.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.467+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/2597715046 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fb5bc0d06e0 con 0x7fb5c4100c50 2024-04-02T13:48:56.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.468+0000 7fb5c9a23640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb5b00a3dc0 0x7fb5b00a6280 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4036040 tx=0x7fb5b4005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:56.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.472+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/2597715046 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb5bc09c9b0 con 0x7fb5c4100c50 2024-04-02T13:48:56.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.681+0000 7fb5cc488640 1 -- 172.21.15.33:0/2597715046 --> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb5c4061f40 con 0x7fb5b00a3dc0 2024-04-02T13:48:56.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.689+0000 7fb5bb7fe640 1 -- 172.21.15.33:0/2597715046 <== mgr.15456 v2:172.21.15.33:6800/619073725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fb5c4061f40 con 0x7fb5b00a3dc0 2024-04-02T13:48:56.693 INFO:teuthology.orchestra.run.smithi033.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-02T00:00:00", "created": "2024-04-02T13:38:55", "first": "2024-04-02T13:39:00", "last": "2024-04-02T13:48:00", "last_pruned": "2024-04-02T13:48:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-02T13:48:56.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.691+0000 7fb5cc488640 1 -- 172.21.15.33:0/2597715046 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb5b00a3dc0 msgr2=0x7fb5b00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:56.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.691+0000 7fb5cc488640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb5b00a3dc0 0x7fb5b00a6280 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4036040 tx=0x7fb5b4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.691+0000 7fb5cc488640 1 -- 172.21.15.33:0/2597715046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4100c50 msgr2=0x7fb5c4196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:56.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.691+0000 7fb5cc488640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4100c50 0x7fb5c4196fc0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00d950 tx=0x7fb5bc00de10 comp rx=0 tx=0).stop 2024-04-02T13:48:56.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.691+0000 7fb5cc488640 1 -- 172.21.15.33:0/2597715046 shutdown_connections 2024-04-02T13:48:56.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.691+0000 7fb5cc488640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fb5b00a3dc0 0x7fb5b00a6280 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.691+0000 7fb5cc488640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5c41059a0 0x7fb5c4197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.691+0000 7fb5cc488640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5c4101610 0x7fb5c4197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.691+0000 7fb5cc488640 1 --2- 172.21.15.33:0/2597715046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5c4100c50 0x7fb5c4196fc0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.691+0000 7fb5cc488640 1 -- 172.21.15.33:0/2597715046 >> 172.21.15.33:0/2597715046 conn(0x7fb5c4077850 msgr2=0x7fb5c40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:56.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.691+0000 7fb5cc488640 1 -- 172.21.15.33:0/2597715046 shutdown_connections 2024-04-02T13:48:56.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.691+0000 7fb5cc488640 1 -- 172.21.15.33:0/2597715046 wait complete. 2024-04-02T13:48:56.706 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-02T13:48:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.807+0000 7f7b60960640 1 -- 172.21.15.33:0/2717271131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b5c10c170 msgr2=0x7f7b5c10e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.807+0000 7f7b60960640 1 --2- 172.21.15.33:0/2717271131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b5c10c170 0x7f7b5c10e5c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009ca0 tx=0x7f7b5402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.807+0000 7f7b60960640 1 -- 172.21.15.33:0/2717271131 shutdown_connections 2024-04-02T13:48:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.807+0000 7f7b60960640 1 --2- 172.21.15.33:0/2717271131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b5c10c170 0x7f7b5c10e5c0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.807+0000 7f7b60960640 1 --2- 172.21.15.33:0/2717271131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b5c103620 0x7f7b5c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.807+0000 7f7b60960640 1 --2- 172.21.15.33:0/2717271131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b5c102d00 0x7f7b5c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.807+0000 7f7b60960640 1 -- 172.21.15.33:0/2717271131 >> 172.21.15.33:0/2717271131 conn(0x7f7b5c077b30 msgr2=0x7f7b5c077f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.807+0000 7f7b60960640 1 -- 172.21.15.33:0/2717271131 shutdown_connections 2024-04-02T13:48:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.807+0000 7f7b60960640 1 -- 172.21.15.33:0/2717271131 wait complete. 2024-04-02T13:48:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b60960640 1 Processor -- start 2024-04-02T13:48:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b60960640 1 -- start start 2024-04-02T13:48:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b60960640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b5c102d00 0x7f7b5c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b60960640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b5c103620 0x7f7b5c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b60960640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b5c10c170 0x7f7b5c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b60960640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c111ee0 con 0x7f7b5c10c170 2024-04-02T13:48:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b60960640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c111d60 con 0x7f7b5c103620 2024-04-02T13:48:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b60960640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c112060 con 0x7f7b5c102d00 2024-04-02T13:48:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b5ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b5c10c170 0x7f7b5c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b5ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b5c10c170 0x7f7b5c19cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41832/0 (socket says 172.21.15.33:41832) 2024-04-02T13:48:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b5ad9d640 1 -- 172.21.15.33:0/4073530402 learned_addr learned my addr 172.21.15.33:0/4073530402 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:48:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b5a59c640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b5c102d00 0x7f7b5c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b53fff640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b5c103620 0x7f7b5c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b5ad9d640 1 -- 172.21.15.33:0/4073530402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b5c102d00 msgr2=0x7f7b5c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b5ad9d640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b5c102d00 0x7f7b5c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b5ad9d640 1 -- 172.21.15.33:0/4073530402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b5c103620 msgr2=0x7f7b5c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b5ad9d640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b5c103620 0x7f7b5c1987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.808+0000 7f7b5ad9d640 1 -- 172.21.15.33:0/4073530402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c19d240 con 0x7f7b5c10c170 2024-04-02T13:48:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.809+0000 7f7b5ad9d640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b5c10c170 0x7f7b5c19cb10 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7b54004af0 tx=0x7f7b54034d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.809+0000 7f7b537fe640 1 -- 172.21.15.33:0/4073530402 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54039050 con 0x7f7b5c10c170 2024-04-02T13:48:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.809+0000 7f7b537fe640 1 -- 172.21.15.33:0/4073530402 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f7b5403a040 con 0x7f7b5c10c170 2024-04-02T13:48:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.809+0000 7f7b60960640 1 -- 172.21.15.33:0/4073530402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5c19d4d0 con 0x7f7b5c10c170 2024-04-02T13:48:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.809+0000 7f7b537fe640 1 -- 172.21.15.33:0/4073530402 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54004670 con 0x7f7b5c10c170 2024-04-02T13:48:56.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.809+0000 7f7b60960640 1 -- 172.21.15.33:0/4073530402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b5c1a4dc0 con 0x7f7b5c10c170 2024-04-02T13:48:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.810+0000 7f7b60960640 1 -- 172.21.15.33:0/4073530402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b5c06ab80 con 0x7f7b5c10c170 2024-04-02T13:48:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.811+0000 7f7b537fe640 1 -- 172.21.15.33:0/4073530402 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7b54003970 con 0x7f7b5c10c170 2024-04-02T13:48:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.812+0000 7f7b537fe640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7b480a3dc0 0x7f7b480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.812+0000 7f7b5a59c640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7b480a3dc0 0x7f7b480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.812+0000 7f7b537fe640 1 -- 172.21.15.33:0/4073530402 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f7b540f3f20 con 0x7f7b5c10c170 2024-04-02T13:48:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.812+0000 7f7b5a59c640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7b480a3dc0 0x7f7b480a6280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c005fd0 tx=0x7f7b4c005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:56.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:56.816+0000 7f7b537fe640 1 -- 172.21.15.33:0/4073530402 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7b5403f3b0 con 0x7f7b5c10c170 2024-04-02T13:48:56.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:56 smithi145 ceph-mon[35646]: pgmap v315: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-02T13:48:56.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:56 smithi050 ceph-mon[35524]: pgmap v315: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-02T13:48:57.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:56 smithi033 ceph-mon[32390]: pgmap v315: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-02T13:48:57.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.024+0000 7f7b60960640 1 -- 172.21.15.33:0/4073530402 --> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7b5c0fdff0 con 0x7f7b480a3dc0 2024-04-02T13:48:57.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.032+0000 7f7b537fe640 1 -- 172.21.15.33:0/4073530402 <== mgr.15456 v2:172.21.15.33:6800/619073725 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7b5c0fdff0 con 0x7f7b480a3dc0 2024-04-02T13:48:57.035 INFO:teuthology.orchestra.run.smithi033.stdout:/ 1m 6m3h 2024-04-02T13:48:57.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.034+0000 7f7b60960640 1 -- 172.21.15.33:0/4073530402 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7b480a3dc0 msgr2=0x7f7b480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:57.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.034+0000 7f7b60960640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7b480a3dc0 0x7f7b480a6280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c005fd0 tx=0x7f7b4c005850 comp rx=0 tx=0).stop 2024-04-02T13:48:57.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.034+0000 7f7b60960640 1 -- 172.21.15.33:0/4073530402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b5c10c170 msgr2=0x7f7b5c19cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:57.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.034+0000 7f7b60960640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b5c10c170 0x7f7b5c19cb10 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7b54004af0 tx=0x7f7b54034d70 comp rx=0 tx=0).stop 2024-04-02T13:48:57.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.034+0000 7f7b60960640 1 -- 172.21.15.33:0/4073530402 shutdown_connections 2024-04-02T13:48:57.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.034+0000 7f7b60960640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f7b480a3dc0 0x7f7b480a6280 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:57.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.034+0000 7f7b60960640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b5c10c170 0x7f7b5c19cb10 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:57.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.034+0000 7f7b60960640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7b5c103620 0x7f7b5c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:57.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.034+0000 7f7b60960640 1 --2- 172.21.15.33:0/4073530402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b5c102d00 0x7f7b5c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:57.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.034+0000 7f7b60960640 1 -- 172.21.15.33:0/4073530402 >> 172.21.15.33:0/4073530402 conn(0x7f7b5c077b30 msgr2=0x7f7b5c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:57.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.034+0000 7f7b60960640 1 -- 172.21.15.33:0/4073530402 shutdown_connections 2024-04-02T13:48:57.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:48:57.034+0000 7f7b60960640 1 -- 172.21.15.33:0/4073530402 wait complete. 2024-04-02T13:48:57.048 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-02T13:48:57.063 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-02T13:48:57.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:57 smithi145 ceph-mon[35646]: from='client.17787 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:57.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:57 smithi050 ceph-mon[35524]: from='client.17787 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:58.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:57 smithi033 ceph-mon[32390]: from='client.17787 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:58.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:58 smithi145 ceph-mon[35646]: from='client.17793 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:58.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:58 smithi145 ceph-mon[35646]: pgmap v316: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-02T13:48:58.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:58 smithi050 ceph-mon[35524]: from='client.17793 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:58.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:58 smithi050 ceph-mon[35524]: pgmap v316: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-02T13:48:59.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:58 smithi033 ceph-mon[32390]: from='client.17793 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:59.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:58 smithi033 ceph-mon[32390]: pgmap v316: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-02T13:48:59.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:48:59 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:59.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:48:59 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:00.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:48:59 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:00.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:00 smithi145 ceph-mon[35646]: pgmap v317: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T13:49:00.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:00 smithi050 ceph-mon[35524]: pgmap v317: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T13:49:00.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:00 smithi033 ceph-mon[32390]: pgmap v317: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T13:49:02.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: pgmap v318: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-02T13:49:02.730 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.731 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:02.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:02.732 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: pgmap v318: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-02T13:49:02.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:02.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:02.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: pgmap v318: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-02T13:49:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:02.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:02.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:02.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:49:03.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:03 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi145 to 176.2M 2024-04-02T13:49:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:03 smithi145 ceph-mon[35646]: Unable to set osd_memory_target on smithi145 to 184772403: error parsing value: Value '184772403' is below minimum 939524096 2024-04-02T13:49:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:03 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-02T13:49:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:03 smithi145 ceph-mon[35646]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-04-02T13:49:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:03 smithi145 ceph-mon[35646]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-02T13:49:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:03 smithi145 ceph-mon[35646]: Unable to set osd_memory_target on smithi050 to 185898496: error parsing value: Value '185898496' is below minimum 939524096 2024-04-02T13:49:03.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:03 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi145 to 176.2M 2024-04-02T13:49:03.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:03 smithi050 ceph-mon[35524]: Unable to set osd_memory_target on smithi145 to 184772403: error parsing value: Value '184772403' is below minimum 939524096 2024-04-02T13:49:03.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:03 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-02T13:49:03.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:03 smithi050 ceph-mon[35524]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-04-02T13:49:03.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:03 smithi050 ceph-mon[35524]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-02T13:49:03.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:03 smithi050 ceph-mon[35524]: Unable to set osd_memory_target on smithi050 to 185898496: error parsing value: Value '185898496' is below minimum 939524096 2024-04-02T13:49:03.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:03 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi145 to 176.2M 2024-04-02T13:49:03.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:03 smithi033 ceph-mon[32390]: Unable to set osd_memory_target on smithi145 to 184772403: error parsing value: Value '184772403' is below minimum 939524096 2024-04-02T13:49:03.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:03 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-02T13:49:03.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:03 smithi033 ceph-mon[32390]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-04-02T13:49:03.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:03 smithi033 ceph-mon[32390]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-02T13:49:03.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:03 smithi033 ceph-mon[32390]: Unable to set osd_memory_target on smithi050 to 185898496: error parsing value: Value '185898496' is below minimum 939524096 2024-04-02T13:49:04.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:04 smithi145 ceph-mon[35646]: pgmap v319: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-04-02T13:49:04.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:04 smithi050 ceph-mon[35524]: pgmap v319: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-04-02T13:49:04.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:04 smithi033 ceph-mon[32390]: pgmap v319: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-04-02T13:49:06.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:06 smithi145 ceph-mon[35646]: pgmap v320: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T13:49:06.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:06 smithi050 ceph-mon[35524]: pgmap v320: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T13:49:06.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:06 smithi033 ceph-mon[32390]: pgmap v320: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T13:49:07.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:49:07.493 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:49:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.609+0000 7f574877c640 1 -- 172.21.15.33:0/2510417949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5740100a20 msgr2=0x7f5740100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.609+0000 7f574877c640 1 --2- 172.21.15.33:0/2510417949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5740100a20 0x7f5740100ea0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f5730009930 tx=0x7f573002bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.609+0000 7f574877c640 1 -- 172.21.15.33:0/2510417949 shutdown_connections 2024-04-02T13:49:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.609+0000 7f574877c640 1 --2- 172.21.15.33:0/2510417949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57401013e0 0x7f574010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.609+0000 7f574877c640 1 --2- 172.21.15.33:0/2510417949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5740100a20 0x7f5740100ea0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.609+0000 7f574877c640 1 --2- 172.21.15.33:0/2510417949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5740106a30 0x7f5740106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.609+0000 7f574877c640 1 -- 172.21.15.33:0/2510417949 >> 172.21.15.33:0/2510417949 conn(0x7f5740077b60 msgr2=0x7f5740077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.609+0000 7f574877c640 1 -- 172.21.15.33:0/2510417949 shutdown_connections 2024-04-02T13:49:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.609+0000 7f574877c640 1 -- 172.21.15.33:0/2510417949 wait complete. 2024-04-02T13:49:07.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f574877c640 1 Processor -- start 2024-04-02T13:49:07.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f574877c640 1 -- start start 2024-04-02T13:49:07.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f574877c640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5740100a20 0x7f5740198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f574877c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57401013e0 0x7f5740198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f574877c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5740106a30 0x7f574019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f574877c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5740111f30 con 0x7f57401013e0 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f574877c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5740111db0 con 0x7f5740106a30 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f574877c640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f57401120b0 con 0x7f5740100a20 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f5745d17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57401013e0 0x7f5740198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f5745d17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57401013e0 0x7f5740198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49156/0 (socket says 172.21.15.33:49156) 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f5745d17640 1 -- 172.21.15.33:0/4147915197 learned_addr learned my addr 172.21.15.33:0/4147915197 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f5746518640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5740100a20 0x7f5740198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f5746d19640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5740106a30 0x7f574019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f5745d17640 1 -- 172.21.15.33:0/4147915197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5740100a20 msgr2=0x7f5740198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f5745d17640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5740100a20 0x7f5740198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f5745d17640 1 -- 172.21.15.33:0/4147915197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5740106a30 msgr2=0x7f574019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f5745d17640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5740106a30 0x7f574019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f5745d17640 1 -- 172.21.15.33:0/4147915197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574019d520 con 0x7f57401013e0 2024-04-02T13:49:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.610+0000 7f5746d19640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5740106a30 0x7f574019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:07.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.611+0000 7f5745d17640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57401013e0 0x7f5740198a70 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f5730000c00 tx=0x7f57300045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:07.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.611+0000 7f57377fe640 1 -- 172.21.15.33:0/4147915197 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730043020 con 0x7f57401013e0 2024-04-02T13:49:07.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.611+0000 7f574877c640 1 -- 172.21.15.33:0/4147915197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574019d7b0 con 0x7f57401013e0 2024-04-02T13:49:07.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.611+0000 7f57377fe640 1 -- 172.21.15.33:0/4147915197 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f573002caa0 con 0x7f57401013e0 2024-04-02T13:49:07.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.611+0000 7f57377fe640 1 -- 172.21.15.33:0/4147915197 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730003b50 con 0x7f57401013e0 2024-04-02T13:49:07.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.611+0000 7f574877c640 1 -- 172.21.15.33:0/4147915197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57401a50a0 con 0x7f57401013e0 2024-04-02T13:49:07.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.613+0000 7f57377fe640 1 -- 172.21.15.33:0/4147915197 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5730046020 con 0x7f57401013e0 2024-04-02T13:49:07.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.613+0000 7f574b436740 1 Processor -- start 2024-04-02T13:49:07.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.613+0000 7f574b436740 1 -- start start 2024-04-02T13:49:07.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f574b436740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 0x5600a1eb5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f574b436740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1ebe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f574b436740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 0x5600a1ec11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f574b436740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5600a1e19830 con 0x5600a1eb61f0 2024-04-02T13:49:07.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f574b436740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5600a1dc1820 con 0x5600a1eb58d0 2024-04-02T13:49:07.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f574b436740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5600a1e157e0 con 0x5600a1ebedd0 2024-04-02T13:49:07.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f57377fe640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f572c0a3d20 0x7f572c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f5746518640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1ebe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f5746518640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1ebe890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49172/0 (socket says 172.21.15.33:49172) 2024-04-02T13:49:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f5746518640 1 -- 172.21.15.33:0/2480711512 learned_addr learned my addr 172.21.15.33:0/2480711512 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f5745d17640 1 --2- 172.21.15.33:0/2480711512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 0x5600a1ec11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f5746d19640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f572c0a3d20 0x7f572c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f5746d19640 1 --2- 172.21.15.33:0/2480711512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 0x5600a1eb5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f57377fe640 1 -- 172.21.15.33:0/4147915197 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f57300f25f0 con 0x7f57401013e0 2024-04-02T13:49:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f5746518640 1 -- 172.21.15.33:0/2480711512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 msgr2=0x5600a1ec11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f5746518640 1 --2- 172.21.15.33:0/2480711512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 0x5600a1ec11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f5746518640 1 -- 172.21.15.33:0/2480711512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 msgr2=0x5600a1eb5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f5746518640 1 --2- 172.21.15.33:0/2480711512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 0x5600a1eb5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.614+0000 7f5746518640 1 -- 172.21.15.33:0/2480711512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a1bf77d0 con 0x5600a1eb61f0 2024-04-02T13:49:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.615+0000 7f5746518640 1 --2- 172.21.15.33:0/2480711512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1ebe890 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f573c009a70 tx=0x7f573c009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45890ce13b38ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.615+0000 7f5746d19640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f572c0a3d20 0x7f572c0a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5738009fd0 tx=0x7f5738009530 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.615+0000 7f5734ff9640 1 -- 172.21.15.33:0/2480711512 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c017070 con 0x5600a1eb61f0 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.615+0000 7f5734ff9640 1 -- 172.21.15.33:0/2480711512 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f573c013470 con 0x5600a1eb61f0 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.615+0000 7f5734ff9640 1 -- 172.21.15.33:0/2480711512 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c012610 con 0x5600a1eb61f0 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.615+0000 7f574b436740 1 -- 172.21.15.33:0/2480711512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 msgr2=0x5600a1ebe890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.615+0000 7f574b436740 1 --2- 172.21.15.33:0/2480711512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1ebe890 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f573c009a70 tx=0x7f573c009e70 comp rx=0 tx=0).stop 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.615+0000 7f574b436740 1 -- 172.21.15.33:0/2480711512 shutdown_connections 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.615+0000 7f574b436740 1 --2- 172.21.15.33:0/2480711512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 0x5600a1ec11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.615+0000 7f574b436740 1 --2- 172.21.15.33:0/2480711512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1ebe890 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.615+0000 7f574b436740 1 --2- 172.21.15.33:0/2480711512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 0x5600a1eb5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.615+0000 7f574b436740 1 -- 172.21.15.33:0/2480711512 >> 172.21.15.33:0/2480711512 conn(0x5600a1eb0000 msgr2=0x5600a1eb03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f574b436740 1 -- 172.21.15.33:0/2480711512 shutdown_connections 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f574b436740 1 -- 172.21.15.33:0/2480711512 wait complete. 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f574b436740 1 -- start start 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f574b436740 1 Processor -- start 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f574b436740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 0x5600a1ebe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f574b436740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1ebb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f574b436740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 0x5600a1ebbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f574b436740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5600a1ec1bc0 con 0x5600a1eb61f0 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f574b436740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5600a1dd4d30 con 0x5600a1eb58d0 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f574b436740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5600a1ec1d40 con 0x5600a1ebedd0 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f574b436740 10 client.0 _ll_register_callbacks cb 0x5600a1dd4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f5745d17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1ebb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f5745d17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1ebb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49186/0 (socket says 172.21.15.33:49186) 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.616+0000 7f5745d17640 1 -- 172.21.15.33:0/3788281644 learned_addr learned my addr 172.21.15.33:0/3788281644 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5746d19640 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 0x5600a1ebbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5746518640 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 0x5600a1ebe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5745d17640 1 -- 172.21.15.33:0/3788281644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 msgr2=0x5600a1ebbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5745d17640 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 0x5600a1ebbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5745d17640 1 -- 172.21.15.33:0/3788281644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 msgr2=0x5600a1ebe610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5745d17640 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 0x5600a1ebe610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5745d17640 1 -- 172.21.15.33:0/3788281644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600a1b2d3f0 con 0x5600a1eb61f0 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5746518640 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 0x5600a1ebe610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5745d17640 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1ebb9a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5730006010 tx=0x7f573008cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef592a8dd0aaa53 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5729ffb640 1 -- 172.21.15.33:0/3788281644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57300be940 con 0x5600a1eb61f0 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5729ffb640 1 -- 172.21.15.33:0/3788281644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f57300054c0 con 0x5600a1eb61f0 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f574b436740 1 -- 172.21.15.33:0/3788281644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f573c013470 con 0x5600a1eb61f0 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5729ffb640 1 -- 172.21.15.33:0/3788281644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f57300be240 con 0x5600a1eb61f0 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f574b436740 10 client.17805 fetch_fsmap learned FSMap version 36 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f574b436740 1 -- 172.21.15.33:0/3788281644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5600a1aed0f0 con 0x5600a1eb61f0 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.617+0000 7f5729ffb640 1 -- 172.21.15.33:0/3788281644 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f573c013470 con 0x5600a1eb61f0 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.618+0000 7f5729ffb640 1 -- 172.21.15.33:0/3788281644 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f57300c6a20 con 0x5600a1eb61f0 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.618+0000 7f574b436740 10 client.17805 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.618+0000 7f574b436740 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.618+0000 7f574b436740 20 client.17805 populate_metadata read hostname 'smithi033' 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.618+0000 7f574b436740 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5600a1eceba0 0x5600a1ed0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.618+0000 7f574b436740 4 client.17805 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.618+0000 7f574b436740 1 -- 172.21.15.33:0/3788281644 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600a1e26d70 con 0x5600a1eceba0 2024-04-02T13:49:07.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.618+0000 7f5746518640 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5600a1eceba0 0x5600a1ed0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.619+0000 7f5746518640 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5600a1eceba0 0x5600a1ed0ff0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a743a61d7791d5ca server_cookie=f2c385a439eb9a34 in_seq=0 out_seq=0 2024-04-02T13:49:07.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.619+0000 7f5729ffb640 10 client.17805 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:49:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.619+0000 7f5729ffb640 1 -- 172.21.15.33:0/3788281644 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600a1e26d70 con 0x5600a1eceba0 2024-04-02T13:49:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.619+0000 7f5729ffb640 10 client.17805 handle_command_reply: tid=0 2024-04-02T13:49:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.619+0000 7f574b436740 1 client.17805 shutdown 2024-04-02T13:49:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.619+0000 7f574b436740 1 -- 172.21.15.33:0/3788281644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 msgr2=0x5600a1ebb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.619+0000 7f574b436740 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1ebb9a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5730006010 tx=0x7f573008cbf0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.620+0000 7f574b436740 1 -- 172.21.15.33:0/3788281644 shutdown_connections 2024-04-02T13:49:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.620+0000 7f574b436740 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5600a1eceba0 0x5600a1ed0ff0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.620+0000 7f574b436740 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 0x5600a1ebbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.620+0000 7f5729ffb640 0 client.17805 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:49:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.620+0000 7f574b436740 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1ebb9a0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.620+0000 7f574b436740 1 --2- 172.21.15.33:0/3788281644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 0x5600a1ebe610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.620+0000 7f574b436740 1 -- 172.21.15.33:0/3788281644 >> 172.21.15.33:0/3788281644 conn(0x5600a1eb0000 msgr2=0x5600a1eb1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.620+0000 7f574b436740 1 -- 172.21.15.33:0/3788281644 shutdown_connections 2024-04-02T13:49:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.620+0000 7f574b436740 1 -- 172.21.15.33:0/3788281644 wait complete. 2024-04-02T13:49:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.620+0000 7f574b436740 20 client.17805 trim_cache size 0 max 16384 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.637+0000 7f574b436740 1 Processor -- start 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.637+0000 7f574b436740 1 -- start start 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.637+0000 7f574b436740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb58d0 0x5600a1eb70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.637+0000 7f574b436740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1eb61f0 0x5600a1eaf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.637+0000 7f574b436740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1ebedd0 0x5600a1ebe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.637+0000 7f574b436740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57401b1a40 con 0x5600a1eb58d0 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.637+0000 7f574b436740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57401af310 con 0x5600a1ebedd0 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.637+0000 7f574b436740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f57401ad160 con 0x5600a1eb61f0 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.637+0000 7f5746518640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb58d0 0x5600a1eb70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.637+0000 7f5746518640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb58d0 0x5600a1eb70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49192/0 (socket says 172.21.15.33:49192) 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.637+0000 7f5746518640 1 -- 172.21.15.33:0/327858711 learned_addr learned my addr 172.21.15.33:0/327858711 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.638+0000 7f5746518640 1 -- 172.21.15.33:0/327858711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1eb61f0 msgr2=0x5600a1eaf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.638+0000 7f5746518640 1 --2- 172.21.15.33:0/327858711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1eb61f0 0x5600a1eaf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.638+0000 7f5746518640 1 -- 172.21.15.33:0/327858711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1ebedd0 msgr2=0x5600a1ebe0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.638+0000 7f5746518640 1 --2- 172.21.15.33:0/327858711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1ebedd0 0x5600a1ebe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.638+0000 7f5746518640 1 -- 172.21.15.33:0/327858711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a1eb6020 con 0x5600a1eb58d0 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.638+0000 7f5746518640 1 --2- 172.21.15.33:0/327858711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb58d0 0x5600a1eb70c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f573c010010 tx=0x7f573c0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbb12907e95166e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.638+0000 7f572bfff640 1 -- 172.21.15.33:0/327858711 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c02b070 con 0x5600a1eb58d0 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.638+0000 7f572bfff640 1 -- 172.21.15.33:0/327858711 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f573c025840 con 0x5600a1eb58d0 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.638+0000 7f572bfff640 1 -- 172.21.15.33:0/327858711 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c017040 con 0x5600a1eb58d0 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.638+0000 7f574b436740 1 -- 172.21.15.33:0/327858711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb58d0 msgr2=0x5600a1eb70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.638+0000 7f574b436740 1 --2- 172.21.15.33:0/327858711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb58d0 0x5600a1eb70c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f573c010010 tx=0x7f573c0092a0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.638+0000 7f574b436740 1 -- 172.21.15.33:0/327858711 shutdown_connections 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 --2- 172.21.15.33:0/327858711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1ebedd0 0x5600a1ebe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 --2- 172.21.15.33:0/327858711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1eb61f0 0x5600a1eaf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 --2- 172.21.15.33:0/327858711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb58d0 0x5600a1eb70c0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 -- 172.21.15.33:0/327858711 >> 172.21.15.33:0/327858711 conn(0x5600a1eb0000 msgr2=0x5600a1eae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 -- 172.21.15.33:0/327858711 shutdown_connections 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 -- 172.21.15.33:0/327858711 wait complete. 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 -- start start 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 Processor -- start 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 0x5600a1eb8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1eb91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 0x5600a1f38510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57401ad7d0 con 0x5600a1eb61f0 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57401aec40 con 0x5600a1eb58d0 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f57401adea0 con 0x5600a1ebedd0 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f574b436740 10 client.0 _ll_register_callbacks cb 0x7f57300be010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f5745d17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1eb91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f5745d17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1eb91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49208/0 (socket says 172.21.15.33:49208) 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f5745d17640 1 -- 172.21.15.33:0/2585565941 learned_addr learned my addr 172.21.15.33:0/2585565941 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.639+0000 7f5746518640 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 0x5600a1eb8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5746d19640 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 0x5600a1f38510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5745d17640 1 -- 172.21.15.33:0/2585565941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 msgr2=0x5600a1f38510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5745d17640 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 0x5600a1f38510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5745d17640 1 -- 172.21.15.33:0/2585565941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 msgr2=0x5600a1eb8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5745d17640 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 0x5600a1eb8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5745d17640 1 -- 172.21.15.33:0/2585565941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600a1f38a50 con 0x5600a1eb61f0 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5746d19640 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 0x5600a1f38510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5745d17640 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1eb91c0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f57300bc4f0 tx=0x7f57300efbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc92258e8b7a396e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5729ffb640 1 -- 172.21.15.33:0/2585565941 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57300c2150 con 0x5600a1eb61f0 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5729ffb640 1 -- 172.21.15.33:0/2585565941 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f57300d7630 con 0x5600a1eb61f0 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f574b436740 1 -- 172.21.15.33:0/2585565941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f573c025840 con 0x5600a1eb61f0 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5729ffb640 1 -- 172.21.15.33:0/2585565941 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f57300be940 con 0x5600a1eb61f0 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f574b436740 10 client.17811 fetch_fsmap learned FSMap version 36 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f574b436740 1 -- 172.21.15.33:0/2585565941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5600a1f39c90 con 0x5600a1eb61f0 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5729ffb640 1 -- 172.21.15.33:0/2585565941 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f573c025840 con 0x5600a1eb61f0 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f5729ffb640 1 -- 172.21.15.33:0/2585565941 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f57300d2ce0 con 0x5600a1eb61f0 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f574b436740 10 client.17811 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f574b436740 10 client.17811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f574b436740 20 client.17811 populate_metadata read hostname 'smithi033' 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.640+0000 7f574b436740 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5600a1eceba0 0x5600a1f3c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.641+0000 7f574b436740 4 client.17811 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.641+0000 7f574b436740 1 -- 172.21.15.33:0/2585565941 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5600a1e210c0 con 0x5600a1eceba0 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.641+0000 7f5746518640 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5600a1eceba0 0x5600a1f3c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.641+0000 7f5746518640 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5600a1eceba0 0x5600a1f3c230 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6813d22a3a7bffd server_cookie=697f84b33fc5a933 in_seq=0 out_seq=0 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.641+0000 7f5729ffb640 10 client.17811 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.641+0000 7f5729ffb640 1 -- 172.21.15.33:0/2585565941 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5600a1e210c0 con 0x5600a1eceba0 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.641+0000 7f5729ffb640 10 client.17811 handle_command_reply: tid=0 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.641+0000 7f574b436740 1 client.17811 shutdown 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574b436740 1 -- 172.21.15.33:0/2585565941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 msgr2=0x5600a1eb91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574b436740 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1eb91c0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f57300bc4f0 tx=0x7f57300efbf0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574b436740 1 -- 172.21.15.33:0/2585565941 shutdown_connections 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574b436740 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x5600a1eceba0 0x5600a1f3c230 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574b436740 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5600a1ebedd0 0x5600a1f38510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574b436740 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600a1eb61f0 0x5600a1eb91c0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f5729ffb640 0 client.17811 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:49:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574b436740 1 --2- 172.21.15.33:0/2585565941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5600a1eb58d0 0x5600a1eb8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574b436740 1 -- 172.21.15.33:0/2585565941 >> 172.21.15.33:0/2585565941 conn(0x5600a1eb0000 msgr2=0x5600a1eaf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574b436740 1 -- 172.21.15.33:0/2585565941 shutdown_connections 2024-04-02T13:49:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574b436740 1 -- 172.21.15.33:0/2585565941 wait complete. 2024-04-02T13:49:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574b436740 20 client.17811 trim_cache size 0 max 16384 2024-04-02T13:49:07.650 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:49:07.650 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-04-02T13:49:07.650 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-04-02T13:49:07.650 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:49:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574877c640 1 -- 172.21.15.33:0/4147915197 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f572c0a3d20 msgr2=0x7f572c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574877c640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f572c0a3d20 0x7f572c0a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5738009fd0 tx=0x7f5738009530 comp rx=0 tx=0).stop 2024-04-02T13:49:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574877c640 1 -- 172.21.15.33:0/4147915197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57401013e0 msgr2=0x7f5740198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574877c640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57401013e0 0x7f5740198a70 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f5730000c00 tx=0x7f57300045e0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574877c640 1 -- 172.21.15.33:0/4147915197 shutdown_connections 2024-04-02T13:49:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574877c640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f572c0a3d20 0x7f572c0a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574877c640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5740106a30 0x7f574019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574877c640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57401013e0 0x7f5740198a70 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574877c640 1 --2- 172.21.15.33:0/4147915197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5740100a20 0x7f5740198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574877c640 1 -- 172.21.15.33:0/4147915197 >> 172.21.15.33:0/4147915197 conn(0x7f5740077b60 msgr2=0x7f57401036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:07.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574877c640 1 -- 172.21.15.33:0/4147915197 shutdown_connections 2024-04-02T13:49:07.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.642+0000 7f574877c640 1 -- 172.21.15.33:0/4147915197 wait complete. 2024-04-02T13:49:07.655 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:07 smithi033 ceph-mon[32390]: osdmap e104: 12 total, 12 up, 12 in 2024-04-02T13:49:07.656 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:49:07.656 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:49:07.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:07 smithi145 ceph-mon[35646]: osdmap e104: 12 total, 12 up, 12 in 2024-04-02T13:49:07.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:07 smithi050 ceph-mon[35524]: osdmap e104: 12 total, 12 up, 12 in 2024-04-02T13:49:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.756+0000 7f196d78d640 1 -- 172.21.15.33:0/2869402759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968102460 msgr2=0x7f196810e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.756+0000 7f196d78d640 1 --2- 172.21.15.33:0/2869402759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968102460 0x7f196810e970 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f1958009cd0 tx=0x7f195802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.756+0000 7f196d78d640 1 -- 172.21.15.33:0/2869402759 shutdown_connections 2024-04-02T13:49:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.756+0000 7f196d78d640 1 --2- 172.21.15.33:0/2869402759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968102460 0x7f196810e970 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.756+0000 7f196d78d640 1 --2- 172.21.15.33:0/2869402759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1968101aa0 0x7f1968101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.756+0000 7f196d78d640 1 --2- 172.21.15.33:0/2869402759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968107ab0 0x7f1968107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.756+0000 7f196d78d640 1 -- 172.21.15.33:0/2869402759 >> 172.21.15.33:0/2869402759 conn(0x7f19680fd590 msgr2=0x7f19680ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.756+0000 7f196d78d640 1 -- 172.21.15.33:0/2869402759 shutdown_connections 2024-04-02T13:49:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.756+0000 7f196d78d640 1 -- 172.21.15.33:0/2869402759 wait complete. 2024-04-02T13:49:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f196d78d640 1 Processor -- start 2024-04-02T13:49:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f196d78d640 1 -- start start 2024-04-02T13:49:07.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f196d78d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968101aa0 0x7f1968198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f196d78d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1968102460 0x7f1968198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f196d78d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968107ab0 0x7f196819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f196d78d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1968111e60 con 0x7f1968107ab0 2024-04-02T13:49:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f196d78d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1968111ce0 con 0x7f1968102460 2024-04-02T13:49:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f196d78d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1968111fe0 con 0x7f1968101aa0 2024-04-02T13:49:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f19677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968107ab0 0x7f196819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f19677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968107ab0 0x7f196819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49230/0 (socket says 172.21.15.33:49230) 2024-04-02T13:49:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f19677fe640 1 -- 172.21.15.33:0/3576546103 learned_addr learned my addr 172.21.15.33:0/3576546103 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f1966ffd640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968101aa0 0x7f1968198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f19667fc640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1968102460 0x7f1968198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f19677fe640 1 -- 172.21.15.33:0/3576546103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968101aa0 msgr2=0x7f1968198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f19677fe640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968101aa0 0x7f1968198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f19677fe640 1 -- 172.21.15.33:0/3576546103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1968102460 msgr2=0x7f1968198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.757+0000 7f19677fe640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1968102460 0x7f1968198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.758+0000 7f19677fe640 1 -- 172.21.15.33:0/3576546103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196819d2f0 con 0x7f1968107ab0 2024-04-02T13:49:07.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.758+0000 7f1966ffd640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968101aa0 0x7f1968198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:07.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.758+0000 7f19677fe640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968107ab0 0x7f196819cbc0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f1958004af0 tx=0x7f1958034d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:07.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.758+0000 7f195ffff640 1 -- 172.21.15.33:0/3576546103 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958039050 con 0x7f1968107ab0 2024-04-02T13:49:07.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.758+0000 7f195ffff640 1 -- 172.21.15.33:0/3576546103 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1958045070 con 0x7f1968107ab0 2024-04-02T13:49:07.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.758+0000 7f196d78d640 1 -- 172.21.15.33:0/3576546103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196819d580 con 0x7f1968107ab0 2024-04-02T13:49:07.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.758+0000 7f195ffff640 1 -- 172.21.15.33:0/3576546103 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19580046b0 con 0x7f1968107ab0 2024-04-02T13:49:07.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.758+0000 7f196d78d640 1 -- 172.21.15.33:0/3576546103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19681a4e70 con 0x7f1968107ab0 2024-04-02T13:49:07.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.759+0000 7f196d78d640 1 -- 172.21.15.33:0/3576546103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19680688f0 con 0x7f1968107ab0 2024-04-02T13:49:07.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.761+0000 7f195ffff640 1 -- 172.21.15.33:0/3576546103 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1958003970 con 0x7f1968107ab0 2024-04-02T13:49:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.761+0000 7f195ffff640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f19540a3d20 0x7f19540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.761+0000 7f1966ffd640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f19540a3d20 0x7f19540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.761+0000 7f195ffff640 1 -- 172.21.15.33:0/3576546103 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f19580f40b0 con 0x7f1968107ab0 2024-04-02T13:49:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.762+0000 7f1966ffd640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f19540a3d20 0x7f19540a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1960005e10 tx=0x7f1960005da0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:07.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.765+0000 7f195ffff640 1 -- 172.21.15.33:0/3576546103 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f19580c0440 con 0x7f1968107ab0 2024-04-02T13:49:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.992+0000 7f196d78d640 1 -- 172.21.15.33:0/3576546103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f196810a030 con 0x7f1968107ab0 2024-04-02T13:49:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.993+0000 7f195ffff640 1 -- 172.21.15.33:0/3576546103 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11873 (secure 0 0 0) 0x7f19580bfd40 con 0x7f1968107ab0 2024-04-02T13:49:07.996 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:49:07.996 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:44:55.702357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:49:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 36 2024-04-02T13:49:07.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.995+0000 7f196d78d640 1 -- 172.21.15.33:0/3576546103 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f19540a3d20 msgr2=0x7f19540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.995+0000 7f196d78d640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f19540a3d20 0x7f19540a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1960005e10 tx=0x7f1960005da0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.995+0000 7f196d78d640 1 -- 172.21.15.33:0/3576546103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968107ab0 msgr2=0x7f196819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:07.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.995+0000 7f196d78d640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968107ab0 0x7f196819cbc0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f1958004af0 tx=0x7f1958034d70 comp rx=0 tx=0).stop 2024-04-02T13:49:07.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.995+0000 7f196d78d640 1 -- 172.21.15.33:0/3576546103 shutdown_connections 2024-04-02T13:49:07.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.995+0000 7f196d78d640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f19540a3d20 0x7f19540a61e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.995+0000 7f196d78d640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968107ab0 0x7f196819cbc0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.995+0000 7f196d78d640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1968102460 0x7f1968198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.995+0000 7f196d78d640 1 --2- 172.21.15.33:0/3576546103 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1968101aa0 0x7f1968198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:07.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.995+0000 7f196d78d640 1 -- 172.21.15.33:0/3576546103 >> 172.21.15.33:0/3576546103 conn(0x7f19680fd590 msgr2=0x7f19680fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:07.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.995+0000 7f196d78d640 1 -- 172.21.15.33:0/3576546103 shutdown_connections 2024-04-02T13:49:07.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:07.995+0000 7f196d78d640 1 -- 172.21.15.33:0/3576546103 wait complete. 2024-04-02T13:49:08.009 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:49:08.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.110+0000 7fadcc6a5640 1 -- 172.21.15.33:0/1722086184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc410c0e0 msgr2=0x7fadc410e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.110+0000 7fadcc6a5640 1 --2- 172.21.15.33:0/1722086184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc410c0e0 0x7fadc410e510 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009ca0 tx=0x7fadbc02c180 comp rx=0 tx=0).stop 2024-04-02T13:49:08.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.110+0000 7fadcc6a5640 1 -- 172.21.15.33:0/1722086184 shutdown_connections 2024-04-02T13:49:08.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.110+0000 7fadcc6a5640 1 --2- 172.21.15.33:0/1722086184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc410c0e0 0x7fadc410e510 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.110+0000 7fadcc6a5640 1 --2- 172.21.15.33:0/1722086184 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4103630 0x7fadc410bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.110+0000 7fadcc6a5640 1 --2- 172.21.15.33:0/1722086184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadc4102d10 0x7fadc41030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.110+0000 7fadcc6a5640 1 -- 172.21.15.33:0/1722086184 >> 172.21.15.33:0/1722086184 conn(0x7fadc4076d80 msgr2=0x7fadc4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.110+0000 7fadcc6a5640 1 -- 172.21.15.33:0/1722086184 shutdown_connections 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.110+0000 7fadcc6a5640 1 -- 172.21.15.33:0/1722086184 wait complete. 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadcc6a5640 1 Processor -- start 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadcc6a5640 1 -- start start 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadcc6a5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc4102d10 0x7fadc4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadcc6a5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4103630 0x7fadc41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadcc6a5640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadc410c0e0 0x7fadc419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadcc6a5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4111e40 con 0x7fadc4102d10 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadcc6a5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4111cc0 con 0x7fadc4103630 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadcc6a5640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4111fc0 con 0x7fadc410c0e0 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadca441640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc4102d10 0x7fadc4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadca441640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc4102d10 0x7fadc4198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49248/0 (socket says 172.21.15.33:49248) 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadca441640 1 -- 172.21.15.33:0/4241076024 learned_addr learned my addr 172.21.15.33:0/4241076024 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadc9c40640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4103630 0x7fadc41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadcac42640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadc410c0e0 0x7fadc419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadca441640 1 -- 172.21.15.33:0/4241076024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadc410c0e0 msgr2=0x7fadc419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadca441640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadc410c0e0 0x7fadc419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadca441640 1 -- 172.21.15.33:0/4241076024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4103630 msgr2=0x7fadc41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadca441640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4103630 0x7fadc41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.111+0000 7fadca441640 1 -- 172.21.15.33:0/4241076024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc419d350 con 0x7fadc4102d10 2024-04-02T13:49:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.112+0000 7fadcac42640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadc410c0e0 0x7fadc419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.112+0000 7fadca441640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc4102d10 0x7fadc4198360 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c970 tx=0x7fadc000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.112+0000 7fadbb7fe640 1 -- 172.21.15.33:0/4241076024 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0007bb0 con 0x7fadc4102d10 2024-04-02T13:49:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.112+0000 7fadcc6a5640 1 -- 172.21.15.33:0/4241076024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc419d640 con 0x7fadc4102d10 2024-04-02T13:49:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.112+0000 7fadbb7fe640 1 -- 172.21.15.33:0/4241076024 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fadc0007d50 con 0x7fadc4102d10 2024-04-02T13:49:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.112+0000 7fadbb7fe640 1 -- 172.21.15.33:0/4241076024 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc000f450 con 0x7fadc4102d10 2024-04-02T13:49:08.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.112+0000 7fadcc6a5640 1 -- 172.21.15.33:0/4241076024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc41a4ed0 con 0x7fadc4102d10 2024-04-02T13:49:08.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.114+0000 7fadbb7fe640 1 -- 172.21.15.33:0/4241076024 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fadc0004fb0 con 0x7fadc4102d10 2024-04-02T13:49:08.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.114+0000 7fadcf36e740 1 Processor -- start 2024-04-02T13:49:08.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcf36e740 1 -- start start 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcf36e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 0x5605ef3208b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcf36e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef320df0 0x5605ef32db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcf36e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef32e0b0 0x5605ef3304a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcf36e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5605ef2847b0 con 0x5605ef3204d0 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcf36e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5605ef223570 con 0x5605ef32e0b0 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcf36e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5605ef280760 con 0x5605ef320df0 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcac42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 0x5605ef3208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcac42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 0x5605ef3208b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49262/0 (socket says 172.21.15.33:49262) 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcac42640 1 -- 172.21.15.33:0/2525870492 learned_addr learned my addr 172.21.15.33:0/2525870492 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadbb7fe640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fadb00a3d20 0x7fadb00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadc9c40640 1 --2- 172.21.15.33:0/2525870492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef32e0b0 0x5605ef3304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcac42640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fadb00a3d20 0x7fadb00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadca441640 1 --2- 172.21.15.33:0/2525870492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef320df0 0x5605ef32db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadbb7fe640 1 -- 172.21.15.33:0/4241076024 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7fadc00d0990 con 0x7fadc4102d10 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcac42640 1 -- 172.21.15.33:0/2525870492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef320df0 msgr2=0x5605ef32db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcac42640 1 --2- 172.21.15.33:0/2525870492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef320df0 0x5605ef32db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcac42640 1 -- 172.21.15.33:0/2525870492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef32e0b0 msgr2=0x5605ef3304a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcac42640 1 --2- 172.21.15.33:0/2525870492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef32e0b0 0x5605ef3304a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadcac42640 1 -- 172.21.15.33:0/2525870492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605eef983f0 con 0x5605ef3204d0 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.115+0000 7fadca441640 1 --2- 172.21.15.33:0/2525870492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef320df0 0x5605ef32db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.116+0000 7fadcac42640 1 --2- 172.21.15.33:0/2525870492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 0x5605ef3208b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fadbc006010 tx=0x7fadbc003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7903e525fd16f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.116+0000 7fadb8ff9640 1 -- 172.21.15.33:0/2525870492 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc039040 con 0x5605ef3204d0 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.116+0000 7fadb8ff9640 1 -- 172.21.15.33:0/2525870492 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fadbc004790 con 0x5605ef3204d0 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.116+0000 7fadb8ff9640 1 -- 172.21.15.33:0/2525870492 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc04e800 con 0x5605ef3204d0 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.116+0000 7fadcf36e740 1 -- 172.21.15.33:0/2525870492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 msgr2=0x5605ef3208b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.116+0000 7fadcf36e740 1 --2- 172.21.15.33:0/2525870492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 0x5605ef3208b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fadbc006010 tx=0x7fadbc003d10 comp rx=0 tx=0).stop 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.116+0000 7fadcac42640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fadb00a3d20 0x7fadb00a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0325b0 tx=0x7fadbc049000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.116+0000 7fadcf36e740 1 -- 172.21.15.33:0/2525870492 shutdown_connections 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.116+0000 7fadcf36e740 1 --2- 172.21.15.33:0/2525870492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef32e0b0 0x5605ef3304a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.116+0000 7fadcf36e740 1 --2- 172.21.15.33:0/2525870492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef320df0 0x5605ef32db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.116+0000 7fadcf36e740 1 --2- 172.21.15.33:0/2525870492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 0x5605ef3208b0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadcf36e740 1 -- 172.21.15.33:0/2525870492 >> 172.21.15.33:0/2525870492 conn(0x5605ef31aed0 msgr2=0x5605ef31b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadcf36e740 1 -- 172.21.15.33:0/2525870492 shutdown_connections 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadcf36e740 1 -- 172.21.15.33:0/2525870492 wait complete. 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadcf36e740 1 -- start start 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadcf36e740 1 Processor -- start 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadcf36e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef3204d0 0x5605ef32af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadcf36e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef320df0 0x5605ef32b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadcf36e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef32e0b0 0x5605ef32b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadcf36e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5605ef330e70 con 0x5605ef320df0 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadcf36e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5605ef23fe80 con 0x5605ef3204d0 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadcf36e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5605ef330ff0 con 0x5605ef32e0b0 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadcf36e740 10 client.0 _ll_register_callbacks cb 0x5605ef27baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadc9c40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef320df0 0x5605ef32b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadc9c40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef320df0 0x5605ef32b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49278/0 (socket says 172.21.15.33:49278) 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.117+0000 7fadc9c40640 1 -- 172.21.15.33:0/1329138961 learned_addr learned my addr 172.21.15.33:0/1329138961 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadcac42640 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef32e0b0 0x5605ef32b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadca441640 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef3204d0 0x5605ef32af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadc9c40640 1 -- 172.21.15.33:0/1329138961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef32e0b0 msgr2=0x5605ef32b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadc9c40640 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef32e0b0 0x5605ef32b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadc9c40640 1 -- 172.21.15.33:0/1329138961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef3204d0 msgr2=0x5605ef32af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadc9c40640 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef3204d0 0x5605ef32af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadc9c40640 1 -- 172.21.15.33:0/1329138961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605ef0627d0 con 0x5605ef320df0 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadca441640 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef3204d0 0x5605ef32af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadc9c40640 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef320df0 0x5605ef32b490 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fadb400b9f0 tx=0x7fadb400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82075789d7ecfb2a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadadffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadadffb640 1 -- 172.21.15.33:0/1329138961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4003e20 con 0x5605ef320df0 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadcf36e740 1 -- 172.21.15.33:0/1329138961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadbc004790 con 0x5605ef320df0 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadadffb640 1 -- 172.21.15.33:0/1329138961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7fadb4005020 con 0x5605ef320df0 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadadffb640 1 -- 172.21.15.33:0/1329138961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fadb4005220 con 0x5605ef320df0 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadcf36e740 10 client.17829 fetch_fsmap learned FSMap version 36 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadcf36e740 1 -- 172.21.15.33:0/1329138961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5605ef33cd80 con 0x5605ef320df0 2024-04-02T13:49:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.118+0000 7fadadffb640 1 -- 172.21.15.33:0/1329138961 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fadbc004790 con 0x5605ef320df0 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.119+0000 7fadadffb640 1 -- 172.21.15.33:0/1329138961 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fadb401b610 con 0x5605ef320df0 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.119+0000 7fadcf36e740 10 client.17829 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.119+0000 7fadcf36e740 10 client.17829 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.119+0000 7fadcf36e740 20 client.17829 populate_metadata read hostname 'smithi033' 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.119+0000 7fadcf36e740 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5605ef33e240 0x5605ef340690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.119+0000 7fadcf36e740 4 client.17829 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.119+0000 7fadcf36e740 1 -- 172.21.15.33:0/1329138961 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605ef291cf0 con 0x5605ef33e240 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.119+0000 7fadca441640 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5605ef33e240 0x5605ef340690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.119+0000 7fadca441640 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5605ef33e240 0x5605ef340690 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62c1175edac7ca5b server_cookie=78abf3b2df6acec4 in_seq=0 out_seq=0 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.119+0000 7fadadffb640 10 client.17829 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadadffb640 1 -- 172.21.15.33:0/1329138961 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605ef291cf0 con 0x5605ef33e240 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadadffb640 10 client.17829 handle_command_reply: tid=0 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadcf36e740 1 client.17829 shutdown 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadcf36e740 1 -- 172.21.15.33:0/1329138961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef320df0 msgr2=0x5605ef32b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadcf36e740 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef320df0 0x5605ef32b490 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fadb400b9f0 tx=0x7fadb400beb0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadcf36e740 1 -- 172.21.15.33:0/1329138961 shutdown_connections 2024-04-02T13:49:08.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadcf36e740 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5605ef33e240 0x5605ef340690 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadcf36e740 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef32e0b0 0x5605ef32b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadcf36e740 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef320df0 0x5605ef32b490 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadadffb640 0 client.17829 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:49:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadcf36e740 1 --2- 172.21.15.33:0/1329138961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef3204d0 0x5605ef32af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadcf36e740 1 -- 172.21.15.33:0/1329138961 >> 172.21.15.33:0/1329138961 conn(0x5605ef31aed0 msgr2=0x5605ef31cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadcf36e740 1 -- 172.21.15.33:0/1329138961 shutdown_connections 2024-04-02T13:49:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadcf36e740 1 -- 172.21.15.33:0/1329138961 wait complete. 2024-04-02T13:49:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.120+0000 7fadcf36e740 20 client.17829 trim_cache size 0 max 16384 2024-04-02T13:49:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadcf36e740 1 Processor -- start 2024-04-02T13:49:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadcf36e740 1 -- start start 2024-04-02T13:49:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadcf36e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 0x5605ef32d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadcf36e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef320df0 0x5605ef339d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadcf36e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef32e0b0 0x5605ef329f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadcf36e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41af890 con 0x5605ef3204d0 2024-04-02T13:49:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadcf36e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41ad160 con 0x5605ef32e0b0 2024-04-02T13:49:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadcf36e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41aafb0 con 0x5605ef320df0 2024-04-02T13:49:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadca441640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 0x5605ef32d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadca441640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 0x5605ef32d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49290/0 (socket says 172.21.15.33:49290) 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadca441640 1 -- 172.21.15.33:0/622560223 learned_addr learned my addr 172.21.15.33:0/622560223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadc9c40640 1 --2- 172.21.15.33:0/622560223 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef320df0 0x5605ef339d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadca441640 1 -- 172.21.15.33:0/622560223 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef320df0 msgr2=0x5605ef339d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadca441640 1 --2- 172.21.15.33:0/622560223 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef320df0 0x5605ef339d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadca441640 1 -- 172.21.15.33:0/622560223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef32e0b0 msgr2=0x5605ef329f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadcac42640 1 --2- 172.21.15.33:0/622560223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef32e0b0 0x5605ef329f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadca441640 1 --2- 172.21.15.33:0/622560223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef32e0b0 0x5605ef329f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadca441640 1 -- 172.21.15.33:0/622560223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605ef3294a0 con 0x5605ef3204d0 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadcac42640 1 --2- 172.21.15.33:0/622560223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef32e0b0 0x5605ef329f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadca441640 1 --2- 172.21.15.33:0/622560223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 0x5605ef32d680 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fadc0098970 tx=0x7fadc00145e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a864b9103a2a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadaffff640 1 -- 172.21.15.33:0/622560223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc00a08c0 con 0x5605ef3204d0 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.134+0000 7fadaffff640 1 -- 172.21.15.33:0/622560223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fadc00a5180 con 0x5605ef3204d0 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.135+0000 7fadaffff640 1 -- 172.21.15.33:0/622560223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0098d10 con 0x5605ef3204d0 2024-04-02T13:49:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.135+0000 7fadcf36e740 1 -- 172.21.15.33:0/622560223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 msgr2=0x5605ef32d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.135+0000 7fadcf36e740 1 --2- 172.21.15.33:0/622560223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 0x5605ef32d680 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fadc0098970 tx=0x7fadc00145e0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.135+0000 7fadcf36e740 1 -- 172.21.15.33:0/622560223 shutdown_connections 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.135+0000 7fadcf36e740 1 --2- 172.21.15.33:0/622560223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef32e0b0 0x5605ef329f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.135+0000 7fadcf36e740 1 --2- 172.21.15.33:0/622560223 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef320df0 0x5605ef339d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.135+0000 7fadcf36e740 1 --2- 172.21.15.33:0/622560223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef3204d0 0x5605ef32d680 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.135+0000 7fadcf36e740 1 -- 172.21.15.33:0/622560223 >> 172.21.15.33:0/622560223 conn(0x5605ef31aed0 msgr2=0x5605ef329820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.135+0000 7fadcf36e740 1 -- 172.21.15.33:0/622560223 shutdown_connections 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.135+0000 7fadcf36e740 1 -- 172.21.15.33:0/622560223 wait complete. 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.135+0000 7fadcf36e740 1 -- start start 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.135+0000 7fadcf36e740 1 Processor -- start 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcf36e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef3204d0 0x5605ef336280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcf36e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef320df0 0x5605ef3367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcf36e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef32e0b0 0x5605ef327780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcf36e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41ab620 con 0x5605ef32e0b0 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcf36e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41aca90 con 0x5605ef320df0 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcf36e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41abcf0 con 0x5605ef3204d0 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcf36e740 10 client.0 _ll_register_callbacks cb 0x7fadb4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcac42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef32e0b0 0x5605ef327780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcac42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef32e0b0 0x5605ef327780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49300/0 (socket says 172.21.15.33:49300) 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcac42640 1 -- 172.21.15.33:0/2903601484 learned_addr learned my addr 172.21.15.33:0/2903601484 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadca441640 1 --2- 172.21.15.33:0/2903601484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef3204d0 0x5605ef336280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcac42640 1 -- 172.21.15.33:0/2903601484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef3204d0 msgr2=0x5605ef336280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcac42640 1 --2- 172.21.15.33:0/2903601484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef3204d0 0x5605ef336280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcac42640 1 -- 172.21.15.33:0/2903601484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef320df0 msgr2=0x5605ef3367c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcac42640 1 --2- 172.21.15.33:0/2903601484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef320df0 0x5605ef3367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcac42640 1 -- 172.21.15.33:0/2903601484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605ef327f80 con 0x5605ef32e0b0 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcac42640 1 --2- 172.21.15.33:0/2903601484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef32e0b0 0x5605ef327780 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fadbc03b000 tx=0x7fadbc04e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7daab076afde2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadadffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadadffb640 1 -- 172.21.15.33:0/2903601484 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc039040 con 0x5605ef32e0b0 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadadffb640 1 -- 172.21.15.33:0/2903601484 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7fadbc04ec70 con 0x5605ef32e0b0 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.136+0000 7fadcf36e740 1 -- 172.21.15.33:0/2903601484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc00a5180 con 0x5605ef32e0b0 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.137+0000 7fadadffb640 1 -- 172.21.15.33:0/2903601484 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fadbc066050 con 0x5605ef32e0b0 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.137+0000 7fadcf36e740 10 client.17835 fetch_fsmap learned FSMap version 36 2024-04-02T13:49:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.137+0000 7fadcf36e740 1 -- 172.21.15.33:0/2903601484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5605ef3a4320 con 0x5605ef32e0b0 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.137+0000 7fadadffb640 1 -- 172.21.15.33:0/2903601484 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fadc00a5180 con 0x5605ef32e0b0 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.137+0000 7fadadffb640 1 -- 172.21.15.33:0/2903601484 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fadbc03f070 con 0x5605ef32e0b0 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.137+0000 7fadcf36e740 10 client.17835 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.137+0000 7fadcf36e740 10 client.17835 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.137+0000 7fadcf36e740 20 client.17835 populate_metadata read hostname 'smithi033' 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.137+0000 7fadcf36e740 1 --2- 172.21.15.33:0/2903601484 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5605ef3a46d0 0x5605ef3a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.137+0000 7fadcf36e740 4 client.17835 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.137+0000 7fadcf36e740 1 -- 172.21.15.33:0/2903601484 --> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605ef28c040 con 0x5605ef3a46d0 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.137+0000 7fadca441640 1 --2- 172.21.15.33:0/2903601484 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5605ef3a46d0 0x5605ef3a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadca441640 1 --2- 172.21.15.33:0/2903601484 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5605ef3a46d0 0x5605ef3a6af0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13dee8dd4f98fe90 server_cookie=f36c2664b4ec9406 in_seq=0 out_seq=0 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadadffb640 10 client.17835 ms_handle_connect on v2:172.21.15.50:6834/1580073716 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadadffb640 1 -- 172.21.15.33:0/2903601484 <== mds.2 v2:172.21.15.50:6834/1580073716 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605ef28c040 con 0x5605ef3a46d0 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadadffb640 10 client.17835 handle_command_reply: tid=0 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadcf36e740 1 client.17835 shutdown 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadcf36e740 1 -- 172.21.15.33:0/2903601484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef32e0b0 msgr2=0x5605ef327780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadcf36e740 1 --2- 172.21.15.33:0/2903601484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef32e0b0 0x5605ef327780 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fadbc03b000 tx=0x7fadbc04e730 comp rx=0 tx=0).stop 2024-04-02T13:49:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadcf36e740 1 -- 172.21.15.33:0/2903601484 shutdown_connections 2024-04-02T13:49:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadcf36e740 1 --2- 172.21.15.33:0/2903601484 >> [v2:172.21.15.50:6834/1580073716,v1:172.21.15.50:6835/1580073716] conn(0x5605ef3a46d0 0x5605ef3a6af0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadcf36e740 1 --2- 172.21.15.33:0/2903601484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5605ef32e0b0 0x5605ef327780 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadcf36e740 1 --2- 172.21.15.33:0/2903601484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605ef320df0 0x5605ef3367c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadadffb640 0 client.17835 ms_handle_reset on v2:172.21.15.50:6834/1580073716 2024-04-02T13:49:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadcf36e740 1 --2- 172.21.15.33:0/2903601484 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5605ef3204d0 0x5605ef336280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadcf36e740 1 -- 172.21.15.33:0/2903601484 >> 172.21.15.33:0/2903601484 conn(0x5605ef31aed0 msgr2=0x5605ef31c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadcf36e740 1 -- 172.21.15.33:0/2903601484 shutdown_connections 2024-04-02T13:49:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadcf36e740 1 -- 172.21.15.33:0/2903601484 wait complete. 2024-04-02T13:49:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.138+0000 7fadcf36e740 20 client.17835 trim_cache size 0 max 16384 2024-04-02T13:49:08.146 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:49:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.139+0000 7fadcc6a5640 1 -- 172.21.15.33:0/4241076024 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fadb00a3d20 msgr2=0x7fadb00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.139+0000 7fadcc6a5640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fadb00a3d20 0x7fadb00a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0325b0 tx=0x7fadbc049000 comp rx=0 tx=0).stop 2024-04-02T13:49:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.139+0000 7fadcc6a5640 1 -- 172.21.15.33:0/4241076024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc4102d10 msgr2=0x7fadc4198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.139+0000 7fadcc6a5640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc4102d10 0x7fadc4198360 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c970 tx=0x7fadc000ce30 comp rx=0 tx=0).stop 2024-04-02T13:49:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.139+0000 7fadcc6a5640 1 -- 172.21.15.33:0/4241076024 shutdown_connections 2024-04-02T13:49:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.139+0000 7fadcc6a5640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fadb00a3d20 0x7fadb00a61e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.139+0000 7fadcc6a5640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadc410c0e0 0x7fadc419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.139+0000 7fadcc6a5640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4103630 0x7fadc41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.139+0000 7fadcc6a5640 1 --2- 172.21.15.33:0/4241076024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc4102d10 0x7fadc4198360 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.139+0000 7fadcc6a5640 1 -- 172.21.15.33:0/4241076024 >> 172.21.15.33:0/4241076024 conn(0x7fadc4076d80 msgr2=0x7fadc40fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.139+0000 7fadcc6a5640 1 -- 172.21.15.33:0/4241076024 shutdown_connections 2024-04-02T13:49:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.139+0000 7fadcc6a5640 1 -- 172.21.15.33:0/4241076024 wait complete. 2024-04-02T13:49:08.152 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:49:08.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.254+0000 7f1584baf640 1 -- 172.21.15.33:0/1369554341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1580102450 msgr2=0x7f158010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.254+0000 7f1584baf640 1 --2- 172.21.15.33:0/1369554341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1580102450 0x7f158010e960 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f1578009cd0 tx=0x7f157802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.254+0000 7f1584baf640 1 -- 172.21.15.33:0/1369554341 shutdown_connections 2024-04-02T13:49:08.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.254+0000 7f1584baf640 1 --2- 172.21.15.33:0/1369554341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1580102450 0x7f158010e960 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.254+0000 7f1584baf640 1 --2- 172.21.15.33:0/1369554341 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1580101a90 0x7f1580101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.254+0000 7f1584baf640 1 --2- 172.21.15.33:0/1369554341 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1580107aa0 0x7f1580107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.254+0000 7f1584baf640 1 -- 172.21.15.33:0/1369554341 >> 172.21.15.33:0/1369554341 conn(0x7f15800fd5a0 msgr2=0x7f15800ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.254+0000 7f1584baf640 1 -- 172.21.15.33:0/1369554341 shutdown_connections 2024-04-02T13:49:08.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.254+0000 7f1584baf640 1 -- 172.21.15.33:0/1369554341 wait complete. 2024-04-02T13:49:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.255+0000 7f1584baf640 1 Processor -- start 2024-04-02T13:49:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.255+0000 7f1584baf640 1 -- start start 2024-04-02T13:49:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.255+0000 7f1584baf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1580101a90 0x7f15801982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.255+0000 7f1584baf640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1580102450 0x7f1580198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.255+0000 7f1584baf640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1580107aa0 0x7f158019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.255+0000 7f1584baf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1580111e50 con 0x7f1580101a90 2024-04-02T13:49:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.255+0000 7f1584baf640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1580111cd0 con 0x7f1580107aa0 2024-04-02T13:49:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.255+0000 7f1584baf640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1580111fd0 con 0x7f1580102450 2024-04-02T13:49:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.255+0000 7f157e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1580101a90 0x7f15801982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.255+0000 7f157e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1580101a90 0x7f15801982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49312/0 (socket says 172.21.15.33:49312) 2024-04-02T13:49:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.255+0000 7f157e59c640 1 -- 172.21.15.33:0/2876137922 learned_addr learned my addr 172.21.15.33:0/2876137922 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.255+0000 7f1577fff640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1580102450 0x7f1580198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.256+0000 7f157ed9d640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1580107aa0 0x7f158019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.256+0000 7f157e59c640 1 -- 172.21.15.33:0/2876137922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1580102450 msgr2=0x7f1580198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.256+0000 7f157e59c640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1580102450 0x7f1580198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.256+0000 7f157e59c640 1 -- 172.21.15.33:0/2876137922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1580107aa0 msgr2=0x7f158019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.256+0000 7f157e59c640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1580107aa0 0x7f158019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.256+0000 7f157e59c640 1 -- 172.21.15.33:0/2876137922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158019d2c0 con 0x7f1580101a90 2024-04-02T13:49:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.256+0000 7f157e59c640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1580101a90 0x7f15801982d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f157000c920 tx=0x7f157000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.256+0000 7f15777fe640 1 -- 172.21.15.33:0/2876137922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570007c70 con 0x7f1580101a90 2024-04-02T13:49:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.256+0000 7f15777fe640 1 -- 172.21.15.33:0/2876137922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1570007e10 con 0x7f1580101a90 2024-04-02T13:49:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.256+0000 7f15777fe640 1 -- 172.21.15.33:0/2876137922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570018850 con 0x7f1580101a90 2024-04-02T13:49:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.256+0000 7f1584baf640 1 -- 172.21.15.33:0/2876137922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158019d5b0 con 0x7f1580101a90 2024-04-02T13:49:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.256+0000 7f1584baf640 1 -- 172.21.15.33:0/2876137922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15801a4e40 con 0x7f1580101a90 2024-04-02T13:49:08.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.258+0000 7f15777fe640 1 -- 172.21.15.33:0/2876137922 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1570018a30 con 0x7f1580101a90 2024-04-02T13:49:08.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.258+0000 7f158779a740 1 Processor -- start 2024-04-02T13:49:08.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f158779a740 1 -- start start 2024-04-02T13:49:08.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f158779a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 0x558951d34b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f158779a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d350c0 0x558951d3d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f158779a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 0x558951d40090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f158779a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558951c987b0 con 0x558951d347a0 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f158779a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558951c37570 con 0x558951d350c0 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f158779a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558951c94760 con 0x558951d3dca0 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f15777fe640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f156c0a3d20 0x7f156c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f157ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 0x558951d34b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f157ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 0x558951d34b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49328/0 (socket says 172.21.15.33:49328) 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f157ed9d640 1 -- 172.21.15.33:0/1540766586 learned_addr learned my addr 172.21.15.33:0/1540766586 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f157e59c640 1 --2- 172.21.15.33:0/1540766586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d350c0 0x558951d3d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f1577fff640 1 --2- 172.21.15.33:0/1540766586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 0x558951d40090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f15777fe640 1 -- 172.21.15.33:0/2876137922 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f15700d0dd0 con 0x7f1580101a90 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f157ed9d640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f156c0a3d20 0x7f156c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f157ed9d640 1 -- 172.21.15.33:0/1540766586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 msgr2=0x558951d40090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f157ed9d640 1 --2- 172.21.15.33:0/1540766586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 0x558951d40090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f157ed9d640 1 -- 172.21.15.33:0/1540766586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d350c0 msgr2=0x558951d3d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f157ed9d640 1 --2- 172.21.15.33:0/1540766586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d350c0 0x558951d3d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.259+0000 7f157ed9d640 1 -- 172.21.15.33:0/1540766586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589519ac3f0 con 0x558951d347a0 2024-04-02T13:49:08.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.260+0000 7f157ed9d640 1 --2- 172.21.15.33:0/1540766586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 0x558951d34b80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1578009430 tx=0x7f1578032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220b23fc8aa342b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.260+0000 7f1574ff9640 1 -- 172.21.15.33:0/1540766586 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578039040 con 0x558951d347a0 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.260+0000 7f1574ff9640 1 -- 172.21.15.33:0/1540766586 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1578032870 con 0x558951d347a0 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.260+0000 7f1574ff9640 1 -- 172.21.15.33:0/1540766586 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15780346a0 con 0x558951d347a0 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 -- 172.21.15.33:0/1540766586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 msgr2=0x558951d34b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f157ed9d640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f156c0a3d20 0x7f156c0a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1578048000 tx=0x7f1578047000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 --2- 172.21.15.33:0/1540766586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 0x558951d34b80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1578009430 tx=0x7f1578032e40 comp rx=0 tx=0).stop 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 -- 172.21.15.33:0/1540766586 shutdown_connections 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 --2- 172.21.15.33:0/1540766586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 0x558951d40090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 --2- 172.21.15.33:0/1540766586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d350c0 0x558951d3d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 --2- 172.21.15.33:0/1540766586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 0x558951d34b80 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 -- 172.21.15.33:0/1540766586 >> 172.21.15.33:0/1540766586 conn(0x558951d2eed0 msgr2=0x558951d2f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 -- 172.21.15.33:0/1540766586 shutdown_connections 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 -- 172.21.15.33:0/1540766586 wait complete. 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 -- start start 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 Processor -- start 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d347a0 0x558951d3d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 0x558951d3a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d3dca0 0x558951d3af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558951d40a90 con 0x558951d350c0 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558951c53e80 con 0x558951d3dca0 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558951d40c10 con 0x558951d347a0 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f158779a740 10 client.0 _ll_register_callbacks cb 0x558951c8faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f1577fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 0x558951d3a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f1577fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 0x558951d3a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49336/0 (socket says 172.21.15.33:49336) 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.261+0000 7f1577fff640 1 -- 172.21.15.33:0/976385599 learned_addr learned my addr 172.21.15.33:0/976385599 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f157e59c640 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d347a0 0x558951d3d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f157ed9d640 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d3dca0 0x558951d3af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f1577fff640 1 -- 172.21.15.33:0/976385599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d347a0 msgr2=0x558951d3d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f1577fff640 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d347a0 0x558951d3d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f1577fff640 1 -- 172.21.15.33:0/976385599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d3dca0 msgr2=0x558951d3af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f1577fff640 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d3dca0 0x558951d3af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f1577fff640 1 -- 172.21.15.33:0/976385599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558951a767d0 con 0x558951d350c0 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f157e59c640 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d347a0 0x558951d3d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f1577fff640 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 0x558951d3a9e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f156800bf30 tx=0x7f15680128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450e4ccdc770a955 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f1565ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f1565ffb640 1 -- 172.21.15.33:0/976385599 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156801f060 con 0x558951d350c0 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f1565ffb640 1 -- 172.21.15.33:0/976385599 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f15680108a0 con 0x558951d350c0 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f158779a740 1 -- 172.21.15.33:0/976385599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1578032870 con 0x558951d350c0 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f1565ffb640 1 -- 172.21.15.33:0/976385599 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1568026050 con 0x558951d350c0 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f158779a740 10 client.17847 fetch_fsmap learned FSMap version 36 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f158779a740 1 -- 172.21.15.33:0/976385599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558951d4cb00 con 0x558951d350c0 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.262+0000 7f1565ffb640 1 -- 172.21.15.33:0/976385599 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1578032870 con 0x558951d350c0 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.263+0000 7f1565ffb640 1 -- 172.21.15.33:0/976385599 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f1568019030 con 0x558951d350c0 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.263+0000 7f158779a740 10 client.17847 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.263+0000 7f158779a740 10 client.17847 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.263+0000 7f158779a740 20 client.17847 populate_metadata read hostname 'smithi033' 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.263+0000 7f158779a740 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558951d4e060 0x558951d504e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.263+0000 7f158779a740 4 client.17847 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.263+0000 7f158779a740 1 -- 172.21.15.33:0/976385599 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558951ca5cf0 con 0x558951d4e060 2024-04-02T13:49:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.263+0000 7f157e59c640 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558951d4e060 0x558951d504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.263+0000 7f157e59c640 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558951d4e060 0x558951d504e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e25d6d463df02290 server_cookie=7e6099eef6b395f6 in_seq=0 out_seq=0 2024-04-02T13:49:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.263+0000 7f1565ffb640 10 client.17847 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:49:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f1565ffb640 1 -- 172.21.15.33:0/976385599 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558951ca5cf0 con 0x558951d4e060 2024-04-02T13:49:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f1565ffb640 10 client.17847 handle_command_reply: tid=0 2024-04-02T13:49:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f158779a740 1 client.17847 shutdown 2024-04-02T13:49:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f158779a740 1 -- 172.21.15.33:0/976385599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 msgr2=0x558951d3a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f158779a740 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 0x558951d3a9e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f156800bf30 tx=0x7f15680128e0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f158779a740 1 -- 172.21.15.33:0/976385599 shutdown_connections 2024-04-02T13:49:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f158779a740 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d3dca0 0x558951d3af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f158779a740 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 0x558951d3a9e0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f158779a740 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558951d4e060 0x558951d504e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f158779a740 1 --2- 172.21.15.33:0/976385599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d347a0 0x558951d3d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f1565ffb640 0 client.17847 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:49:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f158779a740 1 -- 172.21.15.33:0/976385599 >> 172.21.15.33:0/976385599 conn(0x558951d2eed0 msgr2=0x558951d30c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f158779a740 1 -- 172.21.15.33:0/976385599 shutdown_connections 2024-04-02T13:49:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f158779a740 1 -- 172.21.15.33:0/976385599 wait complete. 2024-04-02T13:49:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.264+0000 7f158779a740 20 client.17847 trim_cache size 0 max 16384 2024-04-02T13:49:08.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f158779a740 1 Processor -- start 2024-04-02T13:49:08.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f158779a740 1 -- start start 2024-04-02T13:49:08.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f158779a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 0x558951d499b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f158779a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d350c0 0x558951d3d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f158779a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 0x558951d3d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f158779a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15801b19b0 con 0x558951d347a0 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f158779a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f15801af280 con 0x558951d350c0 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f158779a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f15801ad0d0 con 0x558951d3dca0 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f157e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 0x558951d499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f157e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 0x558951d499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49352/0 (socket says 172.21.15.33:49352) 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f157e59c640 1 -- 172.21.15.33:0/1733721476 learned_addr learned my addr 172.21.15.33:0/1733721476 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f157ed9d640 1 --2- 172.21.15.33:0/1733721476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 0x558951d3d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f1577fff640 1 --2- 172.21.15.33:0/1733721476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d350c0 0x558951d3d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f157e59c640 1 -- 172.21.15.33:0/1733721476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 msgr2=0x558951d3d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f157e59c640 1 --2- 172.21.15.33:0/1733721476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 0x558951d3d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f157e59c640 1 -- 172.21.15.33:0/1733721476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d350c0 msgr2=0x558951d3d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f157e59c640 1 --2- 172.21.15.33:0/1733721476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d350c0 0x558951d3d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f157e59c640 1 -- 172.21.15.33:0/1733721476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558951d3b2f0 con 0x558951d347a0 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f157ed9d640 1 --2- 172.21.15.33:0/1733721476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 0x558951d3d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.278+0000 7f1577fff640 1 --2- 172.21.15.33:0/1733721476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d350c0 0x558951d3d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.279+0000 7f157e59c640 1 --2- 172.21.15.33:0/1733721476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 0x558951d499b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f157000c8f0 tx=0x7f1570013720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbf6aa942e9f0a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.279+0000 7f1567fff640 1 -- 172.21.15.33:0/1733721476 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157009c910 con 0x558951d347a0 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.279+0000 7f1567fff640 1 -- 172.21.15.33:0/1733721476 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1570085e50 con 0x558951d347a0 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.279+0000 7f1567fff640 1 -- 172.21.15.33:0/1733721476 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570098f80 con 0x558951d347a0 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.279+0000 7f158779a740 1 -- 172.21.15.33:0/1733721476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 msgr2=0x558951d499b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.279+0000 7f158779a740 1 --2- 172.21.15.33:0/1733721476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 0x558951d499b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f157000c8f0 tx=0x7f1570013720 comp rx=0 tx=0).stop 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.279+0000 7f158779a740 1 -- 172.21.15.33:0/1733721476 shutdown_connections 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.279+0000 7f158779a740 1 --2- 172.21.15.33:0/1733721476 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 0x558951d3d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.279+0000 7f158779a740 1 --2- 172.21.15.33:0/1733721476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d350c0 0x558951d3d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.279+0000 7f158779a740 1 --2- 172.21.15.33:0/1733721476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d347a0 0x558951d499b0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.279+0000 7f158779a740 1 -- 172.21.15.33:0/1733721476 >> 172.21.15.33:0/1733721476 conn(0x558951d2eed0 msgr2=0x558951d2f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.279+0000 7f158779a740 1 -- 172.21.15.33:0/1733721476 shutdown_connections 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.279+0000 7f158779a740 1 -- 172.21.15.33:0/1733721476 wait complete. 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f158779a740 1 -- start start 2024-04-02T13:49:08.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f158779a740 1 Processor -- start 2024-04-02T13:49:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f158779a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d347a0 0x558951d463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f158779a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 0x558951d3e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f158779a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 0x558951d3f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f158779a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15801ad740 con 0x558951d350c0 2024-04-02T13:49:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f158779a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f15801aebb0 con 0x558951d347a0 2024-04-02T13:49:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f158779a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f15801ade10 con 0x558951d3dca0 2024-04-02T13:49:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f158779a740 10 client.0 _ll_register_callbacks cb 0x7f156800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f1577fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 0x558951d3e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f1577fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 0x558951d3e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49360/0 (socket says 172.21.15.33:49360) 2024-04-02T13:49:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f1577fff640 1 -- 172.21.15.33:0/3762234842 learned_addr learned my addr 172.21.15.33:0/3762234842 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f157ed9d640 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 0x558951d3f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f157e59c640 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d347a0 0x558951d463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f1577fff640 1 -- 172.21.15.33:0/3762234842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 msgr2=0x558951d3f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f1577fff640 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 0x558951d3f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f1577fff640 1 -- 172.21.15.33:0/3762234842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d347a0 msgr2=0x558951d463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f1577fff640 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d347a0 0x558951d463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f1577fff640 1 -- 172.21.15.33:0/3762234842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558951d34ef0 con 0x558951d350c0 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f157e59c640 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d347a0 0x558951d463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f1577fff640 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 0x558951d3e940 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f156801af60 tx=0x7f1568013e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e622df385e64a4c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.280+0000 7f1565ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f1565ffb640 1 -- 172.21.15.33:0/3762234842 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568005510 con 0x558951d350c0 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f1565ffb640 1 -- 172.21.15.33:0/3762234842 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f156801a3f0 con 0x558951d350c0 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f158779a740 1 -- 172.21.15.33:0/3762234842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1570085e50 con 0x558951d350c0 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f1565ffb640 1 -- 172.21.15.33:0/3762234842 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f156801f020 con 0x558951d350c0 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f158779a740 10 client.17853 fetch_fsmap learned FSMap version 36 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f158779a740 1 -- 172.21.15.33:0/3762234842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558951db4660 con 0x558951d350c0 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f1565ffb640 1 -- 172.21.15.33:0/3762234842 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1570085e50 con 0x558951d350c0 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f1565ffb640 1 -- 172.21.15.33:0/3762234842 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f1568019050 con 0x558951d350c0 2024-04-02T13:49:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f158779a740 10 client.17853 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f158779a740 10 client.17853 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f158779a740 20 client.17853 populate_metadata read hostname 'smithi033' 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f158779a740 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558951db4d10 0x558951db7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f158779a740 4 client.17853 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f158779a740 1 -- 172.21.15.33:0/3762234842 --> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558951ca0040 con 0x558951db4d10 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.281+0000 7f157e59c640 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558951db4d10 0x558951db7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f1565ffb640 10 client.17853 ms_handle_connect on v2:172.21.15.50:6836/3788578797 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f157e59c640 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558951db4d10 0x558951db7130 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d04e8b756814d6a0 server_cookie=1ba7b714512ac772 in_seq=0 out_seq=0 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f1565ffb640 1 -- 172.21.15.33:0/3762234842 <== mds.1 v2:172.21.15.50:6836/3788578797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558951ca0040 con 0x558951db4d10 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f1565ffb640 10 client.17853 handle_command_reply: tid=0 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f158779a740 1 client.17853 shutdown 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f158779a740 1 -- 172.21.15.33:0/3762234842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 msgr2=0x558951d3e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f158779a740 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 0x558951d3e940 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f156801af60 tx=0x7f1568013e60 comp rx=0 tx=0).stop 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f158779a740 1 -- 172.21.15.33:0/3762234842 shutdown_connections 2024-04-02T13:49:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f158779a740 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.50:6836/3788578797,v1:172.21.15.50:6837/3788578797] conn(0x558951db4d10 0x558951db7130 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f158779a740 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558951d3dca0 0x558951d3f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f1565ffb640 0 client.17853 ms_handle_reset on v2:172.21.15.50:6836/3788578797 2024-04-02T13:49:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f158779a740 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558951d350c0 0x558951d3e940 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f158779a740 1 --2- 172.21.15.33:0/3762234842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558951d347a0 0x558951d463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f158779a740 1 -- 172.21.15.33:0/3762234842 >> 172.21.15.33:0/3762234842 conn(0x558951d2eed0 msgr2=0x558951d39270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f158779a740 1 -- 172.21.15.33:0/3762234842 shutdown_connections 2024-04-02T13:49:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f158779a740 1 -- 172.21.15.33:0/3762234842 wait complete. 2024-04-02T13:49:08.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.282+0000 7f158779a740 20 client.17853 trim_cache size 0 max 16384 2024-04-02T13:49:08.291 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:49:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.283+0000 7f1584baf640 1 -- 172.21.15.33:0/2876137922 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f156c0a3d20 msgr2=0x7f156c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.283+0000 7f1584baf640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f156c0a3d20 0x7f156c0a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1578048000 tx=0x7f1578047000 comp rx=0 tx=0).stop 2024-04-02T13:49:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.283+0000 7f1584baf640 1 -- 172.21.15.33:0/2876137922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1580101a90 msgr2=0x7f15801982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.283+0000 7f1584baf640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1580101a90 0x7f15801982d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f157000c920 tx=0x7f157000cde0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.283+0000 7f1584baf640 1 -- 172.21.15.33:0/2876137922 shutdown_connections 2024-04-02T13:49:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.283+0000 7f1584baf640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f156c0a3d20 0x7f156c0a61e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.283+0000 7f1584baf640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1580107aa0 0x7f158019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.283+0000 7f1584baf640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1580102450 0x7f1580198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.283+0000 7f1584baf640 1 --2- 172.21.15.33:0/2876137922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1580101a90 0x7f15801982d0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.283+0000 7f1584baf640 1 -- 172.21.15.33:0/2876137922 >> 172.21.15.33:0/2876137922 conn(0x7f15800fd5a0 msgr2=0x7f15800fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.283+0000 7f1584baf640 1 -- 172.21.15.33:0/2876137922 shutdown_connections 2024-04-02T13:49:08.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.283+0000 7f1584baf640 1 -- 172.21.15.33:0/2876137922 wait complete. 2024-04-02T13:49:08.296 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:49:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.398+0000 7f810a7f6640 1 -- 172.21.15.33:0/3317697363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8104100970 msgr2=0x7f8104100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.398+0000 7f810a7f6640 1 --2- 172.21.15.33:0/3317697363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8104100970 0x7f8104100df0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009930 tx=0x7f80f802bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.399+0000 7f810a7f6640 1 -- 172.21.15.33:0/3317697363 shutdown_connections 2024-04-02T13:49:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.399+0000 7f810a7f6640 1 --2- 172.21.15.33:0/3317697363 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8104101330 0x7f810410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.399+0000 7f810a7f6640 1 --2- 172.21.15.33:0/3317697363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8104100970 0x7f8104100df0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.399+0000 7f810a7f6640 1 --2- 172.21.15.33:0/3317697363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8104106980 0x7f8104106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.399+0000 7f810a7f6640 1 -- 172.21.15.33:0/3317697363 >> 172.21.15.33:0/3317697363 conn(0x7f8104077b00 msgr2=0x7f8104077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.399+0000 7f810a7f6640 1 -- 172.21.15.33:0/3317697363 shutdown_connections 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.399+0000 7f810a7f6640 1 -- 172.21.15.33:0/3317697363 wait complete. 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.399+0000 7f810a7f6640 1 Processor -- start 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.399+0000 7f810a7f6640 1 -- start start 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f810a7f6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8104100970 0x7f8104197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f810a7f6640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8104101330 0x7f81041977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f810a7f6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8104106980 0x7f810419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f810a7f6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8104110d60 con 0x7f8104106980 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f810a7f6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8104110be0 con 0x7f8104100970 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f810a7f6640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8104110ee0 con 0x7f8104101330 2024-04-02T13:49:08.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f8108d93640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8104106980 0x7f810419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f8108d93640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8104106980 0x7f810419bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49386/0 (socket says 172.21.15.33:49386) 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f8108d93640 1 -- 172.21.15.33:0/2966953058 learned_addr learned my addr 172.21.15.33:0/2966953058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f8103fff640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8104100970 0x7f8104197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f81037fe640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8104101330 0x7f81041977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f8108d93640 1 -- 172.21.15.33:0/2966953058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8104101330 msgr2=0x7f81041977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f8108d93640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8104101330 0x7f81041977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f8108d93640 1 -- 172.21.15.33:0/2966953058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8104100970 msgr2=0x7f8104197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f8108d93640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8104100970 0x7f8104197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f8108d93640 1 -- 172.21.15.33:0/2966953058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810419c250 con 0x7f8104106980 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f8108d93640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8104106980 0x7f810419bb20 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00eb10 tx=0x7f80fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f81017fa640 1 -- 172.21.15.33:0/2966953058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc004b20 con 0x7f8104106980 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f81017fa640 1 -- 172.21.15.33:0/2966953058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f80fc004cc0 con 0x7f8104106980 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f810a7f6640 1 -- 172.21.15.33:0/2966953058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810419c540 con 0x7f8104106980 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f81017fa640 1 -- 172.21.15.33:0/2966953058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc010700 con 0x7f8104106980 2024-04-02T13:49:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.400+0000 7f810a7f6640 1 -- 172.21.15.33:0/2966953058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81041a3dd0 con 0x7f8104106980 2024-04-02T13:49:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f81017fa640 1 -- 172.21.15.33:0/2966953058 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f80fc004e70 con 0x7f8104106980 2024-04-02T13:49:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f810d3da740 1 Processor -- start 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f810d3da740 1 -- start start 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f810d3da740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd945eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f810d3da740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd945f0c0 0x555dd9467760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f810d3da740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd9467ca0 0x555dd946a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f810d3da740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555dd93c27b0 con 0x555dd945e7a0 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f810d3da740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555dd9361570 con 0x555dd945f0c0 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f810d3da740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555dd93be760 con 0x555dd9467ca0 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f81017fa640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f80ec0a3d20 0x7f80ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f81037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd945eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f81037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd945eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49388/0 (socket says 172.21.15.33:49388) 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f81037fe640 1 -- 172.21.15.33:0/3754482087 learned_addr learned my addr 172.21.15.33:0/3754482087 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f8108d93640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f80ec0a3d20 0x7f80ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f81037fe640 1 --2- 172.21.15.33:0/3754482087 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd9467ca0 0x555dd946a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f8103fff640 1 --2- 172.21.15.33:0/3754482087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd945f0c0 0x555dd9467760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.403+0000 7f81017fa640 1 -- 172.21.15.33:0/2966953058 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f80fc0d1150 con 0x7f8104106980 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.404+0000 7f81037fe640 1 -- 172.21.15.33:0/3754482087 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd9467ca0 msgr2=0x555dd946a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.404+0000 7f81037fe640 1 --2- 172.21.15.33:0/3754482087 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd9467ca0 0x555dd946a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.404+0000 7f81037fe640 1 -- 172.21.15.33:0/3754482087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd945f0c0 msgr2=0x555dd9467760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.404+0000 7f81037fe640 1 --2- 172.21.15.33:0/3754482087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd945f0c0 0x555dd9467760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.404+0000 7f81037fe640 1 -- 172.21.15.33:0/3754482087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dd90d63f0 con 0x555dd945e7a0 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.404+0000 7f81037fe640 1 --2- 172.21.15.33:0/3754482087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd945eb80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f80f8005ab0 tx=0x7f80f8005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7477dde001318276 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.404+0000 7f8108d93640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f80ec0a3d20 0x7f80ec0a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80fc099130 tx=0x7f80fc000f40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.404+0000 7f80ea7fc640 1 -- 172.21.15.33:0/3754482087 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8047020 con 0x555dd945e7a0 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.404+0000 7f80ea7fc640 1 -- 172.21.15.33:0/3754482087 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f80f8005510 con 0x555dd945e7a0 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.404+0000 7f80ea7fc640 1 -- 172.21.15.33:0/3754482087 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f802cc90 con 0x555dd945e7a0 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 -- 172.21.15.33:0/3754482087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 msgr2=0x555dd945eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 --2- 172.21.15.33:0/3754482087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd945eb80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f80f8005ab0 tx=0x7f80f8005df0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 -- 172.21.15.33:0/3754482087 shutdown_connections 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 --2- 172.21.15.33:0/3754482087 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd9467ca0 0x555dd946a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 --2- 172.21.15.33:0/3754482087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd945f0c0 0x555dd9467760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 --2- 172.21.15.33:0/3754482087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd945eb80 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 -- 172.21.15.33:0/3754482087 >> 172.21.15.33:0/3754482087 conn(0x555dd9458ed0 msgr2=0x555dd94592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 -- 172.21.15.33:0/3754482087 shutdown_connections 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 -- 172.21.15.33:0/3754482087 wait complete. 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 -- start start 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 Processor -- start 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd945e7a0 0x555dd946ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945f0c0 0x555dd946b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd9467ca0 0x555dd946b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555dd946aa30 con 0x555dd945f0c0 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555dd937de80 con 0x555dd9467ca0 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.405+0000 7f810d3da740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555dd946abb0 con 0x555dd945e7a0 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f810d3da740 10 client.0 _ll_register_callbacks cb 0x555dd93b9aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f8103fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945f0c0 0x555dd946b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f8103fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945f0c0 0x555dd946b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49404/0 (socket says 172.21.15.33:49404) 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f8103fff640 1 -- 172.21.15.33:0/1491401 learned_addr learned my addr 172.21.15.33:0/1491401 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f81037fe640 1 --2- 172.21.15.33:0/1491401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd9467ca0 0x555dd946b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f81037fe640 1 --2- 172.21.15.33:0/1491401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd945e7a0 0x555dd946ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f8103fff640 1 -- 172.21.15.33:0/1491401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd945e7a0 msgr2=0x555dd946ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f8103fff640 1 --2- 172.21.15.33:0/1491401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd945e7a0 0x555dd946ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f8103fff640 1 -- 172.21.15.33:0/1491401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd9467ca0 msgr2=0x555dd946b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f8103fff640 1 --2- 172.21.15.33:0/1491401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd9467ca0 0x555dd946b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f8103fff640 1 -- 172.21.15.33:0/1491401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dd91a07d0 con 0x555dd945f0c0 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f8103fff640 1 --2- 172.21.15.33:0/1491401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945f0c0 0x555dd946b3b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f80f4012700 tx=0x7f80f4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd0418985bcff83 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f80e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f80e37fe640 1 -- 172.21.15.33:0/1491401 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4004050 con 0x555dd945f0c0 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f80e37fe640 1 -- 172.21.15.33:0/1491401 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f80f40041f0 con 0x555dd945f0c0 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f810d3da740 1 -- 172.21.15.33:0/1491401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f8005510 con 0x555dd945f0c0 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f80e37fe640 1 -- 172.21.15.33:0/1491401 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f80f4025050 con 0x555dd945f0c0 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f810d3da740 10 client.17865 fetch_fsmap learned FSMap version 36 2024-04-02T13:49:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.406+0000 7f810d3da740 1 -- 172.21.15.33:0/1491401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555dd9478d80 con 0x555dd945f0c0 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.407+0000 7f80e37fe640 1 -- 172.21.15.33:0/1491401 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f80f8005510 con 0x555dd945f0c0 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.407+0000 7f80e37fe640 1 -- 172.21.15.33:0/1491401 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f80f40039c0 con 0x555dd945f0c0 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.407+0000 7f810d3da740 10 client.17865 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.407+0000 7f810d3da740 10 client.17865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.407+0000 7f810d3da740 20 client.17865 populate_metadata read hostname 'smithi033' 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.407+0000 7f810d3da740 1 --2- 172.21.15.33:0/1491401 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x555dd947a290 0x555dd947c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.407+0000 7f810d3da740 4 client.17865 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.407+0000 7f810d3da740 1 -- 172.21.15.33:0/1491401 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dd93cfcf0 con 0x555dd947a290 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.407+0000 7f81037fe640 1 --2- 172.21.15.33:0/1491401 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x555dd947a290 0x555dd947c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.407+0000 7f80e37fe640 10 client.17865 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.407+0000 7f81037fe640 1 --2- 172.21.15.33:0/1491401 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x555dd947a290 0x555dd947c6a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8bacde8fd683c8 server_cookie=17b6f6277ad9b7e5 in_seq=0 out_seq=0 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f80e37fe640 1 -- 172.21.15.33:0/1491401 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555dd93cfcf0 con 0x555dd947a290 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f80e37fe640 10 client.17865 handle_command_reply: tid=0 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f810d3da740 1 client.17865 shutdown 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f810d3da740 1 -- 172.21.15.33:0/1491401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945f0c0 msgr2=0x555dd946b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f810d3da740 1 --2- 172.21.15.33:0/1491401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945f0c0 0x555dd946b3b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f80f4012700 tx=0x7f80f4012bc0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f810d3da740 1 -- 172.21.15.33:0/1491401 shutdown_connections 2024-04-02T13:49:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f810d3da740 1 --2- 172.21.15.33:0/1491401 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x555dd947a290 0x555dd947c6a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f810d3da740 1 --2- 172.21.15.33:0/1491401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd9467ca0 0x555dd946b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f810d3da740 1 --2- 172.21.15.33:0/1491401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945f0c0 0x555dd946b3b0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f80e37fe640 0 client.17865 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:49:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f810d3da740 1 --2- 172.21.15.33:0/1491401 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd945e7a0 0x555dd946ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f810d3da740 1 -- 172.21.15.33:0/1491401 >> 172.21.15.33:0/1491401 conn(0x555dd9458ed0 msgr2=0x555dd945ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f810d3da740 1 -- 172.21.15.33:0/1491401 shutdown_connections 2024-04-02T13:49:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.408+0000 7f810d3da740 1 -- 172.21.15.33:0/1491401 wait complete. 2024-04-02T13:49:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.409+0000 7f810d3da740 20 client.17865 trim_cache size 0 max 16384 2024-04-02T13:49:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f810d3da740 1 Processor -- start 2024-04-02T13:49:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f810d3da740 1 -- start start 2024-04-02T13:49:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f810d3da740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd9475930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f810d3da740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd945f0c0 0x555dd9481fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f810d3da740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd9467ca0 0x555dd94824f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f810d3da740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81041af760 con 0x555dd945e7a0 2024-04-02T13:49:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f810d3da740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f81041ad030 con 0x555dd945f0c0 2024-04-02T13:49:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f810d3da740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81041aae80 con 0x555dd9467ca0 2024-04-02T13:49:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f81037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd9475930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f81037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd9475930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49406/0 (socket says 172.21.15.33:49406) 2024-04-02T13:49:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f81037fe640 1 -- 172.21.15.33:0/2098249956 learned_addr learned my addr 172.21.15.33:0/2098249956 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f81037fe640 1 -- 172.21.15.33:0/2098249956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd9467ca0 msgr2=0x555dd94824f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f81037fe640 1 --2- 172.21.15.33:0/2098249956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd9467ca0 0x555dd94824f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f81037fe640 1 -- 172.21.15.33:0/2098249956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd945f0c0 msgr2=0x555dd9481fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f81037fe640 1 --2- 172.21.15.33:0/2098249956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd945f0c0 0x555dd9481fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f81037fe640 1 -- 172.21.15.33:0/2098249956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dd9466530 con 0x555dd945e7a0 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f81037fe640 1 --2- 172.21.15.33:0/2098249956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd9475930 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f80f8036040 tx=0x7f80f8005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1b446531e279df server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f80e9ffb640 1 -- 172.21.15.33:0/2098249956 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8047040 con 0x555dd945e7a0 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f80e9ffb640 1 -- 172.21.15.33:0/2098249956 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f80f805a8b0 con 0x555dd945e7a0 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.422+0000 7f80e9ffb640 1 -- 172.21.15.33:0/2098249956 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f80055e0 con 0x555dd945e7a0 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.423+0000 7f810d3da740 1 -- 172.21.15.33:0/2098249956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 msgr2=0x555dd9475930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.423+0000 7f810d3da740 1 --2- 172.21.15.33:0/2098249956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd9475930 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f80f8036040 tx=0x7f80f8005df0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.423+0000 7f810d3da740 1 -- 172.21.15.33:0/2098249956 shutdown_connections 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.423+0000 7f810d3da740 1 --2- 172.21.15.33:0/2098249956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd9467ca0 0x555dd94824f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.423+0000 7f810d3da740 1 --2- 172.21.15.33:0/2098249956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd945f0c0 0x555dd9481fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.423+0000 7f810d3da740 1 --2- 172.21.15.33:0/2098249956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd9475930 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.423+0000 7f810d3da740 1 -- 172.21.15.33:0/2098249956 >> 172.21.15.33:0/2098249956 conn(0x555dd9458ed0 msgr2=0x555dd9461f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.423+0000 7f810d3da740 1 -- 172.21.15.33:0/2098249956 shutdown_connections 2024-04-02T13:49:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.423+0000 7f810d3da740 1 -- 172.21.15.33:0/2098249956 wait complete. 2024-04-02T13:49:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.423+0000 7f810d3da740 1 -- start start 2024-04-02T13:49:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f810d3da740 1 Processor -- start 2024-04-02T13:49:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f810d3da740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd946fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f810d3da740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd945f0c0 0x555dd946ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f810d3da740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd9467ca0 0x555dd946df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f810d3da740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81041ab4f0 con 0x555dd945e7a0 2024-04-02T13:49:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f810d3da740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f81041ac960 con 0x555dd9467ca0 2024-04-02T13:49:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f810d3da740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81041abbc0 con 0x555dd945f0c0 2024-04-02T13:49:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f810d3da740 10 client.0 _ll_register_callbacks cb 0x7f80f40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f81037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd946fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f81037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd946fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49412/0 (socket says 172.21.15.33:49412) 2024-04-02T13:49:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f81037fe640 1 -- 172.21.15.33:0/3779264216 learned_addr learned my addr 172.21.15.33:0/3779264216 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f8103fff640 1 --2- 172.21.15.33:0/3779264216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd945f0c0 0x555dd946ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f81037fe640 1 -- 172.21.15.33:0/3779264216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd945f0c0 msgr2=0x555dd946ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f81037fe640 1 --2- 172.21.15.33:0/3779264216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd945f0c0 0x555dd946ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f81037fe640 1 -- 172.21.15.33:0/3779264216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd9467ca0 msgr2=0x555dd946df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f81037fe640 1 --2- 172.21.15.33:0/3779264216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd9467ca0 0x555dd946df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f81037fe640 1 -- 172.21.15.33:0/3779264216 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dd946e6b0 con 0x555dd945e7a0 2024-04-02T13:49:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f81037fe640 1 --2- 172.21.15.33:0/3779264216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd946fa60 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f80f802c170 tx=0x7f80f80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfd35d8a9c20747 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.424+0000 7f80e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-02T13:49:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f80e37fe640 1 -- 172.21.15.33:0/3779264216 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f805e020 con 0x555dd945e7a0 2024-04-02T13:49:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f80e37fe640 1 -- 172.21.15.33:0/3779264216 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f80f80054a0 con 0x555dd945e7a0 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f810d3da740 1 -- 172.21.15.33:0/3779264216 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f805a8b0 con 0x555dd945e7a0 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f80e37fe640 1 -- 172.21.15.33:0/3779264216 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f80f8047040 con 0x555dd945e7a0 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f810d3da740 10 client.17871 fetch_fsmap learned FSMap version 36 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f810d3da740 1 -- 172.21.15.33:0/3779264216 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555dd94e1240 con 0x555dd945e7a0 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f80e37fe640 1 -- 172.21.15.33:0/3779264216 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f80f805a8b0 con 0x555dd945e7a0 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f80e37fe640 1 -- 172.21.15.33:0/3779264216 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f80f8059070 con 0x555dd945e7a0 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f810d3da740 10 client.17871 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f810d3da740 10 client.17871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f810d3da740 20 client.17871 populate_metadata read hostname 'smithi033' 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f810d3da740 1 --2- 172.21.15.33:0/3779264216 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x555dd94e1520 0x555dd94e3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f810d3da740 4 client.17871 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f810d3da740 1 -- 172.21.15.33:0/3779264216 --> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555dd93ca040 con 0x555dd94e1520 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.425+0000 7f8103fff640 1 --2- 172.21.15.33:0/3779264216 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x555dd94e1520 0x555dd94e3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f80e37fe640 10 client.17871 ms_handle_connect on v2:172.21.15.50:6838/932333028 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f8103fff640 1 --2- 172.21.15.33:0/3779264216 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x555dd94e1520 0x555dd94e3910 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=546e9bcd81b48f1b server_cookie=4ffcc81d797a49ac in_seq=0 out_seq=0 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f80e37fe640 1 -- 172.21.15.33:0/3779264216 <== mds.0 v2:172.21.15.50:6838/932333028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555dd93ca040 con 0x555dd94e1520 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f80e37fe640 10 client.17871 handle_command_reply: tid=0 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f810d3da740 1 client.17871 shutdown 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f810d3da740 1 -- 172.21.15.33:0/3779264216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 msgr2=0x555dd946fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f810d3da740 1 --2- 172.21.15.33:0/3779264216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd946fa60 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f80f802c170 tx=0x7f80f80058b0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f810d3da740 1 -- 172.21.15.33:0/3779264216 shutdown_connections 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f810d3da740 1 --2- 172.21.15.33:0/3779264216 >> [v2:172.21.15.50:6838/932333028,v1:172.21.15.50:6839/932333028] conn(0x555dd94e1520 0x555dd94e3910 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f810d3da740 1 --2- 172.21.15.33:0/3779264216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555dd9467ca0 0x555dd946df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f810d3da740 1 --2- 172.21.15.33:0/3779264216 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555dd945f0c0 0x555dd946ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f80e37fe640 0 client.17871 ms_handle_reset on v2:172.21.15.50:6838/932333028 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f810d3da740 1 --2- 172.21.15.33:0/3779264216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555dd945e7a0 0x555dd946fa60 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f810d3da740 1 -- 172.21.15.33:0/3779264216 >> 172.21.15.33:0/3779264216 conn(0x555dd9458ed0 msgr2=0x555dd945a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f810d3da740 1 -- 172.21.15.33:0/3779264216 shutdown_connections 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f810d3da740 1 -- 172.21.15.33:0/3779264216 wait complete. 2024-04-02T13:49:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.426+0000 7f810d3da740 20 client.17871 trim_cache size 0 max 16384 2024-04-02T13:49:08.436 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-04-02T13:49:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.427+0000 7f810a7f6640 1 -- 172.21.15.33:0/2966953058 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f80ec0a3d20 msgr2=0x7f80ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.427+0000 7f810a7f6640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f80ec0a3d20 0x7f80ec0a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80fc099130 tx=0x7f80fc000f40 comp rx=0 tx=0).stop 2024-04-02T13:49:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.427+0000 7f810a7f6640 1 -- 172.21.15.33:0/2966953058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8104106980 msgr2=0x7f810419bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.427+0000 7f810a7f6640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8104106980 0x7f810419bb20 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00eb10 tx=0x7f80fc00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.427+0000 7f810a7f6640 1 -- 172.21.15.33:0/2966953058 shutdown_connections 2024-04-02T13:49:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.427+0000 7f810a7f6640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f80ec0a3d20 0x7f80ec0a61e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.427+0000 7f810a7f6640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8104106980 0x7f810419bb20 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.427+0000 7f810a7f6640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8104101330 0x7f81041977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.427+0000 7f810a7f6640 1 --2- 172.21.15.33:0/2966953058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8104100970 0x7f8104197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.427+0000 7f810a7f6640 1 -- 172.21.15.33:0/2966953058 >> 172.21.15.33:0/2966953058 conn(0x7f8104077b00 msgr2=0x7f8104103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.427+0000 7f810a7f6640 1 -- 172.21.15.33:0/2966953058 shutdown_connections 2024-04-02T13:49:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:08.427+0000 7f810a7f6640 1 -- 172.21.15.33:0/2966953058 wait complete. 2024-04-02T13:49:08.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:08 smithi145 ceph-mon[35646]: pgmap v322: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-02T13:49:08.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:08 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3576546103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:49:08.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:08 smithi050 ceph-mon[35524]: pgmap v322: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-02T13:49:08.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:08 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3576546103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:49:08.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:08 smithi033 ceph-mon[32390]: pgmap v322: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-02T13:49:08.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:08 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3576546103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:49:09.441 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-02T13:49:09.441 INFO:tasks.fwd_scrub:done joining 2024-04-02T13:49:09.441 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-02T13:49:09.452 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:49:09.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.567+0000 7f71997fa640 1 -- 172.21.15.33:0/1560096076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194105a00 msgr2=0x7f7194105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:09.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.567+0000 7f71997fa640 1 --2- 172.21.15.33:0/1560096076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194105a00 0x7f7194105de0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f7184009930 tx=0x7f718402bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.567+0000 7f71997fa640 1 -- 172.21.15.33:0/1560096076 shutdown_connections 2024-04-02T13:49:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.567+0000 7f71997fa640 1 --2- 172.21.15.33:0/1560096076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7194101f30 0x7f719410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.567+0000 7f71997fa640 1 --2- 172.21.15.33:0/1560096076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7194101570 0x7f71941019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.567+0000 7f71997fa640 1 --2- 172.21.15.33:0/1560096076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194105a00 0x7f7194105de0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.567+0000 7f71997fa640 1 -- 172.21.15.33:0/1560096076 >> 172.21.15.33:0/1560096076 conn(0x7f7194076b60 msgr2=0x7f71940fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.567+0000 7f71997fa640 1 -- 172.21.15.33:0/1560096076 shutdown_connections 2024-04-02T13:49:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.567+0000 7f71997fa640 1 -- 172.21.15.33:0/1560096076 wait complete. 2024-04-02T13:49:09.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.568+0000 7f71997fa640 1 Processor -- start 2024-04-02T13:49:09.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.568+0000 7f71997fa640 1 -- start start 2024-04-02T13:49:09.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.568+0000 7f71997fa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194101570 0x7f71940fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:09.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.568+0000 7f71997fa640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7194101f30 0x7f71940fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:09.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.568+0000 7f71997fa640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7194105a00 0x7f71940fca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:09.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.568+0000 7f71997fa640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7194110e20 con 0x7f7194101570 2024-04-02T13:49:09.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.568+0000 7f71997fa640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7194110ca0 con 0x7f7194101f30 2024-04-02T13:49:09.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.568+0000 7f71997fa640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7194110fa0 con 0x7f7194105a00 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f7192ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194101570 0x7f71940fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f7192ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194101570 0x7f71940fbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49426/0 (socket says 172.21.15.33:49426) 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f7192ffd640 1 -- 172.21.15.33:0/25795427 learned_addr learned my addr 172.21.15.33:0/25795427 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f71927fc640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7194101f30 0x7f71940fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f71937fe640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7194105a00 0x7f71940fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f7192ffd640 1 -- 172.21.15.33:0/25795427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7194105a00 msgr2=0x7f71940fca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f7192ffd640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7194105a00 0x7f71940fca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f7192ffd640 1 -- 172.21.15.33:0/25795427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7194101f30 msgr2=0x7f71940fc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f7192ffd640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7194101f30 0x7f71940fc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f7192ffd640 1 -- 172.21.15.33:0/25795427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71941a6940 con 0x7f7194101570 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f71927fc640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7194101f30 0x7f71940fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f7192ffd640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194101570 0x7f71940fbfa0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7184000c00 tx=0x7f71840045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f717bfff640 1 -- 172.21.15.33:0/25795427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184043020 con 0x7f7194101570 2024-04-02T13:49:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f717bfff640 1 -- 172.21.15.33:0/25795427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f718402caa0 con 0x7f7194101570 2024-04-02T13:49:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f71997fa640 1 -- 172.21.15.33:0/25795427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71941a6bd0 con 0x7f7194101570 2024-04-02T13:49:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f717bfff640 1 -- 172.21.15.33:0/25795427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184003c20 con 0x7f7194101570 2024-04-02T13:49:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.569+0000 7f71997fa640 1 -- 172.21.15.33:0/25795427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71941a7120 con 0x7f7194101570 2024-04-02T13:49:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.570+0000 7f71997fa640 1 -- 172.21.15.33:0/25795427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7194068d40 con 0x7f7194101570 2024-04-02T13:49:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.571+0000 7f717bfff640 1 -- 172.21.15.33:0/25795427 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7184046020 con 0x7f7194101570 2024-04-02T13:49:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.572+0000 7f717bfff640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f717c0a3d20 0x7f717c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.572+0000 7f71927fc640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f717c0a3d20 0x7f717c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.572+0000 7f717bfff640 1 -- 172.21.15.33:0/25795427 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f71840f27e0 con 0x7f7194101570 2024-04-02T13:49:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.572+0000 7f71927fc640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f717c0a3d20 0x7f717c0a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f71940fda30 tx=0x7f7188008040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.576+0000 7f717bfff640 1 -- 172.21.15.33:0/25795427 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f718404d070 con 0x7f7194101570 2024-04-02T13:49:09.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.766+0000 7f71997fa640 1 -- 172.21.15.33:0/25795427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71941a72f0 con 0x7f7194101570 2024-04-02T13:49:09.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.768+0000 7f717bfff640 1 -- 172.21.15.33:0/25795427 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+25317 (secure 0 0 0) 0x7f71840beb70 con 0x7f7194101570 2024-04-02T13:49:09.771 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:49:09.771 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":104,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:49:06.382749+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/80287983":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/395956530":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3947066882":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1212609914":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:49:09.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.770+0000 7f71997fa640 1 -- 172.21.15.33:0/25795427 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f717c0a3d20 msgr2=0x7f717c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:09.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.770+0000 7f71997fa640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f717c0a3d20 0x7f717c0a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f71940fda30 tx=0x7f7188008040 comp rx=0 tx=0).stop 2024-04-02T13:49:09.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.770+0000 7f71997fa640 1 -- 172.21.15.33:0/25795427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194101570 msgr2=0x7f71940fbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:09.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.770+0000 7f71997fa640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194101570 0x7f71940fbfa0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7184000c00 tx=0x7f71840045e0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.770+0000 7f71997fa640 1 -- 172.21.15.33:0/25795427 shutdown_connections 2024-04-02T13:49:09.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.770+0000 7f71997fa640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f717c0a3d20 0x7f717c0a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.770+0000 7f71997fa640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7194105a00 0x7f71940fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.770+0000 7f71997fa640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7194101f30 0x7f71940fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.770+0000 7f71997fa640 1 --2- 172.21.15.33:0/25795427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194101570 0x7f71940fbfa0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.770+0000 7f71997fa640 1 -- 172.21.15.33:0/25795427 >> 172.21.15.33:0/25795427 conn(0x7f7194076b60 msgr2=0x7f7194100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:09.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.771+0000 7f71997fa640 1 -- 172.21.15.33:0/25795427 shutdown_connections 2024-04-02T13:49:09.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.771+0000 7f71997fa640 1 -- 172.21.15.33:0/25795427 wait complete. 2024-04-02T13:49:09.785 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:36:19.501121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:36:20.161569+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:36:23.482134+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:49:09.785 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:49:09.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.888+0000 7f1f9c311640 1 -- 172.21.15.33:0/1140509839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f941069c0 msgr2=0x7f1f94106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:09.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.888+0000 7f1f9c311640 1 --2- 172.21.15.33:0/1140509839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f941069c0 0x7f1f94106da0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009930 tx=0x7f1f9002bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:09.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.888+0000 7f1f9c311640 1 -- 172.21.15.33:0/1140509839 shutdown_connections 2024-04-02T13:49:09.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.888+0000 7f1f9c311640 1 --2- 172.21.15.33:0/1140509839 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f94101370 0x7f1f9410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.888+0000 7f1f9c311640 1 --2- 172.21.15.33:0/1140509839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f941009b0 0x7f1f94100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.888+0000 7f1f9c311640 1 --2- 172.21.15.33:0/1140509839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f941069c0 0x7f1f94106da0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.888+0000 7f1f9c311640 1 -- 172.21.15.33:0/1140509839 >> 172.21.15.33:0/1140509839 conn(0x7f1f94076d80 msgr2=0x7f1f94077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.888+0000 7f1f9c311640 1 -- 172.21.15.33:0/1140509839 shutdown_connections 2024-04-02T13:49:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.888+0000 7f1f9c311640 1 -- 172.21.15.33:0/1140509839 wait complete. 2024-04-02T13:49:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f9c311640 1 Processor -- start 2024-04-02T13:49:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f9c311640 1 -- start start 2024-04-02T13:49:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f9c311640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f941009b0 0x7f1f94199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f9c311640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f94101370 0x7f1f94199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f9c311640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f941069c0 0x7f1f941a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f9c311640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94110d70 con 0x7f1f94101370 2024-04-02T13:49:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f9c311640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94110bf0 con 0x7f1f941069c0 2024-04-02T13:49:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f9c311640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94110ef0 con 0x7f1f941009b0 2024-04-02T13:49:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f998ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f94101370 0x7f1f94199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f998ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f94101370 0x7f1f94199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49450/0 (socket says 172.21.15.33:49450) 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f998ac640 1 -- 172.21.15.33:0/1735483015 learned_addr learned my addr 172.21.15.33:0/1735483015 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f9a0ad640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f941009b0 0x7f1f94199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f998ac640 1 -- 172.21.15.33:0/1735483015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f941009b0 msgr2=0x7f1f94199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f998ac640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f941009b0 0x7f1f94199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f9a8ae640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f941069c0 0x7f1f941a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f998ac640 1 -- 172.21.15.33:0/1735483015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f941069c0 msgr2=0x7f1f941a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.889+0000 7f1f998ac640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f941069c0 0x7f1f941a42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.890+0000 7f1f998ac640 1 -- 172.21.15.33:0/1735483015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f941a4810 con 0x7f1f94101370 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.890+0000 7f1f9a0ad640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f941009b0 0x7f1f94199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.890+0000 7f1f998ac640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f94101370 0x7f1f94199790 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400d950 tx=0x7f1f8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.890+0000 7f1f8b7fe640 1 -- 172.21.15.33:0/1735483015 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84017070 con 0x7f1f94101370 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.890+0000 7f1f9c311640 1 -- 172.21.15.33:0/1735483015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f941a49e0 con 0x7f1f94101370 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.890+0000 7f1f8b7fe640 1 -- 172.21.15.33:0/1735483015 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f1f84009c30 con 0x7f1f94101370 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.890+0000 7f1f8b7fe640 1 -- 172.21.15.33:0/1735483015 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8400f6b0 con 0x7f1f94101370 2024-04-02T13:49:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.890+0000 7f1f9c311640 1 -- 172.21.15.33:0/1735483015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f941a4f50 con 0x7f1f94101370 2024-04-02T13:49:09.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.891+0000 7f1f9c311640 1 -- 172.21.15.33:0/1735483015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f941020f0 con 0x7f1f94101370 2024-04-02T13:49:09.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.892+0000 7f1f8b7fe640 1 -- 172.21.15.33:0/1735483015 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1f84010a50 con 0x7f1f94101370 2024-04-02T13:49:09.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.892+0000 7f1f8b7fe640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1f800a3d20 0x7f1f800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:09.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.893+0000 7f1f9a0ad640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1f800a3d20 0x7f1f800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:09.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.893+0000 7f1f8b7fe640 1 -- 172.21.15.33:0/1735483015 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f1f840d08c0 con 0x7f1f94101370 2024-04-02T13:49:09.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.893+0000 7f1f9a0ad640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1f800a3d20 0x7f1f800a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f90036040 tx=0x7f1f90005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:09.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:09.897+0000 7f1f8b7fe640 1 -- 172.21.15.33:0/1735483015 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1f84009de0 con 0x7f1f94101370 2024-04-02T13:49:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.084+0000 7f1f9c311640 1 -- 172.21.15.33:0/1735483015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1f94108f20 con 0x7f1f94101370 2024-04-02T13:49:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.084+0000 7f1f8b7fe640 1 -- 172.21.15.33:0/1735483015 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) ==== 93+0+10 (secure 0 0 0) 0x7f1f8409cb40 con 0x7f1f94101370 2024-04-02T13:49:10.087 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:49:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.086+0000 7f1f9c311640 1 -- 172.21.15.33:0/1735483015 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1f800a3d20 msgr2=0x7f1f800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.086+0000 7f1f9c311640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1f800a3d20 0x7f1f800a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f90036040 tx=0x7f1f90005fd0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.086+0000 7f1f9c311640 1 -- 172.21.15.33:0/1735483015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f94101370 msgr2=0x7f1f94199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.086+0000 7f1f9c311640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f94101370 0x7f1f94199790 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400d950 tx=0x7f1f8400de10 comp rx=0 tx=0).stop 2024-04-02T13:49:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.086+0000 7f1f9c311640 1 -- 172.21.15.33:0/1735483015 shutdown_connections 2024-04-02T13:49:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.086+0000 7f1f9c311640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f1f800a3d20 0x7f1f800a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.086+0000 7f1f9c311640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f941069c0 0x7f1f941a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.086+0000 7f1f9c311640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f94101370 0x7f1f94199790 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.086+0000 7f1f9c311640 1 --2- 172.21.15.33:0/1735483015 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f941009b0 0x7f1f94199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.086+0000 7f1f9c311640 1 -- 172.21.15.33:0/1735483015 >> 172.21.15.33:0/1735483015 conn(0x7f1f94076d80 msgr2=0x7f1f940fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.086+0000 7f1f9c311640 1 -- 172.21.15.33:0/1735483015 shutdown_connections 2024-04-02T13:49:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.086+0000 7f1f9c311640 1 -- 172.21.15.33:0/1735483015 wait complete. 2024-04-02T13:49:10.100 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:49:10.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.203+0000 7f5beab9d640 1 -- 172.21.15.33:0/1112597206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be41009b0 msgr2=0x7f5be4100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.203+0000 7f5beab9d640 1 --2- 172.21.15.33:0/1112597206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be41009b0 0x7f5be4100e30 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009930 tx=0x7f5bd002bc20 comp rx=0 tx=0).stop 2024-04-02T13:49:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.203+0000 7f5beab9d640 1 -- 172.21.15.33:0/1112597206 shutdown_connections 2024-04-02T13:49:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.203+0000 7f5beab9d640 1 --2- 172.21.15.33:0/1112597206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be4101370 0x7f5be410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.203+0000 7f5beab9d640 1 --2- 172.21.15.33:0/1112597206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be41009b0 0x7f5be4100e30 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.203+0000 7f5beab9d640 1 --2- 172.21.15.33:0/1112597206 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5be41069c0 0x7f5be4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.203+0000 7f5beab9d640 1 -- 172.21.15.33:0/1112597206 >> 172.21.15.33:0/1112597206 conn(0x7f5be4076d80 msgr2=0x7f5be4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.203+0000 7f5beab9d640 1 -- 172.21.15.33:0/1112597206 shutdown_connections 2024-04-02T13:49:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.203+0000 7f5beab9d640 1 -- 172.21.15.33:0/1112597206 wait complete. 2024-04-02T13:49:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5beab9d640 1 Processor -- start 2024-04-02T13:49:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5beab9d640 1 -- start start 2024-04-02T13:49:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5beab9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be41009b0 0x7f5be4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5beab9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be4101370 0x7f5be4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:10.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5beab9d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5be41069c0 0x7f5be419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5beab9d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4110d70 con 0x7f5be41009b0 2024-04-02T13:49:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5beab9d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4110bf0 con 0x7f5be4101370 2024-04-02T13:49:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5beab9d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4110ef0 con 0x7f5be41069c0 2024-04-02T13:49:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5be8939640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be41009b0 0x7f5be4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5be8939640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be41009b0 0x7f5be4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49470/0 (socket says 172.21.15.33:49470) 2024-04-02T13:49:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5be8939640 1 -- 172.21.15.33:0/3059266473 learned_addr learned my addr 172.21.15.33:0/3059266473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5be913a640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5be41069c0 0x7f5be419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5be3fff640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be4101370 0x7f5be4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5be8939640 1 -- 172.21.15.33:0/3059266473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5be41069c0 msgr2=0x7f5be419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5be8939640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5be41069c0 0x7f5be419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5be8939640 1 -- 172.21.15.33:0/3059266473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be4101370 msgr2=0x7f5be4198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5be8939640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be4101370 0x7f5be4198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5be8939640 1 -- 172.21.15.33:0/3059266473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be419d2f0 con 0x7f5be41009b0 2024-04-02T13:49:10.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5be913a640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5be41069c0 0x7f5be419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:10.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5be8939640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be41009b0 0x7f5be4198300 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800c920 tx=0x7f5bd800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:10.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.204+0000 7f5be1ffb640 1 -- 172.21.15.33:0/3059266473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8007c70 con 0x7f5be41009b0 2024-04-02T13:49:10.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.205+0000 7f5be1ffb640 1 -- 172.21.15.33:0/3059266473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f5bd8007e10 con 0x7f5be41009b0 2024-04-02T13:49:10.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.205+0000 7f5beab9d640 1 -- 172.21.15.33:0/3059266473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be419d5e0 con 0x7f5be41009b0 2024-04-02T13:49:10.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.205+0000 7f5be1ffb640 1 -- 172.21.15.33:0/3059266473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8018850 con 0x7f5be41009b0 2024-04-02T13:49:10.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.205+0000 7f5beab9d640 1 -- 172.21.15.33:0/3059266473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be41a4e70 con 0x7f5be41009b0 2024-04-02T13:49:10.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.206+0000 7f5beab9d640 1 -- 172.21.15.33:0/3059266473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be41020f0 con 0x7f5be41009b0 2024-04-02T13:49:10.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.207+0000 7f5be1ffb640 1 -- 172.21.15.33:0/3059266473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5bd8018a30 con 0x7f5be41009b0 2024-04-02T13:49:10.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.208+0000 7f5be1ffb640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5bd40a3d20 0x7f5bd40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:10.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.208+0000 7f5be3fff640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5bd40a3d20 0x7f5bd40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:10.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.208+0000 7f5be1ffb640 1 -- 172.21.15.33:0/3059266473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f5bd80d0dd0 con 0x7f5be41009b0 2024-04-02T13:49:10.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.208+0000 7f5be3fff640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5bd40a3d20 0x7f5bd40a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0036040 tx=0x7f5bd0005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:10.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.212+0000 7f5be1ffb640 1 -- 172.21.15.33:0/3059266473 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5bd8014030 con 0x7f5be41009b0 2024-04-02T13:49:10.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.396+0000 7f5beab9d640 1 -- 172.21.15.33:0/3059266473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5be402d370 con 0x7f5be41009b0 2024-04-02T13:49:10.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.397+0000 7f5be1ffb640 1 -- 172.21.15.33:0/3059266473 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) ==== 104+0+11 (secure 0 0 0) 0x7f5bd800de20 con 0x7f5be41009b0 2024-04-02T13:49:10.400 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:49:10.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.399+0000 7f5beab9d640 1 -- 172.21.15.33:0/3059266473 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5bd40a3d20 msgr2=0x7f5bd40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.399+0000 7f5beab9d640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5bd40a3d20 0x7f5bd40a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0036040 tx=0x7f5bd0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.399+0000 7f5beab9d640 1 -- 172.21.15.33:0/3059266473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be41009b0 msgr2=0x7f5be4198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.399+0000 7f5beab9d640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be41009b0 0x7f5be4198300 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800c920 tx=0x7f5bd800cde0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.399+0000 7f5beab9d640 1 -- 172.21.15.33:0/3059266473 shutdown_connections 2024-04-02T13:49:10.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.399+0000 7f5beab9d640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f5bd40a3d20 0x7f5bd40a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.399+0000 7f5beab9d640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5be41069c0 0x7f5be419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.399+0000 7f5beab9d640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be4101370 0x7f5be4198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.399+0000 7f5beab9d640 1 --2- 172.21.15.33:0/3059266473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5be41009b0 0x7f5be4198300 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.399+0000 7f5beab9d640 1 -- 172.21.15.33:0/3059266473 >> 172.21.15.33:0/3059266473 conn(0x7f5be4076d80 msgr2=0x7f5be406a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:10.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.399+0000 7f5beab9d640 1 -- 172.21.15.33:0/3059266473 shutdown_connections 2024-04-02T13:49:10.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.399+0000 7f5beab9d640 1 -- 172.21.15.33:0/3059266473 wait complete. 2024-04-02T13:49:10.412 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:49:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.515+0000 7f03d38d3640 1 -- 172.21.15.33:0/798901332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc107aa0 msgr2=0x7f03cc107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.515+0000 7f03d38d3640 1 --2- 172.21.15.33:0/798901332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc107aa0 0x7f03cc107e80 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009cd0 tx=0x7f03c402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.515+0000 7f03d38d3640 1 -- 172.21.15.33:0/798901332 shutdown_connections 2024-04-02T13:49:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.515+0000 7f03d38d3640 1 --2- 172.21.15.33:0/798901332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03cc102450 0x7f03cc10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.515+0000 7f03d38d3640 1 --2- 172.21.15.33:0/798901332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03cc101a90 0x7f03cc101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.515+0000 7f03d38d3640 1 --2- 172.21.15.33:0/798901332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc107aa0 0x7f03cc107e80 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.515+0000 7f03d38d3640 1 -- 172.21.15.33:0/798901332 >> 172.21.15.33:0/798901332 conn(0x7f03cc078f00 msgr2=0x7f03cc079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.515+0000 7f03d38d3640 1 -- 172.21.15.33:0/798901332 shutdown_connections 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.515+0000 7f03d38d3640 1 -- 172.21.15.33:0/798901332 wait complete. 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.515+0000 7f03d38d3640 1 Processor -- start 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.515+0000 7f03d38d3640 1 -- start start 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d38d3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03cc101a90 0x7f03cc073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d38d3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03cc102450 0x7f03cc074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d38d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc107aa0 0x7f03cc074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d38d3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc111e80 con 0x7f03cc107aa0 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d38d3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc111d00 con 0x7f03cc102450 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d38d3640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc112000 con 0x7f03cc101a90 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d1e70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc107aa0 0x7f03cc074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d1e70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc107aa0 0x7f03cc074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49498/0 (socket says 172.21.15.33:49498) 2024-04-02T13:49:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d1e70640 1 -- 172.21.15.33:0/1791579630 learned_addr learned my addr 172.21.15.33:0/1791579630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d0e6e640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03cc102450 0x7f03cc074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d166f640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03cc101a90 0x7f03cc073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d1e70640 1 -- 172.21.15.33:0/1791579630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03cc101a90 msgr2=0x7f03cc073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d1e70640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03cc101a90 0x7f03cc073e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d1e70640 1 -- 172.21.15.33:0/1791579630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03cc102450 msgr2=0x7f03cc074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d1e70640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03cc102450 0x7f03cc074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d1e70640 1 -- 172.21.15.33:0/1791579630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc1a68a0 con 0x7f03cc107aa0 2024-04-02T13:49:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d1e70640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc107aa0 0x7f03cc074880 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00d750 tx=0x7f03bc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03ca7fc640 1 -- 172.21.15.33:0/1791579630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc017070 con 0x7f03cc107aa0 2024-04-02T13:49:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03ca7fc640 1 -- 172.21.15.33:0/1791579630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f03bc0054f0 con 0x7f03cc107aa0 2024-04-02T13:49:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d38d3640 1 -- 172.21.15.33:0/1791579630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03cc1a6b90 con 0x7f03cc107aa0 2024-04-02T13:49:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03ca7fc640 1 -- 172.21.15.33:0/1791579630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc004cf0 con 0x7f03cc107aa0 2024-04-02T13:49:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.516+0000 7f03d38d3640 1 -- 172.21.15.33:0/1791579630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03cc1a70e0 con 0x7f03cc107aa0 2024-04-02T13:49:10.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.517+0000 7f03d38d3640 1 -- 172.21.15.33:0/1791579630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03cc068d40 con 0x7f03cc107aa0 2024-04-02T13:49:10.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.518+0000 7f03ca7fc640 1 -- 172.21.15.33:0/1791579630 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f03bc004e90 con 0x7f03cc107aa0 2024-04-02T13:49:10.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.519+0000 7f03ca7fc640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f03b40a3d20 0x7f03b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:10.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.519+0000 7f03d166f640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f03b40a3d20 0x7f03b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:10.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.519+0000 7f03ca7fc640 1 -- 172.21.15.33:0/1791579630 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7f03bc0d04c0 con 0x7f03cc107aa0 2024-04-02T13:49:10.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.519+0000 7f03d166f640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f03b40a3d20 0x7f03b40a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f03c4005cc0 tx=0x7f03c4005c50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:10.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.524+0000 7f03ca7fc640 1 -- 172.21.15.33:0/1791579630 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f03bc09c0f0 con 0x7f03cc107aa0 2024-04-02T13:49:10.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.709+0000 7f03d38d3640 1 -- 172.21.15.33:0/1791579630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f03cc10a000 con 0x7f03cc107aa0 2024-04-02T13:49:10.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.710+0000 7f03ca7fc640 1 -- 172.21.15.33:0/1791579630 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) ==== 100+0+11 (secure 0 0 0) 0x7f03bc0a03f0 con 0x7f03cc107aa0 2024-04-02T13:49:10.713 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-02T13:49:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.712+0000 7f03d38d3640 1 -- 172.21.15.33:0/1791579630 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f03b40a3d20 msgr2=0x7f03b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.712+0000 7f03d38d3640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f03b40a3d20 0x7f03b40a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f03c4005cc0 tx=0x7f03c4005c50 comp rx=0 tx=0).stop 2024-04-02T13:49:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.712+0000 7f03d38d3640 1 -- 172.21.15.33:0/1791579630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc107aa0 msgr2=0x7f03cc074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.712+0000 7f03d38d3640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc107aa0 0x7f03cc074880 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00d750 tx=0x7f03bc00dc10 comp rx=0 tx=0).stop 2024-04-02T13:49:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.712+0000 7f03d38d3640 1 -- 172.21.15.33:0/1791579630 shutdown_connections 2024-04-02T13:49:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.712+0000 7f03d38d3640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f03b40a3d20 0x7f03b40a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.712+0000 7f03d38d3640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc107aa0 0x7f03cc074880 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.712+0000 7f03d38d3640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03cc102450 0x7f03cc074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.712+0000 7f03d38d3640 1 --2- 172.21.15.33:0/1791579630 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03cc101a90 0x7f03cc073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.712+0000 7f03d38d3640 1 -- 172.21.15.33:0/1791579630 >> 172.21.15.33:0/1791579630 conn(0x7f03cc078f00 msgr2=0x7f03cc104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.712+0000 7f03d38d3640 1 -- 172.21.15.33:0/1791579630 shutdown_connections 2024-04-02T13:49:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.712+0000 7f03d38d3640 1 -- 172.21.15.33:0/1791579630 wait complete. 2024-04-02T13:49:10.724 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:10 smithi033 ceph-mon[32390]: pgmap v323: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:49:10.724 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:10 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/25795427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:49:10.724 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:10 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1735483015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:49:10.725 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:49:10.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:10 smithi145 ceph-mon[35646]: pgmap v323: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:49:10.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:10 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/25795427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:49:10.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:10 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1735483015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:49:10.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:10 smithi050 ceph-mon[35524]: pgmap v323: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:49:10.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:10 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/25795427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:49:10.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:10 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1735483015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:49:10.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.829+0000 7fa3896f2640 1 -- 172.21.15.33:0/700823299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3841013e0 msgr2=0x7fa38410d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.829+0000 7fa3896f2640 1 --2- 172.21.15.33:0/700823299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3841013e0 0x7fa38410d8f0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009cd0 tx=0x7fa37c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.829+0000 7fa3896f2640 1 -- 172.21.15.33:0/700823299 shutdown_connections 2024-04-02T13:49:10.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.829+0000 7fa3896f2640 1 --2- 172.21.15.33:0/700823299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3841013e0 0x7fa38410d8f0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.829+0000 7fa3896f2640 1 --2- 172.21.15.33:0/700823299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa384100a20 0x7fa384100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.829+0000 7fa3896f2640 1 --2- 172.21.15.33:0/700823299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa384106a30 0x7fa384106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.829+0000 7fa3896f2640 1 -- 172.21.15.33:0/700823299 >> 172.21.15.33:0/700823299 conn(0x7fa384077b60 msgr2=0x7fa384077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:10.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.829+0000 7fa3896f2640 1 -- 172.21.15.33:0/700823299 shutdown_connections 2024-04-02T13:49:10.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.829+0000 7fa3896f2640 1 -- 172.21.15.33:0/700823299 wait complete. 2024-04-02T13:49:10.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3896f2640 1 Processor -- start 2024-04-02T13:49:10.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3896f2640 1 -- start start 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3896f2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa384100a20 0x7fa3840fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3896f2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3841013e0 0x7fa3840fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3896f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa384106a30 0x7fa3840fe420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3896f2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa384110e10 con 0x7fa384106a30 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3896f2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa384110c90 con 0x7fa3841013e0 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3896f2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa384110f90 con 0x7fa384100a20 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa384106a30 0x7fa3840fe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa384106a30 0x7fa3840fe420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49508/0 (socket says 172.21.15.33:49508) 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3837fe640 1 -- 172.21.15.33:0/186031970 learned_addr learned my addr 172.21.15.33:0/186031970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3837fe640 1 -- 172.21.15.33:0/186031970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa384100a20 msgr2=0x7fa3840fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3837fe640 1 --2- 172.21.15.33:0/186031970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa384100a20 0x7fa3840fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa37bfff640 1 --2- 172.21.15.33:0/186031970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3841013e0 0x7fa3840fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3837fe640 1 -- 172.21.15.33:0/186031970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3841013e0 msgr2=0x7fa3840fdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:10.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3837fe640 1 --2- 172.21.15.33:0/186031970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3841013e0 0x7fa3840fdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:10.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3837fe640 1 -- 172.21.15.33:0/186031970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3841a6960 con 0x7fa384106a30 2024-04-02T13:49:10.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.830+0000 7fa3837fe640 1 --2- 172.21.15.33:0/186031970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa384106a30 0x7fa3840fe420 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa37c004af0 tx=0x7fa37c034d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:10.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.831+0000 7fa380ff9640 1 -- 172.21.15.33:0/186031970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c039050 con 0x7fa384106a30 2024-04-02T13:49:10.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.831+0000 7fa3896f2640 1 -- 172.21.15.33:0/186031970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3841a6bf0 con 0x7fa384106a30 2024-04-02T13:49:10.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.831+0000 7fa380ff9640 1 -- 172.21.15.33:0/186031970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa37c045070 con 0x7fa384106a30 2024-04-02T13:49:10.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.831+0000 7fa380ff9640 1 -- 172.21.15.33:0/186031970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0046b0 con 0x7fa384106a30 2024-04-02T13:49:10.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.831+0000 7fa3896f2640 1 -- 172.21.15.33:0/186031970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3841a7130 con 0x7fa384106a30 2024-04-02T13:49:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.832+0000 7fa3896f2640 1 -- 172.21.15.33:0/186031970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa384068d40 con 0x7fa384106a30 2024-04-02T13:49:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.833+0000 7fa380ff9640 1 -- 172.21.15.33:0/186031970 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa37c003970 con 0x7fa384106a30 2024-04-02T13:49:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.833+0000 7fa380ff9640 1 --2- 172.21.15.33:0/186031970 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3700a3d20 0x7fa3700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.834+0000 7fa382ffd640 1 --2- 172.21.15.33:0/186031970 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3700a3d20 0x7fa3700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.834+0000 7fa380ff9640 1 -- 172.21.15.33:0/186031970 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7fa37c0f40b0 con 0x7fa384106a30 2024-04-02T13:49:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.834+0000 7fa382ffd640 1 --2- 172.21.15.33:0/186031970 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3700a3d20 0x7fa3700a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa374005d80 tx=0x7fa374005c90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:10.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:10.837+0000 7fa380ff9640 1 -- 172.21.15.33:0/186031970 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa37c0c0440 con 0x7fa384106a30 2024-04-02T13:49:11.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.021+0000 7fa3896f2640 1 -- 172.21.15.33:0/186031970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa384109fa0 con 0x7fa384106a30 2024-04-02T13:49:11.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.022+0000 7fa380ff9640 1 -- 172.21.15.33:0/186031970 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v104) ==== 103+0+11 (secure 0 0 0) 0x7fa37c0bfd40 con 0x7fa384106a30 2024-04-02T13:49:11.025 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:49:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.024+0000 7fa3896f2640 1 -- 172.21.15.33:0/186031970 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3700a3d20 msgr2=0x7fa3700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.024+0000 7fa3896f2640 1 --2- 172.21.15.33:0/186031970 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3700a3d20 0x7fa3700a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa374005d80 tx=0x7fa374005c90 comp rx=0 tx=0).stop 2024-04-02T13:49:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.024+0000 7fa3896f2640 1 -- 172.21.15.33:0/186031970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa384106a30 msgr2=0x7fa3840fe420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.024+0000 7fa3896f2640 1 --2- 172.21.15.33:0/186031970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa384106a30 0x7fa3840fe420 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa37c004af0 tx=0x7fa37c034d70 comp rx=0 tx=0).stop 2024-04-02T13:49:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.024+0000 7fa3896f2640 1 -- 172.21.15.33:0/186031970 shutdown_connections 2024-04-02T13:49:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.024+0000 7fa3896f2640 1 --2- 172.21.15.33:0/186031970 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3700a3d20 0x7fa3700a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.024+0000 7fa3896f2640 1 --2- 172.21.15.33:0/186031970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa384106a30 0x7fa3840fe420 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.024+0000 7fa3896f2640 1 --2- 172.21.15.33:0/186031970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa3841013e0 0x7fa3840fdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:11.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.024+0000 7fa3896f2640 1 --2- 172.21.15.33:0/186031970 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa384100a20 0x7fa3840fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:11.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.024+0000 7fa3896f2640 1 -- 172.21.15.33:0/186031970 >> 172.21.15.33:0/186031970 conn(0x7fa384077b60 msgr2=0x7fa38406a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:11.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.024+0000 7fa3896f2640 1 -- 172.21.15.33:0/186031970 shutdown_connections 2024-04-02T13:49:11.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.024+0000 7fa3896f2640 1 -- 172.21.15.33:0/186031970 wait complete. 2024-04-02T13:49:11.037 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-02T13:49:11.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.138+0000 7fab8b1c8640 1 -- 172.21.15.33:0/4116919125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab84101350 msgr2=0x7fab8410d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:11.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.138+0000 7fab8b1c8640 1 --2- 172.21.15.33:0/4116919125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab84101350 0x7fab8410d860 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fab74009930 tx=0x7fab7402bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:11.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.138+0000 7fab8b1c8640 1 -- 172.21.15.33:0/4116919125 shutdown_connections 2024-04-02T13:49:11.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.138+0000 7fab8b1c8640 1 --2- 172.21.15.33:0/4116919125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab84101350 0x7fab8410d860 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:11.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.138+0000 7fab8b1c8640 1 --2- 172.21.15.33:0/4116919125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab84100990 0x7fab84100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:11.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.138+0000 7fab8b1c8640 1 --2- 172.21.15.33:0/4116919125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab841069a0 0x7fab84106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:11.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.138+0000 7fab8b1c8640 1 -- 172.21.15.33:0/4116919125 >> 172.21.15.33:0/4116919125 conn(0x7fab84077ad0 msgr2=0x7fab84077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:11.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.138+0000 7fab8b1c8640 1 -- 172.21.15.33:0/4116919125 shutdown_connections 2024-04-02T13:49:11.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.138+0000 7fab8b1c8640 1 -- 172.21.15.33:0/4116919125 wait complete. 2024-04-02T13:49:11.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.138+0000 7fab8b1c8640 1 Processor -- start 2024-04-02T13:49:11.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.138+0000 7fab8b1c8640 1 -- start start 2024-04-02T13:49:11.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab8b1c8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab84100990 0x7fab841983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:11.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab8b1c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab84101350 0x7fab841988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab8b1c8640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab841069a0 0x7fab8419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab8b1c8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab84110da0 con 0x7fab84101350 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab8b1c8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fab84110c20 con 0x7fab84100990 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab8b1c8640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fab84110f20 con 0x7fab841069a0 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab83fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab84101350 0x7fab841988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab83fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab84101350 0x7fab841988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49532/0 (socket says 172.21.15.33:49532) 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab83fff640 1 -- 172.21.15.33:0/3962850593 learned_addr learned my addr 172.21.15.33:0/3962850593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab88f64640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab84100990 0x7fab841983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab89765640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab841069a0 0x7fab8419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab83fff640 1 -- 172.21.15.33:0/3962850593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab841069a0 msgr2=0x7fab8419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab83fff640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab841069a0 0x7fab8419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab83fff640 1 -- 172.21.15.33:0/3962850593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab84100990 msgr2=0x7fab841983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab83fff640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab84100990 0x7fab841983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab83fff640 1 -- 172.21.15.33:0/3962850593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8419d390 con 0x7fab84101350 2024-04-02T13:49:11.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab83fff640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab84101350 0x7fab841988e0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fab7800b4a0 tx=0x7fab7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:11.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab81ffb640 1 -- 172.21.15.33:0/3962850593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78014070 con 0x7fab84101350 2024-04-02T13:49:11.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab8b1c8640 1 -- 172.21.15.33:0/3962850593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8419d680 con 0x7fab84101350 2024-04-02T13:49:11.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab81ffb640 1 -- 172.21.15.33:0/3962850593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fab78010ae0 con 0x7fab84101350 2024-04-02T13:49:11.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.139+0000 7fab81ffb640 1 -- 172.21.15.33:0/3962850593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78017c70 con 0x7fab84101350 2024-04-02T13:49:11.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.140+0000 7fab8b1c8640 1 -- 172.21.15.33:0/3962850593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab841a4f10 con 0x7fab84101350 2024-04-02T13:49:11.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.141+0000 7fab8b1c8640 1 -- 172.21.15.33:0/3962850593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab84068d40 con 0x7fab84101350 2024-04-02T13:49:11.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.142+0000 7fab81ffb640 1 -- 172.21.15.33:0/3962850593 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab78017070 con 0x7fab84101350 2024-04-02T13:49:11.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.142+0000 7fab81ffb640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fab6c0a3d20 0x7fab6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:11.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.143+0000 7fab88f64640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fab6c0a3d20 0x7fab6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:11.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.143+0000 7fab81ffb640 1 -- 172.21.15.33:0/3962850593 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9584+0+0 (secure 0 0 0) 0x7fab780cfe40 con 0x7fab84101350 2024-04-02T13:49:11.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.143+0000 7fab88f64640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fab6c0a3d20 0x7fab6c0a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fab7c005fd0 tx=0x7fab7c009680 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:11.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.147+0000 7fab81ffb640 1 -- 172.21.15.33:0/3962850593 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fab7809bb80 con 0x7fab84101350 2024-04-02T13:49:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.383+0000 7fab8b1c8640 1 -- 172.21.15.33:0/3962850593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fab841a51b0 con 0x7fab84101350 2024-04-02T13:49:11.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.392+0000 7fab81ffb640 1 -- 172.21.15.33:0/3962850593 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746056 (secure 0 0 0) 0x7fab7809fe80 con 0x7fab84101350 2024-04-02T13:49:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:49:11.404 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 26, 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "active_gid": 15456, 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "active_name": "x", 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addrs": { 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6800", 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 619073725 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-02T13:49:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6801", 2024-04-02T13:49:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 619073725 2024-04-02T13:49:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-02T13:49:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addr": "172.21.15.33:6801/619073725", 2024-04-02T13:49:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "active_change": "2024-04-02T13:38:46.867619+0000", 2024-04-02T13:49:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "active_mgr_features": 4540701547738038271, 2024-04-02T13:49:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "available": true, 2024-04-02T13:49:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "standbys": [ 2024-04-02T13:49:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 25033, 2024-04-02T13:49:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "z", 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540701547738038271, 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-04-02T13:49:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-04-02T13:49:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-04-02T13:49:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-04-02T13:49:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-04-02T13:49:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-04-02T13:49:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-04-02T13:49:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-04-02T13:49:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-04-02T13:49:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-04-02T13:49:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-04-02T13:49:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-04-02T13:49:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-04-02T13:49:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-04-02T13:49:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-04-02T13:49:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-04-02T13:49:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-04-02T13:49:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T13:49:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-04-02T13:49:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-04-02T13:49:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-04-02T13:49:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-02T13:49:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T13:49:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-04-02T13:49:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-04-02T13:49:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-04-02T13:49:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T13:49:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-04-02T13:49:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-04-02T13:49:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-04-02T13:49:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T13:49:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-04-02T13:49:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-04-02T13:49:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T13:49:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T13:49:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-04-02T13:49:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-04-02T13:49:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-04-02T13:49:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-04-02T13:49:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-04-02T13:49:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T13:49:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-04-02T13:49:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-04-02T13:49:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T13:49:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-02T13:49:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-04-02T13:49:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-04-02T13:49:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T13:49:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-04-02T13:49:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-04-02T13:49:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-04-02T13:49:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T13:49:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-04-02T13:49:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-04-02T13:49:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-04-02T13:49:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T13:49:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-02T13:49:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-04-02T13:49:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-04-02T13:49:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T13:49:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-02T13:49:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-04-02T13:49:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-04-02T13:49:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T13:49:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-04-02T13:49:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-04-02T13:49:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-04-02T13:49:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T13:49:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-04-02T13:49:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-04-02T13:49:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-04-02T13:49:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T13:49:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-04-02T13:49:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-04-02T13:49:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-04-02T13:49:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T13:49:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T13:49:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-04-02T13:49:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T13:49:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-04-02T13:49:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T13:49:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-04-02T13:49:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-04-02T13:49:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-04-02T13:49:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T13:49:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-04-02T13:49:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-04-02T13:49:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-04-02T13:49:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-04-02T13:49:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-04-02T13:49:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-04-02T13:49:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T13:49:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-04-02T13:49:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-04-02T13:49:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-04-02T13:49:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-04-02T13:49:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-04-02T13:49:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T13:49:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-04-02T13:49:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-04-02T13:49:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T13:49:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-04-02T13:49:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-04-02T13:49:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T13:49:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T13:49:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T13:49:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T13:49:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T13:49:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T13:49:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T13:49:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-04-02T13:49:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T13:49:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-04-02T13:49:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-04-02T13:49:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-04-02T13:49:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-04-02T13:49:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-04-02T13:49:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T13:49:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-04-02T13:49:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-04-02T13:49:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-04-02T13:49:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-02T13:49:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-02T13:49:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-04-02T13:49:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-04-02T13:49:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-04-02T13:49:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T13:49:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-04-02T13:49:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-04-02T13:49:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-04-02T13:49:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T13:49:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-04-02T13:49:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-04-02T13:49:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T13:49:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.498 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.498 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T13:49:11.498 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.498 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.498 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.498 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-04-02T13:49:11.498 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-04-02T13:49:11.498 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.498 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.498 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.499 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.499 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.499 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.499 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.499 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T13:49:11.499 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.499 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.499 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-04-02T13:49:11.499 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-04-02T13:49:11.499 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.499 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T13:49:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-04-02T13:49:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-04-02T13:49:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T13:49:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-04-02T13:49:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-04-02T13:49:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T13:49:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-04-02T13:49:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-04-02T13:49:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T13:49:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-04-02T13:49:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-04-02T13:49:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-04-02T13:49:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T13:49:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-04-02T13:49:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-04-02T13:49:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T13:49:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-04-02T13:49:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-04-02T13:49:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T13:49:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-04-02T13:49:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-04-02T13:49:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T13:49:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-04-02T13:49:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-04-02T13:49:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T13:49:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-04-02T13:49:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-04-02T13:49:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T13:49:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-04-02T13:49:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-04-02T13:49:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-04-02T13:49:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-04-02T13:49:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T13:49:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-04-02T13:49:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-04-02T13:49:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-04-02T13:49:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T13:49:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-04-02T13:49:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T13:49:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T13:49:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T13:49:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T13:49:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T13:49:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T13:49:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T13:49:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T13:49:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T13:49:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T13:49:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T13:49:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T13:49:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T13:49:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T13:49:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T13:49:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T13:49:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T13:49:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T13:49:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T13:49:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T13:49:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T13:49:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-02T13:49:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T13:49:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T13:49:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-04-02T13:49:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-04-02T13:49:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T13:49:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T13:49:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-02T13:49:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T13:49:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T13:49:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T13:49:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T13:49:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T13:49:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T13:49:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T13:49:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T13:49:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T13:49:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T13:49:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-04-02T13:49:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T13:49:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T13:49:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T13:49:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T13:49:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T13:49:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T13:49:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T13:49:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T13:49:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T13:49:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T13:49:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T13:49:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T13:49:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T13:49:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T13:49:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T13:49:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T13:49:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T13:49:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T13:49:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T13:49:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T13:49:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T13:49:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T13:49:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T13:49:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-02T13:49:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-04-02T13:49:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-04-02T13:49:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-04-02T13:49:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-04-02T13:49:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-04-02T13:49:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-04-02T13:49:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-04-02T13:49:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-04-02T13:49:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-04-02T13:49:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-04-02T13:49:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-02T13:49:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-02T13:49:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-04-02T13:49:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-04-02T13:49:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T13:49:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-02T13:49:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-02T13:49:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-04-02T13:49:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-04-02T13:49:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-04-02T13:49:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-04-02T13:49:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-02T13:49:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-02T13:49:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-02T13:49:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-02T13:49:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-02T13:49:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-02T13:49:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-04-02T13:49:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-04-02T13:49:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-04-02T13:49:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-04-02T13:49:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-04-02T13:49:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T13:49:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-04-02T13:49:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-04-02T13:49:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-04-02T13:49:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T13:49:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-04-02T13:49:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-04-02T13:49:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-04-02T13:49:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-04-02T13:49:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-04-02T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-04-02T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-02T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-04-02T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-04-02T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-04-02T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-04-02T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-04-02T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-04-02T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-04-02T13:49:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-04-02T13:49:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-04-02T13:49:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-04-02T13:49:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-04-02T13:49:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-04-02T13:49:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-04-02T13:49:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-04-02T13:49:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T13:49:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-04-02T13:49:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-04-02T13:49:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-04-02T13:49:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-02T13:49:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-02T13:49:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-04-02T13:49:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T13:49:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-04-02T13:49:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-04-02T13:49:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-02T13:49:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-02T13:49:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-04-02T13:49:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T13:49:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-04-02T13:49:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-04-02T13:49:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-04-02T13:49:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-04-02T13:49:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-04-02T13:49:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-04-02T13:49:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T13:49:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-04-02T13:49:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-04-02T13:49:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-04-02T13:49:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-04-02T13:49:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T13:49:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-04-02T13:49:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-04-02T13:49:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-04-02T13:49:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T13:49:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-04-02T13:49:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-04-02T13:49:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-04-02T13:49:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-04-02T13:49:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-04-02T13:49:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T13:49:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-04-02T13:49:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-04-02T13:49:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-04-02T13:49:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T13:49:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-04-02T13:49:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-04-02T13:49:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-04-02T13:49:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-04-02T13:49:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-04-02T13:49:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-04-02T13:49:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-04-02T13:49:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-04-02T13:49:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-04-02T13:49:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-04-02T13:49:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-04-02T13:49:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T13:49:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T13:49:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-04-02T13:49:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-04-02T13:49:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-02T13:49:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-04-02T13:49:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-04-02T13:49:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T13:49:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-04-02T13:49:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-04-02T13:49:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-04-02T13:49:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-04-02T13:49:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-04-02T13:49:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-04-02T13:49:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T13:49:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-02T13:49:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-02T13:49:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-04-02T13:49:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-04-02T13:49:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-04-02T13:49:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-04-02T13:49:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-04-02T13:49:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-04-02T13:49:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-04-02T13:49:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-04-02T13:49:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-04-02T13:49:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-04-02T13:49:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-04-02T13:49:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T13:49:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T13:49:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-04-02T13:49:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-04-02T13:49:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-04-02T13:49:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-04-02T13:49:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-02T13:49:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-02T13:49:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-02T13:49:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T13:49:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T13:49:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-04-02T13:49:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-04-02T13:49:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-04-02T13:49:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-02T13:49:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-02T13:49:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-04-02T13:49:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-02T13:49:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-02T13:49:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-02T13:49:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-04-02T13:49:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T13:49:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-04-02T13:49:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-04-02T13:49:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-04-02T13:49:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-04-02T13:49:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-04-02T13:49:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-02T13:49:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-02T13:49:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-02T13:49:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-02T13:49:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-02T13:49:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-02T13:49:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-04-02T13:49:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-04-02T13:49:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-04-02T13:49:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-04-02T13:49:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-04-02T13:49:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-04-02T13:49:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-04-02T13:49:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-04-02T13:49:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-04-02T13:49:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-04-02T13:49:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-04-02T13:49:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-04-02T13:49:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-04-02T13:49:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-04-02T13:49:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-04-02T13:49:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-04-02T13:49:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-04-02T13:49:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T13:49:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-04-02T13:49:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-04-02T13:49:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-04-02T13:49:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-04-02T13:49:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-04-02T13:49:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-04-02T13:49:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T13:49:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-04-02T13:49:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-04-02T13:49:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T13:49:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T13:49:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-04-02T13:49:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-04-02T13:49:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T13:49:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-02T13:49:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-02T13:49:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-04-02T13:49:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-04-02T13:49:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-04-02T13:49:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-04-02T13:49:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-04-02T13:49:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-04-02T13:49:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-04-02T13:49:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-04-02T13:49:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T13:49:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-04-02T13:49:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T13:49:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-04-02T13:49:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T13:49:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-04-02T13:49:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-04-02T13:49:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-04-02T13:49:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-04-02T13:49:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-04-02T13:49:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-04-02T13:49:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-04-02T13:49:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-04-02T13:49:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T13:49:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 25130, 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "y", 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540701547738038271, 2024-04-02T13:49:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-04-02T13:49:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-04-02T13:49:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-04-02T13:49:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-04-02T13:49:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-04-02T13:49:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-04-02T13:49:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-04-02T13:49:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-04-02T13:49:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-04-02T13:49:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-04-02T13:49:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-04-02T13:49:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-04-02T13:49:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-04-02T13:49:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-04-02T13:49:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T13:49:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-04-02T13:49:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-04-02T13:49:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-04-02T13:49:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-04-02T13:49:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-04-02T13:49:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-04-02T13:49:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T13:49:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-04-02T13:49:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-04-02T13:49:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-04-02T13:49:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T13:49:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T13:49:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-04-02T13:49:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-04-02T13:49:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-02T13:49:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-02T13:49:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T13:49:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T13:49:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-04-02T13:49:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-04-02T13:49:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-04-02T13:49:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-02T13:49:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-04-02T13:49:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T13:49:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-04-02T13:49:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-04-02T13:49:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-04-02T13:49:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-04-02T13:49:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-04-02T13:49:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-04-02T13:49:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T13:49:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T13:49:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-04-02T13:49:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-04-02T13:49:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-02T13:49:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-02T13:49:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T13:49:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T13:49:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-02T13:49:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-02T13:49:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-04-02T13:49:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-04-02T13:49:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-04-02T13:49:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-04-02T13:49:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-04-02T13:49:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-04-02T13:49:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-04-02T13:49:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-04-02T13:49:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-04-02T13:49:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T13:49:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T13:49:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-04-02T13:49:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-04-02T13:49:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-02T13:49:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-02T13:49:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T13:49:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T13:49:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-04-02T13:49:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-04-02T13:49:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T13:49:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-04-02T13:49:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-04-02T13:49:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-04-02T13:49:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-02T13:49:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T13:49:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-04-02T13:49:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-04-02T13:49:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-04-02T13:49:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T13:49:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-04-02T13:49:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-04-02T13:49:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-04-02T13:49:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T13:49:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-04-02T13:49:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-04-02T13:49:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T13:49:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-04-02T13:49:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T13:49:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-04-02T13:49:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T13:49:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-04-02T13:49:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-04-02T13:49:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-04-02T13:49:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-04-02T13:49:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T13:49:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-04-02T13:49:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-04-02T13:49:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T13:49:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-04-02T13:49:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-04-02T13:49:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T13:49:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-04-02T13:49:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-04-02T13:49:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T13:49:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-02T13:49:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-04-02T13:49:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-04-02T13:49:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T13:49:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-04-02T13:49:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-04-02T13:49:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-04-02T13:49:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T13:49:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-02T13:49:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-04-02T13:49:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-04-02T13:49:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T13:49:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-04-02T13:49:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-04-02T13:49:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-04-02T13:49:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T13:49:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-04-02T13:49:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-04-02T13:49:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-04-02T13:49:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T13:49:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-04-02T13:49:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-04-02T13:49:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-04-02T13:49:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T13:49:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-04-02T13:49:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-04-02T13:49:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-04-02T13:49:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T13:49:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-04-02T13:49:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-04-02T13:49:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-04-02T13:49:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T13:49:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-02T13:49:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-04-02T13:49:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-04-02T13:49:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T13:49:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-04-02T13:49:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-04-02T13:49:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-04-02T13:49:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T13:49:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-02T13:49:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-04-02T13:49:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-04-02T13:49:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T13:49:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-04-02T13:49:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-04-02T13:49:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-04-02T13:49:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T13:49:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-04-02T13:49:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-04-02T13:49:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-04-02T13:49:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T13:49:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-04-02T13:49:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-04-02T13:49:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-04-02T13:49:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T13:49:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-04-02T13:49:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-04-02T13:49:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T13:49:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-04-02T13:49:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T13:49:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-04-02T13:49:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T13:49:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-04-02T13:49:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-04-02T13:49:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-04-02T13:49:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T13:49:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-04-02T13:49:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-04-02T13:49:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T13:49:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-04-02T13:49:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-04-02T13:49:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-04-02T13:49:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T13:49:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T13:49:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T13:49:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T13:49:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T13:49:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-04-02T13:49:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-04-02T13:49:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-04-02T13:49:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-04-02T13:49:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-04-02T13:49:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-04-02T13:49:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-04-02T13:49:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-04-02T13:49:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T13:49:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-04-02T13:49:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-04-02T13:49:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T13:49:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T13:49:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-04-02T13:49:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-04-02T13:49:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T13:49:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-04-02T13:49:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-04-02T13:49:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T13:49:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-04-02T13:49:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-04-02T13:49:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T13:49:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-04-02T13:49:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-04-02T13:49:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T13:49:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-04-02T13:49:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-04-02T13:49:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-04-02T13:49:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T13:49:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-04-02T13:49:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-04-02T13:49:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-04-02T13:49:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T13:49:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-04-02T13:49:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T13:49:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T13:49:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T13:49:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T13:49:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T13:49:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T13:49:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-04-02T13:49:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T13:49:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T13:49:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T13:49:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T13:49:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T13:49:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T13:49:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T13:49:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T13:49:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T13:49:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T13:49:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T13:49:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T13:49:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T13:49:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T13:49:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T13:49:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T13:49:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T13:49:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-02T13:49:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T13:49:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-04-02T13:49:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T13:49:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T13:49:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-02T13:49:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T13:49:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T13:49:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T13:49:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T13:49:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-04-02T13:49:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T13:49:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T13:49:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T13:49:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T13:49:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T13:49:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T13:49:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T13:49:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T13:49:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-04-02T13:49:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T13:49:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T13:49:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-04-02T13:49:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T13:49:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T13:49:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T13:49:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T13:49:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-02T13:49:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T13:49:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T13:49:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T13:49:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T13:49:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T13:49:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T13:49:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T13:49:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T13:49:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T13:49:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T13:49:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T13:49:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T13:49:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-02T13:49:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-04-02T13:49:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-04-02T13:49:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-04-02T13:49:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-04-02T13:49:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-04-02T13:49:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-04-02T13:49:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-02T13:49:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-02T13:49:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-04-02T13:49:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-04-02T13:49:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T13:49:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-02T13:49:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-02T13:49:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-02T13:49:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-04-02T13:49:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-02T13:49:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-04-02T13:49:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T13:49:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T13:49:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-04-02T13:49:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T13:49:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T13:49:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T13:49:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-04-02T13:49:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-04-02T13:49:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-02T13:49:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-04-02T13:49:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-04-02T13:49:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-04-02T13:49:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T13:49:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-04-02T13:49:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-04-02T13:49:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-04-02T13:49:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T13:49:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-04-02T13:49:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-04-02T13:49:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-04-02T13:49:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-04-02T13:49:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-04-02T13:49:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-04-02T13:49:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-04-02T13:49:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-02T13:49:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-02T13:49:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-04-02T13:49:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-04-02T13:49:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-04-02T13:49:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-04-02T13:49:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-04-02T13:49:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T13:49:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-04-02T13:49:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-04-02T13:49:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-04-02T13:49:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-04-02T13:49:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-04-02T13:49:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T13:49:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-04-02T13:49:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-04-02T13:49:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-04-02T13:49:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T13:49:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-04-02T13:49:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-04-02T13:49:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-04-02T13:49:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-04-02T13:49:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-04-02T13:49:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T13:49:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-04-02T13:49:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-04-02T13:49:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-04-02T13:49:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-04-02T13:49:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-04-02T13:49:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-04-02T13:49:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-04-02T13:49:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T13:49:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-04-02T13:49:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-04-02T13:49:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-04-02T13:49:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-04-02T13:49:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-04-02T13:49:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-04-02T13:49:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T13:49:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T13:49:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-04-02T13:49:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-04-02T13:49:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-04-02T13:49:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-02T13:49:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-04-02T13:49:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-02T13:49:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-02T13:49:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-04-02T13:49:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-04-02T13:49:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T13:49:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T13:49:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-04-02T13:49:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-04-02T13:49:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-04-02T13:49:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-04-02T13:49:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-04-02T13:49:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-04-02T13:49:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-04-02T13:49:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-04-02T13:49:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-04-02T13:49:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-04-02T13:49:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-02T13:49:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-02T13:49:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-02T13:49:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-04-02T13:49:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-02T13:49:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-02T13:49:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-02T13:49:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-04-02T13:49:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-04-02T13:49:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-04-02T13:49:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-04-02T13:49:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-04-02T13:49:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-02T13:49:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-02T13:49:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-04-02T13:49:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-04-02T13:49:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-04-02T13:49:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T13:49:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-04-02T13:49:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-04-02T13:49:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-04-02T13:49:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-04-02T13:49:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-04-02T13:49:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-04-02T13:49:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-04-02T13:49:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-04-02T13:49:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-04-02T13:49:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-04-02T13:49:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-04-02T13:49:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-04-02T13:49:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-04-02T13:49:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-04-02T13:49:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-04-02T13:49:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-04-02T13:49:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-04-02T13:49:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-04-02T13:49:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-04-02T13:49:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-04-02T13:49:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-02T13:49:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-04-02T13:49:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-04-02T13:49:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-04-02T13:49:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-04-02T13:49:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-04-02T13:49:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-04-02T13:49:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-04-02T13:49:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-04-02T13:49:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-04-02T13:49:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-04-02T13:49:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T13:49:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-04-02T13:49:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-04-02T13:49:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-04-02T13:49:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-04-02T13:49:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-04-02T13:49:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-04-02T13:49:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-04-02T13:49:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T13:49:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T13:49:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-04-02T13:49:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-04-02T13:49:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T13:49:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-04-02T13:49:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-04-02T13:49:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T13:49:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-04-02T13:49:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-04-02T13:49:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-04-02T13:49:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-04-02T13:49:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-04-02T13:49:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-04-02T13:49:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-04-02T13:49:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-04-02T13:49:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T13:49:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-04-02T13:49:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-04-02T13:49:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-04-02T13:49:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-04-02T13:49:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T13:49:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "modules": [ 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard", 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "iostat", 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "nfs", 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "restful", 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_schedule" 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-04-02T13:49:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-04-02T13:49:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-04-02T13:49:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-04-02T13:49:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-04-02T13:49:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-04-02T13:49:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-04-02T13:49:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-04-02T13:49:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-04-02T13:49:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-04-02T13:49:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-04-02T13:49:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-04-02T13:49:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-04-02T13:49:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-04-02T13:49:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-04-02T13:49:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-04-02T13:49:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-04-02T13:49:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-04-02T13:49:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-04-02T13:49:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-04-02T13:49:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-04-02T13:49:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-04-02T13:49:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-04-02T13:49:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-04-02T13:49:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T13:49:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T13:49:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T13:49:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T13:49:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-04-02T13:49:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-04-02T13:49:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T13:49:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-04-02T13:49:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-04-02T13:49:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-02T13:49:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-02T13:49:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T13:49:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T13:49:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-04-02T13:49:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-04-02T13:49:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-04-02T13:49:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-04-02T13:49:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T13:49:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T13:49:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-04-02T13:49:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-04-02T13:49:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-02T13:49:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-02T13:49:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T13:49:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T13:49:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-04-02T13:49:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-04-02T13:49:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T13:49:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-04-02T13:49:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-04-02T13:49:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-04-02T13:49:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-02T13:49:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T13:49:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-04-02T13:49:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-04-02T13:49:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-04-02T13:49:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T13:49:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-04-02T13:49:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-04-02T13:49:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-04-02T13:49:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T13:49:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-04-02T13:49:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-04-02T13:49:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T13:49:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T13:49:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-04-02T13:49:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-04-02T13:49:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-04-02T13:49:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T13:49:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-04-02T13:49:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-04-02T13:49:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-04-02T13:49:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-04-02T13:49:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T13:49:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-04-02T13:49:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-04-02T13:49:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T13:49:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-04-02T13:49:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-04-02T13:49:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-04-02T13:49:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T13:49:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-04-02T13:49:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-04-02T13:49:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-04-02T13:49:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T13:49:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-02T13:49:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-04-02T13:49:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-04-02T13:49:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T13:49:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-04-02T13:49:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-04-02T13:49:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-04-02T13:49:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T13:49:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-04-02T13:49:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-04-02T13:49:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-04-02T13:49:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-04-02T13:49:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-04-02T13:49:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-04-02T13:49:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T13:49:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T13:49:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-04-02T13:49:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-04-02T13:49:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-04-02T13:49:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-04-02T13:49:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-04-02T13:49:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T13:49:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-04-02T13:49:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-04-02T13:49:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T13:49:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T13:49:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T13:49:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T13:49:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T13:49:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-04-02T13:49:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T13:49:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-04-02T13:49:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-04-02T13:49:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-04-02T13:49:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-04-02T13:49:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-04-02T13:49:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T13:49:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-04-02T13:49:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-04-02T13:49:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-04-02T13:49:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-04-02T13:49:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-04-02T13:49:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T13:49:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-04-02T13:49:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-04-02T13:49:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T13:49:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T13:49:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-04-02T13:49:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-04-02T13:49:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T13:49:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-04-02T13:49:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-04-02T13:49:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T13:49:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-04-02T13:49:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-04-02T13:49:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T13:49:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-04-02T13:49:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-04-02T13:49:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T13:49:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-04-02T13:49:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-04-02T13:49:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-04-02T13:49:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-04-02T13:49:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-04-02T13:49:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-04-02T13:49:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T13:49:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-04-02T13:49:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-04-02T13:49:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T13:49:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-04-02T13:49:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-04-02T13:49:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T13:49:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-04-02T13:49:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-04-02T13:49:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-04-02T13:49:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-04-02T13:49:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T13:49:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-04-02T13:49:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-04-02T13:49:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-04-02T13:49:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T13:49:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-04-02T13:49:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T13:49:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T13:49:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T13:49:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T13:49:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T13:49:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T13:49:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-04-02T13:49:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T13:49:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T13:49:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T13:49:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T13:49:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T13:49:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T13:49:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T13:49:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T13:49:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T13:49:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T13:49:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T13:49:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-02T13:49:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T13:49:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T13:49:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-04-02T13:49:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-04-02T13:49:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T13:49:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T13:49:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-02T13:49:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T13:49:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T13:49:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T13:49:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T13:49:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T13:49:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T13:49:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T13:49:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T13:49:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T13:49:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T13:49:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-04-02T13:49:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T13:49:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T13:49:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-04-02T13:49:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T13:49:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T13:49:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T13:49:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T13:49:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T13:49:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T13:49:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T13:49:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T13:49:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T13:49:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T13:49:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T13:49:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T13:49:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T13:49:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T13:49:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T13:49:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T13:49:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T13:49:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T13:49:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T13:49:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T13:49:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T13:49:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T13:49:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T13:49:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T13:49:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T13:49:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-04-02T13:49:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T13:49:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T13:49:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-04-02T13:49:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T13:49:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T13:49:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-02T13:49:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T13:49:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T13:49:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T13:49:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T13:49:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T13:49:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T13:49:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T13:49:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T13:49:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T13:49:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T13:49:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T13:49:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T13:49:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-02T13:49:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-04-02T13:49:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-04-02T13:49:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-04-02T13:49:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-04-02T13:49:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-04-02T13:49:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-04-02T13:49:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-04-02T13:49:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-04-02T13:49:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-04-02T13:49:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-04-02T13:49:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-02T13:49:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-02T13:49:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-04-02T13:49:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-04-02T13:49:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-04-02T13:49:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-04-02T13:49:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T13:49:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-02T13:49:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-02T13:49:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-02T13:49:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-02T13:49:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-02T13:49:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-04-02T13:49:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-02T13:49:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-02T13:49:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-04-02T13:49:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-04-02T13:49:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-04-02T13:49:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-02T13:49:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-02T13:49:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-04-02T13:49:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-04-02T13:49:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-02T13:49:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-02T13:49:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-02T13:49:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-02T13:49:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-02T13:49:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-04-02T13:49:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-04-02T13:49:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-04-02T13:49:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-04-02T13:49:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-04-02T13:49:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-04-02T13:49:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-04-02T13:49:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-04-02T13:49:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-04-02T13:49:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T13:49:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-04-02T13:49:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-04-02T13:49:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-04-02T13:49:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T13:49:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-04-02T13:49:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-04-02T13:49:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-04-02T13:49:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-04-02T13:49:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-04-02T13:49:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-04-02T13:49:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-02T13:49:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T13:49:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-04-02T13:49:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-04-02T13:49:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T13:49:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T13:49:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-04-02T13:49:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-04-02T13:49:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-04-02T13:49:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-04-02T13:49:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-04-02T13:49:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-04-02T13:49:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-02T13:49:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-04-02T13:49:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-04-02T13:49:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-04-02T13:49:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-04-02T13:49:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-04-02T13:49:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-04-02T13:49:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T13:49:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-04-02T13:49:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-04-02T13:49:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-04-02T13:49:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T13:49:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-04-02T13:49:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-04-02T13:49:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-04-02T13:49:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T13:49:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-04-02T13:49:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-04-02T13:49:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-04-02T13:49:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-04-02T13:49:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-04-02T13:49:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-04-02T13:49:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-04-02T13:49:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-02T13:49:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-02T13:49:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-04-02T13:49:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T13:49:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-04-02T13:49:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-04-02T13:49:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-02T13:49:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-02T13:49:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-04-02T13:49:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T13:49:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-04-02T13:49:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-04-02T13:49:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-04-02T13:49:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-04-02T13:49:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-04-02T13:49:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-04-02T13:49:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T13:49:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-04-02T13:49:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-04-02T13:49:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-04-02T13:49:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-04-02T13:49:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-04-02T13:49:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T13:49:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-04-02T13:49:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-04-02T13:49:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-04-02T13:49:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-04-02T13:49:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-04-02T13:49:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-04-02T13:49:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T13:49:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-04-02T13:49:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-04-02T13:49:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-04-02T13:49:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-04-02T13:49:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-04-02T13:49:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-04-02T13:49:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-04-02T13:49:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-04-02T13:49:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-04-02T13:49:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-04-02T13:49:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T13:49:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T13:49:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-04-02T13:49:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-04-02T13:49:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T13:49:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-04-02T13:49:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-02T13:49:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-04-02T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-04-02T13:49:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-04-02T13:49:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-02T13:49:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-04-02T13:49:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-04-02T13:49:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-02T13:49:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-04-02T13:49:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-04-02T13:49:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-04-02T13:49:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-04-02T13:49:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T13:49:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T13:49:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T13:49:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T13:49:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-04-02T13:49:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-04-02T13:49:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-04-02T13:49:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-02T13:49:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T13:49:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-02T13:49:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-02T13:49:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-04-02T13:49:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T13:49:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-04-02T13:49:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-04-02T13:49:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-04-02T13:49:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-02T13:49:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-02T13:49:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-04-02T13:49:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-04-02T13:49:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-04-02T13:49:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-02T13:49:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-02T13:49:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-02T13:49:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-02T13:49:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-02T13:49:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-04-02T13:49:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-04-02T13:49:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-04-02T13:49:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-02T13:49:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-04-02T13:49:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T13:49:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-04-02T13:49:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-04-02T13:49:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-04-02T13:49:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-04-02T13:49:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-04-02T13:49:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-02T13:49:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-02T13:49:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-02T13:49:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-02T13:49:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-04-02T13:49:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-04-02T13:49:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-04-02T13:49:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-04-02T13:49:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-04-02T13:49:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-04-02T13:49:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-04-02T13:49:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-04-02T13:49:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-04-02T13:49:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-04-02T13:49:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-04-02T13:49:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-02T13:49:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-04-02T13:49:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-04-02T13:49:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-04-02T13:49:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-04-02T13:49:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-04-02T13:49:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-04-02T13:49:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-02T13:49:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-04-02T13:49:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-04-02T13:49:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-04-02T13:49:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-04-02T13:49:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-04-02T13:49:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-04-02T13:49:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-04-02T13:49:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-04-02T13:49:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-04-02T13:49:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-04-02T13:49:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-04-02T13:49:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T13:49:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-04-02T13:49:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-04-02T13:49:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-04-02T13:49:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-04-02T13:49:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T13:49:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-04-02T13:49:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-04-02T13:49:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T13:49:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-04-02T13:49:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-04-02T13:49:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T13:49:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-04-02T13:49:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-04-02T13:49:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T13:49:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-04-02T13:49:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-04-02T13:49:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-04-02T13:49:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-04-02T13:49:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-04-02T13:49:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T13:49:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-02T13:49:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-02T13:49:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-04-02T13:49:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-04-02T13:49:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-04-02T13:49:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-04-02T13:49:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T13:49:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-04-02T13:49:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-04-02T13:49:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-04-02T13:49:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-04-02T13:49:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-04-02T13:49:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T13:49:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-04-02T13:49:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-04-02T13:49:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-02T13:49:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T13:49:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-04-02T13:49:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-02T13:49:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-02T13:49:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-02T13:49:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-04-02T13:49:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-04-02T13:49:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-02T13:49:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-04-02T13:49:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-02T13:49:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-02T13:49:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-02T13:49:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-02T13:49:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-02T13:49:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-02T13:49:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-02T13:49:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-02T13:49:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-02T13:49:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-02T13:49:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-02T13:49:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-02T13:49:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-02T13:49:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-02T13:49:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-02T13:49:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-02T13:49:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-04-02T13:49:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-04-02T13:49:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-02T13:49:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-04-02T13:49:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-04-02T13:49:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-04-02T13:49:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-02T13:49:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-02T13:49:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-02T13:49:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T13:49:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-02T13:49:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-02T13:49:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-02T13:49:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-02T13:49:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-02T13:49:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-02T13:49:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-02T13:49:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "services": { 2024-04-02T13:49:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard": "https://172.21.15.33:8443/" 2024-04-02T13:49:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "always_on_modules": { 2024-04-02T13:49:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus": [ 2024-04-02T13:49:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-02T13:49:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-02T13:49:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-02T13:49:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-02T13:49:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-02T13:49:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-02T13:49:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-02T13:49:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-02T13:49:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-02T13:49:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-02T13:49:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific": [ 2024-04-02T13:49:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-02T13:49:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-02T13:49:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-02T13:49:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-02T13:49:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-02T13:49:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-02T13:49:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-02T13:49:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-02T13:49:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-02T13:49:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-02T13:49:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy": [ 2024-04-02T13:49:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-02T13:49:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-02T13:49:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-02T13:49:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-02T13:49:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-02T13:49:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-02T13:49:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-02T13:49:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-02T13:49:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-02T13:49:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-02T13:49:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "reef": [ 2024-04-02T13:49:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-02T13:49:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-02T13:49:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-02T13:49:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-02T13:49:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-02T13:49:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-02T13:49:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-02T13:49:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-02T13:49:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-02T13:49:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-02T13:49:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-02T13:49:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "squid": [ 2024-04-02T13:49:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-02T13:49:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-02T13:49:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-02T13:49:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-02T13:49:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-02T13:49:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-02T13:49:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-02T13:49:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-02T13:49:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-02T13:49:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-02T13:49:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-02T13:49:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "last_failure_osd_epoch": 95, 2024-04-02T13:49:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "active_clients": [ 2024-04-02T13:49:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-04-02T13:49:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-02T13:49:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-02T13:49:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-02T13:49:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3830683341 2024-04-02T13:49:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-02T13:49:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "libcephsqlite", 2024-04-02T13:49:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-02T13:49:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-02T13:49:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-02T13:49:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 80898625 2024-04-02T13:49:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-02T13:49:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-04-02T13:49:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-02T13:49:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-02T13:49:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-02T13:49:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 22044734 2024-04-02T13:49:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-02T13:49:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-04-02T13:49:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-02T13:49:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-02T13:49:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-02T13:49:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1515487952 2024-04-02T13:49:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-02T13:49:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-04-02T13:49:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-02T13:49:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-02T13:49:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-02T13:49:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1508188988 2024-04-02T13:49:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-02T13:49:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-04-02T13:49:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-02T13:49:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-02T13:49:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-02T13:49:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-02T13:49:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1376504510 2024-04-02T13:49:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-02T13:49:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-02T13:49:12.818 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:49:12.819 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:11 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3059266473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:49:12.819 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:11 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1791579630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:49:12.819 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:11 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/186031970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:49:12.819 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:11 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3962850593' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T13:49:12.819 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:11 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3059266473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:49:12.819 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:11 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1791579630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:49:12.819 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:11 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/186031970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:49:12.820 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:11 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3962850593' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T13:49:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.397+0000 7fab8b1c8640 1 -- 172.21.15.33:0/3962850593 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fab6c0a3d20 msgr2=0x7fab6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.397+0000 7fab8b1c8640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fab6c0a3d20 0x7fab6c0a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fab7c005fd0 tx=0x7fab7c009680 comp rx=0 tx=0).stop 2024-04-02T13:49:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.397+0000 7fab8b1c8640 1 -- 172.21.15.33:0/3962850593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab84101350 msgr2=0x7fab841988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.397+0000 7fab8b1c8640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab84101350 0x7fab841988e0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fab7800b4a0 tx=0x7fab7800b960 comp rx=0 tx=0).stop 2024-04-02T13:49:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.397+0000 7fab8b1c8640 1 -- 172.21.15.33:0/3962850593 shutdown_connections 2024-04-02T13:49:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.397+0000 7fab8b1c8640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fab6c0a3d20 0x7fab6c0a61e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.397+0000 7fab8b1c8640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fab841069a0 0x7fab8419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.397+0000 7fab8b1c8640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab84101350 0x7fab841988e0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.397+0000 7fab8b1c8640 1 --2- 172.21.15.33:0/3962850593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab84100990 0x7fab841983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.397+0000 7fab8b1c8640 1 -- 172.21.15.33:0/3962850593 >> 172.21.15.33:0/3962850593 conn(0x7fab84077ad0 msgr2=0x7fab840ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.397+0000 7fab8b1c8640 1 -- 172.21.15.33:0/3962850593 shutdown_connections 2024-04-02T13:49:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:11.397+0000 7fab8b1c8640 1 -- 172.21.15.33:0/3962850593 wait complete. 2024-04-02T13:49:12.827 DEBUG:tasks.check_counter:Getting stats from a 2024-04-02T13:49:12.827 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.a perf dump 2024-04-02T13:49:12.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:11 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3059266473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:49:12.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:11 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1791579630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:49:12.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:11 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/186031970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:49:12.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:11 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3962850593' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T13:49:12.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:12 smithi145 ceph-mon[35646]: pgmap v324: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:49:12.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:12 smithi145 ceph-mon[35646]: osdmap e105: 12 total, 12 up, 12 in 2024-04-02T13:49:12.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:12 smithi050 ceph-mon[35524]: pgmap v324: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:49:12.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:12 smithi050 ceph-mon[35524]: osdmap e105: 12 total, 12 up, 12 in 2024-04-02T13:49:13.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:12 smithi033 ceph-mon[32390]: pgmap v324: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-02T13:49:13.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:12 smithi033 ceph-mon[32390]: osdmap e105: 12 total, 12 up, 12 in 2024-04-02T13:49:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:49:14.798 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:14 smithi033 ceph-mon[32390]: pgmap v326: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:49:14.933 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:49:14.933 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:49:14.933 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 4, 2024-04-02T13:49:14.933 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 4, 2024-04-02T13:49:14.933 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3040, 2024-04-02T13:49:14.933 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 448, 2024-04-02T13:49:14.933 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.001566965, 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000039749, 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000638262, 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3, 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000705384, 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000235128 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 448 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 222, 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 211, 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1589984, 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 129344, 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.043559157, 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.013545595, 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.035321321, 2024-04-02T13:49:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.004218635, 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 210, 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.009341338, 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000044482 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1589984, 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 129344 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:14.936 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 161, 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 183968, 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.013537040, 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.009210918, 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001848561, 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 158, 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.015267627, 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000096630 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:14.937 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 183968 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:49:14.938 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:49:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:49:14.940 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 149011, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 32, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-04-02T13:49:14.941 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:49:14.942 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:49:14.942 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-04-02T13:49:14.942 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-04-02T13:49:14.942 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-04-02T13:49:14.942 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-04-02T13:49:14.942 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-04-02T13:49:14.942 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-04-02T13:49:14.942 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-04-02T13:49:14.942 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-04-02T13:49:14.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:14.942 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:49:14.942 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-02T13:49:14.942 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 224, 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1558668, 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 224, 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 224, 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1558668, 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:14.944 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:49:14.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:14 smithi145 ceph-mon[35646]: pgmap v326: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:49:14.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:14 smithi050 ceph-mon[35524]: pgmap v326: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:49:15.801 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-02T13:49:15.801 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-02T13:49:15.802 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-02T13:49:15.802 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-02T13:49:15.802 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-02T13:49:15.802 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-02T13:49:15.802 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-02T13:49:15.802 DEBUG:tasks.check_counter:Getting stats from d 2024-04-02T13:49:15.802 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.d perf dump 2024-04-02T13:49:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:49:16.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:16 smithi033 ceph-mon[32390]: pgmap v327: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 236 B/s wr, 8 op/s 2024-04-02T13:49:16.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:16 smithi145 ceph-mon[35646]: pgmap v327: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 236 B/s wr, 8 op/s 2024-04-02T13:49:16.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:16 smithi050 ceph-mon[35524]: pgmap v327: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 236 B/s wr, 8 op/s 2024-04-02T13:49:17.771 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 228, 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 217, 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1591616, 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 133184, 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.042404479, 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.012989682, 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.034338500, 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.004444901, 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 216, 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.008534638, 2024-04-02T13:49:17.772 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000039512 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1591616, 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 133184 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 165, 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 184672, 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-04-02T13:49:17.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.014142316, 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.009364917, 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001912469, 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 162, 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.015918699, 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000098263 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:17.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:17.775 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T13:49:17.775 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 184672 2024-04-02T13:49:17.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:17.775 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:49:17.775 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 4, 2024-04-02T13:49:17.775 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 4, 2024-04-02T13:49:17.775 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3040, 2024-04-02T13:49:17.775 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 448, 2024-04-02T13:49:17.775 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-04-02T13:49:17.775 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-04-02T13:49:17.775 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.000759197, 2024-04-02T13:49:17.775 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000040779, 2024-04-02T13:49:17.775 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000712587, 2024-04-02T13:49:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:49:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3, 2024-04-02T13:49:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000722286, 2024-04-02T13:49:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000240762 2024-04-02T13:49:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-02T13:49:17.776 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 448 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:49:17.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:49:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148977, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 30, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-04-02T13:49:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:49:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 230, 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1559448, 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 230, 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 230, 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1559448, 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:17.781 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:49:18.411 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-02T13:49:18.412 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-02T13:49:18.412 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-02T13:49:18.412 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-02T13:49:18.412 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-02T13:49:18.412 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-02T13:49:18.412 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-02T13:49:18.412 DEBUG:tasks.check_counter:Getting stats from g 2024-04-02T13:49:18.412 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.g perf dump 2024-04-02T13:49:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:49:18.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:18 smithi033 ceph-mon[32390]: pgmap v328: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:49:18.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:18 smithi145 ceph-mon[35646]: pgmap v328: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:49:18.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:18 smithi050 ceph-mon[35524]: pgmap v328: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-02T13:49:20.416 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:49:20.416 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:49:20.416 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-04-02T13:49:20.416 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-04-02T13:49:20.416 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-04-02T13:49:20.416 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-04-02T13:49:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-04-02T13:49:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-04-02T13:49:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.001361106, 2024-04-02T13:49:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-04-02T13:49:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000344711, 2024-04-02T13:49:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:49:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T13:49:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T13:49:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:49:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-04-02T13:49:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 162, 2024-04-02T13:49:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-04-02T13:49:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 184144, 2024-04-02T13:49:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-04-02T13:49:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-04-02T13:49:20.419 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.014359320, 2024-04-02T13:49:20.419 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.009759295, 2024-04-02T13:49:20.419 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.002084077, 2024-04-02T13:49:20.419 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:49:20.419 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:20.419 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 159, 2024-04-02T13:49:20.419 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.010491619, 2024-04-02T13:49:20.419 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000065985 2024-04-02T13:49:20.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:20.419 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:20.419 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:20.419 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 184144 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 228, 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 215, 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1593568, 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 131072, 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.043176797, 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.013072963, 2024-04-02T13:49:20.420 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.035124816, 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.004204002, 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 215, 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.010640371, 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000049490 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1593568, 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 131072 2024-04-02T13:49:20.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:49:20.422 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:49:20.423 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:49:20.424 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:49:20.424 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-04-02T13:49:20.424 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:49:20.424 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:49:20.424 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:49:20.424 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-04-02T13:49:20.424 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-04-02T13:49:20.424 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-04-02T13:49:20.424 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:49:20.424 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:49:20.424 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:49:20.424 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:49:20.424 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148994, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 31, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-04-02T13:49:20.425 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 226, 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1558928, 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 226, 2024-04-02T13:49:20.427 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-02T13:49:20.427 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-02T13:49:20.427 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 226, 2024-04-02T13:49:20.427 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1558928, 2024-04-02T13:49:20.427 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-02T13:49:20.427 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:20.427 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:20.427 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:20.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:20.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:20.427 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:49:20.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:20 smithi033 ceph-mon[32390]: pgmap v329: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:49:20.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:20 smithi145 ceph-mon[35646]: pgmap v329: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:49:20.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:20 smithi050 ceph-mon[35524]: pgmap v329: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:49:21.173 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-02T13:49:21.173 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-02T13:49:21.173 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-02T13:49:21.173 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-02T13:49:21.173 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-02T13:49:21.173 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-02T13:49:21.173 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-02T13:49:21.173 DEBUG:tasks.check_counter:Getting stats from j 2024-04-02T13:49:21.173 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.j perf dump 2024-04-02T13:49:21.422 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:49:22.842 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:22 smithi033 ceph-mon[32390]: pgmap v330: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:49:22.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:22 smithi145 ceph-mon[35646]: pgmap v330: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:49:22.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:22 smithi050 ceph-mon[35524]: pgmap v330: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-02T13:49:23.108 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-02T13:49:23.108 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:49:23.108 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 4, 2024-04-02T13:49:23.108 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 4, 2024-04-02T13:49:23.108 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3040, 2024-04-02T13:49:23.108 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 448, 2024-04-02T13:49:23.108 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.001659305, 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000052248, 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000674020, 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3, 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000671027, 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000223675 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:23.109 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-02T13:49:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 448 2024-04-02T13:49:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:49:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-04-02T13:49:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 167, 2024-04-02T13:49:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-04-02T13:49:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 185024, 2024-04-02T13:49:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-04-02T13:49:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-04-02T13:49:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.014973120, 2024-04-02T13:49:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.009658629, 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.002972703, 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 164, 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.011108724, 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000067736 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T13:49:23.111 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 185024 2024-04-02T13:49:23.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:23.112 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:49:23.112 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 230, 2024-04-02T13:49:23.112 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 219, 2024-04-02T13:49:23.112 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1592160, 2024-04-02T13:49:23.112 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 134464, 2024-04-02T13:49:23.112 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-04-02T13:49:23.112 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-04-02T13:49:23.112 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.046347337, 2024-04-02T13:49:23.112 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.014338739, 2024-04-02T13:49:23.112 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.037406494, 2024-04-02T13:49:23.112 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.004314418, 2024-04-02T13:49:23.112 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 218, 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.010393725, 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000047677 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1592160, 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 134464 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:23.113 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-04-02T13:49:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:49:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-04-02T13:49:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:49:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:49:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:49:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:49:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:49:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:49:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:49:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:49:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:49:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:49:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:49:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 150008, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 31, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:49:23.116 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:49:23.117 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 232, 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1559708, 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 232, 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 232, 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1559708, 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:23.118 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-02T13:49:23.119 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-02T13:49:23.821 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-02T13:49:23.821 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-02T13:49:23.821 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-02T13:49:23.821 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-02T13:49:23.821 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-02T13:49:23.821 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-02T13:49:23.821 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-02T13:49:23.821 DEBUG:tasks.check_counter:Getting stats from b 2024-04-02T13:49:23.821 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.b perf dump 2024-04-02T13:49:24.080 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:49:24.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:24 smithi145 ceph-mon[35646]: pgmap v331: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 89 B/s wr, 9 op/s 2024-04-02T13:49:24.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:24 smithi050 ceph-mon[35524]: pgmap v331: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 89 B/s wr, 9 op/s 2024-04-02T13:49:25.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:24 smithi033 ceph-mon[32390]: pgmap v331: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 89 B/s wr, 9 op/s 2024-04-02T13:49:26.215 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-04-02T13:49:26.215 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:49:26.215 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 20681, 2024-04-02T13:49:26.215 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 27437, 2024-04-02T13:49:26.215 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 5321876, 2024-04-02T13:49:26.215 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 62896263, 2024-04-02T13:49:26.215 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 16, 2024-04-02T13:49:26.216 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 4, 2024-04-02T13:49:26.216 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 2.045484256, 2024-04-02T13:49:26.216 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.882752046, 2024-04-02T13:49:26.216 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.739492112, 2024-04-02T13:49:26.216 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.404445657, 2024-04-02T13:49:26.216 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:26.216 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 27435, 2024-04-02T13:49:26.216 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.578742669, 2024-04-02T13:49:26.216 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000057544 2024-04-02T13:49:26.216 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.216 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:26.216 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 27881, 2024-04-02T13:49:26.216 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.006651288, 2024-04-02T13:49:26.217 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000238 2024-04-02T13:49:26.217 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.217 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 372256, 2024-04-02T13:49:26.217 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 98512 2024-04-02T13:49:26.217 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.217 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:49:26.217 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 59753, 2024-04-02T13:49:26.217 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 98609, 2024-04-02T13:49:26.217 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 21531537, 2024-04-02T13:49:26.217 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 87693066, 2024-04-02T13:49:26.217 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 163, 2024-04-02T13:49:26.217 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 8, 2024-04-02T13:49:26.217 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 3.934671664, 2024-04-02T13:49:26.218 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 1.533363849, 2024-04-02T13:49:26.218 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 2.237041599, 2024-04-02T13:49:26.218 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.454227453, 2024-04-02T13:49:26.218 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:26.218 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 98606, 2024-04-02T13:49:26.218 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 3.160449883, 2024-04-02T13:49:26.218 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000032051 2024-04-02T13:49:26.218 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.218 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:26.218 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 125958, 2024-04-02T13:49:26.218 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.021567683, 2024-04-02T13:49:26.218 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000171 2024-04-02T13:49:26.218 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 100224, 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 489440 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 23674, 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 29923, 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 7716390, 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 32676310, 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 13, 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 6, 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 2.136234831, 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.919184957, 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.823573098, 2024-04-02T13:49:26.219 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.391503112, 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 29923, 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.371484524, 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000045833 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 29062, 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.007014018, 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000241 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1750304, 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 167552 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.220 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 1, 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-MDSRank": { 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 8275, 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 27.179447651, 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.003284525 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-PurgeQueue": { 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-04-02T13:49:26.221 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 17041, 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 2.944595289, 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000172794 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: "request": 25504, 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: "reply": 25271, 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: "reply_latency": { 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 25271, 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 68.869966322, 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002725256 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.222 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_reply": 0, 2024-04-02T13:49:26.223 INFO:teuthology.orchestra.run.smithi050.stdout: "forward": 233, 2024-04-02T13:49:26.223 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_complete": 0, 2024-04-02T13:49:26.223 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_keys": 0, 2024-04-02T13:49:26.223 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_commit": 183, 2024-04-02T13:49:26.223 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_split": 19, 2024-04-02T13:49:26.223 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_merge": 0, 2024-04-02T13:49:26.223 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes": 16136, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_top": 311, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_bottom": 15499, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pin_tail": 326, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pinned": 15691, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_expired": 727, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_with_caps": 2, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": 3, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "subtrees": 37, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse": 28758, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_hit": 27021, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_forward": 124, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_discover": 9, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_remote_ino": 0, 2024-04-02T13:49:26.224 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_lock": 0, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "load_cent": 0, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "q": 0, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "exported": 45, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "exported_inodes": 612, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "imported": 44, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "imported_inodes": 781, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_dir_fetch": 0, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_backtrace_fetch": 7877, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_peer_discover": 0, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rfiles": 4, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rbytes": 234, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rsnaps": 6, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_fetch": 7877, 2024-04-02T13:49:26.225 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_set_tag": 0, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_inodes": 723, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_base_inodes": 19, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dirfrag_rstats": 704, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_file_inodes": 7154, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_inode_file_caps": 8, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_revoke": 3346, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_grant": 23341, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps": 22930, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps_dirty": 12781, 2024-04-02T13:49:26.226 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_cap_release": 75, 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "process_request_cap_release": 33396 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_cache": { 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update": 1264, 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update_receipt": 41, 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_try_discover": 25, 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_send_discover": 25, 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_handle_discover": 2571, 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays": 600, 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_delayed": 0, 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_created": 10896, 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_enqueued": 10296, 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_reintegrated": 0, 2024-04-02T13:49:26.227 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_migrated": 0, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_processing": 0, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_started": 0, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_completed": 0, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_enqueue_scrub": 19, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_exportdir": 45, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_flush": 0, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragmentdir": 19, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragstats": 0, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_inodestats": 0 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.228 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_log": { 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "evlrg": 0, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "evadd": 50233, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "evex": 0, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "evtrm": 0, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "ev": 50233, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "evexg": 0, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "evexd": 0, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "segadd": 71, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "segex": 0, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "segtrm": 0, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "seg": 71, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "segmjr": 6, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "segexg": 0, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "segexd": 0, 2024-04-02T13:49:26.229 INFO:teuthology.orchestra.run.smithi050.stdout: "expos": 4194304, 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: "wrpos": 126593047, 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: "rdpos": 0, 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: "jlat": { 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 2370, 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 61.174733293, 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.025812123 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: "replayed": 0 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_mem": { 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 7762, 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: "ino+": 18748, 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: "ino-": 10986, 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: "dir": 749, 2024-04-02T13:49:26.230 INFO:teuthology.orchestra.run.smithi050.stdout: "dir+": 809, 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "dir-": 60, 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "dn": 16136, 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "dn+": 27214, 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "dn-": 11078, 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "cap": 3, 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "cap+": 11559, 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "cap-": 11556, 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "rss": 122844, 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "heap": 231800 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_server": { 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_client_request": 32822, 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_server_request": 938, 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_request": 25504, 2024-04-02T13:49:26.231 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_session": 85, 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_peer_request": 2122, 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: "req_create_latency": { 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 10894, 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 27.412365709, 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002516281 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getattr_latency": { 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 150, 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.054918146, 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000366120 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getfilelock_latency": { 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.232 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: "req_link_latency": { 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookup_latency": { 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1580, 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 7.914085925, 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.005008915 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookuphash_latency": { 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupino_latency": { 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupname_latency": { 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupparent_latency": { 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.234 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.235 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.235 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.235 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupsnap_latency": { 2024-04-02T13:49:26.235 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.235 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.235 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.236 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.236 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lssnap_latency": { 2024-04-02T13:49:26.236 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 11, 2024-04-02T13:49:26.236 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.002600897, 2024-04-02T13:49:26.236 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000236445 2024-04-02T13:49:26.236 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.236 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mkdir_latency": { 2024-04-02T13:49:26.236 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 623, 2024-04-02T13:49:26.236 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.353159492, 2024-04-02T13:49:26.236 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002172005 2024-04-02T13:49:26.236 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.236 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mknod_latency": { 2024-04-02T13:49:26.236 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mksnap_latency": { 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 11, 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.603912658, 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.054901150 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: "req_open_latency": { 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 4, 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.001200119, 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000300029 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: "req_readdir_latency": { 2024-04-02T13:49:26.237 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1046, 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.835949875, 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.001755210 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rename_latency": { 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 2, 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.002039772, 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.001019886 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: "req_renamesnap_latency": { 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.238 INFO:teuthology.orchestra.run.smithi050.stdout: "req_snapdiff_latency": { 2024-04-02T13:49:26.239 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.239 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.239 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.239 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.239 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmdir_latency": { 2024-04-02T13:49:26.239 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 607, 2024-04-02T13:49:26.239 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 10.474105590, 2024-04-02T13:49:26.239 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.017255528 2024-04-02T13:49:26.239 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmsnap_latency": { 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 5, 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.021755242, 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.004351048 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmxattr_latency": { 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setattr_latency": { 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 40, 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.144069041, 2024-04-02T13:49:26.240 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.003601726 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setdirlayout_latency": { 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setfilelock_latency": { 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setlayout_latency": { 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.241 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.242 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.242 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setxattr_latency": { 2024-04-02T13:49:26.242 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 9, 2024-04-02T13:49:26.242 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.020657920, 2024-04-02T13:49:26.242 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002295324 2024-04-02T13:49:26.242 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.242 INFO:teuthology.orchestra.run.smithi050.stdout: "req_symlink_latency": { 2024-04-02T13:49:26.242 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.242 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.242 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.242 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.242 INFO:teuthology.orchestra.run.smithi050.stdout: "req_unlink_latency": { 2024-04-02T13:49:26.242 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 10289, 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 19.029145936, 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.001849465 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getvxattr_latency": { 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_sessions": { 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: "session_count": 2, 2024-04-02T13:49:26.243 INFO:teuthology.orchestra.run.smithi050.stdout: "session_add": 7, 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: "session_remove": 5, 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_open": 2, 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_stale": 0, 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: "total_load": 164, 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: "average_load": 82, 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: "avg_session_uptime": 632, 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: "mdthresh_evicted": 0 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:49:26.244 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:49:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:49:26.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 5101335, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 1321, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 9512, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 182, 2024-04-02T13:49:26.247 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 29819881, 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 345562, 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "objecter": { 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "op_active": 0, 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "op_laggy": 0, 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send": 32214, 2024-04-02T13:49:26.248 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send_bytes": 127600331, 2024-04-02T13:49:26.249 INFO:teuthology.orchestra.run.smithi050.stdout: "op_resend": 31, 2024-04-02T13:49:26.249 INFO:teuthology.orchestra.run.smithi050.stdout: "op_reply": 32214, 2024-04-02T13:49:26.249 INFO:teuthology.orchestra.run.smithi050.stdout: "op_latency": { 2024-04-02T13:49:26.249 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 32214, 2024-04-02T13:49:26.249 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 130.382702262, 2024-04-02T13:49:26.249 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.004047392 2024-04-02T13:49:26.250 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.250 INFO:teuthology.orchestra.run.smithi050.stdout: "op_inflight": 0, 2024-04-02T13:49:26.250 INFO:teuthology.orchestra.run.smithi050.stdout: "oplen_avg": { 2024-04-02T13:49:26.250 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 32214, 2024-04-02T13:49:26.250 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 32629 2024-04-02T13:49:26.250 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "op": 32214, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "op_r": 8088, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "op_w": 24126, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "op_rmw": 0, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "op_pg": 0, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_stat": 0, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_create": 11, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_read": 192, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_write": 2543, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writefull": 94, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writesame": 0, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_append": 0, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_zero": 0, 2024-04-02T13:49:26.251 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_truncate": 19, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_delete": 21150, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_mapext": 0, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_sparse_read": 0, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_clonerange": 0, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_getxattr": 7896, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_setxattr": 22, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_rmxattr": 0, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_call": 0, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_watch": 0, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_notify": 0, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls": 0, 2024-04-02T13:49:26.252 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_other": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_active": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_send": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_resend": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_ping": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_active": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_send": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_resend": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_active": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_send": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_resend": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_active": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_send": 0, 2024-04-02T13:49:26.253 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_resend": 0, 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: "command_active": 0, 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: "command_send": 0, 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: "command_resend": 0, 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: "map_epoch": 105, 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: "map_full": 0, 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: "map_inc": 20, 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_sessions": 12, 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_open": 12, 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_close": 0, 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_laggy": 0, 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_wr": 585, 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_rd": 0, 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_del": 117 2024-04-02T13:49:26.254 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: "oft": { 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_objs": 1, 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_updates": 0, 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_removes": 0 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: "osd-slow-ops": { 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_ops_count": 0 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: "purge_queue": { 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops": 0, 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops_high_water": 21, 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing": 0, 2024-04-02T13:49:26.255 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_high_water": 9, 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed_ops": 20609, 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed": 10296, 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_item_in_journal": 0 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 104104, 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 25126689, 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 104104, 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-04-02T13:49:26.256 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 104104, 2024-04-02T13:49:26.257 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 25126689, 2024-04-02T13:49:26.257 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:26.257 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.257 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.257 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.257 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:26.257 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.257 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_bytes": { 2024-04-02T13:49:26.257 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:26.257 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-04-02T13:49:26.257 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:26.257 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-04-02T13:49:26.257 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 32214, 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 128776611, 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 32214, 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 128776611, 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_ops": { 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:26.258 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1024, 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 32214, 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 32214, 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 32214, 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 32214, 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.259 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 10296, 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 1176280, 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 10296, 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 173, 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1176280, 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:26.260 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle-0x55f521a46500": { 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 50233, 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 122398743, 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 50233, 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-04-02T13:49:26.261 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 2370, 2024-04-02T13:49:26.262 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 122398743, 2024-04-02T13:49:26.262 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:26.262 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:26.262 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:26.262 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:26.262 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:26.262 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:26.262 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-04-02T13:49:26.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:26 smithi050 ceph-mon[35524]: pgmap v332: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:26.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:26 smithi145 ceph-mon[35646]: pgmap v332: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:27.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:26 smithi033 ceph-mon[32390]: pgmap v332: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:27.051 INFO:tasks.check_counter:Daemon mds.b mds.exported=45 2024-04-02T13:49:27.051 INFO:tasks.check_counter:Daemon mds.b mds.imported=44 2024-04-02T13:49:27.051 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1264 2024-04-02T13:49:27.051 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=41 2024-04-02T13:49:27.051 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-02T13:49:27.052 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-02T13:49:27.052 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-02T13:49:27.052 DEBUG:tasks.check_counter:Getting stats from e 2024-04-02T13:49:27.052 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.e perf dump 2024-04-02T13:49:27.302 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:49:28.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:28 smithi050 ceph-mon[35524]: pgmap v333: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:28.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:28 smithi145 ceph-mon[35646]: pgmap v333: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:29.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:28 smithi033 ceph-mon[32390]: pgmap v333: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:29.055 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-04-02T13:49:29.055 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:49:29.055 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 18354, 2024-04-02T13:49:29.055 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 10919, 2024-04-02T13:49:29.055 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 7961011, 2024-04-02T13:49:29.055 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 6956856, 2024-04-02T13:49:29.055 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 11, 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 4, 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.957487915, 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.424181378, 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.656166320, 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.037722209, 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 10921, 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.269280098, 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000024657 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 26969, 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.004408705, 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000163 2024-04-02T13:49:29.056 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 928, 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 1216 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 1075, 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 1231, 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 293556, 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 1062772, 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 48, 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 6, 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.087754843, 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.027942753, 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.044908805, 2024-04-02T13:49:29.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.014128870, 2024-04-02T13:49:29.058 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:29.058 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1228, 2024-04-02T13:49:29.058 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.075501512, 2024-04-02T13:49:29.058 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000061483 2024-04-02T13:49:29.058 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.058 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:29.058 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1051, 2024-04-02T13:49:29.058 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000217660, 2024-04-02T13:49:29.058 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000207 2024-04-02T13:49:29.058 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.058 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 6560, 2024-04-02T13:49:29.058 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 145808 2024-04-02T13:49:29.058 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 2820, 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 4371, 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 2572941, 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 2663879, 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 15, 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 7, 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.217920095, 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.088495019, 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.138928749, 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.016614150, 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 4367, 2024-04-02T13:49:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.153365591, 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000035119 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 5171, 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.001029775, 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000199 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1688816, 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 472256 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 1, 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.060 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-MDSRank": { 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1133, 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.965667709, 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000852310 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-PurgeQueue": { 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 922, 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.109673394, 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000118951 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:29.061 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: "request": 1241, 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: "reply": 1066, 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: "reply_latency": { 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1066, 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.771776610, 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.001662079 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_reply": 0, 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: "forward": 175, 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_complete": 0, 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_keys": 0, 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_commit": 11, 2024-04-02T13:49:29.062 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_split": 0, 2024-04-02T13:49:29.063 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_merge": 0, 2024-04-02T13:49:29.063 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes": 416, 2024-04-02T13:49:29.063 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_top": 2, 2024-04-02T13:49:29.063 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_bottom": 2, 2024-04-02T13:49:29.063 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pin_tail": 412, 2024-04-02T13:49:29.063 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pinned": 412, 2024-04-02T13:49:29.063 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_expired": 1474, 2024-04-02T13:49:29.063 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_with_caps": 1, 2024-04-02T13:49:29.063 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": 1, 2024-04-02T13:49:29.063 INFO:teuthology.orchestra.run.smithi050.stdout: "subtrees": 4, 2024-04-02T13:49:29.063 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse": 3874, 2024-04-02T13:49:29.063 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_hit": 2206, 2024-04-02T13:49:29.063 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_forward": 171, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_discover": 608, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_remote_ino": 0, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_lock": 0, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "load_cent": 0, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "q": 0, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "exported": 20, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "exported_inodes": 381, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "imported": 21, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "imported_inodes": 388, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_dir_fetch": 0, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_backtrace_fetch": 399, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_peer_discover": 0, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rfiles": 4, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rbytes": 234, 2024-04-02T13:49:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rsnaps": 6, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_fetch": 399, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_set_tag": 0, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_inodes": 8, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dirfrag_rstats": 8, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_file_inodes": 391, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_inode_file_caps": 2, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_revoke": 98, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_grant": 692, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:49:29.065 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps": 878, 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps_dirty": 535, 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_cap_release": 36, 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "process_request_cap_release": 1127 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_cache": { 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update": 68, 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update_receipt": 1974, 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_try_discover": 1281, 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_send_discover": 1361, 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_handle_discover": 83, 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays": 8, 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_delayed": 0, 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_created": 504, 2024-04-02T13:49:29.066 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_enqueued": 496, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_reintegrated": 0, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_migrated": 0, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_processing": 0, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_started": 0, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_completed": 0, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_exportdir": 20, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_flush": 0, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragmentdir": 0, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragstats": 0, 2024-04-02T13:49:29.067 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_inodestats": 0 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_log": { 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "evlrg": 0, 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "evadd": 2031, 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "evex": 0, 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "evtrm": 0, 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "ev": 2031, 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "evexg": 0, 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "evexd": 0, 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "segadd": 4, 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "segex": 0, 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "segtrm": 0, 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "seg": 4, 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "segmjr": 2, 2024-04-02T13:49:29.068 INFO:teuthology.orchestra.run.smithi050.stdout: "segexg": 0, 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: "segexd": 0, 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: "expos": 4194304, 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: "wrpos": 8417401, 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: "rdpos": 0, 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: "jlat": { 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 368, 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.715505024, 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.001944307 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: "replayed": 0 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_mem": { 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 224, 2024-04-02T13:49:29.069 INFO:teuthology.orchestra.run.smithi050.stdout: "ino+": 1540, 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: "ino-": 1316, 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: "dir": 32, 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: "dir+": 849, 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: "dir-": 817, 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: "dn": 416, 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: "dn+": 2386, 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: "dn-": 1970, 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: "cap": 1, 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: "cap+": 540, 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: "cap-": 539, 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: "rss": 66040, 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: "heap": 223608 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.070 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_server": { 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_client_request": 2222, 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_server_request": 66, 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_request": 1241, 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_session": 35, 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_peer_request": 568, 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: "req_create_latency": { 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 117, 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.275698339, 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002356396 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getattr_latency": { 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 5, 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.001139245, 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000227849 2024-04-02T13:49:29.071 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getfilelock_latency": { 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: "req_link_latency": { 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookup_latency": { 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 381, 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.215851761, 2024-04-02T13:49:29.072 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000566540 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookuphash_latency": { 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupino_latency": { 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupname_latency": { 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.073 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.074 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.074 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupparent_latency": { 2024-04-02T13:49:29.074 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.074 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.074 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.074 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.074 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupsnap_latency": { 2024-04-02T13:49:29.074 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.074 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.074 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.074 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.074 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lssnap_latency": { 2024-04-02T13:49:29.074 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.075 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.075 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.075 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.075 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mkdir_latency": { 2024-04-02T13:49:29.075 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.076 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.076 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.076 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.076 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mknod_latency": { 2024-04-02T13:49:29.076 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.076 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.076 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.076 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.076 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mksnap_latency": { 2024-04-02T13:49:29.076 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.076 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.076 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.076 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: "req_open_latency": { 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: "req_readdir_latency": { 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 59, 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.063282996, 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.001072593 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rename_latency": { 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.077 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.078 INFO:teuthology.orchestra.run.smithi050.stdout: "req_renamesnap_latency": { 2024-04-02T13:49:29.078 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.078 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.078 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.078 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.078 INFO:teuthology.orchestra.run.smithi050.stdout: "req_snapdiff_latency": { 2024-04-02T13:49:29.078 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.078 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.078 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.078 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.078 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmdir_latency": { 2024-04-02T13:49:29.078 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 8, 2024-04-02T13:49:29.078 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.138365148, 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.017295643 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmsnap_latency": { 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmxattr_latency": { 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setattr_latency": { 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.079 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setdirlayout_latency": { 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setfilelock_latency": { 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setlayout_latency": { 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.080 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setxattr_latency": { 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: "req_symlink_latency": { 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: "req_unlink_latency": { 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 496, 2024-04-02T13:49:29.081 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.077439121, 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002172256 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getvxattr_latency": { 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_sessions": { 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: "session_count": 2, 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: "session_add": 6, 2024-04-02T13:49:29.082 INFO:teuthology.orchestra.run.smithi050.stdout: "session_remove": 4, 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_open": 2, 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_stale": 0, 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "total_load": 769, 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "average_load": 384, 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "avg_session_uptime": 632, 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "mdthresh_evicted": 0 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:49:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:49:29.084 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:49:29.085 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 348710, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 131, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 9512, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 182, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:49:29.086 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 848653, 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 10252, 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: "objecter": { 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: "op_active": 0, 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: "op_laggy": 0, 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send": 1988, 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send_bytes": 4347787, 2024-04-02T13:49:29.087 INFO:teuthology.orchestra.run.smithi050.stdout: "op_resend": 33, 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: "op_reply": 1988, 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: "op_latency": { 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1988, 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 6.303840389, 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.003170945 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: "op_inflight": 0, 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: "oplen_avg": { 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1988, 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 2095 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: "op": 1988, 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: "op_r": 443, 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: "op_w": 1545, 2024-04-02T13:49:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: "op_rmw": 0, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "op_pg": 0, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_stat": 0, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_create": 10, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_read": 44, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_write": 412, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writefull": 39, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writesame": 0, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_append": 0, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_zero": 0, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_truncate": 0, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_delete": 1005, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_mapext": 0, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_sparse_read": 0, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_clonerange": 0, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_getxattr": 399, 2024-04-02T13:49:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_setxattr": 20, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_rmxattr": 0, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_call": 0, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_watch": 0, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_notify": 0, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls": 0, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_other": 0, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_active": 0, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_send": 0, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_resend": 0, 2024-04-02T13:49:29.090 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_ping": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_active": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_send": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_resend": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_active": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_send": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_resend": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_active": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_send": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_resend": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "command_active": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "command_send": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "command_resend": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "map_epoch": 105, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "map_full": 0, 2024-04-02T13:49:29.091 INFO:teuthology.orchestra.run.smithi050.stdout: "map_inc": 15, 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_sessions": 12, 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_open": 12, 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_close": 0, 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_laggy": 0, 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_wr": 117, 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_rd": 0, 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_del": 49 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: "oft": { 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_objs": 1, 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_updates": 0, 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_removes": 0 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.092 INFO:teuthology.orchestra.run.smithi050.stdout: "osd-slow-ops": { 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_ops_count": 0 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: "purge_queue": { 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops": 0, 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops_high_water": 16, 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing": 0, 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_high_water": 8, 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed_ops": 992, 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed": 496, 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_item_in_journal": 0 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-04-02T13:49:29.093 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 22245, 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 8805626, 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 22245, 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 22245, 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 8805626, 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.094 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_bytes": { 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 1988, 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 4404883, 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 1988, 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 4404883, 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.095 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_ops": { 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1024, 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 1988, 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 1988, 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 1988, 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1988, 2024-04-02T13:49:29.096 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 496, 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 57096, 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 496, 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-04-02T13:49:29.097 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 44, 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 57096, 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle-0x559a0e74c500": { 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 2031, 2024-04-02T13:49:29.098 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 4223097, 2024-04-02T13:49:29.099 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:29.099 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 2031, 2024-04-02T13:49:29.099 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-04-02T13:49:29.099 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-04-02T13:49:29.099 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 368, 2024-04-02T13:49:29.099 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 4223097, 2024-04-02T13:49:29.099 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:29.099 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:29.099 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:29.099 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:29.099 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:29.099 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:29.099 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-04-02T13:49:29.965 INFO:tasks.check_counter:Daemon mds.e mds.exported=20 2024-04-02T13:49:29.965 INFO:tasks.check_counter:Daemon mds.e mds.imported=21 2024-04-02T13:49:29.965 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=68 2024-04-02T13:49:29.965 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=1974 2024-04-02T13:49:29.965 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-02T13:49:29.965 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:49:29.966 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:49:29.966 DEBUG:tasks.check_counter:Getting stats from h 2024-04-02T13:49:29.966 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.h perf dump 2024-04-02T13:49:30.215 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:49:30.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:30 smithi145 ceph-mon[35646]: pgmap v334: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:49:30.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:30 smithi050 ceph-mon[35524]: pgmap v334: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:49:31.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:30 smithi033 ceph-mon[32390]: pgmap v334: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:49:32.015 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-04-02T13:49:32.015 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:49:32.015 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 1260, 2024-04-02T13:49:32.015 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 1316, 2024-04-02T13:49:32.015 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 390695, 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 4007953, 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 49, 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 9, 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.102441161, 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.038522891, 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.043280944, 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.016253383, 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1309, 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.056093005, 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000042851 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 983, 2024-04-02T13:49:32.016 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000170646, 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000173 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 320784 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 19904, 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 14532, 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 8320761, 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 4521716, 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 10, 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 5, 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 1.024627286, 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.464938184, 2024-04-02T13:49:32.017 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.730842262, 2024-04-02T13:49:32.018 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.011729223, 2024-04-02T13:49:32.018 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:32.018 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 14534, 2024-04-02T13:49:32.018 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.310922156, 2024-04-02T13:49:32.018 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000021392 2024-04-02T13:49:32.018 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.018 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:32.018 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 33161, 2024-04-02T13:49:32.018 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.005526296, 2024-04-02T13:49:32.018 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000166 2024-04-02T13:49:32.018 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.018 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 2672, 2024-04-02T13:49:32.018 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 132976 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 955, 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 930, 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 1899790, 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 519095, 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 13, 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 6, 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.091675469, 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.027486601, 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.054795263, 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.019917729, 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:32.019 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 930, 2024-04-02T13:49:32.020 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.056880264, 2024-04-02T13:49:32.020 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000061161 2024-04-02T13:49:32.020 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.020 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:32.020 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 6, 2024-04-02T13:49:32.020 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000827, 2024-04-02T13:49:32.020 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000137 2024-04-02T13:49:32.020 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.020 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1691584, 2024-04-02T13:49:32.020 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 150512 2024-04-02T13:49:32.021 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 1, 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-MDSRank": { 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1141, 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.592357183, 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000519156 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-PurgeQueue": { 2024-04-02T13:49:32.022 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 708, 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.064911062, 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000091682 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: "request": 1088, 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: "reply": 777, 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: "reply_latency": { 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 777, 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.582568882, 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002036768 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_reply": 0, 2024-04-02T13:49:32.023 INFO:teuthology.orchestra.run.smithi050.stdout: "forward": 311, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_complete": 0, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_keys": 0, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_commit": 11, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_split": 0, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_merge": 0, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes": 60, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_top": 7, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_bottom": 37, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pin_tail": 16, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pinned": 49, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_expired": 1493, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_with_caps": 3, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": 3, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "subtrees": 4, 2024-04-02T13:49:32.024 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse": 3891, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_hit": 1758, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_forward": 309, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_discover": 617, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_remote_ino": 0, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_lock": 0, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "load_cent": 0, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "q": 0, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "exported": 24, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "exported_inodes": 400, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "imported": 24, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "imported_inodes": 224, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_dir_fetch": 0, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_backtrace_fetch": 164, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_peer_discover": 0, 2024-04-02T13:49:32.025 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rfiles": 4, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rbytes": 234, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rsnaps": 6, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_fetch": 164, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_set_tag": 0, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_inodes": 1, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dirfrag_rstats": 1, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_file_inodes": 163, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_inode_file_caps": 0, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_revoke": 70, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_grant": 527, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:49:32.026 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps": 687, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps_dirty": 302, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_cap_release": 43, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "process_request_cap_release": 859 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_cache": { 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update": 63, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update_receipt": 1957, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_try_discover": 1271, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_send_discover": 1335, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_handle_discover": 67, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays": 2, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_delayed": 0, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:49:32.027 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_created": 361, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_enqueued": 359, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_reintegrated": 0, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_migrated": 0, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_processing": 0, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_started": 0, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_completed": 0, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_exportdir": 24, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_flush": 0, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragmentdir": 0, 2024-04-02T13:49:32.028 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragstats": 0, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_inodestats": 0 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_log": { 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "evlrg": 0, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "evadd": 1668, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "evex": 0, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "evtrm": 0, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "ev": 1668, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "evexg": 0, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "evexd": 0, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "segadd": 3, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "segex": 0, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "segtrm": 0, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "seg": 3, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "segmjr": 2, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "segexg": 0, 2024-04-02T13:49:32.029 INFO:teuthology.orchestra.run.smithi050.stdout: "segexd": 0, 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: "expos": 4194304, 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: "wrpos": 7096121, 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: "rdpos": 0, 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: "jlat": { 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 462, 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.042179199, 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002255799 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: "replayed": 0 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_mem": { 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 48, 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: "ino+": 1248, 2024-04-02T13:49:32.030 INFO:teuthology.orchestra.run.smithi050.stdout: "ino-": 1200, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "dir": 30, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "dir+": 846, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "dir-": 816, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "dn": 60, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "dn+": 1919, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "dn-": 1859, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "cap": 3, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "cap+": 405, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "cap-": 402, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "rss": 59656, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "heap": 223608 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_server": { 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_client_request": 2657, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_server_request": 63, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_request": 1088, 2024-04-02T13:49:32.031 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_session": 21, 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_peer_request": 635, 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: "req_create_latency": { 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 137, 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.378354770, 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002761713 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getattr_latency": { 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 4, 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000934571, 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000233642 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getfilelock_latency": { 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.032 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: "req_link_latency": { 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookup_latency": { 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 233, 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.094695687, 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000406419 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookuphash_latency": { 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.033 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupino_latency": { 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupname_latency": { 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupparent_latency": { 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.034 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.035 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.035 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupsnap_latency": { 2024-04-02T13:49:32.035 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.035 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.035 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.035 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.035 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lssnap_latency": { 2024-04-02T13:49:32.035 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mkdir_latency": { 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mknod_latency": { 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mksnap_latency": { 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.036 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: "req_open_latency": { 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: "req_readdir_latency": { 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 42, 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.021348967, 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000508308 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rename_latency": { 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.037 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: "req_renamesnap_latency": { 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: "req_snapdiff_latency": { 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmdir_latency": { 2024-04-02T13:49:32.038 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 2, 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.039101748, 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.019550874 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmsnap_latency": { 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmxattr_latency": { 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setattr_latency": { 2024-04-02T13:49:32.039 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setdirlayout_latency": { 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setfilelock_latency": { 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setlayout_latency": { 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.040 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setxattr_latency": { 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: "req_symlink_latency": { 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: "req_unlink_latency": { 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 359, 2024-04-02T13:49:32.041 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.048133139, 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002919590 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getvxattr_latency": { 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_sessions": { 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: "session_count": 1, 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: "session_add": 4, 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: "session_remove": 3, 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_open": 1, 2024-04-02T13:49:32.042 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_stale": 0, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "total_load": 465, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "average_load": 465, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "avg_session_uptime": 644, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "mdthresh_evicted": 0 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:49:32.043 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-04-02T13:49:32.044 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 315457, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 121, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-04-02T13:49:32.045 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 9512, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 182, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 221473, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 1395, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-04-02T13:49:32.046 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-04-02T13:49:32.047 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.047 INFO:teuthology.orchestra.run.smithi050.stdout: "objecter": { 2024-04-02T13:49:32.047 INFO:teuthology.orchestra.run.smithi050.stdout: "op_active": 0, 2024-04-02T13:49:32.047 INFO:teuthology.orchestra.run.smithi050.stdout: "op_laggy": 0, 2024-04-02T13:49:32.047 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send": 1606, 2024-04-02T13:49:32.047 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send_bytes": 2993469, 2024-04-02T13:49:32.047 INFO:teuthology.orchestra.run.smithi050.stdout: "op_resend": 33, 2024-04-02T13:49:32.047 INFO:teuthology.orchestra.run.smithi050.stdout: "op_reply": 1606, 2024-04-02T13:49:32.047 INFO:teuthology.orchestra.run.smithi050.stdout: "op_latency": { 2024-04-02T13:49:32.047 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1606, 2024-04-02T13:49:32.047 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 4.986873777, 2024-04-02T13:49:32.047 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.003105151 2024-04-02T13:49:32.047 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: "op_inflight": 0, 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: "oplen_avg": { 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1606, 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1727 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: "op": 1606, 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: "op_r": 218, 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: "op_w": 1388, 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: "op_rmw": 0, 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: "op_pg": 0, 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_stat": 0, 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_create": 10, 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_read": 54, 2024-04-02T13:49:32.048 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_write": 516, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writefull": 45, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writesame": 0, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_append": 0, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_zero": 0, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_truncate": 0, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_delete": 730, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_mapext": 0, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_sparse_read": 0, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_clonerange": 0, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_getxattr": 164, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_setxattr": 20, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_rmxattr": 0, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_call": 0, 2024-04-02T13:49:32.049 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_watch": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_notify": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_other": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_active": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_send": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_resend": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_ping": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_active": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_send": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_resend": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_active": 0, 2024-04-02T13:49:32.050 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_send": 0, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_resend": 0, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_active": 0, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_send": 0, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_resend": 0, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "command_active": 0, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "command_send": 0, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "command_resend": 0, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "map_epoch": 105, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "map_full": 0, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "map_inc": 15, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_sessions": 12, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_open": 12, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_close": 0, 2024-04-02T13:49:32.051 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_laggy": 0, 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_wr": 135, 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_rd": 0, 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_del": 53 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: "oft": { 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_objs": 1, 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_updates": 0, 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_removes": 0 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: "osd-slow-ops": { 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_ops_count": 0 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.052 INFO:teuthology.orchestra.run.smithi050.stdout: "purge_queue": { 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops": 0, 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops_high_water": 16, 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing": 0, 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_high_water": 8, 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed_ops": 718, 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed": 359, 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_item_in_journal": 0 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 22115, 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 8601906, 2024-04-02T13:49:32.053 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 22115, 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 22115, 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 8601906, 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_bytes": { 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-04-02T13:49:32.054 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 1606, 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 3034320, 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 1606, 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 3034320, 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.055 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_ops": { 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1024, 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 1606, 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 1606, 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 1606, 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1606, 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.056 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 359, 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 40851, 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 359, 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 54, 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 40851, 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:32.057 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle-0x55c2d6be6500": { 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 1668, 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 2901817, 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 1668, 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 462, 2024-04-02T13:49:32.058 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 2901817, 2024-04-02T13:49:32.059 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:32.059 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:32.059 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:32.059 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:32.059 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:32.059 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:32.059 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-04-02T13:49:32.844 INFO:tasks.check_counter:Daemon mds.h mds.exported=24 2024-04-02T13:49:32.844 INFO:tasks.check_counter:Daemon mds.h mds.imported=24 2024-04-02T13:49:32.844 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=63 2024-04-02T13:49:32.844 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=1957 2024-04-02T13:49:32.844 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-02T13:49:32.844 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:49:32.844 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:49:32.844 DEBUG:tasks.check_counter:Getting stats from k 2024-04-02T13:49:32.844 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.k perf dump 2024-04-02T13:49:32.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:32 smithi145 ceph-mon[35646]: pgmap v335: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:32.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:32 smithi050 ceph-mon[35524]: pgmap v335: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:33.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:32 smithi033 ceph-mon[32390]: pgmap v335: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:33.091 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:49:34.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:34 smithi050 ceph-mon[35524]: pgmap v336: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s 2024-04-02T13:49:34.903 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-04-02T13:49:34.903 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:49:34.903 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 234, 2024-04-02T13:49:34.903 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 223, 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 1593248, 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 137024, 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.047068803, 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.013819374, 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.035850640, 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.005944606, 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 222, 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.011144770, 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000050201 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:34.904 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1593248, 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 137024 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 6, 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 172, 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 3360, 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 186000, 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 5, 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 3, 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.011461717, 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.007379899, 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.001993624, 2024-04-02T13:49:34.905 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 170, 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.015414050, 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000090670 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 186000 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:34.906 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 0, 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 2, 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 0, 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 0, 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.000360690, 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000000000, 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.000304924, 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:34.907 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:49:34.908 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:49:34.909 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 148794, 2024-04-02T13:49:34.910 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 32, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-04-02T13:49:34.911 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 236, 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 1560228, 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 236, 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-04-02T13:49:34.912 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-04-02T13:49:34.913 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 236, 2024-04-02T13:49:34.913 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1560228, 2024-04-02T13:49:34.913 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-04-02T13:49:34.913 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-04-02T13:49:34.913 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-04-02T13:49:34.913 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-04-02T13:49:34.913 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:34.913 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-04-02T13:49:34.913 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-04-02T13:49:34.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:34 smithi145 ceph-mon[35646]: pgmap v336: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s 2024-04-02T13:49:35.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:34 smithi033 ceph-mon[32390]: pgmap v336: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s 2024-04-02T13:49:35.710 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-02T13:49:35.710 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-02T13:49:35.710 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-02T13:49:35.710 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-02T13:49:35.710 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-02T13:49:35.711 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-02T13:49:35.711 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-02T13:49:35.711 DEBUG:tasks.check_counter:Getting stats from c 2024-04-02T13:49:35.711 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.c perf dump 2024-04-02T13:49:35.960 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:49:36.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:36 smithi145 ceph-mon[35646]: pgmap v337: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-02T13:49:36.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:36 smithi050 ceph-mon[35524]: pgmap v337: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-02T13:49:37.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:36 smithi033 ceph-mon[32390]: pgmap v337: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-02T13:49:37.938 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-04-02T13:49:37.938 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:49:37.938 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 2282, 2024-04-02T13:49:37.938 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 2284, 2024-04-02T13:49:37.938 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 3545432, 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 725683, 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 2, 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.201662833, 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.065926909, 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.051294320, 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.068911885, 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2282, 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.081973454, 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000035921 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.939 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 4, 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 169, 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 876, 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 451530, 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 4, 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.011001765, 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.006672301, 2024-04-02T13:49:37.940 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.002165464, 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000079264, 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 166, 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.021363645, 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000128696 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 450832 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.941 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 271, 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 243, 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1684492, 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 139342, 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 3, 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 2, 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.049567863, 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.015854395, 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.037585337, 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.005515521, 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 243, 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.014407747, 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000059291 2024-04-02T13:49:37.942 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1684224, 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 139024 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-04-02T13:49:37.943 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3880, 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.107702064, 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000027758 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 0, 2024-04-02T13:49:37.944 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 0, 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 0, 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 0, 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 0, 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 0, 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 397, 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 0, 2024-04-02T13:49:37.945 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 397, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 0, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 397, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 0, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 0, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 0, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 1, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 0, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 0, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 0, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 0, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 0, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 0, 2024-04-02T13:49:37.946 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-04-02T13:49:37.947 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 0, 2024-04-02T13:49:37.947 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 0, 2024-04-02T13:49:37.947 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 0, 2024-04-02T13:49:37.947 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 0, 2024-04-02T13:49:37.947 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-04-02T13:49:37.947 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 0, 2024-04-02T13:49:37.947 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-04-02T13:49:37.947 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 0, 2024-04-02T13:49:37.947 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 0, 2024-04-02T13:49:37.947 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 0, 2024-04-02T13:49:37.947 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T13:49:37.947 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 0, 2024-04-02T13:49:37.948 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 0, 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 0 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 0, 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 0, 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 0, 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 0, 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 0, 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 0, 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 0, 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 0, 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-04-02T13:49:37.949 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 0, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 0, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.950 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 0, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 1668, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 0, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 3, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 2, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-04-02T13:49:37.951 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 7096121, 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 7094033, 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 1668 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 24, 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 663, 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 639, 2024-04-02T13:49:37.952 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 5, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 109, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 104, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 397, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 899, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 502, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 0, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 0, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 0, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 54648, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 215416 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 0, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 0, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 0, 2024-04-02T13:49:37.953 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 0, 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 0, 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.954 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.955 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.956 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.957 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-04-02T13:49:37.957 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.957 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.957 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.957 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.957 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-04-02T13:49:37.957 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.957 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.957 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.957 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.957 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-04-02T13:49:37.958 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.958 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.958 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.958 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.958 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-04-02T13:49:37.958 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.958 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.958 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.958 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.958 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-04-02T13:49:37.958 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.958 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.959 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-04-02T13:49:37.960 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.961 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-04-02T13:49:37.962 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.963 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 1, 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 4, 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 3, 2024-04-02T13:49:37.964 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 3, 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 0, 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 0, 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 4051, 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:49:37.965 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:49:37.966 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 181313, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 47, 2024-04-02T13:49:37.967 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 9512, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 182, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 344348, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 2221, 2024-04-02T13:49:37.968 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 2285, 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 21, 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 3, 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 2285, 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2285, 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 1.264972829, 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000553598 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.969 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2285, 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 2286 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 2285, 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 2285, 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 0, 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 1450, 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 0, 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 833, 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 0, 2024-04-02T13:49:37.970 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 0, 2024-04-02T13:49:37.971 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-04-02T13:49:37.971 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-04-02T13:49:37.971 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-04-02T13:49:37.971 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 0, 2024-04-02T13:49:37.971 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 0, 2024-04-02T13:49:37.971 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-04-02T13:49:37.971 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-04-02T13:49:37.971 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-04-02T13:49:37.971 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 1, 2024-04-02T13:49:37.971 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 0, 2024-04-02T13:49:37.971 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:49:37.971 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-04-02T13:49:37.971 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:49:37.972 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-04-02T13:49:37.972 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-04-02T13:49:37.972 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-04-02T13:49:37.972 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:49:37.972 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-04-02T13:49:37.972 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:49:37.972 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-04-02T13:49:37.972 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-04-02T13:49:37.972 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-04-02T13:49:37.972 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-04-02T13:49:37.972 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-04-02T13:49:37.972 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-04-02T13:49:37.972 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 105, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 15, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 4, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 4, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 0, 2024-04-02T13:49:37.973 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 0, 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 2, 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 0 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 0, 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: "osd-slow-ops": { 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 0 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 0, 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 0, 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 0, 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 2553, 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 4984408, 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:37.975 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 2553, 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2553, 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 4984408, 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:37.976 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 2285, 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 2901822, 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2285, 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 2901822, 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-04-02T13:49:37.977 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 2285, 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 2285, 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2285, 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 2285, 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.978 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:37.979 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x56355c34e6e0": { 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-04-02T13:49:37.980 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-04-02T13:49:37.981 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-04-02T13:49:37.981 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:37.981 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:37.981 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:37.981 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:37.981 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:37.981 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:37.981 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-04-02T13:49:38.895 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-02T13:49:38.895 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-02T13:49:38.895 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-02T13:49:38.896 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-02T13:49:38.896 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-02T13:49:38.896 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:49:38.896 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:49:38.896 DEBUG:tasks.check_counter:Getting stats from f 2024-04-02T13:49:38.896 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.f perf dump 2024-04-02T13:49:38.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:38 smithi145 ceph-mon[35646]: pgmap v338: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-02T13:49:38.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:38 smithi050 ceph-mon[35524]: pgmap v338: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-02T13:49:39.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:38 smithi033 ceph-mon[32390]: pgmap v338: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-02T13:49:39.138 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:49:40.873 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:40 smithi145 ceph-mon[35646]: pgmap v339: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s 2024-04-02T13:49:40.884 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-04-02T13:49:40.884 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:49:40.884 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 832, 2024-04-02T13:49:40.884 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 834, 2024-04-02T13:49:40.884 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 4412015, 2024-04-02T13:49:40.884 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 264640, 2024-04-02T13:49:40.884 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-04-02T13:49:40.884 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 2, 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.082480962, 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.024631153, 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.024735358, 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.024555939, 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 832, 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.031065530, 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000037338 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.885 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.886 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T13:49:40.886 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T13:49:40.886 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.886 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:49:40.886 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 953, 2024-04-02T13:49:40.886 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 930, 2024-04-02T13:49:40.886 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1876165, 2024-04-02T13:49:40.886 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 358323, 2024-04-02T13:49:40.886 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 4, 2024-04-02T13:49:40.886 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2024-04-02T13:49:40.886 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.112222394, 2024-04-02T13:49:40.886 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.034085626, 2024-04-02T13:49:40.886 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.054821957, 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.027674758, 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 929, 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.035680856, 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000038407 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1670768, 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 139216 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.887 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:49:40.888 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 778, 2024-04-02T13:49:40.888 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 943, 2024-04-02T13:49:40.888 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 262752, 2024-04-02T13:49:40.888 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 698744, 2024-04-02T13:49:40.888 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-04-02T13:49:40.888 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 4, 2024-04-02T13:49:40.888 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.100736040, 2024-04-02T13:49:40.888 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.034828622, 2024-04-02T13:49:40.888 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.025429642, 2024-04-02T13:49:40.888 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.028886348, 2024-04-02T13:49:40.888 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:40.888 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 941, 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.063046151, 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000066999 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 453248 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-04-02T13:49:40.889 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3769, 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.109827083, 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000029139 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.890 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 0, 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 0, 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 0, 2024-04-02T13:49:40.891 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 0, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 0, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 0, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 901, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 0, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 901, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 0, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 901, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 1, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 0, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 0, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 1, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 0, 2024-04-02T13:49:40.892 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 0, 2024-04-02T13:49:40.893 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:49:40.894 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 0, 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 0, 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 0, 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 0 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 0, 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 0, 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 0, 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 0, 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 0, 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 0, 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:49:40.895 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 0, 2024-04-02T13:49:40.896 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 0, 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 2031, 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 0, 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 4, 2024-04-02T13:49:40.897 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 2, 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 8417401, 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 8415313, 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 2031 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 211, 2024-04-02T13:49:40.898 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 963, 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 752, 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 19, 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 117, 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 98, 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 901, 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 1361, 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 460, 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 0, 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 0, 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 0, 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 52836, 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 215416 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-04-02T13:49:40.899 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 0, 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 0, 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 0, 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 0, 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 0, 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.900 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-04-02T13:49:40.901 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.902 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-04-02T13:49:40.903 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.903 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.903 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.903 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.903 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-04-02T13:49:40.903 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.903 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.903 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.903 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.903 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-04-02T13:49:40.903 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.903 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.904 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.904 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.904 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-04-02T13:49:40.904 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.904 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.904 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.904 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.905 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.906 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.907 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.908 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.909 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.910 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 2, 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 6, 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 4, 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 5, 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 0, 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 0, 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 3486, 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:49:40.911 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-04-02T13:49:40.912 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-04-02T13:49:40.913 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 189433, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 52, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:49:40.914 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 9512, 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 182, 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 1120685, 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 10591, 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.915 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 2293, 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 21, 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 3, 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 2293, 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2293, 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 1.295972162, 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000565186 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-04-02T13:49:40.916 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2293, 2024-04-02T13:49:40.917 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 2294 2024-04-02T13:49:40.917 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.917 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 2293, 2024-04-02T13:49:40.917 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 2293, 2024-04-02T13:49:40.917 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 0, 2024-04-02T13:49:40.917 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-04-02T13:49:40.917 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-04-02T13:49:40.917 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 1455, 2024-04-02T13:49:40.917 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 0, 2024-04-02T13:49:40.917 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 836, 2024-04-02T13:49:40.917 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 0, 2024-04-02T13:49:40.917 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 1, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-04-02T13:49:40.918 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-04-02T13:49:40.919 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 105, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 15, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 5, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 5, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 0, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 0, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 2, 2024-04-02T13:49:40.920 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 0 2024-04-02T13:49:40.921 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.921 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-04-02T13:49:40.921 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 0, 2024-04-02T13:49:40.921 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:49:40.921 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-04-02T13:49:40.921 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-04-02T13:49:40.921 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.921 INFO:teuthology.orchestra.run.smithi145.stdout: "osd-slow-ops": { 2024-04-02T13:49:40.921 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 0 2024-04-02T13:49:40.921 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.921 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-04-02T13:49:40.921 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-04-02T13:49:40.921 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 0, 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 0, 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 0, 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 2559, 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 6304079, 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 2559, 2024-04-02T13:49:40.922 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2559, 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 6304079, 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-04-02T13:49:40.923 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-04-02T13:49:40.924 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:40.924 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:40.924 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 2293, 2024-04-02T13:49:40.924 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 4223102, 2024-04-02T13:49:40.924 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2293, 2024-04-02T13:49:40.924 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 4223102, 2024-04-02T13:49:40.924 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:40.924 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.924 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.924 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.924 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:40.924 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.925 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-04-02T13:49:40.925 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:40.925 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-04-02T13:49:40.925 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:40.925 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-04-02T13:49:40.925 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-04-02T13:49:40.925 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:40.926 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:40.926 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 2293, 2024-04-02T13:49:40.926 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 2293, 2024-04-02T13:49:40.926 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2293, 2024-04-02T13:49:40.926 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 2293, 2024-04-02T13:49:40.926 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:40.926 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.926 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.926 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.926 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:40.926 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.926 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:49:40.927 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:40.927 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-04-02T13:49:40.927 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:40.927 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-04-02T13:49:40.927 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-04-02T13:49:40.927 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x55780a1986e0": { 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-04-02T13:49:40.928 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:40.929 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-04-02T13:49:40.929 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-04-02T13:49:40.929 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:40.929 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:40.929 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-04-02T13:49:40.929 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-04-02T13:49:40.929 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-04-02T13:49:40.929 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-04-02T13:49:40.929 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:40.929 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:40.929 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:40.930 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:40.930 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:40.930 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:40.930 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-04-02T13:49:40.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:40 smithi050 ceph-mon[35524]: pgmap v339: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s 2024-04-02T13:49:41.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:40 smithi033 ceph-mon[32390]: pgmap v339: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s 2024-04-02T13:49:41.899 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-02T13:49:41.899 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-02T13:49:41.899 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-02T13:49:41.900 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-02T13:49:41.900 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-02T13:49:41.900 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:49:41.900 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:49:41.900 DEBUG:tasks.check_counter:Getting stats from i 2024-04-02T13:49:41.900 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.i perf dump 2024-04-02T13:49:42.139 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:49:42.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:42 smithi145 ceph-mon[35646]: pgmap v340: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-02T13:49:42.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:42 smithi050 ceph-mon[35524]: pgmap v340: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-02T13:49:43.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:42 smithi033 ceph-mon[32390]: pgmap v340: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-02T13:49:43.862 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-04-02T13:49:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:49:43.862 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 2338, 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 2312, 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 40789967, 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 797318, 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 7, 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.250236304, 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.071542176, 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.102548494, 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.070098118, 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2311, 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.085720778, 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000037092 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.863 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1681344, 2024-04-02T13:49:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 138736 2024-04-02T13:49:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:49:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 216, 2024-04-02T13:49:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 380, 2024-04-02T13:49:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 50397251, 2024-04-02T13:49:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 521534, 2024-04-02T13:49:43.864 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 8, 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 6, 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.046145568, 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.011273581, 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.027642524, 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.004207094, 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 378, 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.020914739, 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000055329 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.865 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 454688 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 353, 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 355, 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 33645426, 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 112261, 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 5, 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.043357833, 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.010384548, 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.020840326, 2024-04-02T13:49:43.866 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.009590963, 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 353, 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.013539771, 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000038356 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-04-02T13:49:43.867 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4211, 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.117633688, 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000027934 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-04-02T13:49:43.868 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 0, 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 0, 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 0, 2024-04-02T13:49:43.869 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 0, 2024-04-02T13:49:43.870 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-04-02T13:49:43.870 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 0, 2024-04-02T13:49:43.870 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 0, 2024-04-02T13:49:43.870 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-04-02T13:49:43.870 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 26445, 2024-04-02T13:49:43.870 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 0, 2024-04-02T13:49:43.870 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 26445, 2024-04-02T13:49:43.870 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 0, 2024-04-02T13:49:43.870 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 26445, 2024-04-02T13:49:43.870 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 0, 2024-04-02T13:49:43.870 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 0, 2024-04-02T13:49:43.870 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 0, 2024-04-02T13:49:43.870 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 2, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-04-02T13:49:43.871 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 0, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 4, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 234, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 6, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 0, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 0, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T13:49:43.872 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 0, 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 0, 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 0, 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 0, 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 0 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 0, 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 0, 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 0, 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 0, 2024-04-02T13:49:43.873 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:49:43.874 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 0, 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 0, 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 0, 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 50233, 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-04-02T13:49:43.875 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 0, 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 71, 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 6, 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 126593047, 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 126592911, 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.876 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 50233 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 7762, 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 18670, 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 10908, 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 744, 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 787, 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 43, 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 26445, 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 27101, 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 656, 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 0, 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 0, 2024-04-02T13:49:43.877 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 0, 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 96028, 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 215416 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 0, 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 0, 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 0, 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 0, 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 0, 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.878 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.879 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-04-02T13:49:43.880 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-04-02T13:49:43.881 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.882 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.882 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.882 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.882 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-04-02T13:49:43.882 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.882 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.882 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.882 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.883 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.884 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.885 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.886 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-04-02T13:49:43.887 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.888 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 2, 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 7, 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 5, 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 5, 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 0, 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 0, 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 3489, 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-04-02T13:49:43.889 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:49:43.890 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:49:43.891 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 230577, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 73, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 9512, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 182, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:49:43.892 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 36583662, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 327658, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 2634, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 26, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 4, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 2634, 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-04-02T13:49:43.893 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2634, 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 1.447968880, 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000549722 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2634, 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 2635 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 2634, 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 2634, 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 0, 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 1680, 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 0, 2024-04-02T13:49:43.894 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 951, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 2, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-04-02T13:49:43.895 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-04-02T13:49:43.896 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 105, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 15, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 10, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 10, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 0, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 0, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 2, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 0 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 0, 2024-04-02T13:49:43.897 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: "osd-slow-ops": { 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 0 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 0, 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 0, 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 0, 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.898 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 2903, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 124554706, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 2903, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2903, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 124554706, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.899 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 2634, 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 122413376, 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2634, 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 122413376, 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:43.900 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 2634, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 2634, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2634, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 2634, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.901 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.902 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x555aef3e26e0": { 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:43.903 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:43.904 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:43.904 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:43.904 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:43.904 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:43.904 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-04-02T13:49:44.680 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-02T13:49:44.680 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-02T13:49:44.680 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-02T13:49:44.680 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-04-02T13:49:44.680 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-02T13:49:44.680 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:49:44.680 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:49:44.680 DEBUG:tasks.check_counter:Getting stats from l 2024-04-02T13:49:44.681 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.l perf dump 2024-04-02T13:49:44.921 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:49:44.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:44 smithi145 ceph-mon[35646]: pgmap v341: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s 2024-04-02T13:49:44.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:44 smithi050 ceph-mon[35524]: pgmap v341: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s 2024-04-02T13:49:45.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:44 smithi033 ceph-mon[32390]: pgmap v341: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s 2024-04-02T13:49:46.724 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:46 smithi145 ceph-mon[35646]: pgmap v342: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 4, 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 4, 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 3040, 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 448, 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 4, 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 0, 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.001224816, 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.000029907, 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.000473118, 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3, 2024-04-02T13:49:46.725 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000648261, 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000216087 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 448 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 237, 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 226, 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1594064, 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 138944, 2024-04-02T13:49:46.726 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 2, 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 1, 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.048648038, 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.015376496, 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.035729590, 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.005873213, 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 225, 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.012439144, 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000055285 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1594064, 2024-04-02T13:49:46.727 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 138944 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 2, 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 173, 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 320, 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 186080, 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 5, 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.011720823, 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.007683676, 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.001517627, 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 170, 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.014623261, 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000086019 2024-04-02T13:49:46.728 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 186080 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 0, 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-04-02T13:49:46.729 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-04-02T13:49:46.730 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-04-02T13:49:46.731 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 148754, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 31, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 0, 2024-04-02T13:49:46.732 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 0, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 928, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 2, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 239, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 1560618, 2024-04-02T13:49:46.733 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-04-02T13:49:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 239, 2024-04-02T13:49:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-04-02T13:49:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-04-02T13:49:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 239, 2024-04-02T13:49:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 1560618, 2024-04-02T13:49:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-04-02T13:49:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-04-02T13:49:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-04-02T13:49:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-04-02T13:49:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:46.734 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-04-02T13:49:46.734 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-04-02T13:49:46.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:46 smithi050 ceph-mon[35524]: pgmap v342: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:47.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:46 smithi033 ceph-mon[32390]: pgmap v342: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:47.565 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-02T13:49:47.565 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-02T13:49:47.565 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-02T13:49:47.565 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-02T13:49:47.565 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-02T13:49:47.565 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-02T13:49:47.565 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-02T13:49:47.565 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-02T13:49:47.576 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-02T13:49:47.576 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:49:47.576 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T13:49:47.597 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:49:47.597 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T13:49:47.657 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:49:47.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.811+0000 7f8e4d99d640 1 -- 172.21.15.33:0/327394704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e48106e70 msgr2=0x7f8e4810e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:47.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.811+0000 7f8e4d99d640 1 --2- 172.21.15.33:0/327394704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e48106e70 0x7f8e4810e9f0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009ca0 tx=0x7f8e4002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:49:47.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.811+0000 7f8e4d99d640 1 -- 172.21.15.33:0/327394704 shutdown_connections 2024-04-02T13:49:47.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.811+0000 7f8e4d99d640 1 --2- 172.21.15.33:0/327394704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e48106e70 0x7f8e4810e9f0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:47.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.811+0000 7f8e4d99d640 1 --2- 172.21.15.33:0/327394704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e48100c80 0x7f8e48106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:47.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.811+0000 7f8e4d99d640 1 --2- 172.21.15.33:0/327394704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e481002d0 0x7f8e481006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:47.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.811+0000 7f8e4d99d640 1 -- 172.21.15.33:0/327394704 >> 172.21.15.33:0/327394704 conn(0x7f8e48076f90 msgr2=0x7f8e480fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:47.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.811+0000 7f8e4d99d640 1 -- 172.21.15.33:0/327394704 shutdown_connections 2024-04-02T13:49:47.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.811+0000 7f8e4d99d640 1 -- 172.21.15.33:0/327394704 wait complete. 2024-04-02T13:49:47.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.812+0000 7f8e4d99d640 1 Processor -- start 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.812+0000 7f8e4d99d640 1 -- start start 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.812+0000 7f8e4d99d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e481002d0 0x7f8e481983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.812+0000 7f8e4d99d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e48106e70 0x7f8e48198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.812+0000 7f8e4d99d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e4819cca0 0x7f8e4819d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.812+0000 7f8e4d99d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e48111ee0 con 0x7f8e48106e70 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.812+0000 7f8e4d99d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8e48111d60 con 0x7f8e4819cca0 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.812+0000 7f8e4d99d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8e48112060 con 0x7f8e481002d0 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e467fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e48106e70 0x7f8e48198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e467fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e48106e70 0x7f8e48198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49562/0 (socket says 172.21.15.33:49562) 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e467fc640 1 -- 172.21.15.33:0/3149902609 learned_addr learned my addr 172.21.15.33:0/3149902609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e477fe640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e4819cca0 0x7f8e4819d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e46ffd640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e481002d0 0x7f8e481983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e467fc640 1 -- 172.21.15.33:0/3149902609 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e481002d0 msgr2=0x7f8e481983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e467fc640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e481002d0 0x7f8e481983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e467fc640 1 -- 172.21.15.33:0/3149902609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e4819cca0 msgr2=0x7f8e4819d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e467fc640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e4819cca0 0x7f8e4819d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e467fc640 1 -- 172.21.15.33:0/3149902609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4819d790 con 0x7f8e48106e70 2024-04-02T13:49:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e467fc640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e48106e70 0x7f8e48198920 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00c940 tx=0x7f8e3c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e4c99b640 1 -- 172.21.15.33:0/3149902609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c004220 con 0x7f8e48106e70 2024-04-02T13:49:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e4c99b640 1 -- 172.21.15.33:0/3149902609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8e3c0043c0 con 0x7f8e48106e70 2024-04-02T13:49:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e4d99d640 1 -- 172.21.15.33:0/3149902609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4819da80 con 0x7f8e48106e70 2024-04-02T13:49:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e4c99b640 1 -- 172.21.15.33:0/3149902609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c004a00 con 0x7f8e48106e70 2024-04-02T13:49:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.813+0000 7f8e4d99d640 1 -- 172.21.15.33:0/3149902609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e481a51c0 con 0x7f8e48106e70 2024-04-02T13:49:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.814+0000 7f8e4d99d640 1 -- 172.21.15.33:0/3149902609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e48068d40 con 0x7f8e48106e70 2024-04-02T13:49:47.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.815+0000 7f8e4c99b640 1 -- 172.21.15.33:0/3149902609 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e3c004ba0 con 0x7f8e48106e70 2024-04-02T13:49:47.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.816+0000 7f8e4c99b640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8e300a3d20 0x7f8e300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:47.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.816+0000 7f8e46ffd640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8e300a3d20 0x7f8e300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:47.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.816+0000 7f8e4c99b640 1 -- 172.21.15.33:0/3149902609 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f8e3c0d0890 con 0x7f8e48106e70 2024-04-02T13:49:47.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.817+0000 7f8e46ffd640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8e300a3d20 0x7f8e300a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8e38005fd0 tx=0x7f8e38005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:47.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:47.821+0000 7f8e4c99b640 1 -- 172.21.15.33:0/3149902609 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8e3c014030 con 0x7f8e48106e70 2024-04-02T13:49:48.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.011+0000 7f8e4d99d640 1 -- 172.21.15.33:0/3149902609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e48100cb0 con 0x7f8e48106e70 2024-04-02T13:49:48.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.013+0000 7f8e4c99b640 1 -- 172.21.15.33:0/3149902609 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+25182 (secure 0 0 0) 0x7f8e3c09cb60 con 0x7f8e48106e70 2024-04-02T13:49:48.016 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:49:48.016 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:49:11.632123+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/80287983":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/395956530":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3947066882":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1212609914":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:49:48.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.015+0000 7f8e4d99d640 1 -- 172.21.15.33:0/3149902609 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8e300a3d20 msgr2=0x7f8e300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.015+0000 7f8e4d99d640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8e300a3d20 0x7f8e300a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8e38005fd0 tx=0x7f8e38005850 comp rx=0 tx=0).stop 2024-04-02T13:49:48.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.015+0000 7f8e4d99d640 1 -- 172.21.15.33:0/3149902609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e48106e70 msgr2=0x7f8e48198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.015+0000 7f8e4d99d640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e48106e70 0x7f8e48198920 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00c940 tx=0x7f8e3c00ce00 comp rx=0 tx=0).stop 2024-04-02T13:49:48.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.015+0000 7f8e4d99d640 1 -- 172.21.15.33:0/3149902609 shutdown_connections 2024-04-02T13:49:48.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.015+0000 7f8e4d99d640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8e300a3d20 0x7f8e300a61e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.015+0000 7f8e4d99d640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e4819cca0 0x7f8e4819d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.015+0000 7f8e4d99d640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e48106e70 0x7f8e48198920 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.015+0000 7f8e4d99d640 1 --2- 172.21.15.33:0/3149902609 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e481002d0 0x7f8e481983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.015+0000 7f8e4d99d640 1 -- 172.21.15.33:0/3149902609 >> 172.21.15.33:0/3149902609 conn(0x7f8e48076f90 msgr2=0x7f8e481050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:48.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.015+0000 7f8e4d99d640 1 -- 172.21.15.33:0/3149902609 shutdown_connections 2024-04-02T13:49:48.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.015+0000 7f8e4d99d640 1 -- 172.21.15.33:0/3149902609 wait complete. 2024-04-02T13:49:48.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:36:19.501121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:36:20.161569+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:36:23.482134+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:49:48.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:49:48.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.133+0000 7f306c73d640 1 -- 172.21.15.33:0/2441374107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064102d40 msgr2=0x7f3064103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.133+0000 7f306c73d640 1 --2- 172.21.15.33:0/2441374107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064102d40 0x7f3064103120 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f305c009ca0 tx=0x7f305c02c180 comp rx=0 tx=0).stop 2024-04-02T13:49:48.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.133+0000 7f306c73d640 1 -- 172.21.15.33:0/2441374107 shutdown_connections 2024-04-02T13:49:48.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.133+0000 7f306c73d640 1 --2- 172.21.15.33:0/2441374107 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f306410c110 0x7f306410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.133+0000 7f306c73d640 1 --2- 172.21.15.33:0/2441374107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3064103660 0x7f306410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.133+0000 7f306c73d640 1 --2- 172.21.15.33:0/2441374107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064102d40 0x7f3064103120 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.133+0000 7f306c73d640 1 -- 172.21.15.33:0/2441374107 >> 172.21.15.33:0/2441374107 conn(0x7f3064076d80 msgr2=0x7f3064077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:48.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.133+0000 7f306c73d640 1 -- 172.21.15.33:0/2441374107 shutdown_connections 2024-04-02T13:49:48.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.133+0000 7f306c73d640 1 -- 172.21.15.33:0/2441374107 wait complete. 2024-04-02T13:49:48.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.133+0000 7f306c73d640 1 Processor -- start 2024-04-02T13:49:48.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f306c73d640 1 -- start start 2024-04-02T13:49:48.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f306c73d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3064102d40 0x7f30641983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f306c73d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064103660 0x7f30641988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f306c73d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f306410c110 0x7f306419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f306c73d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3064111e90 con 0x7f3064103660 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f306c73d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3064111d10 con 0x7f306410c110 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f306c73d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3064112010 con 0x7f3064102d40 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f3069cd8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064103660 0x7f30641988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f3069cd8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064103660 0x7f30641988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49578/0 (socket says 172.21.15.33:49578) 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f3069cd8640 1 -- 172.21.15.33:0/873355793 learned_addr learned my addr 172.21.15.33:0/873355793 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f306a4d9640 1 --2- 172.21.15.33:0/873355793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3064102d40 0x7f30641983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f3069cd8640 1 -- 172.21.15.33:0/873355793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3064102d40 msgr2=0x7f30641983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f3069cd8640 1 --2- 172.21.15.33:0/873355793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3064102d40 0x7f30641983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f3069cd8640 1 -- 172.21.15.33:0/873355793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f306410c110 msgr2=0x7f306419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f3069cd8640 1 --2- 172.21.15.33:0/873355793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f306410c110 0x7f306419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f3069cd8640 1 -- 172.21.15.33:0/873355793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306419d3a0 con 0x7f3064103660 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f3069cd8640 1 --2- 172.21.15.33:0/873355793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064103660 0x7f30641988f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f305800d770 tx=0x7f305800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:48.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f30637fe640 1 -- 172.21.15.33:0/873355793 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058017070 con 0x7f3064103660 2024-04-02T13:49:48.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f30637fe640 1 -- 172.21.15.33:0/873355793 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f30580054f0 con 0x7f3064103660 2024-04-02T13:49:48.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f306c73d640 1 -- 172.21.15.33:0/873355793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306419d690 con 0x7f3064103660 2024-04-02T13:49:48.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.134+0000 7f30637fe640 1 -- 172.21.15.33:0/873355793 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058004d60 con 0x7f3064103660 2024-04-02T13:49:48.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.135+0000 7f306c73d640 1 -- 172.21.15.33:0/873355793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30641a4f20 con 0x7f3064103660 2024-04-02T13:49:48.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.135+0000 7f306c73d640 1 -- 172.21.15.33:0/873355793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f306406a8f0 con 0x7f3064103660 2024-04-02T13:49:48.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.137+0000 7f30637fe640 1 -- 172.21.15.33:0/873355793 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3058004f00 con 0x7f3064103660 2024-04-02T13:49:48.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.137+0000 7f30637fe640 1 --2- 172.21.15.33:0/873355793 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f304c0a3d20 0x7f304c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:48.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.137+0000 7f306a4d9640 1 --2- 172.21.15.33:0/873355793 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f304c0a3d20 0x7f304c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:48.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.137+0000 7f30637fe640 1 -- 172.21.15.33:0/873355793 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f30580d0a10 con 0x7f3064103660 2024-04-02T13:49:48.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.138+0000 7f306a4d9640 1 --2- 172.21.15.33:0/873355793 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f304c0a3d20 0x7f304c0a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f305c004af0 tx=0x7f305c005c50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:48.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.141+0000 7f30637fe640 1 -- 172.21.15.33:0/873355793 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f305809cce0 con 0x7f3064103660 2024-04-02T13:49:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.325+0000 7f306c73d640 1 -- 172.21.15.33:0/873355793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3064199700 con 0x7f3064103660 2024-04-02T13:49:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.326+0000 7f30637fe640 1 -- 172.21.15.33:0/873355793 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f305809c5e0 con 0x7f3064103660 2024-04-02T13:49:48.330 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:49:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.328+0000 7f306c73d640 1 -- 172.21.15.33:0/873355793 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f304c0a3d20 msgr2=0x7f304c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.328+0000 7f306c73d640 1 --2- 172.21.15.33:0/873355793 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f304c0a3d20 0x7f304c0a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f305c004af0 tx=0x7f305c005c50 comp rx=0 tx=0).stop 2024-04-02T13:49:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.328+0000 7f306c73d640 1 -- 172.21.15.33:0/873355793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064103660 msgr2=0x7f30641988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.328+0000 7f306c73d640 1 --2- 172.21.15.33:0/873355793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064103660 0x7f30641988f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f305800d770 tx=0x7f305800dc30 comp rx=0 tx=0).stop 2024-04-02T13:49:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.328+0000 7f306c73d640 1 -- 172.21.15.33:0/873355793 shutdown_connections 2024-04-02T13:49:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.328+0000 7f306c73d640 1 --2- 172.21.15.33:0/873355793 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f304c0a3d20 0x7f304c0a61e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.328+0000 7f306c73d640 1 --2- 172.21.15.33:0/873355793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f306410c110 0x7f306419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.328+0000 7f306c73d640 1 --2- 172.21.15.33:0/873355793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064103660 0x7f30641988f0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.328+0000 7f306c73d640 1 --2- 172.21.15.33:0/873355793 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3064102d40 0x7f30641983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.328+0000 7f306c73d640 1 -- 172.21.15.33:0/873355793 >> 172.21.15.33:0/873355793 conn(0x7f3064076d80 msgr2=0x7f30640fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:48.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.328+0000 7f306c73d640 1 -- 172.21.15.33:0/873355793 shutdown_connections 2024-04-02T13:49:48.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.328+0000 7f306c73d640 1 -- 172.21.15.33:0/873355793 wait complete. 2024-04-02T13:49:48.343 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:49:48.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2141470046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a88101420 msgr2=0x7f3a8810d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 --2- 172.21.15.33:0/2141470046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a88101420 0x7f3a8810d8e0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009cd0 tx=0x7f3a8002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2141470046 shutdown_connections 2024-04-02T13:49:48.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 --2- 172.21.15.33:0/2141470046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a88101420 0x7f3a8810d8e0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 --2- 172.21.15.33:0/2141470046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a88100a60 0x7f3a88100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 --2- 172.21.15.33:0/2141470046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a88106a20 0x7f3a88106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2141470046 >> 172.21.15.33:0/2141470046 conn(0x7f3a88077b60 msgr2=0x7f3a88077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:48.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2141470046 shutdown_connections 2024-04-02T13:49:48.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2141470046 wait complete. 2024-04-02T13:49:48.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 Processor -- start 2024-04-02T13:49:48.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 -- start start 2024-04-02T13:49:48.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a88100a60 0x7f3a88197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:48.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a88101420 0x7f3a88197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a88106a20 0x7f3a88197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88110e00 con 0x7f3a88100a60 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88110c80 con 0x7f3a88106a20 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.445+0000 7f3a8fa93640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88110f80 con 0x7f3a88101420 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8d82f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a88100a60 0x7f3a88197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8d82f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a88100a60 0x7f3a88197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49596/0 (socket says 172.21.15.33:49596) 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8d82f640 1 -- 172.21.15.33:0/2039228190 learned_addr learned my addr 172.21.15.33:0/2039228190 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8e030640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a88106a20 0x7f3a88197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8d02e640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a88101420 0x7f3a88197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8d82f640 1 -- 172.21.15.33:0/2039228190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a88101420 msgr2=0x7f3a88197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8d82f640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a88101420 0x7f3a88197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8d82f640 1 -- 172.21.15.33:0/2039228190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a88106a20 msgr2=0x7f3a88197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8d82f640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a88106a20 0x7f3a88197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8d82f640 1 -- 172.21.15.33:0/2039228190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a881a6830 con 0x7f3a88100a60 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8e030640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a88106a20 0x7f3a88197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:48.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8d82f640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a88100a60 0x7f3a88197050 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400ea20 tx=0x7f3a8400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:48.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a7effd640 1 -- 172.21.15.33:0/2039228190 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400cd80 con 0x7f3a88100a60 2024-04-02T13:49:48.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2039228190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a881a6b20 con 0x7f3a88100a60 2024-04-02T13:49:48.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a7effd640 1 -- 172.21.15.33:0/2039228190 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f3a840054f0 con 0x7f3a88100a60 2024-04-02T13:49:48.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a7effd640 1 -- 172.21.15.33:0/2039228190 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84010490 con 0x7f3a88100a60 2024-04-02T13:49:48.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.446+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2039228190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a881a7070 con 0x7f3a88100a60 2024-04-02T13:49:48.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.447+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2039228190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a88102150 con 0x7f3a88100a60 2024-04-02T13:49:48.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.448+0000 7f3a7effd640 1 -- 172.21.15.33:0/2039228190 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3a84005000 con 0x7f3a88100a60 2024-04-02T13:49:48.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.449+0000 7f3a7effd640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3a740a3d20 0x7f3a740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:48.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.449+0000 7f3a8d02e640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3a740a3d20 0x7f3a740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:48.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.449+0000 7f3a7effd640 1 -- 172.21.15.33:0/2039228190 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f3a840d1290 con 0x7f3a88100a60 2024-04-02T13:49:48.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.449+0000 7f3a8d02e640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3a740a3d20 0x7f3a740a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a78005fd0 tx=0x7f3a78005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.452+0000 7f3a7effd640 1 -- 172.21.15.33:0/2039228190 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3a84053f20 con 0x7f3a88100a60 2024-04-02T13:49:48.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.639+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2039228190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3a8802d370 con 0x7f3a88100a60 2024-04-02T13:49:48.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.640+0000 7f3a7effd640 1 -- 172.21.15.33:0/2039228190 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f3a8409d500 con 0x7f3a88100a60 2024-04-02T13:49:48.644 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:49:48.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.642+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2039228190 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3a740a3d20 msgr2=0x7f3a740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.642+0000 7f3a8fa93640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3a740a3d20 0x7f3a740a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a78005fd0 tx=0x7f3a78005850 comp rx=0 tx=0).stop 2024-04-02T13:49:48.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.642+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2039228190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a88100a60 msgr2=0x7f3a88197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.642+0000 7f3a8fa93640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a88100a60 0x7f3a88197050 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400ea20 tx=0x7f3a8400ee20 comp rx=0 tx=0).stop 2024-04-02T13:49:48.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.642+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2039228190 shutdown_connections 2024-04-02T13:49:48.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.642+0000 7f3a8fa93640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f3a740a3d20 0x7f3a740a61e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.642+0000 7f3a8fa93640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a88106a20 0x7f3a88197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.642+0000 7f3a8fa93640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3a88101420 0x7f3a88197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.642+0000 7f3a8fa93640 1 --2- 172.21.15.33:0/2039228190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a88100a60 0x7f3a88197050 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.642+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2039228190 >> 172.21.15.33:0/2039228190 conn(0x7f3a88077b60 msgr2=0x7f3a881032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:48.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.642+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2039228190 shutdown_connections 2024-04-02T13:49:48.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.642+0000 7f3a8fa93640 1 -- 172.21.15.33:0/2039228190 wait complete. 2024-04-02T13:49:48.657 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:49:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.761+0000 7f471fb52640 1 -- 172.21.15.33:0/1016635961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4718102d40 msgr2=0x7f4718103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.761+0000 7f471fb52640 1 --2- 172.21.15.33:0/1016635961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4718102d40 0x7f4718103120 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f4710009900 tx=0x7f471002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.761+0000 7f471fb52640 1 -- 172.21.15.33:0/1016635961 shutdown_connections 2024-04-02T13:49:48.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.761+0000 7f471fb52640 1 --2- 172.21.15.33:0/1016635961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f471810c110 0x7f471810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.761+0000 7f471fb52640 1 --2- 172.21.15.33:0/1016635961 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4718103660 0x7f471810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.761+0000 7f471fb52640 1 --2- 172.21.15.33:0/1016635961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4718102d40 0x7f4718103120 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.761+0000 7f471fb52640 1 -- 172.21.15.33:0/1016635961 >> 172.21.15.33:0/1016635961 conn(0x7f4718076d80 msgr2=0x7f4718077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:48.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.761+0000 7f471fb52640 1 -- 172.21.15.33:0/1016635961 shutdown_connections 2024-04-02T13:49:48.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.761+0000 7f471fb52640 1 -- 172.21.15.33:0/1016635961 wait complete. 2024-04-02T13:49:48.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471fb52640 1 Processor -- start 2024-04-02T13:49:48.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471fb52640 1 -- start start 2024-04-02T13:49:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471fb52640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4718102d40 0x7f4718100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471fb52640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4718103660 0x7f4718100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471fb52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f471810c110 0x7f4718101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471fb52640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4718111f40 con 0x7f471810c110 2024-04-02T13:49:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471fb52640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4718111dc0 con 0x7f4718102d40 2024-04-02T13:49:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471fb52640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f47181120c0 con 0x7f4718103660 2024-04-02T13:49:48.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471e0ef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f471810c110 0x7f4718101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:48.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471e0ef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f471810c110 0x7f4718101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49610/0 (socket says 172.21.15.33:49610) 2024-04-02T13:49:48.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471e0ef640 1 -- 172.21.15.33:0/3297238236 learned_addr learned my addr 172.21.15.33:0/3297238236 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:48.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471d0ed640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4718103660 0x7f4718100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:48.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471d8ee640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4718102d40 0x7f4718100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:48.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471e0ef640 1 -- 172.21.15.33:0/3297238236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4718103660 msgr2=0x7f4718100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471e0ef640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4718103660 0x7f4718100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471e0ef640 1 -- 172.21.15.33:0/3297238236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4718102d40 msgr2=0x7f4718100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471e0ef640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4718102d40 0x7f4718100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.762+0000 7f471e0ef640 1 -- 172.21.15.33:0/3297238236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47181a6940 con 0x7f471810c110 2024-04-02T13:49:48.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.763+0000 7f471e0ef640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f471810c110 0x7f4718101410 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f470800dac0 tx=0x7f470800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:48.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.763+0000 7f4716ffd640 1 -- 172.21.15.33:0/3297238236 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708017070 con 0x7f471810c110 2024-04-02T13:49:48.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.763+0000 7f471fb52640 1 -- 172.21.15.33:0/3297238236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47181a6c30 con 0x7f471810c110 2024-04-02T13:49:48.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.763+0000 7f4716ffd640 1 -- 172.21.15.33:0/3297238236 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4708003e40 con 0x7f471810c110 2024-04-02T13:49:48.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.763+0000 7f4716ffd640 1 -- 172.21.15.33:0/3297238236 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47080106c0 con 0x7f471810c110 2024-04-02T13:49:48.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.763+0000 7f471fb52640 1 -- 172.21.15.33:0/3297238236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47181a7130 con 0x7f471810c110 2024-04-02T13:49:48.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.764+0000 7f471fb52640 1 -- 172.21.15.33:0/3297238236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4718068d40 con 0x7f471810c110 2024-04-02T13:49:48.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.765+0000 7f4716ffd640 1 -- 172.21.15.33:0/3297238236 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4708003970 con 0x7f471810c110 2024-04-02T13:49:48.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.765+0000 7f4716ffd640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f47000a3d20 0x7f47000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:48.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.765+0000 7f471d8ee640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f47000a3d20 0x7f47000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:48.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.765+0000 7f4716ffd640 1 -- 172.21.15.33:0/3297238236 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f47080d06c0 con 0x7f471810c110 2024-04-02T13:49:48.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.766+0000 7f471d8ee640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f47000a3d20 0x7f47000a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4710006010 tx=0x7f4710000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:48.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.770+0000 7f4716ffd640 1 -- 172.21.15.33:0/3297238236 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4708003c60 con 0x7f471810c110 2024-04-02T13:49:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.958+0000 7f471fb52640 1 -- 172.21.15.33:0/3297238236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4718107200 con 0x7f471810c110 2024-04-02T13:49:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.958+0000 7f4716ffd640 1 -- 172.21.15.33:0/3297238236 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f470809c990 con 0x7f471810c110 2024-04-02T13:49:48.962 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-02T13:49:48.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.960+0000 7f471fb52640 1 -- 172.21.15.33:0/3297238236 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f47000a3d20 msgr2=0x7f47000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.960+0000 7f471fb52640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f47000a3d20 0x7f47000a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4710006010 tx=0x7f4710000bc0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.960+0000 7f471fb52640 1 -- 172.21.15.33:0/3297238236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f471810c110 msgr2=0x7f4718101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:48.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.960+0000 7f471fb52640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f471810c110 0x7f4718101410 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f470800dac0 tx=0x7f470800df80 comp rx=0 tx=0).stop 2024-04-02T13:49:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.960+0000 7f471fb52640 1 -- 172.21.15.33:0/3297238236 shutdown_connections 2024-04-02T13:49:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.960+0000 7f471fb52640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f47000a3d20 0x7f47000a61e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.960+0000 7f471fb52640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f471810c110 0x7f4718101410 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.960+0000 7f471fb52640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4718103660 0x7f4718100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.960+0000 7f471fb52640 1 --2- 172.21.15.33:0/3297238236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4718102d40 0x7f4718100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.960+0000 7f471fb52640 1 -- 172.21.15.33:0/3297238236 >> 172.21.15.33:0/3297238236 conn(0x7f4718076d80 msgr2=0x7f47180fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.960+0000 7f471fb52640 1 -- 172.21.15.33:0/3297238236 shutdown_connections 2024-04-02T13:49:48.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:48.960+0000 7f471fb52640 1 -- 172.21.15.33:0/3297238236 wait complete. 2024-04-02T13:49:48.973 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:48 smithi033 ceph-mon[32390]: pgmap v343: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:48.973 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:48 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3149902609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:49:48.974 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:48 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/873355793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:49:48.974 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:48 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2039228190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:49:48.975 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:49:48.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:48 smithi145 ceph-mon[35646]: pgmap v343: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:48.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:48 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3149902609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:49:48.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:48 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/873355793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:49:48.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:48 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2039228190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:49:48.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:48 smithi050 ceph-mon[35524]: pgmap v343: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:48.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:48 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3149902609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:49:48.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:48 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/873355793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:49:48.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:48 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2039228190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:49:49.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.077+0000 7f8ac58d9640 1 -- 172.21.15.33:0/1574407134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0107d00 msgr2=0x7f8ac01080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.077+0000 7f8ac58d9640 1 --2- 172.21.15.33:0/1574407134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0107d00 0x7f8ac01080e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009930 tx=0x7f8ab002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.077+0000 7f8ac58d9640 1 -- 172.21.15.33:0/1574407134 shutdown_connections 2024-04-02T13:49:49.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.077+0000 7f8ac58d9640 1 --2- 172.21.15.33:0/1574407134 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ac0100d80 0x7f8ac010e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.077+0000 7f8ac58d9640 1 --2- 172.21.15.33:0/1574407134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac01003c0 0x7f8ac0100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.077+0000 7f8ac58d9640 1 --2- 172.21.15.33:0/1574407134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0107d00 0x7f8ac01080e0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.077+0000 7f8ac58d9640 1 -- 172.21.15.33:0/1574407134 >> 172.21.15.33:0/1574407134 conn(0x7f8ac0075d40 msgr2=0x7f8ac0076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.077+0000 7f8ac58d9640 1 -- 172.21.15.33:0/1574407134 shutdown_connections 2024-04-02T13:49:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.077+0000 7f8ac58d9640 1 -- 172.21.15.33:0/1574407134 wait complete. 2024-04-02T13:49:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8ac58d9640 1 Processor -- start 2024-04-02T13:49:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8ac58d9640 1 -- start start 2024-04-02T13:49:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8ac58d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac01003c0 0x7f8ac01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8ac58d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0100d80 0x7f8ac0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8ac58d9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ac0107d00 0x7f8ac019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8ac58d9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac0111f30 con 0x7f8ac0100d80 2024-04-02T13:49:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8ac58d9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac0111db0 con 0x7f8ac01003c0 2024-04-02T13:49:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8ac58d9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac01120b0 con 0x7f8ac0107d00 2024-04-02T13:49:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8abe7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0100d80 0x7f8ac0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8abe7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0100d80 0x7f8ac0198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49636/0 (socket says 172.21.15.33:49636) 2024-04-02T13:49:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8abe7fc640 1 -- 172.21.15.33:0/994266889 learned_addr learned my addr 172.21.15.33:0/994266889 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8abf7fe640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ac0107d00 0x7f8ac019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8abeffd640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac01003c0 0x7f8ac01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8abe7fc640 1 -- 172.21.15.33:0/994266889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ac0107d00 msgr2=0x7f8ac019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8abe7fc640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ac0107d00 0x7f8ac019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8abe7fc640 1 -- 172.21.15.33:0/994266889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac01003c0 msgr2=0x7f8ac01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8abe7fc640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac01003c0 0x7f8ac01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8abe7fc640 1 -- 172.21.15.33:0/994266889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac019d3c0 con 0x7f8ac0100d80 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.078+0000 7f8abf7fe640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ac0107d00 0x7f8ac019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.079+0000 7f8abe7fc640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0100d80 0x7f8ac0198910 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400c920 tx=0x7f8ab400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.079+0000 7f8ac48d7640 1 -- 172.21.15.33:0/994266889 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4007c70 con 0x7f8ac0100d80 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.079+0000 7f8ac58d9640 1 -- 172.21.15.33:0/994266889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac019d6b0 con 0x7f8ac0100d80 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.079+0000 7f8ac48d7640 1 -- 172.21.15.33:0/994266889 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f8ab4007e10 con 0x7f8ac0100d80 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.079+0000 7f8ac48d7640 1 -- 172.21.15.33:0/994266889 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4018850 con 0x7f8ac0100d80 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.079+0000 7f8ac58d9640 1 -- 172.21.15.33:0/994266889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac01a4f40 con 0x7f8ac0100d80 2024-04-02T13:49:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.080+0000 7f8ac58d9640 1 -- 172.21.15.33:0/994266889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac0069c10 con 0x7f8ac0100d80 2024-04-02T13:49:49.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.081+0000 7f8ac48d7640 1 -- 172.21.15.33:0/994266889 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8ab4018a30 con 0x7f8ac0100d80 2024-04-02T13:49:49.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.081+0000 7f8ac48d7640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8aa80a3d20 0x7f8aa80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:49.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.081+0000 7f8abeffd640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8aa80a3d20 0x7f8aa80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:49.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.081+0000 7f8ac48d7640 1 -- 172.21.15.33:0/994266889 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f8ab40d0dd0 con 0x7f8ac0100d80 2024-04-02T13:49:49.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.082+0000 7f8abeffd640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8aa80a3d20 0x7f8aa80a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0006010 tx=0x7f8ab002c7f0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:49.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.086+0000 7f8ac48d7640 1 -- 172.21.15.33:0/994266889 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8ab4014030 con 0x7f8ac0100d80 2024-04-02T13:49:49.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.271+0000 7f8ac58d9640 1 -- 172.21.15.33:0/994266889 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8ac0199720 con 0x7f8ac0100d80 2024-04-02T13:49:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.272+0000 7f8ac48d7640 1 -- 172.21.15.33:0/994266889 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) ==== 103+0+11 (secure 0 0 0) 0x7f8ab400de20 con 0x7f8ac0100d80 2024-04-02T13:49:49.276 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:49:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.274+0000 7f8ac58d9640 1 -- 172.21.15.33:0/994266889 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8aa80a3d20 msgr2=0x7f8aa80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.274+0000 7f8ac58d9640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8aa80a3d20 0x7f8aa80a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0006010 tx=0x7f8ab002c7f0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.274+0000 7f8ac58d9640 1 -- 172.21.15.33:0/994266889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0100d80 msgr2=0x7f8ac0198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.274+0000 7f8ac58d9640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0100d80 0x7f8ac0198910 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400c920 tx=0x7f8ab400cde0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.274+0000 7f8ac58d9640 1 -- 172.21.15.33:0/994266889 shutdown_connections 2024-04-02T13:49:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.274+0000 7f8ac58d9640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f8aa80a3d20 0x7f8aa80a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.274+0000 7f8ac58d9640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ac0107d00 0x7f8ac019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.274+0000 7f8ac58d9640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0100d80 0x7f8ac0198910 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.274+0000 7f8ac58d9640 1 --2- 172.21.15.33:0/994266889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ac01003c0 0x7f8ac01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.274+0000 7f8ac58d9640 1 -- 172.21.15.33:0/994266889 >> 172.21.15.33:0/994266889 conn(0x7f8ac0075d40 msgr2=0x7f8ac00fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.274+0000 7f8ac58d9640 1 -- 172.21.15.33:0/994266889 shutdown_connections 2024-04-02T13:49:49.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.274+0000 7f8ac58d9640 1 -- 172.21.15.33:0/994266889 wait complete. 2024-04-02T13:49:49.289 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T13:49:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.389+0000 7f396380d640 1 -- 172.21.15.33:0/2460130444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c100a20 msgr2=0x7f395c100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.389+0000 7f396380d640 1 --2- 172.21.15.33:0/2460130444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c100a20 0x7f395c100ea0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f394c009930 tx=0x7f394c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.389+0000 7f396380d640 1 -- 172.21.15.33:0/2460130444 shutdown_connections 2024-04-02T13:49:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.389+0000 7f396380d640 1 --2- 172.21.15.33:0/2460130444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f395c1013e0 0x7f395c10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.389+0000 7f396380d640 1 --2- 172.21.15.33:0/2460130444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c100a20 0x7f395c100ea0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.389+0000 7f396380d640 1 --2- 172.21.15.33:0/2460130444 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f395c106a30 0x7f395c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.389+0000 7f396380d640 1 -- 172.21.15.33:0/2460130444 >> 172.21.15.33:0/2460130444 conn(0x7f395c077b60 msgr2=0x7f395c077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.389+0000 7f396380d640 1 -- 172.21.15.33:0/2460130444 shutdown_connections 2024-04-02T13:49:49.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.389+0000 7f396380d640 1 -- 172.21.15.33:0/2460130444 wait complete. 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f396380d640 1 Processor -- start 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f396380d640 1 -- start start 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f396380d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f395c100a20 0x7f395c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f396380d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c1013e0 0x7f395c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f396380d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f395c106a30 0x7f395c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f396380d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f395c110e10 con 0x7f395c1013e0 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f396380d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f395c110c90 con 0x7f395c100a20 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f396380d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f395c110f90 con 0x7f395c106a30 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f3960da8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c1013e0 0x7f395c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f3960da8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c1013e0 0x7f395c197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49644/0 (socket says 172.21.15.33:49644) 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f3960da8640 1 -- 172.21.15.33:0/1054828711 learned_addr learned my addr 172.21.15.33:0/1054828711 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f39615a9640 1 --2- 172.21.15.33:0/1054828711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f395c100a20 0x7f395c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f3960da8640 1 -- 172.21.15.33:0/1054828711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f395c106a30 msgr2=0x7f395c19bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f3960da8640 1 --2- 172.21.15.33:0/1054828711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f395c106a30 0x7f395c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f3960da8640 1 -- 172.21.15.33:0/1054828711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f395c100a20 msgr2=0x7f395c197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f3960da8640 1 --2- 172.21.15.33:0/1054828711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f395c100a20 0x7f395c197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.390+0000 7f3960da8640 1 -- 172.21.15.33:0/1054828711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395c19c320 con 0x7f395c1013e0 2024-04-02T13:49:49.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.391+0000 7f3960da8640 1 --2- 172.21.15.33:0/1054828711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c1013e0 0x7f395c197870 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f394c000c00 tx=0x7f394c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:49.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.391+0000 7f39527fc640 1 -- 172.21.15.33:0/1054828711 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c043020 con 0x7f395c1013e0 2024-04-02T13:49:49.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.391+0000 7f39527fc640 1 -- 172.21.15.33:0/1054828711 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f394c02caa0 con 0x7f395c1013e0 2024-04-02T13:49:49.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.391+0000 7f396380d640 1 -- 172.21.15.33:0/1054828711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395c19c5b0 con 0x7f395c1013e0 2024-04-02T13:49:49.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.391+0000 7f39527fc640 1 -- 172.21.15.33:0/1054828711 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c003bf0 con 0x7f395c1013e0 2024-04-02T13:49:49.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.391+0000 7f396380d640 1 -- 172.21.15.33:0/1054828711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f395c1a3ea0 con 0x7f395c1013e0 2024-04-02T13:49:49.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.392+0000 7f396380d640 1 -- 172.21.15.33:0/1054828711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f395c068d40 con 0x7f395c1013e0 2024-04-02T13:49:49.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.393+0000 7f39527fc640 1 -- 172.21.15.33:0/1054828711 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f394c046020 con 0x7f395c1013e0 2024-04-02T13:49:49.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.394+0000 7f39527fc640 1 --2- 172.21.15.33:0/1054828711 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f39480a3d20 0x7f39480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:49.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.394+0000 7f39527fc640 1 -- 172.21.15.33:0/1054828711 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f394c0f2d60 con 0x7f395c1013e0 2024-04-02T13:49:49.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.394+0000 7f39615a9640 1 --2- 172.21.15.33:0/1054828711 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f39480a3d20 0x7f39480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:49.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.394+0000 7f39615a9640 1 --2- 172.21.15.33:0/1054828711 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f39480a3d20 0x7f39480a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3958005d80 tx=0x7f3958005c90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:49.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.398+0000 7f39527fc640 1 -- 172.21.15.33:0/1054828711 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f394c04d070 con 0x7f395c1013e0 2024-04-02T13:49:49.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.602+0000 7f396380d640 1 -- 172.21.15.33:0/1054828711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f395c198680 con 0x7f395c1013e0 2024-04-02T13:49:49.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.602+0000 7f39527fc640 1 -- 172.21.15.33:0/1054828711 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v105) ==== 81+0+942 (secure 0 0 0) 0x7f394c0473b0 con 0x7f395c1013e0 2024-04-02T13:49:49.606 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/80287983 2024-04-03T13:38:46.867289+0000 2024-04-02T13:49:49.606 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1117299953 2024-04-03T13:29:15.778774+0000 2024-04-02T13:49:49.606 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2562306694 2024-04-03T13:38:46.867289+0000 2024-04-02T13:49:49.606 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/395956530 2024-04-03T13:38:46.867289+0000 2024-04-02T13:49:49.606 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2272183007 2024-04-03T13:29:15.778774+0000 2024-04-02T13:49:49.607 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2666001000 2024-04-03T13:29:15.778774+0000 2024-04-02T13:49:49.607 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2666001000 2024-04-03T13:29:15.778774+0000 2024-04-02T13:49:49.607 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2869265288 2024-04-03T13:28:48.425104+0000 2024-04-02T13:49:49.607 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3947066882 2024-04-03T13:38:46.867289+0000 2024-04-02T13:49:49.607 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3041616812 2024-04-03T13:28:48.425104+0000 2024-04-02T13:49:49.607 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2562306694 2024-04-03T13:38:46.867289+0000 2024-04-02T13:49:49.607 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2869265288 2024-04-03T13:28:48.425104+0000 2024-04-02T13:49:49.607 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1212609914 2024-04-03T13:38:46.867289+0000 2024-04-02T13:49:49.607 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3449207963 2024-04-03T13:28:48.425104+0000 2024-04-02T13:49:49.607 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/917705077 2024-04-03T13:28:48.425104+0000 2024-04-02T13:49:49.607 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1792579346 2024-04-03T13:29:15.778774+0000 2024-04-02T13:49:49.607 INFO:teuthology.orchestra.run.smithi033.stderr:listed 16 entries 2024-04-02T13:49:49.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.604+0000 7f396380d640 1 -- 172.21.15.33:0/1054828711 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f39480a3d20 msgr2=0x7f39480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.604+0000 7f396380d640 1 --2- 172.21.15.33:0/1054828711 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f39480a3d20 0x7f39480a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3958005d80 tx=0x7f3958005c90 comp rx=0 tx=0).stop 2024-04-02T13:49:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.604+0000 7f396380d640 1 -- 172.21.15.33:0/1054828711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c1013e0 msgr2=0x7f395c197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.604+0000 7f396380d640 1 --2- 172.21.15.33:0/1054828711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c1013e0 0x7f395c197870 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f394c000c00 tx=0x7f394c0045e0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.604+0000 7f396380d640 1 -- 172.21.15.33:0/1054828711 shutdown_connections 2024-04-02T13:49:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.604+0000 7f396380d640 1 --2- 172.21.15.33:0/1054828711 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f39480a3d20 0x7f39480a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.604+0000 7f396380d640 1 --2- 172.21.15.33:0/1054828711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f395c106a30 0x7f395c19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.604+0000 7f396380d640 1 --2- 172.21.15.33:0/1054828711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f395c1013e0 0x7f395c197870 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.604+0000 7f396380d640 1 --2- 172.21.15.33:0/1054828711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f395c100a20 0x7f395c197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.604+0000 7f396380d640 1 -- 172.21.15.33:0/1054828711 >> 172.21.15.33:0/1054828711 conn(0x7f395c077b60 msgr2=0x7f395c103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.605+0000 7f396380d640 1 -- 172.21.15.33:0/1054828711 shutdown_connections 2024-04-02T13:49:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.605+0000 7f396380d640 1 -- 172.21.15.33:0/1054828711 wait complete. 2024-04-02T13:49:49.619 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-02T13:49:49.619 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T13:49:49.634 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:49:49.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.774+0000 7ff15bc0d640 1 -- 172.21.15.33:0/621796837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff154103690 msgr2=0x7ff15410bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.774+0000 7ff15bc0d640 1 --2- 172.21.15.33:0/621796837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff154103690 0x7ff15410bb70 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7ff148009900 tx=0x7ff14802bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.774+0000 7ff15bc0d640 1 -- 172.21.15.33:0/621796837 shutdown_connections 2024-04-02T13:49:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.774+0000 7ff15bc0d640 1 --2- 172.21.15.33:0/621796837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff15410c140 0x7ff15410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.774+0000 7ff15bc0d640 1 --2- 172.21.15.33:0/621796837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff154103690 0x7ff15410bb70 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.774+0000 7ff15bc0d640 1 --2- 172.21.15.33:0/621796837 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff154102d70 0x7ff154103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.774+0000 7ff15bc0d640 1 -- 172.21.15.33:0/621796837 >> 172.21.15.33:0/621796837 conn(0x7ff154077b00 msgr2=0x7ff154077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.774+0000 7ff15bc0d640 1 -- 172.21.15.33:0/621796837 shutdown_connections 2024-04-02T13:49:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.774+0000 7ff15bc0d640 1 -- 172.21.15.33:0/621796837 wait complete. 2024-04-02T13:49:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.775+0000 7ff15bc0d640 1 Processor -- start 2024-04-02T13:49:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.775+0000 7ff15bc0d640 1 -- start start 2024-04-02T13:49:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.775+0000 7ff15bc0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff154102d70 0x7ff154198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.775+0000 7ff15bc0d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff154103690 0x7ff154198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.775+0000 7ff15bc0d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff15410c140 0x7ff15419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.775+0000 7ff15bc0d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff154111f10 con 0x7ff154102d70 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.775+0000 7ff15bc0d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff154111d90 con 0x7ff15410c140 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.775+0000 7ff15bc0d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff154112090 con 0x7ff154103690 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.775+0000 7ff1599a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff154102d70 0x7ff154198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.775+0000 7ff1599a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff154102d70 0x7ff154198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49664/0 (socket says 172.21.15.33:49664) 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.775+0000 7ff1599a9640 1 -- 172.21.15.33:0/3351770273 learned_addr learned my addr 172.21.15.33:0/3351770273 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.775+0000 7ff15a1aa640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff15410c140 0x7ff15419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.776+0000 7ff1591a8640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff154103690 0x7ff154198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.776+0000 7ff1599a9640 1 -- 172.21.15.33:0/3351770273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff154103690 msgr2=0x7ff154198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.776+0000 7ff1599a9640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff154103690 0x7ff154198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.776+0000 7ff1599a9640 1 -- 172.21.15.33:0/3351770273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff15410c140 msgr2=0x7ff15419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.776+0000 7ff1599a9640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff15410c140 0x7ff15419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.776+0000 7ff1599a9640 1 -- 172.21.15.33:0/3351770273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15419d420 con 0x7ff154102d70 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.776+0000 7ff15a1aa640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff15410c140 0x7ff15419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.776+0000 7ff1599a9640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff154102d70 0x7ff154198430 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00ea80 tx=0x7ff14c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.776+0000 7ff152ffd640 1 -- 172.21.15.33:0/3351770273 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c00ce40 con 0x7ff154102d70 2024-04-02T13:49:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.776+0000 7ff152ffd640 1 -- 172.21.15.33:0/3351770273 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff14c005020 con 0x7ff154102d70 2024-04-02T13:49:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.776+0000 7ff15bc0d640 1 -- 172.21.15.33:0/3351770273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15419d710 con 0x7ff154102d70 2024-04-02T13:49:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.776+0000 7ff152ffd640 1 -- 172.21.15.33:0/3351770273 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0116d0 con 0x7ff154102d70 2024-04-02T13:49:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.776+0000 7ff15bc0d640 1 -- 172.21.15.33:0/3351770273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1541a4fa0 con 0x7ff154102d70 2024-04-02T13:49:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.777+0000 7ff15bc0d640 1 -- 172.21.15.33:0/3351770273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1540688f0 con 0x7ff154102d70 2024-04-02T13:49:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.778+0000 7ff152ffd640 1 -- 172.21.15.33:0/3351770273 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff14c0051d0 con 0x7ff154102d70 2024-04-02T13:49:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.779+0000 7ff152ffd640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff13c0a3da0 0x7ff13c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.779+0000 7ff1591a8640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff13c0a3da0 0x7ff13c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.779+0000 7ff152ffd640 1 -- 172.21.15.33:0/3351770273 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7ff14c0d1380 con 0x7ff154102d70 2024-04-02T13:49:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.779+0000 7ff1591a8640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff13c0a3da0 0x7ff13c0a6260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff15410dec0 tx=0x7ff148000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.782+0000 7ff152ffd640 1 -- 172.21.15.33:0/3351770273 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff14c016030 con 0x7ff154102d70 2024-04-02T13:49:49.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.971+0000 7ff15bc0d640 1 -- 172.21.15.33:0/3351770273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff154199780 con 0x7ff154102d70 2024-04-02T13:49:49.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.973+0000 7ff152ffd640 1 -- 172.21.15.33:0/3351770273 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+25182 (secure 0 0 0) 0x7ff14c09d670 con 0x7ff154102d70 2024-04-02T13:49:49.978 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:49:49.979 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:49:11.632123+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/80287983":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/395956530":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3947066882":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1212609914":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:49:49.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:49 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3297238236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:49:49.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:49 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/994266889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:49:49.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:49 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1054828711' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:49:49.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.975+0000 7ff15bc0d640 1 -- 172.21.15.33:0/3351770273 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff13c0a3da0 msgr2=0x7ff13c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.975+0000 7ff15bc0d640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff13c0a3da0 0x7ff13c0a6260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff15410dec0 tx=0x7ff148000bc0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.975+0000 7ff15bc0d640 1 -- 172.21.15.33:0/3351770273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff154102d70 msgr2=0x7ff154198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:49.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.975+0000 7ff15bc0d640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff154102d70 0x7ff154198430 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00ea80 tx=0x7ff14c00ef40 comp rx=0 tx=0).stop 2024-04-02T13:49:49.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.975+0000 7ff15bc0d640 1 -- 172.21.15.33:0/3351770273 shutdown_connections 2024-04-02T13:49:49.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.975+0000 7ff15bc0d640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff13c0a3da0 0x7ff13c0a6260 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.975+0000 7ff15bc0d640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff15410c140 0x7ff15419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.975+0000 7ff15bc0d640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff154103690 0x7ff154198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.975+0000 7ff15bc0d640 1 --2- 172.21.15.33:0/3351770273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff154102d70 0x7ff154198430 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.975+0000 7ff15bc0d640 1 -- 172.21.15.33:0/3351770273 >> 172.21.15.33:0/3351770273 conn(0x7ff154077b00 msgr2=0x7ff1540fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.975+0000 7ff15bc0d640 1 -- 172.21.15.33:0/3351770273 shutdown_connections 2024-04-02T13:49:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:49.975+0000 7ff15bc0d640 1 -- 172.21.15.33:0/3351770273 wait complete. 2024-04-02T13:49:49.989 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:49 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3297238236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:49:49.989 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:49 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/994266889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:49:49.989 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:49 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1054828711' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:49:49.990 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:31:41.868850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:36:19.501121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:36:20.161569+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:36:23.482134+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:49:49.991 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:49:49.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:49 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3297238236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:49:49.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:49 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/994266889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:49:49.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:49 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1054828711' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:49:50.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.093+0000 7fa39e7cd640 1 -- 172.21.15.33:0/2033807143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa398068ce0 msgr2=0x7fa398069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.093+0000 7fa39e7cd640 1 --2- 172.21.15.33:0/2033807143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa398068ce0 0x7fa398069160 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fa380009930 tx=0x7fa38002bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:50.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.093+0000 7fa39e7cd640 1 -- 172.21.15.33:0/2033807143 shutdown_connections 2024-04-02T13:49:50.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.093+0000 7fa39e7cd640 1 --2- 172.21.15.33:0/2033807143 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3980696a0 0x7fa39810e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.093+0000 7fa39e7cd640 1 --2- 172.21.15.33:0/2033807143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa398068ce0 0x7fa398069160 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.093+0000 7fa39e7cd640 1 --2- 172.21.15.33:0/2033807143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa398107a80 0x7fa398107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.093+0000 7fa39e7cd640 1 -- 172.21.15.33:0/2033807143 >> 172.21.15.33:0/2033807143 conn(0x7fa398077b00 msgr2=0x7fa398077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:50.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.094+0000 7fa39e7cd640 1 -- 172.21.15.33:0/2033807143 shutdown_connections 2024-04-02T13:49:50.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.094+0000 7fa39e7cd640 1 -- 172.21.15.33:0/2033807143 wait complete. 2024-04-02T13:49:50.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.094+0000 7fa39e7cd640 1 Processor -- start 2024-04-02T13:49:50.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.094+0000 7fa39e7cd640 1 -- start start 2024-04-02T13:49:50.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.094+0000 7fa39e7cd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa398068ce0 0x7fa3981983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:50.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.094+0000 7fa39e7cd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3980696a0 0x7fa398198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.094+0000 7fa39e7cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa398107a80 0x7fa39819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.094+0000 7fa39e7cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa398111e60 con 0x7fa398107a80 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.094+0000 7fa39e7cd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa398111ce0 con 0x7fa398068ce0 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.094+0000 7fa39e7cd640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa398111fe0 con 0x7fa3980696a0 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa39cd6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa398107a80 0x7fa39819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa39cd6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa398107a80 0x7fa39819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49682/0 (socket says 172.21.15.33:49682) 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa39cd6a640 1 -- 172.21.15.33:0/552797675 learned_addr learned my addr 172.21.15.33:0/552797675 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa397fff640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa398068ce0 0x7fa3981983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa3977fe640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3980696a0 0x7fa398198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa39cd6a640 1 -- 172.21.15.33:0/552797675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3980696a0 msgr2=0x7fa398198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa39cd6a640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3980696a0 0x7fa398198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa39cd6a640 1 -- 172.21.15.33:0/552797675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa398068ce0 msgr2=0x7fa3981983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa39cd6a640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa398068ce0 0x7fa3981983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa39cd6a640 1 -- 172.21.15.33:0/552797675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39819d3d0 con 0x7fa398107a80 2024-04-02T13:49:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa39cd6a640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa398107a80 0x7fa39819cca0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa39000cad0 tx=0x7fa39000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa3957fa640 1 -- 172.21.15.33:0/552797675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39000ae20 con 0x7fa398107a80 2024-04-02T13:49:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa3957fa640 1 -- 172.21.15.33:0/552797675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa390005020 con 0x7fa398107a80 2024-04-02T13:49:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa39e7cd640 1 -- 172.21.15.33:0/552797675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39819d6c0 con 0x7fa398107a80 2024-04-02T13:49:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.095+0000 7fa3957fa640 1 -- 172.21.15.33:0/552797675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39000f690 con 0x7fa398107a80 2024-04-02T13:49:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.096+0000 7fa39e7cd640 1 -- 172.21.15.33:0/552797675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3981a4f50 con 0x7fa398107a80 2024-04-02T13:49:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.097+0000 7fa39e7cd640 1 -- 172.21.15.33:0/552797675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3981031b0 con 0x7fa398107a80 2024-04-02T13:49:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.098+0000 7fa3957fa640 1 -- 172.21.15.33:0/552797675 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa3900051d0 con 0x7fa398107a80 2024-04-02T13:49:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.098+0000 7fa3957fa640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3840a3d20 0x7fa3840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.098+0000 7fa397fff640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3840a3d20 0x7fa3840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.098+0000 7fa3957fa640 1 -- 172.21.15.33:0/552797675 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7fa3900d0c10 con 0x7fa398107a80 2024-04-02T13:49:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.099+0000 7fa397fff640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3840a3d20 0x7fa3840a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa388005d00 tx=0x7fa38800b410 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.103+0000 7fa3957fa640 1 -- 172.21.15.33:0/552797675 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa390014030 con 0x7fa398107a80 2024-04-02T13:49:50.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.286+0000 7fa39e7cd640 1 -- 172.21.15.33:0/552797675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa398109a70 con 0x7fa398107a80 2024-04-02T13:49:50.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.287+0000 7fa3957fa640 1 -- 172.21.15.33:0/552797675 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7fa39009cee0 con 0x7fa398107a80 2024-04-02T13:49:50.291 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-02T13:49:50.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.289+0000 7fa39e7cd640 1 -- 172.21.15.33:0/552797675 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3840a3d20 msgr2=0x7fa3840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.289+0000 7fa39e7cd640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3840a3d20 0x7fa3840a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa388005d00 tx=0x7fa38800b410 comp rx=0 tx=0).stop 2024-04-02T13:49:50.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.289+0000 7fa39e7cd640 1 -- 172.21.15.33:0/552797675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa398107a80 msgr2=0x7fa39819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.289+0000 7fa39e7cd640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa398107a80 0x7fa39819cca0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa39000cad0 tx=0x7fa39000cf90 comp rx=0 tx=0).stop 2024-04-02T13:49:50.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.289+0000 7fa39e7cd640 1 -- 172.21.15.33:0/552797675 shutdown_connections 2024-04-02T13:49:50.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.289+0000 7fa39e7cd640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa3840a3d20 0x7fa3840a61e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.289+0000 7fa39e7cd640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa398107a80 0x7fa39819cca0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.289+0000 7fa39e7cd640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3980696a0 0x7fa398198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.289+0000 7fa39e7cd640 1 --2- 172.21.15.33:0/552797675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa398068ce0 0x7fa3981983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.289+0000 7fa39e7cd640 1 -- 172.21.15.33:0/552797675 >> 172.21.15.33:0/552797675 conn(0x7fa398077b00 msgr2=0x7fa3980fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:50.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.289+0000 7fa39e7cd640 1 -- 172.21.15.33:0/552797675 shutdown_connections 2024-04-02T13:49:50.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.290+0000 7fa39e7cd640 1 -- 172.21.15.33:0/552797675 wait complete. 2024-04-02T13:49:50.305 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:49:50.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.408+0000 7ff71e7e9640 1 -- 172.21.15.33:0/3575362287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff718107ab0 msgr2=0x7ff718107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.408+0000 7ff71e7e9640 1 --2- 172.21.15.33:0/3575362287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff718107ab0 0x7ff718107e90 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ff708009930 tx=0x7ff70802bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:50.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.408+0000 7ff71e7e9640 1 -- 172.21.15.33:0/3575362287 shutdown_connections 2024-04-02T13:49:50.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.408+0000 7ff71e7e9640 1 --2- 172.21.15.33:0/3575362287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff718102460 0x7ff71810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.408+0000 7ff71e7e9640 1 --2- 172.21.15.33:0/3575362287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff718101aa0 0x7ff718101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.408+0000 7ff71e7e9640 1 --2- 172.21.15.33:0/3575362287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff718107ab0 0x7ff718107e90 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.408+0000 7ff71e7e9640 1 -- 172.21.15.33:0/3575362287 >> 172.21.15.33:0/3575362287 conn(0x7ff7180fd530 msgr2=0x7ff7180ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:50.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.408+0000 7ff71e7e9640 1 -- 172.21.15.33:0/3575362287 shutdown_connections 2024-04-02T13:49:50.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.408+0000 7ff71e7e9640 1 -- 172.21.15.33:0/3575362287 wait complete. 2024-04-02T13:49:50.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.408+0000 7ff71e7e9640 1 Processor -- start 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.408+0000 7ff71e7e9640 1 -- start start 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff71e7e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff718101aa0 0x7ff718198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff71e7e9640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff718102460 0x7ff718198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff71e7e9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff718107ab0 0x7ff71819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff71e7e9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff718111ee0 con 0x7ff718101aa0 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff71e7e9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff718111d60 con 0x7ff718107ab0 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff71e7e9640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff718112060 con 0x7ff718102460 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff717fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff718101aa0 0x7ff718198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff717fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff718101aa0 0x7ff718198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49706/0 (socket says 172.21.15.33:49706) 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff717fff640 1 -- 172.21.15.33:0/1070576503 learned_addr learned my addr 172.21.15.33:0/1070576503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff7177fe640 1 --2- 172.21.15.33:0/1070576503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff718102460 0x7ff718198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff717fff640 1 -- 172.21.15.33:0/1070576503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff718102460 msgr2=0x7ff718198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff717fff640 1 --2- 172.21.15.33:0/1070576503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff718102460 0x7ff718198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff717fff640 1 -- 172.21.15.33:0/1070576503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff718107ab0 msgr2=0x7ff71819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff717fff640 1 --2- 172.21.15.33:0/1070576503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff718107ab0 0x7ff71819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff717fff640 1 -- 172.21.15.33:0/1070576503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71819d410 con 0x7ff718101aa0 2024-04-02T13:49:50.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff717fff640 1 --2- 172.21.15.33:0/1070576503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff718101aa0 0x7ff718198420 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ff708000c00 tx=0x7ff7080045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:50.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.409+0000 7ff7157fa640 1 -- 172.21.15.33:0/1070576503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708043020 con 0x7ff718101aa0 2024-04-02T13:49:50.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.410+0000 7ff7157fa640 1 -- 172.21.15.33:0/1070576503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff70802caa0 con 0x7ff718101aa0 2024-04-02T13:49:50.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.410+0000 7ff71e7e9640 1 -- 172.21.15.33:0/1070576503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71819d6a0 con 0x7ff718101aa0 2024-04-02T13:49:50.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.410+0000 7ff7157fa640 1 -- 172.21.15.33:0/1070576503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708003ae0 con 0x7ff718101aa0 2024-04-02T13:49:50.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.410+0000 7ff71e7e9640 1 -- 172.21.15.33:0/1070576503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7181a4f90 con 0x7ff718101aa0 2024-04-02T13:49:50.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.411+0000 7ff71e7e9640 1 -- 172.21.15.33:0/1070576503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7181031e0 con 0x7ff718101aa0 2024-04-02T13:49:50.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.412+0000 7ff7157fa640 1 -- 172.21.15.33:0/1070576503 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff708046020 con 0x7ff718101aa0 2024-04-02T13:49:50.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.412+0000 7ff7157fa640 1 --2- 172.21.15.33:0/1070576503 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff7000a3da0 0x7ff7000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:50.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.412+0000 7ff7157fa640 1 -- 172.21.15.33:0/1070576503 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7ff7080f2ba0 con 0x7ff718101aa0 2024-04-02T13:49:50.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.412+0000 7ff7177fe640 1 --2- 172.21.15.33:0/1070576503 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff7000a3da0 0x7ff7000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:50.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.413+0000 7ff7177fe640 1 --2- 172.21.15.33:0/1070576503 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff7000a3da0 0x7ff7000a6260 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7180ffb80 tx=0x7ff70c005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:50.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.417+0000 7ff7157fa640 1 -- 172.21.15.33:0/1070576503 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff70804d020 con 0x7ff718101aa0 2024-04-02T13:49:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.606+0000 7ff71e7e9640 1 -- 172.21.15.33:0/1070576503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff718109a80 con 0x7ff718101aa0 2024-04-02T13:49:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.607+0000 7ff7157fa640 1 -- 172.21.15.33:0/1070576503 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7ff70803c350 con 0x7ff718101aa0 2024-04-02T13:49:50.611 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:49:50.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.609+0000 7ff71e7e9640 1 -- 172.21.15.33:0/1070576503 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff7000a3da0 msgr2=0x7ff7000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.609+0000 7ff71e7e9640 1 --2- 172.21.15.33:0/1070576503 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff7000a3da0 0x7ff7000a6260 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7180ffb80 tx=0x7ff70c005f90 comp rx=0 tx=0).stop 2024-04-02T13:49:50.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.609+0000 7ff71e7e9640 1 -- 172.21.15.33:0/1070576503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff718101aa0 msgr2=0x7ff718198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.609+0000 7ff71e7e9640 1 --2- 172.21.15.33:0/1070576503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff718101aa0 0x7ff718198420 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ff708000c00 tx=0x7ff7080045e0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.609+0000 7ff71e7e9640 1 -- 172.21.15.33:0/1070576503 shutdown_connections 2024-04-02T13:49:50.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.609+0000 7ff71e7e9640 1 --2- 172.21.15.33:0/1070576503 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff7000a3da0 0x7ff7000a6260 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.609+0000 7ff71e7e9640 1 --2- 172.21.15.33:0/1070576503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff718107ab0 0x7ff71819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.609+0000 7ff71e7e9640 1 --2- 172.21.15.33:0/1070576503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff718102460 0x7ff718198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.609+0000 7ff71e7e9640 1 --2- 172.21.15.33:0/1070576503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff718101aa0 0x7ff718198420 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.609+0000 7ff71e7e9640 1 -- 172.21.15.33:0/1070576503 >> 172.21.15.33:0/1070576503 conn(0x7ff7180fd530 msgr2=0x7ff7180fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:50.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.609+0000 7ff71e7e9640 1 -- 172.21.15.33:0/1070576503 shutdown_connections 2024-04-02T13:49:50.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.609+0000 7ff71e7e9640 1 -- 172.21.15.33:0/1070576503 wait complete. 2024-04-02T13:49:50.624 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:49:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.729+0000 7f9e65263640 1 -- 172.21.15.33:0/2645012259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e60107aa0 msgr2=0x7f9e60107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.729+0000 7f9e65263640 1 --2- 172.21.15.33:0/2645012259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e60107aa0 0x7f9e60107e80 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009930 tx=0x7f9e5002bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.729+0000 7f9e65263640 1 -- 172.21.15.33:0/2645012259 shutdown_connections 2024-04-02T13:49:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.729+0000 7f9e65263640 1 --2- 172.21.15.33:0/2645012259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e60102450 0x7f9e6010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.729+0000 7f9e65263640 1 --2- 172.21.15.33:0/2645012259 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e60101a90 0x7f9e60101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.730+0000 7f9e65263640 1 --2- 172.21.15.33:0/2645012259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e60107aa0 0x7f9e60107e80 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.730+0000 7f9e65263640 1 -- 172.21.15.33:0/2645012259 >> 172.21.15.33:0/2645012259 conn(0x7f9e60078f00 msgr2=0x7f9e60079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.730+0000 7f9e65263640 1 -- 172.21.15.33:0/2645012259 shutdown_connections 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.730+0000 7f9e65263640 1 -- 172.21.15.33:0/2645012259 wait complete. 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.730+0000 7f9e65263640 1 Processor -- start 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.730+0000 7f9e65263640 1 -- start start 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.730+0000 7f9e65263640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e60101a90 0x7f9e601983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.730+0000 7f9e65263640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e60102450 0x7f9e60198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.730+0000 7f9e65263640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e60107aa0 0x7f9e6019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.730+0000 7f9e65263640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60111e80 con 0x7f9e60102450 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.730+0000 7f9e65263640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60111d00 con 0x7f9e60101a90 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.730+0000 7f9e65263640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60112000 con 0x7f9e60107aa0 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e567fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e60102450 0x7f9e60198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e567fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e60102450 0x7f9e60198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49722/0 (socket says 172.21.15.33:49722) 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e567fc640 1 -- 172.21.15.33:0/1369827739 learned_addr learned my addr 172.21.15.33:0/1369827739 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e5effd640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e60101a90 0x7f9e601983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e5f7fe640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e60107aa0 0x7f9e6019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e567fc640 1 -- 172.21.15.33:0/1369827739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e60107aa0 msgr2=0x7f9e6019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e567fc640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e60107aa0 0x7f9e6019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e567fc640 1 -- 172.21.15.33:0/1369827739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e60101a90 msgr2=0x7f9e601983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e567fc640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e60101a90 0x7f9e601983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e567fc640 1 -- 172.21.15.33:0/1369827739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6019d3b0 con 0x7f9e60102450 2024-04-02T13:49:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e5effd640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e60101a90 0x7f9e601983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e5f7fe640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e60107aa0 0x7f9e6019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e567fc640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e60102450 0x7f9e60198900 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800e9f0 tx=0x7f9e4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:50.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e5cff9640 1 -- 172.21.15.33:0/1369827739 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4800cd80 con 0x7f9e60102450 2024-04-02T13:49:50.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e5cff9640 1 -- 172.21.15.33:0/1369827739 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f9e480054f0 con 0x7f9e60102450 2024-04-02T13:49:50.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e5cff9640 1 -- 172.21.15.33:0/1369827739 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48010690 con 0x7f9e60102450 2024-04-02T13:49:50.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e65263640 1 -- 172.21.15.33:0/1369827739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6019d6a0 con 0x7f9e60102450 2024-04-02T13:49:50.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.731+0000 7f9e65263640 1 -- 172.21.15.33:0/1369827739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e601a4f30 con 0x7f9e60102450 2024-04-02T13:49:50.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.732+0000 7f9e65263640 1 -- 172.21.15.33:0/1369827739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e60068d40 con 0x7f9e60102450 2024-04-02T13:49:50.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.733+0000 7f9e5cff9640 1 -- 172.21.15.33:0/1369827739 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9e48004f00 con 0x7f9e60102450 2024-04-02T13:49:50.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.734+0000 7f9e5cff9640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9e4c0a3d20 0x7f9e4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:50.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.734+0000 7f9e5cff9640 1 -- 172.21.15.33:0/1369827739 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f9e480d1330 con 0x7f9e60102450 2024-04-02T13:49:50.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.734+0000 7f9e5effd640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9e4c0a3d20 0x7f9e4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:50.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.735+0000 7f9e5effd640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9e4c0a3d20 0x7f9e4c0a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e50006010 tx=0x7f9e50000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:50.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.742+0000 7f9e5cff9640 1 -- 172.21.15.33:0/1369827739 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9e48053f20 con 0x7f9e60102450 2024-04-02T13:49:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.930+0000 7f9e65263640 1 -- 172.21.15.33:0/1369827739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9e6002d370 con 0x7f9e60102450 2024-04-02T13:49:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.931+0000 7f9e5cff9640 1 -- 172.21.15.33:0/1369827739 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f9e4809d600 con 0x7f9e60102450 2024-04-02T13:49:50.935 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-02T13:49:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.933+0000 7f9e65263640 1 -- 172.21.15.33:0/1369827739 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9e4c0a3d20 msgr2=0x7f9e4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.933+0000 7f9e65263640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9e4c0a3d20 0x7f9e4c0a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e50006010 tx=0x7f9e50000bc0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.933+0000 7f9e65263640 1 -- 172.21.15.33:0/1369827739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e60102450 msgr2=0x7f9e60198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.933+0000 7f9e65263640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e60102450 0x7f9e60198900 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800e9f0 tx=0x7f9e4800edf0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.933+0000 7f9e65263640 1 -- 172.21.15.33:0/1369827739 shutdown_connections 2024-04-02T13:49:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.933+0000 7f9e65263640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f9e4c0a3d20 0x7f9e4c0a61e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.933+0000 7f9e65263640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9e60107aa0 0x7f9e6019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.933+0000 7f9e65263640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e60102450 0x7f9e60198900 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.933+0000 7f9e65263640 1 --2- 172.21.15.33:0/1369827739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e60101a90 0x7f9e601983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.933+0000 7f9e65263640 1 -- 172.21.15.33:0/1369827739 >> 172.21.15.33:0/1369827739 conn(0x7f9e60078f00 msgr2=0x7f9e60104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.933+0000 7f9e65263640 1 -- 172.21.15.33:0/1369827739 shutdown_connections 2024-04-02T13:49:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:50.933+0000 7f9e65263640 1 -- 172.21.15.33:0/1369827739 wait complete. 2024-04-02T13:49:50.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:50 smithi033 ceph-mon[32390]: pgmap v344: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:49:50.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:50 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3351770273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:49:50.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:50 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/552797675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:49:50.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:50 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1070576503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:49:50.948 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:49:50.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:50 smithi145 ceph-mon[35646]: pgmap v344: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:49:50.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:50 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3351770273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:49:50.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:50 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/552797675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:49:50.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:50 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1070576503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:49:50.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:50 smithi050 ceph-mon[35524]: pgmap v344: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-02T13:49:50.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:50 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3351770273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:49:50.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:50 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/552797675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:49:50.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:50 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1070576503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:49:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.050+0000 7f4ab20f2640 1 -- 172.21.15.33:0/744661901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac107b50 msgr2=0x7f4aac107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.050+0000 7f4ab20f2640 1 --2- 172.21.15.33:0/744661901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac107b50 0x7f4aac107f30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009930 tx=0x7f4aa002bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.050+0000 7f4ab20f2640 1 -- 172.21.15.33:0/744661901 shutdown_connections 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.050+0000 7f4ab20f2640 1 --2- 172.21.15.33:0/744661901 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aac0696c0 0x7f4aac10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.050+0000 7f4ab20f2640 1 --2- 172.21.15.33:0/744661901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac068d00 0x7f4aac069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.050+0000 7f4ab20f2640 1 --2- 172.21.15.33:0/744661901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac107b50 0x7f4aac107f30 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.050+0000 7f4ab20f2640 1 -- 172.21.15.33:0/744661901 >> 172.21.15.33:0/744661901 conn(0x7f4aac077b00 msgr2=0x7f4aac077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.050+0000 7f4ab20f2640 1 -- 172.21.15.33:0/744661901 shutdown_connections 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.051+0000 7f4ab20f2640 1 -- 172.21.15.33:0/744661901 wait complete. 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.051+0000 7f4ab20f2640 1 Processor -- start 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.051+0000 7f4ab20f2640 1 -- start start 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.051+0000 7f4ab20f2640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aac068d00 0x7f4aac198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.051+0000 7f4ab20f2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac0696c0 0x7f4aac198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.051+0000 7f4ab20f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac107b50 0x7f4aac19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.051+0000 7f4ab20f2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac111e70 con 0x7f4aac107b50 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.051+0000 7f4ab20f2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac111cf0 con 0x7f4aac0696c0 2024-04-02T13:49:51.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4ab20f2640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac111ff0 con 0x7f4aac068d00 2024-04-02T13:49:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac107b50 0x7f4aac19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac107b50 0x7f4aac19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49744/0 (socket says 172.21.15.33:49744) 2024-04-02T13:49:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aabfff640 1 -- 172.21.15.33:0/1880907092 learned_addr learned my addr 172.21.15.33:0/1880907092 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aaaffd640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac0696c0 0x7f4aac198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aab7fe640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aac068d00 0x7f4aac198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aabfff640 1 -- 172.21.15.33:0/1880907092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aac068d00 msgr2=0x7f4aac198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aabfff640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aac068d00 0x7f4aac198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aabfff640 1 -- 172.21.15.33:0/1880907092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac0696c0 msgr2=0x7f4aac198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aabfff640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac0696c0 0x7f4aac198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aabfff640 1 -- 172.21.15.33:0/1880907092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aac19d340 con 0x7f4aac107b50 2024-04-02T13:49:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aabfff640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac107b50 0x7f4aac19cc10 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400eaa0 tx=0x7f4aa400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aa8ff9640 1 -- 172.21.15.33:0/1880907092 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa400cdb0 con 0x7f4aac107b50 2024-04-02T13:49:51.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4ab20f2640 1 -- 172.21.15.33:0/1880907092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aac19d630 con 0x7f4aac107b50 2024-04-02T13:49:51.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aa8ff9640 1 -- 172.21.15.33:0/1880907092 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4aa4005020 con 0x7f4aac107b50 2024-04-02T13:49:51.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.052+0000 7f4aa8ff9640 1 -- 172.21.15.33:0/1880907092 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa40116d0 con 0x7f4aac107b50 2024-04-02T13:49:51.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.053+0000 7f4ab20f2640 1 -- 172.21.15.33:0/1880907092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aac1a4ec0 con 0x7f4aac107b50 2024-04-02T13:49:51.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.054+0000 7f4ab20f2640 1 -- 172.21.15.33:0/1880907092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aac0688f0 con 0x7f4aac107b50 2024-04-02T13:49:51.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.055+0000 7f4aa8ff9640 1 -- 172.21.15.33:0/1880907092 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4aa40051d0 con 0x7f4aac107b50 2024-04-02T13:49:51.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.055+0000 7f4aa8ff9640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4a940a3d20 0x7f4a940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:51.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.055+0000 7f4aab7fe640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4a940a3d20 0x7f4a940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:51.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.055+0000 7f4aa8ff9640 1 -- 172.21.15.33:0/1880907092 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f4aa40d0da0 con 0x7f4aac107b50 2024-04-02T13:49:51.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.056+0000 7f4aab7fe640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4a940a3d20 0x7f4a940a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0006010 tx=0x7f4aa0000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.060+0000 7f4aa8ff9640 1 -- 172.21.15.33:0/1880907092 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4aa4016030 con 0x7f4aac107b50 2024-04-02T13:49:51.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.265+0000 7f4ab20f2640 1 -- 172.21.15.33:0/1880907092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4aac10a060 con 0x7f4aac107b50 2024-04-02T13:49:51.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.265+0000 7f4aa8ff9640 1 -- 172.21.15.33:0/1880907092 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) ==== 103+0+11 (secure 0 0 0) 0x7f4aa409d070 con 0x7f4aac107b50 2024-04-02T13:49:51.270 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-02T13:49:51.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.267+0000 7f4ab20f2640 1 -- 172.21.15.33:0/1880907092 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4a940a3d20 msgr2=0x7f4a940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:51.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.267+0000 7f4ab20f2640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4a940a3d20 0x7f4a940a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0006010 tx=0x7f4aa0000bc0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.267+0000 7f4ab20f2640 1 -- 172.21.15.33:0/1880907092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac107b50 msgr2=0x7f4aac19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:51.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.267+0000 7f4ab20f2640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac107b50 0x7f4aac19cc10 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400eaa0 tx=0x7f4aa400ef60 comp rx=0 tx=0).stop 2024-04-02T13:49:51.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.268+0000 7f4ab20f2640 1 -- 172.21.15.33:0/1880907092 shutdown_connections 2024-04-02T13:49:51.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.268+0000 7f4ab20f2640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f4a940a3d20 0x7f4a940a61e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.268+0000 7f4ab20f2640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aac107b50 0x7f4aac19cc10 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.268+0000 7f4ab20f2640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4aac0696c0 0x7f4aac198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.268+0000 7f4ab20f2640 1 --2- 172.21.15.33:0/1880907092 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aac068d00 0x7f4aac198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.268+0000 7f4ab20f2640 1 -- 172.21.15.33:0/1880907092 >> 172.21.15.33:0/1880907092 conn(0x7f4aac077b00 msgr2=0x7f4aac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:51.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.268+0000 7f4ab20f2640 1 -- 172.21.15.33:0/1880907092 shutdown_connections 2024-04-02T13:49:51.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.268+0000 7f4ab20f2640 1 -- 172.21.15.33:0/1880907092 wait complete. 2024-04-02T13:49:51.282 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T13:49:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.382+0000 7fa985b74640 1 -- 172.21.15.33:0/3445225022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980068d00 msgr2=0x7fa980069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.382+0000 7fa985b74640 1 --2- 172.21.15.33:0/3445225022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980068d00 0x7fa980069180 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fa970009930 tx=0x7fa97002bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.382+0000 7fa985b74640 1 -- 172.21.15.33:0/3445225022 shutdown_connections 2024-04-02T13:49:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.382+0000 7fa985b74640 1 --2- 172.21.15.33:0/3445225022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9800696c0 0x7fa98010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.382+0000 7fa985b74640 1 --2- 172.21.15.33:0/3445225022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980068d00 0x7fa980069180 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.382+0000 7fa985b74640 1 --2- 172.21.15.33:0/3445225022 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa980107b50 0x7fa980107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.382+0000 7fa985b74640 1 -- 172.21.15.33:0/3445225022 >> 172.21.15.33:0/3445225022 conn(0x7fa980077b00 msgr2=0x7fa980077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.382+0000 7fa985b74640 1 -- 172.21.15.33:0/3445225022 shutdown_connections 2024-04-02T13:49:51.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.382+0000 7fa985b74640 1 -- 172.21.15.33:0/3445225022 wait complete. 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.383+0000 7fa985b74640 1 Processor -- start 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.383+0000 7fa985b74640 1 -- start start 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.383+0000 7fa985b74640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980068d00 0x7fa9801009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.383+0000 7fa985b74640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9800696c0 0x7fa980100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.383+0000 7fa985b74640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa980107b50 0x7fa980101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.383+0000 7fa985b74640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa980111e70 con 0x7fa980068d00 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.383+0000 7fa985b74640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa980111cf0 con 0x7fa980107b50 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.383+0000 7fa985b74640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa980111ff0 con 0x7fa9800696c0 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.383+0000 7fa97f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980068d00 0x7fa9801009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.383+0000 7fa97f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980068d00 0x7fa9801009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49766/0 (socket says 172.21.15.33:49766) 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.383+0000 7fa97f7fe640 1 -- 172.21.15.33:0/362249254 learned_addr learned my addr 172.21.15.33:0/362249254 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.383+0000 7fa97ffff640 1 --2- 172.21.15.33:0/362249254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa980107b50 0x7fa980101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.384+0000 7fa97f7fe640 1 -- 172.21.15.33:0/362249254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9800696c0 msgr2=0x7fa980100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.384+0000 7fa97f7fe640 1 --2- 172.21.15.33:0/362249254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9800696c0 0x7fa980100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.384+0000 7fa97f7fe640 1 -- 172.21.15.33:0/362249254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa980107b50 msgr2=0x7fa980101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:51.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.384+0000 7fa97f7fe640 1 --2- 172.21.15.33:0/362249254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa980107b50 0x7fa980101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.384+0000 7fa97f7fe640 1 -- 172.21.15.33:0/362249254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9801a6870 con 0x7fa980068d00 2024-04-02T13:49:51.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.384+0000 7fa97f7fe640 1 --2- 172.21.15.33:0/362249254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980068d00 0x7fa9801009c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fa96800d770 tx=0x7fa96800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:51.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.384+0000 7fa97cff9640 1 -- 172.21.15.33:0/362249254 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968017070 con 0x7fa980068d00 2024-04-02T13:49:51.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.384+0000 7fa985b74640 1 -- 172.21.15.33:0/362249254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9801a6b60 con 0x7fa980068d00 2024-04-02T13:49:51.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.384+0000 7fa97cff9640 1 -- 172.21.15.33:0/362249254 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fa9680054f0 con 0x7fa980068d00 2024-04-02T13:49:51.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.384+0000 7fa97cff9640 1 -- 172.21.15.33:0/362249254 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968004d60 con 0x7fa980068d00 2024-04-02T13:49:51.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.384+0000 7fa985b74640 1 -- 172.21.15.33:0/362249254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9801a7060 con 0x7fa980068d00 2024-04-02T13:49:51.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.385+0000 7fa985b74640 1 -- 172.21.15.33:0/362249254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9800688f0 con 0x7fa980068d00 2024-04-02T13:49:51.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.386+0000 7fa97cff9640 1 -- 172.21.15.33:0/362249254 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa968004f00 con 0x7fa980068d00 2024-04-02T13:49:51.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.387+0000 7fa97cff9640 1 --2- 172.21.15.33:0/362249254 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa96c0a3d20 0x7fa96c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:51.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.387+0000 7fa97cff9640 1 -- 172.21.15.33:0/362249254 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7fa9680d09c0 con 0x7fa980068d00 2024-04-02T13:49:51.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.387+0000 7fa97effd640 1 --2- 172.21.15.33:0/362249254 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa96c0a3d20 0x7fa96c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:51.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.387+0000 7fa97effd640 1 --2- 172.21.15.33:0/362249254 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa96c0a3d20 0x7fa96c0a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa9700096c0 tx=0x7fa970000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.391+0000 7fa97cff9640 1 -- 172.21.15.33:0/362249254 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa96809cc30 con 0x7fa980068d00 2024-04-02T13:49:51.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.589+0000 7fa985b74640 1 -- 172.21.15.33:0/362249254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa98002d370 con 0x7fa980068d00 2024-04-02T13:49:51.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.590+0000 7fa97cff9640 1 -- 172.21.15.33:0/362249254 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v105) ==== 81+0+942 (secure 0 0 0) 0x7fa96809c530 con 0x7fa980068d00 2024-04-02T13:49:51.594 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/80287983 2024-04-03T13:38:46.867289+0000 2024-04-02T13:49:51.594 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1117299953 2024-04-03T13:29:15.778774+0000 2024-04-02T13:49:51.594 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2562306694 2024-04-03T13:38:46.867289+0000 2024-04-02T13:49:51.594 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/395956530 2024-04-03T13:38:46.867289+0000 2024-04-02T13:49:51.594 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2272183007 2024-04-03T13:29:15.778774+0000 2024-04-02T13:49:51.594 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2666001000 2024-04-03T13:29:15.778774+0000 2024-04-02T13:49:51.595 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2666001000 2024-04-03T13:29:15.778774+0000 2024-04-02T13:49:51.595 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2869265288 2024-04-03T13:28:48.425104+0000 2024-04-02T13:49:51.595 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3947066882 2024-04-03T13:38:46.867289+0000 2024-04-02T13:49:51.595 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3041616812 2024-04-03T13:28:48.425104+0000 2024-04-02T13:49:51.595 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2562306694 2024-04-03T13:38:46.867289+0000 2024-04-02T13:49:51.595 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2869265288 2024-04-03T13:28:48.425104+0000 2024-04-02T13:49:51.595 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1212609914 2024-04-03T13:38:46.867289+0000 2024-04-02T13:49:51.595 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3449207963 2024-04-03T13:28:48.425104+0000 2024-04-02T13:49:51.595 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/917705077 2024-04-03T13:28:48.425104+0000 2024-04-02T13:49:51.595 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1792579346 2024-04-03T13:29:15.778774+0000 2024-04-02T13:49:51.596 INFO:teuthology.orchestra.run.smithi033.stderr:listed 16 entries 2024-04-02T13:49:51.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.592+0000 7fa985b74640 1 -- 172.21.15.33:0/362249254 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa96c0a3d20 msgr2=0x7fa96c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:51.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.592+0000 7fa985b74640 1 --2- 172.21.15.33:0/362249254 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa96c0a3d20 0x7fa96c0a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa9700096c0 tx=0x7fa970000bc0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.592+0000 7fa985b74640 1 -- 172.21.15.33:0/362249254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980068d00 msgr2=0x7fa9801009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:51.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.592+0000 7fa985b74640 1 --2- 172.21.15.33:0/362249254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980068d00 0x7fa9801009c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fa96800d770 tx=0x7fa96800dc30 comp rx=0 tx=0).stop 2024-04-02T13:49:51.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.592+0000 7fa985b74640 1 -- 172.21.15.33:0/362249254 shutdown_connections 2024-04-02T13:49:51.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.592+0000 7fa985b74640 1 --2- 172.21.15.33:0/362249254 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fa96c0a3d20 0x7fa96c0a61e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.592+0000 7fa985b74640 1 --2- 172.21.15.33:0/362249254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa980107b50 0x7fa980101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.592+0000 7fa985b74640 1 --2- 172.21.15.33:0/362249254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9800696c0 0x7fa980100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.592+0000 7fa985b74640 1 --2- 172.21.15.33:0/362249254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa980068d00 0x7fa9801009c0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:51.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.592+0000 7fa985b74640 1 -- 172.21.15.33:0/362249254 >> 172.21.15.33:0/362249254 conn(0x7fa980077b00 msgr2=0x7fa9800fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:51.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.592+0000 7fa985b74640 1 -- 172.21.15.33:0/362249254 shutdown_connections 2024-04-02T13:49:51.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:51.592+0000 7fa985b74640 1 -- 172.21.15.33:0/362249254 wait complete. 2024-04-02T13:49:51.607 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2024-04-02T13:49:51.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:49:51.607 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-02T13:49:51.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[84163]: fuse finished with error 0 and tester_r 0 2024-04-02T13:49:51.634 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T13:49:51.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:51 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1369827739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:49:51.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:51 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1880907092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:49:51.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:51 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/362249254' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:49:51.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:51 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1369827739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:49:51.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:51 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1880907092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:49:51.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:51 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/362249254' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:49:52.001 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:51 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1369827739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:49:52.001 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:51 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1880907092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:49:52.001 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:51 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/362249254' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:49:52.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:52 smithi145 ceph-mon[35646]: pgmap v345: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:52.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:52 smithi050 ceph-mon[35524]: pgmap v345: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:53.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:52 smithi033 ceph-mon[32390]: pgmap v345: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-02T13:49:54.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:54 smithi145 ceph-mon[35646]: pgmap v346: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-02T13:49:54.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:54 smithi050 ceph-mon[35524]: pgmap v346: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-02T13:49:55.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:54 smithi033 ceph-mon[32390]: pgmap v346: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-02T13:49:56.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:56 smithi145 ceph-mon[35646]: pgmap v347: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-02T13:49:56.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:56 smithi050 ceph-mon[35524]: pgmap v347: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-02T13:49:57.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:56 smithi033 ceph-mon[32390]: pgmap v347: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-02T13:49:57.736 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-04-02T13:49:57.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:49:57.736 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-02T13:49:57.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:49:57.755 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-02T13:49:57.813 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-02T13:49:57.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:49:57.813 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T13:49:57.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:49:57.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-04-02T13:49:57.992 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-02T13:49:58.004 INFO:tasks.cephadm:Teardown begin 2024-04-02T13:49:58.005 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-02T13:49:58.005 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:49:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.108+0000 7fc6880d3640 1 -- 172.21.15.33:0/3588321895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc680106a40 msgr2=0x7fc680106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.108+0000 7fc6880d3640 1 --2- 172.21.15.33:0/3588321895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc680106a40 0x7fc680106e20 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009930 tx=0x7fc67c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.108+0000 7fc6880d3640 1 -- 172.21.15.33:0/3588321895 shutdown_connections 2024-04-02T13:49:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.108+0000 7fc6880d3640 1 --2- 172.21.15.33:0/3588321895 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc680101440 0x7fc68010d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.108+0000 7fc6880d3640 1 --2- 172.21.15.33:0/3588321895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc680100a80 0x7fc680100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.108+0000 7fc6880d3640 1 --2- 172.21.15.33:0/3588321895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc680106a40 0x7fc680106e20 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.108+0000 7fc6880d3640 1 -- 172.21.15.33:0/3588321895 >> 172.21.15.33:0/3588321895 conn(0x7fc680077b60 msgr2=0x7fc680077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.108+0000 7fc6880d3640 1 -- 172.21.15.33:0/3588321895 shutdown_connections 2024-04-02T13:49:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.108+0000 7fc6880d3640 1 -- 172.21.15.33:0/3588321895 wait complete. 2024-04-02T13:49:58.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.109+0000 7fc6880d3640 1 Processor -- start 2024-04-02T13:49:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.109+0000 7fc6880d3640 1 -- start start 2024-04-02T13:49:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.109+0000 7fc6880d3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc680100a80 0x7fc680198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.109+0000 7fc6880d3640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc680101440 0x7fc680198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.109+0000 7fc6880d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc680106a40 0x7fc68019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.109+0000 7fc6880d3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc680111f20 con 0x7fc680106a40 2024-04-02T13:49:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.109+0000 7fc6880d3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc680111da0 con 0x7fc680100a80 2024-04-02T13:49:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.109+0000 7fc6880d3640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc6801120a0 con 0x7fc680101440 2024-04-02T13:49:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.109+0000 7fc686670640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc680106a40 0x7fc68019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.109+0000 7fc686670640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc680106a40 0x7fc68019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36250/0 (socket says 172.21.15.33:36250) 2024-04-02T13:49:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.109+0000 7fc686670640 1 -- 172.21.15.33:0/3613381878 learned_addr learned my addr 172.21.15.33:0/3613381878 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.109+0000 7fc68566e640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc680101440 0x7fc680198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.109+0000 7fc685e6f640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc680100a80 0x7fc680198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.110+0000 7fc686670640 1 -- 172.21.15.33:0/3613381878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc680101440 msgr2=0x7fc680198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.110+0000 7fc686670640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc680101440 0x7fc680198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.110+0000 7fc686670640 1 -- 172.21.15.33:0/3613381878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc680100a80 msgr2=0x7fc680198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.110+0000 7fc686670640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc680100a80 0x7fc680198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.110+0000 7fc686670640 1 -- 172.21.15.33:0/3613381878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68019d410 con 0x7fc680106a40 2024-04-02T13:49:58.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.110+0000 7fc685e6f640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc680100a80 0x7fc680198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:58.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.110+0000 7fc686670640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc680106a40 0x7fc68019cce0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc67800eb10 tx=0x7fc67800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:58.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.110+0000 7fc676ffd640 1 -- 172.21.15.33:0/3613381878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678004b20 con 0x7fc680106a40 2024-04-02T13:49:58.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.110+0000 7fc676ffd640 1 -- 172.21.15.33:0/3613381878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fc678004cc0 con 0x7fc680106a40 2024-04-02T13:49:58.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.110+0000 7fc6880d3640 1 -- 172.21.15.33:0/3613381878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68019d700 con 0x7fc680106a40 2024-04-02T13:49:58.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.110+0000 7fc676ffd640 1 -- 172.21.15.33:0/3613381878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678010700 con 0x7fc680106a40 2024-04-02T13:49:58.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.110+0000 7fc6880d3640 1 -- 172.21.15.33:0/3613381878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6801a4f90 con 0x7fc680106a40 2024-04-02T13:49:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.111+0000 7fc6880d3640 1 -- 172.21.15.33:0/3613381878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc680068d40 con 0x7fc680106a40 2024-04-02T13:49:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.112+0000 7fc676ffd640 1 -- 172.21.15.33:0/3613381878 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc678004e70 con 0x7fc680106a40 2024-04-02T13:49:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.112+0000 7fc676ffd640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc66c0a3d20 0x7fc66c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.113+0000 7fc676ffd640 1 -- 172.21.15.33:0/3613381878 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7fc6780d11c0 con 0x7fc680106a40 2024-04-02T13:49:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.113+0000 7fc685e6f640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc66c0a3d20 0x7fc66c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.113+0000 7fc685e6f640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc66c0a3d20 0x7fc66c0a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc67c006010 tx=0x7fc67c000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:58.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.117+0000 7fc676ffd640 1 -- 172.21.15.33:0/3613381878 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc67809d430 con 0x7fc680106a40 2024-04-02T13:49:58.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.343+0000 7fc6880d3640 1 -- 172.21.15.33:0/3613381878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc680109000 con 0x7fc680106a40 2024-04-02T13:49:58.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.344+0000 7fc676ffd640 1 -- 172.21.15.33:0/3613381878 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11873 (secure 0 0 0) 0x7fc67809cd30 con 0x7fc680106a40 2024-04-02T13:49:58.348 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:49:58.348 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:44:55.702357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:49:58.349 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 36 2024-04-02T13:49:58.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.346+0000 7fc6880d3640 1 -- 172.21.15.33:0/3613381878 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc66c0a3d20 msgr2=0x7fc66c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.346+0000 7fc6880d3640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc66c0a3d20 0x7fc66c0a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc67c006010 tx=0x7fc67c000bc0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.346+0000 7fc6880d3640 1 -- 172.21.15.33:0/3613381878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc680106a40 msgr2=0x7fc68019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.346+0000 7fc6880d3640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc680106a40 0x7fc68019cce0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc67800eb10 tx=0x7fc67800c2d0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.346+0000 7fc6880d3640 1 -- 172.21.15.33:0/3613381878 shutdown_connections 2024-04-02T13:49:58.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.346+0000 7fc6880d3640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fc66c0a3d20 0x7fc66c0a61e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.346+0000 7fc6880d3640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc680106a40 0x7fc68019cce0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.346+0000 7fc6880d3640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc680101440 0x7fc680198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.346+0000 7fc6880d3640 1 --2- 172.21.15.33:0/3613381878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc680100a80 0x7fc680198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.346+0000 7fc6880d3640 1 -- 172.21.15.33:0/3613381878 >> 172.21.15.33:0/3613381878 conn(0x7fc680077b60 msgr2=0x7fc680103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:58.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.346+0000 7fc6880d3640 1 -- 172.21.15.33:0/3613381878 shutdown_connections 2024-04-02T13:49:58.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.346+0000 7fc6880d3640 1 -- 172.21.15.33:0/3613381878 wait complete. 2024-04-02T13:49:58.361 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:49:58.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.461+0000 7f5642100640 1 -- 172.21.15.33:0/2333452576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c101a90 msgr2=0x7f563c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.461+0000 7f5642100640 1 --2- 172.21.15.33:0/2333452576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c101a90 0x7f563c101f10 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5630009930 tx=0x7f563002bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.461+0000 7f5642100640 1 -- 172.21.15.33:0/2333452576 shutdown_connections 2024-04-02T13:49:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.461+0000 7f5642100640 1 --2- 172.21.15.33:0/2333452576 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f563c102450 0x7f563c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.461+0000 7f5642100640 1 --2- 172.21.15.33:0/2333452576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c101a90 0x7f563c101f10 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.461+0000 7f5642100640 1 --2- 172.21.15.33:0/2333452576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f563c107aa0 0x7f563c107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.461+0000 7f5642100640 1 -- 172.21.15.33:0/2333452576 >> 172.21.15.33:0/2333452576 conn(0x7f563c0fd5a0 msgr2=0x7f563c0ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.461+0000 7f5642100640 1 -- 172.21.15.33:0/2333452576 shutdown_connections 2024-04-02T13:49:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.461+0000 7f5642100640 1 -- 172.21.15.33:0/2333452576 wait complete. 2024-04-02T13:49:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f5642100640 1 Processor -- start 2024-04-02T13:49:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f5642100640 1 -- start start 2024-04-02T13:49:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f5642100640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f563c101a90 0x7f563c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f5642100640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c102450 0x7f563c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f5642100640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f563c107aa0 0x7f563c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f5642100640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f563c111e80 con 0x7f563c102450 2024-04-02T13:49:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f5642100640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f563c111d00 con 0x7f563c101a90 2024-04-02T13:49:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f5642100640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f563c112000 con 0x7f563c107aa0 2024-04-02T13:49:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f563affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c102450 0x7f563c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f563affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c102450 0x7f563c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36264/0 (socket says 172.21.15.33:36264) 2024-04-02T13:49:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f563affd640 1 -- 172.21.15.33:0/2536673028 learned_addr learned my addr 172.21.15.33:0/2536673028 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f563bfff640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f563c107aa0 0x7f563c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f563b7fe640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f563c101a90 0x7f563c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f563affd640 1 -- 172.21.15.33:0/2536673028 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f563c107aa0 msgr2=0x7f563c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f563affd640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f563c107aa0 0x7f563c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f563affd640 1 -- 172.21.15.33:0/2536673028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f563c101a90 msgr2=0x7f563c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f563affd640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f563c101a90 0x7f563c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f563affd640 1 -- 172.21.15.33:0/2536673028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c19d410 con 0x7f563c102450 2024-04-02T13:49:58.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.462+0000 7f563bfff640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f563c107aa0 0x7f563c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:58.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.463+0000 7f563affd640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c102450 0x7f563c198960 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5630005ab0 tx=0x7f5630004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:58.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.463+0000 7f5638ff9640 1 -- 172.21.15.33:0/2536673028 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630043020 con 0x7f563c102450 2024-04-02T13:49:58.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.463+0000 7f5638ff9640 1 -- 172.21.15.33:0/2536673028 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f563002c5d0 con 0x7f563c102450 2024-04-02T13:49:58.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.463+0000 7f5642100640 1 -- 172.21.15.33:0/2536673028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563c19d6a0 con 0x7f563c102450 2024-04-02T13:49:58.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.463+0000 7f5638ff9640 1 -- 172.21.15.33:0/2536673028 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56300039f0 con 0x7f563c102450 2024-04-02T13:49:58.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.463+0000 7f5642100640 1 -- 172.21.15.33:0/2536673028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563c1a4f90 con 0x7f563c102450 2024-04-02T13:49:58.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.464+0000 7f5642100640 1 -- 172.21.15.33:0/2536673028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563c068d40 con 0x7f563c102450 2024-04-02T13:49:58.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.465+0000 7f5638ff9640 1 -- 172.21.15.33:0/2536673028 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5630046020 con 0x7f563c102450 2024-04-02T13:49:58.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.465+0000 7f5638ff9640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f56240a3d20 0x7f56240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.466+0000 7f5638ff9640 1 -- 172.21.15.33:0/2536673028 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f56300f2cf0 con 0x7f563c102450 2024-04-02T13:49:58.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.466+0000 7f563b7fe640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f56240a3d20 0x7f56240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.466+0000 7f563b7fe640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f56240a3d20 0x7f56240a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f562c009ae0 tx=0x7f562c005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:58.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.469+0000 7f5638ff9640 1 -- 172.21.15.33:0/2536673028 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f563004d070 con 0x7f563c102450 2024-04-02T13:49:58.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.699+0000 7f5642100640 1 -- 172.21.15.33:0/2536673028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f563c199770 con 0x7f563c102450 2024-04-02T13:49:58.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.700+0000 7f5638ff9640 1 -- 172.21.15.33:0/2536673028 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11873 (secure 0 0 0) 0x7f56300bf0d0 con 0x7f563c102450 2024-04-02T13:49:58.704 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:49:58.704 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1918898100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1918898100},{"type":"v1","addr":"172.21.15.33:6835","nonce":1918898100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1083419876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1083419876},{"type":"v1","addr":"172.21.15.33:6837","nonce":1083419876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3175531919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3175531919},{"type":"v1","addr":"172.21.15.33:6839","nonce":3175531919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2274071500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2274071500},{"type":"v1","addr":"172.21.15.33:6841","nonce":2274071500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/3754674798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":3754674798},{"type":"v1","addr":"172.21.15.145:6833","nonce":3754674798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/1584719197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":1584719197},{"type":"v1","addr":"172.21.15.50:6833","nonce":1584719197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:36:21.184650+0000","modified":"2024-04-02T13:44:55.702357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24512,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/4110737670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4110737670},{"type":"v1","addr":"172.21.15.145:6835","nonce":4110737670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.50:6835/1580073716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1580073716},{"type":"v1","addr":"172.21.15.50:6835","nonce":1580073716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/4234446304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":4234446304},{"type":"v1","addr":"172.21.15.145:6837","nonce":4234446304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6839/3484016588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":3484016588},{"type":"v1","addr":"172.21.15.145:6839","nonce":3484016588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.50:6837/3788578797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3788578797},{"type":"v1","addr":"172.21.15.50:6837","nonce":3788578797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.50:6839/932333028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":932333028},{"type":"v1","addr":"172.21.15.50:6839","nonce":932333028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24473,24512,24551]},"id":1}]} 2024-04-02T13:49:58.705 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 36 2024-04-02T13:49:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.702+0000 7f5642100640 1 -- 172.21.15.33:0/2536673028 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f56240a3d20 msgr2=0x7f56240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.702+0000 7f5642100640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f56240a3d20 0x7f56240a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f562c009ae0 tx=0x7f562c005f90 comp rx=0 tx=0).stop 2024-04-02T13:49:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.702+0000 7f5642100640 1 -- 172.21.15.33:0/2536673028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c102450 msgr2=0x7f563c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.702+0000 7f5642100640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c102450 0x7f563c198960 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5630005ab0 tx=0x7f5630004600 comp rx=0 tx=0).stop 2024-04-02T13:49:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.702+0000 7f5642100640 1 -- 172.21.15.33:0/2536673028 shutdown_connections 2024-04-02T13:49:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.702+0000 7f5642100640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f56240a3d20 0x7f56240a61e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.702+0000 7f5642100640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f563c107aa0 0x7f563c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.702+0000 7f5642100640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c102450 0x7f563c198960 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.702+0000 7f5642100640 1 --2- 172.21.15.33:0/2536673028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f563c101a90 0x7f563c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.702+0000 7f5642100640 1 -- 172.21.15.33:0/2536673028 >> 172.21.15.33:0/2536673028 conn(0x7f563c0fd5a0 msgr2=0x7f563c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.703+0000 7f5642100640 1 -- 172.21.15.33:0/2536673028 shutdown_connections 2024-04-02T13:49:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.703+0000 7f5642100640 1 -- 172.21.15.33:0/2536673028 wait complete. 2024-04-02T13:49:58.718 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:49:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.817+0000 7f45ae0b7640 1 -- 172.21.15.33:0/547872561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a8103690 msgr2=0x7f45a810bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.817+0000 7f45ae0b7640 1 --2- 172.21.15.33:0/547872561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a8103690 0x7f45a810bb70 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f459c009900 tx=0x7f459c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.817+0000 7f45ae0b7640 1 -- 172.21.15.33:0/547872561 shutdown_connections 2024-04-02T13:49:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.817+0000 7f45ae0b7640 1 --2- 172.21.15.33:0/547872561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a810c140 0x7f45a810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.817+0000 7f45ae0b7640 1 --2- 172.21.15.33:0/547872561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a8103690 0x7f45a810bb70 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.817+0000 7f45ae0b7640 1 --2- 172.21.15.33:0/547872561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45a8102d70 0x7f45a8103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.817+0000 7f45ae0b7640 1 -- 172.21.15.33:0/547872561 >> 172.21.15.33:0/547872561 conn(0x7f45a8077b00 msgr2=0x7f45a8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.817+0000 7f45ae0b7640 1 -- 172.21.15.33:0/547872561 shutdown_connections 2024-04-02T13:49:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.818+0000 7f45ae0b7640 1 -- 172.21.15.33:0/547872561 wait complete. 2024-04-02T13:49:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.818+0000 7f45ae0b7640 1 Processor -- start 2024-04-02T13:49:58.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.818+0000 7f45ae0b7640 1 -- start start 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.818+0000 7f45ae0b7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8102d70 0x7f45a8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.818+0000 7f45ae0b7640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45a8103690 0x7f45a8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.818+0000 7f45ae0b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a810c140 0x7f45a819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.818+0000 7f45ae0b7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8111f10 con 0x7f45a810c140 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.818+0000 7f45ae0b7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8111d90 con 0x7f45a8102d70 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.818+0000 7f45ae0b7640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8112090 con 0x7f45a8103690 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8102d70 0x7f45a8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a6ffd640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45a8103690 0x7f45a8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8102d70 0x7f45a8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.33:47822/0 (socket says 172.21.15.33:47822) 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a77fe640 1 -- 172.21.15.33:0/854158376 learned_addr learned my addr 172.21.15.33:0/854158376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a7fff640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a810c140 0x7f45a819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a6ffd640 1 -- 172.21.15.33:0/854158376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8102d70 msgr2=0x7f45a8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a6ffd640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8102d70 0x7f45a8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a6ffd640 1 -- 172.21.15.33:0/854158376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a810c140 msgr2=0x7f45a819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a6ffd640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a810c140 0x7f45a819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a6ffd640 1 -- 172.21.15.33:0/854158376 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a819d420 con 0x7f45a8103690 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a77fe640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8102d70 0x7f45a8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a7fff640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a810c140 0x7f45a819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a6ffd640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45a8103690 0x7f45a8198970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f459c009230 tx=0x7f459c004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a4ff9640 1 -- 172.21.15.33:0/854158376 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c043020 con 0x7f45a8103690 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.820+0000 7f45ae0b7640 1 -- 172.21.15.33:0/854158376 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a819d6b0 con 0x7f45a8103690 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.819+0000 7f45a4ff9640 1 -- 172.21.15.33:0/854158376 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f459c02c5d0 con 0x7f45a8103690 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.820+0000 7f45a4ff9640 1 -- 172.21.15.33:0/854158376 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c003920 con 0x7f45a8103690 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.820+0000 7f45ae0b7640 1 -- 172.21.15.33:0/854158376 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a81a4fa0 con 0x7f45a8103690 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.820+0000 7f45ae0b7640 1 -- 172.21.15.33:0/854158376 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a80688f0 con 0x7f45a8103690 2024-04-02T13:49:58.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.822+0000 7f45a4ff9640 1 -- 172.21.15.33:0/854158376 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f459c046020 con 0x7f45a8103690 2024-04-02T13:49:58.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.822+0000 7f45a4ff9640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f45900a3d20 0x7f45900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.822+0000 7f45a77fe640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f45900a3d20 0x7f45900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.822+0000 7f45a4ff9640 1 -- 172.21.15.33:0/854158376 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f459c0f2b50 con 0x7f45a8103690 2024-04-02T13:49:58.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.823+0000 7f45a77fe640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f45900a3d20 0x7f45900a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4598011500 tx=0x7f4598005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:58.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:58.826+0000 7f45a4ff9640 1 -- 172.21.15.33:0/854158376 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f459c04d070 con 0x7f45a8103690 2024-04-02T13:49:58.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:58 smithi145 ceph-mon[35646]: pgmap v348: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-02T13:49:58.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:58 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3613381878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:49:58.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:58 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2536673028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:49:58.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:58 smithi050 ceph-mon[35524]: pgmap v348: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-02T13:49:58.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:58 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3613381878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:49:58.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:58 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2536673028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:49:59.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:58 smithi033 ceph-mon[32390]: pgmap v348: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-02T13:49:59.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:58 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3613381878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:49:59.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:58 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2536673028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:49:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.014+0000 7f45ae0b7640 1 -- 172.21.15.33:0/854158376 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45a8107230 con 0x7f45a8103690 2024-04-02T13:49:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.017+0000 7f45a4ff9640 1 -- 172.21.15.33:0/854158376 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+25182 (secure 0 0 0) 0x7f459c0befc0 con 0x7f45a8103690 2024-04-02T13:49:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-02T13:49:59.021 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":105,"fsid":"b5fd982e-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:28:21.504900+0000","modified":"2024-04-02T13:49:11.632123+0000","last_up_change":"2024-04-02T13:35:30.640924+0000","last_in_change":"2024-04-02T13:35:10.519737+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:31:41.868850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:36:19.501121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:36:20.161569+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:36:23.482134+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78695977-281e-41e3-9c9e-9a3c1fca4744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6803","nonce":52311839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6805","nonce":52311839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6809","nonce":52311839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":52311839},{"type":"v1","addr":"172.21.15.33:6807","nonce":52311839}]},"public_addr":"172.21.15.33:6803/52311839","cluster_addr":"172.21.15.33:6805/52311839","heartbeat_back_addr":"172.21.15.33:6809/52311839","heartbeat_front_addr":"172.21.15.33:6807/52311839","state":["exists","up"]},{"osd":1,"uuid":"a6e22709-5642-4aae-a9b3-f7b27c4bed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6801","nonce":1987709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6803","nonce":1987709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6807","nonce":1987709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":1987709736},{"type":"v1","addr":"172.21.15.50:6805","nonce":1987709736}]},"public_addr":"172.21.15.50:6801/1987709736","cluster_addr":"172.21.15.50:6803/1987709736","heartbeat_back_addr":"172.21.15.50:6807/1987709736","heartbeat_front_addr":"172.21.15.50:6805/1987709736","state":["exists","up"]},{"osd":2,"uuid":"5bdfdc32-0526-4f03-bd3f-50b251e80e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6801","nonce":1898887265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6803","nonce":1898887265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6807","nonce":1898887265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":1898887265},{"type":"v1","addr":"172.21.15.145:6805","nonce":1898887265}]},"public_addr":"172.21.15.145:6801/1898887265","cluster_addr":"172.21.15.145:6803/1898887265","heartbeat_back_addr":"172.21.15.145:6807/1898887265","heartbeat_front_addr":"172.21.15.145:6805/1898887265","state":["exists","up"]},{"osd":3,"uuid":"78b02725-b00b-4bdc-927f-b5d72ad92753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6811","nonce":1162388141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6813","nonce":1162388141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6817","nonce":1162388141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1162388141},{"type":"v1","addr":"172.21.15.33:6815","nonce":1162388141}]},"public_addr":"172.21.15.33:6811/1162388141","cluster_addr":"172.21.15.33:6813/1162388141","heartbeat_back_addr":"172.21.15.33:6817/1162388141","heartbeat_front_addr":"172.21.15.33:6815/1162388141","state":["exists","up"]},{"osd":4,"uuid":"b02546aa-52cc-43d0-93c8-ba2ad32ceb0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6809","nonce":367423754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6811","nonce":367423754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6815","nonce":367423754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":367423754},{"type":"v1","addr":"172.21.15.50:6813","nonce":367423754}]},"public_addr":"172.21.15.50:6809/367423754","cluster_addr":"172.21.15.50:6811/367423754","heartbeat_back_addr":"172.21.15.50:6815/367423754","heartbeat_front_addr":"172.21.15.50:6813/367423754","state":["exists","up"]},{"osd":5,"uuid":"39e973fe-56fa-49d5-b155-17569a72a422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6809","nonce":3244269192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6811","nonce":3244269192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6815","nonce":3244269192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3244269192},{"type":"v1","addr":"172.21.15.145:6813","nonce":3244269192}]},"public_addr":"172.21.15.145:6809/3244269192","cluster_addr":"172.21.15.145:6811/3244269192","heartbeat_back_addr":"172.21.15.145:6815/3244269192","heartbeat_front_addr":"172.21.15.145:6813/3244269192","state":["exists","up"]},{"osd":6,"uuid":"523d9581-9067-4a63-8177-403ca4352df7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6819","nonce":529207637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6821","nonce":529207637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6825","nonce":529207637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":529207637},{"type":"v1","addr":"172.21.15.33:6823","nonce":529207637}]},"public_addr":"172.21.15.33:6819/529207637","cluster_addr":"172.21.15.33:6821/529207637","heartbeat_back_addr":"172.21.15.33:6825/529207637","heartbeat_front_addr":"172.21.15.33:6823/529207637","state":["exists","up"]},{"osd":7,"uuid":"53f96776-fc11-4b58-b6c7-dce5e873948e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6817","nonce":2798485676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6819","nonce":2798485676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6823","nonce":2798485676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2798485676},{"type":"v1","addr":"172.21.15.50:6821","nonce":2798485676}]},"public_addr":"172.21.15.50:6817/2798485676","cluster_addr":"172.21.15.50:6819/2798485676","heartbeat_back_addr":"172.21.15.50:6823/2798485676","heartbeat_front_addr":"172.21.15.50:6821/2798485676","state":["exists","up"]},{"osd":8,"uuid":"1957ffe4-4131-4157-b7b1-de97b339f24c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6817","nonce":1364963848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6819","nonce":1364963848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6823","nonce":1364963848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":1364963848},{"type":"v1","addr":"172.21.15.145:6821","nonce":1364963848}]},"public_addr":"172.21.15.145:6817/1364963848","cluster_addr":"172.21.15.145:6819/1364963848","heartbeat_back_addr":"172.21.15.145:6823/1364963848","heartbeat_front_addr":"172.21.15.145:6821/1364963848","state":["exists","up"]},{"osd":9,"uuid":"32d3ed72-1341-4dd0-928e-e0e331751178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6827","nonce":1206256485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6829","nonce":1206256485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6833","nonce":1206256485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1206256485},{"type":"v1","addr":"172.21.15.33:6831","nonce":1206256485}]},"public_addr":"172.21.15.33:6827/1206256485","cluster_addr":"172.21.15.33:6829/1206256485","heartbeat_back_addr":"172.21.15.33:6833/1206256485","heartbeat_front_addr":"172.21.15.33:6831/1206256485","state":["exists","up"]},{"osd":10,"uuid":"0252727c-a17e-4542-b7c3-c409c8a6d0bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6825","nonce":1632195410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6827","nonce":1632195410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6831","nonce":1632195410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1632195410},{"type":"v1","addr":"172.21.15.50:6829","nonce":1632195410}]},"public_addr":"172.21.15.50:6825/1632195410","cluster_addr":"172.21.15.50:6827/1632195410","heartbeat_back_addr":"172.21.15.50:6831/1632195410","heartbeat_front_addr":"172.21.15.50:6829/1632195410","state":["exists","up"]},{"osd":11,"uuid":"7b33c049-e9a7-4f62-a514-4cc24a0c4267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6825","nonce":141823123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6827","nonce":141823123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6831","nonce":141823123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":141823123},{"type":"v1","addr":"172.21.15.145:6829","nonce":141823123}]},"public_addr":"172.21.15.145:6825/141823123","cluster_addr":"172.21.15.145:6827/141823123","heartbeat_back_addr":"172.21.15.145:6831/141823123","heartbeat_front_addr":"172.21.15.145:6829/141823123","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:49.616929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:14.369856+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:38.479363+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:03.057555+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:27.742060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:32:52.785539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:18.774695+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:33:44.305599+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:09.930110+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:34:36.545906+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:02.667914+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:35:28.945848+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/80287983":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/1117299953":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/395956530":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/2272183007":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6801/2666001000":"2024-04-03T13:29:15.778774+0000","172.21.15.33:6800/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/3947066882":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3041616812":"2024-04-03T13:28:48.425104+0000","172.21.15.33:6801/2562306694":"2024-04-03T13:38:46.867289+0000","172.21.15.33:6801/2869265288":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1212609914":"2024-04-03T13:38:46.867289+0000","172.21.15.33:0/3449207963":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/917705077":"2024-04-03T13:28:48.425104+0000","172.21.15.33:0/1792579346":"2024-04-03T13:29:15.778774+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:49:59.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.018+0000 7f45ae0b7640 1 -- 172.21.15.33:0/854158376 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f45900a3d20 msgr2=0x7f45900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.018+0000 7f45ae0b7640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f45900a3d20 0x7f45900a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4598011500 tx=0x7f4598005f90 comp rx=0 tx=0).stop 2024-04-02T13:49:59.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.018+0000 7f45ae0b7640 1 -- 172.21.15.33:0/854158376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45a8103690 msgr2=0x7f45a8198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.018+0000 7f45ae0b7640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45a8103690 0x7f45a8198970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f459c009230 tx=0x7f459c004600 comp rx=0 tx=0).stop 2024-04-02T13:49:59.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.018+0000 7f45ae0b7640 1 -- 172.21.15.33:0/854158376 shutdown_connections 2024-04-02T13:49:59.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.018+0000 7f45ae0b7640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f45900a3d20 0x7f45900a61e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.018+0000 7f45ae0b7640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45a810c140 0x7f45a819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.018+0000 7f45ae0b7640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f45a8103690 0x7f45a8198970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.018+0000 7f45ae0b7640 1 --2- 172.21.15.33:0/854158376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8102d70 0x7f45a8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.018+0000 7f45ae0b7640 1 -- 172.21.15.33:0/854158376 >> 172.21.15.33:0/854158376 conn(0x7f45a8077b00 msgr2=0x7f45a80fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.018+0000 7f45ae0b7640 1 -- 172.21.15.33:0/854158376 shutdown_connections 2024-04-02T13:49:59.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.018+0000 7f45ae0b7640 1 -- 172.21.15.33:0/854158376 wait complete. 2024-04-02T13:49:59.034 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-02T13:49:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.136+0000 7f42b3aba640 1 -- 172.21.15.33:0/959791065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac106980 msgr2=0x7f42ac106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.136+0000 7f42b3aba640 1 --2- 172.21.15.33:0/959791065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac106980 0x7f42ac106d60 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f42a4009930 tx=0x7f42a402bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.136+0000 7f42b3aba640 1 -- 172.21.15.33:0/959791065 shutdown_connections 2024-04-02T13:49:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.136+0000 7f42b3aba640 1 --2- 172.21.15.33:0/959791065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42ac101330 0x7f42ac10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.136+0000 7f42b3aba640 1 --2- 172.21.15.33:0/959791065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42ac100970 0x7f42ac100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.136+0000 7f42b3aba640 1 --2- 172.21.15.33:0/959791065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac106980 0x7f42ac106d60 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.136+0000 7f42b3aba640 1 -- 172.21.15.33:0/959791065 >> 172.21.15.33:0/959791065 conn(0x7f42ac077b00 msgr2=0x7f42ac077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.136+0000 7f42b3aba640 1 -- 172.21.15.33:0/959791065 shutdown_connections 2024-04-02T13:49:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.136+0000 7f42b3aba640 1 -- 172.21.15.33:0/959791065 wait complete. 2024-04-02T13:49:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.136+0000 7f42b3aba640 1 Processor -- start 2024-04-02T13:49:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.136+0000 7f42b3aba640 1 -- start start 2024-04-02T13:49:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b3aba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42ac100970 0x7f42ac0fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b3aba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac101330 0x7f42ac0fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b3aba640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42ac106980 0x7f42ac0fe420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b3aba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac110d60 con 0x7f42ac101330 2024-04-02T13:49:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b3aba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac110be0 con 0x7f42ac100970 2024-04-02T13:49:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b3aba640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac110ee0 con 0x7f42ac106980 2024-04-02T13:49:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b1055640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac101330 0x7f42ac0fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b1055640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac101330 0x7f42ac0fdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36302/0 (socket says 172.21.15.33:36302) 2024-04-02T13:49:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b1055640 1 -- 172.21.15.33:0/281776643 learned_addr learned my addr 172.21.15.33:0/281776643 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b2057640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42ac106980 0x7f42ac0fe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b1856640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42ac100970 0x7f42ac0fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b1055640 1 -- 172.21.15.33:0/281776643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42ac106980 msgr2=0x7f42ac0fe420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b1055640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42ac106980 0x7f42ac0fe420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b1055640 1 -- 172.21.15.33:0/281776643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42ac100970 msgr2=0x7f42ac0fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b1055640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42ac100970 0x7f42ac0fd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b1055640 1 -- 172.21.15.33:0/281776643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac1a6870 con 0x7f42ac101330 2024-04-02T13:49:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b2057640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42ac106980 0x7f42ac0fe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b1055640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac101330 0x7f42ac0fdee0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f42a000d950 tx=0x7f42a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42aaffd640 1 -- 172.21.15.33:0/281776643 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a0017070 con 0x7f42ac101330 2024-04-02T13:49:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42aaffd640 1 -- 172.21.15.33:0/281776643 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f42a0009c30 con 0x7f42ac101330 2024-04-02T13:49:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b3aba640 1 -- 172.21.15.33:0/281776643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ac1a6b60 con 0x7f42ac101330 2024-04-02T13:49:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42aaffd640 1 -- 172.21.15.33:0/281776643 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a000f6b0 con 0x7f42ac101330 2024-04-02T13:49:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.137+0000 7f42b3aba640 1 -- 172.21.15.33:0/281776643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ac1a70b0 con 0x7f42ac101330 2024-04-02T13:49:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.138+0000 7f42b3aba640 1 -- 172.21.15.33:0/281776643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42ac0688f0 con 0x7f42ac101330 2024-04-02T13:49:59.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.140+0000 7f42aaffd640 1 -- 172.21.15.33:0/281776643 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f42a0010a50 con 0x7f42ac101330 2024-04-02T13:49:59.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.141+0000 7f42aaffd640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f42940a3d20 0x7f42940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.141+0000 7f42b1856640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f42940a3d20 0x7f42940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.141+0000 7f42aaffd640 1 -- 172.21.15.33:0/281776643 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9496+0+0 (secure 0 0 0) 0x7f42a00d08d0 con 0x7f42ac101330 2024-04-02T13:49:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.141+0000 7f42b1856640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f42940a3d20 0x7f42940a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f42a4006010 tx=0x7f42a4000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.144+0000 7f42aaffd640 1 -- 172.21.15.33:0/281776643 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f42a0009de0 con 0x7f42ac101330 2024-04-02T13:49:59.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.370+0000 7f42b3aba640 1 -- 172.21.15.33:0/281776643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f42ac1a7280 con 0x7f42ac101330 2024-04-02T13:49:59.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.373+0000 7f42aaffd640 1 -- 172.21.15.33:0/281776643 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7f42a009cb40 con 0x7f42ac101330 2024-04-02T13:49:59.376 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-02T13:49:59.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.375+0000 7f42b3aba640 1 -- 172.21.15.33:0/281776643 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f42940a3d20 msgr2=0x7f42940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.375+0000 7f42b3aba640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f42940a3d20 0x7f42940a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f42a4006010 tx=0x7f42a4000bc0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.375+0000 7f42b3aba640 1 -- 172.21.15.33:0/281776643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac101330 msgr2=0x7f42ac0fdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.375+0000 7f42b3aba640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac101330 0x7f42ac0fdee0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f42a000d950 tx=0x7f42a000de10 comp rx=0 tx=0).stop 2024-04-02T13:49:59.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.375+0000 7f42b3aba640 1 -- 172.21.15.33:0/281776643 shutdown_connections 2024-04-02T13:49:59.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.375+0000 7f42b3aba640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f42940a3d20 0x7f42940a61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.375+0000 7f42b3aba640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42ac106980 0x7f42ac0fe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.375+0000 7f42b3aba640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42ac101330 0x7f42ac0fdee0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.375+0000 7f42b3aba640 1 --2- 172.21.15.33:0/281776643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42ac100970 0x7f42ac0fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.375+0000 7f42b3aba640 1 -- 172.21.15.33:0/281776643 >> 172.21.15.33:0/281776643 conn(0x7f42ac077b00 msgr2=0x7f42ac0ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.375+0000 7f42b3aba640 1 -- 172.21.15.33:0/281776643 shutdown_connections 2024-04-02T13:49:59.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.375+0000 7f42b3aba640 1 -- 172.21.15.33:0/281776643 wait complete. 2024-04-02T13:49:59.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-02T13:49:59.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.497+0000 7ff0fe693640 1 -- 172.21.15.33:0/2305743460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f8102d70 msgr2=0x7ff0f8103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.497+0000 7ff0fe693640 1 --2- 172.21.15.33:0/2305743460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f8102d70 0x7ff0f8103150 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009900 tx=0x7ff0e802bcd0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.497+0000 7ff0fe693640 1 -- 172.21.15.33:0/2305743460 shutdown_connections 2024-04-02T13:49:59.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.497+0000 7ff0fe693640 1 --2- 172.21.15.33:0/2305743460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0f810c140 0x7ff0f810e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.497+0000 7ff0fe693640 1 --2- 172.21.15.33:0/2305743460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f8103690 0x7ff0f810bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.497+0000 7ff0fe693640 1 --2- 172.21.15.33:0/2305743460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f8102d70 0x7ff0f8103150 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.497+0000 7ff0fe693640 1 -- 172.21.15.33:0/2305743460 >> 172.21.15.33:0/2305743460 conn(0x7ff0f8077b60 msgr2=0x7ff0f8077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.497+0000 7ff0fe693640 1 -- 172.21.15.33:0/2305743460 shutdown_connections 2024-04-02T13:49:59.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.497+0000 7ff0fe693640 1 -- 172.21.15.33:0/2305743460 wait complete. 2024-04-02T13:49:59.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0fe693640 1 Processor -- start 2024-04-02T13:49:59.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0fe693640 1 -- start start 2024-04-02T13:49:59.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0fe693640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0f8102d70 0x7ff0f81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0fe693640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f8103690 0x7ff0f81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0fe693640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f810c140 0x7ff0f819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0fe693640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f8111ee0 con 0x7ff0f8103690 2024-04-02T13:49:59.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0fe693640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f8111d60 con 0x7ff0f810c140 2024-04-02T13:49:59.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0fe693640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f8112060 con 0x7ff0f8102d70 2024-04-02T13:49:59.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f8103690 0x7ff0f81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f8103690 0x7ff0f81988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36314/0 (socket says 172.21.15.33:36314) 2024-04-02T13:49:59.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0f77fe640 1 -- 172.21.15.33:0/3132660626 learned_addr learned my addr 172.21.15.33:0/3132660626 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:49:59.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0fcc30640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f810c140 0x7ff0f819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0f7fff640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0f8102d70 0x7ff0f81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0f77fe640 1 -- 172.21.15.33:0/3132660626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0f8102d70 msgr2=0x7ff0f81983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0f77fe640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0f8102d70 0x7ff0f81983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0f77fe640 1 -- 172.21.15.33:0/3132660626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f810c140 msgr2=0x7ff0f819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0f77fe640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f810c140 0x7ff0f819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.498+0000 7ff0f77fe640 1 -- 172.21.15.33:0/3132660626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f819d340 con 0x7ff0f8103690 2024-04-02T13:49:59.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.499+0000 7ff0f77fe640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f8103690 0x7ff0f81988e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00d950 tx=0x7ff0ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.499+0000 7ff0f57fa640 1 -- 172.21.15.33:0/3132660626 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec017070 con 0x7ff0f8103690 2024-04-02T13:49:59.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.499+0000 7ff0f57fa640 1 -- 172.21.15.33:0/3132660626 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7ff0ec009c30 con 0x7ff0f8103690 2024-04-02T13:49:59.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.499+0000 7ff0fe693640 1 -- 172.21.15.33:0/3132660626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f819d630 con 0x7ff0f8103690 2024-04-02T13:49:59.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.499+0000 7ff0f57fa640 1 -- 172.21.15.33:0/3132660626 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec00f6b0 con 0x7ff0f8103690 2024-04-02T13:49:59.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.499+0000 7ff0fe693640 1 -- 172.21.15.33:0/3132660626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f81a4ec0 con 0x7ff0f8103690 2024-04-02T13:49:59.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.500+0000 7ff0fe693640 1 -- 172.21.15.33:0/3132660626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f8068d40 con 0x7ff0f8103690 2024-04-02T13:49:59.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.501+0000 7ff0f57fa640 1 -- 172.21.15.33:0/3132660626 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff0ec010a50 con 0x7ff0f8103690 2024-04-02T13:49:59.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.501+0000 7ff0f57fa640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff0e00a3d20 0x7ff0e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.502+0000 7ff0f7fff640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff0e00a3d20 0x7ff0e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.502+0000 7ff0f57fa640 1 -- 172.21.15.33:0/3132660626 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9666+0+0 (secure 0 0 0) 0x7ff0ec0d0860 con 0x7ff0f8103690 2024-04-02T13:49:59.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.502+0000 7ff0f7fff640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff0e00a3d20 0x7ff0e00a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8005ab0 tx=0x7ff0e8005fd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.507+0000 7ff0f57fa640 1 -- 172.21.15.33:0/3132660626 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff0ec009de0 con 0x7ff0f8103690 2024-04-02T13:49:59.717 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:59 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a[32386]: 2024-04-02T13:49:59.371+0000 7f4f13b36700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T13:49:59.718 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-b[66325]: ignoring --setuser ceph since I am not root 2024-04-02T13:49:59.718 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-b[66325]: ignoring --setgroup ceph since I am not root 2024-04-02T13:49:59.718 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-mds[66329]: -- 172.21.15.50:0/266198734 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561789183a40 con 0x561789157000 2024-04-02T13:49:59.718 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-b[66325]: starting mds.b at 2024-04-02T13:49:59.718 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-c[65619]: ignoring --setuser ceph since I am not root 2024-04-02T13:49:59.719 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-c[65619]: ignoring --setgroup ceph since I am not root 2024-04-02T13:49:59.719 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-mds[65623]: -- 172.21.15.145:0/1288588843 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a6e703a40 con 0x556a6e6d6c00 2024-04-02T13:49:59.719 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-c[65619]: starting mds.c at 2024-04-02T13:49:59.719 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-e[65690]: ignoring --setuser ceph since I am not root 2024-04-02T13:49:59.719 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-e[65690]: ignoring --setgroup ceph since I am not root 2024-04-02T13:49:59.719 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-mds[65694]: -- 172.21.15.50:0/2133033956 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563512667a40 con 0x56351263ac00 2024-04-02T13:49:59.719 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-e[65690]: starting mds.e at 2024-04-02T13:49:59.719 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-f[65309]: ignoring --setuser ceph since I am not root 2024-04-02T13:49:59.720 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-f[65309]: ignoring --setgroup ceph since I am not root 2024-04-02T13:49:59.720 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-mds[65313]: -- 172.21.15.145:0/3916118959 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555904fdfa40 con 0x555904fb3000 2024-04-02T13:49:59.720 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-f[65309]: starting mds.f at 2024-04-02T13:49:59.720 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-h[66015]: ignoring --setuser ceph since I am not root 2024-04-02T13:49:59.720 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-h[66015]: ignoring --setgroup ceph since I am not root 2024-04-02T13:49:59.720 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-mds[66019]: -- 172.21.15.50:0/3824445194 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e63a5f5a40 con 0x55e63a5c8800 2024-04-02T13:49:59.720 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-h[66015]: starting mds.h at 2024-04-02T13:49:59.720 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-i[65924]: ignoring --setuser ceph since I am not root 2024-04-02T13:49:59.720 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-i[65924]: ignoring --setgroup ceph since I am not root 2024-04-02T13:49:59.720 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-mds[65928]: -- 172.21.15.145:0/2391908251 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636b6acba40 con 0x5636b6a9e800 2024-04-02T13:49:59.720 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-i[65924]: starting mds.i at 2024-04-02T13:49:59.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:49:59.734+0000 7ff0fe693640 1 -- 172.21.15.33:0/3132660626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff0f8107290 con 0x7ff0f8103690 2024-04-02T13:49:59.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/854158376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:49:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/281776643' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T13:49:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-mon[35646]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T13:49:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-mon[35646]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T13:49:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/281776643' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T13:49:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-mon[35646]: osdmap e106: 12 total, 12 up, 12 in 2024-04-02T13:49:59.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:49:59 smithi145 ceph-mon[35646]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-02T13:49:59.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/854158376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:49:59.994 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/281776643' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T13:49:59.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-mon[35524]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T13:49:59.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-mon[35524]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T13:49:59.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/281776643' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T13:49:59.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-mon[35524]: osdmap e106: 12 total, 12 up, 12 in 2024-04-02T13:49:59.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:49:59 smithi050 ceph-mon[35524]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-02T13:50:00.002 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:59 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/854158376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:50:00.002 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:59 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/281776643' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T13:50:00.002 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:59 smithi033 ceph-mon[32390]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T13:50:00.002 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:59 smithi033 ceph-mon[32390]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T13:50:00.002 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:59 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/281776643' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T13:50:00.002 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:59 smithi033 ceph-mon[32390]: osdmap e106: 12 total, 12 up, 12 in 2024-04-02T13:50:00.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:49:59 smithi033 ceph-mon[32390]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-02T13:50:00.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a[32386]: 2024-04-02T13:49:59.999+0000 7f4f1633b700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-02T13:50:00.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a[32386]: 2024-04-02T13:49:59.999+0000 7f4f1633b700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-02T13:50:00.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a[32386]: 2024-04-02T13:49:59.999+0000 7f4f1633b700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-04-02T13:50:00.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a[32386]: 2024-04-02T13:49:59.999+0000 7f4f1633b700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-02T13:50:00.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a[32386]: 2024-04-02T13:49:59.999+0000 7f4f1633b700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-04-02T13:50:00.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.375+0000 7ff0f57fa640 1 -- 172.21.15.33:0/3132660626 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7ff0ec09c3d0 con 0x7ff0f8103690 2024-04-02T13:50:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.379+0000 7ff0fe693640 1 -- 172.21.15.33:0/3132660626 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff0e00a3d20 msgr2=0x7ff0e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.379+0000 7ff0fe693640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff0e00a3d20 0x7ff0e00a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8005ab0 tx=0x7ff0e8005fd0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.379+0000 7ff0fe693640 1 -- 172.21.15.33:0/3132660626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f8103690 msgr2=0x7ff0f81988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.379+0000 7ff0fe693640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f8103690 0x7ff0f81988e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00d950 tx=0x7ff0ec00de10 comp rx=0 tx=0).stop 2024-04-02T13:50:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.380+0000 7ff0fe693640 1 -- 172.21.15.33:0/3132660626 shutdown_connections 2024-04-02T13:50:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.380+0000 7ff0fe693640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7ff0e00a3d20 0x7ff0e00a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.380+0000 7ff0fe693640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0f810c140 0x7ff0f819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.380+0000 7ff0fe693640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f8103690 0x7ff0f81988e0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.380+0000 7ff0fe693640 1 --2- 172.21.15.33:0/3132660626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff0f8102d70 0x7ff0f81983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.380+0000 7ff0fe693640 1 -- 172.21.15.33:0/3132660626 >> 172.21.15.33:0/3132660626 conn(0x7ff0f8077b60 msgr2=0x7ff0f80fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.380+0000 7ff0fe693640 1 -- 172.21.15.33:0/3132660626 shutdown_connections 2024-04-02T13:50:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.380+0000 7ff0fe693640 1 -- 172.21.15.33:0/3132660626 wait complete. 2024-04-02T13:50:00.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-02T13:50:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.512+0000 7f1316374640 1 -- 172.21.15.33:0/3295743748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101330 msgr2=0x7f131010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.512+0000 7f1316374640 1 --2- 172.21.15.33:0/3295743748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101330 0x7f131010d840 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1308009cd0 tx=0x7f130802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.512+0000 7f1316374640 1 -- 172.21.15.33:0/3295743748 shutdown_connections 2024-04-02T13:50:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.512+0000 7f1316374640 1 --2- 172.21.15.33:0/3295743748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101330 0x7f131010d840 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.512+0000 7f1316374640 1 --2- 172.21.15.33:0/3295743748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1310100970 0x7f1310100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.512+0000 7f1316374640 1 --2- 172.21.15.33:0/3295743748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1310106980 0x7f1310106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.512+0000 7f1316374640 1 -- 172.21.15.33:0/3295743748 >> 172.21.15.33:0/3295743748 conn(0x7f1310077b00 msgr2=0x7f1310077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.512+0000 7f1316374640 1 -- 172.21.15.33:0/3295743748 shutdown_connections 2024-04-02T13:50:00.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.512+0000 7f1316374640 1 -- 172.21.15.33:0/3295743748 wait complete. 2024-04-02T13:50:00.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f1316374640 1 Processor -- start 2024-04-02T13:50:00.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f1316374640 1 -- start start 2024-04-02T13:50:00.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f1316374640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1310100970 0x7f1310198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f1316374640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101330 0x7f13101988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f1316374640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1310106980 0x7f131019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f1316374640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1310111e60 con 0x7f1310101330 2024-04-02T13:50:00.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f1316374640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1310111ce0 con 0x7f1310106980 2024-04-02T13:50:00.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f1316374640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1310111fe0 con 0x7f1310100970 2024-04-02T13:50:00.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f130f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101330 0x7f13101988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f130f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101330 0x7f13101988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36338/0 (socket says 172.21.15.33:36338) 2024-04-02T13:50:00.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f130f7fe640 1 -- 172.21.15.33:0/1052527359 learned_addr learned my addr 172.21.15.33:0/1052527359 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:50:00.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f1314911640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1310106980 0x7f131019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f130ffff640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1310100970 0x7f1310198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f130f7fe640 1 -- 172.21.15.33:0/1052527359 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1310100970 msgr2=0x7f1310198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f130f7fe640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1310100970 0x7f1310198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f130f7fe640 1 -- 172.21.15.33:0/1052527359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1310106980 msgr2=0x7f131019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f130f7fe640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1310106980 0x7f131019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f130f7fe640 1 -- 172.21.15.33:0/1052527359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131019d350 con 0x7f1310101330 2024-04-02T13:50:00.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.513+0000 7f130ffff640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1310100970 0x7f1310198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:00.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.514+0000 7f130f7fe640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101330 0x7f13101988a0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f130400e9f0 tx=0x7f130400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.514+0000 7f130d7fa640 1 -- 172.21.15.33:0/1052527359 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130400cd80 con 0x7f1310101330 2024-04-02T13:50:00.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.514+0000 7f130d7fa640 1 -- 172.21.15.33:0/1052527359 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f13040054f0 con 0x7f1310101330 2024-04-02T13:50:00.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.514+0000 7f1316374640 1 -- 172.21.15.33:0/1052527359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131019d640 con 0x7f1310101330 2024-04-02T13:50:00.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.514+0000 7f130d7fa640 1 -- 172.21.15.33:0/1052527359 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304010690 con 0x7f1310101330 2024-04-02T13:50:00.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.514+0000 7f1316374640 1 -- 172.21.15.33:0/1052527359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13101a4ed0 con 0x7f1310101330 2024-04-02T13:50:00.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.515+0000 7f1316374640 1 -- 172.21.15.33:0/1052527359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13100688f0 con 0x7f1310101330 2024-04-02T13:50:00.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.516+0000 7f130d7fa640 1 -- 172.21.15.33:0/1052527359 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1304004f20 con 0x7f1310101330 2024-04-02T13:50:00.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.516+0000 7f130d7fa640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12f80a3da0 0x7f12f80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.517+0000 7f130ffff640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12f80a3da0 0x7f12f80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.517+0000 7f130d7fa640 1 -- 172.21.15.33:0/1052527359 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9666+0+0 (secure 0 0 0) 0x7f13040d10e0 con 0x7f1310101330 2024-04-02T13:50:00.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.517+0000 7f130ffff640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12f80a3da0 0x7f12f80a6260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1300009ba0 tx=0x7f1300005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.521+0000 7f130d7fa640 1 -- 172.21.15.33:0/1052527359 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f130409d300 con 0x7f1310101330 2024-04-02T13:50:00.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:00.707+0000 7f1316374640 1 -- 172.21.15.33:0/1052527359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f131002d370 con 0x7f1310101330 2024-04-02T13:50:00.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: pgmap v349: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-02T13:50:00.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3132660626' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:50:00.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-02T13:50:00.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-02T13:50:00.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: fs cephfs is degraded 2024-04-02T13:50:00.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-02T13:50:00.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: fs cephfs is offline because no MDS is active for it. 2024-04-02T13:50:00.979 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: Cluster is now healthy 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3132660626' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.50:6834/91685528,v1:172.21.15.50:6835/91685528] up:boot 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.145:6835/2868795208,v1:172.21.15.145:6837/2868795208] up:boot 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.145:6834/2987144268,v1:172.21.15.145:6836/2987144268] up:boot 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.50:6836/1638101368,v1:172.21.15.50:6837/1638101368] up:boot 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.145:6838/2217880381,v1:172.21.15.145:6839/2217880381] up:boot 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: mds.? [v2:172.21.15.50:6838/2945342656,v1:172.21.15.50:6839/2945342656] up:boot 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: fsmap 12 up:standby 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:50:00.980 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:50:00.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:00 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1052527359' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:50:00.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: pgmap v349: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-02T13:50:00.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3132660626' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:50:00.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-02T13:50:00.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-02T13:50:00.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: fs cephfs is degraded 2024-04-02T13:50:00.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-02T13:50:00.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: fs cephfs is offline because no MDS is active for it. 2024-04-02T13:50:00.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T13:50:00.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T13:50:00.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: Cluster is now healthy 2024-04-02T13:50:00.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3132660626' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:50:00.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.50:6834/91685528,v1:172.21.15.50:6835/91685528] up:boot 2024-04-02T13:50:00.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.145:6835/2868795208,v1:172.21.15.145:6837/2868795208] up:boot 2024-04-02T13:50:00.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.145:6834/2987144268,v1:172.21.15.145:6836/2987144268] up:boot 2024-04-02T13:50:00.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.50:6836/1638101368,v1:172.21.15.50:6837/1638101368] up:boot 2024-04-02T13:50:00.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.145:6838/2217880381,v1:172.21.15.145:6839/2217880381] up:boot 2024-04-02T13:50:00.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: mds.? [v2:172.21.15.50:6838/2945342656,v1:172.21.15.50:6839/2945342656] up:boot 2024-04-02T13:50:00.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: fsmap 12 up:standby 2024-04-02T13:50:00.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:50:00.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:50:00.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:50:00.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:50:00.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:50:00.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:50:00.996 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:00 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1052527359' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:50:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: pgmap v349: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-02T13:50:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3132660626' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:50:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-02T13:50:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-02T13:50:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: fs cephfs is degraded 2024-04-02T13:50:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-02T13:50:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: fs cephfs is offline because no MDS is active for it. 2024-04-02T13:50:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T13:50:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T13:50:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: Cluster is now healthy 2024-04-02T13:50:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3132660626' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:50:01.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.50:6834/91685528,v1:172.21.15.50:6835/91685528] up:boot 2024-04-02T13:50:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.145:6835/2868795208,v1:172.21.15.145:6837/2868795208] up:boot 2024-04-02T13:50:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.145:6834/2987144268,v1:172.21.15.145:6836/2987144268] up:boot 2024-04-02T13:50:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.50:6836/1638101368,v1:172.21.15.50:6837/1638101368] up:boot 2024-04-02T13:50:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.145:6838/2217880381,v1:172.21.15.145:6839/2217880381] up:boot 2024-04-02T13:50:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: mds.? [v2:172.21.15.50:6838/2945342656,v1:172.21.15.50:6839/2945342656] up:boot 2024-04-02T13:50:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: fsmap 12 up:standby 2024-04-02T13:50:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:50:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:50:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:50:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:50:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:50:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:50:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:00 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1052527359' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:50:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.377+0000 7f130d7fa640 1 -- 172.21.15.33:0/1052527359 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) ==== 181+0+0 (secure 0 0 0) 0x7f130409cc00 con 0x7f1310101330 2024-04-02T13:50:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2024-04-02T13:50:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.380+0000 7f1316374640 1 -- 172.21.15.33:0/1052527359 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12f80a3da0 msgr2=0x7f12f80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.380+0000 7f1316374640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12f80a3da0 0x7f12f80a6260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1300009ba0 tx=0x7f1300005f90 comp rx=0 tx=0).stop 2024-04-02T13:50:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.380+0000 7f1316374640 1 -- 172.21.15.33:0/1052527359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101330 msgr2=0x7f13101988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.380+0000 7f1316374640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101330 0x7f13101988a0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f130400e9f0 tx=0x7f130400edf0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.380+0000 7f1316374640 1 -- 172.21.15.33:0/1052527359 shutdown_connections 2024-04-02T13:50:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.380+0000 7f1316374640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f12f80a3da0 0x7f12f80a6260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.380+0000 7f1316374640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1310106980 0x7f131019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.380+0000 7f1316374640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1310101330 0x7f13101988a0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.380+0000 7f1316374640 1 --2- 172.21.15.33:0/1052527359 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1310100970 0x7f1310198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.380+0000 7f1316374640 1 -- 172.21.15.33:0/1052527359 >> 172.21.15.33:0/1052527359 conn(0x7f1310077b00 msgr2=0x7f1310103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.380+0000 7f1316374640 1 -- 172.21.15.33:0/1052527359 shutdown_connections 2024-04-02T13:50:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.380+0000 7f1316374640 1 -- 172.21.15.33:0/1052527359 wait complete. 2024-04-02T13:50:01.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-02T13:50:01.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.513+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/4085875386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f0101330 msgr2=0x7fe4f010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.513+0000 7fe4f5d4d640 1 --2- 172.21.15.33:0/4085875386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f0101330 0x7fe4f010d840 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009cd0 tx=0x7fe4e802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.513+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/4085875386 shutdown_connections 2024-04-02T13:50:01.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.513+0000 7fe4f5d4d640 1 --2- 172.21.15.33:0/4085875386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f0101330 0x7fe4f010d840 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.513+0000 7fe4f5d4d640 1 --2- 172.21.15.33:0/4085875386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f0100970 0x7fe4f0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.513+0000 7fe4f5d4d640 1 --2- 172.21.15.33:0/4085875386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4f0106980 0x7fe4f0106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.513+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/4085875386 >> 172.21.15.33:0/4085875386 conn(0x7fe4f0077b00 msgr2=0x7fe4f0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:01.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.513+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/4085875386 shutdown_connections 2024-04-02T13:50:01.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.513+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/4085875386 wait complete. 2024-04-02T13:50:01.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.514+0000 7fe4f5d4d640 1 Processor -- start 2024-04-02T13:50:01.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.514+0000 7fe4f5d4d640 1 -- start start 2024-04-02T13:50:01.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.514+0000 7fe4f5d4d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4f0100970 0x7fe4f0197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.514+0000 7fe4f5d4d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f0101330 0x7fe4f01977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.514+0000 7fe4f5d4d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f0106980 0x7fe4f019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.514+0000 7fe4f5d4d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f0110d60 con 0x7fe4f0106980 2024-04-02T13:50:01.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.514+0000 7fe4f5d4d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f0110be0 con 0x7fe4f0101330 2024-04-02T13:50:01.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.514+0000 7fe4f5d4d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f0110ee0 con 0x7fe4f0100970 2024-04-02T13:50:01.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.514+0000 7fe4effff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f0106980 0x7fe4f019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4effff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f0106980 0x7fe4f019bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36362/0 (socket says 172.21.15.33:36362) 2024-04-02T13:50:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4effff640 1 -- 172.21.15.33:0/2572789567 learned_addr learned my addr 172.21.15.33:0/2572789567 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:50:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4eeffd640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f0101330 0x7fe4f01977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4ef7fe640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4f0100970 0x7fe4f0197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4effff640 1 -- 172.21.15.33:0/2572789567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4f0100970 msgr2=0x7fe4f0197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4effff640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4f0100970 0x7fe4f0197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4effff640 1 -- 172.21.15.33:0/2572789567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f0101330 msgr2=0x7fe4f01977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4effff640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f0101330 0x7fe4f01977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4effff640 1 -- 172.21.15.33:0/2572789567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f019c270 con 0x7fe4f0106980 2024-04-02T13:50:01.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4eeffd640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f0101330 0x7fe4f01977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:01.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4effff640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f0106980 0x7fe4f019bb40 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8005cc0 tx=0x7fe4e8034b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:01.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4ecff9640 1 -- 172.21.15.33:0/2572789567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8039050 con 0x7fe4f0106980 2024-04-02T13:50:01.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/2572789567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f019c500 con 0x7fe4f0106980 2024-04-02T13:50:01.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4ecff9640 1 -- 172.21.15.33:0/2572789567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7fe4e8045070 con 0x7fe4f0106980 2024-04-02T13:50:01.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4ecff9640 1 -- 172.21.15.33:0/2572789567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8004610 con 0x7fe4f0106980 2024-04-02T13:50:01.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.515+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/2572789567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f01a3df0 con 0x7fe4f0106980 2024-04-02T13:50:01.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.516+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/2572789567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f00688f0 con 0x7fe4f0106980 2024-04-02T13:50:01.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.517+0000 7fe4ecff9640 1 -- 172.21.15.33:0/2572789567 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4e8003970 con 0x7fe4f0106980 2024-04-02T13:50:01.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.518+0000 7fe4ecff9640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe4d80a3d20 0x7fe4d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.518+0000 7fe4ecff9640 1 -- 172.21.15.33:0/2572789567 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9112+0+0 (secure 0 0 0) 0x7fe4e80f3d40 con 0x7fe4f0106980 2024-04-02T13:50:01.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.518+0000 7fe4ef7fe640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe4d80a3d20 0x7fe4d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.518+0000 7fe4ef7fe640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe4d80a3d20 0x7fe4d80a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0005fd0 tx=0x7fe4e000b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:01.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.521+0000 7fe4ecff9640 1 -- 172.21.15.33:0/2572789567 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe4e80c0240 con 0x7fe4f0106980 2024-04-02T13:50:01.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:01.714+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/2572789567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe4f0108f00 con 0x7fe4f0106980 2024-04-02T13:50:02.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.379+0000 7fe4ecff9640 1 -- 172.21.15.33:0/2572789567 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) ==== 169+0+0 (secure 0 0 0) 0x7fe4e80bfb40 con 0x7fe4f0106980 2024-04-02T13:50:02.382 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2024-04-02T13:50:02.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.384+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/2572789567 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe4d80a3d20 msgr2=0x7fe4d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:02.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.384+0000 7fe4f5d4d640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe4d80a3d20 0x7fe4d80a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0005fd0 tx=0x7fe4e000b390 comp rx=0 tx=0).stop 2024-04-02T13:50:02.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.384+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/2572789567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f0106980 msgr2=0x7fe4f019bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:02.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.384+0000 7fe4f5d4d640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f0106980 0x7fe4f019bb40 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8005cc0 tx=0x7fe4e8034b40 comp rx=0 tx=0).stop 2024-04-02T13:50:02.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.384+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/2572789567 shutdown_connections 2024-04-02T13:50:02.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.384+0000 7fe4f5d4d640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7fe4d80a3d20 0x7fe4d80a61e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.384+0000 7fe4f5d4d640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f0106980 0x7fe4f019bb40 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.384+0000 7fe4f5d4d640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f0101330 0x7fe4f01977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.384+0000 7fe4f5d4d640 1 --2- 172.21.15.33:0/2572789567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4f0100970 0x7fe4f0197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.384+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/2572789567 >> 172.21.15.33:0/2572789567 conn(0x7fe4f0077b00 msgr2=0x7fe4f0103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:02.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.384+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/2572789567 shutdown_connections 2024-04-02T13:50:02.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.384+0000 7fe4f5d4d640 1 -- 172.21.15.33:0/2572789567 wait complete. 2024-04-02T13:50:02.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-02T13:50:02.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:02 smithi050 ceph-mon[35524]: pgmap v351: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 921 B/s wr, 8 op/s 2024-04-02T13:50:02.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:02 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/1052527359' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T13:50:02.494 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:02 smithi050 ceph-mon[35524]: osdmap e107: 12 total, 12 up, 12 in 2024-04-02T13:50:02.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:02 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2572789567' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:50:02.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:02 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:02.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.508+0000 7f43a7c0d640 1 -- 172.21.15.33:0/75977615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0107b50 msgr2=0x7f43a0107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:02.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.508+0000 7f43a7c0d640 1 --2- 172.21.15.33:0/75977615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0107b50 0x7f43a0107f30 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4390009cd0 tx=0x7f439002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.508+0000 7f43a7c0d640 1 -- 172.21.15.33:0/75977615 shutdown_connections 2024-04-02T13:50:02.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.508+0000 7f43a7c0d640 1 --2- 172.21.15.33:0/75977615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43a00696c0 0x7f43a010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.508+0000 7f43a7c0d640 1 --2- 172.21.15.33:0/75977615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a0068d00 0x7f43a0069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.508+0000 7f43a7c0d640 1 --2- 172.21.15.33:0/75977615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0107b50 0x7f43a0107f30 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.508+0000 7f43a7c0d640 1 -- 172.21.15.33:0/75977615 >> 172.21.15.33:0/75977615 conn(0x7f43a0077b00 msgr2=0x7f43a0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:02.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.508+0000 7f43a7c0d640 1 -- 172.21.15.33:0/75977615 shutdown_connections 2024-04-02T13:50:02.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.508+0000 7f43a7c0d640 1 -- 172.21.15.33:0/75977615 wait complete. 2024-04-02T13:50:02.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a7c0d640 1 Processor -- start 2024-04-02T13:50:02.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a7c0d640 1 -- start start 2024-04-02T13:50:02.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a7c0d640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43a0068d00 0x7f43a0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a7c0d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a00696c0 0x7f43a0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a7c0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0107b50 0x7f43a019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a7c0d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0111e70 con 0x7f43a0107b50 2024-04-02T13:50:02.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a7c0d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0111cf0 con 0x7f43a00696c0 2024-04-02T13:50:02.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a7c0d640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0111ff0 con 0x7f43a0068d00 2024-04-02T13:50:02.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a61aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0107b50 0x7f43a019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a51a8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a00696c0 0x7f43a0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a61aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0107b50 0x7f43a019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36380/0 (socket says 172.21.15.33:36380) 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a61aa640 1 -- 172.21.15.33:0/3614667415 learned_addr learned my addr 172.21.15.33:0/3614667415 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a59a9640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43a0068d00 0x7f43a0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a51a8640 1 -- 172.21.15.33:0/3614667415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43a0068d00 msgr2=0x7f43a0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a51a8640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43a0068d00 0x7f43a0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a51a8640 1 -- 172.21.15.33:0/3614667415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0107b50 msgr2=0x7f43a019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a51a8640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0107b50 0x7f43a019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a51a8640 1 -- 172.21.15.33:0/3614667415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a019d320 con 0x7f43a00696c0 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.509+0000 7f43a61aa640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0107b50 0x7f43a019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.510+0000 7f43a51a8640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a00696c0 0x7f43a0198870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f439400e9f0 tx=0x7f439400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.510+0000 7f439effd640 1 -- 172.21.15.33:0/3614667415 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439400ccc0 con 0x7f43a00696c0 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.510+0000 7f43a7c0d640 1 -- 172.21.15.33:0/3614667415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a019d610 con 0x7f43a00696c0 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.510+0000 7f439effd640 1 -- 172.21.15.33:0/3614667415 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(27 keys) ==== 1136+0+0 (secure 0 0 0) 0x7f4394010900 con 0x7f43a00696c0 2024-04-02T13:50:02.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.510+0000 7f439effd640 1 -- 172.21.15.33:0/3614667415 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394018960 con 0x7f43a00696c0 2024-04-02T13:50:02.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.510+0000 7f43a7c0d640 1 -- 172.21.15.33:0/3614667415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a01a4ea0 con 0x7f43a00696c0 2024-04-02T13:50:02.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.511+0000 7f43a7c0d640 1 -- 172.21.15.33:0/3614667415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a00688f0 con 0x7f43a00696c0 2024-04-02T13:50:02.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.512+0000 7f439effd640 1 -- 172.21.15.33:0/3614667415 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4394010ab0 con 0x7f43a00696c0 2024-04-02T13:50:02.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.513+0000 7f439effd640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43880a3d20 0x7f43880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.513+0000 7f43a59a9640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43880a3d20 0x7f43880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.513+0000 7f439effd640 1 -- 172.21.15.33:0/3614667415 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8582+0+0 (secure 0 0 0) 0x7f43940d1040 con 0x7f43a00696c0 2024-04-02T13:50:02.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.513+0000 7f43a59a9640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43880a3d20 0x7f43880a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4390005cc0 tx=0x7f4390005c50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:02.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.516+0000 7f439effd640 1 -- 172.21.15.33:0/3614667415 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f439409cff0 con 0x7f43a00696c0 2024-04-02T13:50:02.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:02.722+0000 7f43a7c0d640 1 -- 172.21.15.33:0/3614667415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f43a0109a70 con 0x7f43a00696c0 2024-04-02T13:50:02.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:02 smithi145 ceph-mon[35646]: pgmap v351: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 921 B/s wr, 8 op/s 2024-04-02T13:50:02.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:02 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/1052527359' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T13:50:02.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:02 smithi145 ceph-mon[35646]: osdmap e107: 12 total, 12 up, 12 in 2024-04-02T13:50:02.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:02 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2572789567' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:50:02.728 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:02 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:02 smithi033 ceph-mon[32390]: pgmap v351: 129 pgs: 129 active+clean; 126 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 921 B/s wr, 8 op/s 2024-04-02T13:50:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:02 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/1052527359' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T13:50:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:02 smithi033 ceph-mon[32390]: osdmap e107: 12 total, 12 up, 12 in 2024-04-02T13:50:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:02 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2572789567' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:50:02.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:02 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:03.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:03.720+0000 7f439effd640 1 -- 172.21.15.33:0/3614667415 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v109) ==== 178+0+0 (secure 0 0 0) 0x7f43940a12f0 con 0x7f43a00696c0 2024-04-02T13:50:03.724 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data_ec' removed 2024-04-02T13:50:03.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:03.724+0000 7f43a7c0d640 1 -- 172.21.15.33:0/3614667415 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43880a3d20 msgr2=0x7f43880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:03.724+0000 7f43a7c0d640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43880a3d20 0x7f43880a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4390005cc0 tx=0x7f4390005c50 comp rx=0 tx=0).stop 2024-04-02T13:50:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:03.724+0000 7f43a7c0d640 1 -- 172.21.15.33:0/3614667415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a00696c0 msgr2=0x7f43a0198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:03.724+0000 7f43a7c0d640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a00696c0 0x7f43a0198870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f439400e9f0 tx=0x7f439400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:50:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:03.724+0000 7f43a7c0d640 1 -- 172.21.15.33:0/3614667415 shutdown_connections 2024-04-02T13:50:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:03.724+0000 7f43a7c0d640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.33:6800/619073725,v1:172.21.15.33:6801/619073725] conn(0x7f43880a3d20 0x7f43880a61e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:03.724+0000 7f43a7c0d640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0107b50 0x7f43a019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:03.724+0000 7f43a7c0d640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a00696c0 0x7f43a0198870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:03.724+0000 7f43a7c0d640 1 --2- 172.21.15.33:0/3614667415 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43a0068d00 0x7f43a0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:03.724+0000 7f43a7c0d640 1 -- 172.21.15.33:0/3614667415 >> 172.21.15.33:0/3614667415 conn(0x7f43a0077b00 msgr2=0x7f43a00fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:03.725+0000 7f43a7c0d640 1 -- 172.21.15.33:0/3614667415 shutdown_connections 2024-04-02T13:50:03.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-02T13:50:03.725+0000 7f43a7c0d640 1 -- 172.21.15.33:0/3614667415 wait complete. 2024-04-02T13:50:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:03 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/2572789567' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T13:50:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:03 smithi145 ceph-mon[35646]: osdmap e108: 12 total, 12 up, 12 in 2024-04-02T13:50:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:03 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:03 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:03 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:03 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:03 smithi145 ceph-mon[35646]: from='client.? 172.21.15.33:0/3614667415' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:50:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:03 smithi145 ceph-mon[35646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:50:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:03 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:03 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:03 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:03 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:03.729 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:03 smithi145 ceph-mon[35646]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.742 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/2572789567' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T13:50:03.742 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 ceph-mon[32390]: osdmap e108: 12 total, 12 up, 12 in 2024-04-02T13:50:03.742 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.742 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.742 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.742 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.742 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 ceph-mon[32390]: from='client.? 172.21.15.33:0/3614667415' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:50:03.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 ceph-mon[32390]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:50:03.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:03.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:03.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 ceph-mon[32390]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.744 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:50:03.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:03 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/2572789567' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T13:50:03.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:03 smithi050 ceph-mon[35524]: osdmap e108: 12 total, 12 up, 12 in 2024-04-02T13:50:03.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:03 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:03 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:03 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:03 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:03 smithi050 ceph-mon[35524]: from='client.? 172.21.15.33:0/3614667415' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:50:03.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:03 smithi050 ceph-mon[35524]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:50:03.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:03 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:03 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:03 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:03.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:03 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:03.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:03 smithi050 ceph-mon[35524]: from='mgr.15456 172.21.15.33:0/3860678764' entity='mgr.x' 2024-04-02T13:50:03.775 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:50:03.805 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:50:03.835 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-02T13:50:03.835 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T13:50:03.851 DEBUG:teuthology.orchestra.run.smithi050:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T13:50:03.865 DEBUG:teuthology.orchestra.run.smithi145:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T13:50:03.890 INFO:tasks.cephadm:Stopping all daemons... 2024-04-02T13:50:03.891 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-02T13:50:03.891 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.a 2024-04-02T13:50:04.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:03 smithi033 systemd[1]: Stopping Ceph mon.a for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:04.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:04 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a[32386]: 2024-04-02T13:50:04.208+0000 7f4f19341700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:50:04.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:04 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a[32386]: 2024-04-02T13:50:04.208+0000 7f4f19341700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-02T13:50:04.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:04 smithi033 podman[99509]: 2024-04-02 13:50:04.250479395 +0000 UTC m=+0.172142788 container died 35cc49612e7c9978e7549cf9201b6fab093fd9913d514223163a73692b6ade50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-02T13:50:04.897 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:04 smithi033 podman[99509]: 2024-04-02 13:50:04.634747068 +0000 UTC m=+0.556410456 container cleanup 35cc49612e7c9978e7549cf9201b6fab093fd9913d514223163a73692b6ade50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-02T13:50:04.897 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 02 13:50:04 smithi033 bash[99509]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-a 2024-04-02T13:50:05.066 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.a.service' 2024-04-02T13:50:05.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:50:05.128 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-02T13:50:05.130 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-02T13:50:05.130 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.b 2024-04-02T13:50:05.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:05 smithi050 systemd[1]: Stopping Ceph mon.b for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:05.744 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:05 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-b[35516]: 2024-04-02T13:50:05.454+0000 7f60127b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:50:05.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:05 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-b[35516]: 2024-04-02T13:50:05.454+0000 7f60127b7700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-02T13:50:05.745 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:05 smithi050 podman[71883]: 2024-04-02 13:50:05.47483335 +0000 UTC m=+0.141495269 container died d9039c32ac9c28b12a710aa174ecba1dfec048b4c6ac817d2df3896a71414d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-b, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS) 2024-04-02T13:50:06.117 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:05 smithi050 podman[71883]: 2024-04-02 13:50:05.83836182 +0000 UTC m=+0.505023738 container cleanup d9039c32ac9c28b12a710aa174ecba1dfec048b4c6ac817d2df3896a71414d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-b, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-02T13:50:06.117 INFO:journalctl@ceph.mon.b.smithi050.stdout:Apr 02 13:50:05 smithi050 bash[71883]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-b 2024-04-02T13:50:06.273 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.b.service' 2024-04-02T13:50:06.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:50:06.319 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-02T13:50:06.319 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-02T13:50:06.320 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.c 2024-04-02T13:50:06.641 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:06 smithi145 systemd[1]: Stopping Ceph mon.c for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:06.978 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:06 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-c[35634]: 2024-04-02T13:50:06.638+0000 7f0ecf4eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:50:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:06 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-c[35634]: 2024-04-02T13:50:06.638+0000 7f0ecf4eb700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-02T13:50:06.981 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:06 smithi145 podman[71835]: 2024-04-02 13:50:06.874234745 +0000 UTC m=+0.361471395 container died 6fc32637c96fb23f295b8eeb947efd83e50fd843f68652aebb0f5061c1f93f10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-c, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_CLEAN=True, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:50:07.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:07 smithi145 podman[71835]: 2024-04-02 13:50:07.300103146 +0000 UTC m=+0.787339790 container cleanup 6fc32637c96fb23f295b8eeb947efd83e50fd843f68652aebb0f5061c1f93f10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-04-02T13:50:07.479 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:07 smithi145 bash[71835]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-c 2024-04-02T13:50:07.777 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.c.service' 2024-04-02T13:50:07.801 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:07 smithi145 podman[71848]: 2024-04-02 13:50:07.534911965 +0000 UTC m=+0.655893890 container remove 6fc32637c96fb23f295b8eeb947efd83e50fd843f68652aebb0f5061c1f93f10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mon-c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux ) 2024-04-02T13:50:07.802 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:07 smithi145 systemd[1]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.c.service: Deactivated successfully. 2024-04-02T13:50:07.802 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:07 smithi145 systemd[1]: Stopped Ceph mon.c for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:50:07.802 INFO:journalctl@ceph.mon.c.smithi145.stdout:Apr 02 13:50:07 smithi145 systemd[1]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mon.c.service: Consumed 12.480s CPU time. 2024-04-02T13:50:07.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:50:07.821 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-02T13:50:07.821 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-02T13:50:07.821 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.x 2024-04-02T13:50:08.142 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:50:07 smithi033 systemd[1]: Stopping Ceph mgr.x for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:08.443 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:50:08 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32621]: 2024-04-02T13:50:08.138+0000 7f55741a1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:50:08.443 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:50:08 smithi033 podman[99631]: 2024-04-02 13:50:08.170082172 +0000 UTC m=+0.155187914 container died 0bdee59f1aa5110f95979969ef77694eb11e4cf0e633ec8d57f935a409ed7b10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-02T13:50:08.754 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:50:08 smithi033 podman[99631]: 2024-04-02 13:50:08.532017137 +0000 UTC m=+0.517122887 container cleanup 0bdee59f1aa5110f95979969ef77694eb11e4cf0e633ec8d57f935a409ed7b10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326) 2024-04-02T13:50:08.754 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 02 13:50:08 smithi033 bash[99631]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-x 2024-04-02T13:50:08.995 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.x.service' 2024-04-02T13:50:09.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:50:09.033 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-02T13:50:09.033 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-02T13:50:09.033 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.y 2024-04-02T13:50:09.376 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:50:09 smithi050 systemd[1]: Stopping Ceph mgr.y for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:09.680 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:50:09 smithi050 podman[72006]: 2024-04-02 13:50:09.373212934 +0000 UTC m=+0.144580641 container died aea4ca15923545d6298de6eb1a8ea92347b151d3de47bd0418bde4133818ad7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-02T13:50:09.994 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:50:09 smithi050 podman[72006]: 2024-04-02 13:50:09.777909159 +0000 UTC m=+0.549276867 container cleanup aea4ca15923545d6298de6eb1a8ea92347b151d3de47bd0418bde4133818ad7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD) 2024-04-02T13:50:09.995 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Apr 02 13:50:09 smithi050 bash[72006]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-y 2024-04-02T13:50:10.150 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.y.service' 2024-04-02T13:50:10.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:50:10.188 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-02T13:50:10.188 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-02T13:50:10.188 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.z 2024-04-02T13:50:10.479 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:50:10 smithi145 systemd[1]: Stopping Ceph mgr.z for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:10.825 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:50:10 smithi145 podman[71959]: 2024-04-02 13:50:10.525799846 +0000 UTC m=+0.146384952 container died fb0412f4a58914714965d03d20b25ab96ba4cf3c242c607654c880a19afff1c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-02T13:50:11.159 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:50:10 smithi145 podman[71959]: 2024-04-02 13:50:10.922011887 +0000 UTC m=+0.542596977 container cleanup fb0412f4a58914714965d03d20b25ab96ba4cf3c242c607654c880a19afff1c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-02T13:50:11.159 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:50:10 smithi145 bash[71959]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z 2024-04-02T13:50:11.391 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.z.service' 2024-04-02T13:50:11.414 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:50:11 smithi145 podman[71973]: 2024-04-02 13:50:11.156982566 +0000 UTC m=+0.626631379 container remove fb0412f4a58914714965d03d20b25ab96ba4cf3c242c607654c880a19afff1c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mgr-z, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:50:11.414 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:50:11 smithi145 systemd[1]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-02T13:50:11.414 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:50:11 smithi145 systemd[1]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.z.service: Failed with result 'exit-code'. 2024-04-02T13:50:11.414 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:50:11 smithi145 systemd[1]: Stopped Ceph mgr.z for b5fd982e-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:50:11.414 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Apr 02 13:50:11 smithi145 systemd[1]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mgr.z.service: Consumed 20.535s CPU time. 2024-04-02T13:50:11.427 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:50:11.427 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-02T13:50:11.427 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-02T13:50:11.427 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.0 2024-04-02T13:50:11.747 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:11 smithi033 systemd[1]: Stopping Ceph osd.0 for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:12.004 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:11 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0[44516]: 2024-04-02T13:50:11.744+0000 7f83fc5f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:50:12.004 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:11 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0[44516]: 2024-04-02T13:50:11.744+0000 7f83fc5f2700 -1 osd.0 109 *** Got signal Terminated *** 2024-04-02T13:50:12.004 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:11 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0[44516]: 2024-04-02T13:50:11.744+0000 7f83fc5f2700 -1 osd.0 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:50:17.447 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:17 smithi033 podman[99757]: 2024-04-02 13:50:17.108706899 +0000 UTC m=+5.489224918 container died 5fd88653491d70f97210ebee1b2c0f1ed0e474b6f27793a514e06ee188986bdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-02T13:50:17.448 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:17 smithi033 podman[99757]: 2024-04-02 13:50:17.445265902 +0000 UTC m=+5.825783916 container cleanup 5fd88653491d70f97210ebee1b2c0f1ed0e474b6f27793a514e06ee188986bdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2) 2024-04-02T13:50:17.754 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:17 smithi033 bash[99757]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0 2024-04-02T13:50:17.754 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:17 smithi033 podman[99776]: 2024-04-02 13:50:17.646766064 +0000 UTC m=+0.529734220 container remove 5fd88653491d70f97210ebee1b2c0f1ed0e474b6f27793a514e06ee188986bdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux ) 2024-04-02T13:50:18.353 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:18 smithi033 podman[99869]: 2024-04-02 13:50:17.974630807 +0000 UTC m=+0.017560578 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:18.353 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:18 smithi033 podman[99869]: 2024-04-02 13:50:18.106039651 +0000 UTC m=+0.148969445 container create f424b592f6ad1d176b8b6d6690108cf65811ae3f5c7d18ef5dc5deced43a3ce4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-02T13:50:18.354 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:18 smithi033 podman[99869]: 2024-04-02 13:50:18.351262641 +0000 UTC m=+0.394192394 container init f424b592f6ad1d176b8b6d6690108cf65811ae3f5c7d18ef5dc5deced43a3ce4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:50:18.662 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:18 smithi033 podman[99869]: 2024-04-02 13:50:18.355578555 +0000 UTC m=+0.398508311 container start f424b592f6ad1d176b8b6d6690108cf65811ae3f5c7d18ef5dc5deced43a3ce4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:50:18.662 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:18 smithi033 podman[99869]: 2024-04-02 13:50:18.459290172 +0000 UTC m=+0.502219941 container attach f424b592f6ad1d176b8b6d6690108cf65811ae3f5c7d18ef5dc5deced43a3ce4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-02T13:50:18.939 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:18 smithi033 podman[99869]: 2024-04-02 13:50:18.659671618 +0000 UTC m=+0.702601378 container died f424b592f6ad1d176b8b6d6690108cf65811ae3f5c7d18ef5dc5deced43a3ce4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-02T13:50:19.189 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 02 13:50:19 smithi033 podman[99897]: 2024-04-02 13:50:19.163267659 +0000 UTC m=+0.498325831 container remove f424b592f6ad1d176b8b6d6690108cf65811ae3f5c7d18ef5dc5deced43a3ce4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-0-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:50:19.193 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.0.service' 2024-04-02T13:50:19.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:50:19.235 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-02T13:50:19.235 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-02T13:50:19.235 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.1 2024-04-02T13:50:19.688 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:19 smithi050 systemd[1]: Stopping Ceph osd.1 for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:19.994 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:19 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1[40110]: 2024-04-02T13:50:19.684+0000 7f4f63847700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:50:19.994 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:19 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1[40110]: 2024-04-02T13:50:19.684+0000 7f4f63847700 -1 osd.1 109 *** Got signal Terminated *** 2024-04-02T13:50:19.994 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:19 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1[40110]: 2024-04-02T13:50:19.684+0000 7f4f63847700 -1 osd.1 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:50:25.445 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:25 smithi050 podman[72130]: 2024-04-02 13:50:25.163487905 +0000 UTC m=+5.732742874 container died 5607cb37233d2c09da26ca0924f361e38e365a88d8e10a0d4549c3f04e071496 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-02T13:50:25.744 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:25 smithi050 podman[72130]: 2024-04-02 13:50:25.533420711 +0000 UTC m=+6.102675668 container cleanup 5607cb37233d2c09da26ca0924f361e38e365a88d8e10a0d4549c3f04e071496 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-02T13:50:25.745 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:25 smithi050 bash[72130]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1 2024-04-02T13:50:26.158 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:25 smithi050 podman[72149]: 2024-04-02 13:50:25.809894776 +0000 UTC m=+0.646820718 container remove 5607cb37233d2c09da26ca0924f361e38e365a88d8e10a0d4549c3f04e071496 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:50:26.494 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:26 smithi050 podman[72241]: 2024-04-02 13:50:26.058903523 +0000 UTC m=+0.017070358 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:26.495 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:26 smithi050 podman[72241]: 2024-04-02 13:50:26.20281114 +0000 UTC m=+0.160977995 container create 13ed21433290ad2fbdf5dc30e300570e0cb9c1306f85c97af9611e14e8042f07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326) 2024-04-02T13:50:26.898 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:26 smithi050 podman[72241]: 2024-04-02 13:50:26.59817001 +0000 UTC m=+0.556336843 container init 13ed21433290ad2fbdf5dc30e300570e0cb9c1306f85c97af9611e14e8042f07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:50:26.898 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:26 smithi050 podman[72241]: 2024-04-02 13:50:26.601541487 +0000 UTC m=+0.559708303 container start 13ed21433290ad2fbdf5dc30e300570e0cb9c1306f85c97af9611e14e8042f07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-04-02T13:50:26.899 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:26 smithi050 podman[72241]: 2024-04-02 13:50:26.69784725 +0000 UTC m=+0.656014072 container attach 13ed21433290ad2fbdf5dc30e300570e0cb9c1306f85c97af9611e14e8042f07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2) 2024-04-02T13:50:27.244 INFO:journalctl@ceph.osd.1.smithi050.stdout:Apr 02 13:50:26 smithi050 podman[72241]: 2024-04-02 13:50:26.896492835 +0000 UTC m=+0.854659671 container died 13ed21433290ad2fbdf5dc30e300570e0cb9c1306f85c97af9611e14e8042f07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:50:27.567 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.1.service' 2024-04-02T13:50:27.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:50:27.613 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-02T13:50:27.613 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-02T13:50:27.614 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.2 2024-04-02T13:50:27.978 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:27 smithi145 systemd[1]: Stopping Ceph osd.2 for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:28.478 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2[40798]: 2024-04-02T13:50:27.975+0000 7fc993839700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:50:28.479 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2[40798]: 2024-04-02T13:50:27.975+0000 7fc993839700 -1 osd.2 109 *** Got signal Terminated *** 2024-04-02T13:50:28.479 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2[40798]: 2024-04-02T13:50:27.975+0000 7fc993839700 -1 osd.2 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:50:33.832 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:33 smithi145 podman[72087]: 2024-04-02 13:50:33.530959193 +0000 UTC m=+5.726190369 container died 3872fc45c5b29003342e49eca5dac191ef8e98ccec17f2794cd457875844f10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-02T13:50:34.157 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:33 smithi145 podman[72087]: 2024-04-02 13:50:33.920090061 +0000 UTC m=+6.115321236 container cleanup 3872fc45c5b29003342e49eca5dac191ef8e98ccec17f2794cd457875844f10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-02T13:50:34.157 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:33 smithi145 bash[72087]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2 2024-04-02T13:50:34.467 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:34 smithi145 podman[72115]: 2024-04-02 13:50:34.154849045 +0000 UTC m=+0.626673805 container remove 3872fc45c5b29003342e49eca5dac191ef8e98ccec17f2794cd457875844f10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:50:34.729 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:34 smithi145 podman[72207]: 2024-04-02 13:50:34.368222575 +0000 UTC m=+0.017925749 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:34.729 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:34 smithi145 podman[72207]: 2024-04-02 13:50:34.480927027 +0000 UTC m=+0.130630245 container create 8e75bf1e6d78aea8e1a9ba8f4c45f8d6a2f5bd041e70a6b3f1cb0bbddc787633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-02T13:50:35.044 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:34 smithi145 podman[72207]: 2024-04-02 13:50:34.748910486 +0000 UTC m=+0.398613654 container init 8e75bf1e6d78aea8e1a9ba8f4c45f8d6a2f5bd041e70a6b3f1cb0bbddc787633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-02T13:50:35.044 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:34 smithi145 podman[72207]: 2024-04-02 13:50:34.752084526 +0000 UTC m=+0.401787679 container start 8e75bf1e6d78aea8e1a9ba8f4c45f8d6a2f5bd041e70a6b3f1cb0bbddc787633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:50:35.044 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:34 smithi145 podman[72207]: 2024-04-02 13:50:34.850921993 +0000 UTC m=+0.500625158 container attach 8e75bf1e6d78aea8e1a9ba8f4c45f8d6a2f5bd041e70a6b3f1cb0bbddc787633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-02T13:50:35.358 INFO:journalctl@ceph.osd.2.smithi145.stdout:Apr 02 13:50:35 smithi145 podman[72207]: 2024-04-02 13:50:35.041953062 +0000 UTC m=+0.691656237 container died 8e75bf1e6d78aea8e1a9ba8f4c45f8d6a2f5bd041e70a6b3f1cb0bbddc787633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-02T13:50:35.665 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.2.service' 2024-04-02T13:50:35.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:50:35.714 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-02T13:50:35.714 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-02T13:50:35.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.3 2024-04-02T13:50:36.001 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:35 smithi033 systemd[1]: Stopping Ceph osd.3 for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:36.254 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:35 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3[51469]: 2024-04-02T13:50:35.998+0000 7fd30d244700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:50:36.254 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:35 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3[51469]: 2024-04-02T13:50:35.998+0000 7fd30d244700 -1 osd.3 109 *** Got signal Terminated *** 2024-04-02T13:50:36.254 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:35 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3[51469]: 2024-04-02T13:50:35.998+0000 7fd30d244700 -1 osd.3 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:50:38.728 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:38 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:38.487+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:39.728 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:39 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:39.440+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:40.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:40.453+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:40.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:40.499+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:40.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:40 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:40.732+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:41.754 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:41 smithi033 podman[99976]: 2024-04-02 13:50:41.485899892 +0000 UTC m=+5.575780810 container died b33b1bfbe7d6053b28868bed5c2d68c0d57ee01223606b4ad126995b85c85216 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:50:41.978 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:41.520+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:41.979 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:41.495+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:41.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:41 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:41.727+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:42.127 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:41 smithi033 podman[99976]: 2024-04-02 13:50:41.857051589 +0000 UTC m=+5.946932517 container cleanup b33b1bfbe7d6053b28868bed5c2d68c0d57ee01223606b4ad126995b85c85216 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:50:42.127 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:41 smithi033 bash[99976]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3 2024-04-02T13:50:42.453 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:42 smithi033 podman[100005]: 2024-04-02 13:50:42.125025929 +0000 UTC m=+0.633281236 container remove b33b1bfbe7d6053b28868bed5c2d68c0d57ee01223606b4ad126995b85c85216 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True) 2024-04-02T13:50:42.747 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:42 smithi033 podman[100096]: 2024-04-02 13:50:42.450848451 +0000 UTC m=+0.094573193 container create 895e8791c4a37b0f44ee16f1806c2c6880c356bdb99189173b17995d281c925c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326) 2024-04-02T13:50:42.747 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:42 smithi033 podman[100096]: 2024-04-02 13:50:42.374056073 +0000 UTC m=+0.017780838 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:42.748 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:42 smithi033 podman[100096]: 2024-04-02 13:50:42.670437062 +0000 UTC m=+0.314161804 container init 895e8791c4a37b0f44ee16f1806c2c6880c356bdb99189173b17995d281c925c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:50:42.748 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:42 smithi033 podman[100096]: 2024-04-02 13:50:42.673618707 +0000 UTC m=+0.317343448 container start 895e8791c4a37b0f44ee16f1806c2c6880c356bdb99189173b17995d281c925c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:50:42.748 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:42 smithi033 podman[100096]: 2024-04-02 13:50:42.745057836 +0000 UTC m=+0.388782580 container attach 895e8791c4a37b0f44ee16f1806c2c6880c356bdb99189173b17995d281c925c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-02T13:50:42.978 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:42 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:42.527+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:42.979 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:42 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:42.503+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:42.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:42 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:42.695+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:43.004 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 02 13:50:42 smithi033 podman[100096]: 2024-04-02 13:50:42.978338869 +0000 UTC m=+0.622063610 container died 895e8791c4a37b0f44ee16f1806c2c6880c356bdb99189173b17995d281c925c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-02T13:50:43.504 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:43 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:43.231+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:43.662 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.3.service' 2024-04-02T13:50:43.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:50:43.706 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-02T13:50:43.706 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-02T13:50:43.706 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.4 2024-04-02T13:50:43.978 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:43.531+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:43.979 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:43.552+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:43.994 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:43 smithi050 systemd[1]: Stopping Ceph osd.4 for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:43.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:43 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:43.694+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:44.494 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:44 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4[46903]: 2024-04-02T13:50:44.030+0000 7f1120290700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:50:44.494 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:44 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4[46903]: 2024-04-02T13:50:44.030+0000 7f1120290700 -1 osd.4 109 *** Got signal Terminated *** 2024-04-02T13:50:44.494 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:44 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4[46903]: 2024-04-02T13:50:44.030+0000 7f1120290700 -1 osd.4 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:50:44.504 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:44 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:44.238+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:44.979 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:44 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:44.534+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:44.979 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:44 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:44.554+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:44.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:44 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:44.678+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:45.504 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:45 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:45.233+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:45.656 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:45 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:45.285+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:45.978 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:45 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:45.485+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:45.979 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:45 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:45.562+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:45.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:45 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:45.653+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:46.504 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:46 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:46.238+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:46.622 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:46 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:46.329+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:46.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:46 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:46.563+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:46.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:46 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:46.464+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:46.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:46 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:46.487+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:46.994 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:46 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4[46903]: 2024-04-02T13:50:46.619+0000 7f111c884700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.194170+0000 front 2024-04-02T13:50:24.194194+0000 (oldest deadline 2024-04-02T13:50:45.894309+0000) 2024-04-02T13:50:46.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:46 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:46.640+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:46.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:46 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:46.640+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:47.653 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:47 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:47.292+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:47.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:47.278+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:47.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:47.278+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:50:47.979 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:47 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:47.520+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:47.979 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:47 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:47.531+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:47.979 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:47 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:47.513+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:47.994 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:47 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4[46903]: 2024-04-02T13:50:47.650+0000 7f111c884700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.194170+0000 front 2024-04-02T13:50:24.194194+0000 (oldest deadline 2024-04-02T13:50:45.894309+0000) 2024-04-02T13:50:47.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:47 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:47.660+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:47.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:47 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:47.660+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:48.552 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:48 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:48.286+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:48.552 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:48 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:48.286+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:50:48.658 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:48 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:48.257+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:48.978 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:48 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:48.524+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:48.979 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:48 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:48.477+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:48.979 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:48 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:48.550+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:48.979 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:48 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:48.550+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:23.674667+0000 front 2024-04-02T13:50:23.674721+0000 (oldest deadline 2024-04-02T13:50:47.774668+0000) 2024-04-02T13:50:48.994 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:48 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4[46903]: 2024-04-02T13:50:48.690+0000 7f111c884700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.194170+0000 front 2024-04-02T13:50:24.194194+0000 (oldest deadline 2024-04-02T13:50:45.894309+0000) 2024-04-02T13:50:48.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:48 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:48.656+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:48.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:48 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:48.656+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:49.003 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:48 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:48.550+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:50:49.495 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:49 smithi050 podman[72360]: 2024-04-02 13:50:49.395800801 +0000 UTC m=+5.487349065 container died 5f7dcf531e342fa21f481c5eb3aa690ed6064991d8d229df4fc5031d569194ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:50:49.495 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:49 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:49.216+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:49.584 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:49 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:49.327+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:49.584 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:49 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:49.327+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:50:49.728 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:49 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:49.488+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:49.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:49 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:49.428+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:49.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:49 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:49.552+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:49.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:49 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:49.552+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:23.674667+0000 front 2024-04-02T13:50:23.674721+0000 (oldest deadline 2024-04-02T13:50:47.774668+0000) 2024-04-02T13:50:49.994 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:49 smithi050 podman[72360]: 2024-04-02 13:50:49.797512911 +0000 UTC m=+5.889061174 container cleanup 5f7dcf531e342fa21f481c5eb3aa690ed6064991d8d229df4fc5031d569194ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-02T13:50:49.995 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:49 smithi050 bash[72360]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4 2024-04-02T13:50:49.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:49 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:49.698+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:49.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:49 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:49.698+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:50.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:49 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:49.581+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:50:50.371 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:50 smithi050 podman[72379]: 2024-04-02 13:50:50.040955246 +0000 UTC m=+0.643868363 container remove 5f7dcf531e342fa21f481c5eb3aa690ed6064991d8d229df4fc5031d569194ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-02T13:50:50.372 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:50 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:50.245+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:50.543 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:50 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:50.282+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:50.544 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:50 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:50.282+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:50:50.660 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:50 smithi050 podman[72469]: 2024-04-02 13:50:50.272486822 +0000 UTC m=+0.017913395 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:50.660 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:50 smithi050 podman[72469]: 2024-04-02 13:50:50.400088187 +0000 UTC m=+0.145514734 container create c790efebbcac0e8d3b68c0a9e121cf88e79fc2178bbc8a3cc55f49497a1d301a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326) 2024-04-02T13:50:50.728 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:50 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:50.483+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:50.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:50 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:50.557+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:50.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:50 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:50.557+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:23.674667+0000 front 2024-04-02T13:50:23.674721+0000 (oldest deadline 2024-04-02T13:50:47.774668+0000) 2024-04-02T13:50:50.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:50 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:50.380+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:50.965 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:50 smithi050 podman[72469]: 2024-04-02 13:50:50.661222325 +0000 UTC m=+0.406648892 container init c790efebbcac0e8d3b68c0a9e121cf88e79fc2178bbc8a3cc55f49497a1d301a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-02T13:50:50.965 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:50 smithi050 podman[72469]: 2024-04-02 13:50:50.664297822 +0000 UTC m=+0.409724369 container start c790efebbcac0e8d3b68c0a9e121cf88e79fc2178bbc8a3cc55f49497a1d301a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-02T13:50:50.965 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:50 smithi050 podman[72469]: 2024-04-02 13:50:50.869778001 +0000 UTC m=+0.615204563 container attach c790efebbcac0e8d3b68c0a9e121cf88e79fc2178bbc8a3cc55f49497a1d301a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:50:50.966 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:50 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:50.657+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:50.966 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:50 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:50.657+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:51.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:50 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:50.540+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:50:51.230 INFO:journalctl@ceph.osd.4.smithi050.stdout:Apr 02 13:50:50 smithi050 podman[72469]: 2024-04-02 13:50:50.962918294 +0000 UTC m=+0.708344851 container died c790efebbcac0e8d3b68c0a9e121cf88e79fc2178bbc8a3cc55f49497a1d301a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-02T13:50:51.495 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:51 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:51.227+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:51.649 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.4.service' 2024-04-02T13:50:51.692 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:50:51.692 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-02T13:50:51.692 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-02T13:50:51.692 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.5 2024-04-02T13:50:51.722 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:51 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:51.395+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:51.722 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:51 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:51.603+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:51.722 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:51 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:51.603+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:23.674667+0000 front 2024-04-02T13:50:23.674721+0000 (oldest deadline 2024-04-02T13:50:47.774668+0000) 2024-04-02T13:50:51.722 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:51 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:51.442+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:51.753 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:51 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:51.538+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:50:51.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:51 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:51.295+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:51.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:51 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:51.295+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:50:51.978 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:51 smithi145 systemd[1]: Stopping Ceph osd.5 for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:51.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:51 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:51.655+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:51.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:51 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:51.655+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:52.414 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:52 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:52.016+0000 7f363caee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:50:52.414 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:52 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:52.016+0000 7f363caee700 -1 osd.5 109 *** Got signal Terminated *** 2024-04-02T13:50:52.414 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:52 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:52.016+0000 7f363caee700 -1 osd.5 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:50:52.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:52 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:52.184+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:52.583 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:52 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:52.321+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:52.583 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:52 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:52.321+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:50:52.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:52 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:52.411+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:52.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:52 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:52.628+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:52.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:52 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:52.628+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:23.674667+0000 front 2024-04-02T13:50:23.674721+0000 (oldest deadline 2024-04-02T13:50:47.774668+0000) 2024-04-02T13:50:52.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:52 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:52.465+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:52.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:52 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:52.643+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:52.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:52 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:52.643+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:53.003 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:52 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:52.580+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:50:53.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:53 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:53.186+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:53.668 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:53 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:53.408+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:53.668 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:53 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:53.464+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:53.712 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:53.557+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:50:53.713 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:53.368+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:53.713 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:53 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:53.368+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:50:53.979 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:53 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:53.665+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:53.979 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:53 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:53.665+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:23.674667+0000 front 2024-04-02T13:50:23.674721+0000 (oldest deadline 2024-04-02T13:50:47.774668+0000) 2024-04-02T13:50:53.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:53 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:53.636+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:53.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:53 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:53.636+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:54.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:54 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:54.166+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:54.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:54 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:54.166+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:50:54.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:54 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:54.494+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:54.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:54 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:54.451+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:54.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:54 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:54.451+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:50:54.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:54 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:54.648+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:54.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:54 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:54.648+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:23.674667+0000 front 2024-04-02T13:50:23.674721+0000 (oldest deadline 2024-04-02T13:50:47.774668+0000) 2024-04-02T13:50:54.730 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:54 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:54.648+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.775140+0000 front 2024-04-02T13:50:27.775097+0000 (oldest deadline 2024-04-02T13:50:53.675295+0000) 2024-04-02T13:50:54.753 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:54 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:54.577+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:50:54.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:54 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:54.338+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:54.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:54 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:54.338+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:50:54.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:54 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:54.674+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:54.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:54 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:54.674+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:55.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:55 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:55.179+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:55.495 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:55 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:55.179+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:50:55.607 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:55 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:55.309+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:55.607 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:55 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:55.309+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:50:55.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:55 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:55.510+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:55.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:55 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:55.459+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:55.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:55 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:55.459+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:50:55.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:55 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:55.624+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:55.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:55 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:55.624+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:23.674667+0000 front 2024-04-02T13:50:23.674721+0000 (oldest deadline 2024-04-02T13:50:47.774668+0000) 2024-04-02T13:50:55.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:55 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:55.624+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.775140+0000 front 2024-04-02T13:50:27.775097+0000 (oldest deadline 2024-04-02T13:50:53.675295+0000) 2024-04-02T13:50:55.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:55 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:55.666+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:55.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:55 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:55.666+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:56.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:55 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:55.602+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:50:56.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:56 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:56.202+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:56.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:56 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:56.202+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:50:56.622 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:56 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:56.349+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:56.622 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:56 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:56.349+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:50:56.622 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:56 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:56.349+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:50:56.979 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:56 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:56.502+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:56.979 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:56 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:56.502+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:50:56.979 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:56 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:56.475+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:56.979 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:56 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:56.475+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:50:56.979 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:56 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:56.638+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.073573+0000 front 2024-04-02T13:50:16.073620+0000 (oldest deadline 2024-04-02T13:50:39.573630+0000) 2024-04-02T13:50:56.979 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:56 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:56.638+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:23.674667+0000 front 2024-04-02T13:50:23.674721+0000 (oldest deadline 2024-04-02T13:50:47.774668+0000) 2024-04-02T13:50:56.979 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:56 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5[47283]: 2024-04-02T13:50:56.638+0000 7f36390e2700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.775140+0000 front 2024-04-02T13:50:27.775097+0000 (oldest deadline 2024-04-02T13:50:53.675295+0000) 2024-04-02T13:50:56.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:56 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:56.649+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:56.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:56 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:56.649+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:56.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:56 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:56.649+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:50:57.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:56 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:56.619+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:50:57.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:57 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:57.237+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:57.495 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:57 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:57.237+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:50:57.754 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:57 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:57.579+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:50:57.754 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:57 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:57.579+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.882814+0000 front 2024-04-02T13:50:27.883055+0000 (oldest deadline 2024-04-02T13:50:56.683576+0000) 2024-04-02T13:50:57.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:57 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:57.357+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:57.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:57 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:57.357+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:50:57.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:57 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:57.357+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:50:57.787 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:57 smithi145 podman[72314]: 2024-04-02 13:50:57.496703345 +0000 UTC m=+5.610567200 container died 74cfd1be6218407322f04ec0e136b26b7f4b051a6a1a5453fec439225c639694 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-02T13:50:57.788 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:57 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:57.514+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:57.788 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:57 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:57.514+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:50:57.788 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:57 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:57.476+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:57.788 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:57 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:57.476+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:50:57.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:57 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:57.641+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:57.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:57 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:57.641+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:57.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:57 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:57.641+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:50:58.154 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:57 smithi145 podman[72314]: 2024-04-02 13:50:57.892355358 +0000 UTC m=+6.006219213 container cleanup 74cfd1be6218407322f04ec0e136b26b7f4b051a6a1a5453fec439225c639694 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:50:58.154 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:57 smithi145 bash[72314]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5 2024-04-02T13:50:58.462 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:58 smithi145 podman[72332]: 2024-04-02 13:50:58.152224812 +0000 UTC m=+0.649969497 container remove 74cfd1be6218407322f04ec0e136b26b7f4b051a6a1a5453fec439225c639694 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:50:58.598 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:58 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:58.311+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:58.598 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:58 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:58.311+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:50:58.598 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:58 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:58.311+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:50:58.691 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:58 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:58.249+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:58.691 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:58 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:58.249+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:50:58.728 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:58 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:58.484+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:58.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:58 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:58.484+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:50:58.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:58 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:58.523+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:58.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:58 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:58.523+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:50:58.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:58 smithi145 podman[72421]: 2024-04-02 13:50:58.363296134 +0000 UTC m=+0.018080498 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:58.729 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:58 smithi145 podman[72421]: 2024-04-02 13:50:58.486734054 +0000 UTC m=+0.141518385 container create 347078e730b085239cbf398518d6532e29a1d30c781dcfc0d4e7033b55475a8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:50:58.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:58 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:58.687+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:58.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:58 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:58.687+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:58.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:58 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:58.687+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:50:59.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:58 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:58.595+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:50:59.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:58 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:58.595+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.882814+0000 front 2024-04-02T13:50:27.883055+0000 (oldest deadline 2024-04-02T13:50:56.683576+0000) 2024-04-02T13:50:59.051 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:58 smithi145 podman[72421]: 2024-04-02 13:50:58.756155289 +0000 UTC m=+0.410939622 container init 347078e730b085239cbf398518d6532e29a1d30c781dcfc0d4e7033b55475a8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-02T13:50:59.051 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:58 smithi145 podman[72421]: 2024-04-02 13:50:58.759395463 +0000 UTC m=+0.414179795 container start 347078e730b085239cbf398518d6532e29a1d30c781dcfc0d4e7033b55475a8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True) 2024-04-02T13:50:59.051 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:58 smithi145 podman[72421]: 2024-04-02 13:50:58.856211323 +0000 UTC m=+0.510995662 container attach 347078e730b085239cbf398518d6532e29a1d30c781dcfc0d4e7033b55475a8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-04-02T13:50:59.330 INFO:journalctl@ceph.osd.5.smithi145.stdout:Apr 02 13:50:59 smithi145 podman[72421]: 2024-04-02 13:50:59.049657669 +0000 UTC m=+0.704442021 container died 347078e730b085239cbf398518d6532e29a1d30c781dcfc0d4e7033b55475a8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True) 2024-04-02T13:50:59.645 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:59 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:59.290+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:50:59.645 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:59 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:59.290+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:50:59.645 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:50:59 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:50:59.290+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:50:59.670 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:59.255+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:50:59.671 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:50:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:50:59.255+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:50:59.704 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:59 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:59.475+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:50:59.704 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:50:59 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:50:59.475+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:50:59.704 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:59 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:59.482+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:50:59.705 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:50:59 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:50:59.482+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:50:59.722 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.5.service' 2024-04-02T13:50:59.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:50:59.804 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-02T13:50:59.804 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-02T13:50:59.804 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.6 2024-04-02T13:50:59.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:59.667+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:50:59.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:59.667+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:50:59.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:50:59 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:50:59.667+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:00.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:59 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:59.642+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:51:00.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:59 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:50:59.642+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.882814+0000 front 2024-04-02T13:50:27.883055+0000 (oldest deadline 2024-04-02T13:50:56.683576+0000) 2024-04-02T13:51:00.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:50:59 smithi033 systemd[1]: Stopping Ceph osd.6 for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:51:00.504 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:00.301+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:00.504 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:00.301+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:00.504 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:00.301+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:00.504 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:51:00.108+0000 7efed8693700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:51:00.504 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:51:00.108+0000 7efed8693700 -1 osd.6 109 *** Got signal Terminated *** 2024-04-02T13:51:00.504 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:51:00.108+0000 7efed8693700 -1 osd.6 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:51:00.703 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:00 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:00.293+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:51:00.703 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:00 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:00.293+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:51:00.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:00 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:00.456+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:00.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:00 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:00.456+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:00.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:00 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:00.454+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:00.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:00 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:00.454+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:00.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:00 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:00.700+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:00.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:00 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:00.700+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:00.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:00 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:00.700+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:01.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:51:00.684+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:51:01.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:00 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:51:00.684+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.882814+0000 front 2024-04-02T13:50:27.883055+0000 (oldest deadline 2024-04-02T13:50:56.683576+0000) 2024-04-02T13:51:01.694 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:01 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:01.247+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:51:01.694 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:01 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:01.247+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:51:01.719 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:01 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:01.300+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:01.719 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:01 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:01.300+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:01.720 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:01 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:01.300+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:01.728 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:01 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:01.452+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:01.728 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:01 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:01.452+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:01.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:01 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:01.411+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:01.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:01 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:01.411+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:01.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:01 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:01.691+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:01.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:01 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:01.691+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:01.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:01 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:01.691+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:02.003 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:01 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:51:01.716+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:51:02.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:01 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:51:01.716+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.882814+0000 front 2024-04-02T13:50:27.883055+0000 (oldest deadline 2024-04-02T13:50:56.683576+0000) 2024-04-02T13:51:02.663 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:02 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:02.248+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:51:02.663 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:02 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:02.248+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:51:02.694 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:02 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:02.277+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:02.694 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:02 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:02.277+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:02.694 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:02 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:02.277+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:02.728 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:02 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:02.443+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:02.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:02 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:02.443+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:02.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:02 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:02.367+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:02.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:02 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:02.367+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:02.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:02 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:02.367+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:02.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:02 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:02.659+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:02.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:02 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:02.659+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:02.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:02 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:02.659+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:03.003 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:02 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:51:02.691+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:51:03.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:02 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:51:02.691+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.882814+0000 front 2024-04-02T13:50:27.883055+0000 (oldest deadline 2024-04-02T13:50:56.683576+0000) 2024-04-02T13:51:03.645 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:03 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:03.264+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:51:03.646 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:03 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:03.264+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:51:03.661 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:03 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:03.318+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:03.662 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:03 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:03.318+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:03.662 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:03 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:03.318+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:03.662 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:03 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:03.318+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:03.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:03 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:03.399+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:03.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:03 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:03.399+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:03.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:03 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:03.337+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:03.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:03 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:03.337+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:03.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:03 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:03.337+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:03.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:03 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:03.642+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:03.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:03 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:03.642+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:03.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:03 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:03.642+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:03.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:03 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:03.642+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:04.003 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:03 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:51:03.658+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:51:04.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:03 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:51:03.658+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.882814+0000 front 2024-04-02T13:50:27.883055+0000 (oldest deadline 2024-04-02T13:50:56.683576+0000) 2024-04-02T13:51:04.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:04 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:04.227+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:51:04.495 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:04 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:04.227+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:51:04.661 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:04 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:04.367+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:04.661 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:04 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:04.367+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:04.661 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:04 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:04.367+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:04.661 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:04 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:04.367+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:04.728 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:04 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:04.336+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:04.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:04 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:04.336+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:04.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:04 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:04.336+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:04.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:04 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:04.367+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:04.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:04 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:04.367+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:04.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:04 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:04.367+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:04.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:04 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:04.611+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:04.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:04 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:04.611+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:04.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:04 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:04.611+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:04.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:04 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:04.611+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:05.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:04 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:51:04.658+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:21.481853+0000 front 2024-04-02T13:50:21.481759+0000 (oldest deadline 2024-04-02T13:50:47.882412+0000) 2024-04-02T13:51:05.004 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:04 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6[58477]: 2024-04-02T13:51:04.658+0000 7efed4c87700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.882814+0000 front 2024-04-02T13:50:27.883055+0000 (oldest deadline 2024-04-02T13:50:56.683576+0000) 2024-04-02T13:51:05.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:05 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:05.216+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:51:05.495 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:05 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:05.216+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:51:05.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:05 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:05.352+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:05.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:05 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:05.352+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:05.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:05 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:05.352+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:05.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:05 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:05.346+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:05.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:05 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:05.346+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:05.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:05 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:05.346+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:05.754 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:05 smithi033 podman[100202]: 2024-04-02 13:51:05.591230298 +0000 UTC m=+5.594834802 container died 37c10c1a396d80f8adeee04d8a7555f4828fb8518b8d65fa93308d225c1a53b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-02T13:51:05.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:05 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:05.348+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:05.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:05 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:05.348+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:05.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:05 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:05.348+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:05.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:05 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:05.348+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:05.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:05 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:05.591+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:05.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:05 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:05.591+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:05.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:05 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:05.591+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:05.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:05 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:05.591+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:06.046 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:05 smithi033 podman[100202]: 2024-04-02 13:51:05.976500549 +0000 UTC m=+5.980105039 container cleanup 37c10c1a396d80f8adeee04d8a7555f4828fb8518b8d65fa93308d225c1a53b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:51:06.046 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:05 smithi033 bash[100202]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6 2024-04-02T13:51:06.391 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:06 smithi033 podman[100219]: 2024-04-02 13:51:06.244632599 +0000 UTC m=+0.654947495 container remove 37c10c1a396d80f8adeee04d8a7555f4828fb8518b8d65fa93308d225c1a53b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-02T13:51:06.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:06 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:06.237+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:51:06.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:06 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:06.237+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:51:06.656 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:06 smithi033 podman[100319]: 2024-04-02 13:51:06.531186471 +0000 UTC m=+0.018666774 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:51:06.745 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:06 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:06.388+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:06.745 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:06 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:06.388+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:06.745 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:06 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:06.388+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:06.745 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:06 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:06.388+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:06.746 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:06 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:06.328+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:06.746 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:06 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:06.328+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:06.746 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:06 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:06.328+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:06.746 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:06 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:06.314+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:06.746 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:06 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:06.314+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:06.746 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:06 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:06.314+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:06.983 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:06 smithi033 podman[100319]: 2024-04-02 13:51:06.653891852 +0000 UTC m=+0.141372155 container create 1e5cf43b7a43e08d397d30cac58b226976aa3b09ce9f3cb0737c9b092c0d382f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-02T13:51:06.983 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:06 smithi033 podman[100319]: 2024-04-02 13:51:06.897985386 +0000 UTC m=+0.385465702 container init 1e5cf43b7a43e08d397d30cac58b226976aa3b09ce9f3cb0737c9b092c0d382f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6-deactivate, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-02T13:51:06.984 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:06 smithi033 podman[100319]: 2024-04-02 13:51:06.901596267 +0000 UTC m=+0.389076578 container start 1e5cf43b7a43e08d397d30cac58b226976aa3b09ce9f3cb0737c9b092c0d382f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6-deactivate, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-02T13:51:06.984 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:06 smithi033 podman[100319]: 2024-04-02 13:51:06.981286494 +0000 UTC m=+0.468766813 container attach 1e5cf43b7a43e08d397d30cac58b226976aa3b09ce9f3cb0737c9b092c0d382f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:51:06.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:06 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:06.599+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:06.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:06 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:06.599+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:06.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:06 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:06.599+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:06.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:06 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:06.599+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:07.254 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 02 13:51:07 smithi033 podman[100319]: 2024-04-02 13:51:07.201723116 +0000 UTC m=+0.689203414 container died 1e5cf43b7a43e08d397d30cac58b226976aa3b09ce9f3cb0737c9b092c0d382f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:51:07.580 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:07 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:07.253+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:51:07.580 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:07 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:07.253+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:51:07.580 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:07 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:07.253+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:40.682532+0000 front 2024-04-02T13:50:40.682399+0000 (oldest deadline 2024-04-02T13:51:06.582489+0000) 2024-04-02T13:51:07.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:07 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:07.287+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:07.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:07 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:07.287+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:07.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:07 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:07.287+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:07.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:07 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:07.342+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:07.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:07 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:07.342+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:07.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:07 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:07.342+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:07.753 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:07 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:07.386+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:07.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:07 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:07.386+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:07.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:07 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:07.386+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:07.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:07 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:07.386+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:07.772 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.6.service' 2024-04-02T13:51:07.814 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:51:07.814 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-02T13:51:07.814 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-02T13:51:07.815 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.7 2024-04-02T13:51:07.843 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:07 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:07.577+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:07.843 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:07 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:07.577+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:07.843 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:07 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:07.577+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:07.844 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:07 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:07.577+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:08.128 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:07 smithi050 systemd[1]: Stopping Ceph osd.7 for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:51:08.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:08 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:08.125+0000 7f15f104c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:51:08.495 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:08 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:08.125+0000 7f15f104c700 -1 osd.7 109 *** Got signal Terminated *** 2024-04-02T13:51:08.495 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:08 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:08.125+0000 7f15f104c700 -1 osd.7 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:51:08.495 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:08 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:08.294+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:51:08.495 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:08 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:08.294+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:51:08.495 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:08 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:08.294+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:40.682532+0000 front 2024-04-02T13:50:40.682399+0000 (oldest deadline 2024-04-02T13:51:06.582489+0000) 2024-04-02T13:51:08.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:08 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:08.323+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:08.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:08 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:08.323+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:08.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:08 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:08.323+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:08.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:08 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:08.240+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:08.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:08 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:08.240+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:08.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:08 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:08.240+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:08.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:08 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:08.390+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:08.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:08 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:08.390+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:08.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:08 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:08.390+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:08.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:08 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:08.390+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:08.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:08 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:08.545+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:08.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:08 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:08.545+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:08.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:08 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:08.545+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:08.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:08 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:08.545+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:09.409 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:09 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:09.246+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:51:09.409 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:09 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:09.246+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:51:09.409 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:09 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:09.246+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:40.682532+0000 front 2024-04-02T13:50:40.682399+0000 (oldest deadline 2024-04-02T13:51:06.582489+0000) 2024-04-02T13:51:09.728 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:09 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:09.242+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:09.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:09 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:09.242+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:09.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:09 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:09.242+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:09.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:09 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:09.334+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:09.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:09 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:09.334+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:09.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:09 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:09.334+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:09.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:09 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:09.524+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:09.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:09 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:09.524+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:09.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:09 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:09.524+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:09.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:09 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:09.524+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:09.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:09 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:09.369+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:09.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:09 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:09.369+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:09.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:09 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:09.369+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:09.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:09 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:09.369+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:10.574 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:10 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:10.271+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:51:10.574 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:10 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:10.271+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:51:10.574 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:10 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:10.271+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:40.682532+0000 front 2024-04-02T13:50:40.682399+0000 (oldest deadline 2024-04-02T13:51:06.582489+0000) 2024-04-02T13:51:10.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:10 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:10.269+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:10.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:10 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:10.269+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:10.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:10 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:10.269+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:10.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:10 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:10.378+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:10.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:10 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:10.378+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:10.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:10 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:10.378+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:10.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:10 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:10.322+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:10.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:10 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:10.322+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:10.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:10 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:10.322+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:10.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:10 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:10.322+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:10.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:10 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:10.322+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:10.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:10 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:10.570+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:10.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:10 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:10.570+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:10.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:10 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:10.570+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:10.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:10 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:10.570+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:11.494 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:11 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:11.237+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:51:11.495 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:11 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:11.237+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:51:11.495 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:11 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:11.237+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:40.682532+0000 front 2024-04-02T13:50:40.682399+0000 (oldest deadline 2024-04-02T13:51:06.582489+0000) 2024-04-02T13:51:11.728 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:11 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:11.290+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:11.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:11 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:11.290+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:11.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:11 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:11.290+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:11.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:11 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:11.381+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:11.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:11 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:11.381+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:11.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:11 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:11.381+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:11.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:11 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:11.381+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:11.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:11 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:11.361+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:11.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:11 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:11.361+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:11.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:11 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:11.361+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:11.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:11 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:11.361+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:11.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:11 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:11.361+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:11.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:11 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:11.562+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:11.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:11 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:11.562+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:11.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:11 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:11.562+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:11.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:11 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:11.562+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:12.524 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:12 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:12.244+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:19.277911+0000 front 2024-04-02T13:50:19.277862+0000 (oldest deadline 2024-04-02T13:50:45.177906+0000) 2024-04-02T13:51:12.524 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:12 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:12.244+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.880167+0000 front 2024-04-02T13:50:30.880341+0000 (oldest deadline 2024-04-02T13:50:53.780204+0000) 2024-04-02T13:51:12.524 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:12 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:12.244+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:40.682532+0000 front 2024-04-02T13:50:40.682399+0000 (oldest deadline 2024-04-02T13:51:06.582489+0000) 2024-04-02T13:51:12.524 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:12 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7[53897]: 2024-04-02T13:51:12.244+0000 7f15e9638700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:46.582898+0000 front 2024-04-02T13:50:46.582921+0000 (oldest deadline 2024-04-02T13:51:11.883062+0000) 2024-04-02T13:51:12.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:12 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:12.264+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:12.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:12 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:12.264+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:12.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:12 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:12.264+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:12.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:12 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:12.377+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:12.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:12 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:12.377+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:12.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:12 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:12.377+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:12.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:12 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:12.378+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:12.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:12 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:12.356+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:12.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:12 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:12.356+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:12.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:12 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:12.356+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:12.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:12 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:12.356+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:12.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:12 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:12.356+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:12.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:12 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:12.521+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:12.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:12 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:12.521+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:12.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:12 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:12.521+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:12.995 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:12 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:12.521+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:13.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:13 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:13.304+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:13.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:13 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:13.304+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:13.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:13 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:13.304+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:13.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:13 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:13.304+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.167864+0000 front 2024-04-02T13:50:47.167771+0000 (oldest deadline 2024-04-02T13:51:12.467494+0000) 2024-04-02T13:51:13.730 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:13 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:13.426+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:13.730 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:13 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:13.426+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:13.730 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:13 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:13.426+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:13.730 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:13 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:13.426+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:13.744 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:13 smithi050 podman[72575]: 2024-04-02 13:51:13.490204051 +0000 UTC m=+5.483970213 container died e36070aae524eefbc2a79f19288d2666edc09fb6df9b4ff193b07bf4700b3f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD) 2024-04-02T13:51:13.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:13 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:13.476+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:13.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:13 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:13.476+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:13.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:13 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:13.476+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:13.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:13 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:13.476+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:13.753 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:13 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:13.381+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:13.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:13 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:13.381+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:13.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:13 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:13.381+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:13.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:13 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:13.381+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:13.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:13 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:13.381+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:14.131 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:13 smithi050 podman[72575]: 2024-04-02 13:51:13.893856682 +0000 UTC m=+5.887622855 container cleanup e36070aae524eefbc2a79f19288d2666edc09fb6df9b4ff193b07bf4700b3f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:51:14.131 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:13 smithi050 bash[72575]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7 2024-04-02T13:51:14.459 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:14 smithi050 podman[72604]: 2024-04-02 13:51:14.128519694 +0000 UTC m=+0.636803857 container remove e36070aae524eefbc2a79f19288d2666edc09fb6df9b4ff193b07bf4700b3f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-02T13:51:14.726 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:14 smithi050 podman[72694]: 2024-04-02 13:51:14.360173592 +0000 UTC m=+0.016785678 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:51:14.726 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:14 smithi050 podman[72694]: 2024-04-02 13:51:14.462680508 +0000 UTC m=+0.119292582 container create 3ebd6d19f023d3756cb418f703b0b3dc086e0748fd91ec3476547a76db1e7ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:51:14.726 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:14 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:14.514+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:14.727 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:14 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:14.514+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:14.727 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:14 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:14.514+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:14.727 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:14 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:14.514+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:14.727 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:14 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:14.514+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:14.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:14 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:14.380+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:14.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:14 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:14.380+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:14.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:14 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:14.380+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:14.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:14 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:14.380+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:14.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:14 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:14.260+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:14.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:14 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:14.260+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:14.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:14 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:14.260+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:14.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:14 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:14.260+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.167864+0000 front 2024-04-02T13:50:47.167771+0000 (oldest deadline 2024-04-02T13:51:12.467494+0000) 2024-04-02T13:51:14.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:14 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:14.347+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:14.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:14 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:14.347+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:14.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:14 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:14.347+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:14.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:14 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:14.347+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:14.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:14 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:14.347+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:14.995 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:14 smithi050 podman[72694]: 2024-04-02 13:51:14.724007018 +0000 UTC m=+0.380619098 container init 3ebd6d19f023d3756cb418f703b0b3dc086e0748fd91ec3476547a76db1e7ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:51:14.995 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:14 smithi050 podman[72694]: 2024-04-02 13:51:14.727841613 +0000 UTC m=+0.384453688 container start 3ebd6d19f023d3756cb418f703b0b3dc086e0748fd91ec3476547a76db1e7ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-02T13:51:14.995 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:14 smithi050 podman[72694]: 2024-04-02 13:51:14.80691506 +0000 UTC m=+0.463527169 container attach 3ebd6d19f023d3756cb418f703b0b3dc086e0748fd91ec3476547a76db1e7ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:51:15.312 INFO:journalctl@ceph.osd.7.smithi050.stdout:Apr 02 13:51:15 smithi050 podman[72694]: 2024-04-02 13:51:15.019016682 +0000 UTC m=+0.675628757 container died 3ebd6d19f023d3756cb418f703b0b3dc086e0748fd91ec3476547a76db1e7ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0) 2024-04-02T13:51:15.479 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:15 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:15.218+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:15.479 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:15 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:15.218+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:15.479 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:15 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:15.218+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:15.479 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:15 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:15.218+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.167864+0000 front 2024-04-02T13:50:47.167771+0000 (oldest deadline 2024-04-02T13:51:12.467494+0000) 2024-04-02T13:51:15.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:15 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:15.351+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:15.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:15 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:15.351+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:15.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:15 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:15.351+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:15.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:15 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:15.351+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:15.613 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:15 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:15.474+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:15.613 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:15 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:15.474+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:15.613 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:15 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:15.474+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:15.614 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:15 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:15.474+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:15.614 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:15 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:15.474+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:15.643 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.7.service' 2024-04-02T13:51:15.729 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:51:15.729 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-02T13:51:15.729 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-02T13:51:15.729 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.8 2024-04-02T13:51:15.753 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:15 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:15.348+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:15.753 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:15 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:15.348+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:15.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:15 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:15.348+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:15.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:15 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:15.348+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:15.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:15 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:15.348+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:16.040 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:15 smithi145 systemd[1]: Stopping Ceph osd.8 for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:51:16.321 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:16 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:16.037+0000 7f74a65c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:51:16.321 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:16 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:16.037+0000 7f74a65c8700 -1 osd.8 109 *** Got signal Terminated *** 2024-04-02T13:51:16.321 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:16 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:16.037+0000 7f74a65c8700 -1 osd.8 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:51:16.321 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:16 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:16.264+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:16.321 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:16 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:16.264+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:16.321 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:16 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:16.264+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:16.321 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:16 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:16.264+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.167864+0000 front 2024-04-02T13:50:47.167771+0000 (oldest deadline 2024-04-02T13:51:12.467494+0000) 2024-04-02T13:51:16.728 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:16 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:16.317+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:16.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:16 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:16.317+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:16.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:16 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:16.317+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:16.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:16 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:16.317+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:16.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:16 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:16.479+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:16.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:16 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:16.479+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:16.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:16 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:16.479+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:16.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:16 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:16.479+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:16.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:16 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:16.479+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:16.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:16 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:16.314+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:16.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:16 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:16.314+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:16.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:16 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:16.315+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:16.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:16 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:16.315+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:16.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:16 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:16.315+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:17.504 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:17 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:17.347+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:17.504 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:17 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:17.347+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:17.504 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:17 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:17.347+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:17.504 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:17 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:17.347+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:17.504 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:17 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:17.347+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:17.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:17 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:17.275+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:17.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:17 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:17.275+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:17.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:17 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:17.275+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:17.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:17 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:17.275+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.167864+0000 front 2024-04-02T13:50:47.167771+0000 (oldest deadline 2024-04-02T13:51:12.467494+0000) 2024-04-02T13:51:17.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:17 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:17.330+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:17.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:17 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:17.330+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:17.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:17 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:17.330+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:17.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:17 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:17.330+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:17.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:17 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:17.446+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:17.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:17 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:17.446+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:17.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:17 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:17.446+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:17.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:17 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:17.446+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:17.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:17 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:17.446+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:18.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:18 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:18.284+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:18.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:18 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:18.284+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:18.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:18 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:18.284+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:18.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:18 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:18.284+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.167864+0000 front 2024-04-02T13:50:47.167771+0000 (oldest deadline 2024-04-02T13:51:12.467494+0000) 2024-04-02T13:51:18.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:18 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:18.355+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:18.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:18 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:18.356+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:18.730 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:18 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:18.356+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:18.730 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:18 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:18.356+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:18.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:18 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:18.413+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:18.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:18 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:18.413+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:18.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:18 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:18.413+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:18.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:18 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:18.413+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:18.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:18 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:18.413+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:18.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:18 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:18.328+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:18.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:18 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:18.328+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:18.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:18 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:18.328+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:18.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:18 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:18.328+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:18.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:18 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:18.328+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:19.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:19 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:19.308+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:19.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:19 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:19.308+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:19.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:19 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:19.308+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:19.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:19 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:19.308+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:19.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:19 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:19.281+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:19.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:19 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:19.281+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:19.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:19 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:19.281+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:19.729 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:19 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:19.281+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.167864+0000 front 2024-04-02T13:50:47.167771+0000 (oldest deadline 2024-04-02T13:51:12.467494+0000) 2024-04-02T13:51:19.730 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:19 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:19.281+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:52.467788+0000 front 2024-04-02T13:50:52.467865+0000 (oldest deadline 2024-04-02T13:51:18.367847+0000) 2024-04-02T13:51:19.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:19 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:19.412+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:19.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:19 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:19.412+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:19.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:19 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:19.412+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:19.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:19 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:19.412+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:19.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:19 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:19.412+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:19.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:19 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:19.375+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:19.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:19 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:19.375+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:19.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:19 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:19.375+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:19.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:19 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:19.375+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:19.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:19 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:19.375+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:19.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:19 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:19.375+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:54.019710+0000 front 2024-04-02T13:50:54.019789+0000 (oldest deadline 2024-04-02T13:51:18.719498+0000) 2024-04-02T13:51:20.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:20 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:20.276+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:20.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:20 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:20.276+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:20.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:20 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:20.276+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:20.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:20 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:20.276+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:20.730 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:20 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:20.277+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:24.364477+0000 front 2024-04-02T13:50:24.364503+0000 (oldest deadline 2024-04-02T13:50:46.064527+0000) 2024-04-02T13:51:20.730 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:20 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:20.277+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:27.765453+0000 front 2024-04-02T13:50:27.765379+0000 (oldest deadline 2024-04-02T13:50:53.665384+0000) 2024-04-02T13:51:20.730 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:20 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:20.277+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:37.766352+0000 front 2024-04-02T13:50:37.766318+0000 (oldest deadline 2024-04-02T13:51:03.666514+0000) 2024-04-02T13:51:20.730 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:20 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:20.277+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.167864+0000 front 2024-04-02T13:50:47.167771+0000 (oldest deadline 2024-04-02T13:51:12.467494+0000) 2024-04-02T13:51:20.730 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:20 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8[53813]: 2024-04-02T13:51:20.277+0000 7f749ebb4700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:52.467788+0000 front 2024-04-02T13:50:52.467865+0000 (oldest deadline 2024-04-02T13:51:18.367847+0000) 2024-04-02T13:51:20.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:20 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:20.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:20.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:20 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:20.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:20.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:20 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:20.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:20.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:20 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:20.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:20.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:20 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:20.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:20.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:20 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:20.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:20.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:20 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:20.383+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:20.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:20 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:20.383+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:20.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:20 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:20.383+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:20.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:20 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:20.383+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:20.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:20 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:20.383+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:20.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:20 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:20.383+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:54.019710+0000 front 2024-04-02T13:50:54.019789+0000 (oldest deadline 2024-04-02T13:51:18.719498+0000) 2024-04-02T13:51:21.590 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:21 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:21.264+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:21.590 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:21 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:21.264+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:21.590 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:21 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:21.264+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:21.590 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:21 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:21.264+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:21.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:21 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:21.360+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:21.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:21 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:21.360+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:21.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:21 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:21.360+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:21.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:21 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:21.360+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:21.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:21 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:21.360+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:21.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:21 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:21.360+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:21.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:21 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:21.379+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:21.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:21 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:21.379+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:21.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:21 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:21.379+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:21.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:21 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:21.379+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:21.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:21 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:21.379+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:21.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:21 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:21.379+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:54.019710+0000 front 2024-04-02T13:50:54.019789+0000 (oldest deadline 2024-04-02T13:51:18.719498+0000) 2024-04-02T13:51:21.848 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:21 smithi145 podman[72541]: 2024-04-02 13:51:21.58780284 +0000 UTC m=+5.667343336 container died d0eb6b2b3da25608a76205e4d55b6a641f0ae2c789a9b10421276a08289beb4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326) 2024-04-02T13:51:22.174 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:21 smithi145 podman[72541]: 2024-04-02 13:51:21.937364417 +0000 UTC m=+6.016904925 container cleanup d0eb6b2b3da25608a76205e4d55b6a641f0ae2c789a9b10421276a08289beb4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-02T13:51:22.174 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:21 smithi145 bash[72541]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8 2024-04-02T13:51:22.479 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:22 smithi145 podman[72558]: 2024-04-02 13:51:22.171932121 +0000 UTC m=+0.579903173 container remove d0eb6b2b3da25608a76205e4d55b6a641f0ae2c789a9b10421276a08289beb4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, ceph=True, GIT_CLEAN=True) 2024-04-02T13:51:22.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:22 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:22.304+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:22.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:22 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:22.304+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:22.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:22 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:22.304+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:22.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:22 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:22.304+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:22.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:22 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:22.347+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:22.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:22 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:22.347+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:22.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:22 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:22.347+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:22.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:22 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:22.347+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:22.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:22 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:22.347+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:22.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:22 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:22.347+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:22.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:22 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:22.370+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:22.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:22 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:22.370+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:22.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:22 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:22.370+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:22.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:22 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:22.370+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:22.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:22 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:22.370+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:22.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:22 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:22.370+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:54.019710+0000 front 2024-04-02T13:50:54.019789+0000 (oldest deadline 2024-04-02T13:51:18.719498+0000) 2024-04-02T13:51:22.786 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:22 smithi145 podman[72652]: 2024-04-02 13:51:22.400087489 +0000 UTC m=+0.017416175 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:51:22.786 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:22 smithi145 podman[72652]: 2024-04-02 13:51:22.51457432 +0000 UTC m=+0.131902977 container create 7175dc989786744794aca382105fc771b924ad08244f071523cdedaed78a36c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-02T13:51:22.786 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:22 smithi145 podman[72652]: 2024-04-02 13:51:22.783976118 +0000 UTC m=+0.401304797 container init 7175dc989786744794aca382105fc771b924ad08244f071523cdedaed78a36c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-04-02T13:51:23.075 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:22 smithi145 podman[72652]: 2024-04-02 13:51:22.787321868 +0000 UTC m=+0.404650527 container start 7175dc989786744794aca382105fc771b924ad08244f071523cdedaed78a36c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-04-02T13:51:23.075 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:22 smithi145 podman[72652]: 2024-04-02 13:51:22.892481118 +0000 UTC m=+0.509809785 container attach 7175dc989786744794aca382105fc771b924ad08244f071523cdedaed78a36c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-02T13:51:23.325 INFO:journalctl@ceph.osd.8.smithi145.stdout:Apr 02 13:51:23 smithi145 podman[72652]: 2024-04-02 13:51:23.073117278 +0000 UTC m=+0.690445944 container died 7175dc989786744794aca382105fc771b924ad08244f071523cdedaed78a36c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-02T13:51:23.325 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:23 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:23.321+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:23.325 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:23 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:23.321+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:23.724 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:23 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:23.321+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:23.724 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:23 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:23.321+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:23.724 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:23 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:23.321+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:23.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:23 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:23.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:23.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:23 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:23.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:23.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:23 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:23.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:23.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:23 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:23.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:23.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:23 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:23.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:23.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:23 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:23.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:23.753 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.8.service' 2024-04-02T13:51:23.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:23 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:23.392+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:23.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:23 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:23.392+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:23.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:23 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:23.392+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:23.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:23 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:23.392+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:23.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:23 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:23.392+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:23.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:23 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:23.392+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:54.019710+0000 front 2024-04-02T13:50:54.019789+0000 (oldest deadline 2024-04-02T13:51:18.719498+0000) 2024-04-02T13:51:23.793 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:51:23.793 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-02T13:51:23.793 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-02T13:51:23.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.9 2024-04-02T13:51:24.105 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:23 smithi033 systemd[1]: Stopping Ceph osd.9 for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:51:24.360 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:24 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:24.102+0000 7febee8f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:51:24.360 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:24 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:24.102+0000 7febee8f1700 -1 osd.9 109 *** Got signal Terminated *** 2024-04-02T13:51:24.360 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:24 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:24.102+0000 7febee8f1700 -1 osd.9 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:51:24.360 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:24 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:24.357+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:24.360 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:24 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:24.357+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:24.360 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:24 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:24.357+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:24.360 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:24 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:24.357+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:24.360 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:24 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:24.357+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:24.360 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:24 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:24.357+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:54.019710+0000 front 2024-04-02T13:50:54.019789+0000 (oldest deadline 2024-04-02T13:51:18.719498+0000) 2024-04-02T13:51:24.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:24 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:24.338+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:24.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:24 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:24.338+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:24.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:24 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:24.338+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:24.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:24 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:24.338+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:24.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:24 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:24.338+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:24.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:24 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:24.340+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:24.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:24 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:24.340+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:24.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:24 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:24.340+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:24.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:24 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:24.340+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:24.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:24 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:24.340+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:24.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:24 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:24.340+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:25.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:25 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:25.292+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:25.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:25 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:25.292+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:25.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:25 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:25.292+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:25.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:25 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:25.292+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:25.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:25 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:25.292+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:25.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:25 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:25.300+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:25.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:25 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:25.300+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:25.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:25 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:25.300+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:25.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:25 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:25.300+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:25.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:25 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:25.300+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:25.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:25 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:25.300+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:25.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:25 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:25.376+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:25.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:25 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:25.376+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:25.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:25 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:25.376+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:25.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:25 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:25.376+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:25.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:25 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:25.376+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:25.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:25 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:25.376+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:54.019710+0000 front 2024-04-02T13:50:54.019789+0000 (oldest deadline 2024-04-02T13:51:18.719498+0000) 2024-04-02T13:51:26.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:26 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:26.304+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:26.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:26 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:26.304+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:26.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:26 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:26.304+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:26.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:26 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:26.304+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:26.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:26 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:26.304+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:26.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:26 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:26.313+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:26.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:26 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:26.313+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:26.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:26 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:26.313+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:26.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:26 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:26.313+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:26.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:26 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:26.313+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:26.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:26 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:26.313+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:26.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:26 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:26.400+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:26.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:26 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:26.400+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:26.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:26 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:26.400+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:26.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:26 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:26.400+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:26.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:26 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:26.400+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:26.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:26 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:26.400+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:54.019710+0000 front 2024-04-02T13:50:54.019789+0000 (oldest deadline 2024-04-02T13:51:18.719498+0000) 2024-04-02T13:51:27.630 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:27.346+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:27.630 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:27.346+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:27.630 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:27.346+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:27.630 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:27.346+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:27.630 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:27.346+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:27.630 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:27 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:27.346+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:27.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:27 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:27.316+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:27.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:27 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:27.316+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:27.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:27 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:27.316+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:27.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:27 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:27.316+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:27.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:27 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:27.316+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:27.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:27 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:27.316+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:27.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:27 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:27.448+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:27.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:27 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:27.448+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:27.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:27 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:27.448+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:27.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:27 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:27.448+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:27.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:27 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:27.448+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:27.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:27 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:27.448+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:54.019710+0000 front 2024-04-02T13:50:54.019789+0000 (oldest deadline 2024-04-02T13:51:18.719498+0000) 2024-04-02T13:51:28.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:28 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:28.368+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:28.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:28 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:28.368+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:28.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:28 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:28.368+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:28.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:28 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:28.368+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:28.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:28 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:28.368+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:28.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:28 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:28.368+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:28.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:28 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:28.296+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:28.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:28 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:28.296+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:28.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:28 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:28.296+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:28.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:28 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:28.296+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:28.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:28 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:28.296+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:28.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:28 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:28.296+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:28.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:28 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:28.484+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.614720+0000 front 2024-04-02T13:50:16.614773+0000 (oldest deadline 2024-04-02T13:50:42.514896+0000) 2024-04-02T13:51:28.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:28 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:28.484+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:22.515299+0000 front 2024-04-02T13:50:22.515323+0000 (oldest deadline 2024-04-02T13:50:47.215425+0000) 2024-04-02T13:51:28.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:28 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:28.484+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:29.516508+0000 front 2024-04-02T13:50:29.516323+0000 (oldest deadline 2024-04-02T13:50:55.416382+0000) 2024-04-02T13:51:28.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:28 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:28.484+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:39.417447+0000 front 2024-04-02T13:50:39.417514+0000 (oldest deadline 2024-04-02T13:51:02.917574+0000) 2024-04-02T13:51:28.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:28 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:28.484+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:47.618743+0000 front 2024-04-02T13:50:47.618643+0000 (oldest deadline 2024-04-02T13:51:09.918554+0000) 2024-04-02T13:51:28.754 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:28 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9[65250]: 2024-04-02T13:51:28.484+0000 7febe6edd700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:54.019710+0000 front 2024-04-02T13:50:54.019789+0000 (oldest deadline 2024-04-02T13:51:18.719498+0000) 2024-04-02T13:51:29.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:29.378+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:29.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:29.378+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:29.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:29.378+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:29.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:29.378+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:29.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:29.378+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:29.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:29 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:29.378+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:29.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:29 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:29.329+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:29.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:29 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:29.329+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:29.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:29 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:29.329+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:29.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:29 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:29.329+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:29.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:29 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:29.329+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:29.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:29 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:29.329+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:29.930 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:29 smithi033 podman[100424]: 2024-04-02 13:51:29.592166085 +0000 UTC m=+5.605393908 container died ca187e333dd69ea41e65aa6382a0480bc6ba9f4e217b2d8eb782a9a535bc232a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-02T13:51:29.930 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:29 smithi033 podman[100424]: 2024-04-02 13:51:29.9274632 +0000 UTC m=+5.940691010 container cleanup ca187e333dd69ea41e65aa6382a0480bc6ba9f4e217b2d8eb782a9a535bc232a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-02T13:51:30.254 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:29 smithi033 bash[100424]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9 2024-04-02T13:51:30.254 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:30 smithi033 podman[100442]: 2024-04-02 13:51:30.103694074 +0000 UTC m=+0.510814811 container remove ca187e333dd69ea41e65aa6382a0480bc6ba9f4e217b2d8eb782a9a535bc232a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:51:30.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:30 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:30.401+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:30.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:30 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:30.401+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:30.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:30 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:30.401+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:30.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:30 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:30.401+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:30.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:30 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:30.401+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:30.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:30 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:30.401+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:30.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:30 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:30.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:30.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:30 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:30.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:30.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:30 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:30.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:30.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:30 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:30.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:30.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:30 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:30.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:30.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:30 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:30.377+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:30.784 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:30 smithi033 podman[100536]: 2024-04-02 13:51:30.521140087 +0000 UTC m=+0.111152826 container create 53369cbbaa6ab72068194896b6ec13d1fe8e6cba9954f0f9172b5d1d18831fbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-02T13:51:30.784 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:30 smithi033 podman[100536]: 2024-04-02 13:51:30.427883328 +0000 UTC m=+0.017896017 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:51:30.784 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:30 smithi033 podman[100536]: 2024-04-02 13:51:30.781964804 +0000 UTC m=+0.371977478 container init 53369cbbaa6ab72068194896b6ec13d1fe8e6cba9954f0f9172b5d1d18831fbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-04-02T13:51:31.083 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:30 smithi033 podman[100536]: 2024-04-02 13:51:30.785113137 +0000 UTC m=+0.375125812 container start 53369cbbaa6ab72068194896b6ec13d1fe8e6cba9954f0f9172b5d1d18831fbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:51:31.083 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:30 smithi033 podman[100536]: 2024-04-02 13:51:30.865193558 +0000 UTC m=+0.455206233 container attach 53369cbbaa6ab72068194896b6ec13d1fe8e6cba9954f0f9172b5d1d18831fbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9-deactivate, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:51:31.369 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 02 13:51:31 smithi033 podman[100536]: 2024-04-02 13:51:31.081378551 +0000 UTC m=+0.671391231 container died 53369cbbaa6ab72068194896b6ec13d1fe8e6cba9954f0f9172b5d1d18831fbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:51:31.672 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.9.service' 2024-04-02T13:51:31.713 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:51:31.713 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-02T13:51:31.713 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-02T13:51:31.713 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.10 2024-04-02T13:51:31.728 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:31 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:31.401+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:31.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:31 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:31.401+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:31.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:31 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:31.401+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:31.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:31 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:31.401+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:31.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:31 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:31.401+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:31.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:31 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:31.401+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:31.743 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:31 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:31.356+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:31.743 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:31 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:31.356+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:31.743 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:31 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:31.356+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:31.743 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:31 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:31.356+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:31.743 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:31 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:31.356+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:31.743 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:31 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:31.356+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:31.743 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:31 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:31.356+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.598480+0000 front 2024-04-02T13:51:04.598638+0000 (oldest deadline 2024-04-02T13:51:30.498396+0000) 2024-04-02T13:51:31.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:31 smithi050 systemd[1]: Stopping Ceph osd.10 for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:51:32.383 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:32.005+0000 7f5e84532700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:51:32.383 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:32.005+0000 7f5e84532700 -1 osd.10 109 *** Got signal Terminated *** 2024-04-02T13:51:32.383 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:32.005+0000 7f5e84532700 -1 osd.10 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:51:32.383 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:32.380+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:32.383 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:32.380+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:32.383 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:32.380+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:32.383 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:32.380+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:32.383 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:32.380+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:32.384 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:32.380+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:32.384 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:32 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:32.380+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.598480+0000 front 2024-04-02T13:51:04.598638+0000 (oldest deadline 2024-04-02T13:51:30.498396+0000) 2024-04-02T13:51:32.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:32 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:32.400+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:32.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:32 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:32.400+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:32.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:32 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:32.400+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:32.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:32 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:32.400+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:32.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:32 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:32.400+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:32.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:32 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:32.400+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:33.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:33 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:33.355+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:33.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:33 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:33.355+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:33.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:33 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:33.355+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:33.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:33 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:33.355+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:33.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:33 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:33.355+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:33.730 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:33 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:33.355+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:33.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:33 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:33.415+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:33.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:33 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:33.416+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:33.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:33 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:33.416+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:33.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:33 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:33.416+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:33.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:33 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:33.416+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:33.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:33 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:33.416+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:33.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:33 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:33.416+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.598480+0000 front 2024-04-02T13:51:04.598638+0000 (oldest deadline 2024-04-02T13:51:30.498396+0000) 2024-04-02T13:51:34.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:34 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:34.355+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:34.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:34 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:34.355+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:34.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:34 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:34.355+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:34.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:34 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:34.355+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:34.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:34 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:34.355+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:34.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:34 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:34.355+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:34.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:34 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:34.451+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:34.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:34 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:34.451+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:34.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:34 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:34.451+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:34.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:34 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:34.451+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:34.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:34 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:34.451+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:34.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:34 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:34.451+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:34.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:34 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:34.451+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.598480+0000 front 2024-04-02T13:51:04.598638+0000 (oldest deadline 2024-04-02T13:51:30.498396+0000) 2024-04-02T13:51:35.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:35 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:35.345+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:35.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:35 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:35.345+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:35.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:35 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:35.345+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:35.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:35 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:35.345+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:35.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:35 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:35.345+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:35.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:35 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:35.345+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:35.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:35 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:35.430+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:35.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:35 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:35.430+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:35.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:35 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:35.430+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:35.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:35 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:35.430+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:35.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:35 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:35.430+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:35.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:35 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:35.430+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:35.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:35 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:35.430+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.598480+0000 front 2024-04-02T13:51:04.598638+0000 (oldest deadline 2024-04-02T13:51:30.498396+0000) 2024-04-02T13:51:36.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:36 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:36.298+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:36.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:36 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:36.298+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:36.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:36 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:36.298+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:36.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:36 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:36.298+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:36.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:36 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:36.298+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:36.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:36 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:36.298+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:36.744 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:36 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:36.389+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:13.890552+0000 front 2024-04-02T13:50:13.890427+0000 (oldest deadline 2024-04-02T13:50:39.790469+0000) 2024-04-02T13:51:36.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:36 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:36.389+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-04-02T13:50:20.291394+0000 front 2024-04-02T13:50:20.291548+0000 (oldest deadline 2024-04-02T13:50:46.191564+0000) 2024-04-02T13:51:36.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:36 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:36.389+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:30.193330+0000 front 2024-04-02T13:50:30.193239+0000 (oldest deadline 2024-04-02T13:50:56.093156+0000) 2024-04-02T13:51:36.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:36 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:36.389+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:38.294864+0000 front 2024-04-02T13:50:38.294961+0000 (oldest deadline 2024-04-02T13:51:03.594807+0000) 2024-04-02T13:51:36.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:36 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:36.389+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:48.796230+0000 front 2024-04-02T13:50:48.796192+0000 (oldest deadline 2024-04-02T13:51:13.496325+0000) 2024-04-02T13:51:36.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:36 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:36.389+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:56.997684+0000 front 2024-04-02T13:50:56.997611+0000 (oldest deadline 2024-04-02T13:51:19.897399+0000) 2024-04-02T13:51:36.745 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:36 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10[60944]: 2024-04-02T13:51:36.389+0000 7f5e7cb1e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.598480+0000 front 2024-04-02T13:51:04.598638+0000 (oldest deadline 2024-04-02T13:51:30.498396+0000) 2024-04-02T13:51:37.724 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:37 smithi050 podman[72801]: 2024-04-02 13:51:37.337345338 +0000 UTC m=+5.435900893 container died 4b100a3a758cd0a37246a7722ddc2a94503d5062d86aef919209723c58a19f99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-02T13:51:37.728 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:37 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:37.262+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:37.728 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:37 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:37.262+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:37.728 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:37 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:37.262+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:37.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:37 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:37.262+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:37.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:37 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:37.262+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:37.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:37 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:37.262+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:37.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:37 smithi050 podman[72801]: 2024-04-02 13:51:37.804737231 +0000 UTC m=+5.903292770 container cleanup 4b100a3a758cd0a37246a7722ddc2a94503d5062d86aef919209723c58a19f99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-02T13:51:37.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:37 smithi050 bash[72801]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10 2024-04-02T13:51:38.423 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:38 smithi050 podman[72818]: 2024-04-02 13:51:38.081239893 +0000 UTC m=+0.746462008 container remove 4b100a3a758cd0a37246a7722ddc2a94503d5062d86aef919209723c58a19f99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326) 2024-04-02T13:51:38.704 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:38 smithi050 podman[72912]: 2024-04-02 13:51:38.323300826 +0000 UTC m=+0.017570994 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:51:38.704 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:38 smithi050 podman[72912]: 2024-04-02 13:51:38.42367182 +0000 UTC m=+0.117941978 container create b1ca921c2fbc066ed624a20674257212fc96326ff11f1461e50bab5417f8ae90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:51:38.704 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:38 smithi050 podman[72912]: 2024-04-02 13:51:38.618153425 +0000 UTC m=+0.312423597 container init b1ca921c2fbc066ed624a20674257212fc96326ff11f1461e50bab5417f8ae90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-02T13:51:38.704 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:38 smithi050 podman[72912]: 2024-04-02 13:51:38.621505706 +0000 UTC m=+0.315775865 container start b1ca921c2fbc066ed624a20674257212fc96326ff11f1461e50bab5417f8ae90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-02T13:51:38.704 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:38 smithi050 podman[72912]: 2024-04-02 13:51:38.701175189 +0000 UTC m=+0.395445364 container attach b1ca921c2fbc066ed624a20674257212fc96326ff11f1461e50bab5417f8ae90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-04-02T13:51:38.728 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:38 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:38.311+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:38.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:38 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:38.311+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:38.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:38 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:38.311+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:38.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:38 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:38.311+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:38.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:38 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:38.311+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:38.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:38 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:38.311+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:38.994 INFO:journalctl@ceph.osd.10.smithi050.stdout:Apr 02 13:51:38 smithi050 podman[72912]: 2024-04-02 13:51:38.918279847 +0000 UTC m=+0.612550008 container died b1ca921c2fbc066ed624a20674257212fc96326ff11f1461e50bab5417f8ae90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-10-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2) 2024-04-02T13:51:39.705 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.10.service' 2024-04-02T13:51:39.728 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:39 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:39.280+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:39.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:39 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:39.280+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:39.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:39 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:39.280+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:39.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:39 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:39.280+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:39.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:39 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:39.280+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:39.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:39 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:39.280+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:39.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:39 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:39.280+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-04-02T13:51:12.460269+0000 front 2024-04-02T13:51:12.460256+0000 (oldest deadline 2024-04-02T13:51:38.360079+0000) 2024-04-02T13:51:39.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:51:39.788 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-02T13:51:39.788 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-02T13:51:39.788 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.11 2024-04-02T13:51:40.117 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:39 smithi145 systemd[1]: Stopping Ceph osd.11 for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:51:40.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:40.114+0000 7fc5429e1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:51:40.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:40.114+0000 7fc5429e1700 -1 osd.11 109 *** Got signal Terminated *** 2024-04-02T13:51:40.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:40.114+0000 7fc5429e1700 -1 osd.11 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:51:40.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:40.261+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:40.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:40.261+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:40.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:40.261+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:40.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:40.261+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:40.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:40.261+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:40.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:40.261+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:40.479 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:40 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:40.261+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-04-02T13:51:12.460269+0000 front 2024-04-02T13:51:12.460256+0000 (oldest deadline 2024-04-02T13:51:38.360079+0000) 2024-04-02T13:51:41.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:41.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:41.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:41.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:41.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:41.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:41.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:41.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:41.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:41.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:41.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:41.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:41.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:41 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:41.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-04-02T13:51:12.460269+0000 front 2024-04-02T13:51:12.460256+0000 (oldest deadline 2024-04-02T13:51:38.360079+0000) 2024-04-02T13:51:42.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:42 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:42.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:42.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:42 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:42.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:42.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:42 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:42.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:42.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:42 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:42.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:42.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:42 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:42.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:42.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:42 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:42.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:42.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:42 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:42.256+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-04-02T13:51:12.460269+0000 front 2024-04-02T13:51:12.460256+0000 (oldest deadline 2024-04-02T13:51:38.360079+0000) 2024-04-02T13:51:43.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:43.248+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:43.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:43.248+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:43.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:43.248+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:43.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:43.248+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:43.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:43.248+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:43.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:43.248+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:43.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:43.248+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-04-02T13:51:12.460269+0000 front 2024-04-02T13:51:12.460256+0000 (oldest deadline 2024-04-02T13:51:38.360079+0000) 2024-04-02T13:51:43.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:43 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:43.248+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6820 osd.8 since back 2024-04-02T13:51:18.360425+0000 front 2024-04-02T13:51:18.360384+0000 (oldest deadline 2024-04-02T13:51:43.060406+0000) 2024-04-02T13:51:44.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:44 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:44.266+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-02T13:50:16.149370+0000 front 2024-04-02T13:50:16.149512+0000 (oldest deadline 2024-04-02T13:50:38.449606+0000) 2024-04-02T13:51:44.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:44 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:44.266+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-04-02T13:50:31.653660+0000 front 2024-04-02T13:50:31.653514+0000 (oldest deadline 2024-04-02T13:50:56.353762+0000) 2024-04-02T13:51:44.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:44 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:44.266+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-02T13:50:36.354378+0000 front 2024-04-02T13:50:36.354407+0000 (oldest deadline 2024-04-02T13:51:01.654341+0000) 2024-04-02T13:51:44.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:44 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:44.266+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-04-02T13:50:45.155662+0000 front 2024-04-02T13:50:45.155738+0000 (oldest deadline 2024-04-02T13:51:11.055436+0000) 2024-04-02T13:51:44.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:44 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:44.266+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-04-02T13:50:53.856812+0000 front 2024-04-02T13:50:53.856832+0000 (oldest deadline 2024-04-02T13:51:23.257484+0000) 2024-04-02T13:51:44.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:44 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:44.266+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-02T13:51:04.358417+0000 front 2024-04-02T13:51:04.358545+0000 (oldest deadline 2024-04-02T13:51:27.258396+0000) 2024-04-02T13:51:44.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:44 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:44.266+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-04-02T13:51:12.460269+0000 front 2024-04-02T13:51:12.460256+0000 (oldest deadline 2024-04-02T13:51:38.360079+0000) 2024-04-02T13:51:44.729 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:44 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11[60711]: 2024-04-02T13:51:44.266+0000 7fc53afcd700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.145:6820 osd.8 since back 2024-04-02T13:51:18.360425+0000 front 2024-04-02T13:51:18.360384+0000 (oldest deadline 2024-04-02T13:51:43.060406+0000) 2024-04-02T13:51:45.768 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:45 smithi145 podman[72769]: 2024-04-02 13:51:45.480455519 +0000 UTC m=+5.496077656 container died 2a8ec003a4bf469b181d3f024b8330fe56ef4dfc4795a8504b832ae0886d3863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2) 2024-04-02T13:51:46.135 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:45 smithi145 podman[72769]: 2024-04-02 13:51:45.840452008 +0000 UTC m=+5.856074152 container cleanup 2a8ec003a4bf469b181d3f024b8330fe56ef4dfc4795a8504b832ae0886d3863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-02T13:51:46.135 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:45 smithi145 bash[72769]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11 2024-04-02T13:51:46.444 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:46 smithi145 podman[72787]: 2024-04-02 13:51:46.133421065 +0000 UTC m=+0.643644274 container remove 2a8ec003a4bf469b181d3f024b8330fe56ef4dfc4795a8504b832ae0886d3863 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:51:46.722 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:46 smithi145 podman[72877]: 2024-04-02 13:51:46.442511535 +0000 UTC m=+0.112289942 container create 3b43b08d8ef4ba3e5e0a1ef16088d17dc8d5f2a1fd9924a293f701f01ace259a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-02T13:51:46.722 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:46 smithi145 podman[72877]: 2024-04-02 13:51:46.347959434 +0000 UTC m=+0.017737873 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:51:46.722 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:46 smithi145 podman[72877]: 2024-04-02 13:51:46.636804215 +0000 UTC m=+0.306582628 container init 3b43b08d8ef4ba3e5e0a1ef16088d17dc8d5f2a1fd9924a293f701f01ace259a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-02T13:51:46.723 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:46 smithi145 podman[72877]: 2024-04-02 13:51:46.640060167 +0000 UTC m=+0.309838576 container start 3b43b08d8ef4ba3e5e0a1ef16088d17dc8d5f2a1fd9924a293f701f01ace259a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:51:46.723 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:46 smithi145 podman[72877]: 2024-04-02 13:51:46.72029534 +0000 UTC m=+0.390073764 container attach 3b43b08d8ef4ba3e5e0a1ef16088d17dc8d5f2a1fd9924a293f701f01ace259a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-02T13:51:46.979 INFO:journalctl@ceph.osd.11.smithi145.stdout:Apr 02 13:51:46 smithi145 podman[72877]: 2024-04-02 13:51:46.926552087 +0000 UTC m=+0.596330509 container died 3b43b08d8ef4ba3e5e0a1ef16088d17dc8d5f2a1fd9924a293f701f01ace259a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:51:47.488 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@osd.11.service' 2024-04-02T13:51:47.528 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:51:47.528 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-02T13:51:47.529 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-02T13:51:47.529 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.a 2024-04-02T13:51:47.823 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 02 13:51:47 smithi033 systemd[1]: Stopping Ceph mds.a for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:51:48.254 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 02 13:51:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-a[71944]: 2024-04-02T13:51:47.820+0000 7f0ae69ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:51:48.254 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 02 13:51:47 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-a[71944]: 2024-04-02T13:51:47.820+0000 7f0ae69ce700 -1 mds.a *** got signal Terminated *** 2024-04-02T13:51:49.116 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 02 13:51:48 smithi033 podman[100657]: 2024-04-02 13:51:48.835661413 +0000 UTC m=+1.112353490 container died 213ccd759d4c0e7fce24f9767fbb358f990dfbf344546f85dbbaad464135ec10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-a, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0) 2024-04-02T13:51:49.457 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 02 13:51:49 smithi033 podman[100657]: 2024-04-02 13:51:49.212083199 +0000 UTC m=+1.488775269 container cleanup 213ccd759d4c0e7fce24f9767fbb358f990dfbf344546f85dbbaad464135ec10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-a, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:51:49.457 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 02 13:51:49 smithi033 bash[100657]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-a 2024-04-02T13:51:49.583 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.a.service' 2024-04-02T13:51:49.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:51:49.619 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-02T13:51:49.619 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-02T13:51:49.619 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.d 2024-04-02T13:51:49.753 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:51:49 smithi033 systemd[1]: Stopping Ceph mds.d for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:51:50.254 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:51:49 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-d[70989]: 2024-04-02T13:51:49.939+0000 7fde707c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:51:50.254 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:51:49 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-d[70989]: 2024-04-02T13:51:49.939+0000 7fde707c9700 -1 mds.d *** got signal Terminated *** 2024-04-02T13:51:53.136 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:51:52 smithi033 podman[100778]: 2024-04-02 13:51:52.863504283 +0000 UTC m=+3.022405845 container died 34b9deba6824cbc2314f52e10e4a07805d7c6f053beae312bcf34fe4b97d5a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-02T13:51:53.477 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:51:53 smithi033 podman[100778]: 2024-04-02 13:51:53.224311568 +0000 UTC m=+3.383213138 container cleanup 34b9deba6824cbc2314f52e10e4a07805d7c6f053beae312bcf34fe4b97d5a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-02T13:51:53.477 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 02 13:51:53 smithi033 bash[100778]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-d 2024-04-02T13:51:53.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.d.service' 2024-04-02T13:51:53.689 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:51:53.690 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-02T13:51:53.690 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-02T13:51:53.690 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.g 2024-04-02T13:51:54.004 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:51:53 smithi033 systemd[1]: Stopping Ceph mds.g for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:51:54.504 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:51:54 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-g[71583]: 2024-04-02T13:51:54.001+0000 7f49054b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:51:54.504 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:51:54 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-g[71583]: 2024-04-02T13:51:54.001+0000 7f49054b8700 -1 mds.g *** got signal Terminated *** 2024-04-02T13:51:58.197 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:51:57 smithi033 podman[100898]: 2024-04-02 13:51:57.911059673 +0000 UTC m=+4.009278178 container died ed7fe4339c384e2243f6a211b87ef99923fa59606126bad3e4fda72fa90e6391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:51:58.504 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:51:58 smithi033 podman[100898]: 2024-04-02 13:51:58.278046974 +0000 UTC m=+4.376265478 container cleanup ed7fe4339c384e2243f6a211b87ef99923fa59606126bad3e4fda72fa90e6391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-g, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-04-02T13:51:58.504 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 02 13:51:58 smithi033 bash[100898]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-g 2024-04-02T13:51:58.711 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.g.service' 2024-04-02T13:51:58.744 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:51:58.744 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-02T13:51:58.744 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-02T13:51:58.744 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.j 2024-04-02T13:51:59.075 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:51:58 smithi033 systemd[1]: Stopping Ceph mds.j for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:51:59.504 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:51:59 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-j[70634]: 2024-04-02T13:51:59.071+0000 7f7f33adf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:51:59.504 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:51:59 smithi033 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-j[70634]: 2024-04-02T13:51:59.071+0000 7f7f33adf700 -1 mds.j *** got signal Terminated *** 2024-04-02T13:52:02.117 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:52:01 smithi033 podman[101019]: 2024-04-02 13:52:01.852568145 +0000 UTC m=+2.885825242 container died e135d8098509da5639a4aeaecf1105f5c454f6315646b931dee895280408dea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-j, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:52:02.459 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:52:02 smithi033 podman[101019]: 2024-04-02 13:52:02.214480462 +0000 UTC m=+3.247737558 container cleanup e135d8098509da5639a4aeaecf1105f5c454f6315646b931dee895280408dea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-j, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0) 2024-04-02T13:52:02.459 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 02 13:52:02 smithi033 bash[101019]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-j 2024-04-02T13:52:02.591 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.j.service' 2024-04-02T13:52:02.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:52:02.625 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-02T13:52:02.625 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-02T13:52:02.625 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.b 2024-04-02T13:52:02.943 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:52:02 smithi050 systemd[1]: Stopping Ceph mds.b for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:52:03.244 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:52:02 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-b[66325]: 2024-04-02T13:52:02.939+0000 7f49d5670700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:52:03.245 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:52:02 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-b[66325]: 2024-04-02T13:52:02.939+0000 7f49d5670700 -1 mds.b *** got signal Terminated *** 2024-04-02T13:52:07.218 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:52:06 smithi050 podman[73030]: 2024-04-02 13:52:06.945451588 +0000 UTC m=+4.122712962 container died a6141b466874b9569b0927ffa20a80b8b868e6cbea229503438c084ad1251725 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-b, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:52:07.494 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:52:07 smithi050 podman[73030]: 2024-04-02 13:52:07.306300287 +0000 UTC m=+4.483561650 container cleanup a6141b466874b9569b0927ffa20a80b8b868e6cbea229503438c084ad1251725 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:52:07.494 INFO:journalctl@ceph.mds.b.smithi050.stdout:Apr 02 13:52:07 smithi050 bash[73030]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-b 2024-04-02T13:52:07.746 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.b.service' 2024-04-02T13:52:07.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:52:07.784 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-02T13:52:07.784 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-02T13:52:07.784 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.e 2024-04-02T13:52:08.121 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:52:07 smithi050 systemd[1]: Stopping Ceph mds.e for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:52:08.494 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:52:08 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-e[65690]: 2024-04-02T13:52:08.118+0000 7fc32a941700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:52:08.494 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:52:08 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-e[65690]: 2024-04-02T13:52:08.118+0000 7fc32a941700 -1 mds.e *** got signal Terminated *** 2024-04-02T13:52:11.713 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:52:11 smithi050 podman[73152]: 2024-04-02 13:52:11.444461929 +0000 UTC m=+3.440819870 container died 985dba5ca7f63f0e2567ae6d5f68c438b8938b77aa69cc7a53a5b8981a635d6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:52:11.994 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:52:11 smithi050 podman[73152]: 2024-04-02 13:52:11.810487115 +0000 UTC m=+3.806845043 container cleanup 985dba5ca7f63f0e2567ae6d5f68c438b8938b77aa69cc7a53a5b8981a635d6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-02T13:52:11.995 INFO:journalctl@ceph.mds.e.smithi050.stdout:Apr 02 13:52:11 smithi050 bash[73152]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-e 2024-04-02T13:52:12.232 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.e.service' 2024-04-02T13:52:12.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:52:12.269 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-02T13:52:12.269 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-02T13:52:12.269 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.h 2024-04-02T13:52:12.600 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:52:12 smithi050 systemd[1]: Stopping Ceph mds.h for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:52:12.994 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:52:12 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-h[66015]: 2024-04-02T13:52:12.597+0000 7fb0ad6d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:52:12.995 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:52:12 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-h[66015]: 2024-04-02T13:52:12.597+0000 7fb0ad6d5700 -1 mds.h *** got signal Terminated *** 2024-04-02T13:52:15.709 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:52:15 smithi050 podman[73285]: 2024-04-02 13:52:15.448916231 +0000 UTC m=+2.953758964 container died 398fcfd4da0bb586cd61fdea485b5367c3f6372e26d1bd343cf95967d4824f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-h, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:52:15.994 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:52:15 smithi050 podman[73285]: 2024-04-02 13:52:15.800174878 +0000 UTC m=+3.305017605 container cleanup 398fcfd4da0bb586cd61fdea485b5367c3f6372e26d1bd343cf95967d4824f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-h, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-02T13:52:15.995 INFO:journalctl@ceph.mds.h.smithi050.stdout:Apr 02 13:52:15 smithi050 bash[73285]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-h 2024-04-02T13:52:16.223 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.h.service' 2024-04-02T13:52:16.258 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:52:16.259 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-02T13:52:16.259 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-02T13:52:16.259 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.k 2024-04-02T13:52:16.745 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:52:16 smithi050 systemd[1]: Stopping Ceph mds.k for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:52:16.745 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:52:16 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-k[65375]: 2024-04-02T13:52:16.559+0000 7f891d6fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:52:16.745 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:52:16 smithi050 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-k[65375]: 2024-04-02T13:52:16.559+0000 7f891d6fb700 -1 mds.k *** got signal Terminated *** 2024-04-02T13:52:19.179 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:52:18 smithi050 podman[73408]: 2024-04-02 13:52:18.866604792 +0000 UTC m=+2.388324696 container died a9da2e12226d724711d396934a9bb65d8c1a0c8fe41431fae04af5877ac7fa8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-02T13:52:19.179 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:52:19 smithi050 podman[73408]: 2024-04-02 13:52:19.177198562 +0000 UTC m=+2.698918469 container cleanup a9da2e12226d724711d396934a9bb65d8c1a0c8fe41431fae04af5877ac7fa8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:52:19.494 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:52:19 smithi050 bash[73408]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-k 2024-04-02T13:52:19.495 INFO:journalctl@ceph.mds.k.smithi050.stdout:Apr 02 13:52:19 smithi050 podman[73422]: 2024-04-02 13:52:19.378343371 +0000 UTC m=+0.514106261 container remove a9da2e12226d724711d396934a9bb65d8c1a0c8fe41431fae04af5877ac7fa8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-k, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-02T13:52:19.612 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.k.service' 2024-04-02T13:52:19.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:52:19.646 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-02T13:52:19.646 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-02T13:52:19.646 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.c 2024-04-02T13:52:19.925 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:52:19 smithi145 systemd[1]: Stopping Ceph mds.c for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:52:20.183 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:52:19 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-c[65619]: 2024-04-02T13:52:19.922+0000 7f6573bc4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:52:20.183 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:52:19 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-c[65619]: 2024-04-02T13:52:19.922+0000 7f6573bc4700 -1 mds.c *** got signal Terminated *** 2024-04-02T13:52:23.729 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:52:23 smithi145 podman[72983]: 2024-04-02 13:52:23.450703391 +0000 UTC m=+3.616855681 container died 21a7f37a992bf39db3c07f28ddc005c7350a84a310ffda028da624fedf019ef6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-c, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:52:24.185 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:52:23 smithi145 podman[72983]: 2024-04-02 13:52:23.780432898 +0000 UTC m=+3.946585212 container cleanup 21a7f37a992bf39db3c07f28ddc005c7350a84a310ffda028da624fedf019ef6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0) 2024-04-02T13:52:24.186 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:52:23 smithi145 bash[72983]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-c 2024-04-02T13:52:24.186 INFO:journalctl@ceph.mds.c.smithi145.stdout:Apr 02 13:52:23 smithi145 podman[73007]: 2024-04-02 13:52:23.998411837 +0000 UTC m=+0.549610440 container remove 21a7f37a992bf39db3c07f28ddc005c7350a84a310ffda028da624fedf019ef6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326) 2024-04-02T13:52:24.204 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.c.service' 2024-04-02T13:52:24.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:52:24.244 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-02T13:52:24.244 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-02T13:52:24.244 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.f 2024-04-02T13:52:24.478 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:52:24 smithi145 systemd[1]: Stopping Ceph mds.f for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:52:24.979 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:52:24 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-f[65309]: 2024-04-02T13:52:24.575+0000 7f9da1175700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:52:24.979 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:52:24 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-f[65309]: 2024-04-02T13:52:24.575+0000 7f9da1175700 -1 mds.f *** got signal Terminated *** 2024-04-02T13:52:27.729 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:52:27 smithi145 podman[73117]: 2024-04-02 13:52:27.458361768 +0000 UTC m=+2.990689286 container died feb4f52be8ac04c85cb29bf8691834ec5110b720f636e591037ddbe3b1dbd7d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-f, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:52:28.079 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:52:27 smithi145 podman[73117]: 2024-04-02 13:52:27.842740611 +0000 UTC m=+3.375068174 container cleanup feb4f52be8ac04c85cb29bf8691834ec5110b720f636e591037ddbe3b1dbd7d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-f, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-02T13:52:28.080 INFO:journalctl@ceph.mds.f.smithi145.stdout:Apr 02 13:52:27 smithi145 bash[73117]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-f 2024-04-02T13:52:28.227 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.f.service' 2024-04-02T13:52:28.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:52:28.264 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-02T13:52:28.264 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-02T13:52:28.264 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.i 2024-04-02T13:52:28.729 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:52:28 smithi145 systemd[1]: Stopping Ceph mds.i for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:52:28.729 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:52:28 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-i[65924]: 2024-04-02T13:52:28.571+0000 7fa900116700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:52:28.729 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:52:28 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-i[65924]: 2024-04-02T13:52:28.571+0000 7fa900116700 -1 mds.i *** got signal Terminated *** 2024-04-02T13:52:31.729 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:52:31 smithi145 podman[73240]: 2024-04-02 13:52:31.449238302 +0000 UTC m=+2.977611687 container died df105ad19076c883d511eeb3f9766328f637701e164620d48143e84c5b6c5de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-i, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:52:32.100 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:52:31 smithi145 podman[73240]: 2024-04-02 13:52:31.82141204 +0000 UTC m=+3.349785433 container cleanup df105ad19076c883d511eeb3f9766328f637701e164620d48143e84c5b6c5de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-i, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-02T13:52:32.100 INFO:journalctl@ceph.mds.i.smithi145.stdout:Apr 02 13:52:31 smithi145 bash[73240]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-i 2024-04-02T13:52:32.237 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.i.service' 2024-04-02T13:52:32.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:52:32.271 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-02T13:52:32.271 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-02T13:52:32.271 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.l 2024-04-02T13:52:32.729 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:52:32 smithi145 systemd[1]: Stopping Ceph mds.l for b5fd982e-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:52:32.729 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:52:32 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-l[64993]: 2024-04-02T13:52:32.574+0000 7f97e4e63700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:52:32.729 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:52:32 smithi145 ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-l[64993]: 2024-04-02T13:52:32.574+0000 7f97e4e63700 -1 mds.l *** got signal Terminated *** 2024-04-02T13:52:36.178 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:52:35 smithi145 podman[73364]: 2024-04-02 13:52:35.902691093 +0000 UTC m=+3.420263261 container died cb3d3318fbc9a13bb3805b4f839c42d1e1f1ef8144f0087ea89e0a5f407fe5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-l, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:52:36.478 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:52:36 smithi145 podman[73364]: 2024-04-02 13:52:36.266764983 +0000 UTC m=+3.784337162 container cleanup cb3d3318fbc9a13bb3805b4f839c42d1e1f1ef8144f0087ea89e0a5f407fe5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-l, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-02T13:52:36.479 INFO:journalctl@ceph.mds.l.smithi145.stdout:Apr 02 13:52:36 smithi145 bash[73364]: ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4-mds-l 2024-04-02T13:52:36.653 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b5fd982e-f0f4-11ee-b647-cb9ed24678a4@mds.l.service' 2024-04-02T13:52:36.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:52:36.687 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-02T13:52:36.687 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T13:52:36.885 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:52:39.285 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T13:52:39.484 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:52:42.015 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T13:52:42.207 INFO:teuthology.orchestra.run.smithi145.stdout:Deleting cluster with fsid: b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:52:44.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:52:44.580 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:52:44.615 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:52:44.645 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-02T13:52:44.647 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593/remote/smithi033/crash 2024-04-02T13:52:44.648 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T13:52:44.677 INFO:teuthology.orchestra.run.smithi033.stderr:tar: /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T13:52:44.677 INFO:teuthology.orchestra.run.smithi033.stderr:tar: Error is not recoverable: exiting now 2024-04-02T13:52:44.680 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593/remote/smithi050/crash 2024-04-02T13:52:44.680 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T13:52:44.727 INFO:teuthology.orchestra.run.smithi050.stderr:tar: /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T13:52:44.727 INFO:teuthology.orchestra.run.smithi050.stderr:tar: Error is not recoverable: exiting now 2024-04-02T13:52:44.728 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593/remote/smithi145/crash 2024-04-02T13:52:44.729 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T13:52:44.764 INFO:teuthology.orchestra.run.smithi145.stderr:tar: /var/lib/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T13:52:44.765 INFO:teuthology.orchestra.run.smithi145.stderr:tar: Error is not recoverable: exiting now 2024-04-02T13:52:44.766 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-02T13:52:44.766 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-02T13:52:44.803 INFO:teuthology.orchestra.run.smithi033.stdout:2024-04-02T13:50:00.000165+0000 mon.a (mon.0) 1430 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-02T13:52:44.804 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-02T13:52:44.804 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[SEC\]' /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-02T13:52:44.880 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]' /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-02T13:52:44.956 INFO:teuthology.orchestra.run.smithi033.stdout:2024-04-02T13:50:00.000165+0000 mon.a (mon.0) 1430 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-02T13:52:44.957 INFO:tasks.cephadm:Compressing logs... 2024-04-02T13:52:44.957 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T13:52:44.999 DEBUG:teuthology.orchestra.run.smithi050:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T13:52:45.002 DEBUG:teuthology.orchestra.run.smithi145:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T13:52:45.025 INFO:teuthology.orchestra.run.smithi033.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T13:52:45.025 INFO:teuthology.orchestra.run.smithi033.stderr:: No such file or directory 2024-04-02T13:52:45.025 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-02T13:52:45.025 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-02T13:52:45.025 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.6.log 2024-04-02T13:52:45.028 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.a.log 2024-04-02T13:52:45.029 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T13:52:45.029 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.6.log: /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.g.log 2024-04-02T13:52:45.029 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.j.log 2024-04-02T13:52:45.029 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.g.log: 92.6%gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.3.log 2024-04-02T13:52:45.029 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-02T13:52:45.029 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.x.log 2024-04-02T13:52:45.029 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.j.log: /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.0.log 2024-04-02T13:52:45.030 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.x.log: 90.2% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T13:52:45.034 INFO:teuthology.orchestra.run.smithi145.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T13:52:45.034 INFO:teuthology.orchestra.run.smithi145.stderr:: No such file or directory 2024-04-02T13:52:45.034 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.l.log 2024-04-02T13:52:45.034 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T13:52:45.034 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.l.log: --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.i.log 2024-04-02T13:52:45.035 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.11.log 2024-04-02T13:52:45.035 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.z.log 2024-04-02T13:52:45.035 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.11.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T13:52:45.035 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.8.log 2024-04-02T13:52:45.035 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.5.log 2024-04-02T13:52:45.035 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.z.log: /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.f.log 2024-04-02T13:52:45.035 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T13:52:45.035 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T13:52:45.035 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T13:52:45.035 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T13:52:45.035 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.2.log 2024-04-02T13:52:45.035 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.log: 94.5% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.l.log.gz 2024-04-02T13:52:45.045 INFO:teuthology.orchestra.run.smithi145.stderr: 87.7% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T13:52:45.045 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.c.log 2024-04-02T13:52:45.045 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T13:52:45.045 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.c.log: 92.0% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.z.log.gz 2024-04-02T13:52:45.046 INFO:teuthology.orchestra.run.smithi033.stderr: 94.6% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.g.log.gz 2024-04-02T13:52:45.046 INFO:teuthology.orchestra.run.smithi033.stderr: 94.6% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.j.log.gz 2024-04-02T13:52:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T13:52:45.047 INFO:teuthology.orchestra.run.smithi033.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T13:52:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T13:52:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T13:52:45.047 INFO:teuthology.orchestra.run.smithi033.stderr: 84.0% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T13:52:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.log: 87.7% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T13:52:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.9.log 2024-04-02T13:52:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.d.log 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.10.log 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.b.log 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.4.log 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr: 92.1%/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.b.log: -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.1.log 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.k.log 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.4.log: /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.h.log 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.7.log 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.e.log 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.7.log: 90.3% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T13:52:45.048 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.y.log 2024-04-02T13:52:45.049 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.e.log: 94.5% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.k.log.gz 2024-04-02T13:52:45.049 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T13:52:45.049 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.y.log: 91.9% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.y.log.gz 2024-04-02T13:52:45.050 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T13:52:45.051 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T13:52:45.051 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T13:52:45.053 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.a.log 2024-04-02T13:52:45.053 INFO:teuthology.orchestra.run.smithi033.stderr: 90.9% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T13:52:45.055 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.log: 87.7% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T13:52:45.055 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.b.log 2024-04-02T13:52:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.d.log: /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.a.log: 94.6% 94.5% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.a.log.gz 2024-04-02T13:52:45.058 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.d.log.gz 2024-04-02T13:52:45.059 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.c.log 2024-04-02T13:52:45.070 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log: /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T13:52:45.073 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T13:52:45.241 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.c.log: 94.3% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.c.log.gz 2024-04-02T13:52:45.275 INFO:teuthology.orchestra.run.smithi145.stderr: 94.6% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.f.log.gz 2024-04-02T13:52:45.409 INFO:teuthology.orchestra.run.smithi033.stderr: 90.3% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.x.log.gz 2024-04-02T13:52:45.533 INFO:teuthology.orchestra.run.smithi145.stderr: 92.1% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.c.log.gz 2024-04-02T13:52:45.546 INFO:teuthology.orchestra.run.smithi050.stderr: 92.1% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.b.log.gz 2024-04-02T13:52:45.974 INFO:teuthology.orchestra.run.smithi033.stderr: 90.6% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.a.log.gz 2024-04-02T13:52:47.397 INFO:teuthology.orchestra.run.smithi050.stderr: 94.2% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.h.log.gz 2024-04-02T13:52:47.828 INFO:teuthology.orchestra.run.smithi145.stderr: 95.6% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.i.log.gz 2024-04-02T13:52:47.904 INFO:teuthology.orchestra.run.smithi050.stderr: 94.0% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.e.log.gz 2024-04-02T13:52:50.532 INFO:teuthology.orchestra.run.smithi050.stderr: 91.9% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.10.log.gz 2024-04-02T13:52:51.053 INFO:teuthology.orchestra.run.smithi033.stderr: 91.5% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.0.log.gz 2024-04-02T13:52:51.070 INFO:teuthology.orchestra.run.smithi145.stderr: 91.7% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.11.log.gz 2024-04-02T13:52:51.418 INFO:teuthology.orchestra.run.smithi145.stderr: 91.8% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.8.log.gz 2024-04-02T13:52:51.593 INFO:teuthology.orchestra.run.smithi145.stderr: 91.5% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.2.log.gz 2024-04-02T13:52:52.003 INFO:teuthology.orchestra.run.smithi050.stderr: 91.7% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.1.log.gz 2024-04-02T13:52:52.366 INFO:teuthology.orchestra.run.smithi033.stderr: 91.5% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.3.log.gz 2024-04-02T13:52:52.997 INFO:teuthology.orchestra.run.smithi050.stderr: 91.6% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.7.log.gz 2024-04-02T13:52:54.089 INFO:teuthology.orchestra.run.smithi033.stderr: 91.4% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.6.log.gz 2024-04-02T13:52:54.395 INFO:teuthology.orchestra.run.smithi145.stderr: 91.4% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.5.log.gz 2024-04-02T13:52:54.397 INFO:teuthology.orchestra.run.smithi145.stderr: 2024-04-02T13:52:54.397 INFO:teuthology.orchestra.run.smithi145.stderr:real 0m9.380s 2024-04-02T13:52:54.397 INFO:teuthology.orchestra.run.smithi145.stderr:user 0m31.102s 2024-04-02T13:52:54.397 INFO:teuthology.orchestra.run.smithi145.stderr:sys 0m0.956s 2024-04-02T13:52:55.409 INFO:teuthology.orchestra.run.smithi033.stderr: 91.6% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.9.log.gz 2024-04-02T13:52:55.587 INFO:teuthology.orchestra.run.smithi050.stderr: 91.5% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.4.log.gz 2024-04-02T13:52:57.642 INFO:teuthology.orchestra.run.smithi033.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-02T13:52:57.643 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-04-02T13:52:57.644 INFO:teuthology.orchestra.run.smithi033.stderr:real 0m12.632s 2024-04-02T13:52:57.644 INFO:teuthology.orchestra.run.smithi033.stderr:user 0m45.407s 2024-04-02T13:52:57.644 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m1.320s 2024-04-02T13:53:27.205 INFO:teuthology.orchestra.run.smithi050.stderr: 93.4% -- replaced with /var/log/ceph/b5fd982e-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.b.log.gz 2024-04-02T13:53:27.207 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-04-02T13:53:27.207 INFO:teuthology.orchestra.run.smithi050.stderr:real 0m42.190s 2024-04-02T13:53:27.207 INFO:teuthology.orchestra.run.smithi050.stderr:user 1m16.567s 2024-04-02T13:53:27.207 INFO:teuthology.orchestra.run.smithi050.stderr:sys 0m2.173s 2024-04-02T13:53:27.207 INFO:tasks.cephadm:Archiving logs... 2024-04-02T13:53:27.208 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593/remote/smithi033/log 2024-04-02T13:53:27.209 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T13:53:33.698 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593/remote/smithi050/log 2024-04-02T13:53:33.699 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T13:53:44.502 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593/remote/smithi145/log 2024-04-02T13:53:44.503 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T13:53:49.462 INFO:tasks.cephadm:Removing cluster... 2024-04-02T13:53:49.463 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 --force 2024-04-02T13:53:49.661 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:53:49.965 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 --force 2024-04-02T13:53:50.163 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:53:50.516 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b5fd982e-f0f4-11ee-b647-cb9ed24678a4 --force 2024-04-02T13:53:50.710 INFO:teuthology.orchestra.run.smithi145.stdout:Deleting cluster with fsid: b5fd982e-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:53:51.009 INFO:tasks.cephadm:Removing cephadm ... 2024-04-02T13:53:51.009 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T13:53:51.028 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T13:53:51.044 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T13:53:51.060 INFO:tasks.cephadm:Teardown complete 2024-04-02T13:53:51.060 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-02T13:53:51.072 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-02T13:53:51.073 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T13:53:51.075 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T13:53:51.087 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T13:53:51.199 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-02T13:53:51.210 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-02T13:53:51.210 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T13:53:51.212 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T13:53:51.215 DEBUG:teuthology.orchestra.run.smithi145:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T13:53:51.226 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-04-02T13:53:51.227 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-04-02T13:53:51.228 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-04-02T13:53:51.230 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T13:53:51.231 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-04-02T13:53:51.231 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 -534us[ -545us] +/- 71ms 2024-04-02T13:53:51.231 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -1418us[-1429us] +/- 54ms 2024-04-02T13:53:51.231 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 +2612us[+2601us] +/- 79ms 2024-04-02T13:53:51.231 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:53:51.232 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T13:53:51.233 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-04-02T13:53:51.233 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 26 -543us[ -543us] +/- 71ms 2024-04-02T13:53:51.233 INFO:teuthology.orchestra.run.smithi050.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -1421us[-1409us] +/- 54ms 2024-04-02T13:53:51.233 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +2609us[+2622us] +/- 79ms 2024-04-02T13:53:51.233 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:53:51.234 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T13:53:51.234 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-04-02T13:53:51.234 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 -518us[ -561us] +/- 71ms 2024-04-02T13:53:51.234 INFO:teuthology.orchestra.run.smithi145.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -1397us[-1440us] +/- 54ms 2024-04-02T13:53:51.234 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +2623us[+2580us] +/- 79ms 2024-04-02T13:53:51.234 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:53:51.234 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-02T13:53:51.245 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-02T13:53:51.246 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-02T13:53:51.257 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T13:53:51.364 DEBUG:teuthology.orchestra.run.smithi050:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T13:53:51.440 DEBUG:teuthology.orchestra.run.smithi145:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T13:53:51.513 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T13:53:51.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:53:51.546 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T13:53:51.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:53:51.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T13:53:51.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:53:51.607 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-02T13:53:51.619 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-02T13:53:51.630 INFO:teuthology.task.internal:Duration was 1976.827032 seconds 2024-04-02T13:53:51.630 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-02T13:53:51.643 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-02T13:53:51.643 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T13:53:51.646 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T13:53:51.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T13:53:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T13:53:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T13:53:51.691 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T13:53:52.044 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-02T13:53:52.045 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-04-02T13:53:52.045 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T13:53:52.066 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2024-04-02T13:53:52.190 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T13:53:52.218 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi145.front.sepia.ceph.com 2024-04-02T13:53:52.218 DEBUG:teuthology.orchestra.run.smithi145:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T13:53:52.252 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-02T13:53:52.252 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T13:53:52.256 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T13:53:52.266 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T13:53:52.326 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-02T13:53:52.326 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T13:53:52.329 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T13:53:52.337 DEBUG:teuthology.orchestra.run.smithi145:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T13:53:52.717 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-02T13:53:52.732 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-02T13:53:52.733 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T13:53:52.759 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T13:53:52.786 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T13:53:52.813 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-02T13:53:52.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T13:53:52.828 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T13:53:52.830 DEBUG:teuthology.orchestra.run.smithi145:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T13:53:52.851 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-04-02T13:53:52.853 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2024-04-02T13:53:52.879 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = core 2024-04-02T13:53:52.901 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T13:53:52.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:53:52.921 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T13:53:52.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:53:52.947 DEBUG:teuthology.orchestra.run.smithi145:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T13:53:52.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:53:52.962 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-02T13:53:52.980 INFO:teuthology.task.internal:Transferring archived files... 2024-04-02T13:53:52.982 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593/remote/smithi033 2024-04-02T13:53:52.982 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T13:53:53.027 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593/remote/smithi050 2024-04-02T13:53:53.027 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T13:53:53.066 DEBUG:teuthology.misc:Transferring archived files from smithi145:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593/remote/smithi145 2024-04-02T13:53:53.066 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T13:53:53.105 INFO:teuthology.task.internal:Removing archive directory... 2024-04-02T13:53:53.105 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T13:53:53.107 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T13:53:53.110 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T13:53:53.156 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-02T13:53:53.170 INFO:teuthology.task.internal:Not uploading archives. 2024-04-02T13:53:53.170 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-02T13:53:53.182 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-02T13:53:53.182 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T13:53:53.184 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T13:53:53.187 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T13:53:53.196 INFO:teuthology.orchestra.run.smithi033.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 13:53 /home/ubuntu/cephtest 2024-04-02T13:53:53.199 INFO:teuthology.orchestra.run.smithi050.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 13:53 /home/ubuntu/cephtest 2024-04-02T13:53:53.210 INFO:teuthology.orchestra.run.smithi145.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 13:53 /home/ubuntu/cephtest 2024-04-02T13:53:53.211 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-02T13:53:53.224 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-02T13:53:53.249 INFO:teuthology.nuke:Checking targets against current locks 2024-04-02T13:53:53.276 DEBUG:teuthology.nuke:shortname: smithi033 2024-04-02T13:53:53.276 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T13:53:53.300 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:12:23.631068', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLhTnsnNgTEqvZ0xH6rc6tnsPMCrui9NGPD3iGUtxEMvHGDqTEvq7KKmVY4TO0VTw0K1wbELcKMqMk8zzOdAznQ='} 2024-04-02T13:53:53.304 DEBUG:teuthology.nuke:shortname: smithi050 2024-04-02T13:53:53.305 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T13:53:53.328 DEBUG:teuthology.nuke:shortname: smithi145 2024-04-02T13:53:53.328 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T13:53:53.331 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:12:23.632177', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbXHHwuJ3dMVVWHiHynexISlW2DTUTPBSllC2jgAn4HenuiXzJZonUO7EZFtpyDiOH8hsbyyGD5kvyFEZaO9jI='} 2024-04-02T13:53:53.361 INFO:teuthology.orchestra.console.smithi033:Power off 2024-04-02T13:53:53.361 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T13:53:53.373 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635593', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:12:23.629403', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG4bNyFXHEQGaXTNcf2ir12eNpiralK8fj87y7QdLX0e6bOBhm/YW4OGLshUUeUizs1UMAnlk2Mxoekaq1y0euQ='} 2024-04-02T13:53:53.383 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off 2024-04-02T13:53:53.383 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T13:53:53.406 INFO:teuthology.orchestra.console.smithi050:Power off 2024-04-02T13:53:53.406 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T13:53:53.416 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-02T13:53:53.431 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off 2024-04-02T13:53:53.431 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T13:53:53.447 INFO:teuthology.orchestra.console.smithi145:Power off 2024-04-02T13:53:53.447 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T13:53:53.456 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-04-02T13:53:53.468 DEBUG:teuthology.orchestra.console.smithi145:power off output: Chassis Power Control: Down/Off 2024-04-02T13:53:53.468 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T13:53:53.492 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2024-04-02T13:53:57.418 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T13:53:57.458 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T13:53:57.493 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T13:53:57.529 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-02T13:53:57.572 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-04-02T13:53:57.607 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2024-04-02T13:54:01.533 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T13:54:01.573 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T13:54:01.608 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T13:54:01.644 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-02T13:54:01.684 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off 2024-04-02T13:54:01.718 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2024-04-02T13:54:01.785 INFO:teuthology.orchestra.console.smithi050:Power off completed 2024-04-02T13:54:05.649 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T13:54:05.722 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T13:54:05.761 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2024-04-02T13:54:05.834 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is off 2024-04-02T13:54:05.862 INFO:teuthology.orchestra.console.smithi033:Power off completed 2024-04-02T13:54:05.935 INFO:teuthology.orchestra.console.smithi145:Power off completed 2024-04-02T13:54:06.040 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} duration: 1976.8270318508148 failure_reason: '"2024-04-02T13:50:00.000165+0000 mon.a (mon.0) 1430 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-04-02T13:54:06.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T13:54:06.130 INFO:teuthology.run:FAIL